Yan Wang, MD, DrPH1,2; Markus Bidell, PhD3; Matthew B. Schabath, PhD4; Mandi L. Pratt-Chapman, PhD2,5
1The George Washington University, Milken Institute School of Public Health, Washington, DC
2GW Cancer Center, School of Medicine and Health Sciences, The George Washington University, Washington, DC
3City University of New York, Hunter College, New York, NY
4H. Lee Moffitt Cancer Center & Research Institute, Tampa, FL
5Department of Medicine, School of Medicine and Health Sciences, The George Washington University, Washington, DC
Purpose: Patients’ sexual orientation and gender identity have critical implications for cancer continuum of care needs. Yet, no extant scales allow for comparisons of patient- and provider-reported experiences of sexual and gender minority (SGM)-affirming care. This study conducted psychometric analyses to validate companion scales entitled the QUeering Individual and Relational Knowledge Scale (QUIRKS)-Provider and QUIRKS-Patient for providers and patients, respectively.
Methods: Based on theorized factors, confirmatory factor analyses (CFAs) were conducted using a convenience sample of cancer care providers (n = 143) and cancer survivors (n = 173). Criterion-related validity was conducted by assessing correlation between the QUIRKS-Provider and the Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS). Correlation analyses compared provider- and patient-reported Environmental Cues.
Results: CFA confirmed 5 factors for the QUIRKS-Provider and 3 for the QUIRKS-Patient. The final 24-item QUIRKS-Provider included Environmental Cues (3 items, McDonald omega coefficient ω = .920), Knowledge (7 items, ω = .750), Clinical Preparedness (5 items, ω = .934), Clinical Behaviors (5 items, ω =.609), and Attitudes (4 items, ω = .685). The final 13-item QUIRKS-Patient included Environmental Cues (5 items, ω = .837), Comfort Discussing Sexual Orientation and Gender Identity (5 items, ω = .887), and Satisfaction with Care (3 items, ω = .850). In correlation analyses, Knowledge, Clinical Preparedness, and Attitudes factors of the QUIRKS-Provider were strongly correlated with corresponding factors from the LGBT-DOCSS (range of 0.542-0.659, ps <.001). Comparisons of patient- and provider-reported environmental cues differed.
Conclusion: The QUIRKS-Patient and QUIRKS-Provider are valid companion scales to use in health services research to assess SGM-affirming care.
Healthcare experiences of lesbian, gay, bisexual, transgender, queer, and intersex (LGBTQI) patients are not well understood due to a lack of routine collection of sexual orientation and gender identity (SOGI) data. The National Academies of Sciences, Engineering, and Medicine called for improved data collection and services for LGBTQI people in its 2020 publication Understanding the Well-Being of LGBTQI+ Populations.1 The American Society of Clinical Oncology has also called for systematic SOGI data collection, provider training, and improved cancer care services for LGBTQI patients.
LGBTQI people have specific health and healthcare needs. For cancer care, unique needs include behavioral risks, differential screening, heterogeneity of treatment effects, and inadequate supportive care. For example, bisexual women have the highest rates of alcohol and substance use, increasing the risk for cancers of the mouth, throat, larynx, esophagus, colon, rectum, liver, and breast.2,3 Evidence suggests that transgender and nonbinary persons with a cervix have heightened gender dysphoria in gynecological healthcare settings, reducing cervical cancer screening.4 Past research showing differential outcomes by sex for some pharmacotherapies suggests that hormone therapy may moderate treatment effects.2,5 Men who have sex with men have inadequate therapeutic interventions for erectile dysfunction following prostate cancer treatment.6 Finally, same-sex couples have reported discrimination in end-of-life care.7 These disparities call for measuring, improving, and monitoring health services for LGBTQI patients.
Yet, few scales measure provider or patient experiences with affirming care for sexual and gender minorities (SGMs), making it difficult to establish a baseline or to measure progress. Only 1 validated measure specifically assesses provider competence in caring for SGM persons—the Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS).8 While that scale separates aggregate sexual orientation “LGB” from gender identity (T),8 there remains a need to examine differences in caring for lesbian, gay, and bisexual patients, respectively, as well as for patients with intersex variations.8 To our knowledge, there are no extant scales assessing patient-reported experiences of SGM-affirming care. To fill these gaps, this study developed companion scales entitled the QUeering Individual and Relational Knowledge Scale (QUIRKS)-Provider and the QUIRKS-Patient, respectively.
The study was approved by the George Washington University (GW) Institutional Review Board (NCR213247). All methods were carried out in accordance with this approval. Prior to completing the online survey, each respondent checked a box documenting informed consent.
Scales were developed based on theorized constructs, including patient-reported environmental cues relevant to affirming care; comfort being asked about SOGI; and satisfaction with care as well as provider-reported attitudes, clinical preparedness, clinical behaviors, environmental cues, and objective knowledge. Items were refined through cognitive interviews with cancer survivors and providers, respectively. Participants were asked about their thought process when reading a question, comprehension, ease of retrieving the scenario asked about, relevance, and appropriate response options. Detailed notes from interviews were used to refine the scales. Cognitive interviews confirmed face and content validity of items, and results will be published separately (Cognitive Testing).
QUIRKS-Provider. The original QUIRKS-Provider (32 items) included 5 questions about clinical preparedness, 4 questions about environmental cues, 8 questions assessing attitudes, 6 questions about clinical behaviors, and 9 objective knowledge questions. Six questions were reverse coded. Questions regarding clinical preparedness disaggregated feeling prepared to meet the needs of gay, lesbian, bisexual, transgender, and intersex patients, respectively. Questions focused on clinical behaviors assessed consideration of hormones in medical management, initiation of conversations about patient values and preferences, actions in response to witnessing discrimination or jokes about SGM people, ability for patients to designate medical decision makers, and documentation of advance directives. Questions assessing provider attitudes included comfort in discussing sexual health, the importance of knowing patients’ sexual orientation and/or gender identity, assumptions about patient comfort in being asked about sex assigned at birth and gender identity, assumptions about patient comfort in being asked about sex assigned at birth or gender identity if the provider explained why they were asking, wish for additional SGM-specific training, beliefs about professional responsibility to learn about SGM patient needs, and beliefs about professional responsibility to champion system changes to support SGM patients. These 4 subscales were designed to measure SGM-affirming care in any setting.
A fifth subscale, Knowledge, was cancer-specific to allow for assessment of the Together, Equitable, Accessible, Meaningful Cancer Care for SGM Patients (TEAM SGM)9 training. Knowledge items asked about implicit bias, gender as biological, transgender risk of family rejection, SGM trauma, knowledge of transgender-specific supportive care resources, cancer risks, and association of breast cancer reconstruction with sexual orientation. This subscale was designed to measure SGM-affirming care in cancer care settings.
QUIRKS-Patient. The original 14-item QUIRKS-Patient included 5 questions about environmental cues; 4 questions about patient experiences of discrimination, clinical communication, quality of and satisfaction with care; and 6 questions about healthcare providers inquiring about sex assigned at birth, sexual orientation, gender identity, and sexual and psychosocial health.
Comparison of Scales. To compare patient-reported and provider-reported experiences, there were 8 questions relevant to environmental cues included in each scale. Patients were asked about cues in the healthcare practice that welcome LGBTQI patients and racial/ethnic minorities, respectively, and opportunities to easily document name in use, pronouns, and sexual orientation. Providers were asked similar questions.
QUIRKS-Patient. Two samples of patients were merged to create an aggregate analytical sample for this study. First, a convenience sample of individuals with a history of cancer were recruited using convenience sampling through social media (ie, Twitter, LinkedIn) and professional networks (ie, the GW Cancer Center’s healthcare professional listserv). Second, patients from institutions that participated in TEAM SGM were asked to complete the scale. After eliminating incomplete records, these 2 data sets were combined for the present analysis (n = 173). There were no duplicates and no incentives for participation. Heterosexual, cisgender patients and LGBTQI patients were included in the study sample since a primary area of interest was visibility of affirming cues and behaviors regardless of patients’ sexual orientation or gender identity.
QUIRKS-Provider. Three subsamples of providers were merged to create the analytical sample for this study. First, a convenience sample of healthcare professionals were recruited to complete the QUIRKS-Provider through convenience sampling via GW Cancer Center newsletters, social media, and via professional networks (eg, the GW Cancer Center healthcare professional listserv). Second, healthcare professionals participating in TEAM SGM9 completed the QUIRKS-Provider at baseline, and third, immediately postintervention in summer 2021. No incentives were provided. Incomplete responses were eliminated, and there were no duplicates. The final sample was n = 143.
Data were collected using a unique link via the REDCap database. For patients, demographic questions captured age, race and ethnicity, sex assigned at birth, gender identity, sexual orientation, and state of residence. For providers, demographics captured age, race and ethnicity, sex assigned at birth, gender identity, sexual orientation, state of residence, professional role, specialty, and number of training hours specific to SGM health.
Four items in the QUIRKS-Provider and 5 items in the QUIRKS-Patient relevant to environmental cues had answer responses: Yes, No, or I don’t know. Since the investigators were most interested in whether affirming care indicators were clearly apparent to participants, these items were recoded to 0 = Yes and 1 = No/I don’t know. Six items in the QUIRKS-Patient relevant to comfort in being asked about sexual orientation, gender identity, and sexual health included a 5-item Likert scale from 0 = Strongly agree to 4 = Strongly disagree, and a fifth response option: “I have never been asked this question.” The fifth response was recoded as missing.
Since the scales were developed based on theorized constructs, validity of factors was assessed with confirmatory factor analyses (CFAs) for both scales. Diagonally weighted least squares (WLSMV) was used for analyses as the variables were a mix of categorical and continuous variables. Missing values were handled with pairwise deletion. A threshold of .40 was set as acceptable for factor loadings. Items were dropped from theorized factors if the factor loading was under the a priori threshold or there was cross-loading.
Convergent validity was assessed based on McDonald omega coefficient. McDonald omega (ω) is computed as the ratio of the variance due to the latent factor to the total variance. It has the advantage over Cronbach’s alpha as it does not assume Tau-equivalence (eg, equal factor loading of all items). Like Cronbach’s alpha, a value >0.7 indicates unidimensionality or convergent validity. Criterion validity was assessed by testing correlations between 3 factors for the QUIRKS-Provider (Knowledge, Attitudes, Clinical Preparedness), and the corresponding similar factors in the LGBT-DOCSS for providers.8 Statistical significance was set to P <.05. Chi-square tests were conducted to test association between each item (dichotomous variable) in the QUIRKS-Patient and QUIRKS-Provider reported Environmental Cues (the only factor in common for both scales). Mplus 8.4 statistical software was used for the CFA analyses and SPSS statistical software was used for other analyses.
Mean, SD, and skewness of each item were assessed. Descriptive statistics were reported. Correlations between items were assessed using Pearson coefficients.
Participants for both samples varied by race and ethnicity, with most respondents identifying as White (82.7% for patients vs 81.1% for providers). The majority of respondents identified as cisgender women (72.3% for patients vs 83.2% for providers). Approximately 73% of both samples identified as straight, with the remaining participants reporting a sexual minority status. The mean age of patient respondents was 58.5 (SD = 12.63) and 44.3 (SD = 11.9) for providers. Providers reported residence in 29 different US states, with the greatest representation from Ohio (n = 27) followed by Pennsylvania (n = 14) and New York (n = 13). Patients reported residence in 26 different states, with the greatest representation from Oregon (n = 63) followed by Illinois (n = 28), and Georgia (n = 10) (Table 1).
The 5-factor CFA was conducted based on theorized factors for the QUIRKS-Provider. However, the initial model fit was not satisfactory (X2: 828.48; degrees of freedom [df] = 454; P <.001; root mean square error of approximation [RMSEA]: .076; comparative fit index (CFI): .534; Tucker-Lewis index [TLI]: .490; standardized root mean squared residual [SRMR]: .139). Based on factor loading, cross-loadings, and modification indices, we revised the model.
First, items with factor loadings <0.4 were removed, which included 1 item (“Sexual orientation is relevant when it comes to breast reconstruction preferences among breast cancer survivors”) from Knowledge, 1 item (“I encourage my LGBTQI patients to document advance directives”) from Clinical Behaviors, and 4 items (“My patients would not understand why they are being asked about both sex assigned at birth and gender identity” [reverse coded]; “I am as comfortable discussing sexual health concerns with LGBTQI patients as I am with heterosexual, cisgender patients”; “I would like more training on how to better care for LGBTQI patients”; and “If I explained why I was asking about both sex assigned at birth and gender identity, my patients would be comfortable with me asking”) from Attitudes.
Several items had cross-loadings, eg, the items of “There are cues in my healthcare practice that show I welcome lesbian, gay, bisexual, and transgender patients” had high loadings (≥0.4) onto 3 latent factors (Knowledge, Clinical Behaviors, and Attitudes). The item “If a transgender patient wanted psychosocial support, I would know who to refer them to” had high loadings (>0.4) onto 3 other latent constructs (Environmental Cues, Clinical Preparedness, and Clinical Behaviors). We removed these items from the model due to cross-loadings. Based on the modification indices, we correlated the residual errors of the item “It is my professional responsibility to champion positive system changes to support LGBTQI patients” and the item “It is my professional responsibility to learn about LGBTQI patient needs” (standardized residual covariance = .820; SE = .011; P <.001) and correlated the residual errors of the item “If I witness people discriminating against an LGBTQI patient, I actively challenge that behavior” and the item “If I witness people making jokes about LGBTQI people, I actively challenge that behavior” (standardized residual covariance = .630; SE = .041; P <.001). The residual errors may be correlated due to similar wording in the questions.
The final model showed satisfactory model fit (X2: 268.89; df = 240; P =.02; RMSEA: .037; CFI: .926; TLI: .915; SRMR: .079) (Table 2). In summary, the CFA confirmed 5 factors for the QUIRKS-Provider. Based on McDonald’s omega, the scale also supports unidimentionality for each latent factor: Environmental Cues (ω = .920), Knowledge (ω = .750), Clinical Preparedness (ω = .934), Clinical Behaviors (ω = .609), and Attitudes (ω = .685).
Correlation coefficients among similar items from the Knowledge, Clinical Preparedness, and Attitudes factors of the QUIRKS-Provider and the same latent constructs from LGBT-DOCSS showed strong and significant correlations between each pair of corresponding factors: Knowledge (r = –0.616, P <.001), Clinical Preparedness (r = –0.659, P <.001), and Attitudes (r = –0.580, P <.001). Notably, the Clinical Behaviors factor of the QUIRKS-Provider was statistically significantly correlated with all of the other factors of the QUIRKS-Provider as well as all of the LGBT-DOCSS factors at P <.001. Note that the correlations between the QUIRKS-Provider and LGBT-DOCSS constructs are negatively associated due to reverse directionality: higher scores on the LGBT-DOCSS reflect more affirming care, whereas lower scores of the QUIRKS represent more affirming care (Table 3).
The initial CFA based on the theorized factors with WLSMV did not fit the data satisfactorily (X2 = 120.38; df = 74; P <.001; RMSEA: .060; CFI: .841; TLI: .805; SRMR: .092). To revise the model, one item was removed with factor loading <.4 (standardized factor loading: .233 for the item, “I have personally experienced discrimination in a cancer care interaction (R)”). Based on the modification indices, we correlated the residual errors between “There are cues in my cancer care provider’s office that welcome various racial and ethnic groups” and “There are cues in my cancer care provider’s office that welcome lesbian, gay, bisexual, and transgender (LGBT) patients” (standardized covariance between residual errors: .581; SE = 0.1; P <.001). The correlated errors are likely attributed to the similar wording between the questions. The final CFA provides satisfactory fit (X2 = 82.31; df = 61; P = .036; RMSEA: .045; CFI: .923; TLI: .902; SRMR: .084) for Environmental Cues.
The final QUIRKS-Patient included 13 items and 3 factors: Environmental Cues (5 items, ω = .837), Comfort Discussing Sexual Orientation and Gender Identity (5 items, ω = .887), and Satisfaction with Care (3 items, ω = .850). For final factors and factor loadings, see Table 4. No extant validated scales measure these constructs, thus criteria validity was unable to be assessed.
Chi-square tests showed a significant difference between patient-reported and provider-reported responses to the question regarding cues that welcome LGBT patients in the healthcare practice (20.81% vs 60.14%; P <.001) and ability to document legal name (42.77% vs 58.04%; P = .01). Ability to document pronouns (32.95% vs 41.26%) and sexual orientation (41.62% vs 45.45%) were less often endorsed by patients than providers, but differences were not significant.
Growing attention to the health and healthcare needs of LGBTQI persons demands validated measures to assess the healthcare experiences of these patients. There are few validated instruments to measure antigay bias10 and only 1 scale specific to measuring healthcare provider knowledge, attitudes, and clinical preparedness in working with LGBT patients, collectively.8 Prior to the development of the QUIRKS-Provider, we know of no scales that disaggregated L-G-B-T patients or included intersex persons when reporting provider-reported preparedness. Most prior research has focused on instrumentation to assess attitudes regarding homosexuality specifically, but measures have not been specific to healthcare or oncology experiences.11 More recently, there has been work to develop and validate scales that measure healthcare provider trans-inclusivity12 and discrimination.13 Currently, a study is ongoing to measure patient-reported outcomes of adolescents receiving gender-affirming treatment.14
Our recruitment strategy included patients and providers of all sexual orientations and gender identities to capture the degree of visibility of affirming cues and behaviors for all patients regardless of sexual orientation or gender identity. This strategy was informed by the belief that both heterosexual, cisgender patients and LGBTQI patients should be able to report their sexual orientation, gender identity, and name in use on intake forms and medical documentation. Furthermore, if SOGI data collection is to become normalized, patients of all sexual orientations and genders need to be asked these questions routinely.
A major contribution of this study is the inclusion of provider-reported behaviors critical to the actual provision of affirming care to SGM people. Another major contribution of this study was the development of companion items to examine patient-reported and provider-reported experiences of SGM-affirming environmental cues. The fact that providers reported more affirming environmental cues compared with patients in this study may be due to social desirability bias on the part of providers or lack of relevance to—and thus lack of attention by—some participants in the sample. The purpose of capturing both perspectives is to allow researchers and clinicians to observe differences in assumptions between healthcare practitioners and the patients they serve in terms of provision of a welcoming, affirming care environment. Differences may provide opportunities for healthcare practices to improve the intentionality and visibility of welcoming cues for SGM patients.
In terms of limitations, our study included a participant sample that was disproportionately White. Further, participants recruited from TEAM SGM for the QUIRKS-Provider sample may represent healthcare providers with more affirming stances toward SGM persons given self-selection into an educational intervention designed to improve care for this population. Finally, while inclusion of cisgender, heterosexual individuals may be perceived as a weakness, the inclusion of these participants was intentional in order to normalize SOGI data collection and assess experiences of affirming environments regardless of sexual orientation or gender.
Future research should test these scales in diverse settings with diverse populations. Research in areas that are not cancer-specific would be appropriate for 4 of the 5 subscales of the QUIRKS-Provider and the entire QUIRKS-Patient scale. Additional comparisons of patient- and provider-reported environmental cues in diverse settings would help researchers determine the degree of correlation in patient- and provider-reported experiences. Regression analyses examining which provider-reported constructs (eg, more Affirming Attitudes, Clinical Preparedness, Clinical Behaviors) as measured via the QUIRKS-Provider are associated with greater patient satisfaction as reported through the QUIRKS-Patient could guide which subscales are most critical to include in research to meaningfully advance health equity in SGM populations.
The QUIRKS scales can be used in health services research to compare patient- and provider-reported experiences with SGM-affirming care. Each scale measures relevant constructs for patients and providers, respectively, while measuring one common construct to allow for comparisons between patient-reported and provider-assumed experiences with SGM-affirming care. Use of these validated scales can strengthen rigor of future research focused on improving care for SGM patients.
- National Academies of Sciences. Understanding the Well-Being of LGBTQI+ Populations. Washington, DC: National Academies Press; 2020.
- Özdemir BC, Csajka C, Dotto GP, Wagner AD. Sex differences in efficacy and toxicity of systemic treatments: an undervalued issue in the era of precision oncology. J Clin Oncol. 2018;36:2680-2683.
- Centers for Disease Control and Prevention. Alcohol and Cancer. www.cdc.gov/cancer/alcohol/index.htm. Accessed December 23, 2021.
- Peitzmeier SM, Khullar K, Reisner SL, Potter J. Pap test use is lower among female-to-male patients than non-transgender women. Am J Prev Med. 2014;47:808-812.
- Schmetzer O, Flörcken A. Sex differences in the drug therapy for oncologic diseases. Handb Exp Pharmacol. 2012;(214):411-442.
- Pratt-Chapman ML, Alpert AB, Castillo DA. Health outcomes of sexual and gender minorities after cancer: a systematic review. Syst Rev. 2021;10:183.
- Stein GL, Berkman C, O’Mahony S, et al. Experiences of lesbian, gay, bisexual, and transgender patients and families in hospice and palliative care: perspectives of the palliative care team. J Palliat Med. 2020;23:817-824.
- Bidell MP. The Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS): establishing a new interdisciplinary self-assessment for health providers. J Homosex. 2017;64:1432-1460.
- Together, Equitable, Accessible, Meaningful (TEAM) Cancer Care. GW Cancer Center. https://cancercenter.gwu.edu/training-and-education/professional-education/team-together-equitable-accessible-meaningful. Accessed November 29, 2021.
- Pratt-Chapman ML, Ward AR. The Index of Attitudes Toward Homosexuals: factor analysis in a national sample of oncology clinicians. J Homosex. 2021;69:1238-1251.
- Grey JA, Robinson BB, Coleman E, Bockting WO. A systematic review of instruments that measure attitudes toward homosexual men. J Sex Res. 2013;50:329-352.
- Kattari SK, Curley KM, Bakko M, Misiolek BA. Development and validation of the Trans-Inclusive Provider Scale. Am J Prev Med. 2020;58:707-714.
- Watson LB, Allen LR, Flores MJ, et al. The development and psychometric evaluation of the Trans Discrimination Scale: TDS-21. J Couns Psychol. 2019;66:14-29.
- Klassen AF, Kaur M, Johnson N, et al. International phase I study protocol to develop a patient-reported outcome measure for adolescents and adults receiving gender-affirming treatments (the GENDER-Q). BMJ Open. 2018;8:e025435.