Cognitive Testing of Healthcare Professional and Patient Scales to Assess Affirming Care for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Persons: The QUeering Individual and Relational Knowledge Scales (QUIRKS)

March 2023 Vol 14, No 3


Original Research

Leila Habib,1 Sylvia Shirima,1,2 Yan Wang,1 Markus Bidell,3 Gwendolyn Quinn,4 Asa Radix,5 Kristen Eckstrand,6 Barbara Warren,7 Erin Loubier,8 Krisztina Szabo,8 Charles Kamen,9 Mandi L. Pratt-Chapman2,10

1The George Washington University, Milken Institute School of Public Health, Washington, DC

2The George Washington University, School of Medicine and Health Sciences, GW Cancer Center, Washington, DC

3City University of New York, Hunter College, New York, NY

4Grossman School of Medicine, Departments of OB-GYN and Population Health, Perlmutter Cancer Center, New York University, New York, NY

5Callen-Lorde Community Health Center, New York, NY

6University of Pittsburgh, Department of Psychiatry, Pittsburg, PA

7Mount Sinai Health System and School of Medicine, New York, NY

8Whitman-Walker Health, Washington, DC

9University of Rochester, Rochester, NY

10The George Washington University, School of Medicine and Health Sciences, Department of Medicine, Washington, DC

This study aimed to develop a pair of scales to assess impact of health education on patient- and provider-reported experiences that could signal a welcoming cancer care environment for sexual and gender minority (SGM) patients. Both scales assessed clinical environment and clinician behaviors. The provider-facing scale also assessed objective knowledge and self-reported preparedness in meeting the healthcare needs of SGM patients. The patient-facing scale assessed experiences of care and satisfaction. Participants were recruited using purposive sampling for maximum variation. Each 60-minute interview (n = 9) was conducted via Zoom. Participants were asked about their thought process when reading a question, comprehension, ease of retrieving the scenario asked about, relevance, and appropriate response options. Detailed notes from interviews were used to refine the scales. Changes to scale items were made to improve comprehension, response formulation, and relevance based on feedback. A few additional items were crafted based on topics voiced as important to patients and providers. This study supports the use of cognitive interviewing prior to scale finalization, particularly when using terminology that is not common among all potential respondents. Additional research to refine scales to compare provider- and patient-reported experiences of SGM-affirming healthcare is warranted.

Sexual and gender minority (SGM) patients experience a cluster of risk factors that may lead to disparate cancer health outcomes. For example, bisexual women are among the highest users of tobacco,1 a known risk factor for many cancers. During the COVID-19 pandemic, gay, lesbian, and bisexual persons reported greater anxiety, depression, and problematic alcohol use compared with cisgender straight counterparts.2 Chronic stress and distress along with these maladaptive coping mechanisms support iatrogenic conditions for SGM people at the cellular and social levels. However, it is challenging to know the degree of need among SGM people since sexual orientation and gender identity are not routinely collected in clinical practice or research.3 The National Academies of Sciences, Engineering, and Medicine4 and the American Society of Clinical Oncology5 have called for greater attention to addressing the needs of SGM patients with cancer.

While there is increasing research on physician knowledge, attitudes, and behaviors when interacting with lesbian, gay, bisexual, transgender, queer, and intersex (LGBTQI) patients, less is known about other health professionals or about in-clinic cues that might signal a welcoming environment for SGM patients.6,7 Emerging research has indicated suboptimal knowledge of SGM health issues,8 as well as heteronormativity and homophobia in healthcare settings9 that vary by degree of exposure to SGM persons,10 amount of SGM-specific education received,10 and demographic characteristics.11 Providers with more education and greater clinical exposure to SGM patients, as well as women, tend to have more accepting attitudes and greater knowledge on SGM care.

Learning interventions have demonstrated improvements in knowledge and confidence among healthcare professionals in caring for SGM patients.6,12 However, most studies have not used validated scales, instead using internally developed, study-specific scales. Extant scales have historically excluded assessment of clinical preparedness in working with people with intersex conditions, and few scales assess clinical behaviors. Additionally, few scales disaggregate subpopulations under the LGBTQI umbrella.

The purpose of this study was to develop a pair of data collection instruments that could be used for health education research and clinical quality improvement initiatives to assess clinic environment and clinician behaviors that may support affirming SGM clinical care. While most of the items can be used to assess LGBTQI-affirming care in any healthcare setting, knowledge questions were specific to cancer care. Nurse and patient navigators are uniquely positioned to assess and address challenges that SGM patients might experience by serving as a broker to healthcare teams regarding SGM patient challenges; this study provides considerations for language when assessing SGM patient experiences. This was the first of a 2-part process of measure development to first ensure face validity through cognitive testing and then conduct psychometric validity testing for items (see page 71).

Instrument Development

Two scales were developed: the QUeering Individual and Relational Knowledge Scale (QUIRKS)-Provider and QUIRKS-Patient, respectively.

Provider scale. The only known validated scale to assess clinician self-reported knowledge, skills, attitudes, and preparedness in caring for LGBT people is the Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS).13 However, the LGBT-DOCSS is generic, does not have cancer-specific knowledge measures, and uses “LGBT” throughout the scale rather than disaggregating patient confidence and preparedness for specific subpopulations under the LGBT umbrella. QUIRKS items were developed based on metrics for LGBT-affirming care recommended by the Human Rights Campaign Healthcare Equality Index to address the following gaps in the literature: (1) provide a cancer-specific scale; (2) disaggregate provider self-reported preparedness in caring for gay, lesbian, bisexual, transgender, and intersex patients to allow for comparisons between these groups; (3) assess environmental and behavioral cues that could signal welcoming and affirming care environments for LGBTQI patients; and (4) assess objective knowledge regarding gender, bias, health risks, and healthcare considerations relevant to SGM patients in general and cancer risks in particular.

Item refinement was iterative: first, the provider scale was refined after discussion with the author of the LGBT-DOCSS, and then feedback was obtained through an open-ended survey to a small group of health services researchers (n = 5). The open-ended survey asked experts what they thought each question was asking, whether the item was important to measure LGBTQI clinical competence, and recommendations to improve the wording of the item. The resulting scale is referred to as the “original QUIRKS-Provider.” This study reports on changes to the scale after conducting cognitive interviews with diverse healthcare professionals (the “revised QUIRKS-Provider”).

Patient scale. The patient scale was developed to be brief and to ask only about environmental and behavioral cues for affirming SGM care as well as patient perceptions of care quality. This study refers to the “original QUIRKS-Patient” as the scale developed to correspond with the original provider scale and the “revised QUIRKS-Patient” as the version refined after conducting cognitive interviews.

Participant Recruitment

Participants were recruited through phone and e-mail personal invitations from the study team’s professional networks using purposive sampling for maximum variation of clinical expertise (provider instrument) and lived experiences (patient instrument). For the patient scale, cancer survivors diverse in sexual orientation, gender identity, and race/ethnicity were sought to obtain a range of reactions to wording of items from those both inside and outside the SGM community.

Data Collection

Participants were interviewed via Zoom. Interviews lasted approximately 60 minutes. The interview methodology and interview guide (Appendix 1) were developed based on Schildmann et al.14 Participants were first asked to describe their thought process as they read and answered each item in the QUIRKS questionnaire, which is a technique known as “think aloud.” After the initial think aloud, the interviewer (LH) probed participants for what they thought the question was asking (comprehension) and whether the response options aligned with the question (response formulation). Feedback on important topics missing and acceptability of items was also solicited. Detailed notes were taken during interviews.

Data Analysis

Notes from interviews were analyzed based on Tourangeau’s (1984) 4-stage question response model as described by Schildmann et al.14 Interviews were conducted until no new themes emerged and analyzed for comprehension, retrieval, judgment, and response formulation. Suggested changes to the topics as well as comments about the general acceptability of the scale were noted.

Ethical Considerations

The study was approved by the George Washington University Institutional Review Board (NCR213247). Participants provided verbal informed consent after the information sheet (Appendix 1) was reviewed.

Sample Characteristics

Cognitive interviews included healthcare providers (n = 5) and individuals with a cancer diagnosis history (n = 5), one of whom fit both descriptions and was counted twice (total n = 9). Providers represented a range of disciplines, and patients were from various racial backgrounds and survived different types of cancer.

Provider Interview Findings

Original questions, comments, and revisions for the QUIRKS-Provider scale can be accessed through the QR code at the end of the article.

Comprehension. Providers varied in their understanding of “signs” for welcoming SGM patients in Question 4. Most participants (n = 3) interpreted the word literally, thinking about physical objects on the wall: “If you want it to be not literal, you need to add more words...If you’re trying to ask people, ‘Was the environment welcoming?’ That would be a different question, because a welcoming environment could mean a million different things” (Participant 3). The phrase “intake form or process” gave rise to multiple interpretations (Questions 5-7). Most respondents (n = 4) focused on a physical form completed by patients during their visit, while 1 participant interpreted the phrase as flexible: “I’m assuming it could be whatever (is) available—it could be electronic, it could be you write on pen and paper, what someone’s asking you” (Participant 2). One participant did not understand the term “implicit bias,” and another participant suggested adding an answer option that read, “I don’t know what this means,” or “I don’t understand the question.” The phrase “clinically prepared” (Questions 15-18) was challenging for some respondents: “Does it mean I need special equipment for something?” (Participant 5). Lastly, participants were uncertain about what it meant to “respectfully confront” a colleague about discrimination or discriminatory jokes. One respondent said: “Confront is a little aggressive...People may have discussions with other providers around their discrimination, but I don’t think they would characterize it as confront” (Participant 2). Another respondent asked what was meant by “confront”: “Calling them out, sending them a passive-aggressive e-mail, making a scene in public, reprimanding them?” (Participant 5).

Retrieval. No participants reported difficulty remembering information to answer the questions from the scale, except for 1 participant who could not answer the question about an intake form: “I don’t know what the form looks like. It’s all computerized now. We don’t see the paper that they’re filling out” (Participant 3).

Judgment. Nonclinicians had some difficulty answering questions about clinical topics (eg, patients’ hormonal balance, clinical preparedness, and screening considerations), and another participant indicated that the word “clinic” did not apply to them: “I don’t work in a clinic; I work in a nonprofit…‘practice’ might be more inclusive. ‘In my practice setting,’” (Participant 5). Others felt some questions did not apply due to their specialty: “If you were taking care of someone’s basal cancer, are you going to consider the hormonal balance?” (Participant 2). Participants felt questions about sexual health and sexual orientation would be more applicable to certain specialties over others: “I make it available for them to disclose their sexual orientation because I deal with breast and cervical where that is extremely important. But those who do other cancers don’t build the same rapport and intimacy level with patients to ask and disclose their sexual orientation” (Participant 4). Participants recommended adding an option: “Not relevant to my specialty” and the disclaimer: “Some of these questions may require you to think of yourself outside of your current specialty practice. Please give your best answer.” One individual proposed adding, “for cancers that are potentially impacted by hormones,” to the question about hormonal balance.

Participants noted that some questions were double-barreled by including values and preferences in 1 question. Similar difficulties were noted in answering questions about “treating all patients the same”: “I do treat all my patients the same, but I disagree with that second part…I don’t like this question. It’s a divided question” (Participant 1). Participants had difficulty answering questions about experiences they never had, such as confronting a colleague: “You might have someone who would feel comfortable doing it, but they’ve never had to…I would say I’m not sure, because I can’t think of a specific instance where I’ve had to go out of my way to confront someone about this” (Participant 5). Two participants were unsure about how to respond if they may have confronted individuals, but not respectfully. “Sometimes my confrontation is not respectfully done—that’s why I couldn’t say ‘always’... Do you want to know if people confront, or if they want people to respectfully confront?” (Participant 3). Participants also had difficulty evaluating inclusion of same-sex partners in decision-making, because they would base that decision on the patients’ desires: “It doesn’t matter to me as a provider who the person is—it’s who the patient wants the person to be” (Participant 5). Lack of inclusivity of the language was also noted: “Shouldn’t you just have partners? Why would you put same-sex partners? should include all partners” (Participant 2). Two participants felt that some questions, about championing change or professional learning, would lead to social response bias: “What are they gonna say, ‘No, I strongly disagree that it’s my professional responsibility?’...I think people will feel badly about saying anything other than strongly agree or agree” (Participant 3). Instead, the participant suggested asking if providers have participated in change-making efforts in the past, or if they would be willing to support change-making efforts in the future.

Response Formulation. Providers demonstrated some difficulty with the response options for sex assigned at birth, gender identity, and sexual orientation due to a lack of familiarity with the terms intersex, two-spirit, same-gender loving, and pansexual: “Definitions are extremely important. Even though I consider myself a champion, I need them. If we’re going to be collecting data and asking the questions, we should know what the definitions are” (Participant 4). One participant thought there were answer options missing for the gender identity question, suggesting the addition of “Man” and “Woman”: “Some people may not identify as cisgender man or transgender...I understand why you have it, because you don’t want people think you’re defaulting to cis when you say ‘man’ or ‘woman,’ but in reality, that doesn’t happen. A lot of trans folk pick man or woman instead of trans man or woman” (Participant 2). Two participants demonstrated difficulty responding when questions that were asked negatively—such as in Question 31 (“impossible”) and Question 33 (“irrelevant”). One participant felt they would be better able to match their responses to answer options on an Always/Never scale than the Strongly agree/Strongly disagree scale for Question 22, which asks if patients may feel insulted when asked about their identities, and Question 27, which asks if transgender people are more likely to experience rejection by their families. However, another participant disliked Always/Never in scales: “I hate always and never—I just feel like nobody does anything all the time and never is always the wrong answer. I’m just not a fan of those never/always” (Participant 3).

Additional Topics to Cover. Some participants had suggestions for questions to include in the scale that were not already addressed. One person recommended adding questions about cancer screening, stating, “We’ve all known people who’ve gone to their provider and they go, ‘you’re not at risk,’ and they don’t do the screening” (Participant 2). Other suggestions were to include questions about advance directives and providers’ comfort discussing sexual health: “Are people discussing those types of things with patients who identify as LGBT? Because I’m sure they’re not” (Participant 2).

Acceptability. Generally, participants found the scale acceptable, but a few expressed concerns about length, terminology, applicability, and order of questions. No participants found any of the questions upsetting, embarrassing, or inappropriate. Two participants believed the answer options for gender identity and sexual orientation were too lengthy. One provider stated, “If this is a general form that’ll be seen, 13 is a lot to choose from” (Participant 1). Another concern raised was whether all providers should be expected to know all the information in the scale: “If any provider needs to know the answers to these questions, I disagree wholeheartedly...that’s why we have specialists. And I don’t feel bad about that” (Participant 3). One participant suggested that asking about sex assigned at birth up front could make scale respondents uncomfortable. Concern about the demographic questions were echoed by another participant, who expressed a general dislike of similar questions but understood why they were asked in this context: “I am who I am—I don’t need to tell people who I am and validate who I am, so in the context of this in collecting’s important...but I feel weird requiring to be identified and requiring others to identify themselves” (Participant 4).

Patient Interview Findings

The original questions, feedback, and revisions for the QUIRKS-Patient can be accessed through the QR code at the end of the article.

Comprehension. Similar to providers, patient respondents had difficulty with the term “signs” (Questions 4 and 5). One participant understood “signs” to include magazines that are of interest to members of the LGBT community, while another comprehended “signs” to mean a physical object on the wall that read “LGBT is [sic] welcome here at our offices.” There was general consensus about the meaning of “intake form or process,” in Questions 6, 7, and 8, where all but 1 participant understood the phrase to refer to paper forms completed upon one’s arrival at the healthcare setting: “I was thinking about papers, because every time I…go to a new practice or a new physician, they would tell me to arrive early to complete forms” (Participant 6). However, 2 participants noted the intake process may be more “complex” than solely a paper form due to the various entry points one may have into a medical system and myriad individuals that may be involved: “Where my mind goes is thinking about how patients come into the system...‘Was there a point at which someone gave me the opportunity to disclose my sexual orientation?’ makes more room for an ability to understand the question” (Participant 8). All respondents had a common understanding of what “discrimination” meant (Question 13). One participant was confused with the word “can” in Question 14 due to multiple possible interpretations of the phrase: “It could mean ‘I feel comfortable,’ ‘I feel like they make the time for me,’ ‘I feel like they ask,’ ‘I feel like I can give an answer’” (Participant 5).

Retrieval. Participants had little difficulty remembering information to answer questions, with 3 exceptions: “It’s been too long to feel like I can definitively answer [the question about discrimination]” (Participant 5). Another participant had trouble remembering the intake form at their doctor’s office (Participant 7). Lastly, 1 participant who had not been to their cancer clinic for an extended period stated that it was challenging to decide how to answer the questions about signs in the clinic (Questions 4 and 5) and the intake process (Questions 6, 7, and 8): “The last time I was there was about a year ago. If they’ve been put up since then, I don’t know...I would say I don’t know” (Participant 5).

Judgment. Participants had difficulty responding to scenarios they had not encountered, such as being asked about sex assigned at birth or gender identity: “It’s a clear-cut statement, but because I’m not of the LGBT community, I can’t answer, because no one has ever asked me about my sex assigned at birth, so it does not apply to me. So I don’t know if it’s an uncomfortable or comfortable feeling” (Participant 9). Another difficulty participants reported was determining which doctors to consider when answering the question. For instance, Questions 6 to 8 prompted respondents to think about their “doctor’s intake form or process,” but participants shared feelings of uncertainty about which doctor(s) to consider: “The question is, are you asking someone about 1 specific doctor’s office or all of their doctors’ offices? Rarely are you seeing 1 doctor—you’re seeing 20 doctors, especially if you have cancer” (Participant 5). Two participants reported trouble determining if they experienced discrimination in their cancer care (Question 13) due to lack of clarity about the type of discrimination (personal vs group) and perpetrator inquired about: “Was it discrimination at the front desk? Was it discrimination by your provider? By your healthcare company? By other patients? By another member of the team?” (Participant 5). In the other case, the respondent was not the recipient of discrimination but experienced seeing discrimination (Participant 7).

Response Formulation. The most common problem with response formulation was regarding questions about gender identity and sexual orientation (Questions 2-3). Almost all participants who completed the patient survey (n = 4) had difficulty understanding at least 1 of the answer options to describe gender identity and sexual orientation. Participants were unfamiliar with “two-spirit” (n = 3), “agender” (n = 3), “cisgender” (n = 3), “pansexual” (n = 3), “asexual” (n = 2), “questioning” (n = 1), and the difference between “genderqueer” and “nonbinary” (n = 2): “Seeing the other [terms] there that I had never heard of before threw a wrench in my thought pattern. I recognize cisgender, but you don’t hear it often” (Participant 7). “In my generation, cisgender was not a term we were using to describe gender identity. I know that’s a newer term now” (Participant 6).

Participants also had challenges when answering negatively worded statements, such as Question 9, which begins with “I don’t understand” and asks respondents to state how much they agree with the statement: “I don’t think ‘I don’t understand’ is a good way to start that question...Is it ‘I strongly agree that I don’t understand...’ ‘I strongly disagree that I don’t understand why they ask me...’ I really can’t answer that question the way it’s posed” (Participant 9). Similar confusion occurred with Questions 10 and 11, which began with “I feel uncomfortable,” and 1 participant answered the questions as if they read “I feel comfortable.”

Due to uncertainty about which providers to consider in responses to Questions 6 to 8 about intake forms, some participants had trouble matching their answer to the “Yes/No/I don’t know” options provided: “Some doctors will ask about it, but I would say more often than not, they don’t...It depends on the context of the doctor’s office. More likely at a PCP or gyno” (Participant 5).

Additional Topics to Cover. Some participants had suggestions of other topics to include, such as asking patients if they felt they could talk to their providers about their psychosocial health needs or social support: “I think care goes beyond clinical care and getting good clinical treatment. Some of that has to do with how patients’ access and preferences and needs are assessed, not just around gender and sexual identity, but around a whole host of other issues like financials, support system, transportation” (Participant 8). One participant suggested asking questions about different members of an individual’s care team separately: “Sometimes when you think about an oncology care team, there are a lot of different players that come into play and are part of the team...1 experience with 1 person that is very disaffirming or very bad to them may ruin the experience of the whole team” (Participant 5).

Acceptability. Overall, participants had favorable attitudes toward the scale. No participants found any questions upsetting, embarrassing, or inappropriate. No participants suggested that any topics be removed from the scale. One participant noted that the order of the questions may affect other respondents’ acceptance of the questionnaire, since questions related to sex assigned at birth, gender identity, and sexual orientation come first, while general questions about general cancer care are at the end: “It makes me think that perhaps it’s a little more skewed towards people who are queer” (Participant 8).


This study assessed constructs that are often tested to assess face validity of items before psychometric testing. Participant responses greatly improved clarity of items for both the QUIRKS-Provider and QUIRKS-Patient. These findings identify ways of optimizing measurements of LGBT-affirming care constructs that can be used by patient support teams to assess and measure quality improvements in cancer care.

While some feedback was consistent across a majority of participants (ie, the word “signs” was confusing to most participants), other feedback was voiced by 1 or 2 participants. The research team took a pragmatic approach to addressing feedback, refining the language of items where there was clear confusion or consensus and optimizing items wherever possible. For original and revised items, please see QR codes at the end of the article.

Similar Revisions Made to Both Scales

To improve comprehension for both scales, the term “signs” was changed to “cues” (Questions 4 and 5), and rather than asking about “intake form or process,” items asked about easy documentation of name, pronouns, and/or sexual orientation. Questions that were negatively phrased (eg, “impossible” and “irrelevant”) were streamlined, but some reverse coding was retained to ensure that respondents slowed down to process directionality of the questions. To improve applicability, the answer options, “I don’t know” and “I have never been asked this question” were added as options for some questions. Due to lack of familiarity with all response options for gender identity, definitions as well as a note reading, “It is okay if you are not familiar with all of these terms. Choose the term or terms that best fit with your identity” were added to the questions about gender identity and sexual orientation. While 2 participants expressed concerns about the large number of answer options provided for sexual orientation and gender identity questions, another participant noted, “It’s really great for so many people to see each other in there.” The difference in perception suggests that those in mainstream culture who routinely see themselves represented in healthcare encounters have starkly different perceptions of questions asking about sexual orientation and gender identity than those who have been historically excluded in healthcare data collection.

Revisions to QUIRKS-Provider Scale

The following additional changes were made to improve the provider scale: Due to uncertainty about what it means to “respectfully confront” a colleague about discriminatory jokes or actions, the phrase was changed to “actively challenge.” References to “clinic” were revised to “practice setting,” and skip logic was used to avoid clinically oriented questions for nonclinicians. Based on feedback to include additional topics, a question asking about provider initiation of advance directives was added. Some participants also felt there were leading questions, such as questions asking about championing systems change (Question 14) and asking about patients’ values and preferences (Question 8). While Question 14 was retained, Question 8 was revised to ask providers about frequency with which they “initiate conversations about patient values and preferences when establishing goals of care” to distinguish from voluntary disclosure.

Revisions to QUIRKS-Patient Scale

The following additional revisions were made to the patient scale: To alleviate the confusion about the multiple implications of the word “can” (Question 14), the item was revised to “I appreciate when my healthcare provider asks me about my sexual health.” Another item was revised to “I have personally experienced discrimination in a cancer care interaction,” to clarify that the question is inquiring about a respondent’s individual experience rather than witnessing of discrimination. The emphasis on personal experience was also applied to other questions such as Question 15, which was revised to read, “My cancer care team provides high-quality clinical care to me personally.” The scale was revised to prompt respondents to answer the questions while thinking about the last time they visited their cancer care provider. With the new prompt, the scale clarifies that any unknown changes since the last time the respondent visited the clinic are not under question.

A major difficulty with judgment that participants expressed was discerning which doctors to include when thinking about their responses. In response, the scale was revised such that the first section prompted respondents to think about the last time they visited their cancer care provider, the second section about all aspects of their cancer care, and the last about all aspects of their healthcare. In addition, “doctor” was changed to “cancer care provider” and “healthcare provider” (as applicable) to account for care from diverse clinicians.

Based on feedback for additional topics to include, a question was added about psychosocial health needs in the QUIRKS-Patient. However, despite suggestions to include multiple questions about perceived social support in cancer care and questions about different members of an individual’s care team separately, for purposes of brevity the research team added only 1 item that broadly asked patients the following: “I appreciate when my healthcare provider asks me about my psychosocial health.” For revised scales, see Appendix 2 and Appendix 3.

Limitations. This study was limited by its small convenience sample. The sample was small due to challenges in recruiting busy clinicians and patients with no incentives and due to pragmatism: the scales were meant to be developed, tested, and then used for evaluation in an educational intervention bound within a designated time line.15

Future Directions. Next steps in this research are to field the revised scale and conduct psychometric testing on the scales based on larger convenience samples. Also, it may be important to disaggregate items in the QUIRKS-Provider that inquire about confronting discriminatory behaviors and professional responsibilities regarding SGM subpopulations. On the other hand, repetitive questions that ask about disaggregated subgroups may contribute to survey fatigue. It may also be important to distinguish types of healthcare professionals by discipline in the QUIRKS-Patient to allow respondents to report variation in care experiences.


This study supports the use of cognitive interviewing prior to scale finalization, particularly when using terminology that is not commonly used among all potential respondents. Participant responses greatly improved the clarity of scale items, although lack of consensus among respondents and research pragmatism prevented responding to every piece of feedback. Additional research to refine scales that allow for comparison of provider- and

patient-reported experiences with LGBTQI-affirming healthcare is warranted. g

Acknowledgment: Thank you to Ruta Brazinskaite for administrative support for this manuscript submission.

Funding: This study was made possible through funding from NIH 5R01CA203604-4-REVISED.

Conflicts of interest/Competing interests: The authors have no conflicts to report.

Availability of data and material: Notes from interviews will be made available upon request to the corresponding author (MPC; This email address is being protected from spambots. You need JavaScript enabled to view it.).

Authors’ contributions: Conceptualization: MPC; Methodology: MPC; Formal analysis and investigation: LH; Writing, original draft preparation: LH, MPC; Writing, review and editing: LH, SS, MB, GQ, AR, KE, BW, EL, KS, CK, MPC; Funding acquisition: MPC.

Ethics approval (include appropriate approvals or waivers): The study was approved by the George Washington University Institutional Review Board (NCR213247).


  1. Romm KF, Huebner DM, Pratt-Chapman ML, et al. Disparities in traditional and alternative tobacco product use across sexual orientation groups of young adult men and women in the US. Subst Abus. 2022;43:815-824.
  2. Akré E-R, Anderson A, Stojanovski K, et al. Depression, anxiety, and alcohol use among LGBTQ+ people during the COVID-19 pandemic. Am J Public Health. 2021;111:1610-1619.
  3. Kamen CS, Pratt-Chapman ML, Meersman SC, et al. Sexual orientation and gender identity data collection in oncology practice: findings of an ASCO survey. JCO Oncol Pract. 2022;8:e1297-e1305.
  4. National Academies of Sciences, Engineering, and Medicine. Understanding the Well-Being of LGBTQI+ Populations. Washington, DC: The National Academies Press; 2020.
  5. Griggs J, Maingi S, Blinder V, et al. American Society of Clinical Oncology Position Statement: Strategies for Reducing Cancer Health Disparities Among Sexual and Gender Minority Populations. J Clin Oncol. 2017;35:2203-2208.
  6. Morris M, Cooper RL, Ramesh A, et al. Training to reduce LGBTQ-related bias among medical, nursing, and dental students and providers: a systematic review. BMC Med Educ. 2019;19:325.
  7. Aleshire M, Ashford K, Fallin-Bennett A, Hatcher J. Primary care providers’ attitudes related to LGBTQ people: a narrative literature review. Health Promot Pract. 2019;20:173-187.
  8. Berner AM, Hughes DJ, Tharmalingam H, et al. An evaluation of self-perceived knowledge, attitudes and behaviours of UK oncologists about LGBTQ+ patients with cancer. ESMO Open. 2020;5:e000906.
  9. Çakır H, Seren AKH. Are nurses homophobic? A survey study in public hospitals in Turkey. J Nurs Scholarsh. 2020;52:613-622.
  10. Nowaskie DZ, Patel AU, Fang RC. A multicenter, multidisciplinary evaluation of 1701 healthcare professional students’ LGBT cultural competency: comparisons between dental, medical, occupational therapy, pharmacy, physical therapy, physician assistant, and social work students. PLoS One. 2020;15:e0237670.
  11. Taskiran Eskici G, Alan H, Bacaksiz FE, et al. Under the same rainbow: a study on homophobia and discrimination among private sector health care professionals. J Nurs Manag. 2021;29:3-15.
  12. Pratt-Chapman ML, Phillips S. Health professional student preparedness to care for sexual and gender minorities: efficacy of an elective interprofessional educational intervention. J Interprof Care. 2020;34:418-421.
  13. Bidell M. The Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS): establishing a new interdisciplinary self-assessment for health providers. J Homosex. 2017;64:1432-1460.
  14. Schildmann EK, Groeneveld EI, Denzel J, et al. Discovering the hidden benefits of cognitive interviewing in two languages: the first phase of a validation study of the Integrated Palliative care Outcome Scale. Palliat Med. 2016;30:599-610.
  15. Pratt-Chapman ML, Wang Y, Eckstrand K, et al. Together-equitable-accessible-meaningful (TEAM) training to improve cancer care for sexual and gender minorities (SGM): outcomes from a pilot study. J Cancer Educ. 2022.

Related Articles
Nurse Navigators as the Drivers to an Enhanced Patient Referral Process
May 2023 Vol 14, No 5
Within the structure of a center-wide patient access redesign initiative, the nurse navigators evaluated their contributions and activities as professional registered nurses. They noticed a lack of standard interview questions and a patient referral process that holistically supported patients.
Community Navigation and Supportive Care Experiences of Low-Income Black and Latina Cancer Survivors: Patient and Navigator Perspectives
April 2023 Vol 14, No 4
Little is known about the role of community-based navigation in supportive care delivery for historically marginalized cancer survivors. The purposes of this study were to evaluate supportive care experiences of low-income Black and Latina cancer survivors and examine the care role of their community navigator.
The QUeering Individual Relational and Knowledge Scales (QUIRKS): Validation of Companion Measures Assessing Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex–Affirming Healthcare
March 2023 Vol 14, No 3
This study conducted psychometric analyses to validate companion scales entitled the QUeering Individual and Relational Knowledge Scale (QUIRKS)-Provider and QUIRKS-Patient for providers and patients, respectively.
Last modified: March 22, 2023

Subscribe Today!

To sign up for our print publication or e-newsletter, please enter your contact information below.

I'd like to receive:

  • First Name *
    Last Name *
    Profession or Role
    Primary Specialty or Disease State