Towards a Reconsideration of the Use of Agree-Disagree Questions in Measuring Subjective Evaluations
Document Type
Article
Publication Date
2-2022
Publication Title
Research in Social and Administrative Pharmacy
Volume
18
Issue
2
Pages
2335-2344
Publisher Name
Elsevier
Abstract
Agree-disagree (AD) or Likert questions (e.g., “I am extremely satisfied: strongly agree … strongly disagree”) are among the most frequently used response formats to measure attitudes and opinions in the social and medical sciences. This review and research synthesis focuses on the measurement properties and potential limitations of AD questions. The research leads us to advocate for an alternative questioning strategy in which items are written to directly ask about their underlying response dimensions using response categories tailored to match the response dimension, which we refer to as item-specific (IS) (e.g., “How satisfied are you: not at all … extremely”). In this review we: 1) synthesize past research comparing data quality for AD and IS questions; 2) present conceptual models of and review research supporting respondents’ cognitive processing of AD and IS questions; and 3) provide an overview of question characteristics that frequently differ between AD and IS questions and may affect respondents’ cognitive processing and data quality. Although experimental studies directly comparing AD and IS questions yield some mixed results, more studies find IS questions are associated with desirable data quality outcomes (e.g., validity and reliability) and AD questions are associated with undesirable outcomes (e.g., acquiescence, response effects, etc.). Based on available research, models of cognitive processing, and a review of question characteristics, we recommended IS questions over AD questions for most purposes. For researchers considering the use of previously administered AD questions and instruments, issues surrounding the challenges of translating questions from AD to IS response formats are discussed.
Recommended Citation
Dykema, Jennifer; Schaeffer, Nora Cate; Garbarski, Dana; Assad, Nadia; and Bank of America Corp.. Towards a Reconsideration of the Use of Agree-Disagree Questions in Measuring Subjective Evaluations. Research in Social and Administrative Pharmacy, 18, 2: 2335-2344, 2022. Retrieved from Loyola eCommons, Sociology: Faculty Publications and Other Works, http://dx.doi.org/10.1016/j.sapharm.2021.06.014
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.
Copyright Statement
© Elsevier Inc, 2021
Comments
Author Posting © Elsevier Inc, 2021. This is the author's version of the work. It is posted here by permission of Elsevier for personal use. The original article was published in Research in Social and Administrative Pharmacy, VOL.18, ISS.2 ( February 2022), https://doi.org/10.1016/j.sapharm.2021.06.014