Response Styles and Aberrant Responding

In this line of research, we seek to develop and evaluate approaches for addressing atypical responses that may occur for some individuals. Atypical responses can undermine the quality of research data and resulting conclusions, as well as decisions for particular test-takers. Such atypical responses may take multiple forms and have multiple different underlying causes. Examples include:

To elaborate, questionnaires that make use of Likert-type items (e.g., rate your agreement to a statement on a scale from 1 – Disagree to 7 – Agree) may be used for assessing personality, emotions, child temperament, patient reported/health outcomes, etc. One problem with such self-report items is known as response styles, which are characterized by a tendency to agree to items (acquiescence), use mostly the endpoints of the scale (1 and 7; extreme response style) or the midpoint of the scale (4; midpoint response style). Response styles are thought to vary depending on age, education, cultural background, etc.

Additional aberrant responding may occur in the context of questionnaire-based research as well as other test-taking situations. Participants in research studies may not take the questionnaire seriously or complete items carefully (i.e., careless responding). Survey bots may contaminate responses to online studies and there is worry of large language models providing responses. Students or licensure examinees test-takers may be motivated to cheat. And prospective employees may wish to appear desirable and competent to prospective employers.

Filtering out noise to measure the construct(s) of interest remains challenging. This line of research employs both model-based (item response theory or factor analytic methods) and model-agnostic / machine-learning approaches to addressing these problems. Identification of such individuals or particular aberrant responses, and how to appropriately handle such responses remains a pressing issue.

Representative Publications

  • Falk, C. F., Huang, A., & Ilagan, M. J. (in press). Unsupervised [randomly responding] survey bot detection: In search of high classification accuracy. Psychological Methods. doi: 10.1037/met0000746
    [Preprint]   [OSF]

  • Ilagan, M.J., & Falk, C. F. (2024). Model-agnostic unsupervised detection of bots in a Likert-type questionnaire. Behavior Research Methods, 56, 5068-5085. doi: 10.3758/s13428-023-02246-7
    [Preprint]   [OSF]   [R Package (dev version)]   [Workshop]

  • Hong, S.E., Monroe, S., & Falk, C.F. (2020). Performance of person-fit statistics under model misspecification. Journal of Educational Measurement, 57, 423-442. doi: 10.1111/jedm.12207

  • Falk, C.F., & Ju, U. (2020). Estimation of response styles using the multidimensional nominal response model: A tutorial and comparison with sum-scores. Frontiers in Psychology: Quantitative Psychology and Measurement, 11:72, 1-17. doi: 10.3389/fpsyg.2020.00072

  • Falk, C.F., & Cai, L. (2016). A flexible full-information approach to the modeling of response styles. Psychological Methods, 21, 328-347. doi: 10.1037/met0000059