Model Fit, Selection, and Complexity

It is not enough to simply continue to invent models in order to address substantive research problems. At some point, there is often a need for individual researchers to select among one of several competing models. This model selection process is partly holistic in that it balances theoretical concerns, evaluation of overall model fit and relative model fit, and concerns regarding overfitting or model complexity. The ultimate goal is to pick an interim model that we believe has good utility, yet is also parsimonious and may generalize to other datasets. This line of research seeks to provide computationally efficient ways to perform model selection, winnowing down the number of available models, and help inform model selection choices based in part on parsimony.

Representative Publications

  • Somer, E., Falk, C.F., & Miočević, M. (in press). A comparison of regularization, alignment, and a traditional method for estimating structural relationships across multiple groups. Multivariate Behavioral Research. doi: 10.1080/00273171.2026.2650840
    [Preprint]   [OSF]

  • Bonifay, W., Cai, L., Falk, C. F., & Preacher, K. J. (in press). Reassessing the fitting propensity of factor models. Psychological Methods. doi: 10.1037/met0000735

  • Fujimoto, K. A., & Falk, C. F. (2024). The accuracy of Bayesian model fit indices in selecting among multidimensional item response theory models. Educational and Psychological Measurement, 84, 217-244. doi: 10.1177/00131644231165520
    [Preprint]

  • Falk, C.F., & Muthukrishna, M. (2023). Parsimony in model selection: Tools for assessing fit propensity. Psychological Methods, 28, 123-136. doi: 10.1037/met0000422
    [Preprint]   [R Package (GitHub)]

    • Both authors contributed equally to this manuscript
    • Won Research Award from the Quantitative Methods Section of the Canadian Psychological Association (2024)

  • Starr, J., Falk, C.F., Monroe, S., & Vachon, D.D. (2021). A comparison of limited information fit statistics for a response style MIRT model. Multivariate Behavioral Research, 56, 687-702. doi: 10.1080/00273171.2020.1828024

  • Falk, C.F. (2019). Model selection for monotonic polynomial item response models. In In M. Wiberg, S. Culpepper, R. Janssen, J. González, and D. Molenaar (Eds.), Quantitative Psychology: The 83rd Annual Meeting of the Psychometric Society, New York, NY, 2018 (pp. 75–85). Cham, Switzerland: Springer Nature.

  • Falk, C.F., & Monroe, S. (2018). On Lagrange Multiplier tests in multidimensional item response theory: Information matrices and model misspecification. Educational and Psychological Measurement, 78, 653-678. doi: 10.1177/0013164417714506

  • Marcoulides, K.M., & Falk, C.F. (2018). Model specification searches in structural equation modeling with R. Structural Equation Modeling: A Multidisciplinary Journal, 23, 484-491. doi: 10.1080/10705511.2017.1409074
    [R Functions]   [R Package]