Justin A. DeSimone

Research on Careless Responding: Publications and References

Is it Also Time to Revisit Situational Specificity?

Industrial and Organizational Psychology, Perspectives on Science and Practice

This commentary revisits the debate between situational specificity and validity generalization in selection contexts. It argues that contextual differences may meaningfully affect validity estimates and should not be dismissed as artifacts. The piece encourages more nuanced thinking about how job-specific factors interact with generalizable validity evidence.

Attenuation, Correction for

The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation

This encyclopedia entry explains the correction for attenuation, a statistical adjustment that estimates true correlations by accounting for measurement error. It describes the formula, assumptions, and criticisms, noting potential misuse when reliability estimates are inaccurate. The entry provides guidance for researchers on when and how to apply the correction responsibly.

When It’s Incorrect to Correct: A Brief History and Cautionary Note

Industrial and Organizational Psychology, Perspectives on Science and Practice

This article reviews the history of psychometric corrections, especially attenuation and range restriction, and cautions against their uncritical application. It highlights controversies surrounding inflated validity estimates and violations of underlying assumptions. The work underscores the importance of careful methodological judgment when applying statistical adjustments.

New Techniques for Evaluating Temporal Consistency

Organizational Research Methods

This article introduces new statistics for assessing temporal consistency beyond traditional test–retest reliability. It demonstrates how item-level information, interitem correlations, and component structures can reveal instability that test-level metrics may overlook. The methods provide richer diagnostic tools for test developers and researchers evaluating longitudinal measurement.

If It Were Only That Easy: The Use of Meta-Analytic Research by Organizational Scholars

Organizational Research Methods

This article critiques how organizational researchers often oversimplify citations of meta-analyses, reducing them to evidence of a basic relationship. It documents widespread neglect of effect size magnitude, heterogeneity, and methodological nuance. The work calls for more accurate and sophisticated interpretation to fully leverage meta-analytic evidence.

Recommendations for Reviewing Meta-Analyses in Organizational Research

Organizational Research Methods

This piece offers best-practice recommendations for reviewers evaluating meta-analyses in organizational research. It highlights common shortcomings in transparency and replicability, from search and coding to reporting effect sizes and moderators. A reviewer checklist is provided to encourage higher standards and improve the quality of cumulative evidence.

Exploring Joint Variance Between Independent Variables and a Criterion: Meaning, Effect, and Size

Organizational Research Methods

This article introduces joint variance as an often-overlooked explanatory component in regression models. It shows how predictors can share variance in explaining outcomes, revealing substantive relationships missed when focusing only on unique effects. The approach provides new ways to interpret predictor interdependencies in behavioral research.

References

  1. DeSimone, J. A. (2015). New techniques for evaluating temporal consistency. Organizational Research Methods, 18, 133-152.

  2. DeSimone, J. A. (2014). When it’s incorrect to correct: A brief history and cautionary note. Industrial and Organizational Psychology: Perspectives on Science and Practice, 7, 527-531.

  3. DeSimone, J. A., Brannick, M. T., O’Boyle, E. H., & Ryu, J. W. (2021). Recommendations for reviewing meta-analyses in organizational research. Organizational Research Methods, 24, 694-717.

  4. DeSimone, J. A., & Fezzey, T. (2023). Is it also time to revisit situational specificity? Industrial and Organizational Psychology: Perspectives on Science and Practice, 16, 317-321.

  5. DeSimone, J. A., Köhler, T., & Schoen, J. L. (2019). If it were only that easy: The use of meta-analytic research by organizational scholars. Organizational Research Methods, 22, 867-891.

  6. DeSimone, J. A. & LeBreton, J. M. (2018). Attenuation, Correction for. In B. B. Frey (Ed.), The SAGE encyclopedia of educational research, measurement and evaluation (pp. 138-139). Thousand Oaks, CA: Sage.

  7. Schoen, J. L., DeSimone, J. A., James, L. R. (2011). Exploring joint variance between independent variables and a criterion: Meaning, effect, and size. Organizational Research Methods, 14, 674-695.