Episode 81: Analysis Multiversi

In this episode, we will discuss multiverse analysis–what it is, why we use it, and how we should use it. We also discuss many analyst projects, and Daniel's new preprint on multiverse analysis. 

 

Shownotes

• Auspurg, K. (2025). Robustness is better assessed with a few thoughtful models than with billions of regressions. Proceedings of the National Academy of Sciences, 122(43), e2521917122.
• Chamberlin, T. C. (1890). The method of multiple working hypotheses. Science, (366), 92–96.
• Dochtermann, N. A., & Jenkins, S. H. (2011). Developing multiple hypotheses in behavioral ecology. Behavioral Ecology and Sociobiology, 65(1), 37–45.
• Dreber, A., & Johannesson, M. (2025). The credibility gap: Evaluating and improving empirical research in the social sciences. Routledge.
• Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University, 348(1-17), 3.
• Knief, U., & Forstmeier, W. (2025). Are our 95% CIs only worth 45% confidence? bioRxiv, 2025-08.
• Lakens, D., Rasti, S., & Tunç, M. N. (2026). There is only one correct analysis. (Preprint)
• Platt, J. R. (1964). Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others. Science, 146(3642), 347–353.
• Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2020). Specification curve analysis. Nature Human Behaviour, 4(11), 1208–1214.
• Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712.
• Wimsatt, W. C. (2007). Re-engineering philosophy for limited beings: Piecewise approximations to reality. Harvard University Press.