Friday Oct 10, 2025
Episode 68: Fraus P-Valoris - I
In this two-part episode, we delve into the phenomenon of p-hacking. What are the various terms used to describe practices that inflate error rates? How does terminology shape our understanding and bring about change? What are its necessary and sufficient conditions, and which practices are most common?
Shownotes
- Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: a key to the file-drawer. Journal of Experimental Psychology: General, 143(2), 534.
- Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366.
- Stefan, A. M., & Schönbrodt, F. D. (2023). Big little lies: A compendium and simulation of p-hacking strategies. Royal Society Open Science, 10(2), 220346.
- John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524-532.
- Fiedler, K., & Schwarz, N. (2016). Questionable research practices revisited. Social Psychological and Personality Science, 7(1), 45-52.
2 months ago
@Edoardo “Dado” Marcora : Yes, indeed! And keep up the good work!
2 months ago
@Edoardo “Dado” Marcora : ... in a future episode
2 months ago
@lakens : I am trying to do my part both in my teaching and in professional interactions. Many students complain I am not teaching them what their supervisor wants them to use (mindless NHST) and in professional interactions I always get shutdown with ”everyone else does it”. I start to become hopeless, especially now that AI is being used to do data analysis without any understanding of the code that is produced, the models being used, and the inferences being produced. Maybe a topic worth pursuing in a
2 months ago
Ignorantia juris non excusat, but the law needs to be properly promulgated - that is the problem with p-hacking, as many disciplines do not teach or discuss it.
2 months ago
Does the law admit ignorance? I don’t see ”not knowing one is doing p-hacking” as an excuse, especially when statistical claims can affect policy or medical decisions.