Nullius in Verba

Episode 9: Praeiudicium Publicandi

In this episode, we discuss the issue of publication bias. We discuss issues like: Do we learn anything from null results, given the current state of research practices? Is poorly done research still worth sharing with the scientific community? And how can we move toward a system where null results are informative and worth publishing?
 
Shownotes
Bones, A. K. (2012). We Knew the Future All Along Scientific Hypothesizing is Much More Accurate Than Other Forms of Precognition—A Satire in One Part. Perspectives on Psychological Science, 7(3), 307–309. https://doi.org/10.1177/1745691612441216
Carter, E. C., & McCullough, M. E. (2014). Publication bias and the limited strength model of self-control: Has the evidence for ego depletion been overestimated? Frontiers in Psychology, 5. https://doi.org/10.3389/fpsyg.2014.00823
Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82(1), 1–20. https://doi.org/10.1037/h0076157
Fidler, F., Singleton Thorn, F., Barnett, A., Kambouris, S., & Kruger, A. (2018). The epistemic importance of establishing the absence of an effect. Advances in Methods and Practices in Psychological Science, 1(2), 237-244.
Pickett, J. T., & Roche, S. P. (2017). Questionable, Objectionable or Criminal? Public Opinion on Data Fraud and Selective Reporting in Science. Science and Engineering Ethics, 1–21. https://doi.org/10.1007/s11948-017-9886-2
Scheel, A. M., Schijen, M. R. M. J., & Lakens, D. (2021). An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports. Advances in Methods and Practices in Psychological Science, 4(2), 25152459211007468. https://doi.org/10.1177/25152459211007467
Sterling, T. D. (1959). Publication Decisions and Their Possible Effects on Inferences Drawn from Tests of Significance—Or Vice Versa. Journal of the American Statistical Association, 54(285), 30–34. JSTOR. https://doi.org/10.2307/2282137
The FDA Trial tracker to see which trials have not shared their results: https://fdaaa.trialstracker.net/ 
 

Episode 8: Scientia Cultus Sarcinarum

In this episode, we discuss physicist Richard Feynman’s famous speech ‘Cargo Cult Science,’ which refers to work that has all the affectations of science without the actual application of the scientific method. We also discuss topics like: What is pathological science? How might cargo cult science and pathological be different from pseudo-science? How do we know whether or not we’re in a cargo cult, and what can we do to make sure we're not fooling ourselves?
 
Shownotes
Cargo Cult Science (Feynman, 1974)
Gergen, K. J. (1973). Social psychology as history. Journal of personality and social psychology, 26(2), 309–320.
Langmuir, I. (1989). Pathological science. Research-Technology Management, 32(5), 11-17.
Sabine Hossenfelder. No one in physics dare say so, but the race to invent new particles is pointless. The Guardian.
Young, P. T. (1932). Relative food preferences of the white rat. Journal of Comparative Psychology, 14(3), 297.
Young, P. T. (1941). The experimental analysis of appetite. Psychological Bulletin, 38(3), 129.
 

Prologus 8: Cargo Cult Science (R.P. Feynman)

In this bonus episode, we present a reading of the famous speech by physicist Richard Feynman on "science that isn't science," Cargo Cult Science, which will be the topic of the next episode. Enjoy. 

Episode 7: Corpora Regulatoria

In this episode we discuss regulatory bodies their influence on the generation and dissemination of knowledge. Should regulatory bodies have the authority to affect the topics and methods of science? Is more highly regulated research actually better? And should we just give up on our own lines of research and become potato researchers? 
 
Shownotes
Kitcher, P. (2001). Science, truth, and democracy. Oxford University Press.
Indian Council of Agricultural Research (ICAR)-Central Potato Research Institute (CPRI)
Emergence of the NIH 
DARPA SCORE
Lakens, D. (2023). Is my study useless? Why researchers need methodological review boards. Nature, 613(7942), 9-9.
PSA: Legate, N., Ngyuen, T. V., Weinstein, N., Moller, A., Legault, L., Vally, Z., ... & Ogbonnaya, C. E. (2022). A global experiment on motivating social distancing during the COVID-19 pandemic. Proceedings of the National Academy of Sciences, 119(22).
 

Episode 6: Consentio

In this episode, we discuss the importance of consensus in science, both as means of establishing true knowledge and for determining which research questions might be worth pursuing. We also discuss barriers to reaching consensus and the different frameworks that are currently employed for trying to reach consensus among important stakeholders. 
Shownotes
The Popper quote is from: Popper, K. R. (1959). The logic of scientific discovery. Routledge.
The Polanyi quote is from: Polanyi, M. (1950). Freedom in Science. Bulletin of the Atomic Scientists, 6(7), 195–198. https://doi.org/10.1080/00963402.1950.11461263
Planck's Principle: A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it.
The Many Smiles collaboration: Coles, N. A., et al., (2020). The Many Smiles collaboration: A multi-Lab foundational test of the facial feedback hypothesis. Nature Human Behaviour. https://doi.org/10.31234/osf.io/cvpuw
Paul Meehl's 50 year rule: Meehl, P. E. (1992). Cliometric metatheory: The actuarial approach to empirical, history-based philosophy of science. Psychological Reports, 71, 339–339.
Mulkay, M. (1978). Consensus in science. Social Science Information, 17(1), 107-122.
Deliberative Polling
Laudan, L. (1986). Science and values: The aims of science and their role in scientific debate. Univ of California Press.
 

Episode 5: Insanabile Scribendi Cacoethes

In this episode, we discuss the insatiable itch to publish, starting with a quote from 1927 by sociologist Clarence Case on the dictum “Publish or perish.” We discuss ways in which individual goals to publish conflict with the broader scientific goal of producing useful knowledge. We also question the assumptions behind the notion that publishing less would be beneficial for science.
 
Shownotes
Case, C. M. (1927). Scholarship in sociology. Sociology and Social Research, 12, 323-340 (Publish or perish)
Phaf, R. H. (2020). Publish less, read more. Theory & Psychology, 30(2), 263–285. https://doi.org/10.1177/0959354319898250
The term "insanabile scribendi cacoethes" comes from: Merton, R. K. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56-63.
 

Episode 4: Eminentia

In this episode, we discuss the role of eminence in science. What ask questions like: What makes scientists eminent? What role does eminence play in science? Can eminence be spread across scientific teams instead of individuals? And how can we recognize and applaud scientists for their contributions, while avoiding conferring too many benefits on scientists who do become eminent?
 
Shownotes
Eminent psychologists of the 20th century
Intel - Our rock stars aren't like your rock stars

Episode 3: Confirmatio Praeiudicia

In our third episode, we discuss confirmation bias, which affects not only how scientists generate and test their own hypotheses, but also how they evaluate the scientific evidence presented by others. We discuss guardrails against confirmation bias that are already in place, and others that could potentially improve scientific practice if adopted. 
 
Shownotes
Wason, P. C. (1960). On the failure to eliminate hypotheses in a conceptual task. Quarterly Journal of Experimental Psychology, 12(3), 129-140.
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220.
Mellers, B., Hertwig, R., & Kahneman, D. (2001). Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science, 12(4), 269-275.
Coles, N. A., March, D. S., Marmolejo-Ramos, F., Larsen, J. T., Arinze, N. C., Ndukaihe, I. L., ... & Liuzza, M. T. (2022). A multi-lab test of the facial feedback hypothesis by the many smiles collaboration. Nature Human Behaviour, 1-12.
Dutilh, G., Sarafoglou, A., & Wagenmakers, E. J. (2021). Flexible yet fair: Blinding analyses in experimental psychology. Synthese, 198(23), 5745-5772.
Sarafoglou, A., Hoogeveen, S., & Wagenmakers, E. J. (2023). Comparing analysis blinding with preregistration in the many-analysts religion project. Advances in Methods and Practices in Psychological Science, 6(1), 25152459221128319. 
Faster-than-light neutrino anomaly
 

Episode 2: Scepticismus

In our second episode, we discuss the role of skepticism in science, a topic that relates closely to the title of our podcast. Given that the scientific enterprise is essentially an exercise in organized skepticism, how can we maintain a healthy amount of skepticism while also ensuring that scientists don't slip into cynicism or nihilism? 
 
Shownotes
Opening quote by Imre Lakatos from Science and Pseudoscience. Hear it from the man himself. 
Ego depletion
Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124
Wacholder, S., Chanock, S., Garcia-Closas, M., El Ghormli, L., & Rothman, N. (2004). Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. Journal of the National Cancer Institute, 96(6), 434-442. DOI: 10.1093/jnci/djh075
Quote by Debra Mayo. Original reference: Mayo, D. G. (2018). Statistical inference as severe testing: How to get beyond the statistics wars. Cambridge University Press.

Episode 1: Motivus

In our first episode, we discuss a quote from the preface to The Instauratio Magna (of which Novum Organum is a part), in which Bacon claims that scientists should be motivated to do science for the betterment of mankind, and not for personal motives like fame, fortune, or even fun. 
Here is the tweet (by Heidi Seibold) on academia not being aligned with good scientific practices.
 
An unedited transcript of the episode can be found here.