Nullius in Verba

In this episode, we discuss review boards for research with human subjects. Are they necessary? Are they efficient? Are scientists well equipped to make judgements about ethics? And are economists more ethical than psychologists? 
 
Shownotes
Whitney, S. N. (2015). Balanced ethics review: A guide for institutional review board members. Springer.
Schrag, Z. M. (2010). Ethical imperialism: Institutional review boards and the social sciences, 1965–2009. JHU Press.
Kinsey ReportsMasters & Johnson
How Institutional Review Boards can be (and are) Weaponized Against Academic Freedom
Weaponizing the IRB 2.0
Psychologists’ Involvement in Torture and the APA. Psychology Today.
 

In this episode, we discuss activism in science. How do political and personal values affect science? When is activism just part of the job? And should one be careful about activism in the classroom? Enjoy.  
 
Shownotes: 
Frisby, C. L., Redding, R. E., & O’Donohue, W. T. (2023). Ideological and Political Bias in Psychology: An Introduction. In Ideological and Political Bias in Psychology: Nature, Scope, and Solutions (pp. 1-14). Cham: Springer International Publishing.
McCaughey, M. (2023). The Trouble with Scholar-Activists. AAUP. 
McCaughey, M. (2024). Against Scholar Activists. The James G. Martin Center for Academic Renewal. 
Honeycutt, N., & Jussim, L. (2023). Political bias in the social sciences: A critical, theoretical, and empirical review. Ideological and Political Bias in Psychology: Nature, Scope, and Solutions, 97-146.
Sargent, R. M. (2012). From Bacon to Banks: The vision and the realities of pursuing science for the common good. Studies in History and Philosophy of Science Part A, 43(1), 82-90.
Weber, M. (1946). Science as a Vocation. In Science and the Quest for Reality (pp. 382-394). London: Palgrave Macmillan UK.
 

In this episode, we continue our discussion of replications. We talk about how to analyze replication studies, which studies are worth replicating, and what is the status of replications in other scientific disciplines. 
 
Shownotes
Mack, R. W. (1951). The Need for Replication Research in Sociology. American Sociological Review, 16(1), 93–94. https://doi.org/10.2307/2087978
Smith, N. C. (1970). Replication studies: A neglected aspect of psychological research. American Psychologist, 25(10), 970–975. https://doi.org/10.1037/h0029774
Sidman, M. (1960). Tactics of Scientific Research: Evaluating Experimental Data in Psychology (New edition). Cambridge Center for Behavioral.
Ebersole, C. R., Mathur, M. B., Baranski, E., Bart-Plange, D.-J., Buttrick, N. R., Chartier, C. R., Corker, K. S., Corley, M., Hartshorne, J. K., IJzerman, H., Lazarević, L. B., Rabagliati, H., Ropovik, I., Aczel, B., Aeschbach, L. F., Andrighetto, L., Arnal, J. D., Arrow, H., Babincak, P., … Nosek, B. A. (2020). Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245920958687
Isager, P. M., van Aert, R. C. M., Bahník, Š., Brandt, M. J., DeSoto, K. A., Giner-Sorolla, R., Krueger, J. I., Perugini, M., Ropovik, I., van ’t Veer, A. E., Vranka, M., & Lakens, D. (2023). Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints. Psychological Methods, 28(2), 438–451. https://doi.org/10.1037/met0000438
Aldhous, P. (2011). Journal rejects studies contradicting precognition. New Scientist. https://www.newscientist.com/article/dn20447-journal-rejects-studies-contradicting-precognition/
Stanley, D. J., & Spence, J. R. (2014). Expectations for Replications: Are Yours Realistic? Perspectives on Psychological Science, 9(3), 305–318. https://doi.org/10.1177/1745691614528518
Simonsohn, U. (2015). Small telescopes: Detectability and the evaluation of replication results. Psychological Science, 26(5), 559–569. https://doi.org/10.1177/0956797614567341
Nosek, B.A., Errington, T.M. (2017) Reproducibility in Cancer Biology: Making sense of replications. eLife 6:e23383. https://doi.org/10.7554/eLife.23383
 
 
 

In the next two episodes, we will discuss replication studies, which are essential to building reliable scientific knowledge.
 
Shownotes
Mack, R. W. (1951). The Need for Replication Research in Sociology. American Sociological Review, 16(1), 93–94. https://doi.org/10.2307/2087978
Smith, N. C. (1970). Replication studies: A neglected aspect of psychological research. American Psychologist, 25(10), 970–975. https://doi.org/10.1037/h0029774
Sidman, M. (1960). Tactics of Scientific Research: Evaluating Experimental Data in Psychology (New edition). Cambridge Center for Behavioral.
Ebersole, C. R., Mathur, M. B., Baranski, E., Bart-Plange, D.-J., Buttrick, N. R., Chartier, C. R., Corker, K. S., Corley, M., Hartshorne, J. K., IJzerman, H., Lazarević, L. B., Rabagliati, H., Ropovik, I., Aczel, B., Aeschbach, L. F., Andrighetto, L., Arnal, J. D., Arrow, H., Babincak, P., … Nosek, B. A. (2020). Many Labs 5: Testing Pre-Data-Collection Peer Review as an Intervention to Increase Replicability. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245920958687
Isager, P. M., van Aert, R. C. M., Bahník, Š., Brandt, M. J., DeSoto, K. A., Giner-Sorolla, R., Krueger, J. I., Perugini, M., Ropovik, I., van ’t Veer, A. E., Vranka, M., & Lakens, D. (2023). Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints. Psychological Methods, 28(2), 438–451. https://doi.org/10.1037/met0000438
Aldhous, P. (2011). Journal rejects studies contradicting precognition. New Scientist. https://www.newscientist.com/article/dn20447-journal-rejects-studies-contradicting-precognition/
Stanley, D. J., & Spence, J. R. (2014). Expectations for Replications: Are Yours Realistic? Perspectives on Psychological Science, 9(3), 305–318. https://doi.org/10.1177/1745691614528518
Simonsohn, U. (2015). Small telescopes: Detectability and the evaluation of replication results. Psychological Science, 26(5), 559–569. https://doi.org/10.1177/0956797614567341
 

Smith, N. C. (1970). Replication studies: A neglected aspect of psychological research. American Psychologist, 25(10), 970–975. https://doi.org/10.1037/h0029774

In this episode, we discuss a fun mix of eponymous laws, which are laws named after individuals who postulate them. 
 
Shownotes
Campbell, D. T. (1979). Assessing the impact of planned social change. Evaluation and Program Planning, 2(1), 67–90. https://doi.org/10.1016/0149-7189(79)90048-X
Merton, R. K. (1995). The Thomas Theorem and the Matthews Effect. Social Forces, 74(2), 379–422.
Stigler, S. M. (1980). Stigler’s Law of Eponymy*. Transactions of the New York Academy of Sciences, 39(1 Series II), 147–157. https://doi.org/10.1111/j.2164-0947.1980.tb02775.x
Clarke, A. C. (Arthur C. (1962). Profiles of the future: An inquiry into the limits of the possible. New York : Bantam Books. http://archive.org/details/profilesoffuture00clar
Brandolini’s Law: Based on a tweet, after reading Kahneman Thinking fast and slow: https://twitter.com/ziobrando/status/289635060758507521
Preston, I. L. (1980). Researchers at the Federal Trade Commission—Peril and Promise. Current Issues and Research in Advertising, 3(1), 1–15.
Twyman’s Law: “The more unusual or interesting the data, the more likely they are to have been the result of an error of one kind or another.” Earliest scholarly reference is in Journal of the Royal Statistical Society, Series A, Vol 138, No 4, 1975. The Teaching of Statistics by A. S. C. Ehrenberg.
Bloch, A. (1990). Murphy's law complete: All the reasons why everything goes wrong. Arrow Books Limited. 
 

In this final episode of the three-part series on the Philosophical Psychology lectures by Paul Meehl, we discuss lectures 6-8, which cover the ten obfuscating factors in "soft areas" of psychology and a host of advice Meehl provides for researchers, reviewers, editors, and educators on how to improve practice. 
 
Shownotes
Krefeld-Schwalb, A., Sugerman, E. R., & Johnson, E. J. (2024). Exposing omitted moderators: Explaining why effect sizes differ in the social sciences. Proceedings of the National Academy of Sciences, 121(12), e2306281121. https://doi.org/10.1073/pnas.2306281121
Lakens, D., & Etz, A. J. (2017). Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True. Social Psychological and Personality Science, 8(8), 875–881. https://doi.org/10.1177/1948550617693058
 

In this episode, we continue the discussion of Meehl's Philosophy of Psychology course, focusing on lectures 3, 4, and 5. 
 
Shownotes
The quote "Don't make a mockery of honest ad-hockery" is probably from Clark Glymour: https://en.wikipedia.org/wiki/Clark_Glymour
Good, I. J. (1965). The Estimation of Probabilities: An Essay on Modern Bayesian Methods. M.I.T. Press.
Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science, 237(4820), 1317–1323.
 

Faust, D., & Meehl, P. E. (1992). Using scientific methods to resolve questions in the history and philosophy of science: Some illustrations. Behavior Therapy, 23(2), 195–211. https://doi.org/10.1016/S0005-7894(05)80381-8

Video lectures: https://meehl.umn.edu/video 
Faust, D., & Meehl, P. E. (1992). Using scientific methods to resolve questions in the history and philosophy of science: Some illustrations. Behavior Therapy, 23(2), 195–211. https://doi.org/10.1016/S0005-7894(05)80381-8
Serlin, R. C., & Lapsley, D. K. (1985). Rationality in psychological research: The good-enough principle. American Psychologist, 40(1), 73–83. https://doi.org/10.1037/0003-066X.40.1.73
Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1(2), 108–141. https://doi.org/10.1207/s15327965pli0102_1
Meehl, P. E. (1992). Cliometric metatheory: The actuarial approach to empirical, history-based philosophy of science. Psychological Reports, 71, 339–467.