Debunking decreases mistaken causal inferences: A direct replication of Irving et al. (2022)

Social Impact and Responsibility

Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 investigated how debunking statistical misinformation can reduce mistaken causal inferences. Empirical findings may be presented in public domains using non-scientific language or inadequate statistical reporting which can lead to mistaken inferences. A common risk with such misinformation is mistaking a correlational finding for a causal one. Such mistaken inferences may not only influence public opinion, but also domains such as policy decision making and medical research. This may be more salient for populations where the original scientific source is inaccessible. Our success in replicating the results of Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 suggests that the debunking effect is robust and the inclusion of a correction statement can reduce the risk of mistaken causal inferences. However, our study only compared debunking to a non-debunking condition, so we cannot make any conclusions about the effectiveness compared to alternative techniques.

Introduction

Misinformation affects the whole spectrum of society from everyday conversations, to the media, to publishing scientific research. Misinformation may be described as any false or incorrect information that is shared without the awareness that it is false (Ecker et al., 2022Ecker, U. K. H., Lewandowsky, S., Cook, J., Schmid, P., Fazio, L. K., Brashier, N., Kendeou, P., Vraga, E. K., & Amazeen, M. A. (2022). The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1), 13–29. https://doi.org/10.1038/s44159-021-00006-y; Lewandowsky et al., 2024Lewandowsky, S., Ecker, U. K. H., Cook, J., van der Linden, S., Roozenbeek, J., Oreskes, N., & McIntyre, L. C. (2024). Liars know they are lying: Differentiating disinformation from disagreement. Humanities and Social Sciences Communications, 11(1), Article 986. https://doi.org/10.1057/s41599-024-03503-6). In contrast, disinformation is where someone intentionally or willfully shares information they know to be false (Hameleers, 2023Hameleers, M. (2023). Disinformation as a context-bound phenomenon: Toward a conceptual clarification integrating actors, intentions and techniques of creation and dissemination. Communication Theory, 33(1), 1–10. https://doi.org/10.1093/ct/qtac021; Lewandowsky et al., 2024Lewandowsky, S., Ecker, U. K. H., Cook, J., van der Linden, S., Roozenbeek, J., Oreskes, N., & McIntyre, L. C. (2024). Liars know they are lying: Differentiating disinformation from disagreement. Humanities and Social Sciences Communications, 11(1), Article 986. https://doi.org/10.1057/s41599-024-03503-6). Regardless of whether it gets debunked or not, information is classified as misinformation or disinformation based on the intent behind sharing it. Though unintentional, misinformation may lead to misinformed policy decisions and downstream changes in individual behavior and beliefs. For instance, Loomba et al. (2021)Loomba, S., de Figueiredo, A., Piatek, S. J., de Graaf, K., & Larson, H. J. (2021). Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nature Human Behaviour, 5(3), 337–348. https://doi.org/10.1038/s41562-021-01056-1 found that exposure to misinformation led to lower vaccination intent among residents across the USA and the UK during the COVID-19 global health crisis.

Debunking can be an effective strategy to counter misinformation where corrective messages are included to caution viewers (Chan et al., 2017Chan, M.-P. S., Jones, C. R., Hall Jamieson, K., & Albarracín, D. (2017). Debunking: A meta-analysis of the psychological efficacy of messages countering misinformation. Psychological Science, 28(11), 1531–1546. https://doi.org/10.1177/0956797617714579). While inherent attitudes and beliefs may be resilient (Jang et al., 2019Jang, J., Lee, E.-J., & Shin, S. Y. (2019). What debunking of misinformation does and doesn’t. Cyberpsychology, Behavior, and Social Networking, 22(6), 423–427. https://doi.org/10.1089/cyber.2018.0608; Schmid, 2026Schmid, P. (2026). Debunking health misinformation with empathy. Current Opinion in Psychology, 67, Article 102213. https://doi.org/10.1016/j.copsyc.2025.102213), debunking can reduce mistaken inferences (Ecker et al., 2022Ecker, U. K. H., Lewandowsky, S., Cook, J., Schmid, P., Fazio, L. K., Brashier, N., Kendeou, P., Vraga, E. K., & Amazeen, M. A. (2022). The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1), 13–29. https://doi.org/10.1038/s44159-021-00006-y; Fasce et al., 2026Fasce, A., Rosales-Trabuco, J., Barberia, I., Pinedo-Yzaguirre, E., Espinoza Nuñez, J. M., Ortiz-Treviños, W. M., Pizarro-Moncada, C. D., Reyes-Bossio, M., Carbajal-León, C., & Rodríguez-Ferreiro, J. (2026). Effects of debunking interventions on endorsement of alternative medicine: A randomized controlled experiment in Peru. Scientific Reports, 16(1), 4995. https://doi.org/10.1038/s41598-026-38260-w). In the current study, we focus on misinformation around media articles reporting on scientific research reports and whether that information can be debunked. We present a direct replication of Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 who found that debunking statistical misinformation through a caveat statement has a large effect on reducing the number of mistaken causal inferences. Given the importance of reducing the impact of misinformation, it is important to independently replicate and verify the robustness of their results to ensure a trustworthy foundation for future research.

Debunking statistical misinformation in the media is particularly important as lay readers may not have the quantitative skills to debunk information on their own. Journalists are an easy target to blame for introducing misinformation, but Sumner et al. (2014)Sumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Venetis, C. A., Davies, A., Ogden, J., Whelan, L., Hughes, B., Dalton, B., Boy, F., & Chambers, C. D. (2014). The association between exaggeration in health related science news and academic press releases: Retrospective observational study. BMJ, 349, g7015. https://doi.org/10.1136/bmj.g7015 found 33% of university press releases exaggerated causal claims in research and 81% of articles using those press releases contained a similar exaggeration. In a follow-up, Adams et al. (2019)Adams, R. C., Challenger, A., Bratton, L., Boivin, J., Bott, L., Powell, G., Williams, A., Chambers, C. D., & Sumner, P. (2019). Claims of causality in health news: A randomised trial. BMC Medicine, 17(1), Article 91. https://doi.org/10.1186/s12916-019-1324-7 manipulated press releases to include more cautious headlines and explicit caveat statements around concluding causality, which led to headlines that were better aligned with the evidence. Conversely, Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 focused on the readers of news articles to see whether debunking statements affected what participants would understand about the study. Participants completed a continuous influence paradigm where they read a fictitious news story one sentence at a time. The story reported a new study showing extended TV screen time causes cognitive decline in older adults. Irving et al. manipulated one key sentence where participants were randomly allocated to read one of two versions. In the correction group, the story included a quote from the study’s author warning they only demonstrated a correlational, not causal link. In the no-correction group, the story stated they could not reach the author for comment. Irving et al. found strong evidence that correcting statistical misinformation decreased the number of mistaken causal inferences (δ = 0.64 as a standardised mean difference) and increased the number of correlational inferences (δ = 1.27) compared to the no-correction group. Given the importance of developing impactful interventions to combat statistical misinformation, it is important to check the consistency of this finding.

Choice of Study for Replication

The ability to reproduce and replicate a study is critical to science, where reproducibility refers to taking the same data and applying the same analyses as an original study, and investigating whether someone can recreate the results (National Academies of Sciences, Engineering, and Medicine [NASEM], 2019). On the other hand, replicability involves collecting new data to address the same research question with the same research design as an original study (Nosek & Errington, 2017Nosek, B. A., & Errington, T. M. (2017). Making sense of replications. eLife, 6, e23383. https://doi.org/10.7554/eLife.23383). While reproducibility is a minimum standard for scientific research, interpreting replicability is more nuanced. Successfully reproducing a study can increase confidence in the finding and establish a more robust basis for future research (Zwaan et al., 2018Zwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2018). Making replication mainstream. Behavioral and Brain Sciences, 41, Article e120. https://doi.org/10.1017/S0140525X17001972). But if a study does not replicate, there are helpful and non-helpful sources of replication failure (NASEM, 2019). Helpful sources include previously unknown contextual moderators while unhelpful sources range from mistakes to fraud. Despite the importance of replication in science, they are still rarely published, representing an estimated 1% or less of articles (Clarke et al., 2026Clarke, B., Lee, P. Y., Schiavone, S. R., Rhemtulla, M., & Vazire, S. (2026). The prevalence of direct replication articles in top-ranking psychology journals. American Psychologist, 81(2), 238–253. https://doi.org/10.1037/amp0001385; Makel et al., 2012Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in psychology research: How often do they really occur? Perspectives on Psychological Science, 7(6), 537–542. https://doi.org/10.1177/1745691612460688).

It is not feasible to replicate every published study, meaning there must be features or characteristics that influence why a given study is chosen to be the target of a direct replication. Field et al. (2019)Field, S. M., Hoekstra, R., Bringmann, L., & van Ravenzwaaij, D. (2019). When and why to replicate: As easy as 1, 2, 3? Collabra: Psychology, 5(1), Article 46. https://doi.org/10.1525/collabra.218 highlight three facets to select a study for replication when resources are scarce: statistical, theoretical, and methodological. These include whether the study’s results are based on weak statistical evidence, whether the study addressed a theoretically important topic or currently relevant research question, and elements of the study’s method like the sample size. Isager et al. (2023)Isager, P. M., van Aert, R. C. M., Bahník, Š., Brandt, M. J., DeSoto, K. A., Giner-Sorolla, R., Krueger, J. I., Perugini, M., Ropovik, I., van ’t Veer, A. E., Vranka, M., & Lakens, D. (2023). Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints. Psychological Methods, 28(2), 438–451. https://doi.org/10.1037/met0000438 proposed a formal decision model to describe this process and calculate replication value as a product of value and uncertainty of the original claim. Value relates to the scientific or societal impact of the findings and uncertainty relates to quantitative indicators like the sample size or statistical evidence. Alister et al. (2021)Alister, M., Vickers-Jones, R., Sewell, D. K., & Ballard, T. (2021). How do we choose our giants? Perceptions of replicability in psychological science. Advances in Methods and Practices in Psychological Science, 4(2), Article 25152459211018199. https://doi.org/10.1177/25152459211018199 surveyed researchers on which study attributes would affect their confidence in whether a study would replicate. High statistical power, open materials, and study pre-registration were among the attributes that most increased researchers’ confidence in replication, while low statistical power, the methods and data were not being available, and weak statistical evidence like a p-value close to .05 decreased researchers’ confidence in replication.

Using these three perspectives of replication value, we selected Irving et al. as our replication target to check the reproducibility and replicability of their finding that correcting misinformation reduced mistaken causal inferences. Through the frame of Field et al. (2019)Field, S. M., Hoekstra, R., Bringmann, L., & van Ravenzwaaij, D. (2019). When and why to replicate: As easy as 1, 2, 3? Collabra: Psychology, 5(1), Article 46. https://doi.org/10.1525/collabra.218, theoretically we consider correcting statistical misinformation to be a contemporary research question worth addressing; statistically, Irving et al. found very strong evidence for their effect; and methodologically, they justified their sample size through a power analysis, pre-registered their study, and shared their materials and data. By replication value (Isager et al., 2023Isager, P. M., van Aert, R. C. M., Bahník, Š., Brandt, M. J., DeSoto, K. A., Giner-Sorolla, R., Krueger, J. I., Perugini, M., Ropovik, I., van ’t Veer, A. E., Vranka, M., & Lakens, D. (2023). Deciding what to replicate: A decision model for replication study selection under resource and knowledge constraints. Psychological Methods, 28(2), 438–451. https://doi.org/10.1037/met0000438) we consider Irving et al. to have high value given the societal importance of reducing the impact of misinformation, while having low uncertainty due to their robust findings. Finally, in accordance with the findings of Alister et al. (2021)Alister, M., Vickers-Jones, R., Sewell, D. K., & Ballard, T. (2021). How do we choose our giants? Perceptions of replicability in psychological science. Advances in Methods and Practices in Psychological Science, 4(2), Article 25152459211018199. https://doi.org/10.1177/25152459211018199, high statistical power, open materials, and study pre-registration signal high confidence in replication.

Research Question and Hypotheses

Our research question is: Do the findings of Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 replicate in a study independent from the original research team? There were two primary hypotheses which are the focus of our direct replication:

H1. Participants in the correction group would make fewer causal inferences than those in the no-correction group, in response to the coded inference questions.

H2. Participants in the correction group would make more correlational inferences than those in the no-correction group, in response to the coded inference questions.

The original study found strong evidence in favour of both primary hypotheses, and for the purposes of our direct replication, we pose the same hypotheses.

Method

Transparency and Openness

All materials, data, and analysis scripts are publicly available on our Open Science Framework (OSF) project: https://osf.io/jpxbu. We included a .qsf file to import our study materials into Qualtrics and a PDF version. We used the statistics software R (R Core Team, R version 4.6.1 [2026-06-24 ucrt]) for all analyses. This manuscript is available as a reproducible R Markdown document, and we provide a full report outlining our reproducibility checks of Irving et al. and outlining all our data processing and modelling steps. This study was not pre-registered as we collected the data as part of an MSc dissertation project and we missed the opportunity due to the short turnaround of projects. The lack of pre-registration is a limitation of our study as we cannot verify our sampling plan, but our reproducibility checks demonstrated our analyses precisely followed the original study. We report how we determined our sample size, all data exclusions (if any), all manipulations, and all measures in the study.

Sample Size Justification

In the original study by Irving et al., they reported an a priori power analysis to detect a smallest effect size of interest of d = 0.54 based on the meta-analysis by Chan et al. (2017)Chan, M.-P. S., Jones, C. R., Hall Jamieson, K., & Albarracín, D. (2017). Debunking: A meta-analysis of the psychological efficacy of messages countering misinformation. Psychological Science, 28(11), 1531–1546. https://doi.org/10.1177/0956797617714579. They aimed to collect 75 participants per group (N = 150, alpha = 5%, power = 95%), but their final sample size was 129, making their study sensitive to effects of d = 0.58 with 95% power. The smallest effect size reported by Irving et al. was δ = 0.64 as roughly equivalent to Cohen’s d, and if we use that in an a priori power analysis, we would need 54 participants per group (N = 108) for 95% power. As a replication study, we did not want to aim for fewer participants than the original study, so we recruited the same sample size of 129 to be sensitive to effects of d = 0.58 with 95% power.

Participants

With our target sample size of 129, we recruited participants online through Prolific between the 8th and 9th of June 2023. We paid participants £2.25 (prorated £9 per hour) for an estimated study time of 15 minutes. Our only sampling restriction was participants should be in the United Kingdom or the United States of America.

Given the online recruitment, we included a data quality question: “We want to ensure that all data we analyse is meaningful and accurately reflects engagement with the study content. In your honest opinion, should we use your data in our analyses of this study?” We removed one participant who answered “No, do not use my data” and recruited one additional participant to ensure the final sample size was 129 and consistent with our sampling plan. There were 65 participants in the correction group and 64 in the no-correction group.

Table 1 shows the demographic breakdown of our sample compared to the original study by Irving et al. Participants in our study were approximately five years younger on average and had more male participants, but the education profile and English as a first language was similar between the two studies.

Materials and Procedure

We created the study using Qualtrics based on the original study’s open materials. The open materials allowed us to precisely replicate the study where we used the same order and wording of instructions, materials, and measures. The only differences were using our own institution ethics forms and including two additional data quality items after all of the original study materials and measures but prior to the debrief. The only details not clearly documented by Irving et al. were the location of page breaks in the materials and measures. We had no contact with any of the original authors during the design of our study. The study received ethical approval from the University of Glasgow Medical, Veterinary, and Life Sciences ethics review committee (Ref: 200220281).

After being redirected from Prolific, participants first read an information sheet explaining the purpose of the study. Participants then provided informed consent and completed demographic questions on age, country of origin, gender, highest education level, and whether English was their first language. Participants completed the 21-item Trust in Science and Scientists scale (Nadelson et al., 2014Nadelson, L., Jorcyk, C., Yang, D., Jarratt Smith, M., Matson, S., Cornell, K., & Husting, V. (2014). I just don’t trust them: The development and validation of an assessment instrument to measure trust in science and scientists. School Science and Mathematics, 114(2), 76–86. https://doi.org/10.1111/ssm.12051). Participants responded on a 1-5 Likert scale ranging from strongly disagree to strongly agree. After reverse coding 12 items, we calculated the mean of the 21 items to create an overall scale score, where higher values indicate higher trust in science and scientists. We included a 22-item spirituality scale (Delaney, 2005Delaney, C. (2005). The Spirituality Scale: Development and psychometric testing of a holistic instrument to assess the human spiritual dimension. Journal of Holistic Nursing, 23(2), 145–167. https://doi.org/10.1177/0898010105276180) to create doubt in the participants’ perception of the study’s aim. Like Irving et al., we did not process or analyse the participants’ responses to this scale.

The critical materials for the study featured a continuous influence paradigm where participants read 10 short passages. Participants read each passage individually and could advance - but not go back - by clicking an arrow to show the next passage. The passages described a fictitious news article of a new study reporting the causal link between the number of hours someone watches TV and increased cognitive decline in adults over 65. The passages included fictitious quotes from a health and lifestyle expert, a member of parliament, and a spokesperson from an online streaming service.

The independent variable controls the ninth passage, where participants were randomly allocated to read one of two variations. Participants in the correction group read a passage which corrects the mistaken causal inference: “The lead author of the study, Professor Alastair McKay from Durham University, has since specified that the results have been misrepresented by Professor James Evans and that TV screen time has not yet been found to cause cognitive decline. He informs us that only a correlation has been found between the two and no causal link is stated in the study.” In contrast, participants in the no-correction group read a passage which does not correct the mistaken causal inference: “The lead author of the study, Professor Alastair McKay from Durham University could not be reached for comment.”

After reading the 10 passages, we provided participants with 10 free-text responses asking them what they understood about the study. We manually coded 5 of the 10 passages for mention of a causal or correlational relationship (we provide further details of the coding process below). Participants then completed 10 multiple-choice questions on facts about the passages, such as how old the participants in the news story were.

Participants completed a manipulation check question on whether they remember the results of the study being corrected or not, and a statistical understanding check for whether they can explain the difference between correlation and causation. Unlike the original study, we included a self-report response check for whether we should have included their data in the analysis and the opportunity to leave a free-text response on their experience completing the study. Finally, participants read a debrief sheet fully explaining the aims of the study and the key manipulation.

Coding

As participants provided free-text responses, we had to manually code the responses for whether participants made a causal inference or a correlational inference. Author 3 (JB) prepared the responses by isolating the participant ID and inference questions from all other variables. This meant there was a level of blinding as the condition was not present while coding the data. Author 2 (KZ) then independently coded all responses by following the coding scheme Irving et al. shared with their pre-registration:

1. If a participant makes a correlational inference, they will receive a score of 0 on DV1 (causal inference) and a score of 1 on DV2 (correlational inference).
2. If a participant makes a causal inference, they will receive a score of 1 on DV1 (causal inference) and score 0 on DV2 (correlational inference).
3. If a participant gives an off-topic answer, they will receive a score of 0 on both DVs.

Like Irving et al., we coded responses using a harsh and lenient coding scheme. Under the harsh scheme, any ambiguous responses which did not directly comment on a causal or correlational relationship received 0. Under the lenient scheme, ambiguous responses for questions three and four received 0 for causal and 1 for correlational if the participant answered “No” or “None”.

After Author 2 coded all responses, Author 3 randomly sampled the coding from 10 participants (50 individual codes). There was 94% agreement (47/50 codes), or a moderate Cohen's Kappa value of .64, where Author 3 provided the same code as Author 2, before any disagreements were resolved through discussion to provide a final spreadsheet of codes. For each participant, this meant there were four columns of data: five codes for causal inferences under the harsh and lenient scheme, and five codes for correlational inferences under the harsh and lenient scheme. Unless noted, our analyses focus on the harsh coding scheme to be conservative.

Replication Closeness Evaluation

Table 2 outlines replication classification criteria (LeBel et al., 2018LeBel, E. P., McCarthy, R. J., Earp, B. D., Elson, M., & Vanpaemel, W. (2018). A unified framework to quantify the credibility of scientific findings. Advances in Methods and Practices in Psychological Science, 1(3), 389–402. https://doi.org/10.1177/2515245918787489) and we consider all design facets to be the same as the original study apart from two. We recruited people using Prolific, whereas the original was a convenience and snowball sample using social media. We also paid participants, whereas the original did not. Given the similarity in demographic variables (Table 1), we consider our study a very close direct replication under the replication taxonomy (LeBel et al., 2018LeBel, E. P., McCarthy, R. J., Earp, B. D., Elson, M., & Vanpaemel, W. (2018). A unified framework to quantify the credibility of scientific findings. Advances in Methods and Practices in Psychological Science, 1(3), 389–402. https://doi.org/10.1177/2515245918787489).

Design and Data Analysis

Our study was a direct replication of Irving et al., meaning we collected the same variables and applied the same modelling techniques. To ensure we performed the analyses the same as the original study, we used the open data from Irving et al. to check computational reproducibility. We could reproduce their primary outcomes precisely, meaning we were confident we were using the same modelling techniques to ensure a direct replication.

In the online reproducibility and replication report, we provide all primary and secondary analyses that Irving et al. reported. For brevity, we focus on the primary hypotheses in this manuscript, in addition to our own exploratory analyses to check the robustness of the findings to alternative modelling techniques.

The primary analyses focus on one independent variable (Condition) where participants were randomly allocated to the correction group or no-correction group. There were two dependent variables for two ways of coding participants’ inferences:

1) sum of causal inferences and 2) sum of correlational inferences. We calculated the sum of causal/correlational inferences by adding up the coded responses, where the values could range from 0 to 5.

To address the primary hypotheses 1 and 2, we applied a Bayesian one-sided independent samples t-test with a medium (r = 0.707) half-Cauchy prior using the BayesFactor R package (Morey & Rouder, 2023Morey, R. D., & Rouder, J. N. (2023). BayesFactor: Computation of Bayes factors for common designs [Computer software]. https://CRAN.R-project.org/package=BayesFactor). The model and prior were the same as the original study. For hypothesis 1, the one-sided t-test expected only positive effects for the no-correction - correction comparison, meaning we expected participants in the correction group to produce fewer causal inferences compared to participants in the no-correction group. For hypothesis 2, the one-sided t-test expected only negative effects for the no-correction - correction comparison, meaning we expected participants in the correction group to produce more correlational inferences compared to participants in the no-correction group. For both of these analyses, the null hypothesis represents the opposite effect size space, such as negative effects for hypothesis 1, rather than a point-null of zero.

To quantify the effect size of these analyses and compare to those included by Irving et al., we report the mean difference, delta (δ), as the standardised mean difference (Rouder et al., 2009Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. https://doi.org/10.3758/PBR.16.2.225), and the probability of superiority as a common language effect size. For the mean difference and delta, we report the 95% Highest Density Interval (HDI) as the Bayesian version of an interval estimate provided by the BayesFactor R package (Morey & Rouder, 2023Morey, R. D., & Rouder, J. N. (2023). BayesFactor: Computation of Bayes factors for common designs [Computer software]. https://CRAN.R-project.org/package=BayesFactor). For the probability of superiority, we report the 95% confidence interval (CI) as the frequentist version of an interval estimate provided by the effectsize R package (Ben-Shachar et al., 2020Ben-Shachar, M. S., Lüdecke, D., & Makowski, D. (2020). effectsize: Estimation of effect size indices and standardized parameters. Journal of Open Source Software, 5(56), Article 2815. https://doi.org/10.21105/joss.02815).

Results

Direct Replication

Causal Inferences. On average, those in the no-correction group (M = 2.06, SD = 1.15) made more mistaken causal inferences than those in the correction group (M = 1.28, SD = 1.13). Figure 1 shows the mean sum of causal inferences in the correction group compared to the no-correction group in our replication study compared to the original findings by Irving et al.

The Bayesian one-sided t-test showed the data were 308.69 times more likely under the directional hypothesis than the null hypothesis. On average, participants in the correction group provided 0.74, 95% HDI = [0.34, 1.13] fewer causal inferences than participants in the no-correction group, supporting H1. Expressed as a standardised effect size, this translates to δ = 0.65, 95% HDI = [0.29, 1.00] and probability of superiority of 0.69, 95% CI = [0.59, 0.77]. This means there is a 69% chance that someone selected at random from the correction group would make fewer mistaken causal inferences than someone selected at random from the no-correction group.

Correlational Inferences. On average, those in the correction group (M = 0.75, SD = 1.02) made more correlational inferences than those in the no-correction group (M = 0.16, SD = 0.44). Figure 2 shows the mean sum of correlational inferences in the correction group compared to the no-correction group in our replication study compared to the original findings by Irving et al.

The Bayesian one-sided t-test showed the data were 1212.41 times more likely under the directional hypothesis than the null. On average, participants in the correction group provided 0.56, 95% HDI = [0.29, 0.84] more correlational inferences than participants in the no-correction group, supporting H2. Expressed as a standardised effect size, this translates to δ = 0.72, 95% HDI = [0.36, 1.08] and probability of superiority of 0.70, 95% CI = [0.61, 0.79]. This means there is a 70% chance that someone selected at random from the correction group would make more correlational inferences than someone selected at random from the no-correction group.

Replication Evaluation. We used the statistical interpretation criteria from LeBel et al. (2019)LeBel, E. P., Vanpaemel, W., Cheung, I., & Campbell, L. (2019). A brief guide to evaluate replications. Meta-Psychology, 3. https://doi.org/10.15626/MP.2018.843 to judge to what extent we replicated the findings from Irving et al. There are three criteria: (a) whether we detected a signal, (b) the consistency of the effect size, and (c) the precision of the effect size estimate compared to the original study.

For the sum of causal inferences, we consider the replication outcome to be signal detected, consistent with the original study. For the purposes of comparison, we focus on delta as a standardised effect size. Both our study (BF10 = 308.69) and Irving et al. (BF10 = 279.35) detected a signal, where the Bayesian one-sided t-tests found strong evidence in favour of the alternative hypothesis compared to the null. The estimate for delta was almost identical between both studies (δ = 0.65, 95% HDI = [0.29, 1.00] compared to δ = 0.64, 95% HDI = [0.28, 0.99] in Irving et al., page 5). The 95% HDIs were also similar between both studies, where the original effect size was within the replication 95% HDI, and the replication effect size was within the original 95% HDI. Figure 3 demonstrates this visually by comparing delta and its 95% HDI side by side, where the original and replication estimates overlap almost entirely.

For the sum of correlational inferences, the replication outcome is signal detected, inconsistent with the original study with a smaller effect size. Both our study (BF10 = 1212.41) and Irving et al. (BF10 = 1.36 × 10⁹) detected a signal, where the Bayesian one-sided t-tests found strong evidence in favour of the alternative hypothesis compared to the null. The estimate for delta was smaller in our replication (δ = 0.72, 95% HDI = [0.36, 1.08]) compared to Irving et al., page 5 (δ = 1.27, 95% HDI = [0.90, 1.66]), but in the same direction where those in the correction group reported more correlational inferences than those in the no-correction group. The 95% HDI in both studies had a similar range but the estimate in our replication shifted to a smaller magnitude closer to the sum of causal inferences estimates (see Figure 3).

Exploratory Analyses: Logistic Regression

Given the outcome distributions in Figures 1 and 2 that Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 did not visualise, independent samples t-tests may not be the appropriate choice of analysis. The sum of causal and correlational inferences are on a bounded 0-5 scale, whereas the t-test assumes the outcome is normally distributed and theoretically boundless. Therefore, the analysis may not capture important features of the data generating process, particularly for the sum of correlational inferences where responses were close to floor levels. We checked the robustness of the conclusions by fitting a mixed effects logistic regression model using brms (Bürkner, 2017Bürkner, P.-C. (2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01). The online reproducibility and replication report provides full details of the modelling approach and setting informed priors consistent with the WAMBS checklist for Bayesian modelling (Van de Schoot et al., 2021van de Schoot, R., Depaoli, S., King, R., Kramer, B., Märtens, K., Tadesse, M. G., Vannucci, M., Gelman, A., Veen, D., Willemsen, J., & Yau, C. (2021). Bayesian statistics and modelling. Nature Reviews Methods Primers, 1(1), Article 1. https://doi.org/10.1038/s43586-020-00001-2), but we provide an overview below of our final models.

Instead of the sum of causal/correlation inferences, we modelled the outcome as binary using a Bernoulli distribution and logit link. For each inference question, the outcome was 1 if the participant made a causal/correlational inference and 0 if the participant did not make a causal/correlational inference. We included a categorical fixed effect for Condition to compare the correction and no-correction groups. We then included random slopes and intercepts for participant and question ID to take variation within participants and questions into account.

Causal Inferences. We used the open data from Irving et al. to fit our planned model with default priors and used the model estimates to set informed priors for our replication data. Note the following values are expressed on a log scale due to the logit link. For the intercept and slope priors, we used a normal distribution with a mean of -1.00 (SD = 0.65) for the intercept and a mean of -1.20 (SD = 0.40) for Condition slope. For the residual standard deviation prior, we used an exponential distribution with a rate parameter of 1. We fitted the model using four chains, each containing 3,000 post-warmup samples (12,000 total) which passed model fitting and convergence checks.

The no-correction group was set as the reference group and the mean estimate for the intercept was -0.51, 95% HDI = [-1.51, 0.50]. The correction group was set as the target group where the slope for Condition as a fixed effect was -1.16, 95% HDI = [-1.77, -0.55], suggesting lower log odds of making a mistaken causal inference in the correction group. To aid interpretation, we can convert the slope estimate from log odds to an odds ratio, where being in the correction group decreased the odds of making a mistaken causal inference 3.2, 95% HDI = [1.74, 5.85] times compared to the no-correction group. Expressed another way, the predicted probability of making a mistaken causal inference was lower in the correction group (.16, 95% HDI = [.06, .35]) compared to the no-correction group (.38, 95% HDI = [.18, .62]). Figure 4 compares the model predicted probabilities in our replication data using informed priors compared to the same model applied to the data from Irving et al. using default priors.

This shows the results were robust to the modelling approach as the correction group had lower odds of making a mistaken causal inference compared to the no-correction group, consistent with the sum of causal inferences being lower in the correction group compared to the no-correction group.

Correlational Inferences. We fitted the same model to correlational inferences and the original data from Irving et al. to set informed priors. For the intercept and slope priors, we used a normal distribution with a mean of -3.40 (SD = 0.5) for the intercept and a mean of 1.9 (SD = 0.6) for Condition slope. With no-correction as the reference group, we expected higher log odds in the correction group as the target. For the residual standard deviation prior, we used an exponential distribution with a rate parameter of 1. The model with informed priors passed model fitting and convergence checks.

The mean estimate for the intercept was -4.46, 95% HDI = [-5.42, -3.55] and the slope for Condition as a fixed effect was 1.90, 95% HDI = [1.00, 2.80], suggesting higher log odds of making a correlational inference in the correction group compared to the no-correction group. Expressed as an odds ratio, being in the correction group increased the odds of making a correlational inference 6.71, 95% HDI = [2.72, 16.51] times compared to the no-correction group. The probability of making a correlational inference was higher in the correction group (.07, 95% HDI = [.03, .16]) compared to the no-correction group (.01, 95% HDI = [0, .03]). Figure 5 compares the model-predicted probabilities in our replication data, using informed priors, compared to the same model applied to the data from Irving et al., using default priors.

This shows the results were robust to the modelling approach, as the correction group had higher odds of making a correlational inference compared to the no-correction group, but we should be cautious in the interpretation. On the one hand, we can see clear advantages from using a logistic regression model compared to the analyses from Irving et al. There are few correlational inferences in the data with many zeros, so the estimates and lower bound of the HDI are close to zero predicted probability and respecting the bounded nature of the outcome. However, there is a lot of uncertainty in these estimates shown by the wide 95% HDI around the odds ratio and predicted probability estimates, potentially due to few correlational inferences in the data.

Discussion

In this direct replication of Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408, we found that debunking statistical misinformation led to fewer causal inferences among readers of a news article. Using the replication success criteria of LeBel et al. (2019)LeBel, E. P., Vanpaemel, W., Cheung, I., & Campbell, L. (2019). A brief guide to evaluate replications. Meta-Psychology, 3. https://doi.org/10.15626/MP.2018.843, we categorise causal inferences to be signal detected, consistent with the original study. The standardised effect size we observed (δ = 0.65, 95% HDI = [0.29, 1.00]) was almost identical to the estimate reported by Irving et al. (δ = 0.64, 95% HDI = [0.28, 0.99]). For the second outcome, we similarly found that debunking statistical information led to more correlational inferences, but we categorised the replication success criteria as signal detected, inconsistent with the original study with a smaller effect size. The standardised effect size we observed (δ = 0.72, 95% HDI = [0.36, 1.08]) was smaller than Irving et al. reported (δ = 1.27, 95% HDI = [0.90, 1.66]). We describe our findings as a successful direct replication where we confirmed both hypotheses H1 and H2, with the effect size for causal inferences being virtually identical and the effect size for correlational inferences being smaller.

The open research practices of Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 allowed us to assess all four dimensions of reproducible research (The Turing Way Community, 2025): reproducible, replicable, robust, and generalisable. First, the study was computationally reproducible as we obtained identical results for the primary analyses by using open data from Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408. Second, the study was replicable as we were able to produce similar results in the primary analyses using their analysis approach on our newly collected data. Third, the original findings were robust, when we applied an alternative analysis technique and made qualitatively similar conclusions. Finally, the findings were generalisable, as we also made qualitatively similar conclusions using the alternative analysis technique on our newly collected data. In sum, we consider the findings of Irving et al. to meet all the criteria for reproducible research.

Through this replication, we present evidence supporting the robustness and effectiveness of caveat statements as a debunking strategy for statistical misinformation. Misinformation is a growing phenomenon in today’s digital world where information is consumed rapidly and continuously through mediums such as social media (Xu et al., 2025Xu, G., Qian, M., & Meng, L. (2025). Misinformation dissemination on social media: Key research themes and evolutionary paths between 2013 and 2023. Humanities and Social Sciences Communications, 12(1), Article 1775. https://doi.org/10.1057/s41599-025-06067-1). This has further been exacerbated by the sharp increase in generative artificial intelligence content (Park & Nan, 2026Park, S., & Nan, X. (2026). Generative AI and misinformation: A scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation. AI & Society, 41(2), 1501–1515. https://doi.org/10.1007/s00146-025-02620-3). Our successful replication of Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 supports the effectiveness of a debunking caveat statement in reducing misinformation across different topics (Lorko et al., 2026Lorko, M., Čavojová, V., Šrol, J., Priesol, R., Jalakšová, P., & Tužilová, B. (2026). Timing matters: The effects of prebunking versus debunking on trust in disinformation. Personality and Social Psychology Bulletin. Advance online publication.. https://doi.org/10.1177/01461672251411571), consistent with the strategy of providing an alternative narrative in reducing the impact of misinformation (Lewandowsky et al., 2012Lewandowsky, S., Ecker, U. K. H., Seifert, C. M., Schwarz, N., & Cook, J. (2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13(3), 106–131. https://doi.org/10.1177/1529100612451018). Debunking strategies such as caveat statements represent an effective tool to reduce misinformation, perhaps in conjunction with other strategies, such as combining media literacy interventions and fact-checkers (Hameleers, 2023Hameleers, M. (2023). Disinformation as a context-bound phenomenon: Toward a conceptual clarification integrating actors, intentions and techniques of creation and dissemination. Communication Theory, 33(1), 1–10. https://doi.org/10.1093/ct/qtac021).

Limitations and Future Research

While we consider our findings to represent a successful direct replication of our target study, there are limitations to consider around external validity (Clarke et al., 2024Clarke, B., Alley, L. J., Ghai, S., Flake, J. K., Rohrer, J. M., Simmons, J. P., Schiavone, S. R., & Vazire, S. (2024). Looking our limitations in the eye: A call for more thorough and honest reporting of study limitations. Social and Personality Psychology Compass, 18(7), Article e12979. https://doi.org/10.1111/spc3.12979). In terms of sample diversity, our replication was similar to Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408, where participants were predominantly educated to an undergraduate degree level or higher, and located in the United Kingdom or United States of America, where English was their first language. Future research could explore the generalisability of caveat statements as a debunking strategy across more diverse samples to identify potential moderators, such as through a big-team science approach (Ghai et al., 2024Ghai, S., Forscher, P. S., & Chuan-Peng, H. (2024). Big-team science does not guarantee generalizability. Nature Human Behaviour, 8(6), 1053–1056. https://doi.org/10.1038/s41562-024-01902-y; see Klein et al., 2018Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., Batra, R., Berkics, M., Bernstein, M. J., Berry, D. R., Bialobrzeska, O., Binan, E. D., Bocian, K., Brandt, M. J., Busching, R., … Nosek, B. A. (2018). Many Labs 2: Investigating variation in replicability across samples and settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225).

A second limitation relating to external validity is the low realism of the task and online data collection. Participants completed a continuous influence paradigm where they read a fictitious news article one sentence or passage at a time. We included a free-text box prior to the debrief to offer participants an opportunity to comment on their experience completing the study. One participant mentioned “There was a lot of memory recall which wasn’t ideal, I’m glad I made plenty of notes.” In the real world, it is unlikely someone is reading isolated sentences or making notes as they read a news article. To maintain overall data quality, we included an item for participants to self-report inattentive responding (Meade & Craig, 2012Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in survey data. Psychological Methods, 17(3), 437–455. https://doi.org/10.1037/a0028085) and while there are few differences between in-person and online data collection, online participants do report more distractions (Clifford & Jerit, 2014Clifford, S., & Jerit, J. (2014). Is there a cost to convenience? An experimental comparison of data quality in laboratory and online studies. Journal of Experimental Political Science, 1(2), 120–131. https://doi.org/10.1017/xps.2014.5). This means we are relatively confident in the overall quality of responses across the two samples, but the methods potentially have low realism. The next step in this line of research is for future studies to capture more real-life behaviour using realistic articles on a wider range of topics such as medicine and finance. Instead of relying solely on self-report data, the design could also use more direct measures such as eye-tracking to monitor how participants interact with the materials. For instance, Lee et al. (2024)Lee, Y.-I., Mu, D., Hsu, Y.-C., Wojdynski, B. W., & Binford, M. (2024). Misinformation or hard to tell? An eye-tracking study to investigate the effects of food crisis misinformation on social media engagement. Public Relations Review, 50(4), Article 102483. https://doi.org/10.1016/j.pubrev.2024.102483 compared gaze time on mock Facebook posts about food crises between different types of corrective rebuttals.

Finally, much like the original study, our replication also lacked a true control condition, which in turn did not allow us to measure misinformation persistence. Misinformation persistence here refers to the effect wherein individuals continue referring to misinformation even after correction (Ecker & Antonio, 2021Ecker, U. K. H., & Antonio, L. M. (2021). Can you believe it? An investigation into the impact of retraction source credibility on the continued influence effect. Memory & Cognition, 49(4), 631–644. https://doi.org/10.3758/s13421-020-01129-y; Irving et al., 2022Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408). As seen in the results, debunking through caveat statements does not fully eradicate misinformation, as participants in the correction group still made causal inferences. Measuring misinformation persistence would provide a better metric to evaluate the efficacy of debunking as a strategy to counter misinformation. Future research may expand the current design to include a control condition to establish baseline effects of viewing the stimuli, excluding any mention of evidence (misinformation) or correction.

Conclusion

Taken together, the results of our study demonstrated a successful direct replication, where debunking statistical misinformation reduced mistaken causal inferences. Through the open science practices of Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 sharing their data and materials, we could computationally reproduce their analyses, replicate their findings with new data, and show the robustness and generalisability of their conclusions to alternative analysis techniques. Collectively, Irving et al. (2022)Irving, D., Clark, R. W. A., Lewandowsky, S., & Allen, P. J. (2022). Correcting statistical misinformation about scientific findings in the media: Causation versus correlation. Journal of Experimental Psychology: Applied, 28(1), 1–9. https://doi.org/10.1037/xap0000408 and our findings support previous research and theory demonstrating the effectiveness of debunking techniques. Future research can address limitations around external validity to demonstrate whether the effectiveness of correcting statistical misinformation can generalise to more diverse samples and more realistic materials.

Disclosures

Conflicts of Interest

We declare no conflict of interest associated with the publication of this manuscript.

Funding

There were no external funds supporting this project. We paid the participant fees via Prolific using supervisor funds, where dissertation supervisors at the University of Glasgow receive funding per student to support their projects.

AI Use Statement

We used Microsoft Copilot GPT-5 for minor text creation. We generated the first draft of the 100-word abstract before manually editing the text.

Open Research Practices

Open data: https://osf.io/jpxbu/. Our direct replication data are available under a CC-BY-4.0 licence and the original data from Irving et al. are publicly available on the OSF (https://osf.io/ps98w/overview). Open materials and analysis script: https://osf.io/jpxbu/ Pre-registration: This study was not pre-registered.

CRediT Statement

The authors made the following contributions. Dhruv Aggarwal: Validation, Investigation, Writing - Original Draft Preparation, Writing - Review & Editing; Keyi Zhang: Conceptualization, Investigation, Writing - Original Draft Preparation; James E Bartlett: Conceptualization, Data curation, Formal analysis, Investigation, Writing - Original Draft Preparation, Writing - Review & Editing, Supervision.

R Packages

We used R (Version 4.6.1; R Core Team, 2026) and the R-packages BayesFactor (Version 0.9.12.4.8; Morey & Rouder, 2023Morey, R. D., & Rouder, J. N. (2023). BayesFactor: Computation of Bayes factors for common designs [Computer software]. https://CRAN.R-project.org/package=BayesFactor), bayesplot (Version 1.15.0; Gabry et al., 2019Gabry, J., Simpson, D., Vehtari, A., Betancourt, M., & Gelman, A. (2019). Visualization in Bayesian workflow. Journal of the Royal Statistical Society A (Statistics in Society), 182(2), 389–402. https://doi.org/10.1111/rssa.12378), bayestestR (Version 0.18.1; Makowski et al., 2019Makowski, D., Ben-Shachar, M. S., & Lüdecke, D. (2019). bayestestR: Describing effects and their uncertainty, existence and significance within the Bayesian framework. Journal of Open Source Software, 4(40), Article 1541. https://doi.org/10.21105/joss.01541), brms (Version 2.23.0; Bürkner, 2017Bürkner, P.-C. (2017). brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software, 80(1), 1–28. https://doi.org/10.18637/jss.v080.i01), cowplot (Version 1.2.0; Wilke, 2020Wilke, C. O. (2020). cowplot: Streamlined plot theme and plot annotations for ‘ggplot2’ [Computer software]. https://CRAN.R-project.org/package=cowplot), dplyr (Version 1.2.1; Wickham, François, et al., 2023), effectsize (Version 1.0.3; Ben-Shachar et al., 2020Ben-Shachar, M. S., Lüdecke, D., & Makowski, D. (2020). effectsize: Estimation of effect size indices and standardized parameters. Journal of Open Source Software, 5(56), Article 2815. https://doi.org/10.21105/joss.02815), emmeans (Version 2.0.4; Lenth, 2023Lenth, R. V. (2023). emmeans: Estimated marginal means, aka least-squares means [Computer software]. https://CRAN.R-project.org/package=emmeans), ggplot2 (Version 4.0.3; Wickham, 2016Wickham, H. (2016). ggplot2: Elegant graphics for data analysis. Springer. https://doi.org/10.1007/978-3-319-24277-4), janitor (Version 2.2.1; Firke, 2023Firke, S. (2023). janitor: Simple tools for examining and cleaning dirty data [Computer software]. https://CRAN.R-project.org/package=janitor), knitr (Version 1.51; Xie, 2015Xie, Y. (2015). Dynamic documents with R and knitr (2nd ed.). Chapman & Hall/CRC. https://yihui.org/knitr/), papaja (Version 0.1.5.9001; Aust & Barth, 2023Aust, F., & Barth, M. (2023). papaja: Prepare reproducible APA journal articles with R Markdown [Computer software]. https://github.com/crsh/papaja), patchwork (Version 1.3.2; Pedersen, 2023Pedersen, T. L. (2023). patchwork: The composer of plots [Computer software]. https://CRAN.R-project.org/package=patchwork), performance (Version 0.18.0; Lüdecke et al., 2021Lüdecke, D., Ben-Shachar, M. S., Patil, I., Waggoner, P., & Makowski, D. (2021). performance: An R package for assessment, comparison and testing of statistical models. Journal of Open Source Software, 6(60), Article 3139. https://doi.org/10.21105/joss.03139), psych (Version 3.2.9; Revelle, 2023Revelle, W. (2023). psych: Procedures for psychological, psychometric, and personality research [Computer software]. https://CRAN.R-project.org/package=psych), pwr (Version 1.3.0; Champely, 2020Champely, S. (2020). pwr: Basic functions for power analysis [Computer software]. https://CRAN.R-project.org/package=pwr), qualtRics (Version 3.3.0; Ginn et al., 2022Ginn, J., O’Brien, J., & Silge, J. (2022). qualtRics: Download ‘Qualtrics’ survey data [Computer software]. https://CRAN.R-project.org/package=qualtRics), readr (Version 2.2.0; Wickham, Hester, et al., 2023), tidybayes (Version 3.0.7; Kay, 2023), tidyr (Version 1.3.2; Wickham, Vaughan, et al., 2023), and tidyverse (Version 2.0.0; Wickham et al., 2019Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L. D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T. L., Miller, E., Bache, S. M., Müller, K., Ooms, J., Robinson, D., Seidel, D. P., Spinu, V., … Yutani, H. (2019). Welcome to the tidyverse. Journal of Open Source Software, 4(43), Article 1686. https://doi.org/10.21105/joss.01686) for all our analyses.