Misuse of Bayesian inference in applied research: Current status and constructive feedback

Abstract

The use of Bayesian statistics in many scientific fields is a relatively new endeavor. The social sciences, for example, are witnessing a growing interest for Bayesian inference for the last 10 years. Several reasons exist to explain this phenomenon. One explanation is the fact that commonly used frequentist inferential tools, such as the p-value and the confidence interval, seem to fail to provide answers to the questions that applied researchers wish to answer. Bayesian inference, if correctly used, may help practitioners in this regard. As such, many people are being attracted to the relatively new and unknown Bayesian framework. However, it is unclear whether applied researchers are correctly using Bayesian inference in their work, both while writing their research reports or while reviewing manuscripts that employ Bayesian methodology. The fear that misunderstanding of Bayesian inference in applied research may be occurring is not in vain. After all, there is abounding evidence indicating that p-values and confidence intervals have been consistently misused, in spite of the fact that frequentist statistics has been around for about 100 years. Wong et al. (2022; https://psyarxiv.com/86p4k) conducted a first pilot study that shed some light onto the misuse of Bayesian inference in the social sciences literature. Specifically, they focused on null hypothesis Bayesian testing and the associated Bayes factor, which is the Bayesian counterpart to the frequentist p-value. Wong and colleagues inspected a small sample of 73 papers that used the Bayes factor to test statistical hypotheses. Results provided evidence that, indeed, problems in understanding, reporting, and interpreting hypothesis testing outcomes through the Bayes factor do exist. In this talk, I will present the results of an ongoing study that further extend those from Wong and colleagues. This work is the result of a collaboration between five researchers, with experience in both the frequentist as well as in the Bayesian inferential frameworks. In this study, we conducted a large search through the Web of Science and Google. Our target were papers in the social sciences published since 2010 that used the Bayes factor to test hypotheses or, more generally, to perform model comparison. Our final sample includes 167 papers. In this talk, I will start by offering a quick introduction to basic frequentist statistics and its shortcomings. I will then introduce the basics of Bayesian statistics, specifically in the realm of hypothesis testing and the Bayes factor. Besides presenting the results of our literature study, I will also report the outcomes on the joint discussions that our research team had. Specifically, we attempted to diagnose each of the problems that we found (over 10 in total). This is crucial if we are to contribute to improve the current state of affairs in the near future; I will also offer some specific suggestions in this regard. Participants in this talk may expect to acquire a deeper perspective on both frequentist and Bayesian inference. Participants will benefit from this talk by improving their skills in correctly reporting and interpreting statistical outcomes. More broadly, this talk really concerns anyone who may be considering adding Bayesian statistics to their analysis toolbox (or those who have already done so but feel a bit uneasy about it). The contents to be presented are of an introductory nature, thus little prior knowledge is required, in particular regarding Bayesian statistics.