In this episode, our guest is Rotem Botvinik-Nezer, a postdoc at Dartmouth University, working with Dr. Tor Wager in his Cognitive and Affective Neuroscience Lab. In 2020, Dr. Botvinik-Nezer was first author of an influential paper published in Nature, titled Variability in the analysis of a single neuroimaging dataset by many teams, where the results were compared from 70 independent teams analyzing a single data set having 9 hypotheses. This paper made it clear that there are many points of variability in data analysis pipelines, and provided further incentives for sharing data and code to grow consensus and replicability. While the popular press suggested that this paper was yet another hit to fMRI, we discuss how even papers that critique the results of this seminal paper ultimately converge in agreement with the overall message of systematic transparency. Dr. Botvinik-Nezer also has a strong interest in how our brains influence our perception of pain, having just published a recent paper showing evidence that regions associated with painful stimuli remain active even when subjects experience less pain while having the belief that a placebo is effective.

In this conversation, Peter and Rotem delve into all these topics and more, but spend the bulk of the discussion on the interplay between choices in analyses, such as determining a statistical threshold, and variability in results. We also discuss incentives for users to share data and code and possible ways to create a more solid scaffolding for best practices.

Episode producers:
Omer Faruk Gulban
Xuqian Michelle Li