Interview with the SPM Team, winner of the Open Science Award 2023
Author: Elisa Guma
Editors: Simon Steinkamp, Elizabeth DuPre
Learn more from the SPM team about open science.
Queen Square, home of the FIL and other neuroscience / neurology departments, in the winter snow from a previous year (courtesy of Peter Zeidman)
Next in our award winner interview series, we had the chance to hear from this year’s Open Science Award winner, the Statistical Parametric Mapping (SPM) Team based out of the Functional Imaging Laboratory (FIL) at University College London. SPM is a free, open source, and widely used software suite designed for the analysis of brain imaging data across various modalities including PET, fMRI, EEG, MEG, and SPECT. Additionally, SPM provides different analysis approaches for neuroimaging data that go beyond the classic General Linear Model (GLM), such as Dynamic Causal Modelling (DCM) and Voxel Based Morphometry (VBM). SPM was first developed by Karl Friston (see an interview with him from 2017 on this blog) for the statistical analysis of Positron Emission Tomography (PET) and functional Magnetic Resonance Imaging (fMRI) data. Since then, it has gone through several technical improvements to reflect theoretical advances in the field (here is a history on their website and a retrospective piece about the software by Dr. John Ashburner). The current software version can be found here. In addition to maintaining and improving the software, the SPM team also offers in-person courses to help neuroimagers learn how to use their tools.
Since the early 1990s, SPM has been at the forefront of open science—even before the notion of open science was widespread in the neuroimaging community. Indeed, Dr. Karl Friston used to give away the software on floppy disks to those who asked; now it is freely available for download. Additionally, the SPM team has been leading substantial efforts to teach SPM and its methods by providing courses or publishing tutorials.
We are grateful to Peter Zeidman, Olivia Kowalczyk and the SPM team for being willing to answer a few questions about their work. Read on to learn more about the SPM team and their thoughts on open science!
Q1: How would you characterize the state of open science today, and what aspects do we still need to improve on?
SPM Team: First, thank you again to OHBM for our 2023 Open Science Award! It recognises the hard work of generations of researchers who have developed new analysis methods in the SPM software package and made them freely available to the community. A lot has changed since Karl Friston travelled the world in the 1990s, giving away SPM on floppy disks. But the culture of making analysis methods openly available—and dedicating time every week to supporting users with their analyses—has remained constant through SPM’s 30-year history.
Open research has brought enormous benefits to the neuroimaging community. It is now entirely unremarkable to obtain a dataset using OpenNeuro, in standard machine-readable BIDS format, then apply an open-source analysis package like SPM or FSL to test hypotheses, and use a meta-analysis tool like Neurosynth to help contextualise results against the literature. These brilliant advances have come through the hard work of dedicated researchers, who care deeply about the research we conduct as a community.
To keep up this pace of change, we need to provide researchers with incentives for open research beyond “knowing it’s the right thing to do.” Grassroots initiatives have been essential in promoting open research, but these are often driven by early career researchers, who may not have the decision-making power, budget or duration of employment to deliver the projects they envisage. Now, the key drive needs to come from the top. Institutions, journals and in particular, funders, need to get involved to incentivise the practices that we all know are best for research.
Q2: Even though we know it’s best practice, what are some challenges that make open science harder for researchers to achieve?
SPM Team: There are many remaining challenges for open research we could talk about—lack of awareness of good practice, lack of time, lack of know-how, the need for support from line managers and need for greater diversity. But we’ll highlight one pressing issue that we have been thinking a lot about recently: how to ensure that people understand the theory and assumptions that underlie their analyses.
It is now easier than ever to perform neuroimaging analyses using open-source software. However, if these tools are used as “black boxes” without understanding what’s going on under the hood, then there is a real danger of generating meaningless results. For analysis software to be truly “open source”, it means more than simply making the code freely available; we need to provide training and meaningful documentation on the methods and algorithms that are written into analysis software.
This issue is particularly acute where pipelines are used, which are wrappers that enable multiple analysis packages to be flexibly chained together. Pipelines have become increasingly popular and have delivered clear benefits—such as making analyses reproducible at the click of a button, and for baking-in quality control procedures. However, with software packages like SPM, FSL and AFNI being combined in ways that the developers hadn’t envisaged, there can be unintended consequences that may not be obvious such as changing the statistical properties of data that violate the assumptions of subsequent tests. To avoid these problems, it’s vital that pipelines are carefully considered from a theoretically-principled perspective, and that researchers have some understanding of the theory when applying neuroimaging analysis software.
We are hard at work in the SPM Dev Team to provide better training on the methods that underlie neuroimaging analyses. We have just released the first stage of a new documentation website at (https://www.fil.ion.ucl.ac.uk/spm/docs/). The development of this website was led by Dr Olivia Kowalczyk, and it includes click-by-click tutorials on how to use SPM, together with all new video lectures on the relevant theory, delivered by our international faculty. We have also revamped our bi-annual SPM courses, which take place in London and online, to better balance the delivery of theory and interactive workshops. We’ve made videos of all the lectures freely available at https://www.fil.ion.ucl.ac.uk/spm/course/
Q3: Focusing on SPM, what are some challenges in maintaining such a large but essential toolbox for the neuroimaging community?
SPM Team: An academic saying that they need more funding is about as surprising as a child saying they need more candy. That said… we need more funding. Specifically, we need funding opportunities for developing and maintaining research software. The teams who develop neuroimaging packages like SPM tend to be far smaller than their commercial counterparts, and they typically have many academic commitments beyond writing software. There are projects that we’d love to conduct, such as refactoring particular sections of the code, if only we had the time and resources. This work may not be as seductive as discovery science, but it is vital for enabling future discoveries—so we need this funding gap to be addressed.
Other key challenges in the SPM Dev Team relate to managing a large codebase in a changing software landscape. SPM is open-source, written 90% in MATLAB and 10% in C++. We also provide Standalone SPM as a compiled version that people can use without a MATLAB license. Our code is highly optimized, having been continuously developed and refined for over 30 years. This makes SPM very stable and bug reports rare. By writing our code primarily in MATLAB, we create analysis tools that work on any platform with good performance. This gives us more time to focus on methods development.
Nevertheless, people are increasingly preferring programming languages that are fully open source— Python in particular. In the SPM Dev Team, we get it, and a priority for us is making SPM accessible to everyone. There are excellent projects like Nipype, which already enable Python users to integrate SPM into their analysis pipelines. But our long-term plan is to go further, enabling people to have the same quality of experience using SPM, regardless of whether they are in a Python or MATLAB environment. Of course, developing this kind of deep integration with Python will require funding – so watch this space!
Finally, I will mention that we are changing how we do software development, aimed at greatly improving transparency. Thanks to the hard work of Dr Guillaume Flandin, we have recently moved from a private subversion repository to a public Github account (https://github.com/spm/), which means that all of our development work is now public facing. We are expanding the team who actively contribute to SPM, refreshing our course content and writing new training materials. In terms of features, I can highlight new tools in SPM for OP-MEG analysis (led by Dr Tim Tierney), https://www.fil.ion.ucl.ac.uk/spm/docs/tutorials/opm/. We will also be implementing novel methods for analysing naturalistic neuroscience experiments which tend to involve free movement and long recordings. This work will be funded by a Wellcome Discovery Research Platform grant (https://www.fil.ion.ucl.ac.uk/news-item/department-of-imaging-neuroscience-awarded-funding-to-discover-how-the-brain-allows-us-to-live-our-lives/). We are looking forward to a very exciting few years ahead for SPM—and for open research in neuroimaging!