OHBM 2022 Keynote Interview with Jonathan R. Polimeni: Modeling to invert the fMRI signal

By Steven Baete

Assistant Professor of Radiology, Center for Biomedical Imaging, New York University

Dr. Jonathan Polimeni is Assistant Professor of Radiology at Harvard Medical School and of Biomedical Engineering at Massachusetts General Hospital, Athinoula A. Martinos Center for Biomedical Imaging. In his research, he focuses on the fundamental understanding of neural activity in the brain, often in the visual cortex. In pursuing this understanding, Dr. Polimeni has along the way pushed the boundaries of fMRI. His work has resulted in many contributions to both neuroscience and functional imaging science, both in insights gained and in technical advancements. We had the opportunity to chat with Dr. Polimeni about his experience as a scientist and his vision on functional imaging.

Steven Baete (SB): To start things off, if you were not talking to brain mappers or scientists, how would you describe your research and your most proud scientific accomplishment?

Jon Polimeni (JP): I would first say that MRI tracks brain function not by detecting neural activity directly. Instead, you can see where the blood flow is increased in the brain in order to deliver oxygen to where it is needed. And because of the magnetic properties of the blood, we can track this with MRI. The blood vessels of the brain are quite smart, and can deliver blood exactly to where it is needed, when it is needed. The goal of my work is to understand how the blood flow is delivered to the brain and to build technologies to image this delivery more clearly. To make functional MRI a better tool to see neural activity and brain function in working brains.

My proudest scientific accomplishment is just to be able to contribute. As a domain, I feel like we have been able to both develop technologies to improve our abilities to track brain function with fMRI and to shed a few insights into this blood flow regulation. I am not sure if I can point to a single achievement, I am just happy to be a part of this endeavor.

SB: When I look through literature, you have many contributions to both neuroscience and MRI imaging elements of fMRI. As far as I know though, you started out your science career in neuroscience. How did you enter the field of MRI physics? And how did the two fields come together for you?

JP: I have long been interested in both neuroscience and in imaging, although my degrees are all in electrical and computer engineering, which I think is excellent training for experimental science, honestly. My very first research experience was actually in neuromorphic engineering. We were building analog integrated electronics that mimic the retina which we used as imaging systems. From there, I later moved to the computational neuroscience and computer vision laboratory of Eric Schwartz. There I first started working on the computer vision side, but I soon became more interested and more involved in the computational neuroscience side. I was building models of the functional architecture of the visual cortex, mainly these retinotopic models, and sought out data to test these models. We used some optical imaging data and electrophysiology data, plus a lot of histological data that Eric acquired back when he was in New York. We were really interested in testing these models with human data. Eric had done some human PET imaging many years back. fMRI seemed like the best way to acquire the human data that we needed, or we thought that we needed.

At first, we were a bit skeptical of fMRI, because it was based on blood, and not neural activity. But we had seen some impressive data by, for example, Marty Sereno and Anders Dale, who were performing retinotopic mapping in humans using fMRI. And so we decided to give it a try. We wrote a grant, an R01, with Anders and Bruce Fischl, who was also a graduate of our lab. That was my start.
We knew the limitations of fMRI in terms of spatial resolution and imaging distortions. So I was really eager to see if we could push the technology to be able to acquire more accurate data that we could use to test our models. I knew, from my early work in intrinsic signal optical imaging, that the resolution of the hemodynamic response should be sufficient. So the issues then seemed to us to be mainly how to achieve high enough resolution and SNR with MRI.

It is funny, I flipped through my thesis last night, just to see what my thinking was back then. It was remarkable to see how many of the issues that we are still concerned about today were listed in my very-long future directions chapter. I could tell I was definitely excited about the potential of fMRI. I was very interested in how we could acquire better data. This is how I got connected with Larry Wald through Eric, who ended up being my postdoctoral supervisor and mentor. I learned MR physics from Larry, both because I needed to understand how I could acquire the data that I wanted to collect and because it is also a lot of fun to learn. In the end, I ended up spending quite some time applying what I learned to develop new methods for fMRI acquisition that went beyond what I needed for my own experiments. Also, Eric’s own training was in high energy physics, so we were already in an environment that was very heavy in terms of mathematical physics and trying to understand these MRI signals at a deeper level. I was already sort of primed for this postdoc with Larry.

SB: A big interest of yours is, of course, laminar fMRI. How did you get interested in studying brain activation in a laminar fashion?

JP: Again, rereading my thesis, I see that I was thinking about this during our first fMRI experiments back with Bruce and Anders. Back then we were modeling the columnar systems in the visual cortex and there are many differences across cortical layers that we had to be mindful of. For example, in our early retinotopic mapping data we knew that receptive fields were smallest in layer IV. I saw in my thesis a few examples of performing cortical surface based analysis of our retinotopic data from layer four to see whether the map was less blurry than the map that we had sampled near the white matter. But, spoiler alert, it was not. I had not quite appreciated how the hemodynamic blurring was a much larger factor than any kind of “neuronal blurring”. We also knew that the vascular density varied across layers, but I was not sure then how this might affect our data. I remember completing my thesis and not knowing how this blurring was going to affect the results.

I would help process some of the histological data that our lab was acquiring, mostly in the monkey visual cortex, including ocular dominance column data. When you saw this ocular dominance pattern, clearly it was important to sample the data from layer four, where the cells are monocularly driven.

Eric was one of the only researchers who was able to—I don't know how they quite did this—physically flatten the cortical tissue, basically by pressing this really fragile tissue between two glass slides. Then once it was flattened, he would section the tissue tangentially, to study these columnar patterns as a function of cortical depth or cortical layer. In a way, this is exactly analogous to what I later did with fMRI when Bruce Fischl and I applied surface based analyses to generate maps of activation as a function of cortical depth. This was our 2010 article with Doug Greve and Larry Wald, where we used retinotopy to impose activation in the shape of this letter M on the surface of V1. In that study, we looked at the activation across cortical depths, similar to what Eric had done with histology data.

We also generated laminar profiles of activation and we thought we saw a bump around layer IV in the activation, a slightly larger BOLD signal there. At the time we thought there was just higher vascular density there. But then, Larry, I think it was Larry, had the idea for us to start using these tools that we had developed to look at functional connectivity between layers. We wanted to use fMRI to try to decipher directionality of information flow between cortical areas, and to add arrows to the undirected graphs that we have in functional connectivity. Ever since, I have been hooked on laminar fMRI.

SB: Do you think that laminar fMRI has broad applications in general neuroscience studies or is it more of a niche topic?

JP: First of all, I think laminar fMRI is conceptually very simple, right? You just sample the fMRI data across layers, and look for differences. In practice however, laminar fMRI is really hard to do. With laminar fMRI or any high resolution MRI, we typically operate at the limits of the imaging resolution that we can achieve to acquire small enough voxels. The data is thus just noisy and fragile. Accurate registration of the data and segmentation of the layers is critical. There are so many opportunities to make mistakes. The biggest challenge maybe is the interpretation of the data though, which I am going to talk about in this keynote lecture.

The vasculature does impose many biases in the data, and not just the large vessels. As I sort of alluded to, there is such a close relationship between the vascular and functional architecture. It makes understanding these laminar fMRI patterns really difficult. For example, layer four coincides with a band of high capillary density, which may bias some of the fMRI signals there. Relating the fMRI signals back to the neural activity for laminar fMRI, is especially tough.

Another big challenge to broader applicability of laminar fMRI is that there is a lack of ground truth regarding the functional properties of neurons across layers. There are some general rules of thumb about input and output layers for feedforward, feedback and lateral pathways, but they do not always hold. Sometimes I think I know a lot about the visual system or I know a lot about the brain, but I have been humbled trying to generate new stimuli to activate certain populations of neurons. It is often really unclear how to stimulate specific pathways experimentally. This lack of ground truth makes it challenging to validate observations made in laminar fMRI.

On the other hand, maybe as we get better at interpreting high-resolution fMRI data, we can use this approach to learn something new about the human brain. Ultra high resolution fMRI is often used to confirm fine-scale features of functional architecture, originally seen in animal models. Laminar fMRI may be capable of making some new discoveries about human brain circuitry, which is both exciting and a bit scary. Since it is an unknown territory we have to proceed carefully and cautiously. One step at a time. But I do think that there is some potential.

SB: In your publications, and some previous talks, I have seen that you are interested in more neurally specific fMRI contrast than BOLD MRI. Which contrasts do I have in mind? And why do you think they're important?

JP: Actually, I think that BOLD fMRI can work really well for many applications. But what I would like to see is a broader accessibility of fMRI techniques, so that researchers can use the most appropriate tool for any given question. For cases where sensitivity is really paramount, such as when we are looking for very small effect sizes or really subtle cognitive tasks, BOLD fMRI is really tough to beat. However, non-BOLD techniques, I think, do have promise. Certainly the classic methods for measuring cerebral blood flow or perfusion, and newer methods for measuring cerebral blood volume. These techniques seem capable of achieving higher neuronal specificity and are therefore better at localizing activation at small spatial scales. The main limitations of these methods today seem to be the limited sensitivity and also the ease of use. A lot of these sequences are still very challenging to optimize and to utilize.

I am becoming more and more interested in newer versions of the basic technique for measuring CBV, since it appears to achieve a good balance between sensitivity and specificity. Not the most sensitive or the most specific, but I think it has a pretty good balance. Renzo Huber, the VASO expert and laminar fMRI guru, has taught us this. I still do not fully understand the signal mechanism and aspects of the micro-vascular and hemodynamic response that VASO reflects. But, we are starting to dig into this with help from Renzo and I agree with him that VASO does have great potential. Not only for laminar fMRI, which is what Renzo has been applying VASO to the most, with the most success also, but also for a broader range of experiments if some of these practical engineering issues can be sorted out.

As much as I like BOLD, since it is so robust and easy to use, if more neurally specific fMRI contrasts could become more available and more accepted by the community, I think that this could go a long way towards improving the overall quality of fMRI. I think there is a lot of potential when using these powerful techniques.

SB: Probably even more interesting is the combination of these different contracts in a generalized modeling framework. What is the value of such a model and what is the ideal model you have in mind?

JP: I think that there are two ways to answer this. One is more conventional. Each contrast has varying levels of sensitivity to these small activations in the presence of noise and specificity to the neuronal signal of interest. Many researchers are interested in combining these contrasts, to achieve the best of both worlds, both high sensitivity and high specificity. We have already seen some examples of this, in fMRI as well as in diffusion MRI.

Another answer, maybe more interesting, is that none of these fMRI contrasts is directly reporting on the aspects of the vascular and hemodynamic responses that are most closely coupled to neural activity. The hope is thus that we can improve our understanding of what these fMRI contrasts really reflect in terms of vascular and hemodynamic responses by having a model for how the neural activity affects these vascular and hemodynamic signals that we measure. Then we may be able to, or at least we hope to, relate these measurements back to the neural activity. I am thinking in terms of spatial localization, also temporal localization, and in terms of different forms of neural signaling. I have seen some intriguing new work looking at how inhibitory and excitatory neural activity may couple into the vasculature in slightly different ways. That could potentially be teased apart with such a model.

Just like our friends using MEG and EEG, who build sophisticated models that they attempt to invert to localize neural activity from measurements made on the surface of the head, I envision similar vascular or hemodynamic models that can be inverted. The inversion will work best if we can observe signals from as many aspects of the vascular and hemodynamic responses as possible.
I think that one of the things that we are learning, and we will be presenting some of this work at the meeting, is that it is not enough to have a variety of observations or acquisitions of contrasts that we understand. It is also important to stimulate the system in different ways, to try to interrogate the system and perform a system identification. We can then use that information to build up a model that can be inverted. I would almost put more effort into experimental design than in deciding which contrasts to use. How would I stimulate the neural activity in order to best observe these responses so that I can engage as many components of the vasculature as possible to try to build a model that can be inverted?

SB: Another topic that you have published on over the years is the impact of more subtle fMRI artifacts, such as respiratory variation or heart rate changes, the dependence of BOLD on the orientation of the vasculature, and the impact of smoothing during post processing steps. To what extent do you think that the field has found and implemented good solutions for these problems?

JP: For the systemic, physiological changes that you mentioned, there has been a lot of work on identifying and removing these artifacts. I am not sure though if any one method can work well for all datasets. I work mainly in very high resolution fMRI and I often find that methods designed with more conventional data in mind can underperform with high resolution data. I do think that for conventional data, at least, there are many approaches available.

What may be more interesting to me on this topic about physiological dynamics is that in some cases the distinction between neuronal activity and physiological noise becomes less clear. They melt into one another. Of course, neurons in the brain drive these physiological cycles, often through autonomic pathways. However, there can be physiological variations that are affected by stimulus themselves. We could call it “stimulus correlated motion” and “stimulus correlated physiological changes”. I have spoken to Matt Glasser quite a bit about this. He has thought about this a lot, how physiological changes may affect neuronal function as well. For example, you take a deep breath, this can have kind of a calming effect.

I hope that future studies that look at brain function more holistically and we can start to integrate information from systemic physiological dynamics to help interpret the data rather than just throwing it out as noise. In my lecture, I will show that there is some evidence that several large-scale neuronal networks are accompanied by physiological or vascular networks, which I also find to be very intriguing. There seem to be some interesting relationships between the patterns of neural activity and the patterns of physiological activity which make this distinction a bit harder to make.

You mentioned the orientation effects? It is quite a new finding. We have been working on a method to correct for the associated detection biases. We have integrated some tools into the FreeSurfer package, but these are works in progress. Stay tuned on that!
The inadvertent smoothing, this is an interesting point. I think that the HCP, the Human Connectome Project, certainly did a nice job of developing some best-practice processing pipelines, reducing this effect. It is really important to provide people with tools that they can use to perform these best-practice analyses. They really made this widely available and credit goes again to Matt Glasser for this.

What I would like to see more of in the future is a broader recognition of how different processing choices can inadvertently affect the smoothness of the data. There was this amazing Nature article in 2020 that showed how 70 different research groups who all analyzed the exact same dataset got quite different results. The authors of the study attributed this to differences in the level of either implicit or explicit smoothing performed during preprocessing. I hope that this really striking example can help increase awareness of these effects, and at least get people to consider this potential issue as they're designing their preprocessing strategy.

SB: What do you think are the biggest challenges you hope to overcome with your research in the next 10 years?

JP: Tough question. I think that there are several challenges that are faced by specific projects in the lab. One sort of big challenge that spans all of the projects, is the question of how do we bridge the divide between the neuroscience and fMRI communities? These two communities are thinking about brain function at very different scales. If we can convince our friends in classical neuroscience that fMRI can be used not only to accurately localize but also to understand brain computation, I think that could go a long way. I think today these two communities just do not interact as much as they should, as much as we should.

fMRI has come a long way, especially in the past 10 to 15 years. Our understanding of how faithful fMRI signals can be in terms of reflecting neural activity has changed enormously. I am not sure if the neuroscience community has really appreciated this. I would like to try to help bridge these two communities and establish some sort of common ground.

SB: Stepping away from the science for a moment, let's talk about your career path. What were the main challenges and opportunities you came across in your career so far?

JP: I do not think I have any sort of major, major challenges to report. One sort of personal challenge that I think others can relate to, is how to prioritize. There are so many fun problems to tackle and so many interesting mysteries to solve. It can be hard to focus. How to pick the right problem to focus on.

Opportunities, gosh, I feel really lucky to be at the Martinos Center. It is such an open and collaborative environment that is brimming with energy. I was talking to Peter Bandettini a few months ago and he was mentioning his time at the Martinos Center as a postdoc. He said then it was crackling with energy. And I said: “Gosh, Peter, it's the same way today.” The center has such amazing resources in terms of MRI scanners and instrumentation and capabilities in other forms of imaging that can help us tackle tough questions. Of course, the main resources are the people. Just walking down the hall, bumping into somebody and striking up a conversation about current research often results in a lot of new ideas and new opportunities. Just being in such a collaborative and open environment, it has been a huge opportunity for me.

SB: Who are the people that have inspired you throughout your career?

JP: Gosh, so many to list! Maybe the main people? I have been really fortunate to have a lot of amazing mentors. I have mentioned several of them already in this discussion. The person who has inspired me for the longest period of time is my PhD supervisor, Eric Schwartz. He was really a no BS guy who taught us by example what it means to be a scientist and to be skeptical and thorough and do the work. He also taught us to just have the confidence to dive into tough problems. He was a real inspiration. I already mentioned Larry Wald, Bruce Fischl, Bruce Rosen. I have been really fortunate to have terrific mentors throughout my career.

SB: What is the best piece of scientific advice you have received? And from who?

JP: It is hard to think of just one. One may be, I think that it was from my postdoc mentor, Larry. It was not really any one piece of advice he gave me on one occasion, but something that he said in different versions on different occasions. He really pushed me to think bigger and to tackle important problems with my research. Not just problems that I had an intellectual curiosity to solve. I have been fortunate to find some problems that are both important and interesting to me. I will certainly indulge occasionally on a fun project, but sometimes I have to take a step back from what I'm working on and to try to understand whether it is going to have an impact, and whether it will help push the field forward. This is not an attempt to make an impact just to advance my career, but life is short and as scientists we want to make a contribution and help the world as best we can. That is what I really took from Larry's advice, to think big and try my best to work on important problems.

SB: That is indeed good advice. Without spoiling any surprises you may have prepared, what can attendees expect to hear in your keynote at OHBM this year?

JP: Since OHBM is such a diverse audience, I am trying, hopefully, to put some interesting material in for as many people as possible. I will be touching upon several topics we discussed today, including, the vascular and hemodynamic modeling. I will not be focusing too much on laminar fMRI or fMRI technologies. What I am trying to do in this lecture is to make the case that the blood vessels have an interesting story to tell, both in terms of anatomical layout of the blood vessels and how precisely they respond to neural activity. By paying attention by listening to the blood vessels, we can learn a lot about the brain.

SB: Lastly, what do you enjoy doing beyond research?

JP: I like to spend time with family and friends as much as possible. I also enjoy traveling, especially to far-off places. Both to experience other cultures, mostly through their food, and to experience natural beauty and bustling cities. I really enjoy eating while traveling. Music has always been kind of a big part of my life outside of research. I have not made enough time for this lately, but maybe I will have to correct this after I finish preparing my lecture!

SB: Of course! Thank you very much Dr. Polimeni for your time and the interview.