Cleared for Takeoff - Medium

The tipping point: From animal intelligence to human intelligence

Wed, 20 Jul 2022 17:43:00 GMT

There are many obvious things that we humans do to a much larger degree than other animals. We construct great civilizations, we create advanced technology, we use complex language, we make art and tell stories. How do our unique capabilities guide us in figuring out how our brains are different from those of other animals, if they are?

To me, the most revealing feature of human intelligence is that it is primarily societal, rather than individual. Most of what each of us knows or understands is taught to us, rather than things we figured out. We have found a way to accumulate intelligence across individuals and across generations, and because of this, collective human intelligence has exploded over the past few thousand years. This accumulation is the basis of nearly all of our advances. Each human who pushes the envelope of human knowledge is first a prodigious student of the state of the art at the time.

So, what does the brain need to do to support this kind of capability, and what brain architecture might be employed to implement it? My guesses at the answers to these questions are described in an article posted on Arxiv entitled A Reservoir Model of Explicit Human Intelligence , and here is a brief summary.

Our first innovation was imagination. By this I mean the ability to perform mental processing on things that are hypothetical rather than the immediate physical present. Without imagination, the brain is restricted to being an input-output mapping machine. The development of imagination seems to me to be the hardest evolutionary step. To support off-line processing, we had to develop mechanisms to switch between a real-world mode, vigilant of our surroundings and reacting appropriately to them, and an off-line mode, where we are free to consider hypothetical scenarios, predict potential outcomes, to ponder. This required neural mechanisms in the brain, likely involving the default mode network, but also community and societal mechanisms to provide safety to those who are ‘daydreaming’. Some point to the stone tool industry as early evidence of imagination, starting around 1M years ago, but imagination was clearly solidified by the time we were making sophisticated art on cave walls about 80K years ago.

Enabled by imagination, the second innovation was language. Even with access to an off-line world model, without labels for things that are not present at the moment, we are limited in our communication to direct demonstration of objects and actions that we wish to convey, like a traveler with no knowledge of the local language. But with labels for both objects and actions, we can describe, record, and accumulate. Words also allow us to categorize, define, and produce higher levels of abstraction, as we do with mathematical theorems.

With imagination and language, I think that humans just expanded existing associative networks and mechanisms to develop what is now called explicit, reportable, or explainable intelligence, the stuff we accumulate and pass on. Lower animals can easily be taught to make associations between previously unrelated stimuli by simply juxtaposing them, like in the classic experiments performed by Pavlov on dogs. Using that same kind of network, we build a web of associations, organized by the curricular plan that our teachers, parents, and mentors define, and construct in our students a distillation of human knowledge. Excitation of elements of the network can initiate excitation that produces output actions, or run along recurrent paths representing internal thought. It’s a big web, anchored by the 20,000 or so words we learn, with hundreds of thousands more abstractions added in including all of our long term memories. Words serve as a random access addressing system to directly excite sequences of abstractions in our brains, and also influence others by exciting sequences in their brains as well.

The previous billion years of evolution has done a slow but steady job of accumulating ever increasing intelligence in our genomes. But a tipping point occurred only a few thousand years ago, when intelligence began to be accumulated by the society itself, rather than by mutations in the genome. Accumulable intelligence requires that the knowledge be describable in a compact form for communication, so the intelligence must be stored in a form that is transparent, and a simple (though large) associative network may suffice. “Lower level” processes like visual processing are actually more complex, but do not need to be reportable in detail, and so have the luxury of utilizing deep networks with layers of hidden representations when they are discoverable by evolution.

I think that the two enabling developments for accumulable intelligence, capacities for imagination and language, were evolutionary innovations, probably driven by intelligence as a competitive advantage in changing natural environments. However, once this accumulation began, acceleration of collective intelligence became inevitable, despite the fact that the original evolutionary pressure largely evaporated when we mastered our environment.

The Challenge of BWAS: Unknown Unknowns in Feature Space and Variance

Mon, 04 Jul 2022 13:42:00 GMT

The paper by Marek et al ( Reproducible brain-wide association studies require thousands of individuals, Nature, 602, 7902, pp 654-660, 2022 ) came out recently, and caused a bit of a stir in the field for a couple of reasons: First, the title, while an accurate description of the findings of the paper, is bold and lacking just enough qualifiers to quell immediate questions. “Does this imply that fMRI or other measures used in BWAS are lacking intrinsic sensitivity?” “Is this a general statement about all studies now and into the future?” “Is fMRI doomed to require thousands of individuals for all studies?” The answers to all these questions is “no,” as becomes clear on reading the paper.

Secondly, I think that the reaction of many on reading the title was a sigh and a thought that this is yet another paper in the same vein as the dead salmon study, the double dipping paper, or the cluster failure paper that makes a cautionary statement about fMRI that is then wildly spun by the popular media to imply more damning impact than brain imaging experts would gather. Again, it’s not this kind of paper, however there was a bit of hyperbole in places. The Nature News article titled “Can brain scans reveal behavior? Bombshell study says not yet” discusses this in an overall reasonable manner but the need for an attention-grabbing title was unfortunate. The study was not a bombshell. The Marek study was a clear, even-handed, well-done (clearly a huge amount of work!) description of a specific type of comparison in fMRI and MRI performed in a specific way. While my reaction to the Merek paper was that of mild surprise that the reported correlation values were a bit lower than expected, I was more curious than anything, and thankful that such a study was performed to clarify precisely where the field – again, for a specific type of study performed in a specific manner – was.

I was asked by several groups to comment on it. First, I discussed my thoughts with Nature News. At the time of my discussion, I was still not certain what I thought of the paper, and was suggesting that there may be sources of error and low power that might be improved upon: such as population selection, the choice of resting state as the measure, time series noise, or even spatial normalization pipelines that might be smearing out much of the useful information. I aimed to emphasize in that discussion that it should be made clear that the Marek paper is emphatically NOT a statement about the intrinsic sensitivity of fMRI – which sensitive enough to reliably detect activation in single subjects – and even in single runs or with single events. It was more a statement on the challenges of extracting subtle differences between populations having different behaviors. While I feel that there is quite a bit that can be done to push the necessary numbers down (as a field, we are really just getting started), I can’t rule out the fact that people may just be too different in how their brains manifest differences in behavior – thus confounding attempts to capture population effects. It’s really an interesting question for future study.

I was also asked to write something for an upcoming collection of opinions on the Marek paper to be published in Aperture Neuro – a new publishing platform associated with the Organization for Human Brain Mapping. I finally submitted it a few weeks ago.

In the mean time, four of the authors (Scott Marek, Brenden Tervo-Clemmens, Damian Fair, and Nico Dosenbach) graciously agreed to be interviewed by me on the OHBM Neurosalience Podcast . This episode can be reached here . During this truly outstanding conversation, the authors further clarified the methods and impact of the paper. I pushed them on all the things that could be improved, methodologically, to bring these numbers down but was just a bit further swayed that one implication of these results may be that the variability of people, as we currently sort them based on their behavior, really might be larger than we fully appreciate. It should be emphasized that the authors main message was overall extremely positive on the potential impact and importance of these large N studies as well as the many other ways that fMRI can be used with small N or even individual subjects to assess activity or changes in activity with interventions.

I was lastly asked to write a commentary for Cell Press’s new flagship medical and translational journal, Med , which I just submitted yesterday and am adding to this blog post, below. However, before you read that, I wanted to leave you with a thought experiment that might help illustrate the challenge – at least as I see it:

It’s been shown that fMRI can track changes in brain activity or connectivity with specific interventions. Let’s say, after a month of an intervention, we clearly see a change. This is not unreasonable and has been reported often. We repeat this for 100 or 1000 subjects. In each subject, we can track a change! Now, here’s the problem. If we repurposed this study as a BWAS study by grouping all subjects together before and then after the intervention and compare the groups, the implication (as I understand Marek et al) is that we would likely not see a reliable effect that comes through, and those effects that we did see from this BWAS-style approach would lack the richness of the individual changes that we are able to see longitudinally with every one of the subjects. The implication is that each subject’s brain changed in a way that was reliably measured with fMRI, but each brain changed in a way that was just different enough so that when grouped, the effects mostly disappeared. Again, this is just a hypothetical thought experiment. I would love to see such a study done as it would shed light on specifically what it is about BWAS studies that result in effect sizes that are lower than intuition suggests.

Either way, here is the paper that I just submitted to Med. I would like to thank my coauthors, Javier Gonzalez-Castillo, Dan Handwerker, Paul Taylor, Gang Chen, and Adam Thomas for all their insights and in helping to write it. On last note, since this paper was a commentary, I was limited to 3000 words and 15 references. Otherwise it would have been much longer with many more relevant references.

The challenge of BWAS: Unknown Unknowns in Feature Space and Variance

Peter A. Bandettini ^1,2 , Javier Gonzalez-Castillo ¹ , Dan Handwerker ¹ , Paul Taylor ³ , Gang Chen ³ , Adam Thomas ⁴

¹ Section on Functional Imaging Methods

² Functional MRI Core Facility

³ Scientific and Statistical Computing Core Facility

⁴ Data Science and Sharing Team

National Institute of Mental Health

Bethesda, MD 20817

Abstract:

The recent paper by Marek et al. ( Reproducible brain-wide association studies require thousands of individuals, Nature, 602, 7902, pp 654-660, 2022 ) has shown that to capture brain-behavioral phenotype associations using brain measures of cortical thickness, resting state connectivity, and task fMRI, thousands of individuals are required. For those outside the field of human brain mapping and even for some within, these results are potentially misunderstood to imply that MRI or fMRI lack sensitivity or specificity. This commentary expands and develops on what was touched upon in the Marek et al. paper and focuses a bit more fMRI. First it is argued that fMRI is exquisitely sensitive to brain activity and modulations in brain activity in individual subjects. Here, fMRI advancement over the years is described, including examples of sensitivity to robustly map activity and connectivity in individuals. Secondly, the potential underlying – yet still unknown – factors that may be determining for the need for thousands of subjects, as described in the Marek paper, are discussed. These factors may include variation in individuals’ anatomy or function that are not accounted for in the processing pipeline, sub-optimal choice of features in the data from which to differentiate individuals, or the sobering reality that the mapping between behavior (including behavior differences) and brain features, while readily tracked within individuals, may truly vary across individuals enough to confound and limit the power of group comparison approaches – even with fully optimized pipelines and feature extraction approaches. True human variability is a potentially rich area of future research – that of more fully understanding how individuals expressing similar behavior vary in anatomy and function. A final source of variance may be inaccurate grouping of populations to compare. Behavior is highly complex, and it is possible that alternative grouping schemes based on insights into brain-behavior relationships may stratify differences more readily. Alternatively, allowing self-sorting of data may inform dimensions of behavior that have not been fully appreciated. Potential ways forward to explore and correct for the unknown unknowns in feature space and unwanted variance are finally discussed.

The Emergence and Growth of fMRI:

Human behavior originates in the brain and differences in human behavior also have brain correlates. The daunting task of neuroscience is to trace differences and similarities in behavior over time scales of milliseconds to decades back to the brain which is organized across temporal and spatial scales of milliseconds to years and spatial scales of microns to centimeters. Capturing the salient features across these scales that determine behavior is perhaps the defining challenge of human neuroscience. Insights derived from this effort shape our understanding of brain organization and may provide clinical utility in diagnosis and treatment. Advances in this effort are fundamentally driven by more powerful tools coupled with more sophisticated questions, experiments, models, and analyses.

When functional MRI (fMRI) emerged, it was embraced because activation-induced signal changes are robust and repeatable. Blood oxygen level dependent (BOLD) contrast allows non-invasive mapping of neuronal activity changes in human brains with high consistency and fidelity on the scales of seconds and millimeters. Because it was able to be implemented on the already vast number of clinical MRI scanners in the world, its growth was explosive. The activation-induced hemodynamic response, while limited in many ways, has become a widely used and effective tool for indirectly mapping brain human activation. It is indirect because it relies on the spatially localized and consistent relationship between brain activation and hemodynamic changes that result in an increase in flow, volume, and oxygenation. Increases in flow are measured with techniques such as arterial spin labeling (ASL), volume with techniques such as vascular space occupancy imaging (VASO), and blood oxygenation with T2* or T2 weighted contrast (i.e. BOLD contrast). BOLD contrast is far and away the most common of the techniques because of its ease in implementation and highest functional contrast of the three.

Early on, richly featured and high-fidelity motor and sensory activation maps were produced, followed quickly by maps of cognitive processes and more subtle activation. Then resting state fMRI emerged in the late 1990’s, demonstrating that temporally correlated spontaneous fluctuations in the BOLD signal organized themselves into coarse networks across 100’s of nodes. The study of the functional significance of these networks rapidly followed, accompanied by revelations that these networks dynamically reconfigured over time, and were modulated in association with specific tasks, brain states, or measures of performance(1).

Functional MRI has flourished over three decades in a large part because of its success in creating detailed and informative maps of brain activation in individuals in single scanning sessions. At typical resolutions, the functional contrast to noise of fMRI is about 5/1, depending on many factors. This robustness has enabled fMRI to delineate, at the individual level, activity changes associated with vanishingly subtle variations in stimuli or task, learning, attention, and adaptation to name a few. Additionally, in quasi-real time, fMRI has successfully provided neuro-feedback to individuals, leading to changes in connectivity and, in some cases, behavior(2). Clinically, fMRI is increasingly used for presurgical mapping of individuals(3). There is no doubt that the method itself is sufficiently robust and sensitive to be applied to individual subjects to map detailed organization patterns as well as subtle changes with interventions.

Functional MRI has been taken further. Voxel-wise patterns of activity within regions of activation in individuals were shown to delineate subtle variations in task or stimuli. This pattern-effect mapping, known as representational similarity analysis(4), has shown continued success and growth. Because each pattern is subject and even session-specific, it currently defies multi-subject averaging; however, approaches such as hyper-alignment(5) show promise even at this level of detail.

Over time, fMRI signal has been shown to be stable, repeatable, and sensitive enough to reveal induced differences in activity as an individual brain learns, adapts, and engages. Functional MRI can consistently delineate functional activation in individual brains – going so far as to be able to allow approximate reconstruction of the original stimuli, from activation patterns associated with movie viewing or sentence reading(6,7). All these approaches rely on within-individual contrasts, thus sidestepping the less tractable problem of variance across subjects.

For “central tendency” mapping, it was determined that combining data across subjects shows the generalizability from individuals to a population. The “central tendency” effects and derived time courses are more stable but inevitably minimize or remove more subtle effects that population subsets may reveal. These approaches are negatively impacted by variation in structure and function that may be unaccounted for or defy current best practices in spatial normalization and alignment.

Over the past three decades, since fMRI and structural MRI have been able to provide individualized information, the desire has been to go beyond central tendency mapping to reveal individual differences in activation, connectivity, and function. With “standard” clinical MRI, scans of the brain, lesions, tumors, vascular, or gross structural abnormalities have been straightforward for a trained radiologist to identify; however, psychiatric and most behavioral differences have brain correlates that are much too subtle for standard clinical MRI approaches. An effort has been made over at least the past two decades to pool and average functional and/or structural images together towards the creation of reproducible and clinically useful biomarkers. No one doubts that differences between individuals or truly homogeneous groups reside in the brain; however, whether they can be seen robustly or at all at the specific temporal and spatial niche offered by structural and functional MRI remains an open question. This question remains open because the brain is organized across a wide range of temporal and spatial scales and the causal physical mechanisms that lead to trait or state differences are not currently understood. At this stage, neuroscientists and clinicians are using fMRI to determine if any signatures related to behavioral or state differences can be robustly seen at all. It may well be that distinct brain differences across many scales can lead to similar trait differences or it may be that they reside at a spatial or temporal scale – or even magnitude – that is outside of what fMRI or MRI can capture. It remains to be fully determined.

The challenge of the Marek paper:

The recent paper by Marek et al. (8) has argued that behavioral phenotype variations associated with variations in cortical thickness, activation, and resting state connectivity, which they termed Brain-Wide Associations (BWAS) as measured with MRI, are reproducible only after thousands of individuals are considered. The authors of the paper suggest that the unfortunate reality is that the effect sizes are so small that reproducible studies require about two thousand subjects, and would benefit somewhat from further reduction in time series noise and multivariate analysis approaches. It is good news that we can get an effect, but for many invested in fMRI studies with this goal, this may be cause for despair and confusion. How is it that we can map individual brains so robustly, efficiently, and precisely, yet require so many subjects to derive any meaningful result when looking for differences in this readily mapped functional and structural information?

While single subjects can produce robust activation and connectivity maps, the differences in activation or structure as they relate to differences in traits across individuals are either so subtle and/or so variable that thousands of subjects are required to see emerging (i.e., “central tendency”) effects – and these may be just the most robust effects. Put another way, if the unwanted variability across subjects were vanishingly small, then the results of Marek et al would suggest that the BWAS – related differences in measured activation, structure, or connectivity would be about three orders of magnitude smaller than the main effect that is commonly seen in individual maps (1 subject required for an activation map vs 1000 subjects required for reliable difference). Given the much more readily observed changes observed while tracking individuals longitudinally as they change state, the small difference explanation seems highly unlikely. Therefore, the need for thousands of subjects is more likely explained predominantly by the unwanted and unaccounted for variance in trait-relevant or processing pipeline-related structural, activation, or connectivity patterns.

The problem or challenge, as it exists, is not primarily with the sensitivity or specificity of fMRI or structural MRI. Rather it likely resides in the uncharacterized and tremendously large variation in observed brain-behavior relationships across individuals. The underlying brain structure-function relationships, as measured with fMRI or MRI, that may be different for, say, a depressed individual may be numerous, subtle, and idiosyncratic. The study of BWAS is an attempt to determine the most common brain-based causes from a turbulent sea of possible causes across individuals. The Marek et al study has shown that this challenge is more profound than most of us may have imagined – at least on the temporal and spatial scale that we have access to through our tools. It may also be true that those effects that we do eventually see after studying groups of thousands of subjects are but a small fraction of the dispersed effects unique to each individual – and that those that we are able to observe are not necessarily the most influential to the trait observed, as they are simply, by definition, the most commonly observed.

Marek et al have done a service to the field by pointing out concerns for a type of fMRI study that has wide-spread interest but so far, relatively few reported studies. Their work may be interpreted to suggest that, given the formidable number of subjects needed, BWAS-style studies are not a practically tenable use of fMRI. This conclusion should be tempered by an alternate view. Large databases of deeply characterized subjects may be queried in many different ways, potentially increasing their utility into the future. The authors also point out that the effect sizes shown are at least comparable to large database gene-wide association studies (GWAS). Improvement is still likely. It’s important to make sure the field of fMRI has done due diligence in being certain that it has minimized the irrelevant variance across subjects as it is manifest through our techniques for determining function and in our techniques for pooling multi-subject data.

The unknown unknowns in feature space and variance:

Is there something we are missing – hidden sources of irrelevant variance, inaccurate choices in feature space, or mischaracterization and therefore mis-grouping of behavioral phenotypes – that are suppressing the more informative features and thus reducing effect size? In the tables below, the “unknown unknowns” in understanding BWAS power and possible approaches to address them are described. Table 1 lists potential unknown confounders that may be reducing BWAS power. Within this table are some considerations on how to understand and address these unknowns. Much more could be said for any of these, and indeed work is already taking place worldwide on all these topics. Table 2 lists other considerations that are not necessarily unknowns but areas of active research that should also be considered when designing BWAS or perhaps any fMRI study.

Table 1: Potential Confounders that are not fully understood nor addressed:

Table 2: Other Avenues to Improvement

In summary, Marek et al provide a sobering snapshot of the state of BWAS using MRI and fMRI. The study of brain wide associations(13), like the study of gene-wide associations(14), does have promise however has barely just begun work towards objectively identifying and extracting the most meaningful features and identifying and removing the confounding variance from the signal – in time and space. We are at an early stage in this promising research. The Marek at al study has performed a profound service by clarifying, quantifying, and highlighting the challenge.

The study of individuals and how they change with time and natural disease progression, or interventions will continue. In fact, large population longitudinal studies in which each participant is directly compared with themselves at an earlier time, and then compared across the cohort will likely have a high yield of deep insights into brain differences and similarities(15). These studies are difficult but are worth pursuing as they avoid many of the potential pitfalls of BWAS, related to between-subject variability, as described in Marek et al.

Individual or small N fMRI will continue as insights into healthy brain organization and function are still being derived at an increasingly rapid rate as the field develops methods to extract more subtle information from the data. Individual fMRI for presurgical mapping, real time feedback, and neuromodulation guidance also continues with extremely promising progress.

Evolving fMRI from central tendency mapping to identifying differences in individuals has proven to be deeply challenging. As the field continues working to address this challenge, it will likely uncover unique sources of variance residing in every step of acquisition and analysis; as well as yet-uncovered structure in idiosyncratic brain-behavior relationships. The fMRI signal is intrinsically strong, reproducible, and robust, as has been shown over the past 30 years. To use it to compare individuals, we need to delve much more deeply into how individuals and their brains vary so we can identify and minimize the still unknown nuisance variance and maximally use the still unknown informative variance. Once we can do this, the effect sizes and replicability promise to reach a useful level with fewer required subjects. In the process of this work, new principles of brain organization may likely be derived. Perhaps before the field rushes ahead to collect more two-thousand subject cohorts, it should explore, understand, and minimize the unknown unknowns in the feature space and variance among individuals.

References

1. Newbold DJ, Laumann TO, Hoyt CR, Hampton JM, Montez DF, Raut RV, et al. Plasticity and Spontaneous Activity Pulses in Disused Human Brain Circuits. Neuron. 2020;1–10.

2. Ramot M, Kimmich S, Gonzalez-Castillo J, Roopchansingh V, Popal H, White E, et al. Direct modulation of aberrant brain network connectivity through real-time NeuroFeedback. Elife. 2017;6:e28974.

3. Silva MA, See AP, Essayed WI, Golby AJ, Tie Y. Challenges and techniques for presurgical brain mapping with functional MRI. NeuroImage Clin. 2018 Jan 1;17:794–803.

4. Kriegeskorte N, Mur M, Bandettini P. Representational similarity analysis – connecting the branches of systems neuroscience. Front Syst Neurosci. 2008 Nov;2(NOV):2007–8.

5. Haxby JV, Guntupalli JS, Nastase SA, Feilong M. Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies. Elife. 2020;9:e56601.

6. Pereira F, Lou B, Pritchett B, Ritter S, Gershman SJ, Kanwisher N, et al. Toward a universal decoder of linguistic meaning from brain activation. Nat Commun. 2018 Mar 6;9(1):963.

7. Nishimoto S, Vu AT, Naselaris T, Benjamini Y, Yu B, Gallant JL. Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies. Curr Biol. 2011 Oct 11;21(19):1641–6.

8. Marek S, Tervo-Clemmens B, Calabro FJ, Montez DF, Kay BP, Hatoum AS, et al. Reproducible brain-wide association studies require thousands of individuals. Nature. 2022 Mar;603(7902):654–60.

9. Finn ES, Glerean E, Khojandi AY, Nielson D, Molfese PJ, Handwerker DA, et al. Idiosynchrony: From shared responses to individual differences during naturalistic neuroimaging. NeuroImage. 2020 Jul;215:116828–116828.

10. Gonzalez-Castillo J, Kam JWY, Hoy CW, Bandettini PA. How to Interpret Resting-State fMRI: Ask Your Participants. J Neurosci. 2021 Feb 10;41(6):1130–41.

11. Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R. Intersubject Synchronization of Cortical Activity during Natural Vision. Science. 2004 Mar;303(5664):1634–40.

12. Finn ES. Is it time to put rest to rest? Trends Cogn Sci. 2021 Dec 1;25(12):1021–32.

13. Sui J, Jiang R, Bustillo J, Calhoun V. Neuroimaging-based Individualized Prediction of Cognition and Behavior for Mental Disorders and Health: Methods and Promises. Biol Psychiatry. 2020 Dec 1;88(11):818–28.

14. Visscher PM, Wray NR, Zhang Q, Sklar P, McCarthy MI, Brown MA, et al. 10 Years of GWAS Discovery: Biology, Function, and Translation. Am J Hum Genet. 2017 Jul 6;101(1):5–22.

15. Douaud G, Lee S, Alfaro-Almagro F, Arthofer C, Wang C, McCarthy P, et al. SARS-CoV-2 is associated with changes in brain structure in UK Biobank. Nature. 2022 Apr;604(7907):697–707.

The Unique Relationship Between fMRI and MRI Scanner Vendors

Wed, 16 Dec 2020 04:51:00 GMT

One defining and often overlooked aspect of fMRI as a field is that it is has been riding on the back of and directly benefitting from the massive clinical MRI industry. Even though fMRI has not yet hit the clinical mainstream – as there are no widely used standard clinical practices that include fMRI, it has reaped many benefits from the clinical impact of “standard” MRI. Just about every clinical scanner can be used for fMRI with minimal modification, as most vendors have rudimentary fMRI packages that are sold. Just imagine if MRI was only useful for fMRI – how much slower fMRI methods and applications would have developed and how much more expensive and less advanced MRI scanners would be. Without a thriving clinical MRI market only a few centers would be able to afford scanners that would likely be primitive compared to the technology that exists today.

Looking back almost 40 years to the early 1980’s when the first MRI scanners were being sold, we see that the clinical impact of MRI was almost immediate and massive. For the first time, soft tissue was able to be imaged non invasively with unprecedented resolution, providing immediate clinical applications for localization of brain and body lesions. Commercial scanners, typically 1.5T, were rapidly installed in hospitals worldwide. By the late 1980’s the clinical market for MRI scanners was booming. The clinical applications continued to grow. MRI was used to image not only brain, but just about every other part of the body. As long as it had water it was able to be imaged. Sequences were developed to capture the heart in motion and even characterize trabecular bone structure. Tendons, muscles, and lungs were imaged. Importantly, the information provided by MRI was highly valuable, non-invasively obtained, and unique relative to other approaches. The clinical niches were increasing.

In 1991, fMRI came along. Two of the first three results were produced on commercially sold clinical scanners that were tricked out to allow for high speed imaging. In the case of Massachusetts General Hospital, they used a “retrofitted” (I love that word) resonant gradient system sold by ANMR. The system at MCW had a home built, sewer pipe, epoxy, and wire local head gradient coil, that, because of its extremely low inductance, could perform echo planar imaging at relatively high resolution. Only The University Minnesota’s scanner, a 4 Tesla research device, was non-commercial.

Since 1991, advancement of fMRI was initially gradual as commercial availability of EPI, almost essential for fMRI, was limited. Finally, in 1996, EPI was included on commercial scanners and to the best that I can recall, mostly marketed as a method for tracking bolus injections of gadolinium for cerebral blood volume/perfusion assessment and for freezing cardiac motion. The first demonstration for EPI that I recall was shown in 1989 by Robert Weisskoff from MGH on the their GE / retrofitted ANMR system – capturing a spectacular movie of a beating heart. EPI was great for moving organs like the heart or rapidly changing contrast like a bolus injection of Gadolinium. EPI as a pulse sequence for imaging the heart was eventually superseded by fast multi-shot, gated, “cine” methods that were more effective and higher resolution. However, thanks to EPI being sold with commercial scanners, functional MRI began to propagate more rapidly after 1996. Researchers could now negotiate for time on their hospital scanners to collect pilot fMRI data. Eventually, as research funding for fMRI grew, more centers were able to afford research-dedicated fMRI scanners. That said, the quantity of scanners today that are sold for the purposes of fMRI are such a small fraction of the clinical market (I might venture 1000 (fMRI scanners) /50,000 (clinical scanners) or 2%), that the buyers’ needs as they relate to fMRI typically don’t influence vendor product development in any meaningful way. Vendors can’t devote a large fraction of their R & D time to a research market. Almost all benefit that the field of fMRI receives from advances in what vendors provide is incidental as it likely relates to the improvement of more clinically relevant techniques. Recent examples include high field, multi-channel coil arrays, and parallel reconstruction – all beneficial to clinical MRI but also highly valued by the fMRI community. This also applies to 3T scanners back in the early 2000’s. Relative to 1.5 T, 3T provided more signal to noise and in some cases better contrast (in particular susceptibility contrast) for structural images – and therefore helped clinical applications, so that market grew, to the benefit of fMRI. Some may argue that the perceived potential of fMRI back in the early 2000’s had some influence on getting the 3T product lines going (better BOLD contrast), and perhaps it did, however, today 20 years later, even though I’m more hopeful than ever about robust daily clinical applications of fMRI, this potential still remains just over the horizon, so the prospect of a golden clinical fMRI market has lost some of its luster to vendors.

This is the current state of fMRI: benefitting from the development of clinically impactful products such as higher field strength, more sophisticated pulse sequences, recon, analysis, shimming, and RF coils, however not strongly driving the production pipelines of vendors in a meaningful way. Because fMRI is not yet a robust and widely used clinical tool, vendors are understandably reluctant to redirect their resources to further develop fMRI platforms. This can be frustrating as fMRI would tremendously benefit from increased vendor development and product dissemination.

There can be a healthy debate as to how much the fMRI research, development, and application community has influenced vendor products. While there may have been some influence, I believe it to be minimal – less than what I think that the clinical long term potential of fMRI may justify. That said, there is nothing bad or good about vendor decisions on what products they produce and support. Especially in today’s large yet highly competitive clinical market, they have to think slightly shorter term and highly strategically. We, as the fMRI community, need to up our game to incentivize either the big scanner vendors or smaller third party vendors to help catalyze its clinical implementation.

For instance, if vendors saw a large emerging market in fMRI, they would likely create a more robust fMRI-tailored platform – including a suite of fMRI pulse sequences sensitive to perfusion, blood volume changes, and of course BOLD – with multi-echo EPI being standard. They would also have a sophisticated yet clinically robust processing pipeline to make sense of resting state and activation data in ways that are easily interpretable and usable by clinicians. One could also imaging a package of promising fMRI-based “biomarkers” for a clinician or AI algorithm to incorporate in research and basic practice.

Regarding pulse sequence development, the current situation is that large academic and/or hospital centers have perhaps one or more physicist who knows the vendor pulse sequence programming language. They program and test various pulse sequences and present their data at meetings, where ideas catch on – or not. Those that show promise are eventually patented and vendors employ their programmers to incorporate these sequences, with the appropriate safety checks, into their scanner platforms. Most sequences don’t make it this far. Many are considered as, using Siemens’ terminology, “works in progress” or WIPS. These are only distributed to those centers who sign a research agreement and have the appropriate team of people to incorporate the sequence at the research scanner in their center. This approach, while effective to some degree to share sequences in a limited and focused manner, is not optimal from a pulse sequence development, dissemination and testing standpoint. It’s not what it could be. One could imagine alternatively, that vendors could create a higher level pulse sequence development platform that allows rapid iteration for creation and testing of sequences, with all checks in place so that sharing and testing is less risky. This type of environment would not only benefit standard MRI pulse sequences but would catalyze the development and dissemination of fMRI pulse sequences. There are so many interesting potential pulse sequences for fMRI – involving embedded functional contrasts, real time adaptability, and methods for noise mitigation that remain unrealized due to the bottleneck in the iteration of pulse sequence creation, testing, dissemination, application, and finally the big step of productization, not to mention FDA approval.

Functional MRI – specific hardware is also another area where growth is possible. It’s clear that local gradient coils would be a huge benefit to both DTI and fMRI, as the smaller coils can achieve higher gradients, switch faster, don’t induce as high of the nerve stimulating dB/dt, don’t heat up as easily, produce less eddy currents, and are generally more stable than whole body gradients. Because of space and patient positioning restrictions however, they would have limited day to day clinical applicability and currently have no clear path to become a robust vendor product. Another aspect of fMRI that would stand to benefit are the tools for subject interfacing – stimulus devices, head restraints, subject feedback, physiologic monitoring, eye tracking, EEG, etc.. Currently, a decked out subject interface suite is cobbled together from a variety of products and is awkward and time consuming to set up and use – at best. I can imagine the vendors creating a fully capable fMRI interface suite, that has all these tools engineered in a highly integrated manner, increasing standardization and ease of all our studies and catalyzing the propagation of fundamentally important physiological monitoring, subject interface, and multimodal integration.

Along a similar avenue, I can imagine so many clinicians who want to try fMRI but don’t have the necessary team of people to handle the entire experiment/processing pipeline for practical use. One could imagine if a clinical fMRI experimental platform and analysis suite were created and optimized through the vendors. Clinicians could test out various fMRI approaches to determine their efficacy and, importantly, work out the myriad of practical kinks unique to a clinical setting that researchers don’t have to typically deal with. Such a platform would almost certainly catalyze clinical development and implementation of fMRI.

Lastly, a major current trend is the collection and analysis of data collected across multiple scanner platforms: different vendors and even slightly different protocols. So far the most useful large data sets have been collected on a single scanner or on a small group of identical scanners or even with a single subject being repeatedly scanned on one scanner over many months. Variance across scanners and protocols appears to wreak havoc with the statistics and reproducibility, especially when looking for small effect sizes. Each vendor has proprietary reconstruction algorithms and typically only outputs the images rather than the raw unreconstructed data. Each scan setup varies as the patient cushioning, motion constraints, shimming procedures, RF coil configurations, and auto prescan (for determining the optimal flip angle) all vary not only across vendors but also potentially from subject to subject. To even start alleviating these problems it is important to have a cross vendor reconstruction platform that takes in the raw data and reconstructs the images in an identical, standardized manner. First steps of this approach have been taken in the emergence of the “ Gadgetron ” as well as an ISMRM standard raw data format. There have emerged some promising third party approaches to scanner independent image recon, including one via a Swiss company called Skope . One concern with third party recon is that the main vendors have put in at least 30 years of work perfecting and tweaking their pulse-sequence specific recon, and, understandably, the code is strictly proprietary – although most of the key principles behind the recon strategies are published. Third party recon engines have had to play catchup, and perhaps in the open science environment, have been on a development trajectory that is faster than that of industry. If they have not already done so, they will likely surpass the standard vendor recon in image quality and sophistication. So far, with structural imaging – but not EPI, open source recon software is likely ahead of that of vendors. While writing this I was reminded that parallel imaging, compressed sensing, model based recon, and deep learning recon were all open access code before many of them were used by industry. These need to be adopted to EPI recon to be useful for fMRI.

A primary reason why the entire field of fMRI is not all doing recon offline is because most fMRI centers don’t have the setup or even the expertise to easily port raw data to free-standing recon engines. If this very achievable technology were disseminated more completely across fMRI centers – and if it were simply easier to quickly take raw data of the scanner – the field of fMRI would make an important advance as images would likely become more artifact free, more stable, and more uniform across scanners. This platform would also be much more nimble – able to embrace the latest advances in image recon and artifact mitigation.

My group, specifically Vinai Roopchansingh, and others at the NIH and elsewhere, have worked with Gadgetron, have also been working on approaches to independent image reconstruction: including scripts for converting raw data to the ismrmrd format, an open access Jupyter notebook script running python for recon of EPI data.

Secondly, vendors could work together – in a limited capacity – to create standard research protocols that are as identical as possible – specifically constructed for sharing and pooling of data across vendors. Third, to alleviate the problem of so much variability across vendors and subjects in terms of time series instability, there should be a standard in image and time series quality metrics reporting . I can imagine such metrics as tSNR, image SNR, ghosting, outliers, signal dropout, and image contrast to be reported for starters. This would take us a long way towards immediately recognizing and mitigating deviations in time series quality and thus producing better results from pooled data sets. This metric reporting could be carried out by each vendor – tagging these on a quality metric file at the end of each time series. Vendors would likely have to work together to establish these. Currently programs that generate metrics exist (i.e. Oscar Esteban’s MRIQC ), however there remains insufficient incentives and coordination to adopt them on a larger scale.

I am currently part of the OHBM standards and best practices committee, and we are discussing starting a push to more formally advise all fMRI users to report or have tagged to each time series, an agreed upon set of image quality metrics.

In general the relationship between fMRI and the big vendors currently is a bit of a Catch-22 situation. All of the above mentioned features would catalyze clinical applications of fMRI, however for vendors to take note and devote the necessary resources to these, it seems that there needs to be clinical applications in place, or at least a near certainty that a clinical market would emerge from these efforts in the near term, which cannot be guaranteed. How can vendors be incentivized to take the longer term and slightly more risky approach here – or if not this, cater slightly more closely to a smaller market? Many of these advances to help catalyze potential clinical fMRI don’t require an inordinate amount of investment, so could be initiated by either public or private grants. On the clinical side, clinicians and hospital managers could speak up to vendors on the need for testing and developing fMRI by having a rudimentary but usable pipeline. Some of these goals are simply achievable if vendors open up to work together in a limited manner on cross-scanner harmonization and standardization. This simply requires a clear and unified message from the researchers of such a need and how it may be achieved while maintaining the proprietary status of most vendor systems. FMRI is indeed an entirely different beast than structural MRI – requiring a higher level of subject and researcher/clinician engagement, on-the-fly, robust, yet flexible time series analysis, and rapid collapsing of multidimensional data that can be easily and accurately assessed and digested by a technologist and clinician – definitely not an easy task.

Over the years, smaller third party vendors have attempted to cater to the smaller fMRI research market, with mixed success. Companies have built RF coils, subject interface devices, and image analysis suites. There continues to be opportunities here as there is much more that could be done, however the delivery of products that bridge the gap between what fMRI is and what it could be from a technological standpoint requires that the big vendors “open the hood” of their scanners to some degree, allowing increased access to proprietary engineering and signal processing information. Again, since the clinical market is small, there is little, on first glance, to gain and thus no real incentive for the vendors to do this. I think that the solution is to lead the vendors to realize that there is something to gain – in the long run – if they work to nurture, through more open access platforms or modules within their proprietary platforms, the tremendous untapped intellectual resources of highly skilled and diverse fMRI community. At a very small and limited scale this already exists. I think that a key variable in many fMRI scanner purchase decisions has been the ecosystem of sharing research pulse sequences -which some vendors do better than others. This creates a virtuous circle as pulse programmers want to maximize their impact and leverage collaborations through ease of sharing – to the benefit of all users – and ultimately to the benefit of the field which will result in increasing the probability of fMRI being a clinically robust and useful technique, thus opening up a large market. Streamlining the platform for pulse sequence development and sharing, allowing raw data to be easily ported from the scanner, sharing the necessary information for the highest quality EPI image reconstruction, and working more effectively with third party vendors and with researchers with no interest in starting a business would be a great first step towards catalyzing the clinical impact of fMRI.

Overall, the relationship between fMRI and scanner vendors remains quite positive and still dynamic, with fMRI slowly getting more leverage as the research market grows, and as clinicians start taking notice of the growing number of promising fMRI results. I have had outstanding interactions and conversations with vendors over the past 30 years about what I, as an fMRI developer and researcher, would really like. They always listen and sometimes improvements to fMRI research sequences and platforms happen. Other times, they don’t. We are all definitely going in the right direction. I like to say that fMRI is one amazing clinical application away from having vendors step in and catalyze the field. To create that amazing clinical application will likely require approaches to better leverage the intellectual resources and creativity of the fMRI community – providing better tools for them to collectively find solutions to the daunting challenge of integrating fMRI into clinical practice as well as of course, more efficiently searching for that amazing clinical application. We are working in that direction and there are many reasons to be hopeful.

ISMRM Gold Medal 2020

Tue, 10 Nov 2020 19:56:00 GMT

This year I was among the four ISMRM Gold Medal recipients for 2020 . These were Ken Kwong, Robert Turner, and Kaori Togashi. It was a deep honor to win this along side my two friends: Ken Kwong, who arguably was the first to demonstrate fMRI in humans, and Bob Turner, who has been a constant pioneer in all aspects of fast imaging since even before my time and then fMRI since the beginning. I have always looked up to and respected past ISMRM gold medal winners, and am very deeply humbled to be among this highly esteemed company. I’m also grateful to Hanbing Lu for nominating me, as well as to those who wrote support letters for me. It’s also an acknowledgement by ISMRM of the importance of fMRI as a field, which while so successful in brain mapping for research purposes, has not yet fully entered into clinical utility.

While the event was virtual, there was no actual physical presentation of the Gold Medal to the recipients, however, a couple of weeks ago I came back to my office to pick up a few things after vacating it on March 16 due to Covid. At the base of the door I found a Fedex box, which I was deeply delighted to find this pleasant surprise inside:

Here is what I said for my acceptance speech, which I feel is important to share.

“I would like to thank ISMRM for this incredible honor. Throughout my career, and especially at the start, I enjoyed quite a bit of serendipity. Back in 1989, when I was starting graduate school at the Medical College of Wisconsin, I was extremely lucky to be at just the right place at the right time and wouldn’t be here accepting this without the help of my mentors, colleagues, and lab over the years.

Before starting graduate school, before fMRI, I had absolutely no idea what was ahead of me, but I did know one thing: that I wanted to image brain function with MRI…somehow. My parents instilled a sense of curiosity, and dinnertime conversations with my Dad sparked my fascination with the brain.

Jim Hyde, my advisor, set up the Biophysics Dept at MCW to excel in MRI hardware and basic research. His confidence and bold style were infused into the center’s culture.

Scott Hinks my co-advisor, helped me during a critical and uncertain time in my graduate career, and I’m grateful for his taking me on. His clear thinking set an inspiringly high standard.

Eric Wong, my dear friend, colleague and mentor, was a fellow graduate student with me at the time, and it’s to him that I have my most profound gratitude. He designed and built the local head gradient and RF coils and wrote from scratch the EPI pulse sequence and reconstruction necessary to perform our first fMRI experiments. He taught me almost everything I know about MRI, but more importantly he trained me well through his example. He constantly came up with great ideas, and one of his most common phrases was “let’s try it.” This phrase set the optimistic and proactive approach I have taken to this day. In September of 1991, one month after Ken Kowng’s jaw-dropping results shown by Tom Brady at the then called SMR meeting in San Francisco, we collected our first successful fMRI data and from then on were well positioned to help push the field. Without Eric’s work, MCW would have had no fMRI, and my career would have looked very different.

The late Andre Jesmanowicz, a professor at MCW, helped in a big way through his fundamental contribution to our paper introducing correlation analysis of fMRI time series.

My post doc experience at the Mass General Hospital lasted less than 2 years but felt like 10, in a good way, as I learned so much from the great people there. That place just hums with intellectual energy.

One of my best decisions was to accept an offer to join Leslie Ungerleider’s Laboratory of Brain and Cognition as well as to create a joint NINDS/NIMH functional MRI facility. It’s here that I have been provided with so much support. My colleague at the NIH, Alan Korestky, has been source of insight, and is perhaps my favorite NIH person to talk to. In general NIH is just teeming with great people in both MRI and neuroscience. The environment is perfect.

My neuroscientist and clinician collaborators have been essential for disseminating fMRI as they embraced new methods and findings.

I have been lucky to have an outstanding multidisciplinary team. Many have gone on to be quite successful, including Rasmus Birn, Jerzy Bodurka, Natalia Petridou, Kevin Murphy, Prantik Kundu, Niko Kriegeskorte, Carlton Chu, Emily Finn, and Renzo Huber.

My current team of staff scientists have shown outstanding commitment over the years and especially during these difficult times. These include Javier Gonzalez-Castillo, Dan Handwerker, Sean Marrett, Pete Molfese, Vinai Roopchansingh, Linqing Li, Andy Derbyshire, Francisco Pereira, and Adam Thomas.

The worldwide community of friends I have gained through this field is special to me, and a reminder that science, on so many levels, is a positive force for cohesion across countries and cultures.

Lastly, I am also so very lucky and thankful for my brilliant, adventurous, and supportive wife, Patricia, and my three precocious boys who challenge me every day.

An approach to research that has always worked well at least for me has been to be completely open with sharing ideas, not to care about credit, and perhaps most importantly, to think broadly, deeply, and simply and then proceed optimistically and boldly. To just try it. There are many possible reasons for an idea not to work, but in most cases it’s worthwhile to test it anyway.

Someday, we will figure out the brain, and I believe that fMRI will help us get there. It’s a bright future. Thank you.”

My talk on layer-fMRI in the Brain Space Initiative Speaker Series.

Sat, 07 Nov 2020 14:56:00 GMT

The BrainSpace Initiative is an outreach program that allows researchers to present their work, currently on non-invasive technique. It is also a meeting space to discuss papers and issues. I was invited to both be a member of the advisory committee and to give a talk. I decided to present a talk on all the work on layer fMRI has come out of my lab over the past 4 years. Here it is:

My brain space initiative talk

Layer fMRI, requiring high field, advanced pulse sequences, and sophisticated processing methods, has emerged in the last decade. The rate of layer fMRI papers published has grown sharply as the delineation of mesoscopic scale functional organization has shown success in providing insight into human brain processing. Layer fMRI promises to move beyond being able to simply identify where and when activation is taking place as inferences made from the activation depth in the cortex will provide detailed directional feedforward and feedback related activity. This new knowledge promises to bridge invasive measures and those typically carried out on humans. In this talk, I will describe the challenges in achieving laminar functional specificity as well as possible approaches to data analysis for both activation studies and resting state connectivity. I will highlight our work demonstrating task-related laminar modulation of primary sensory and motor systems as well as layer-specific activation in dorsal lateral prefrontal cortex with a working memory task. Lastly, I will present recent work demonstrating cortical hierarchy in visual cortex using resting state connectivity laminar profiles.

Revision: Defending Brain Mapping, fMRI, and Discovery Science

Thu, 18 Jun 2020 00:21:00 GMT

We submitted our rebuttal to Brain and received a prompt reply from the Editor-In-Chief, Dr. Kullman himself, offering us an opportunity to revise – with the main criticism that our letter contained unfounded insinuations and allegations. We tried to interpret his message as best we could and respond accordingly. To most readers it was pretty clear what he wrote and the message he intended to convey. Nevertheless, in our revision, we stayed much closer to the words of editorial itself. We also tried to bolster our response with tighter arguments and a few salient references.

Essentially our message was:

Defending Brain Mapping, fMRI, and Discovery Science: A Rebuttal to Editorial ( Brain, Volume 143, Issue 4, April 2020, Page 1045) Revision 1

Vince Calhoun ¹ and Peter Bandettini ²

¹ Tri-institutional Center for Translational Research in Neuroimaging and Data Science: Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, Georgia, USA.

² National Institute of Mental Health

In his editorial in Brain (Volume 143, Issue 4, April 2020, Page 1045), Dr. Dimitri Kullmann presents an emotive and uninformed set of criticisms about research where “… the route to clinical application or to improved understanding of disease mechanisms is very difficult to infer…” The editorial starts with a criticism about a small number of submissions, then it quickly pivots to broadly criticize discovery science, brain mapping, and the entire fMRI field: “ Such manuscripts disproportionately report on functional MRI in groups of patients without a discernible hypothesis. Showing that activation patterns or functional connectivity motifs differ significantly is, on its own, insufficient justification to occupy space in Brain.”

The description of activity patterns and their differences between populations and even individuals is fundamental in characterizing and understanding how the healthy brain is organized, how it changes, and how it varies with disease – often leading directly to advances in clinical diagnosis and treatment (Matthews et al. , 2006). The first such demonstrations were over 20 years ago with presurgical mapping of individual patients (Silva et al. , 2018). Functional MRI is perfectly capable of obtaining results in individual subjects(Dubois and Adolphs, 2016). These maps are windows into the systems level organization of the brain that inform hypotheses that are generated within this specific spatial and temporal scale. The brain is clearly organized across a wide range of temporal and spatial scales – with no one scale emerging yet as the “most” informative(Lewis et al. , 2015).

Dr. Kullmann implies in the above statement that the only hypotheses-driven studies are legitimate. This view dismisses out of hand the value of discovery science, which casts a wide and effective net in gathering and making sense of large amounts of data that are being collected and pooled(Poldrack et al. , 2013). In this age of large neuroscience data repositories, discovery science research can be deeply informative (Miller et al. , 2016). Both hypotheses-driven and discovery science have importance and significance.

Finally, in his opening salvo, he sets up his attack on fMRI: “ Given that functional MRI is ∼ 30 years old and continues to divert many talented young researchers from careers in other fields of translational neuroscience it is worth reiterating two of the most troubling limitations of the method..” The author, who is also the editor-in-chief of Brain, sees fMRI research as problematic not only because a disproportionally large number of studies from it are reporting group differences and are not hypothesis-driven, but also because it has been diverting all the good young talent from more promising approaches. The petty lament about diverted young talent reveals a degree of cynicism of the natural and fair process by which the best science reveals itself and attracts good people. It implies that young scientists are somehow being misled to waste their brain power on fMRI rather than naturally gravitating towards the best science.

His “most troubling limitations of the method” are two hackneyed criticisms of fMRI that suggest for the past 30 years, he has not been following the fMRI literature published worldwide and in his own journal. Kullman’s two primary criticisms about fMRI are: “ First, the fundamental relationship between the blood oxygenation level-dependent (BOLD) signal and neuronal computations remains a complete mystery.” and “ Second, effect sizes are quasi-impossible to infer, leading to an anomaly in science where statistical significance remains the only metric reported.”

Both of these criticisms, to the degree that they are valid, can apply to all neuroscience methods to various degrees. The first criticism is partially true, as the relationship between ANY measure of neuronal firing or related physiology and neuronal computations IS still pretty much a complete mystery. While theoretical neuroscience is making rapid progress, we still do not know what a neuronal computation would look like no matter what measurement we observe. However, the relationship between neuronal activity and fMRI signal changes is far from a complete mystery, rather it has been extensively studied (Logothetis, 2003; Ma et al. , 2016). While this relationship is imperfectly understood, literally hundreds of papers have established the relationship between localized hemodynamic changes and neuronal activity, measured using a multitude of other modalities. Nearly all cross-modal verification has provided strong confirmation that where and when neuronal activity changes, hemodynamic changes occur – in proportion to the degree of neuronal activity.

While inferences about brain connectivity from measures of temporal correlation have been supported by electrophysiologic measures, they have inherent assumptions about the degree to which synchronized neuronal activity is driving the fMRI-based connectivity as well as a degree of uncertainty about what is meant by “connectivity.” It has never been implied that functional connectivity gives an unbiased estimation of information transfer across regions. Furthermore, this issue has little to do with fMRI. Functional connectivity – as implied by temporal co-variance – is a commonly used metric in all neurophysiology studies. Functional MRI – based measures of “connectivity” have been demonstrated to clearly and consistently show correspondence with differences in behavior and traits of populations and individuals(Finn et al. , 2015; Finn et al. , 2018; Finn et al. , 2020). These data, while not fully understood, and thus not yet perfectly interpretable, are beginning to inform systems-level network models with increasing levels of sophistication(Bertolero and Bassett, 2020).

Certainly, issues related to spatially and temporally confounding effects of larger vascular and other factors continue to be addressed. Sound experimental design, analysis, and interpretation can take these factors into account, allowing useful and meaningful information on functional organization, connectivity, and dynamics to be derived. Acquisition and processing strategies involving functional contrast manipulations and normalization approaches have effectively mitigated these vascular confounds (Menon, 2012). Most of these approaches have been known for over 20 years, yet until recently we didn’t have hardware that would enable us to use these methods broadly and robustly.

In contrast to what is claimed in the editorial, high field allows substantial reduction of large blood vessel and “draining vein” effects thanks to higher sensitivity at high field enabling scientists to use contrast manipulations more exclusively sensitive to small vessel and capillary effects(Polimeni and Uludag, 2018). Hundreds of ultra-high resolution fMRI studies are revealing cortical depth dependent activation that shows promise in informing feedback vs. feedforward connections(Huber et al. , 2017; Huber et al. , 2018; Finn et al. , 2019; Huber et al. , 2020).

Regarding the second criticism involving effect sizes. In stark contrast to the criticism in Dr. Kullmann’s editorial, effect sizes in fMRI are quite straight-forward to compute using standard approaches and are very often reported. In fact, you can estimate prediction accuracy relative to the noise ceiling. What is challenging is that there are many different fMRI-related variables that could be utilized. One might compare voxels, regions, patterns of activation, connectivity measures, or dynamics using an array of functional contrasts including blood flow, oxygenation, or blood volume. In fact, you can fit models under one set of conditions and test them under another set of conditions if you want to look at generalization. Thus, there are many different types of effects, depending on what is of interest. Rather than a weakness, this is a powerful strength of fMRI in that it is so rich and multi-dimensional.

The challenge of properly characterizing and modeling the meaningful signal as well as the noise is an ongoing area of research that is, shared by virtually every other brain assessment technique. In fMRI, the challenge is particularly acute because of the wealth and complexity of potential neuronal and physiological information provided. Clinical research in neuroscience generally suffers most from limitations of statistical analysis and predictive modeling because of the limited size of the available clinical data sets and the enormous individual variability in patients and healthy subjects. Again, this is a limitation for all measures, including fMRI. Singling out these issues as if they were specific to fMRI is indicative of a narrow and biased perspective. Dr. Kullmann is effectively stating that indeed fMRI is different from all the rest – a particularly efficient generator of a disproportionately high fraction of poor and useless studies. This perspective is cynical and wrong and ignores that ALL modalities have their limits and associated bad science, ALL modalities have their range of questions that they can appropriately ask.

Dr. Kullmann’s editorial oddly backpedals near the end. He does admit that: “This is not to dismiss the potential importance of the method when used with care and with a priori hypotheses, and in rare cases functional MRI has found a clinical role. One such application is in diagnosing consciousness in patients with cognitive-motor dissociation.” He then goes on to praise one researcher, Dr. Adrian Owen, who has pioneered fMRI use in clinical settings with “locked in” patients. The work he refers to in this article and the work of Dr. Owen are both outstanding, however, the perspective verbalized by Dr. Kullmann here is breathtaking as there are literally thousands of similar quality papers and hundreds of similarly accomplished and pioneering researchers in fMRI.

In summary, we argue that location and timing of brain activity on the scales that fMRI allows is useful for both understanding the brain and aiding clinical practice. One just has to take a more in-depth view of the literature and growth of fMRI over the past 30 years to appreciate the impact it has had. His implication that most fMRI users are misguided appears to dismiss the flawed yet powerful process of peer review in deciding in the long run what the most fruitful research methods are. His specific criticisms of fMRI are incorrect as they bring up legitimate challenges but completely fail to appreciate how the field has dealt – and continues to effectively deal with them. These two criticisms also fail to acknowledge that limits in interpreting any measurements are common to all other brain assessment techniques – imaging or otherwise. Lastly, his highlighting of a single researcher and study in this issue of Brain is myopic as he appears to imply that these are the extreme exceptions – inferred from his earlier statements – rather than simply examples of a high fraction of outstanding fMRI papers. He mentions the value of hypothesis driven studies without appreciating the growing literature of discovery science studies.

Functional MRI is a tool and not a catalyst for categorically mediocre science. How it is used is determined by the skill of the researcher. The literature is filled with examples of how fMRI has been used with inspiring skill and insight to penetrate fundamental questions of brain organization and reveal subtle, meaningful, and actionable differences between clinical populations and individuals. Functional MRI is advancing in sophistication at a very rapid rate, allowing us to better ask fundamental questions about the brain, more deeply interpret its data, as well as to advance its clinical utility. Any argument that an entire modality should be categorically dismissed in any manner is troubling and should in principle be strongly rebuffed.

Bertolero MA, Bassett DS. On the Nature of Explanations Offered by Network Science: A Perspective From and for Practicing Neuroscientists. Top Cogn Sci 2020.

Dubois J, Adolphs R. Building a Science of Individual Differences from fMRI. Trends Cogn Sci 2016; 20(6): 425-43.

Finn ES, Corlett PR, Chen G, Bandettini PA, Constable RT. Trait paranoia shapes inter-subject synchrony in brain activity during an ambiguous social narrative. Nat Commun 2018; 9(1): 2043.

Finn ES, Glerean E, Khojandi AY, Nielson D, Molfese PJ, Handwerker DA , et al. Idiosynchrony: From shared responses to individual differences during naturalistic neuroimaging. NeuroImage 2020; 215: 116828.

Finn ES, Huber L, Jangraw DC, Molfese PJ, Bandettini PA. Layer-dependent activity in human prefrontal cortex during working memory. Nat Neurosci 2019; 22(10): 1687-95.

Finn ES, Shen X, Scheinost D, Rosenberg MD, Huang J, Chun MM , et al. Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nat Neurosci 2015; 18(11): 1664-71.

Huber L, Finn ES, Chai Y, Goebel R, Stirnberg R, Stocker T , et al. Layer-dependent functional connectivity methods. Prog Neurobiol 2020: 101835.

Huber L, Handwerker DA, Jangraw DC, Chen G, Hall A, Stüber C , et al. High-Resolution CBV-fMRI Allows Mapping of Laminar Activity and Connectivity of Cortical Input and Output in Human M1. Neuron 2017; 96(6): 1253-63.e7.

Huber L, Ivanov D, Handwerker DA, Marrett S, Guidi M, Uludağ K , et al. Techniques for blood volume fMRI with VASO: From low-resolution mapping towards sub-millimeter layer-dependent applications. NeuroImage 2018; 164: 131-43.

Lewis CM, Bosman CA, Fries P. Recording of brain activity across spatial scales. Curr Opin Neurobiol 2015; 32: 68-77.

Logothetis NK. The underpinnings of the BOLD functional magnetic resonance imaging signal. J Neurosci 2003; 23(10): 3963-71.

Ma Y, Shaik MA, Kozberg MG, Kim SH, Portes JP, Timerman D , et al. Resting-state hemodynamics are spatiotemporally coupled to synchronized and symmetric neural activity in excitatory neurons. Proc Natl Acad Sci U S A 2016; 113(52): E8463-E71.

Matthews PM, Honey GD, Bullmore ET. Applications of fMRI in translational medicine and clinical practice. Nat Rev Neurosci 2006; 7(9): 732-44.

Menon RS. The great brain versus vein debate. NeuroImage 2012; 62(2): 970-4.

Miller KL, Alfaro-Almagro F, Bangerter NK, Thomas DL, Yacoub E, Xu J , et al. Multimodal population brain imaging in the UK Biobank prospective epidemiological study. Nat Neurosci 2016; 19(11): 1523-36.

Poldrack RA, Barch DM, Mitchell JP, Wager TD, Wagner AD, Devlin JT , et al. Toward open sharing of task-based fMRI data: the OpenfMRI project. Front Neuroinform 2013; 7: 12.

Polimeni JR, Uludag K. Neuroimaging with ultra-high field MRI: Present and future. NeuroImage 2018; 168: 1-6.

Silva MA, See AP, Essayed WI, Golby AJ, Tie Y. Challenges and techniques for presurgical brain mapping with functional MRI. Neuroimage Clin 2018; 17: 794-803.

Defending fMRI, Brain Mapping, and Discovery Science

Fri, 22 May 2020 15:28:00 GMT

This blog post was initiated by Dr. Vince Calhoun, director of the Tri-institutional Center for Translational Research in Neuroimaging and Data Science and of Georgia State University, Georgia Institute of Technology, and Emory University. Vince shot me an email asking if I saw this editorial in Brain by Dimitri Kullman ( Brain , Volume 143, Issue 4, April 2020, Page 1045) https://academic.oup.com/brain/article/143/4/1045/5823483 . He also made the suggestion that we write something together as a counterpoint. I heartily agreed. While there are many valid criticisms of fMRI and brain mapping in general, this particular editorial struck me as uninformed, myopic and cynical – thus requiring a response. I usually err on the side of giving the benefit of the doubt when reading or hearing of a different opinion, but my first visceral reaction to reading this article was simply: “Wow…” Vince and I quickly got to work and within a week submitted the below counterpoint to Brain.

Rebuttal to Editorial ( Brain, Volume 143, Issue 4, April 2020, Page 1045)

Vince Calhoun ¹ and Peter Bandettini ²

¹ Tri-institutional Center for Translational Research in Neuroimaging and Data Science: Georgia State University, Georgia Institute of Technology, Emory University, Atlanta, Georgia, USA.

² National Institute of Mental Health

In his editorial in Brain (Volume 143, Issue 4, April 2020, Page 1045), Dr. Dimitri Kullmann takes several cheap shots at fMRI as a field and at most of the research findings that it produces. He argues that fMRI-based findings describing functional differences in activation or connectivity have no place in Brain and that fMRI functional contrast is fundamentally flawed. He rants that fMRI is drawing away talented young researchers whose time and energy would be better spent using other modalities. This salvo misses the mark however, as it is woefully uninformed and incorrect.

Dr. Kullmann seems to equate brain mapping itself with flawed and non-hypothesis driven research: “Showing that activation patterns or functional connectivity motifs differ significantly is, on its own, insufficient justification to occupy space in Brain.” There is no need to argue the utility of brain mapping, as the thousands of outstanding papers in the literature speak for themselves. One just has to attend the Organization for Human Brain Mapping or Society for Neuroscience meetings to appreciate the traction that has been made by fMRI in generating insight into brain organization of healthy and clinical subjects.

Dimitri Kullmann’s central premise is that somehow the science performed with fMRI, to a greater degree than other modalities, is ineffective in penetrating meaningful neuroscience questions or leading to clinical applications – something akin to doing astronomy with a microscope. He states two reasons. The first: “… the fundamental relationship between the blood oxygenation level-dependent (BOLD) signal and neuronal computations remains a complete mystery. As a direct consequence, it is extremely difficult to conclude that functional connectivity as measured by functional MRI genuinely measures information exchange between brain regions.” This is partially true, as the relationship between ANY measure of neuronal firing or related physiology and neuronal computations IS a complete mystery. We really do not know what a neuronal computation would even look like no matter what is measured. However, the relationship between neuronal activity and fMRI signal changes is far from a complete mystery, rather it has been extensively studied. While this relationship is imperfectly understood, literally hundreds of papers have established the relationship between localized hemodynamic changes and neuronal activity, measured using a multitude of other modalities. Nearly all cross-modal verification has provided strong confirmation that where and when neuronal activity changes, hemodynamic changes occur – in proportion to the degree of neuronal activity. Certainly, issues related to spatial and temporally confounding effects of larger vascular and other factors are still being addressed, yet, sound experimental design, analysis, and interpretations can take these limits into account, allowing useful information to be derived. Additionally, multiple functional contrast manipulations and normalization approaches have reduced these vascular confounds. In contrast to what is claimed in the editorial, high field in fact does allow mitigation of large blood vessels thanks to higher sensitivity that enables scientists to use contrast manipulations less sensitive to large vein effects. Hundreds of ultra-high resolution fMRI studies are revealing cortical depth dependent activation that shows promise in informing feedback vs. feedforward connections.

The second of his reasons: “…effect sizes are quasi-impossible to infer, leading to an anomaly in science where statistical significance remains the only metric reported.” Effect sizes in fMRI are in fact quite straight-forward to compute using standard approaches and are very often reported. What is challenging is that there are many different fMRI-related variables that could be utilized. One might compare voxels, regions, patterns of activation, connectivity measures, or dynamics using an array of functional contrasts including blood flow, oxygenation, or blood volume. Thus, there are many different types of effects, depending on what is of interest. Rather than a weakness, this is a powerful strength of fMRI in that it is so rich and multi-dimensional.

The challenge of properly characterizing and modeling the meaningful signal as well as the noise is an ongoing point of research that is, in fact, shared by virtually every other brain assessment technique. In fMRI, the challenge is particularly acute because of the wealth and complexity of potential neuronal and physiological information provided. Singling out these issues as if they were specific to fMRI is indicative of a very narrow and perhaps biased perspective. Dr. Kullmann is effectively stating that indeed fMRI is different from all the rest – a particularly efficient generator of a disproportionately high fraction of poor and useless studies. This perspective is cynical and wrong and ignores that ALL modalities have their limits and associated bad science, ALL modalities have their range of questions that they can appropriately ask.

An additional point to emphasize in this age of big neuroscience data is that the editorial also expresses a cynicism against science that generates results that it cannot fully seal into a tight-fitting story. Describing a unique activation or connectivity pattern with a specific paradigm or demonstrating differences between populations or even individuals, while not always groundbreaking, usually advances our understanding of the brain, and can lead to clinical insights or even advances in clinical practice. Dr. Kullmann implies that the only legitimate use of fMRI in a study is in an hypothesis driven study. This view dismisses out of hand the value of discovery science, which casts a wide and effective net in gathering and making sense of large amounts of data. Both hypothesis driven and discovery science have importance and significance.

In summary, Dr. Kullmann argues that studies that compare activity or connectivity maps, as many fMRI studies do have no place in Brain . He claims that fMRI attracts too many talented researchers at the expense of better science performed with other tools. He describes two aspects of fMRI: the vascular origin of the signal and reporting on statistical measures, as being fatal flaws of the technique. However, he states that there are very rare exceptions – certain rare people are doing fMRI well.

We argue that location and timing of brain activity on the scales that fMRI allows is informative and useful information for both understanding the brain and clinical practice. One just has to take a more in depth view of the literature and growth fMRI over the past 30 years to appreciate the impact it has had. His cynicism that most fMRI users are misguided appears to dismiss the flawed yet powerful process of peer review. His specific criticisms of fMRI are incorrect as they bring up legitimate challenges but completely fail to appreciate how the field has dealt – and continues to effectively deal with them. These two criticisms also fail to acknowledge that limits in interpreting the measurements are inherent to all other brain assessment techniques – imaging or otherwise. Lastly, his highlighting of a single researcher and study in this issue of Brain is myopic as he appears to imply that these are the extreme exceptions – inferred from his earlier statements – rather than simply examples of a high fraction of outstanding fMRI papers. He mentions the value of hypothesis driven studies without appreciating the vast literature of hypothesis driven fMRI studies nor acknowledging the power of discovery science. Functional MRI is a tool and not a catalyst for categorically mediocre science. How it is used is determined by the skill of the researcher. The literature is filled with examples of how fMRI has been used with inspiring skill and insight to penetrate fundamental questions of brain organization and reveal subtle, meaningful, and actionable differences between clinical populations and individuals. Functional MRI is advancing in sophistication at a very rapid rate, allowing us to better ask fundamental questions about the brain, more deeply interpret its data, as well as to advance its clinical utility. Any argument that an entire modality should be categorically dismissed in any manner is troubling and should in principle be strongly rebuffed.

The New Age of Virtual Conferences

Thu, 16 Apr 2020 00:56:00 GMT

For decades, the scientific community has witnessed a growing trend towards online collaboration, publishing, and communication. The next natural step, started over the past decade, has been the emergence of virtual lectures, workshops, and conferences. My first virtual workshop took place back in about 2011 when I was asked to co-moderate a virtual session about 10 talks on MRI methods and neurophysiology. It was put on jointly by the International Society for Magnetic Resonance in Medicine (ISMRM) and the Organization for Human Brain Mapping (OHBM) and considered an innovative experiment at the time. I recall running it from a hotel room with spotty internet in Los Angeles as I was also participating in an in-person workshop at UCLA at the same time. It went smoothly, as the slides displayed well, speakers came through clearly, and, at the end of each talk, participants were able to ask questions by text which I could read to the presenter. It was easy, perhaps a bit awkward and new, but definitely worked and was clearly useful.

Since then, the virtual trend has picked up momentum. In the past couple of years, most talks that I attended at the NIH were streamed simultaneously using Webex. Recently, innovative use of twitter has allowed virtual conferences consisting of twitter feeds. An example of such twitter-based conferences is #BrainTC , which was started in 2017 and is now putting these on annually.

Using the idea started with #BrainTC, Aina Puce spearheaded OHBMEquinoX or OHBMx . This “conference” took place on the Spring Equinox involving sequential tweets from speakers and presenters from around the world. It started in Asia and Australia and worked its way around with the sun during this first day of spring where the sun was directly above the equator and the entire planet had precisely the same number of hours of sunlight.

Recently, conferences with live streaming talks have been assembled in record time, with little cost overhead, providing a virtual conference experience to audiences numbering in the 1000’s at extremely low or even no registration cost. An outstanding recent example of a successful online conference is neuromatch.io . An insightful blog post summarized logistics of putting this on.

Today, the pandemic has thrown in-person conference planning, at least for the spring and summer of 2020, into chaos. The two societies with which I am most invested, ISMRM and OHBM, have taken different solutions to cancellations in their meetings. ISMRM has chosen to delay their meeting to August. ISMRM’s delay will hopefully be enough time for the current situation to return to normal, however, given the uncertainty of the precise timeline, even this delayed in-person meeting may have to be cancelled. OHBM has chosen to make this year’s conference virtual and are currently scrambling to organize it – aiming for the same start date in June that they had originally planned.

What we will see in June with OHBM will be a spectacular, ambitious, and extremely educational experiment. While we will be getting up to date on the science, most of us will also be having our first foray into a multi-day, highly attended, highly multi-faceted conference that was essentially organized in a couple of months.

Virtual conferences, now catalyzed by COVID-19 constraints, are here to stay. These are the very early days. Formats and capabilities of virtual conferences will be evolving for quite some time. Now is the time to experiment with everything, embracing all the available online technology as it evolves. Below is an incomplete list of the advantages, disadvantages, and challenges of virtual conferences, as I see them.

What are the advantages of a virtual conference?

1. Low meeting cost. There is no overhead cost to rent a venue. Certainly, there are some costs in hosting websites however these are a fraction of the price of renting conference halls.

2. No travel costs. No travel costs or time and energy are incurred for travel for the attendees and of course a corresponding reduction in carbon emissions from international travel. Virtual conferences allow an increased inclusivity to those who cannot afford to travel to conferences, potentially opening up access to a much more diverse audience – resulting in corresponding benefits to everyone.

3. Flexibility. Because there is no huge venue cost the meeting can last as long or short as necessary and can take place for 2 hours a day or several hours interspersed throughout the day to accommodate those in other time zones. It can last the normal 4 or 5 days or can be extended for three weeks if necessary. There will likely be many discussions on what the optimal virtual conference timing and spacing should be. We are in the very early days here.

5. Ease of access to information within the conference. With, hopefully, a well-designed website, session attendance can be obtained with a click of a finger. Poster viewing and discussing, once the logistics are fully worked out, might be efficient and quick. Ideally, the poster “browsing” experience will be preserved. Information on poster topics, speakers, and perhaps a large number of other metrics will be cross referenced and categorized such that it’s easy to plan a detailed schedule. One might even be able to explore a conference long after it is completed, selecting the most viewed talks and posters, something like searching articles using citations as a metric. Viewers might also be able to rate each talk or poster that they see, adding to usable information to search.

6. Ease of preparation and presentation. You can present from your home and prepare up to the last minute in your home.

7. Direct archival. It should be trivial to directly archive the talks and posters for future viewing, so that if one doesn’t need real-time interaction or misses the live feed, one can participate in the conference any time in the future at their own convenience. This is a huge advantage that is certainly also possible even for in-person conferences, but has not yet been achieved in a way that quite represents the conference itself. With a virtual conference, there can be a one-to-one conference “snapshot” preservation of precisely all the information contained in the conference as it’s already online and available.

What are the disadvantages of a virtual conference?

1. Socialization. To me the biggest disadvantage is the lack of directly experiencing all the people. Science is a fundamentally human pursuit. We are all human, and what we communicate by our presence at a conference is much more than the science. It’s us, our story, our lives and context. I’ve made many good friends at conferences and look forward to seeing them and catching up every year. We have a shared sense of community that only comes from discussing something in front of a poster or over a beer or dinner. This is the juice of science. At our core we are all doing what we can towards trying to figure stuff out and creating interesting things. Here we get a chance to share it with others in real time and gauge their reaction and get their feedback in ways so much more meaningful than that provided virtually. One can also look at it in terms of information. There is so much information that is transferred during in-person meetings that simply cannot be conveyed with virtual meetings. These interactions are what makes the conference experience real, enjoyable, and memorable, which all feeds into the science.

2. Audience experience. Related to 1, is the experience of being part of a massive collective audience. There is nothing like being in a packed auditorium of 2000 people as a leader of the field presents their latest work or their unique perspective. I recall the moment I first saw the first preliminary fMRI results presented by Tom Brady at ISMRM. My jaw dropped and I looked at Eric Wong, sitting next to me, in amazement. After the meeting, there was a group of scientists huddled in a circle outside the doors talking excitedly about the results. FMRI was launched into the world and everyone felt it and shared that experience. These are the experiences that are burnt into people’s memories and which fuel their excitement.

3. No room for randomness. This could be built into a virtual conference, however at an in-person conference, one of the joys is to experience first-hand, the serendipitous experiences – the bit of randomness. Chance meetings of colleagues or passing by a poster that you didn’t anticipate. This randomness is everywhere at a conference venue perhaps more important than we realize. There may be clever ways to engineer a degree of randomness into a virtual conference experience, however.

4. No travel. At least to me, one of the perks of science is the travel. Physically traveling to another lab, city, country, or continent is a deeply immersive experience that enriches our lives and perspectives. On a regular basis, while it can turn into a chore at times, is almost always worth it. The education and perspective that a scientist gets about our world community is immense and important.

5. Distraction. Going to a conference is a commitment. The problem I always have when a conference is in my own city is that as much as I try to fully commit to it, I am only half there. The other half is attending to work, family, and the many other mundane and important things that rise up and demand my attention for no other reason than I am still here in my home and dealing with work. Going to a conference separates one from that life, as much as can be done in this connected world. Staying in a hotel or AirBnB is a mixed bag – sometimes delightful and sometimes uncomfortable. However, once at the conference, you are there. You assess your new surroundings, adapt, and figure out a slew of minor logistics. You immerse yourself in the conference experience, which is, on some level, rejuvenating – a break from the daily grind. A virtual conference is experienced from your home or office and can be filled with the distraction of your regular routine pulling you back. The information might be coming at you but the chances are that you are multi-tasking and interrupted. The engagement level during virtual sessions, and importantly, after the sessions are over, is less. Once you leave the virtual conference you are immediately surrounded by your regular routine. This lack of time away from work and home life I think is also a lost chance to ruminate and discuss new ideas outside of the regular context.

What are the challenges?

1. Posters. Posters are the bread and butter of “real” conferences. I’m perhaps a bit old school in that I think that electronic posters presented at “real” conferences are absolutely awful. There’s no way to efficiently “scan” electronic posters as you are walking by the lineup of computer screens. You have to know what you’re looking for and commit fully to looking at it. There’s a visceral efficiency and pleasure of walking up and down the aisles of posters, scanning, pausing, and reading enough to get the gist, or stopping for extended times to dig in. Poster sessions are full of randomness and serendipity. We find interesting posters that we were not even looking for. Here we see colleagues and have opportunities to chat and discuss. Getting posters right in virtual conferences will likely be one of the biggest challenges. I might suggest creating a virtual poster hall with full, multi-panel posters as the key element of information. Even the difference between clicking on a title vs scrolling through the actual posters in full multi-panel glory will make a massive difference in the experience. These poster halls, with some thought, can be constructed for the attendee to search and browse. Poster presentations can be live with the attendee being present to give an overview or ask questions. This will require massive parallel streaming but can be done. An alternative is to have the posters up, a pre-recorded 3 minute audio presentation, and then a section for questions and answers – with the poster presenter being present live to answer in text questions that may arise and having the discussion text preserved with the poster for later viewing.

2. Perspective. Keeping the navigational overhead low and whole meeting perspective high. With large meetings, there is a of course a massive amount of information that is transferred that no one individual can take in. Meetings like SFN, with 30K people, are overwhelming. OHBM and ISMRM, with 3K to 7K people, are also approaching this level. The key to making these meetings useful is creating a means by which the attendee can gain a perspective and develop a strategy for delving in. Simple to follow schedules with enough information but not too much, customized schedule-creation searches based on a wide rage of keywords and flags for overlap are necessary. The room for innovation and flexibility is likely higher at virtual conferences than at in-person conferences, as there are less constraints on temporal overlap.

3. Engagement. Fully engaging the listener is always a challenge, with a virtual conference it’s even more so. Sitting at a computer screen and listening to a talk can get tedious quickly. Ways to creatively engage the listener – real time feedback, questions to the audience, etc.. might be useful to try. Also, conveying effectively with clever graphics the size or relative interests of the audience might also be useful in creating this crowd experience.

4. Socializing. Neuromatch.io included a socializing aspect to their conference. There might be separate rooms of specific scientific themes for free discussion, perhaps led by a moderator. There might also be simply rooms for completely theme-less socializing or discussion about any aspect of the meeting. Nothing will compare to real meetings in this regard, but there are some opportunities to potentially exploit the ease of accessing information about the meeting virtually to be used to enrich these social gatherings.

5. Randomness. As I mentioned above, randomness and serendipity play a large role in making a meeting successful and worth attending. Defining a schedule and sticking to it is certainly one way of attacking a meeting, but others might want to randomly sample and browse and randomly run into people. It might be possible for this to be done in the meeting scheduling tool but designing opportunities for serendipity in the website experience itself should be given careful thought. One could decide on a time when they view random talks or posters or meet random people based on a range of keywords.

6. Scalability. It would be useful to have virtual conferences constructed of scalable elements such as poster sessions, keynotes, discussion, proffered talks, that could start to become standardized to increase ease of access and familiarity across conferences of different sizes from 20 to 200,000 as it’s likely that virtual meeting sizes will vary more widely yet will be generally larger than “real” meetings.

7. Costs vs. Charges? This will be of course determined on its own in a bottom up manner based on regular economic principles, however, in these early days, it’s useful to for meeting organizers to work through a set of principles of what to charge or if to make a profit at all. It is possible that if the web-elements of virtual meetings are open access, many of costs could disappear. However, for regular meetings of established societies there will be always be a need to support the administration to maintain the infrastructure.

Beyond Either-Or:

Once the unique advantages of virtual conferences are realized, I imagine that even as in-person conferences start up again, there will remain a virtual component, allowing a much higher number and wider range of participants. These conferences will perhaps simultaneously offer something to everyone – going well beyond simply keeping talks and posters archived for access – as is the current practice today.

While I have helped organize meetings for almost three decades, I have not yet been part of organizing a virtual meeting, so in this area, I don’t have much experience. I am certain that most thoughts expressed here have been thought through and discussed many times already. I welcome any discussion on points that I might have wrong or aspects I may have missed.

Virtual conferences are certainly going to be popping up at an increasing rate, throwing open a relatively unexplored wide open space for creativity with the new constraints and opportunities of this venue. I am very much looking forward to seeing them evolve and grow – and helping as best I can in the process.

So I finally wrote a book…

Fri, 24 Jan 2020 19:55:00 GMT

One day, back in the mid 2010’s, feeling just a bit on top of my work duties, and more than a little ambitious, I decided that writing a book would be a worthwhile way to spend my extra time. I wanted to write an accessible book on fMRI, imbued with my own perspective of the field. Initially, I had thought of taking on the daunting task of writing a popular book on the story of fMRI – it’s origins and interesting developments (there are great stories there!) – but decided that I’ll put that off until my skill in that medium has improved. I approached Robert Prior from MIT Press to discuss the idea of a book on fMRI for audiences ranging from the interested beginner to expert. He liked it and, after about a couple of years of our trying to decide on the precise format, he approached me with he idea of making it part of the MIT Essential Knowledge Series. This is a series being put out by MIT Press containing relatively short “Handbooks” on a wide variety of topics with writing at the level of about a “Scientific American” article. Technical and accessible to anyone who has the interest but not overly technical or textbook dry – highly readable to people who want to get a good in depth summary of a topic or field from an expert.

I agreed to give this a try. The challenge was that it had to be about 30K to 50K words and containing minimal figures with no color. The audience was tricky. I didn’t want to make it so simple as to present nuanced facts incorrectly and disgruntle my fellow experts, but I also didn’t want to have it go too much in depth on any particular issue thus leaving beginners wading through content that was not really enjoyable. My goal was to first describe the world of brain imaging that existed when fMRI was developed, and then outline some of the more interesting history from someone who lived it, all while giving essential facts about the technique itself. Later chapters deal with topics involving acquisition, paradigm design, processing and so forth – all while striving to keep the perspective broad and interesting. At the end of the book, I adopted a blog post as a chapter on the “26 controversies and challenges” of fMRI, adding a concluding perspective that while fMRI is mature, it still has more than its share of controversies and unknowns, and that these are in fact good things that keep the field moving along and advancing as they tend to focus and drive the efforts of many of the methodologists.

After all was done, I was satisfied with what I wrote and pleasantly surprised that my own unique perspective – that of someone who has been involved with the field since its inception – came clearly through. My goal, which I think I achieved, was incorporate as much insight into the book as possible, rather than just giving the facts. I am now in the early stages of attempting to write a book of the story of fMRI, perhaps adding perspective on where all this is eventually going, but for now I look forward to the feedback about this MIT Essential Knowledge Series book on fMRI.

Some takeaway thoughts on my first major writing project since the composition of my Ph.D. thesis over 26 years ago. By the way, for those interested, my thesis can be downloaded from figshare: DOI and link below:
10.6084/m9.figshare.11711430. Peter Bandettini’s Ph.D. Thesis 1994 .

Below is the preface to the book fMRI. I hope you will take a look and enjoy reading it when it comes out. Also, I welcome any feedback at all (good or bad). Writing directly to me via: bandettini@nih.gov will get my attention.

Preface to FMRI:

In taking the first step and picking up this book, you may be wondering if this is just another book on fMRI (functional magnetic resonance imaging). To answer: This is not just another book on fMRI. While it contains all the basics and some of the more interesting advanced methods and concepts, it is imbued, for better or worse, with my unique perspective on the field. I was fortunate to be in the right place at the right time when fMRI first began. I was a graduate student at the Medical College of Wisconsin looking for a project. Thanks in large part to Eric Wong, my brilliant fellow graduate student who had just developed, for his own non-fMRI purposes, the hardware and pulse sequences essential to fMRI, and my co-advisors Scott Hinks and Jim Hyde who gave me quite a bit of latitude to find my own project, we were ready to perform fMRI before the first results were publicly presented by the Massachusetts General Hospital group on August 12, 1991, at the Society for Magnetic Resonance Meeting in San Francisco. After that meeting, I started doing fMRI, and in less than a month I saw my motor cortex light up when I tapped my fingers. As a graduate student, it was a mind-blowingly exciting time—to say the least. My PhD thesis was on fMRI contrast mechanisms, models, paradigms, and processing methods. I’ve been developing and using fMRI ever since. Since 1999, I have been at the National Institute of Mental Health, as chief of the Section on Functional Imaging Methods and director of the FunctionalMRI Core Facility that services over thirty principle investigators. This facility has grown to five scanners—one 7T and four 3Ts.

Thousands of researchers in the United States and elsewhere are fortunate that the National Institutes of Health (NIH) has provided generous support for fMRI development and applications continuously over the past quarter century. The technique has given us an unprecedented window into human brain activation and connectivity in healthy and clinical populations. However, fMRI still has quite a long way to go toward making impactful clinical inroads and yielding deep insights into the functional organization and computational mechanisms of the brain. It also has a long way to go from group comparisons to robust individual classifications.

The field is fortunate because in 1996, fMRI capability (high-speed gradients and time-series echo planar imaging) became available on standard clinical scanners. The thriving clinical MRI market supported and launched fMRI into its explosive adoption worldwide. Now an fMRI-capable scanner was in just about every hospital and likely had quite a bit of cheap free time for a research team to jump on late at night or on a weekend to put a subject in the scanner and have them view a flashing checkerboard or tap their fingers.

Many cognitive neuroscientists changed their career paths entirely in order to embrace this new noninvasive, relatively fast, sensitive, and whole-brain method for mapping human brain function. Clinicians took notice, as did neuroscientists working primarily with animal models using more invasive techniques. It looked like fMRI had potential. The blood oxygen level–dependent (BOLD) signal change was simply magic. It just worked—every time. That 5% signal change started revealing, at an explosive rate, what our brains were doing during an ever-growing variety and number of tasks and stimuli, and then during “rest.”

Since the exciting beginnings of fMRI, the field has grown in different ways. The acquisition and processing methods have become more sophisticated, standardized, and robust. The applications have moved from group comparisons where blobs were compared—simple cartography— to machine learning analysis of massive data sets that are able to draw out subtle individual differences in connectivity between individuals. In the end, it’s still cartography because we are far from looking at neuronal activity directly, but we are getting much better at gleaning ever more subtle and usefulinformation from the details of the spatial and temporal patterns of the signal change. While things are getting more standardized and stable on one level, elsewhere there is a growing amount of innovation and creativity, especially in the realm of post-processing. The field is just starting to tap into the fields of machine learning, network science, and big data processing.

The perspective I bring to this book is similar to that of many who have been on the front lines of fMRI methodology research—testing new processing approaches and new pulse sequences, tweaking something here or there, trying to quantify the information and minimize the noise and variability, attempting to squeeze every last bit of interesting information from the time series—and still working to get rid of those large vessel effects!

This book reflects my perspective of fMRI as a physicist and neuroscientist who is constantly thinking about how to make fMRI better—easier, more informative, and more powerful. I attempt to cover all the essential details fully but without getting bogged down in jargon and complex concepts. I talk about trade-offs—those between resolution and time and sensitivity, between field strength and image quality, between specificity and ease of use.

I also dwell a bit on the major milestones—the start of resting state fMRI, the use and development of event-related fMRI, the ability to image columns and layers, the emergence of functional connectivity imaging and machine learning approaches—as reflecting on these is informative and entertaining. As a firsthand participant and witness to the emergence of these milestones, I aim to provide a nuanced historical context to match thescience.

A major part of fMRI is the challenge to activate the brain in just the right way so that functional information can be extracted by the appropriate processing approach against the backdrop of many imperfectly known sources of variability. My favorite papers are those with clever paradigm designs tailored to novel processing approaches that result in exciting findings that open up vistas of possibilities. Chapter 6 covers paradigm designs, and I keepthe content at a general level: after learning the basics of scanning and acquisition, learning the art of paradigm design is a fundamental part of doing fMRI well. Chapter 7 on fMRI processing ties in with chapter 6 and again, is kept at a general level in order to provide perspective and appreciation without going into too much detail.

Chapter 8 presents an overview of the controversies and challenges that have faced the field as it has advanced. I outline twenty-six of them, but there are many more. Functional MRI has had its share of misunderstandings, nonreproducible findings, and false starts. Many are not fully resolved. As someone who has dealt with all of these situations firsthand, I believe that they mark how the field progresses—one challenge, one controversy at a time. Someone makes a claim that catalyzes subsequent research, which then either confirms, advances, or nullifies it. This is a healthy process in such a dynamic research climate, helping to focus the field.

This book took me two years longer to write than I originally anticipated. I appreciate the patience of the publisher Robert Prior of MIT Press who was always very encouraging. I also thank my lab members for their constant stimulation, productivity, and positive perspective. Lastly, I want to thank my wife and three boys for putting up with my long blocks of time ensconced in my office at home, struggling to put words on the screen. I hope you enjoy this book. It offers a succinct overview of fMRI against the backdrop of how it began and has developed and—even more important—where it may be going.

The book “FMRI” can be purchased at MIT Press and Amazon, among other places:

MIT Press
Amazon

Starting a Podcast: NIMH Brain Experts Podcast

Mon, 06 Jan 2020 22:13:00 GMT

About a year or so ago, I was thinking of ways to improve NIMH outreach – to help show the world of non-scientists what NIMH-related researchers are doing. I wanted to not only convey the issues, insights, and implications of their work but also provide a glimpse into the world of clinical and basic brain research – to reveal the researchers themselves and what their day to day work looks like, what motivates and excites them, and what their challenges are. Initially, I was going to organize public lectures or a public forum, but the overall impact of this seemed limited. I wanted an easily accessible medium that also preserved the information for future access, so I decided to take the leap into podcasting. I love a good conversation and felt I was pretty good at asking good questions and keeping a conversation flowing. There have been so many great conversations that I have with my colleagues that I wish that I could have preserved and saved in some way. The podcast structure is slightly awkward (“interviewing” colleagues), and of course, there is always the pressure of not saying the wrong thing or not knowing some basic piece of information that I should know. I had and still have – for quite some time – much to learn with regard to perfecting this skill.

I decided to go through official NIMH channels to get this off the ground, and happily the people in the public relations department loved the idea. I had to provide them with two “pilot” episodes to make sure that it was all ok. Because the podcast was under the “official” NIMH label, I had to be careful not to say anything that could be misunderstood as an official NIMH position or at least I had to qualify any potentially controversial positions. Next were the logistics.

Before it started, I had to do a few things: pick an introduction musical piece and a graphic to show with the podcast. Also I had to pick a name for the podcast. I was introduced into the world of non-copyrighted music. I learned that there are many services out there that give you rights to a wide range of music for a flat fee. I used a website service: www.premiumbeat.com . I picked a tune that seemed thoughtful, energetic, and positive. As for the graphic, I chose an image that comes from a highly processed photo of a 3D printout of my own brain. It’s the image at the top of this post. Both the music and graphic were approved, and we finally arrived on a name “The Brain Experts” which pretty much what it was all about.

For in-person podcasts I use a multi-directional Yeti microphone and Quicktime on my Mac to record. This seems to work pretty well. I really should be making simultaneous backup recordings though – just in case IT decides to reboot my computer during a podcast. I purchased a muli-microphone & mixer setup to be used for future episodes. For remote podcasts, I use Zoom which has a super simple recording feature and has generally had the best performance of any videoconferencing software that I have used. I can also save only the audio files to a surprisingly small (much smaller than with Quicktime) file. Once the files are saved, it’s my responsibility to get them transcribed. There are many cheap and efficient transcription services out there. I also provide a separate introduction to the podcast and the guest – recorded at a separate time. Once the podcast and transcript are done, I send them to the public relations people, who do the editing and packaging.

The general format of the podcast is as follows: I interview the guest for about an hour and some of the interview is edited out – resulting in a podcast that is generally about 30 minutes in length. I wish it could be longer but the public relations people decided that 30 minutes was a good digestible time. I start with the guests’ backgrounds and how they got to where they are. I ask about what motivates them and what excites them. I then get into the science – the bulk of the podcast – bringing up recent work or perhaps discussing a current issue related to their own research. After that, I end by discussing any challenges they have going on, what their future plans are, and also if they had any advice to new researchers. I’ve been pleased that so far, no one has refused an offer to be on my podcast. I think most of gone well! I certainly learned quite a bit. Also, importantly, about a week before I interview the guests, I provide them with a rough outline of questions that I may ask and papers that I may want to discuss.

For the first four podcasts, I have chosen guests that I know pretty well: Francisco Pereira – an NIMH staff scientist heading up the Machine Learning Team that I started, Niko Kriegeskorte – a computational cognitive neuroscientist at Columbia University who was a former post doc of mine, Danny Pine – a Principle Investigator in the NIMH intramural program who has been a colleague of mine for almost 20 years, and Chris Baker – a Principle Investigator in the NIMH intramural program who has been a co-PI with me in the Laboratory of Brain and Cognition at the NIMH for over a decade. Most recently, I interviewed Laura Lewis, from Boston University, who is working on some exciting advancements in fMRI methods that are near and dear to my heart. In the future I plan to branch out more to cover the broad landscape of brain assessment – beyond fMRI and imaging, however in these first few, I figured I would start in my comfort zone.

Brain research can be roughly categorized into: Understanding the brain, and Clinical applications. Of course, there is considerable overlap between the two, and the best research establishes a strong link between fundamental understanding and clinical implementation. Not all brain understanding leads directly to clinical applications as the growing field of artificial intelligence tries to glean organizational and functional insights from neural circuitry. The podcasts, while focused on a guest, each have a theme that is related to either of the above two categories. So far, Danny Pine has had a clinical focus – on the problem of how to make fMRI more clinically relevant in the context of psychiatric disorders, and Niko and Chris have had a more basic neuroscience focus. With Niko I focused on the sticky question of how relevant can fMRI be for informing mechanistic models of the brain. With Chris, we talked at length about the unique approach he takes to fMRI paradigm design and processing with regard to understanding visual processing and learning. Francisco straddled the two since machine learning methods promise to enhance both basic research and provide more powerful statistical tools for clinical implementation of fMRI.

In the future I plan to interview both intramural and extramural scientists covering the entire gamut of neuroscience topics. Podcasting is fascinating and exhausting. After each interview, I’m exhausted in that the level of “on” that I have to be is much higher in casual conversation. The research – even in areas that I know well – takes a bit of time, but is time well spent. Importantly, I try to not only glean over the topics, but dig for true insight into issues that we all are grappling with. The intended audience is broad: from the casual listener to the scientific colleague, so I try to guide the conversation to include something for everyone. The NIH agreed to 7 podcasts and it looks like they will wrap it up after the 7th due to the fact that they don’t have the personnel for the labor intensive editing and producing process, so it looks like I have one more to go. My last interview will be with Dr. Susan Amara, who is the director of the NIMH intramural program and will take place in December. I have other plans to continue podcasting, so stay tuned!

The podcasts can be found using most podcast apps: iTunes, Spotify, Castro, etc.. Just do a search for “NIMH Brain Experts Podcast.”

The youtube versions of these can be found at https://www.youtube.com/playlist?list=PLV9WJDAawyhaMmciHR6SCwop-9BzsbsIl

The “official” posting of the first 6 podcasts can be found (with transcripts) here:

Lastly, if you would like to be interviewed or know someone who you think would make a great guest, please give me an email at bandettini@nih.gov. I’m setting up my list now. The schedule is about one interview every three months.

Twenty-Six Controversies and Challenges in fMRI

Sun, 23 Dec 2018 17:53:00 GMT

•Neurovascular Coupling •Draining Veins •Linearity •Pre-undershoot •Post-undershoot •Long duration •Mental Chronometry •Negative Signal Changes •Resting state source •Dead fish activation •Voodoo correlations •Global signal regression •Motion artifacts •The decoding signal •non-neuronal BOLD •relationship to other measures •contrast: spin-echo vs gradient-echo •contrast: SEEP contrast •contrast: diffusion changes •contrast: neuronal currents •contrast: NMR phase imaging •lie detection •correlation ≠ connection •clustering conundrum •reproducibility •dynamic connectivity changes

This will be a chapter in my upcoming book “Functional MRI” in the MIT Press Essential Knowledge Series

Functional MRI is unique in that, in spite of being almost 30 years old as a method, it continues to progress in terms of sophistication of acquisition, hardware, processing, in our understanding of the signal itself. There has been no plateau in any of these areas. In fact, by looking at the literature, one gets the impression that this advancement is accelerating. Every new advance opens the potential range where it might have an impact, allowing new questions about the brain to be addressed.

In spite of its success – perhaps as a result of its success – it has had its share of controversies coincident with methods advancements, new observations, and novel applications. Some controversies have been more contentious than others. Over the years, I’ve been following these controversies and have at times performed research to resolve them or at least better understand them. A good controversy can help to move the field forward as it can focus and motivate groups of people to work on the issue itself, shedding a broader light on the field as these are overcome.

While a few of the controversies or issues of contention have been fully resolved, most remain to some degree unresolved. Understanding fMRI through its controversies allows a deeper appreciation for how the field advances as a whole – and how science really works – the false starts, the corrections, and the various claims made by those with potentially useful pulse sequences, processing methods, or applications. Below is the list of twenty-six major controversies in fMRI – in approximately chronological order.

#1: The Neurovascular coupling debate.

Since the late 1800’s, the general consensus, hypothesized by Roy and Sherrington in 1890 (1), was that activation-induced cerebral flow changes were driven by local changes in metabolic demand. In 1986, a publication by Fox et al. (2)challenged that view, demonstrating that with activation, localized blood flow seemingly increased beyond oxidative metabolic demand, suggesting an “uncoupling” of the hemodynamic response from metabolic demand during activation. Many, including Louis Sokolof, a well-established neurophysiologist at the National Institutes of Health, strongly debated the results. Fox nicely describes this period in history from his perspective in “The coupling controversy” (3).

I remember well, in the early days of fMRI, Dr. Sokolof standing up from the audience to debate Fox on several circumstances, arguing that the flow response should match the metabolic need and there should be no change in oxygenation. He argued that what we are seeing in fMRI is something other than an oxygenation change.

In the pre-fMRI days, I recall not knowing what direction the signal should go – as when I first laid eyes on the impactful video presented by Tom Brady during his plenary lecture on the future of MRI at the Society for Magnetic Resonance (SMR) Meeting in August of 1991, it was not clear from these time series movies of subtracted images the direction he performed the subtraction operation. Was it task minus rest or rest minus task? Did the signal go up or down with activation? I also remember very well, analyzing my first fMRI experiments, expecting to see a decrease in BOLD signal – as Ogawa, in an earlier paper(4), hypothesized that metabolic rate increases would lead to a decrease blood oxygenation thus a darkening of the BOLD signal during brain activation. Instead, all I saw were signal increases. It was Fox’s work that helped me to understand why the BOLD signal should increase with activation. Flow goes up and oxygen delivery exceeds metabolic need, leading to an increase in blood oxygenation.

While models of neurovascular coupling have improved, we still do not understand the precise need for flow increases. First, we had the “watering the whole garden to feed one thirsty flower” hypothesis which suggested that flow increases matched metabolic need for one area but since vascular control was coarse, the abundant oxygenation was delivered to a wider area than was needed, causing the increase in oxygenation. We also had the “diffusion limited” model, where it was hypothesized that in order to deliver enough oxygen to the furthest neurons from the oxygen supplying vessels, an overabundance of oxygen was needed at the vessel itself since the decrease of oxygen as it diffused from the vessel to the furthest cell was described as an exponential. This theory has fallen a bit from favor as the increases in CMRO ₂ or the degree to which the diffusion of oxygen to tissue from blood is limited tend to be higher than physiologic measures. The alternative to the metabolic need hypothesis involves neurogenic “feed-forward” hypotheses – which still doesn’t get at the “why” of the flow response.

Currently, this is where the field stands. We know that the flow response is robust and consistent. We know that in active areas, oxygenation in healthy brains always increases, however we just don’t understand specifically why it’s necessary. Is it a neurogenic, metabolic, or some other mechanism to satisfy some evolutionary critical need that extends beyond simple need for more oxygen? We are still figuring that out. Nevertheless, it can be said that whatever the reason for this increase in flow, it is fundamentally important, as the BOLD response is stunningly consistent.

#2: The Draining Vein Effect

The question: “what about the draining veins?” I think was first posited at an early fMRI conference by Kamil Ugurbil of the University of Minnesota. Here and for the next several years he alerted the community to the observation that, especially at low field, draining veins are a problem as they smear and distort the fMRI signal change such that it’s hard to know specifically where the underlying neuronal activation is with a high degree of certainty. In the “20 years of fMRI: the science and the stories” special issue of NeuroImage, Ravi Menon writes a thorough narrative of the “Great brain vs vein” debate (5). When the first fMRI papers were published, only one, by Ogawa et al. (6), was at high field – 4 Tesla – and relatively high resolution. In Ogawa’s paper, it was shown that there was a differentiation between veins (very hot spots) and capillaries (more diffuse weaker activation in grey matter). Ravi followed this up with another paper (7)using multi-echo imaging, to show that blood in veins had an intrinsically shorter T2* decay than gray matter at 4T and appeared as dark dots in T2* weighted structural images yet appeared as bright functional hot spots in the 4T functional images. Because of the low SNR and CNR at 1.5T, allowing only the strongest BOLD effects to be seen, and because models suggested that at low field strengths, large vessels contributed the most to the signal, the field worried that all fMRI was looking at was veins – at least at 1.5T.

The problem of large vein effects is prevalent using standard gradient-echo EPI – even at high fields. Simply put, the BOLD signal is directly proportional to the venous blood volume contribution in each voxel. If the venous blood volume is high – as with the case of a vein filling a voxel – then the potential for high BOLD changes is high if there is a blood oxygenation change in the area. At high field, indeed, there is not much intravascular signal left in T2* weighted Gradient-echo sequences, however, the extravascular effect of large vessels still exists. Spin-echo sequences (sensitive to small compartments) still are sensitive to the susceptibility effects around intravascular red blood cells within large vessels – even at 7T where intravascular contribution is reduced. Even with some vein sensitivity, promising high resolution orientation column results have been produced at 7T using gradient-echo and spin-echo sequences (8). The use of arterial spin labeling has potential as a method insensitive to large veins, although the temporal efficiency, intrinsic sensitivity, and brain coverage limitations blunt its utility. Vascular Space Occupancy (VASO), a method sensitive to blood volume changes, has been shown to be exquisitely sensitive to blood volume changes in small vessels and capillaries. Preliminary results have shown clear layer dependent activation using VASO where other approaches have provided less clear delineation(9).

Methods have arisen to identify and mask large vein effects – including thresholding based on percent signal change (large veins tend to fill voxels and thus exhibit a larger fractional signal change), as well as temporal fluctuations (large veins are pulsatile thus exhibit more magnitude and phase noise). While these seem to be workable solutions, they have not been picked up and used extensively. With regard to using the phase variations as a regressor to eliminate pulsatile blood and tissue, perhaps the primary reason for this not being adopted is because standard scanners do not produce these images readily, thus users do not have easy access to this information.

The draining vein issue is least problematic at voxel sizes larger than 2mm, as at these resolutions, mostly the region of activation- as defined as >1 cm “blob” is used and interpreted. Other than enhancing the magnitude of activation, vein effects do not distort these “blobs” thus are typically of no concern for low resolution studies. In fact, the presence of veins helps to amplify the signal in these cases. Spatial smoothing and multi-subject averaging – still commonly practiced – also ameliorate vein effects as they tend to be averaged out as each subject has a spatially variant macroscopic venous structure.

The draining vein problem is most significant where details of high resolution fMRI maps need to be interpreted for understanding small and tortuous activation patterns in the context of layer and column level mapping. So far no fully effective method works at this resolution as the goal is not to mask the voxels containing veins since there may be useful capillary and therefore neuronal effects still within the voxel. We need to eliminate vein effects more effectively on the acquisition side.

#3: The linearity of the BOLD response

The BOLD response is complex and not fully understood. In the early days of fMRI, it was found that BOLD contrast was both linear and nonlinear. The hemodynamic response tends to overestimate activation at very brief (<3 seconds) stimulus durations (10)or at very low stimulus intensities (11). With stimuli that were of duration of 2 seconds or less, the response was much larger than predicted by a linear system. The reasons for these nonlinearities are still not fully understood, however for interpreting transient or weak activations relative to longer duration activation, a clear understanding of the nonlinearity of neurovascular coupling across all activation intensities and durations needs to be well established.

#4: The pre-undershoot

The fMRI pre-undershoot was first observed in the late 90’s. With activation, it was sometimes observed that the fMRI signal, in the first 0.5 second of stimulation, first deflected slightly downwards before it increased (12, 13). Only a few groups were able to replicate this finding in fMRI however it appears to be ubiquitous in optical imaging work on animal models. The hypothesized mechanism is that before the flow response has a chance to start, a more rapid increase in oxidative metabolic rate causes the blood to become transiently less oxygenated. This transient effect is then washed away by the large increase in flow that follows.

Animal studies have demonstrated that this pre-undershoot could be enhanced by decreasing blood pressure. Of the groups that have seen the effect in humans, a handful claim that it is more closely localized to the locations of “true” neuronal activation. These studies were all in the distant past (>15 years ago) and since then, very few papers have come out revisiting the study of the elusive pre-undershoot. It certainly may be that it exists, but the precise characteristics of the stimuli and physiologic state of the subject may be critically important to produce it.

While simulations of the hemodynamic response can readily reproduce this effect (14), the ability to robustly reproduce and modulate this effect experimentally in healthy humans has proven elusive. Until these experiments are possible, this effect remains incompletely understood and not fully characterized.

#5: The post-undershoot

In contrast to the pre-undershoot, the post-undershoot is ubiquitous and has been studied extensively (15). Similar to the pre-undershoot, its origins are still widely debated. The basic observation is that following brain activation, the BOLD signal decreases and then passes below baseline for up to 40 seconds. The hypothesized reasons for this include: 1) a perseveration of elevated blood volume causing the amount of deoxyhemoglobin to remain elevated even though oxygenation and flow are back to baseline levels, 2) a perseveration of an elevated oxidative metabolic rate causing the blood oxygenation to decrease below baseline levels as the flow and total blood volume have returned to baseline states, 3) a post stimulus decrease in flow below baseline levels. A decrease in flow with steady state blood volume and oxidative metabolic rate would cause a decrease in blood oxygenation. Papers have been published arguing for, and showing evidence suggesting each of these three hypotheses, so the mechanism of the post undershoot, as common as it is, stays unresolved. It has also been suggested that if it is perhaps due to a decrease in flow, then this decrease might indicate a refractory period where neuronal inhibition is taking place (16). For now, the physiologic underpinnings of the post-undershoot remain a mystery.

#6: Long duration stimulation effects

This is a controversy that has long been resolved (17). In the early days of fMRI, investigators were performing the basic tests to determine if the response was a reliable indicator of neuronal activity and one of the tests was to determine if, with long duration steady state neuronal activation, the BOLD response remains elevated. A study by Kruger et al came out suggesting that the BOLD response habituated after 5 minutes (18). A counter study came out showing that with a flashing checkerboard on for 25 minutes, the BOLD response and flow response (measured simultaneously) remained elevated (19). It was later concluded that the stimuli in the first study was leading to some degree of attentional and neuronal habituation and not a long duration change in the relationship between the level of BOLD and the level of neuronal activity. Therefore, it is now accepted that as long as neurons are firing, and as long as the brain is in a normal physiologic state, the fMRI signal will remain elevated for the entire duration of activation.

#7: Mental Chronometry with fMRI.

A major topic of study over the entire history of fMRI has been how much temporal resolution can be extracted from the fMRI signal. The fMRI response is relatively slow, taking about two seconds to start to increase and, with sustained activation, takes about 10 seconds to reach a steady-state “on” condition. On cessation, it takes a bit longer to return to baseline – about 10 to 12 seconds, and has a long post-stimulus undershoot lasting up to 40 seconds. In addition to this slow response, it has been shown that the spatial distribution in delay is up to four seconds due to spatial variations in the brain vasculature.

Given this sluggishness and spatial variability of the hemodynamic response, it may initially seem that there wouldn’t be any hope in sub-second temporal resolution. However, the situation is more promising than one would expect. Basic simulations demonstrate that, assuming no spatial variability in the hemodynamic response, and given a typical BOLD response magnitude of 5% and a typical temporal standard deviation of about 1%, after 11 runs of 5 minutes each, a relative delay of 50 to 100ms could be discerned from one area to the next. However, the spatial variation in the hemodynamic response, is plus or minus 2 seconds depending on where one is looking in the brain and depends mostly on what aspect of the underlying vasculature is captured with each voxel. Large veins tend to have longer delays.

Several approaches have attempted to bypass or to calibrate the hemodynamic response. One way of bypassing the slow variable hemodynamic response problem is to modulate the timing of experiment, so the relative onset delays with task timing delays can be observed. This is using the understanding that in each voxel, the hemodynamic response is extremely well behaved and repeatable. Using this approach, relative delays of 500ms to 50 ms have been discerned (20-23). While these measures are not absolute, they are useful in discriminating which areas show delay modulations with specific tasks. This approach has been applied – to 500ms accuracy – to uncover the underlying dynamics and the relative timings of specific regions involved with word rotation and word recognition (23). Multivariate decoding approaches have been able to robustly resolve sub second (and sub-TR) relative delays in the hemodynamic response(24). By slowing down the paradigm itself, Formisano et al. (25)have been able to resolve absolute timing of mental operations down to the order of a second. The fastest brain activation on-off rate that has been able to be resolved has recently been published by Lewis et al.(26)and is in the range of 0.75 Hz. While this is not mental chronometry in the strict sense of the term, it does indicate an upper limit at which high-speed changes in neuronal activation may be extracted.

#8: Negative signal changes .

Negative BOLD signal changes were mostly ignored for the first several years of fMRI as researchers did not know precisely how to interpret them. After several years, with a growing number of negative signal change observations, the issue arose in the field, spawning several hypotheses to explain them. One hypothesis invoked a “steal” effect, where active regions received an increase in blood flow at the expense of adjacent areas which would hypothetically experience a decrease in flow. If flow decreases from adjacent areas, these areas would exhibit a decrease in BOLD signal but not actually be “deactivated.” Another hypothesis was that these were areas that were more active during rest, thus becoming “deactivated” during a task as neuronal activity was re-allocated to other regions of the brain. A third was that they represented regions that were actively inhibited by the task. While in normal healthy subjects, the evidence for the steal effects is scant, the other hypotheses are clear possibilities. In fact, the default mode network was first reported as a network that showed consistent deactivation during most cognitive tasks(27). This network deactivation was also seen in the PET literature(28). A convincing demonstration of neuronal suppression associated with negative BOLD changes was carried out by Shmuel et al (29)showing simultaneous decreased neuronal spiking and decreased fMRI signal in a robustly deactivated ring of visual cortex surrounding activation to a ring annulus. These observations seem to point to the idea that the entire brain is tonically active and able to be inhibited by activity in other areas through several mechanisms. This inhibition is manifest as a decrease in BOLD.

#9: Sources of resting state signal fluctuations

Since the discovery that the resting state signal showed temporal correlations across functionally related regions in the brain, there has been an effort to determine their precise origin as well as their evolutionary purpose. The predominant frequency of these fluctuations is in the range of 0.1 Hz, which was eye opening to the neuroscience community since, previously, most had not considered that neuronally-meaningful fluctuations in brain activity occurred on such a slow time scale. The most popular model for the source of resting state fluctuations is that spontaneously activated regions induce fluctuations in the signal. As measured with EEG, MEG, or ECoG, these spontaneous spikes or local field potentials occur across a wide range of frequencies. When this rapidly changing signal is convolved with a hemodynamic response, the resulting fluctuations approximate the power spectrum of a typical BOLD time series. Direct measures using implanted electrodes combined with BOLD imaging show a temporal correspondence of BOLD fluctuations with spiking activity(30). Recent work with simultaneous calcium imaging – a more direct measure of neuronal activation – has also shown a close correspondence both spatially and temporally with BOLD fluctuations(31), thus strongly suggesting that these spatially and temporally correlated fluctuations are in fact, neuronal. Mention of these studies is only the tip of a very large iceberg of converging evidence that resting state fluctuations are in fact related to ongoing, synchronize, spontaneous neuronal activity.

While the basic neurovascular mechanisms behind resting state fluctuations may be understood to some degree, the mystery of the origins and purpose of these fluctuations remains. To complicate the question further, it appears that there are different types of correlated resting state fluctuations. Some are related to the task being performed. Some may be related to brain “state” or vigilance. Some are completely insensitive to task or vigilance state. It has been hypothesized that the spatially broad, global fluctuations may relate more closely to changes in vigilance or arousal. Some are perhaps specific to a subject, and relatively stable across task, brain state, or vigilance state – reflecting characteristics of an individual’s brain that may change only very slowly over time or with disease. A recent study suggests that resting state networks, as compared in extremely large data sets of many subjects, reveal clear correlations to demographics, life style, and behavior (32).

Regarding the source or purpose of the fluctuations, some models simply state it’s an epiphenomenon of a network at a state of criticality – ready to go into action. The networks have to be spontaneously active to be able to transition easily into engagement. The areas that are typically most engaged together are resultantly fluctuating together during “rest.” In a sense, resting state may be the brain continually priming itself in a readiness state for future engagement. Aside from this issue there is the issue of whether or not there are central “regulators” of resting state fluctuations. Do the resting state fluctuations arise from individual nodes of circuits simply firing on their own or is there a central hub that sends out spontaneous signals to these networks to fire. There has also been growing evidence suggesting that this activity represents more than just a subconscious priming. The default mode network, for instance, has been shown to be central to cognitive activity as rumination and self-directed attention.

Work is ongoing in trying to determine if specific circuits have signature frequency profiles that might help to differentiate them. Work is also ongoing to determine what modulates resting state fluctuations. So far, it’s clear that brain activation tasks, vigilance state, lifestyle, demographics, disease and immediately previous task performance can have an effect. There are currently no definitive conclusions as to the deep origin and purpose of resting state fluctuations.

#10: Dead Fish (false positive) Activation.

In about 2006, a poster at the Organization for Human Brain Mapping was published by Craig Bennett, mostly as a joke (I think) – but with the intent to illustrate the shaky statistical ground that fMRI was standing on with regard to the multiple comparisons problem. This study was picked up by the popular media and went a bit viral. The study showed BOLD activation in a dead salmon’s brain. Here, it was clearly suggested that fMRI based on BOLD contrast has some problems if it shows activation where there should be none. In fact, it was a clear indication of false positives that can happen by chance even if the appropriate statistical tests are not used. The basic problem in creation of statistical maps is that the maps not appropriately normalized to “multiple comparisons.” It’s known, that purely by chance, if enough comparison is made (in this case it’s one comparison every voxel), then some voxels will appear to have significantly changed in signal intensity. Bonferroni corrections and false discovery rate corrections are almost always used and are all available in current statistical packages. Bonferroni is likely too conservative as each voxel is not fully independent. False discovery rate is perhaps closer to the appropriate test. When using these, false activations are minimized, however, they can still occur for other reasons. The structure of the brain and skull, having edges which can enhance any small motion or system instability, resulting in false positives. While this poster brought out a good point and was perhaps among the most cited fMRI works in the popular literature and blogs, it failed to convey a more nuanced and important message, that no matter what statistical test is used, the reality is that the signal and the noise are not fully understood, therefore all are actually approximations of truth, subject to errors. That said, a well-designed study with clear criteria and models of activation as well as appropriately conservative statistical tests, will minimize this false positive effect. In fact, it is likely that we are missing much of what is really going on by using over-simplified models of what we should expect of the fMRI signal.

A recent paper by Gonzalez-Castillo et al (33)showed that with a more open model of what to expect from brain activation, and 9 hours of averaging, nearly all grey matter becomes “active” in some manner. Does this mean that there is no null hypothesis? Likely not, but the true nature of the signal is still not fully understood, and both false positives and false negatives permeate the literature.

#11: Voodoo correlations and double dipping.

In about 2009, Vul et al published a paper that caused a small commotion in the field in that it identified a clear error in neuroimaging data analysis. It listed the papers – some quite high profile – that used this erroneous procedure that resulted in elevated correlations. The basic problem that was identified was that studies were performing circular analysis rather than pure selective analysis(34). A circular analysis is a form of selective analysis in which biases involved with the selection are not taken into account. Analysis is “selective” when a subset of data is first selected before performing secondary analysis on the selected data. This is otherwise known as “double dipping.” Because data always contain noise, the selected subset will never be determined by true effects only. Even if the data have no true effects at all, the selected data will show tendencies that they were selected for.

So, a simple solution to this problem is to analyze the selected regions using independent data (not data that were used to select the regions). Therefore, effects and not noise will replicate. Thus, the results will reflect actual effects without bias due to the influence of noise on the selection.

This was an example of an adroit group of statisticians helping to correct a problem as it was forming in fMRI. Since this paper, the number of papers published with this erroneous approach has sharply diminished.

#12: Global signal regression for time series cleanup

The global signal is obtained simply by averaging up the MRI signal intensity in every voxel of the brain for each time point. For resting state correlation analysis, global variations of the fMRI signal are often considered nuisance effects and are commonly removed(35, 36)by regression of the global signal against the fMRI time series. However the removal of global signal has been shown artifactually cause anticorrelated resting state networks in functional connectivity analyses (37). Before this was known, papers were published showing large anticorrelated networks in the brain and interpreted as large networks that were actively inhibited by the activation of another network. If global regression was not performed, these so called “anti-correlated” networks simply showed minimal correlation with – positive or negative – the spontaneously active network. Studies have shown that the removing the global signal not only induces negative correlation, but distorts positive correlations – leading to errors in interpretation (38). Since then the field has mostly moved away from global signal regression. However, some groups use it as it does clean up the artifactual signal to some degree.

The global signal has been studied directly. Work has shown that it is a direct measure of vigilance as assessed by EEG (39). The monitoring of the global signal may be an effective way to ensure that when the resting state data is collected, subjects are in a similar vigilance state – which can have a strong influence on brain connectivity (40). In general, the neural correlates of the global signal change fluctuations are still not fully understood, however, it appears that the field has reached a consensus that, as a pre-processing step, the global signal should not be removed. Simply removing the global signal will not only induce temporal artifacts that look like interesting effects but will remove potentially useful and neuronally relevant signal.

The global signal is obtained simply by averaging up the MRI signal intensity in every voxel of the brain for each time point. For resting state correlation analysis, global variations of the fMRI signal are often considered nuisance effects and are commonly removed(35, 36)by regression of the global signal against the fMRI time series. However the removal of global signal has been shown artifactually cause anticorrelated resting state networks in functional connectivity analyses (37). Before this was known, papers were published showing large anticorrelated networks in the brain and interpreted as large networks that were actively inhibited by the activation of another network. If global regression was not performed, these so called “anti-correlated” networks simply showed minimal correlation with – positive or negative – the spontaneously active network. Studies have shown that the removing the global signal not only induces negative correlation, but distorts positive correlations – leading to errors in interpretation (38). Since then the field has mostly moved away from global signal regression. However, some groups use it as it does clean up the artifactual signal to some degree.

The global signal has been studied directly. Work has shown that it is a direct measure of vigilance as assessed by EEG (39). The monitoring of the global signal may be an effective way to ensure that when the resting state data is collected, subjects are in a similar vigilance state – which can have a strong influence on brain connectivity (40). In general, the neural correlates of the global signal change fluctuations are still not fully understood, however, it appears that the field has reached a consensus that, as a pre-processing step, the global signal should not be removed. Simply removing the global signal will not only induce temporal artifacts that look like interesting effects but will remove potentially useful and neuronally relevant signal.

#13: Motion Artifacts

Functional MRI is extremely sensitive to motion, particularly in voxels that have a large difference in signal intensity relative to adjacent voxels. Typical areas that manifest motion effects are edges, sinuses and ear canals where susceptibility dropout is common. In these areas, even a motion of a fraction of a voxel can induce a large fractional signal change, leading to incorrect results. Motion can be categorized into task correlated, slow, and pulsatile. Work has been performed over the past 27 years to develop methods to avoid or eliminate in acquisition or post processing, motion induced signal changes. In spite of this effort, motion is still a major challenge today. The most difficult kind of motion to eliminate is task-correlated motion that occurs when a subject tenses up or moves during a task or strains to see a display. Other types of motion include slow settling of the head during a scan, rotation of the head, swallowing, pulsation of blood and csf, and breathing induced motion-like effects.

Typical correction for motion is carried out by the use of motion regressors that are obtained by most image registration software. An a hoc method for dealing with motion – that can be quite effective – include visual inspection of the functional images and manually choosing time series signals that clearly contain motion effects that can then be regressed out or “orthogonalized.” Other approaches include image registration and time series “scrubbing.” Scrubbing involves automated detection of “outlier” images and eliminating them from analysis. Other ways of working around motion have included paradigm designs that involve brief tasks such that any motion from the task itself is able to be identified as a rapid change whereas a change in the hemodynamic response is slow, thus allowing the signals to be separable by their temporal signatures.

In recent years, an effort has been made to proactively reduce motion effects by tracking optical sensors positioned on the head, and then feeding the position information back to the imaging gradients such that the gradients themselves are slightly adjusted to maintain a constant head position through changing the location or orientation of the imaging volume. The primary company selling such capability is KinetiCor.

A more direct strategy for dealing with motion is the implementation of more effective methods to keep the head rigid and motionless. This approach has included bite bars and plastic moldable head casts. These have some effectiveness in some subjects but run the risk of being uncomfortable – resulting in abbreviated scanning times or worse, more motion due to active repositioning during the scan due to discomfort of the subject.

Aside from the problem of motion of the head, motion of the abdomen, without any concomitant head motion, can have an effect on the MRI signal. With each breath, the lungs fill with air, altering the difference in susceptibility between the signal in the chest cavity and the outside air. This alteration has an effect on the main magnetic field that can extend all the way into the brain, leading to breathing induced image distortions and signal dropout. This problem is increased at higher fields where the effects of susceptibility differences between tissue are enhanced. Possible solutions to this problem are direct measurement of the magnetic field in the proximity to the head using a “field camera” such as the one sold by the Swiss company called Skope, and then perhaps using these dynamically measured field perturbations as regressors in post processing or by feeding this signal to the gradients and shims prior to data collection in an attempt to compensate.

In resting state fMRI, motion is even more difficult to identify and remove as slow motion or breathing artifacts may have similar temporal signatures. Also, if there are systematic differences between intrinsic motion in specific groups such as children or those with Attention Deficit Disorder (ADD), then interpretation of group differences in resting state fMRI results is particularly problematic as the degree of motion can vary with the degree to which individuals suffer from these disorders.

In high resolution studies, specifically looking at small structures at the base of the brain, motion is a major confound as with each cardiac cycle, the base of the brain physically moves. Solutions to this have included cardiac gating and simple averaging. Gating is promising, however signal changes associated with the inevitably varying TR, which would vary with the cardiac cycle length, need to be accounted for by so far imperfect post processing approaches.

A novel approach to motion elimination has been with the use of multi-echo EPI, allowing the user to differentiate BOLD effects from non-BOLD effects based on the signal fit to a T2* change model.

Another motion artifact is that which arises from through plane movement. If the motion is such that a slightly different head position causes a slightly different slice to be excited (with an RF pulse), then some protons will not experience that RF pulse and will have magnetization that is no longer in equilibrium (achieved by a constant TR). If this happens, even if the slice position is corrected, there will be some residual non-equilibrium signal remaining that will take a few TR’s to get back into equilibrium. Methods have been put forward to correct for this, modeling the T1 effects of the tissue, however these can also eliminate the BOLD signal changes, so this problem still remains.

Lastly, apparent motion can also be caused by scanner drift. As the scanner is running, gradients and gradient amplifiers and RF coils can heat up, causing a drift in the magnetic field as well as the resonance frequency of the RF coils, causing a slow shifting of the image location and quality of image reconstruction. Most vendors have implemented software to account for this, but it is not an ideal solution. It would be better to have a scanner that does not have this instability to begin with.

In general, motion is still a problem in the field and one which every user still struggles with, however it is becoming better managed as we gain greater understanding of the sources of motion, the spatial and temporal signatures of motion artifacts, and the temporal and spatial properties of the BOLD signal itself.

There’s a payoff waiting once complete elimination of motion and non-neuronal physiologic fluctuations in solved. The upper temporal signal to noise of an fMRI time series is no higher than about 120/1 due to physiologic noise setting an upper limit. If this noise source were to be eliminated, then the temporal signal to noise ratio would only be limited by coil sensitivity and field strength, perhaps allowing fMRI time series SNR values to approach 1000/1. This improvement would transform the field.

#14: Basis of the decoding signal

Processing approaches have advanced beyond univariate processing that involves comparison of all signal changes against temporal model with the assumption that there is no spatial relationship between voxels or even blobs of activation. Instead, fine-grained voxel wise patterns of activation in fMRI have been shown to carry previously unappreciated information regarding specific brain activity associated with a region. Using multivariate models which compare the voxel wise pattern of activity associated with each stimulus, detailed information related to such discriminations as visual angle and face identity, as well as object category have been differentiated(41). A model to explain these voxel-specific signal change patterns hypothesizes that while each voxel is not small enough to capture the unique pool of neurons that are selectively activated by specific stimuli, the relative population of neurons that are active in a voxel causes a modulation in the signal that, considered with the array of other uniquely activated vowels, makes a pattern that, while having limited meaningful macroscopic topographic information, conveys information as to what activity the functional area is performing. The proposed mechanism of these multi-voxel signal changes of sub voxel activations, taken as a pattern, convey useful and unique functional information. Another proposed mechanism for these changes is that there remain subtle macroscopic changes that occur.

One study has attempted to answer the question of whether the source of pattern effect mapping is multi-voxel or sub-voxel by testing the performance of pattern effect mapping as the activation maps were spatially smoothed (42). The hypothesis was that if the information were sub-voxel and varied from voxel to voxel, then the performance of the algorithm would diminish with spatial smoothing. If the information was distributed at a macroscopic level, smoothing would improve detection. This study showed that both voxel-wise and multi-voxel information contributed in different areas to the multivariate decoding success, thus while the decoding signal is robust, the origins, are, as with many other fMRI signals, complicated and varying.

#15: Signal change but no neuronal activity?

Several years ago, Sirotin and Das reported that they observed a hemodynamic response where no neuronal activity was present. They recorded simultaneously hemodynamic and electrical signals in an animal model during repeated and expected periodic stimuli. When the stimulus was removed when the animal was expecting it, then there was no neuronal firing but a hemodynamic response remained (43). Following this controversial observation, there were some papers that disputed their claim, suggesting that, in fact, there was, in fact, very subtle electrical activity still present. The hemodynamic response is known to consistently over-estimate neuronal activity for very low level or very brief stimuli. This study appears to be an example of just such an effect, despite what the authors claimed. While there was no clear conclusion to this controversy, the study was not replicated. In general, a claim such as this is extremely difficult to make as it is nearly impossible to show that that something doesn’t exist when one is simply unable to detect it.

#16: Curious relationships to other measures

Over the years the hemodynamic response magnitude has been compared with other neuronal measures such as EEG, PET, and MEG. These measures show comparable effects, yet at times, there have been puzzling discrepancies reported. One example is the following paper (44), comparing the visual checkerboard flashing rate dependence of BOLD and of MEG. At high spatial stimulus frequencies, the flashing visual checkerboard showed similar monotonic relationships between BOLD and MEG. At low spatial frequency, the difference between BOLD and MEG was profound – showing a much stronger BOLD signal than MEG signal. The reasons for this discrepancy are still not understood yet studies like these are important for revealing differences where it is otherwise easy to assume that they are measuring very similar effects.

#17: Contrast mechanisms: spin-echo vs gradient-echo.

Spin-echo sequences are known to be sensitive to susceptibility effects from small compartments and gradient-echo sequences are known to be sensitive to susceptibility effects from compartments of all sizes(45). From this observation it has been incorrectly inferred that spin-echo sequences are be sensitive to capillaries rather than large draining veins. The mistake in this inferences is that the blood within draining veins is not taken into account. Within draining veins are small compartments – red blood cells. So, while spin-echo sequences may be less sensitive to large vessel extravascular effects, they are still sensitive to the intravascular signal in vessels where the blood signal is present.

There have been methods proposed for removing the intravascular signal, such as the application of diffusion gradients that null out any rapidly moving spins. However, spin-echo sequences have lower BOLD contrast than gradient-echo by at least a factor of 2 to 4, and with the addition of diffusion weighting, the contrast almost completely disappears.

Another misunderstanding is that spin-echo EPI is truly a spin-echo. EPI takes time to form an image, having a long “readout” window – at least 20 ms whereas with spin-echo sequences there is only a moment when the echo forms. All other times, the signal is being refocused by gradient-echoes – as with T2* sensitive gradient echo imaging. Therefore, it is nearly impossible in EPI sequences to obtain a perfect spin-echo contrast. Most of spin-echo EPI contrast is actually T2* weighted. “Pure” spin-echo contrast is where the readout window is isolated to just the echo – only obtainable with multi-shot sequences. However, even at high field strengths, spin-echo sequences are considerably less sensitive to BOLD contrast than gradient-echo sequences.

There is hope for “pure” spin-echo sequences at very high field might be effective in eliminating large vessel effects as, at high field, blood T2 rapidly shortens, and therefore the intravascular signal contributes minimally to the functional contrast. Spin-echo sequence have been used at 7T to visualize extremely fine functional activation structures such as ocular dominance and orientation columns (46, 47).

For these reasons, spin-echo sequences have not caught on for all but a small handful of fMRI studies at very high field. If high field is used and a short readout window is employed, then spin-echo sequences may be one of sequences of choice, along with the blood volume sensitive sequence, VASO – for spatial localization of activation.

#18: Contrast mechanisms: SEEP contrast.

Over the years, after tens of thousands of observations of the fMRI response and the testing of various pulse sequence parameters, some investigators have claimed that the signal does not always behave in a manner that is BOLD-like. One interesting example appeared about 15 years ago. Here Stroman et al.(48)was investigating the spinal cord and failed to find a clear BOLD signal. On using a T2-weighted sequence they claimed to see a signal change that did not show a TE dependence and was therefore not T2-based. Rather they claimed that it was based proton density changes – but not perfusion. It was also curiously most prevalent in the spinal cord.

The origin of this signal has not been completely untangled and SEEP contrast has disappeared from the current literature. Those investigating spinal cord activation have been quite successful using standard BOLD contrast.

#19: Contrast Mechanisms: Activation-Induced Diffusion Changes.

Denis LeBihan is a pioneer in fMRI and diffusion imaging. In the early 1990’s, he was an originator of diffusion tensor mapping(49). Later, he advanced the concept of intra-voxel incoherent motion (IVIM)(50, 51). The idea behind IVIM is that with extremely low levels of diffusion weighting, a pulse sequence may become selectively sensitized to randomly, slowly flowing blood rather than free water diffusion. The pseudo-random capillary network supports blood flow patterns that may resemble, imperfectly, rapid diffusion, and thus could be imaged as rapid diffusion using gradients sensitized to this high diffusion rate of random flow rather than pure diffusion. This concept was advanced in the late 1980’s and excited the imaging community in that it suggested that, if MRI were sensitive to capillary perfusion, it could be sensitive to activation-induced changes in perfusion. This contrast mechanism, while theoretically sound, was unable to be clearly demonstrated in practice as relative blood volume is only 2% and sensitization of diffusion weighting to capillary perfusion also, unfortunately, sensitizes the sequences to CSF pulsation and motion in the brain.

Le Bihan emerged several years later with yet another potential functional contrast that is sensitive not to hemodynamics but, hypothetically to activation-induced cell swelling. He claimed that diffusion weighting revealed that on activation, measurable decreases in diffusion coefficient occur in the brain. The proposed mechanism is that with neuronal activation, active neurons swell, thus increasing the intracellular water content. High levels of diffusion weighting are used to detect subtle activation-induced shifts in neuronal water content. A shift in water distribution from extracellular space which has slightly higher diffusion coefficient, to intracellular space which slightly lower diffusion coefficient, would cause an increase in signal in a highly diffusion weighted sequences.

While LeBihan has published these findings(52, 53), the idea has not achieved wide acceptance. First, the effect itself is relatively small and noisy, if present at all. Secondly, there have been papers published demonstrating that the mechanism behind this signal is vascular rather than neuronal (54). Lastly, and perhaps most importantly, many groups, including my own, have tried this approach, coming up with null results. If the technique gives noisy and inconsistent results, it is likely not going to compete with BOLD, regardless of how selective it is to neuronal effects. Of course, it’s always worthwhile to work on advancing such methodology!

#20: Contrast mechanisms: Neuronal Current Imaging.

In the late 90’s it was proposed that MRI is theoretically directly sensitive to electric currents produced by neuronal activity. A current carried by a wire is known to set up magnetic fields around the wire. These magnetic fields when superimposed on the primary magnetic field of the scanner can cause NMR phase shifts or, if the wires are very small and randomly distributed, an NMR phase dispersion – or a signal attenuation. In the brain, dendrites and white matter tracts behave as wires that carry current. Basic models have calculated that the field in the vicinity of these fibers can be as high as 0.2 nT(55). MEG in fact, does detect these subtle magnetic fields as they fall off near the skull. At the skull surface the magnetic fields produced are on the order of 100 fT, inferring that at the source, they are on the order of 0.1 nT.

Over the last 15 years, work has continued around the world toward the goal of using MRI to detect neuronal currents. This work includes the attempts to observe rapid activation-induced phase shifts and magnitude shifts in susceptibility weighted sequences as the superimposed field distortion would cause either a phase shift or dephasing, depending on the geometry. Using these methods, no conclusive results have been reported in vivo. Other attempted methods include “Lorentz” imaging(56). Here the hypothesis is that when current is passing through a nerve within a magnetic field, a net torque is produced, causing the nerve to move just a small amount – potentially detectable by well-timed diffusion weighting. Again, no clear results have emerged. More recent approaches are based on the hypothesis that the longitudinal relaxation properties of spins may be affected if in resonance with intrinsic resonances in the brain such as alpha (10Hz) frequencies(57, 58). Spin-locking approaches that involve adiabatic pulses at these specific frequencies aim to observe neuronal activity based changes in the resonance of the NMR tissue. In such manner, maps of predominant oscillating frequency may be made. Again, such attempts have resulted in suggestive but not conclusive results.

Many are working on methods to detect neuronal currents directly, however, by most calculations, the effect is likely an order of magnitude to small. Coupled with the fact that physiologic noise and even BOLD contrast would tend to overwhelm neuronal current effects, the challenge remains daunting. Such approaches would require extreme acquisition speed (to detect transient effects that cascade over 200 ms throughout the brain), insensitivity to BOLD effects and physiologic noise, yet likely an order of magnitude higher sensitivity overall.

#21: Contrast mechanisms: NMR phase imaging.

The idea of observing phase changes rather than magnitude changes induced by oxygenation changes is not a new one. Back in the early 90’s it was shown that large vessels show a clear NMR phase change with activation. It was suggested that this approach may allow the clean separation of large vessel from tissue effects when interpreting BOLD. In general, it is known that large, relative to a voxel size, changes in susceptibility induce a net phase shift as well.

The concept of observing phase changes has been revisited. Studies have suggested that large activated regions, while mostly showing magnitude changes, may act as a single large susceptibility perturber and induce a large shift in phase. These studies suggest that use of both phase and magnitude information would boost both sensitivity and specificity. This approach has not yet caught on in the field. Perhaps one reason for it not catching on is that vendors typically only provide magnitude reconstruction of the scanner data. Most users simply don’t have access to NMR phase information. It may also turn out that any gains are only very small, thus making the extra effort in obtaining double the data and spending twice the time on pre and post processing not worth the effort.

#22: fMRI for Lie Detection

Over the past 20 years, fMRI has provided evidence that the brain of someone who is lying is active in distinctly different ways than that which is telling the truth. There is additional prefrontal and parietal activity with story fabrication(59). There have been papers that have demonstrated this effect in many different ways. In fact, there exist studies that show not only that lie detection is possible but extraction of the hidden knowledge of the truth is also possible using fMRI(60).

Because of this success, companies touting MRI-based lie detection services have cropped up (e.g. No Lie MRI— http://noliemri.com/ and CEPHOS— http://www.cephoscorp.com/) . The problem with this is that lie detection has never been fully tested in motivated and at times psychopathic criminals and negative results are more common than not. Inconclusive fMRI based lie detection results, if allowed in court, could potentially bias favor on the defense because, a negative or inconclusive result would provide suggestion that the individual is innocent.

The difference between research success and success or readiness for implementation in public use illustrates much of what the rest of fMRI faces. In real world applications, there are many variables which make fMRI nearly uninterpretable. Generalizing from group studies or highly controlled studies on motivated volunteers to individual studies in patients or criminals is extremely challenging to say the least.

Nevertheless, in spite of scientific, legal, ethical, operational, and social hurdles, machine learning and classification methods may ultimately prove capable in interpreting individual subject activation in association with lie detection with actual criminals or other real-world situations. There may in fact be regions of the brain or patterns of activity that tell truth from lies regardless of whether the person is a psychopathic hardened criminal or a motivated college student volunteer. No one has done those comparison studies yet.

#23: Does Correlation Imply Connectivity?

In resting state and sometimes task based fMRI, the correlation between voxels or regions is calculated. A growing trend is to replace the word correlation with connectivity. The assumption is that a high temporal correlation between disparate regions of the brain directly implies a high level of functional connectivity. It’s also assumed that any change in correlation between these regions implies that there is a corresponding change in connectivity. As a first approximation, these statements may be considered true, however there are many situations in which they are not true.

First, other processes can lead to a high correlation between disparate regions. Bulk movement, cardiac pulsation, and respiration are three primary sources of artifactual correlation. For the most part these are well dealt with – as motion is relatively easily identified and cardiac pulsation is at a distinct frequency (1Hz) that is fortunately far from resting state correlation frequencies (0.1 Hz). However aliased cardiac frequencies and respiration-induced correlations present a bit more of a challenge to remove. Respiration also can create signal changes that show T2* changes, so multi-echo sequences optimized to separate BOLD from non-BOLD effects would be less effective in removing respiration effects. Respiration is however identifiable by the fact that it is more spatially diffuse than correlations between distinct regions. This separation based on spatial pattern and spatial diffusivity is still extremely difficult to perform robustly and without error.

Modulations in correlation can occur for a number of reasons. First, if, when looking at a pair of regions that show correlation, if both signals contain noise (as they all do), an increase in the amplitude of one signal will naturally increase the correlation value, but no “real” increase in connectivity likely occurred. Likewise, if the noise in one or both of the signals is reduced, then, again, there will be a measurable reduction in correlation but likely no change in actual functional connectivity. If the relative latency or shape of one of the responses changes, then a change in correlation will occur without a change in connectivity perhaps. Let’s say an additional frequency was added from another oscillating signal that has nothing to do with the signal driving the “connectivity” between the two regions. If this happens, then again, the correlation between the two signals will be reduced without the connectivity between the regions being altered.

All of the above issues have not been addressed in any systematic manner as we are still in the early days of just figuring out the best ways to cleanly and efficiently extract correlation data. In the future, in order to make more meaningful interpretations of these signals, we will need to control for a the potentially confounding effects mentioned above.

#24: The clustering conundrum.

In 2016, a bombshell paper by Eklund et al (61)identified an error in most of the statistical fMRI packages, including SPM, FSL, and AFNI. Quoting a part of the abstract of that paper:

“ In theory, we should find 5% false positives (for a significance threshold of 5%), but instead we found that the most common software packages for fMRI analysis (SPM, FSL, AFNI) can result in false-positive rates of up to 70%. These results question the validity of a number of fMRI studies and may have a large impact on the interpretation of weakly significant neuroimaging results.”

The results showed that the packages correctly inferred statistical significance when considering independent voxels, however, when considering clusters of voxels as a single activation – as most activations are considered “clusters” or spatially correlated activations – the estimations of cluster sizes, smoothness, or the statistical threshold for what constitutes a “significantly activated” cluster were incorrect, leading to incorrectly large clusters.

The implication of their paper was that for the past 15 years, these commonly used packages have overestimated activation extent. For well-designed studies with highly significant results, this would have virtually no effect on how results are interpreted. The same conclusions would be likely have been made. Most studies’ conclusions do not rely on absolute cluster size for their conclusions. Instead they draw conclusions based on the center of mass of activation, or whether a region was activated or not. Again, these studies would not be significantly affected.

Perhaps the greatest implications might be for pooled data in large data sets. If incorrectly large clusters are averaged across thousands of studies, then perhaps errors in interpretation of the extent of activation my crop up.

Unfortunately, the paper also went on to make a highly emotive statement: “These results question the validity of some 40,000 fMRI studies and may have a large impact on the interpretation of neuroimaging results.” Later in a correction, they removed the figure of 40,000 papers. Unfortunately, the media picked up on this figure with the sensational statement that most fMRI studies are “wrong,” which in itself is completely untrue by most definitions of the word “wrong.” The studies may have simply slightly overestimated the size of activation. If a paper relied on this error to reach a conclusion, it was naturally treading on thin statistical ice in the first place and would likely be in question anyway. Most activation (and most of the voxels in these clusters found) are highly significant.

After many popular articles raised concern in the public, the issue eventually died down. Most of the scientists in the field who understood the issue were completely unfazed as the slightly larger clusters did not have any influence on the conclusions drawn by their papers or papers that they relied on for guiding their work.

This brings up an issue related to statistical tests. All statistical tests involve estimates on the nature of the signal and noise, so, by definition are always somewhat “wrong.” They are close enough however. So much more goes into the proper interpretation of results. Most seasoned fMRI practitioners have gained experience in not over interpreting aspects of the results that are not extremely robust.

While the Eklund paper has performed a service to the field by alerting it to a highly propagated error, it also raised false concerns about the high-level of robustness and reproducibility of fMRI in drawing inferences about brain activation. Yes, brain activation extent has been slightly overestimated, but no, most of the papers produced using these analyses need to change in any manner, their conclusions.

#25: The issue of reproducibility

While fMRI is a robust and reproducible method, there is room for improvement. In science in general, it’s been suggested that up to 70% of published results have failed to be reproduced. The reasons for this high fraction may be in part related to what we mean by “successfully reproduced” as well as pressure to publish findings that push the limits of what may be a reasonable level of interpretation. Some might argue that this number is an indication of the health of the scientific process by which a study’s result either lives or dies based on whether it is replicated. If a study is not able to be replicated, the conclusions generally fade away from acceptance.

One researcher, Russ Poldrack, of Stanford is spearheading an effort to increase transparency and reproducibility of fMRI data as he heads the Stanford Center for Reproducible Science. The goal is to increase further the reproducibility of fMRI and therefore move the field towards a more full realization of its potential and less wasted work in the long run. He specifically is encouraging more replication papers, more shared data and code, more “data papers” contributing a valuable data set that may be re-analyzed, and an increased number of “registered studies” where the hypotheses and methods are stated up front before any data are collected. All of these approaches will make the entire process of doing science with fMRI much more transparent and able to be better leveraged by a growing number of researchers as they develop better processing methods or generate more penetrating hypotheses.

#26: Dynamic Connectivity Changes

The most recent “controversy” in the field of fMRI revolves around the interpretation of resting state scans. Typically, in the past, maps of resting state connectivity were created from entire time series lasting up to 30 minutes in length. The assumption was that brain connectivity remained somewhat constant over these long time periods. There is evidence that this assumption is not correct. In fact, it’s been shown that the connectivity profile of the brain changes fluidly over time. Today, fMRI practitioners now regard individual fMRI scans as rich 4D datasets with meaningful functional connectivity dynamics (dFC) requiring updated models able to accommodate this additional time-varying dimension. For example, individual scans today are often described in terms of a limited set of recurring, short-duration (tens of seconds), whole-brain FC configurations named FC states. Metrics describing their dwell times, ordering and frequency of transitions can then be used to quantify different aspects of empirically observed dFC. Many questions remain both about the etiology of empirically observed FC dynamics; as well as regarding the ability of models, such as FC states, to accurately capture behavioral, cognitive and clinically relevant dynamic phenomena.

Despite reports of dFC in resting humans, macaques and rodents, a consensus does not exist regarding the underlying meaning or significance of dFC while at rest. Those who hypothesize it to be neuronally relevant have explored resting dFC in the context of consciousness, development and clinical disorders. These studies have shown how the complexity of dFC decreases as consciousness levels decline; how dynamic inter-regional interactions can be used to predict brain maturity, and how dFC derivatives (e.g., dwell times) can be diagnostically informative for conditions such as schizophrenia, mild cognitive impairment, and autism; to name a few.

Yet, many others have raised valid concerns regarding the ability of current dFC estimation methods to capture neuronally relevant dFC at rest. These concerns include lack of appropriate null models to discern real dynamics from sampling variability, improper pre-processing leading to spurious dynamics, and excessive temporal smoothing (a real concern for sliding window techniques used to estimate FC states) that hinder our ability to capture sharp and rapid transitions of interest (62). Finally, some have even stated that resting dFC is primarily a manifestation of sampling variability, residual head motion artifacts, and fluctuations in sleep state; and as such, mostly irrelevant.

One reason causing such discrepant views is that it is challenging to demonstrate the potential cognitive correlates of resting dFC, given the unconstrained cognitive nature of rest and scarcity of methods to infer the cognitive correlates of whole-brain FC patterns. When subjects are instructed to quietly rest, retrospective reports demonstrate that subjects often engage in a succession of self-paced cognitive processes including inner speech, musical experience, visual imagery, episodic memory recall, future planning, mental manipulation of numbers, and periods of heightened somatosensory sensations. Reconfigurations of FC patterns during rest could, to some extent, be a manifestation of this flow of covert self-paced cognition; even if other factors (e.g., random exploration of cognitive architectures, fluctuations in autonomic system activity and arousal levels, etc.) also contribute.

Research is ongoing in fMRI to determine the neural correlates of dynamic connectivity changes, to determine the best methods for extracting this rich data, as well as to determine how this information may be used to better understand ongoing cognition in healthy and clinical populations and individuals.

—

In conclusion, other interesting controversies also come to mind – such as BOLD activation in white matter, BOLD tensor mapping, the problem of reverse inference from fMRI data, the difference between mapping connectivity (ahem..correlation) vs mapping fMRI magnitude, inferring brain region causation from different brain regions during activation using fMRI, and other non-BOLD contrast mechanisms such as temperature or elasticity. Challenges also come to mind – including the robust extraction of individual similarities and differences from fMRI data, the problem of how to best parcellate brains, the creation of fMRI-based “biomarkers,” and potential utility of gradient coils.

The field of fMRI has advanced and further defined itself through many of these controversies. In my opinion, these indicate that the field is still growing and is quite robust and dynamic. Also, in the end, a consensus is usually reached – thus pushing the understanding forward. Controversies are healthy and welcomed. Let’s keep them coming!

References

If, how, and when fMRI goes clinical

Fri, 18 May 2018 14:31:00 GMT

#CCNeuro asks: “How can we find out how the brain works?”

Sun, 03 Sep 2017 13:04:00 GMT

The organizers of the upcoming conference Cognitive Computational Neuroscience (#CCNeuro) have done a very cool thing ahead of the meeting. They asked their keynote speakers the same set of 5 questions, and posted their responses on the conference blog .

The first of these questions is “ How can we find out how the brain works?”. In addition to recommending reading the insightful responses of the speakers, I offer here my own unsolicited suggestion.

A common theme among the responses is the difficulty posed by the complexity of the brain and the extraordinary expanse of scales across which it is organized.

The most direct approach to this challenge may be to focus on the development of recording technologies to measure neural activity that more and more densely span the scales until ultimately the entire set of neural connections and synaptic weights is known. At that point the system would be known but not understood.

In the machine learning world, this condition (known but not understood) is just upon us with AlphaGo and other deep networks. While it has not been proven that AlphaGo works like a brain, it seems close enough that it would be silly not to use as a testbed for any theory that tries to penetrate the complexity of the brain a system that has human level performance in a complex task, is perfectly and noiselessly known, and was designed to learn specifically because we could not make it successful by programming it to execute known algorithms (contrast Watson).

Perhaps the most typical conceptual approach to understanding the brain is based on the idea (hope) that the brain is modular in some fashion, and that models of lower scale objects such as cortical columns may encapsulate their function with sufficiently few parameters that the models can be built up hierarchically and arrive at a global model whose complexity is in some way still humanly understandable, whatever that means.

I think that modularity, or something effectively like modularity is necessary in order to distill understanding from the complexity. However, the ‘modularity’ that must be exploited in understanding the brain will likely need to be at a higher level of abstraction than spatially contiguous structures such as columns, built up into larger structures. The idea of brain networks that can be overlapping is already such an abstraction, but considering the density of long range connections witnessed by the volume of our white matter, the distributed nature of representations, and the intricate coding that occurs at the individual neuron level, it is likely that the concept of overlapping networks will be necessary all the way down to the neuron, and that the brain is like an extremely fine sparse sieve of information flow, with structure at all levels, rather than a finite set of building blocks with countable interactions.

Review of “Incognito: The Secret Lives of the Brain” by David Eagleman

Tue, 14 Mar 2017 14:50:00 GMT

I recently blogged about a meeting at Stanford on wearable technology. One of the best talks at this conference was by David Eagleman – all about the “umwelt” or how we experience the world through our relatively limited senses, as well as how we may expand and enhance our umwelt with devices that convert previously unperceived information to sensory experience. The moment his talk was finished, I bought two of his books – it was that good. This book was one of them.

He summarizes his book early on as follows: “Your consciousness is like a tiny stowaway on a transatlantic steamship, taking credit for the journey without acknowledging the massive engineering underfoot. This book is about that amazing fact: how we know it, what it means, and what it explains about people, markets, secrets, strippers, retirement accounts, criminals, artists, Ulysses, drunkards, stroke victims, gamblers, athletes, bloodhounds, racists, lovers, and every decision you’ve ever taken to be yours.” (p.4)

He starts with an example of why we may find another person attractive. We may verbalize characteristics but these fall short. The deep mechanisms for attraction are hardwired into us – below our conscious processes. We have surprisingly little access or control of these mechanisms. Other examples include how experts perform well-practiced movements. Again, these actions may have started as conscious processes, but over time, they have become automatic. Playing a piano well depends on repeated practice moving the neural processes involved with the actions from slow and awkward conscious space to unconscious execution. He also claims that unconscious processes are behind our of our best ideas and insights.

Eagleman delves into our perceptual world and all that we don’t experience – describing how our senses are exquisitely tuned to information critical to our survival and how what we experience is a fine sliver of possible sensations. He then takes this further to draw the comparison to the tiny sliver of mental processes that we have access to: “By analogy to your perception of the world, your mental life is built to range over a certain territory, and it is restricted from the rest. There are thoughts you cannot think. You cannot comprehend the sextillion stars of our universe, nor picture a five-dimensional cube, nor feel attracted to a frog. If these examples seem obvious (Of course I can’t!), just consider them in analogy to seeing in infrared, or picking up on radio waves, or detecting butyric acid as a tick does. Your “thought umwelt” is a tiny fraction of the “thought umgebung.” (p.82)

So just as our senses are limited, so is our consciousness – it has many blind spots.

Drawing upon an eye-opening experiment that he has the reader perform, he gives an example of our social hardwiring that we are not consciously aware of. To illustrate how our brains are best at social interactions but less so in logic, he first shares a logic puzzle that when posed without a social context, most get wrong, but when posed in a social framework (i.e. detecting cheaters) is solved easily. It’s the same problem presented in two contexts – one which is alien to our brain (pure logic) and one which we evolved to master (social situations): “The brain cares about social interaction so much that it has evolved special programs devoted to it: primitive functions to deal with issues of entitlement and obligation. In other words, your psychology has evolved to solve social problems such as detecting cheaters— but not to be smart and logical in general.” (p.86).

Within his discussion of unconscious processes he includes some classic insights into known brain functions that are better described than anywhere I’ve seen in the literature. In one example he eloquently describes how the amygdala is invoked to store emotionally charged memories. “For instance, under normal circumstances, your memories of daily events are consolidated (that is, “cemented in”) by an area of the brain called the hippocampus. But during frightening situations— such as a car accident or a robbery— another area, the amygdala, also lays down memories along an independent, secondary memory track. Amygdala memories have a different quality to them: they are difficult to erase and they can pop back up in “flashbulb” fashion— as commonly described by rape victims and war veterans. In other words, there is more than one way to lay down memory.” (p.126)

Also included is perhaps the clearest description of one of the more famous cognitive neuroscience experiments of all time – and still the best example of the “inference engine” that I know. It’s worth quoting in full here (bold print is my own): “Not only do we run alien subroutines; we also justify them . We have ways of retrospectively telling stories about our actions as though the actions were always our idea. As an example at the beginning of the book, I mentioned that thoughts come to us and we take credit for them (“I just had a great idea!”), even though our brains have been chewing on a given problem for a long time and eventually served up the final product. We are constantly fabricating and telling stories about the alien processes running under the hood. To bring this sort of fabrication to light, we need only look at another experiment with split-brain patients. As we saw earlier, the right and left halves are similar to each other but not identical. In humans, the left hemisphere (which contains most of the capacity to speak language) can speak about what it is feeling, whereas the mute right hemisphere can communicate its thoughts only by commanding the left hand to point, reach, or write. And this fact opens the door to an experiment regarding the retrospective fabrication of stories. In 1978, researchers Michael Gazzaniga and Joseph LeDoux flashed a picture of a chicken claw to the left hemisphere of a split-brain patient and a picture of a snowy winter scene to his right hemisphere. The patient was then asked to point at cards that represented what he had just seen. His right hand pointed to a card with a chicken, and his left hand pointed to a card with a snow shovel. The experimenters asked him why he was pointing to the shovel. Recall that his left hemisphere (the one with the capacity for language), had information only about a chicken, and nothing else. But the left hemisphere, without missing a beat, fabricated a story: “Oh, that’s simple. The chicken claw goes with the chicken, and you need a shovel to clean out the chicken shed. When one part of the brain makes a choice, other parts can quickly invent a story to explain why.” (pp.133-134)

I personally find this story so important to explain so much of human behavior. We grow up and live in an environment which is constantly shaping our beliefs – mostly at a level beneath our awareness, and then “without missing a beat” we justify our beliefs so quickly with a seemingly rational explanation. To me this is the foundation of some of the most deep divisions in our world and perhaps the source of so many conflicts. We can see it happening today. Perfectly intelligent people have wildly divergent beliefs that cannot be moved no matter how rational the arguments are on either side. We tend to talk right by each other because no one is fully aware of the true sources of our deeply held beliefs, therefore cannot find the verbal/rational leverage to change them.

“ As Gazzaniga put it, ‘These findings all suggest that the interpretive mechanism of the left hemisphere is always hard at work, seeking the meaning of events. It is constantly looking for order and reason, even when there is none— which leads it continually to make mistakes. ’”(p.134)

Later, he puts forth his own hypothesis for the role of consciousness itself: “From an evolutionary point of view, the purpose of consciousness seems to be this: an animal composed of a giant collection of zombie systems would be energy efficient but cognitively inflexible. It would have economical programs for doing particular, simple tasks, but it wouldn’t have rapid ways of switching between programs or setting goals to become expert in novel and unexpected tasks. In the animal kingdom, most animals do certain things very well (say, prying seeds from the inside of a pine cone), while only a few species (such as humans) have the flexibility to dynamically develop new software.” (p.142)

So consciousness allows flexibility – or put another way, substantially increases the possible actions that the organism can take. Engleman argues that it’s likely possessed across all animals – with the degree of intellectual flexibility reflecting the degree of consciousness.

Later in the book, Engleman delves into the difficult and charged question of free will: “So in our current understanding of science, we can’t find the physical gap in which to slip free will— the uncaused causer— because there seems to be no part of the machinery that does not follow in a causal relationship from the other parts.” (p. 166)

If our actions, decisions, and beliefs are a result of causal interactions of subsystems in our brains, is free will an illusion? Can neuroscience test for free will? He brings up a fascinating example of an early test and surprising results: “In the 1960s, a scientist named Benjamin Libet placed electrodes on the heads of subjects and asked them to do a very simple task: lift their finger at a time of their own choosing. They watched a high-resolution timer and were asked to note the exact moment at which they “felt the urge” to make the move. Libet discovered that people became aware of an urge to move about a quarter of a second before they actually made the move. But that wasn’t the surprising part. He examined their EEG recordings— the brain waves— and found something more surprising: the activity in their brains began to rise before they felt the urge to move. And not just by a little bit. By over a second. In other words, parts of the brain were making decisions well before the person consciously experienced the urge.” (p. 167)

So where did the will come from? He takes this concept further to suggest that criminal action is mostly the result of processes outside of conscious control. However, he still argues that of course such criminals should be taken off the streets, but perhaps understanding this process may foster better ways of changing their brains such that their behavior eventually becomes more socially acceptable. He does venture that the prefrontal cortex has “veto power” which perhaps can be trained.

Engleman then appears to pull back just a bit: “Given the steering power of our genetics, childhood experiences, environmental toxins, hormones, neurotransmitters, and neural circuitry, enough of our decisions are beyond our explicit control that we are arguably not the ones in charge. In other words, free will may exist— but if it does, it has very little room in which to operate. So I’m going to propose what I call the principle of sufficient automatism. The principle arises naturally from the understanding that free will, if it exists, is only a small factor riding on top of enormous automated machinery. So small that we may be able to think about bad decision making in the same way we think about any other physical process, such as diabetes or lung disease. The principle states that the answer to the free-will question simply does not matter.” (p. 170)

He gives a compelling argument that criminal action can be placed in a spectrum similar to other brain disorders that have been characterized and treated with varying success: “What accounts for the shift from blame to biology? Perhaps the largest driving force is the effectiveness of the pharmaceutical treatments. No amount of beating will chase away depression, but a little pill called fluoxetine often does the trick. Schizophrenic symptoms cannot be overcome by exorcism, but can be controlled by risperidone. Mania responds not to talking or to ostracism, but to lithium. These successes, most of them introduced in the past sixty years, have underscored the idea that it does not make sense to call some disorders brain problems while consigning others to the ineffable realm of the psychic. Instead, mental problems have begun to be approached in the same way we might approach a broken leg.” (p. 172)

Eric Wong adds his perspective about consciousness and free will here: “In computer program speak, I think of both consciousness and free will as properties of an ‘event handler’ like piece of software that just happens to work at the top level, able to execute other software, and the process to which control is returned after any other subroutine completes. The concept of free will is just a perceptual artifact of the fact that our brains are programmed to hang out at critical points, so that apparently meaningful ‘decisions’ are frequent occurrences.”

Later in the book, he takes on the limits of modern neuroimaging methods for understanding our unconscious processes, stating that the imaging resolution is much too coarse and sensitivity too small to understand the multitudes of processes that may play a role. He then hits the nail on the head regarding a current focus and major challenge of neuroimaging today: “For example, a study by psychologists Angela Scarpa and Adrian Raine found that there are measurable differences in the brain activity of convicted murderers and control subjects, but these differences are subtle and reveal themselves only in group measurement. Therefore, they have essentially no diagnostic power for an individual.” (p. 174)

So true for all neuroimaging – we need to accelerate the movement away from group averages to individuals. I believe that there is reason to have high hopes as promising results have recently been demonstrated in EEG, MEG, and fMRI.

Back to blameworthiness for those who carry out actions or have beliefs so far outside of the social norms that they need to be removed from society. Here he shifts the focus and states that blameworthiness is simply the wrong question. Most criminals don’t have measurable biologic problems, therefore are thought to be freely acting. On the other hand many with brain disorders do not carry out criminal acts. The sources of human behavior are incredibly complex. He flips the argument, stating that the actions themselves suggest that there are indeed biologic issues and that we simply don’t have the technology to detect them yet. We should instead focus on the best methods for rehabilitation. He mentions work by Stephen LaConte in real time feedback for giving the frontal lobes practice in suppressing impulsive short-term circuits. Beyond this, he does not have many solid suggestions for this as it is indeed a hard problem.

So is that all there is? Is our very essence the result of a vastly complex array of subconscious processes with us having the illusion of free will? His view, as expected, is hopeful for more nuance: “The situation is likely to be the opposite: as we plumb further down, we will discover ideas much broader than the ones we currently have on our radar screens, in the same way that we have begun to discover the gorgeousness of the microscopic world and the incomprehensible scale of the cosmos.” The sense of agency is so strong it’s hard to fathom that it’s an illusion. On other other hand, as we dig deeper into our unconsious influences, the picture might become more clear. Perhaps, in some future time, armed with this deeper awareness of the hidden influences of our thoughts – and perhaps some sophisticated biofeedback tools, we may be able to pull ourselves further out of our subjective experience where we can more optimally train our brains or change our beliefs…

From here, he takes on the problem of a “soul.” “All of this leads to a key question: do we possess a soul that is separate from our physical biology— or are we simply an enormously complex biological network that mechanically produces our hopes, aspirations, dreams, desires, humor, and passions? The majority of people on the planet vote for the extra biological soul, while the majority of neuroscientists vote for the latter: an essence that is a natural property that emerges from a vast physical system, and nothing more besides.” (p.203)

Like most scientists, he agrees with the materialist view: “The materialist viewpoint states that we are, fundamentally, made only of physical materials. In this view, the brain is a system whose operation is governed by the laws of chemistry and physics— with the end result that all of your thoughts, emotions, and decisions are produced by natural reactions following local laws to lowest potential energy. We are our brain and its chemicals, and any dialing of the knobs of your neural system changes who you are.”

…however he draws the line when the materialist view moves to a reductionist view. In fact, here he hits at perhaps the central problem in neuroscience in trying to understand the brain. “Just because a system is made of pieces and parts, and just because those pieces and parts are critical to the working of the system, that does not mean that the pieces and parts are the correct level of description.” (p. 216).

He then clarifies a bit: “The future of understanding the mind lies in deciphering the patterns of activity that live on top of the wetware, patterns that are directed both by internal machinations and by interactions from the surrounding world. Laboratories all over the world are working to figure out how to understand the relationship between physical matter and subjective experience, but it’s far from a solved problem.” (p. 204).

This is an easily misunderstood point. All the details are important, however, the principles of human thought and behavior cannot be explained by one level of description. Understanding the action potential or even networked activity in the brain is but one spatial and temporal scale. “A meaningful theory of human biology cannot be reduced to chemistry and physics, but instead must be understood in its own vocabulary of evolution, competition, reward, desire, reputation, avarice, friendship, trust, hunger, and so on…” (p. 218) These wider contexts of understanding may be critical for truly understanding conscious brain processes.

And finally, a final quote that lines up with much of what my co-blogger, Eric Wong, has been saying in his posts. There are many many layers of understanding that span spatial and temporal scales and perhaps defy explanation given our current understanding, that need to be uncovered to truly understand the brain in full. “Each day neuroscientists go into the laboratory and work under the assumption that understanding enough of the pieces and parts will give an understanding of the whole. This break-it-down-to-the-smallest-bits approach is the same successful method that science has employed in physics, chemistry, and the reverse-engineering of electronic devices. But we don’t have any real guarantee that this approach will work in neuroscience. The brain, with its private, subjective experience, is unlike any of the problems we have tackled so far. Any neuroscientist who tells you we have the problem cornered with a reductionist approach doesn’t understand the complexity of the problem. Keep in mind that every single generation before us has worked under the assumption that they possessed all the major tools for understanding the universe, and they were all wrong, without exception. Just imagine trying to construct a theory of rainbows before understanding optics, or trying to understand lightning before knowledge of electricity, or addressing Parkinson’s disease before the discovery of neurotransmitters.” (p. 223)

So, Eagleman’s view is that understanding the brain is not impossible, but realistically, we have not started to even figure out how to approach some of the unknowns. The human brain is much more than its conscious processes and likely an embodiment of principles more subtle and profound than those that we infer by basic reductionistic approaches.

Eagleman, with this book, has succeeded in drawing us in, opening our eyes, then making us uncomfortable with questions of free will, souls, and reductionistic views of the brain. He dissects these concepts with skill, presenting a convincing argument that while collecting more data is useful, learning to dismiss outdated concepts and form new better questions is what drives our understanding.

A satisfying read on a relatively unexplored subject: The book is clear, entertaining, and thought-provoking.

Mini Book Review: “Explaining the Brain,” by Carl Craver

Fri, 24 Feb 2017 04:18:00 GMT

“ Explaining the Brain ” is a 2007 book by Carl Craver, who applies philosophical principles to comment on the current state of neuroscience. This is my first and only exposure to the philosophy of science , so my viewpoint is very naive, but here are some main points from the book that I found insightful.

The book starts by making a distinction between two broad goals in neuroscience: explanation , which is concerned with how the brain works; and control , which is concerned with practical things like diagnosis, repair, and augmentation of the brain. In my previous post on this blog , I tried to highlight that same distinction. This book focuses on explanation, which is essentially defined as the ability to fully describe the mechanisms by which a system operates.

A major emphasis is on the question of what it takes to establish a mechanism, and the notion of causality is integral to this question.

Point 1: Mechanistic explanation and causality are difficult to establish . In Chapters 3 and 4: “Explanation and Causal Relevance” and “Causal Relevance and Manipulation”, Craver reviews the formal logic that is needed for the establishment of a mechanistic explanation. In a very multidisciplinary field like neuroscience, underlying mechanisms are complex, and true causality is very difficult to establish. We all know this, but it is still sobering to see it laid out formally. The conclusions that can be drawn from most neuroscientific data is often simply that some correlation exists between two observations or conditions. However, when striving for more mechanistic conclusions, we invoke the concept of causality, looking for the identity and order of the cogs in the machine. Unfortunately, to actually demonstrate mechanistic causality, you have to show that a ‘clean’ isolation or manipulation of X changes Y, and Craver talks in detail about what that means. In neuroscience, observed phenomena are usually multifactorial, difficult to isolate, and embedded in a complex system that is poorly understood in general, so the establishment of mechanistic causality is especially tricky.

Point 2: Filler terms . In Chapter 4: “The norms of mechanistic explanation”, Craver reviews what typically constitutes ‘explanation’ in neuroscience. He notes that while we are all trying to be as rigorous as we can about our conclusions, the line between what we have actually shown and what is merely suggested by our data is often blurry, largely due to the complexity of underlying mechanisms in neuroscience, as noted above. He cautions us to watch out for what he calls ‘filler terms’, which frequently appear in our publications and (sometimes) betray this blurriness. Among this list of filler terms: “activate,” “inhibit,” “encode,” “cause,” “produce,” “process,” and “represent.” Of course these terms can all be used in a rigorous way if applied carefully, but often they are not, and what results is what Craver calls a ‘sketch’, rather than a complete description of a mechanism.

Point 3: The mosaic unity of neuroscience . This term, which is described in Chapter 7 and is part of the subtitle of the book, is meant to describe the idea that the brain simply has too many layers of complexity for us to expect to find a compact set of explanatory mechanisms that describe how it works. Unlike many other areas of science, which search for compact goals by nature (eg high energy physics), Craver suggests that perhaps the brain is destined to be explained by a layered mosaic of pieces that cumulatively constrain the set of possible mechanisms, rather than a grand unified theory. This viewpoint is illustrated in the context of the relationship between the macroscopic phenomena of learning and memory, and the neuron scale phenomenon of long term potentiation. As a staunch reductionist , my personal bias is that this is giving up too soon, and that we should be applying heroic and concerted efforts to force things to become simpler.

The Wearable Tech + Digital Health Conference at Stanford University

Sun, 12 Feb 2017 23:20:00 GMT

The future of healthcare both small and big. It’s big data, machine learning, and massive amounts of data coming from tiny robust devices or phone apps of individuals. It’s individualized medicine – not only for patients who need care but for healthy individuals. The data will come from devices that will become ever more ubiquitous – stickers on skin, tattoos, clothing, contact lenses, and more. T his conference , organized by Applysci , and held on Feb 7 and 8, 2017 at Stanford University, involved a slate of some of the most creative, ambitious, and successful people in the digital health industry. I was both mesmerized and inspired.

I decided to venture outside my comfort zone of fMRI and brain imaging conferences to get a glimpse of the future of wearable technology and digital health by attending this conference. The speakers were mostly academics who have started companies related to their particular area of expertise. Others were solidly in industry or government. Some were quite famous and others were just getting started. All were great communicators – many having night jobs as writers. My goal for being here was to see how these innovations could complement fMRI – or vise versa. Were there new directions to go, strategies to consider, or experiments to try? What were the neural correlates of expanding one’s “ umwelt ?” – a fascinating concept elegantly described by one of the speakers, David Engleman.

On a personal level, I just love this stuff. I feel that use of the right data can truly provide insight into so many aspects of an individual’s health, fitness, and overall well-being, and can be used for prediction and classification. There’s so much untapped data that can be measured and understood on an individual level.

Many talks were focussed on flexible, pliable, wearable, and implantable devices that can measure, among other things, hemodynamics, neuronal activity, sweat content, sweat rate, body heat, solar radiation, body motion, heart rate, heart rate variability, skin conductance, blood pressure, electrocardiogram measures, then communicate this to the user and the cloud – all for analysis, feedback, and diagnosis. Other talks were on the next generation of brain analysis and imaging techniques. Others focussed on brain computer interfaces to allow for wired and wireless prosthetic interfacing. Frankly, the talks at this conference were almost all stunning. The prevailing theme that ran through each talk could be summarized as: In five or so years, not much will happen, but in ten to fifteen years, brace yourselves. The world will change! Technophiles see this future as a huge leap forward – as information will be more accessible and usable – reducing the cost of healthcare and, in some contexts – bypassing clinicians altogether and increasing the well-being of a very large fraction of the population. Others may see a dystopia wrought with the inevitable ethical issues of who can use and control the data.

Below are abbreviated notes, highlights, and personal thoughts from each of the talks that I attended. I don’t talk about the speakers themselves as they are easily googled – and most are more or less famous. I focus simply on what the highlights were for me.

Zhenan Bao

-Stanford University

-Director of eWear

Dr. Bao started with a quote – “Future wearable tech must enhance users abilities” and listed a sequence of where the area is going:

She focused much of her talk on electronics for the skin where a flexible wrap could measure heart rate, blood oxygenation, heart rate variability, skin conductance, protein in sweat, pH of sweat, blood pressure, and more…

She demonstrated impressive properties of these materials – maintaining their electronic integrity when being stretched, twisted, and even punctured. These apply for conductive and semiconductive materials.

She talked about a startup PyrAmes that was developing an implanted device for measuring intracranial brain pressure.

She suggested that her collaboration with Karl Deisseroth may allow direct communication of mechanoreceptors to the brain via optogenetics.

Lastly, she addressed the issue of batteries. If they get too hot, the problems multiply if they are implanted. She discussed work on safer batteries that can sense temperature and then shut down – rather than cause damage.

Marcus Weldon

-President of Bell Labs and CTO of Nokia

Dr. Weldon gave a visionary talk on the future of digital fabric and focussed on the point that we are entering the era of “automation of everything” which will be considered another major transformation of economy and society. He started with the classic Maslow’s hierarchy of needs and added to it – at the pinnacle was “time.” To him, we all spend so much time dealing with problems that could be anticipated and resolved with better and more distributed information.

He argued that the fundamental good of many of these technologies was to create time – allowing us to pursue our higher needs.

He emphasized the need for extremely fast processing (“latency and bandwidth matter”) with regard to immersive virtual reality environments as our vestibular-ocular reflex is on the order of 7 to 10 ms. The immersive virtual world needs to keep up with our head movement on this time scale otherwise many will become nauseated rather than reap the benefits of such environments.

Overall this was an inspiring and information-rich “big picture” talk that was not only optimistic but conveyed a sense of inevitability of an incredible future.

Roozbeh Ghaffari

-founder of MC10

He started his talk with showing what’s already out there – a cheap and flexible stick-on patch that measures UV exposure. It’s made by Lorealle and is disposable – very useful to gauge when one should get out of the sun to avoid sunburn.

He went on to talk about “biostamps” – stick-on devices that can measure continuously and with high fidelity, EKG as well as blood oxygenation, blood perfusion, and pulse-related pressure wave arrival time, and showed data indicating that the time delay between EKG and pulse arrival time is an alternative and high fidelity measure of blood pressure.

(My own thought is that we potentially can do this for the entire body already with MRI/fMRI – as pulsation is clearly visible, and accurately measurable in nearly all MR images – which might be worth looking into.)

Dirk Schapeler

-Director of Digital Health, Bayer

He started his talk lamenting the extremely long, expensive, and flawed process of clinical trials for drugs (1% success rate and taking up to 12 years) and suggested that there should be a new model to potentially bypass this – that of giving health advice rather than drugs.

He went on to talk about his ongoing work on a “smart shirt” that measures heart rate, skin temperature, impedance, respiration, posture, and steps.

He also pointed out that with more data that we can measure continuously in individuals, we may find that we currently are not even measuring the most useful things in current clinical practice.

Mary Lou Jepsen

-Founder of Openwater

Her talk was mostly about her vision for Holographic Near Infrared Light devices. She suggests that because NIR light can penetrate skin and scatters in relation to different densities and different blood oxygenation levels, this technology could replace the much more expensive MRI and fMRI technology today.

While I would love to think that this was possible, I’m still a bit skeptical as NIR at current energy levels has a penetration problem – at least as I understand it. I don’t fully understand how this can be overcome – especially when trying to resolve blood oxygenation changes below a depth of a few cm at most. Perhaps I missed a key point of her talk but I remain interested but skeptical.

Vivek Wadhwa

-Distinguished fellow at Carnegie Mellon University and Director of Research at Duke University’s Pratt School of Engineering

He started with a prediction that by 2023 the iphone will have the computing power of the human brain. He’s also a bit more optimistic in that he states that in 5 to 7 years there will be fundamentally transformative changes as AI and smart cloud-connected devices will permeate every aspect of our lives.

In 2030, he posits that we’ll have our first bionic man.

Miguel Nicolelis

-Duke University

His talk was all about brain-machine interfaces for helping the paralyzed walk again, showing many examples of how monkeys could move cursors and carts and arms by learning to control neuronal activity being picked up by only a few electrodes in their brains.

He was part of the famous demonstration before the world cup game where a paralyzed person made the first kick of a game. It was awe-inspiring.

Lastly he introduced an interesting expansion on this technology termed “brainet.” This is where two brains are learning to generate signals together to move a cursor to the appropriate place – suggesting that this comes easily to primates as they evolved to work together – to pick up on what each is doing. Fascinating stuff.

Tom Insel

-Verily

My former boss at NIMH has been a denizen of Silicon Valley for over a year and I was very curious as to what he was up to. He started his talk with some eye opening statistics. One was that in the US, suicide rate is 3 times that of homicide rate. This is a very big problem.

He also quoted Sydney Brenner “Science depends on New Techniques, New Discoveries, and New Ideas, likely in that order”

He talked a bit on genetics and it’s challenges.

First, in using genetics to understand disorders, it’s clear that the area is complicated. All it can do is assess “risk” as many with the clear genetic signatures associated with disorders never acquire the disorder. He went on to emphasize that genetics of mental illness is really genetics of brain development.

As a founder of RDoC , his emphasis of three points was not new: 1. Mental disorders are brain disorders. 2. Circuits matter and individual differences are profound. 3. Brain imaging may contribute to better psychiatric health care…music to my ears..and likely true.

Along these lines, he emphasized that DSM-5 diagnostics are imprecise and really have no clear biomarkers – essentially a dead end in data driven science.

He emphasized that we need measurement based cures (both before and after treatment – we need more follow-up studies!)

As a side note, I was intrigued by his mention of two pieces of data.

He outlined a three pronged strategy for new health care: 1. Mobile interventions 2. Care management, and 3. Digital phenotyping.

He also re-emphasized the prevailing themes that we need to show that cool tools make a difference and that we need, importantly, trust and transparency and to give people agency – the capacity of individuals to act independently and to make their own free choices .

Again, similarly to several other speakers, he mentioned how exponential growth (this industry) will take us by surprise as we are more sensitive to linear changes, quoting Bill Gates: “ Things will change much less in the next two years than we expect but much more in the next ten years than we expect.”

His main point was the need for more data collected directly from the patient, suggesting phone apps could do much more.

(My own thought is that we might also want to give physicians an app in which they could report – of course anonymously – on their patients with perhaps more curated information.)

Nathan Intrator

-Tel Aviv University and Founder of Neurosteer

This talk was exciting. One of the best of the conference.

He started with a puzzle. How, with one microphone above an orchestra, can humans so easily pick out any instrument they decide to listen to? The answer is that the instruments are differentiated not by frequency but by timbre. He then went on to apply this concept to his revolutionary idea.

He started his talk on the history of various signal processing methods for extracting unique time series information from complicated signals…FFT, wavelets, and his own Mother Wavelet Optimization. (my own thought – I have to look this up for use with fMRI data!)

He then went on to describe his big idea:

EEG signals can be fed into an EEG database where “basis packets” are discerned. These are used as templates (generated from many experiments in which the brain activity is known) to then be used to accurately decode whatever the brain is doing at that moment or the state of the brain during a longer time period.

(my own thought – resting state fMRI could potentially do the same thing – generate a library of pairwise correlation templates – to be applied to the time series to understand what the subject is thinking in real time).

He took this further. He has developed a simple two electrode patch that can be worn discretely by anyone. The data can be continuously fed to an iphone or ipad where it’s then uploaded to the cloud where the above analysis is performed. Over time, templates can be perfected such that the conscious state of each individual can be assessed in real time – allowing direct treatment or intervention, bypassing a clinician – all from two electrodes.

I certainly don’t have a good feel for this processing technique and wonder about the efficacy of using only two electrodes – however, I’m very much looking forward to see how this amazing idea will play out.

Vinod Khosia

-Co Founder of Sun Microsystems

Vinod is an individual who was treated (with good reason!) with special reverence by the organizers of this meeting. He didn’t give a talk, but rather gave an interview where he answered questions from the moderator and audience.

He suggested that not only will algorithms improve but they will also substantially improve human functioning – giving us insights into avenues of thought that we had not realized. One telling example that he brought up: The best Go player in the world who was beaten by an algorithm last year. After his match in which he was defeated by AlphaGo, his winning percentage against other humans shot up substantially. Apparently he gleaned some significant insights from playing the computer. This is particularly interesting as this did not happen in the context of Chess. However AlphaGo was a trained neural net, while Deep Blue (the algorithm that defeated the best chess player) was a brute force deep ply digger.

He suggested that conversational AI may be a good way for the elderly to fend of loneliness and depression as they can have conversations with such a system whenever they want.

Another interesting quote from him about predicting the future: “Extrapolating the past tends to underestimate progress. Inventing the future comes from future focused people.”

Another quote on a problem that all big data people grapple with: How do we motivate people to share their data?” We need to give people reasons to share. They won’t do it without a motive – preferably short term…a very hard problem. Of course, we could work harder to create a culture where it’s a given that it’s simply the right thing to do!

John Rogers

-Northwestern University

He spoke on two systems: 1. Epidermal wireless electronics and 2. Skin integrated microfluidic systems.

Regarding electronics, the goal is to make electronics fully biocompatible and progress is being made: Science 333, 838 (2011), Science 344, 70 (2014), Nature Communications DOI 10.1038 (2014).

He provided a vision for electronic temporary tattoos and stretchy electronic materials.

Such devices can measure ECG, hydration, temperature, sweat and more.

He talked on an ECG patch that also measured respiration rate (from the time dependent modulation of ECG)

His talk really got interesting when he started talking about Epidermal Microfluidics that could measure lactate, chloride, pH, glucose. Apparently Dr. Whitney on the “Rachael Ray” show advertised this.

He talked of a neat device that performed time-stamped chrono-sampling using multiple reservoirs that were filled with sweat in a sequential manner.

David Eagleman

-Stanford University

This was also one of the best most inspiring talks. He’s an amazing communicator.

He started with the concept of the “umwelt.” Every animal has it’s own window of the world, determined by how their senses take in and code outside reality.

He then went on to describe how the human “umwelt” could be expanded not only to treat those who are blind or deaf but for normal healthy humans. He mentioned specific people working on various devices:

Leslie Kay – “sonic landscape” converts visual information to auditory information

Kanno Kajimoto – tactile device on forehead that converts visual (or sonic) information to tactile information

He also described the BrainPort which converts visual information to tactile tongue-specific information.

He demonstrated a very interesting thing called the VEST – which is, in fact, a vest that one wears that converts sound information to coded tactile information on the torso and suggested that the information that can be converted to torso “feel” is vast: internet information (stocks), drone control information (pitch, roll), smell, UV light, and more.

(Personally this seemed like a perfect way to test how the brain learns to code new information from other senses. An fMRI study of receiving the stimuli before it’s coded vs after it’s coded and learned would be helpful for understanding plasticity as well as cross modal interaction).

Phillip Alvelda

-DARPA

He is a proponent of the use of photoactive information for tracking neuronal activity rather than being limited by electronic arrays. He showed some spectacular high resolution images of optically detected calcium activity throughout the brain – performed by Mark Schnitzer. He suggested one can decode this information using automated segmentation approaches.

He left us with an interesting puzzle: “We don’t know how motor precision is represented in the nervous system” We have uncovered correlates of movement, but precise movement has eluded us. (My own thought is that again this might be an interesting fMRI task – comparing subjects doing low precision motor tasks vs high precision motor tasks. Perhaps this has been done. I’ll have to search the literature). Email: phillip.alveda@darpa.mil

David Borton

-Brown University

Demonstrated high density multi-spike activity and suggested that a significant aspect of the neural code is not simply in the discrete bits of spiking activity but a more continuous discrimination of spiking synchronicity across multiple neurons.

He left us with a question of what would we do with a high bandwidth neural interface? What information could we obtain?

Karl Deisseroth

-Stanford University

Karl talked on his work on CLARITY and explained clearly that the reason brains are not translucent is because of the lipid content. CLARITY is a method that carefully removes all lipids, rendering the neuronal structure in great detail.

He also mentioned a method by which he could create CLARITY type maps that are sensitive to neuronal activity over a period of time before the process.

He demonstrated – using a mouse model – optogenetic targeting of the nucleus accumbens that modulated risky behavior.

Interestingly, he brought up the use of TMS to modulate dorsal-lateral prefrontal cortex to turn down cocaine craving. (Frenczi et al, Nature Neuroscience, 2016)

Mounir Zok

-US Olympic Committee

He gave a pretty light but inspiring talk on how smart sensors, smart fabrics, computer vision, and augmented or virtual reality are directly impacting performance of athletes (better than drugs).

Email: mounir.zok@usa.org

Sky Christopherson

-Olympic Cyclist, World Record Holder in Velodrome, Founder of Optimized Athlete

He gave an energetic talk on his own personal experience using various high tech strategies for maintaining health especially for older athletes.

For sleep tracking, he suggested the use of “emfit qs” which also provides information on heart rate variability – an increasingly useful measure of state of recovery. Low HRV typically indicates overtraining.

He also emphasized his pleasure in the use of the “Chilipad” for sleep. It’s a water chilled mattress pad that can precisely control temperature. He sets his to 66 degrees. This helps significantly in increasing the quality of sleep during the night – especially hot humid summer nights.

He also mentioned that there is an app coming out called “Gold” that will help intelligent decision making for training.

Understanding ‘Understanding’: Comments on “Could a neuroscientist understand a microprocessor?”

Fri, 03 Feb 2017 04:31:00 GMT

In a very revealing paper: “ Could a neuroscientist understand a microprocessor? ”, Jonas and Kording tested a battery of neuroscientific methods to see if they were useful in helping to understand the workings of a basic microprocessor. This paper has already stirred quite a response, including from Numenta , the Spike , Arstechnica , the Atlantic , and lots of chatter on Twitter.

This is a fascinating paper. To a large degree, the answer to the title question as addressed by their methods (connectomics, lesion studies, tuning properties, LFPs, Granger causality, and dimensionality reduction), is simply ‘no’, but perhaps even more importantly, the paper brings focus to the question of what it means to ‘understand’ something that processes information, like a brain or a microprocessor.

Indeed, the authors devoted more than a page of words to trying to define this question before launching into their results. Unfortunately, they do not propose any specific definition of understanding, but instead state that the data should “guide us towards” a known descriptive understanding of the workings of the microprocessor, such as the storage of information in registers, decoding and execution of instructions, etc. It is useful to realize that even in this case where we already know the answer, it is not easy to articulate a clear definition of understanding.

Some of my initial thoughts about defining ‘understanding’ in the context of brain science are outlined in my first post for this blog: “ What does it mean to understand the brain? “. Here is a little more structure that might be useful for the discussion.

Two possible approaches to defining and articulating goals related to understanding the brain.

From the perspective of the end users of the understanding, one way to categorize our goals is to declare whether they are primarily aimed at trying to satisfy our curiosity about how the brain works, whether we are reverse engineering the brain to inform computational science, or whether they are aimed at the practical goals of curing disease or augmenting the brain. The balance between these types of goals should driven by society at large. Where do we put our resources? How much do we value basic knowledge? Of course the hope is that on our way towards basic knowledge about how the brain works we find that practically useful information and technologies will fall out, as for the human genome project and the mission to the moon. However, unlike the genome and moon projects, the complexity of the brain is entirely unprecedented, and the future utility of obtaining a detailed understanding of the function of every neuron in the brain is much less certain. So, it is probably useful for now to think about basic curiosity driven exploration, reverse engineering, and healthcare driven search for biomarkers as separate goals, and frame our overarching questions accordingly.

From an analytical perspective, a clear distinction should be made between studying the substrate for computation vs studying the algorithms that run on that substrate. It is very different to understand the function of transistors and gates and neurons and synapses than to understand the algorithms that are implemented as computer programs or neural connections . Studying the substrate is primarily a bottom-up endeavor, where the biology and physiology are likely not much different between lower animals and humans. It is much less clear how to chip away at uncovering algorithms. From the bottom up, I believe we are certainly on our way to understanding real computational algorithms in very simple organisms, but scaling up is daunting to put it very mildly. Understanding the human brain in particular in an algorithmic way requires figuring out what a brain can do with 20 billion cortical neurons that it can’t do with 6 billion (chimps). Imagine the complexity of algorithms that run on 6 billion neurons with several trillion synapses. I for one, can’t. Now imagine that that level of complexity just doesn’t cut it, and we need to build an understanding of algorithms that apparently can’t be implemented without more neurons in order to understand the human brain. From the top down (as with bottom up approaches) the initial steps are well underway and clearly informative. The functional organization of the whole human brain is being mapped down to (few) millimeter scale resolution, and the richness of data at this level of many hundreds of parcels will already give us a good handle on how information is handled (in an org chart kind of way), and what is normal. From there, drilling down to something one could label as the implementation of a computational algorithm is much more dicey, and I think that what the field can really use (as for the bottom up approaches) is a clear statement of specific technical goals, and a clear description of exactly what kinds of ‘understanding’ are likely to be revealed by the attainment of those goals. Such a statement would be a great way to rally the field towards a finite set of goals. Any takers?

My Wish List for the Ultimate fMRI System

Wed, 01 Feb 2017 04:16:00 GMT

I recently had a meeting where the topic discussed was: “What would we like to see in the ideal cutting edge and future-focussed fMRI/DTI scanner?” While those who use fMRI are used to some progress being made in pulse sequences and scanner hardware, the technological capability exists to create something substantially better than we have now.

In this blog posting, I start out with a brief overview of what
we currently have now in terms of scanner technology. The second part of this blog is then focussed on what my ideal fMRI system would have. Lastly, the article ends with a summary outline of my wish list – so if you want to get the gist of this blog, scroll to the list at the bottom. Enjoy and enter your comments! Feedback, pushback, and more ideas are welcome!

What is the current state of fMRI technology now ?

Field Strength:

The world has over fifty 7T systems. I would estimate that at least a third of these are, for the most part, “turn-key.” The field has wide bore 3T systems – allowing for greater patient comfort and the ability to fit more or bulkier subject interface devices inside. GE has come out with a prototype head only 3T system at Mayo Clinic that has an incredibly small footprint and much less need for liquid helium as a coolant. The NIH is also planning to receive a human 11.7T scanner, hopefully by next year, after the first one quenched several years ago. Soon, 7T will achieve FDA clearance, and when it does the price of a 7T will perhaps (hopefully) drop from the currently prohibitive $8M to $10M down to (a less prohibitive) $7M or less as the clinical market grows.

Pulse Sequences and Image Reconstruction:

With few exceptions, the current fMRI scanners are clinical scanners. This means that they have the latest clinically relevant pulse sequences – some that carry over to fMRI applications, including a version of susceptibility weighted imaging (SWI), diffusion imaging, and high tissue contrast capability.

Most scanners now have some form of parallel acceleration (such as SENSE, GRAPPA, or derivatives) – allowing higher resolution for fMRI and DTI, and shorter scan times for structural imaging. Some scanners, assuming there is a research agreement in place, have access to the latest novel pulse sequences for fMRI, namely “simultaneous multi-slice” (SMS) and multi-echo. SMS allows entire multi-slice volumes to be obtained in ½ to ⅛ the time of standard acquisition. Multi-echo has advantages that come with collecting multiple (up to five) simultaneous and differentially T2*-weighted time series – allowing time series cleanup and at the very least, increased temporal SNR and CNR.

What the field does NOT have is an easy way for researchers to create, share, and test pulse sequences and reconstruction methodology across scanners and vendors. It’s difficult, even for a skilled MR physicist, to modify a product pulse sequence – not because pulse sequence programming is intrinsically complicated but because the operating environment many vendors is somewhat of a mess – patched together from outdated platforms, and modified over the years without major overhauls. While it may be slightly easier to modify an extremely simplified pulse sequence, it’s then much more difficult to then add the appropriate safety checks that are required for distribution. This entire mechanism could be cleaned up substantially, allowing researchers to focus more on pulse sequence innovation than navigating the pulse sequence programming idiosyncrasies.

Regarding image reconstruction, with some work, it’s possible to save raw data and then port to another computer to perform reconstruction. This is generally cumbersome and vendors don’t typically give out the code to their proprietary reconstruction methods, so the images that are independently reconstructed are typically never as good as those of the scanners themselves. This is also a source of concern as such things as image smoothing, image unwarping, and other operations are performed in the background, further amplifying the differences in images coming from different vendors and scanners.

At the NIH, a gadgetron was created that takes raw data from scanners across several vendors, so that such a reconstruction, comparison test-bed could be used. This is a model for what should be commonly available on all future scanners – helping recon development and reducing variability across scanners. This would be a more effective tool if it could start with the vendors’ latest recon code, and then have it modified to reduce variability across scanners or at the very least allow for a more accurate scanner comparison.

Gradient Coils:

Gradients have improved. The most advanced commercial scanner at the moment, the Siemens 3T Prisma, have a maximum gradient strength of 80 mT/m or 8 Gauss/cm, and a maximum slew rate of 200T/m/s. This slew rate, coming from a whole body gradient system, is enough to exceed biologic limits on dB/dT (the rate of change, per unit time, of the B-field), causing peripheral nerve stimulation.

The most powerful human scanner built to date is the Human Connectome Scanner – also a 3T and made by Siemens. The first system resides at the Massachusetts General Hospital Martinos Center. A second system is planned for Cardiff University in the UK. It has gradients in which each axis has four parallel drives that are independently powered by two sources per gradient, performance specs of 300 mT/m gradient strength, a standard slew rate of 200T/m/s, and has the power requirements of a nuclear submarine.

Manufacturer constructed head gradient coils have claimed performance specs that include a maximum gradient strength of 85 mT/m and a stunning 700 T/m/s slew rate. Indeed the connectome scanner is optimized for diffusion imaging, requiring large gradients, and the prototype head gradient coils mentioned are optimized for high speed imaging with a higher possible – and allowed – high slew rate. With the high slew rates, gains can also be made in diffusion imaging, as the higher slew rate enables the appropriate diffusion weighting to be achieved in less time, thus boosting signal to noise.

An important factor when thinking of gradient coils is that the local head gradient coil can switch much faster than a whole body gradient coil when scanning humans not only because it has lower inductance, but most importantly, because the gradients do not extend much beyond the head coil itself, thus keeping the maximum dB/dT at the extreme ends around isocenter at a level below that which would cause peripheral nerve stimulation. This shorter length of the coil allows greater flexibility in how fast the gradients can switch, benefitting other sequences than just diffusion weighting.

Radiofrequency Coils:

RF coils have steadily improved since the 90’s when single channel “quadrature” coils were the norm. Since about 2000, multi-channel receive coils have become more popular, starting with 4 then 8 then 16 channels. Now, at 3T, 32 channel receive coils are the norm. The advantages are two-fold. First, with smaller coils (and an array of small coils to maintain whole brain coverage), there is an increase in signal to noise ratio. Second, with the various parallel imaging techniques, the coils themselves, because of their non-overlapping sensitivity profiles and spatial distribution, aid in spatially encoding the data, saving time and improving some aspects of image quality.

Shim Coils:

When a head is placed in a scanner, it distorts the magnetic field. If these field inhomogeneities are not corrected, then the image quality suffers. Warping and signal dropout are common manifestations of an inhomogeneous magnetic field that has been poorly shimmed. Typically, shimming is performed to smooth out the magnetic field. The basic operation of shimming involves adjustment of the current in coils situated in the bore of the magnet. In the past decade, auto-shim algorithms have been able to take the user out of this process, speeding up the procedure. The shims on most magnets are designed to approximate the spherical-harmonic functions. These functions are orthogonal (independent) over any sphere centered at the origin. This approach (resistive shims up to 2nd or 3rd degree) works sufficiently well for most high resolution, multi-shot, clinical pulse sequences, however, as anyone doing EPI at 7T can tell you, they still fall far short of satisfactory. Especially at high fields, low resolution EPI results in signal dropout, and high resolution EPI results in extreme warping. I believe we have the technology to solve this. It’s just a matter of implementation. A possible remedy is described in the second part of this blog.

Motion Correction:

Post processing methods perform well but not perfectly in motion correction. Still, any motion near a signal intensity gradient or beyond about 1 mm, is imperfectly corrected – and the data are typically thrown out. Couple that with spin-history effects, etc..we have room to improve. Motion compensation on the acquisition end has shown promise. We have navigator pulses for multi-shot imaging as well as devices that measure displacement optically then feed that information back to the gradient settings to compensate for each line of k-space. These approaches also have variable success and are mostly aimed at cleaning up single, multi-shot structural images associated with standard clinical use of the scanners.

Other Features:

What are the other features that are potentially more widely useful in a scanner that have been implemented in some form now? One that comes to mind are simultaneous positron emission tomography (PET) and MRI scanners. These have PET detectors positioned in the bore so that both MRI and PET images may be obtained simultaneously. This is admittedly a very niche market as most PET/MRI comparisons can easily be done separately. Only with the observation of either very transient effects or non-repeatable effects does this approach shine. My sense is that more experiments will be devised that capitalize on the simultaneity of the collection of this information.

Another feature would be better integration of subject interface devices with the scanning environment. Currently visual and auditory stimuli as well as button boxes, eye trackers, etc.. are fitted in an ad-hoc manner into clinical scanners, requiring a relatively large amount of setup time. All these could be integrated into an fMRI scanner such that they are always available and ready to go.

What would I want in the ultimate fMRI system?

Field Strength:

7T is definitely where both fMRI and structural imaging are going. The gains in SNR, CNR, and they qualitative types of unique contrast are clear for both fMRI and clinical applications. For clinical applications, one can much more easily see gray matter plaques in MS and iron deposits with various disorders. Susceptibility contrast provides exquisite detail in imaging the venous vasculature and iron deposits, and the higher SNR allows for gains in resolution and/or speed. The issues involved with specific absorption rate (SAR), shim, RF inhomogeneity, and different relaxation rates are all solvable at 7T. For functional MRI, the gains in studying resting state fMRI are clear, as physiologic noise (including spontaneous neuronal activation induced fluctuations) even further dominates the signal. The imaging of cortical layers and orientation columns has only been demonstrated at fields at or above 7T. This resolution and the increased sensitivity to fluctuations balanced with our ability to solve the engineering hurdles clearly points to 7T as the desired field strength. The gains made at higher fields than 7T are not clear yet, as the engineering problems associated with going higher currently appear much more challenging. So, my perfect fMRI scanner field strength would be a 7T.

Pulse sequences and Reconstruction:

The field of fMRI has been neglected by vendors for the past quarter century. Creating and implementing a pulse sequence is perhaps analogous to writing a novel in machine language rather than using a word processing program. Regarding fMRI, there is the untapped potential of novel pulse sequences for looking at different functional contrasts, simultaneous acquisition of complementary contrasts (flow, BOLD, volume, etc..), high resolution, etc.. The development and testing of pulse sequences would be significantly accelerated if we had the equivalent of Microsoft Word for pulse sequence development, or at the very least, and open access structure for writing, testing, and disseminating pulse sequences. The rate at which new pulse sequences for fMRI are disseminated through clinical vendor product releases or even as works in progress to researchers is somewhat anemic. An example: Arterial Spin Labelling. This sequence, tremendously useful for noninvasively measuring baseline perfusion and activation-induced perfusion changes, was invented and patented in the early 90’s. It was only until the patent ran out (17 years later) that it became a feature on most clinical scanners.

We need a more robust research environment for developing, testing, and disseminating new pulse sequences. If a Microsoft Word – type platform is out of the question. It would be extremely helpful for pulse sequence development if the vendors created a modular sequence allowing researchers to play with multiple knobs easily. If such a sequence were developed, fMRI acquisition sophistication and diversity would grow more rapidly, but of course ultimately, we are limited by our own imaginations.

One example of what has worked for me over the years was an all purpose modular and easily modifiable pulse sequence. Back in the 90’s, Eric Wong and his colleagues developed a “swiss army knife” of pulse sequences called “spep.” It was highly modular, allowing almost complete control of RF pulses, gradient placement, readout window placement, resolution – all adjustable using adjustment of single variables, called CV’s or control variables, in the code. It led to the development of several novel ASL sequences (QUIPPS, Q2TIPS for example) and has been instrumental to my own research exploring multi-echo EPI, the effects of diffusion weighting on fMRI, and many other projects.

Regarding image reconstruction, all fMRI researchers would benefit substantially from an open and entirely modifiable reconstruction platform provided by the vendor. As mentioned above, one platform created by users, called the gadgetron, takes raw k-space data from multiple scanners and applies a reconstruction algorithm to it. It cannot be emphasized how important it is for something like this to exist and be vendor supported – as it opens up a source of potential standardization across scanners as large multi-scanner fMRI and DTI databases are being created. Currently, vendors do not typically supply recon code, so those programming the gadgetron have been starting mostly from scratch.

In today’s environment of open, fully sharable data, it’s essential, at the very least, to know precisely how the images were created, and then perhaps establish a standard reconstruction across vendors, substantially reducing image variability across vendors. Again, we referring to our home built pulse sequence mentioned above: “spep” as an example. This sequence relies on a stripped down recon, and the images differ substantially from those produced using the product recon – thus limiting its utility. If researchers develop pulse sequences, they should have access to the image recon code.

Gradient Coils:

The issue of gradient coils is tricky as there is not one ideal configuration. The need for whole body imaging is sometimes useful – even for fMRI, thus requiring whole body gradients. However, most fMRI researchers would be happy with a head gradient coil that allows easy patient access and all the advantages that come with it.

The ideal fMRI scanner would perhaps have both a body coil and a modular insert gradient coil with two modes – high slew rate and high gradient strength. The high slew rate would use less windings and the high gradient strength would use more windings – all changeable by the flip of a switch – or even electronically – allowing each configuration to be activated during a pulse sequence (i.e. very short readout window EPI with high diffusion weighting gradient lobes).

The head coil would be able to switch gradients faster without inducing peripheral nerve stimulation. Simulations have shown that a head gradient coil could slew to 100mT/m (more than sufficient for most purposes) at a slew rate of 700 mT/m/s without inducing peripheral nerve stimulation, while a whole body gradient could slew to 100mT/m with a slew rate of only 100 mT/m/s without inducing peripheral nerve stimulation.

As I mentioned before, until vendors start caring more about fMRI development (which is until fMRI becomes more clinically relevant) these exciting, and quite achievable capabilities will likely go unfulfilled.

Radiofrequency Coils:

RF receive: While the standard RF coil arrays used today consist of up to 32 channels, it’s not clear to me if more will result in clear improvements. It is clear that there is considerable room for improvement in coil configuration. In an array coil placement could be optimized either for SNR gains or for spatial encoding as with SENSE, SMASH or even SMS imaging. Currently, we make do with one configuration for both. Coil placement could also be much closer to the head. Flexible coil caps or rigid configurations that fit much more snugly at least above the eyes and down the back of the head would substantially improve SNR.

RF excitation: RF receive coils have been discussed but, so far, no vendor has a seamless multi-channel excitation package. Having multiple excitation coils is useful for several purposes. The first and foremost is that their power can be independently adjusted to “RF-shim” or make a more homogenous excitation distribution such that the RF flip angles are uniform throughout the brain – making for more interpretable contrast. This feature is still being developed. A major obstacle is the risk that once RF power is set for each coil, there man be unforeseen “hotspots” in RF power, exceeding the current SAR limits and potentially heating the subject. So far, no clear solution has been proposed to this problem. The second feature is that different coils, in theory, could be tuned to excite different frequencies or frequency widths – allowing for simultaneous spectroscopic imaging or magnetization transfer imaging. This would be extremely advantageous to multiple fMRI studies.

Shim Coils:

As suggested above, there are clear limits to adding more orthogonal shim coils. The amount of current needed for fully shimming the head at 7T is currently not practical. Even with third order shims are not effective in creating a homogenous field around sharply defined field inhomogeneities near the ears and sinuses. While the effect of the currently poor shimming performance is minimal with normal clinical imaging, at 7T with either high resolution (warping) or low resolution (signal dropout) it is still prohibitive. For the perfect fMRI scanner, I think alternative solutions are possible. Each solution is also “stackable” in that they can be used together in a synergistic manner. Shim can indeed be “solved.”

Shim coils closer to the head: Strategies have been proposed to use the RF coils to support an adjustable DC current. This should be standard on all scanners.

Specialized, anatomically specific shim coils: Strategies have also been proposed that involve placement of specific shim coil loops on the nose or at the ears. These have been shown to be quite effective as the small field distribution around these small, close to the head coils can effectively target the small sharply varying fields around sinuses and ears.

Non-orthogonal multi-shim array: A recently introduced alternative has been multi-coil shim array – up to 128 loops of wire driven independently offers more power and flexibility in counteracting field homogeneities induced around and inside the head at high fields. Here, the solutions are arrived by iteratively calculating the fields necessary for each coil to target specifically focused inhomogeneities.

Shim strategy: slice specific shims. When shimming is carried out – in particular with orthogonal shim coils – eliminating field inhomogeneity in one part of the brain can come at a cost of creating greater field inhomogeneities in another part of the brain. One solution that has been demonstrated is slice specific shimming. This solution would require rapidly switching the shim coil currents between slice collection, requiring more power in the shim current amplifiers – all very doable.

Strategically placed passive shims: It has been shown many years ago that strategic placement of diamagnetic materials of specific shapes on or near the head – or even in the mouth , can help smooth out some of the more problematic field inhomogeneities. While perhaps a bit cumbersome, one could imaging subjects wearing a “shim mask” to achieve this purpose.

Field cameras: Lastly, the effect on Bo by breathing has not been mentioned at all because it’s not at all compensated for in standard scanners. At high field, it has a very pronounce effect, especially if performing multi-shot EPI for fMRI. There currently exist “field cameras” that measure the dynamic changes in Bo that occur. These are quite expensive but potentially game changing when it comes to removing breathing related artifacts in fMRI. Having a field camera setup with each scanner and then using it for prospective motion correction by feeding the Bo field information either to the shims and/or to the gradients, would substantially increase stability.

Shim parameter settings: Lastly, it’s clear that most heads are more or less the same. Typically shimming takes time because the shimming procedure starts from scratch for each subject. Long ago our 3T Bruker scanner had the option for saving shim settings for each subject. An alternative solution would be to have perhaps 5 differing “starting” shim settings to account for most head types. This would save considerable time in shimming as the algorithms would start much closer to the “solution.”

Motion Correction:

Again, the clinical vendors have favored motion correction strategies that target standard clinical imaging. While these are somewhat effective, they fall short for EPI sequences used in EPI. One approach to prospective motion involving using optical sensors on specific targets (currently a moire pattern), involves detection of motion and then feeding back that information to the gradients to compensate. This approach has been a disappointment so far. I believe that a simple optical camera that images the entire head, then using that information fed back to the scanners, would perhaps be more robust for prospective motion correction.

With fMRI, high resolution is a desired feature. Common single shot EPI solutions have been parallel imaging, zoomed imaging methods, or partial k-space imaging. Recently multi-shot techniques, particularly 3D EPI, have been experiencing a resurgence. However, these approaches suffer from time series instability that only becomes a non-factor when the time series SNR is so low that thermal noise matches time series instability. In the past navigator pulses have been used for phase correction in multi-shot time series. Having navigators implemented in these sequences would greatly enhance their utility for fMRI.

Other Features:

As PET is experiencing something of a resurgence, especially since it is so complementary to many fMRI studies, it makes sense to have a built in PET capability that is not too spatially intrusive and does not interfere with scanner, gradient, or shimming performance. Currently, robust PET/MRI capability exists.

In addition to a PET scanner it seems reasonable that such capabilities as in-scanner EEG, TMS, optical imaging, and tDCS could be engineered into the scanner, allowing easy use when needed.

Real time fMRI, with an open pipeline allowing for open access real time analysis, would also be essential – especially for pushing fMRI into the clinical realm. In the clinical setting, it’s essential to obtain usable information quickly. This information would range from time series quality and subject motion – determining if the scans need to be redone – to functional maps that allow clinicians to make decisions. While vendors provide the ability to see the raw images come up in real time, a seamless, open access pipeline to raw k-space data or image data, and a seamless time series processing platform would be essential.

Quieter pulse sequences, quieter gradients, or active noise cancellation in the bore: A major drawback in MRI and fMRI is the extreme loudness of the scanner during the scanning process. Sound levels are in the range of 130 dB. In the base RARE or BURST sequences have proven to be substantially quieter , and, perhaps with navigator pulses (as they are multi-shot) could be useful for many fMRI applications that require less scanner noise.

While vendors have worked towards developing quieter gradients, more EPI specific engineering could go further – as the higher frequency “vibrational modes” associated with a range of EPI gradient readouts could be dampened with the correct reinforcement.

Active noise cancellation. This technology has been around for some time and has been implemented with variable success in headphones in the scanner, a major limitation of sound canceling headphones is that most of the sound is still transmitted through the skull so the net attenuation is only about 30dB. I’m not certain if it is possible, but perhaps an entire bore sound cancellation device might prove effective in canceling the noise before it even reaches the head.

Lastly, it would be tremendously useful to have some sort of non-compete agreement between vendors in the research arena, so that the scanner business is forced to be consistent with the movement towards open science. While companies can continue to compete in the clinical arena, this competition is no reason to hold back progress in fMRI research. Accompanying this open access environment would be an open technical fMRI community that would be able to share vendor experiences, pulse sequences, image reconstruction algorithms, processing methods, hardware, and other aspects of fMRI seamlessly.

—-

Below is a list summarizing all the desired features on an fMRI-optimized scanner that I have discussed above.

Review of “The Distracted Mind” by Adam Gazzaley and Larry D. Rosen

Thu, 19 Jan 2017 20:21:00 GMT

http://www.fcc.gov/guides/texting-while-driving.

Ten Unique Characteristics of fMRI

Sun, 08 Jan 2017 16:57:00 GMT

A motivation for this blog is that since our graduate student days, Eric Wong and I have had hundreds of great conversations about MRI, fMRI, brain imaging, neuroscience, machine learning, and more. We finally decided to go ahead and start posting some of these, as well as thoughts of our own. It’s better – for us and hopefully others – to publicly share our thoughts, perspectives, and questions, than to keep them to ourselves. The posts are varied in topic and format. In certain areas, we know what we’re talking about, and in other others, we might be naïve or just wrong, so we welcome feedback! We also welcome guest blogs as we hope to grow the list of guest contributors and readers.

I’ve been working to advance Functional MRI (fMRI) since its inception. On Sept 14, 1991, two years into graduate school, and a month after seeing preliminary Massachusetts General Hospital results at the SMR meeting in San Francisco, Eric Wong and I performed our first successful fMRI experiment (Bandettini 2012).

That time period was indescribably exciting – especially for a graduate student just looking for a Ph.D. thesis project he could pursue. While we knew that the groups from Massachusetts General Hospital and possibly the group from Minnesota were also making progress along these lines, we were no less excited. The more complete story around this experiment, from my own perspective, was published in NeuroImage volume 62, Issue 2, pages 575-1324, 2012 The NeuroImage Special Issue “20 Years of fMRI: the Science and the Stories.” The stories from labs producing first results, Kamil Ugurbil (Uğurbil 2012), Ken Kwong (Kwong 2012), and Seiji Ogawa (Ogawa 2012) are all good reads.

Functional MRI, to this day, over a quarter century later, remains as exciting to me as on Day 1 as developments and applications continue at a rapid rate. While human brain imaging methodologies have arisen and grown over the years, and all of them, I’m certain, have interesting stories behind them, I wanted to share why I feel fMRI is unique:

1. fMRI was a surprising, rapid discovery.

Elements leading up to the discovery of fMRI were the discovery of BOLD by Ogawa et al (Ogawa 2012), the discovery of the dependence of blood T2 on oxygenation by Thulborn et al(Thulborn 2012), the advent of arterial spin labelling techniques by Williams et al. (Williams, Detre et al. 1992), the technical capability to perform EPI (Cohen and Schmitt 2012), and for the Minnesota group, higher field strengths (Uğurbil 2012).

The first fMRI results came from Ken Kwong’s penchant for trying interesting experiments. With Ken’s experiment, the method was discovered rather than incrementally developed. In fact, the pulse sequence and basic parameters used by Ken for BOLD were not anything overly complex or new – simple T2* weighted gradient-echo EPI at 1.5 Tesla. There was minimal time series processing involved then – in stark contrast to processing methods today. Interestingly, aside from the explosion in the sophistication of time series processing, the details of Ken’s first experiment have not qualitatively changed in terms of general practice over the years. He was just the first to realize that such a straightforward thing could be done!

This discovery surprised and excited the MRI community. To provide an analogy, it was as if we realized that if one sets the exposure settings of a standard camera just right, rather than just getting a photograph, you can get a picture of, say, subatomic particles. While the MRI scanner vendors adopted a wait-and-see approach before putting any resources into developing fMRI, the clinical and basic neuroscientists were highly motivated to start scanning.

2. fMRI was a revolutionary advance in functional imaging capability.

Functional MRI was and still is the only non-invasive, whole-brain method that has enough sensitivity to see human brain activity with about 2mm detail as it is happening in real time. This made for good science fiction before 1991. No one imagined it would become reality so quickly.

3. fMRI is deeply multidisciplinary.

Functional MRI brought disparate disciplines together in a way that was unprecedented. Suddenly, cognitive neuroscientists were having intense conversations with MR physicists. Computer programmers were talking with clinicians. The best fMRI research today has a signature of advancing methodology and insight into brain function – requiring close collaborations between physicists, statisticians, programmers, and neuroscientists.

4. fMRI is riding on the back of the clinical MR industry.

A huge factor that many people overlook is that fMRI was able to launch and propagate so rapidly because it leveraged the massive clinical MRI industry. In the early 90’s there were at least 20,000 clinical MRI scanners worldwide. By 1998, most MRI scanners were equipped with EPI – for other more clinically relevant purposes such as following a bolus of gadolinium for perfusion imaging or visualizing the heart beating.

Even though fMRI had minimal clinical impact, almost every MRI scanner in every hospital in the world was a potential brain function imaging machine. There was no need for a manufacturer to make fMRI machines. They already existed! These scanners were priced at over $1M each and were paid for and supported by hospital revenue – not neuroscience research grants. Today, that’s changing somewhat as the fMRI market grows and research grant revenue towards fMRI increases, but the reality is that fMRI depends on the clinical MRI market to survive.

FMRI has tremendously benefited from essentially riding on the back of the clinical MRI industry. This relationship has clear drawbacks too. Many interesting pulse sequences and custom fMRI setups are not being disseminated worldwide because the scanner vendors do not yet see a large enough market of fMRI to necessitate adding more development resources. Until fMRI becomes a thriving clinical technique (hopefully soon), it will be at the mercy of the clinical focus of the MRI scanner vendors – namely Siemens, General Electric, and Philips.

5. The degrees of freedom in fMRI acquisition is vast and unexplored.

We can do so much more than collect a simple time series of T2* weighted echo planar images. The ability to derive physiologic and neuronal information from MRI is still being explored as there are so many “knobs” you can adjust on the acquisition side to highlight gray matter, white matter, CSF, flowing blood, perfusion, iron deposits, vascular territories, trauma, leaks in the blood brain barrier, hemorrhage, deoxygenated blood, metabolism, pulsation, macromolecules, temperature, water diffusion, diffusion anisotropy, and much more. Additionally, the information that may be useful to fMRI is also still relatively untapped. Along with mapping the magnitude of the hemodynamic response as is most commonly done, we can derive information about latency, fluctuations, oxidative metabolic rate changes, blood vessel sizes, oxygenation, and more.

6. Processing methods are exploding in variety and sophistication.

Functional MRI processing methods continue to surprise – as it seems that the field continues to find new and better ways to extract, compare, and display new hemodynamic and neuronal information in groups and individuals. With the emergence of massive shared data sets, ever more subtle information about individual differences and similarities is being plumbed with the help of modern machine learning approaches.

7. Functional MRI just works.

Functional MRI just works – almost every time! It’s a stunningly robust technique. The functional effect size to noise ratio (from 6/1 to 1/1) is still perhaps too small and subject-wise variability a bit to large (with current post processing techniques) for robust clinical use (at least 10/1 is considered essential) but is large enough to see a significant effect within a few minutes of averaging. If it took 6 hours of averaging to see something, ambitious people would still do it but it would be much more difficult and the field would be much more anemic at this point.

8. fMRI requires two highly serendipitous properties

Another key to fMRI that is commonly taken for granted: It requires two very subtle yet all-important properties to be possible at all. The first is that hemoglobin has to change its magnetic susceptibility in a non-trivial manner between being oxygenated and deoxygenated. This is an extremely rare property of a biologic tissue. If our blood were copper based – as with mollusks – rather than iron based, this would not happen. We would not have BOLD contrast as copper based blood does not change susceptibility with oxygenation. The second all-important property is that, with activation, a localized flow increase in the active region creates a highly focal overabundance of oxygenation. Why didn’t nature just require that the oxygenation stay the same in the active regions? We are still trying to figure that out, but the fact that it does – every single time with every person – similarly across species – in the same precise way in a stunningly consistent manner is highly fortunate. Perhaps our brains could have evolved a system where localized activation-induced changes in flow increased to simply match the increased metabolic needs rather than apparently overshoot them. If this happened, there would be no BOLD changes. We are lucky!

9. fMRI fills a unique temporal and spatial niche

The information that fMRI provides fills a large and interesting temporal and spatial niche in understanding brain organization. Our brains are highly modular, and fortunately, the larger modules (motor cortex, visual cortex, etc..) are easily large enough to be discerned with fMRI. If our largest brain modules happened to be no larger than ocular dominance columns, fMRI would have never taken off, and if it did, interpretation of the results would have been a challenge at best. We’ll likely gain enough sensitivity and resolution soon to routinely probe the columnar and layer level organization of the brain soon – which brings us to the next unique property…

10. The highest fMRI resolution matches the intrinsic precision of hemodynamic control

It appears that the highest resolution achievable to fMRI (limited by scanning technology) – that of cortical columns or layers – matches the intrinsic precision of hemodynamic control. In other words, the smallest homogeneously activated region that causes a focal change in blood flow is on the order of columns or layers (<1mm). This perhaps suggests that this is the smallest scale in which groups of neurons are activated together. This last point is potentially controversial as it may suggest that looking any finer than this scale at neuronal activity may not necessarily lend insight into modular brain organization. Either way, again it’s fortuitous that fMRI resolution limits match the hemodynamic control limits – at least in humans.

P.S. It’s also a lucky thing that MRI can exist at all. To quote an early fMRI pioneer, Robert Weisskoff:

“It still seems remarkable to me that an entire segment of the medical imaging industry could be built on something as unlikely as MRI … The small energy difference between the spin up and spin down state for protons even at 1.5 T creates a net polarization of only a few protons per million, and everything we see in MRI (and fMRI) is the result of that small difference.”

MRI can easily detect only those few-in-a-million protons…a tiny signal that gives so much information.

Bibliography

Bandettini, P. A. (2012). “Sewer pipe, wire, epoxy, and finger tapping: The start of fMRI at the Medical College of Wisconsin.” NeuroImage 62 (2): 620-631.

Cohen, M. S. and F. Schmitt (2012). “Echo planar imaging before and after fMRI: A personal history.” NeuroImage 62 (2): 652-659.

Kwong, K. K. (2012). “Record of a single fMRI experiment in May of 1991.” NeuroImage 62 (2): 610-612.

Ogawa, S. (2012). “Finding the BOLD effect in brain images.” NeuroImage 62 (2): 608-609.

Thulborn, K. R. (2012). “My starting point: The discovery of an NMR method for measuring blood oxygenation using the transverse relaxation time of blood water.” NeuroImage 62 (2): 589-593.

Uğurbil, K. (2012). “Development of functional imaging in the human brain (fMRI); the University of Minnesota experience.” NeuroImage 62 (2): 613-619.

Uğurbil, K. (2012). “The road to functional imaging and ultrahigh fields.” NeuroImage 62 (2): 726-735.

Williams, D. S., J. A. Detre, J. S. Leigh and A. P. Koretsky (1992). “Magnetic-Resonance-Imaging of Perfusion Using Spin Inversion of Arterial Water.” Proceedings of the National Academy of Sciences of the United States of America 89 (1): 212-216.

What does it mean to understand the brain?

Fri, 06 Jan 2017 21:55:00 GMT

Thanks to Peter Bandettini for the idea of starting a blog, and for offering to let me partner with him in this endeavor. We hope you find it interesting.

In this my first contribution to theBrainBlog, I would like to outline some of my initial thoughts about what a useful understanding of the human brain might look like.

Starting at the bottom, I think we largely understand C. elegans . Yes, there are many details to be filled in, but we have the wiring diagram, we more or less know how the circuits work, and filling in the details is a foreseeable task using current technology. Importantly, the thing that most will point to as ‘understanding’ will be the statements in published papers that say things like: when the animal gets stimulus A, sensor B send signals to neurons C and D, which relay signals to neurons E, F, and G, which together decide whether to excite H and I to produce behavior J. The point is that what we think of as understanding is usually expressible in a reasonable number of sentences, and those sentences define a compact algorithm or fully described concept. When you ask an expert how something works, he/she always just starts talking, and keeps talking until you say “Oh, I get it”.

So, what happens when we look at the mouse? At this level of complexity ( 70 million neurons ), even if we can divide the brain into subsystems of a million or so neurons, I think there is no longer any hope of providing an exact algorithmic description of the function of these neural circuits, unless the system has many orders of magnitude of redundancy and is in fact a compactly describable system, which I think is doubtful. Otherwise, we are left with two classes of approaches. One is to give up the concept that ‘understanding’ involves distilling a phenomenon into a handful of sentences and declare instead that the wiring diagram and weighted connections themselves constitute understanding. Not ridiculous but certainly not very satisfying. The other is to hope that we can describe what we measure and learn about the subsystems of the brain (assuming that these can be identified) in a way that is significantly compact and yet complete enough that the interaction between subsystems can be modeled, and describe the function of the whole brain to our satisfaction. I think this is the common view (hope), that there is modularity. Break the system into functional modules and describe the whole as a hierarchy of connected components.

This general approach works for most things. In mechanical systems, components made up of 10 ²³ molecules are well described by bulk mechanical properties like tensile strength, shear modulus, and the ideal gas law, allowing for analysis that ignores molecules entirely. This provides us with a very clean modularity in that the net effect of smaller scale properties like molecular interactions are described to extremely high accuracy by these bulk properties, and allow us to bring only the bulk properties up to the next higher scale of analysis without significant compromise. Even in complex biological systems (like us), the function of many entire organs can be reduced to a handful of parameters. Witness the kidney, heart, and lung, each of which we can replace at least temporarily with a relatively simple machine. And even in the liver, which we cannot yet replicate, the number individual chemical functions are likely in the thousands, not millions or billions.

Will this approach work in the brain? I think this goes back to what kind of understanding we want. For asking fundamental questions about how basic functions like locomotion, visual processing, and foraging work, I think we can go right back down to the most basic organisms that do these things. The fairy fly for example, can walk, fly, find food, and reproduce, and his entire brain is about 20 microns. In this guy, going neuron by neuron seems like a good idea if we want compact algorithmic descriptions of how basic tasks are performed. But presumably we are interested in the mouse because we want to understand the more sophisticated and subtle processing that apparently requires 70 million neurons to support. So, modularity is the hope, but do we really believe that the subtleties in the brain function we are interested in won’t be lost if our models ignore billions of connections in order to impose a manageable degree of modularity? If we implicitly ignore these billions of connections by forcing our models into a countable number of functional units, aren’t we just modeling the fairy fly?

But what about finding correlates of behavior or other phenotypic expression in neural recordings? Doesn’t that bring a direct connection between neuron level information and behavior? In my opinion, sort of but not really. Imagine for example that you have trained an artificial neural network with 1000 neurons to discriminate between pictures of 6 types of birds. If you then start probing the network with virtual electrodes, you will find that by the time you have employed more than about 6 electrodes, you can find some combination of those 6+ signals that correlate with the 6 outcomes. Great, but does that teach you how the network works? I would argue that it only gives you a tiny peek in the window, no more than you would learn about how a CPU works by polling 6 of it’s transistors. By looking for correlates, we have defined a 6 dimensional question, and asking this question implicitly projects our 1000 dimensional system into this 6 dimensional space, and in this space a random sampling of 6+ neurons is likely to reveal correlations. But correlations in my mind don’t constitute understanding. If you want to know how the network actually works, and if it really needs 1000 neurons to perform its task, I would argue that you really need to know what every neuron is doing.

So this is what I really think. I think it is likely that the concept of crawling our way either up or down the size scale through the 9 orders of magnitude between neurons and the brain trying to determine the algorithms that are being implemented is probably not a useful way to pursue an understanding of the human brain. In a sense we already know how the brain works. It is a group of neurons that communicate using electrical impulses and synapses, that learns by adjusting weights by trial and error, and after 20 or 30 years of constant human instruction, it sometimes ends up not crazy. I think that fortunately, most of the interesting and useful questions are at the two ends of the size scale. At the big end there is the functional organization of the brain, which is clearly important, highly programmed, and largely accessible using fMRI and other imaging methods. At the small end there are things like cell type specialization, genetic factors and chemical transmitters and modulators that will likely be the levers that we push when we are ready for the next generation of brain tuning and restoration.

At the middle scales there are masses of neurons that implement unimaginably subtle and complex algorithms that do all the hard work, and those algorithms are only completely described by the 100 trillion dynamically changing synaptic weights, along with the chemical millieu, etc., that operate the machinery. I think we will never know algorithmically how AlphaGo beat Lee Sedol , let alone how 80 billion neurons conspire to decide the trustworthiness of the stranger at the door in 1 second.

So I think we already know where to get the bulk of what we need in order to arrive at a useful ‘understanding’. The macroscopic functional organization of the normal brain, which we can get from fMRI and other imaging methods, will help us to make a nice PBS series on ‘how the brain works’ that most people can watch and say “Oh, I get it”, which I would suggest is the popular definition of understanding. The functional (dis)organization of the brain in disease will tell our future nanobots where to deliver our cell type/genetic/chemical imbalance specific potions to fix the brain when it is broken, and the neuron scale information that we learn from electrodes, genetics, chemistry and microscopes will show us how to brew those potions.