It’s New York City in the riotous 1970s. The population wants to get in touch with itself and so has unbuttoned its stiff collars, marched for its rights, experimented with drugs. Not far from midtown, by contrast, I imagine the cadence is systematic. A man hails a taxicab in one motion, and his colleague steps in beside him. They ask the driver to take them to the Algonquin Hotel where they’re scheduled to attend a meeting over dinner. The conversation turns to a merger that’s been a long time coming, a way of unifying efforts to pick apart the workings of the human brain. By the time they arrive, a new field of science is christened.
Cognitive neuroscience was famously coined by George A. Miller and Michael Gazzaniga on a taxi ride to a scientific meeting, as Russ Poldrack describes in his recent book. Up until that point, psychology and neuroscience had proceeded in parallel lanes to understand the mind and the brain. It’s not a coincidence that with the 1970s came the first serious investigation of how the human brain creates the mind. The culture was self-curious. Moreover, this new science depended critically on the invention of technologies for noninvasively imaging the brain in living, behaving humans. The patron of the first efforts to image human mental processing, James McDonnell, had earned his fortune as a supplier of fighter aircraft. In the wake of two world wars, the country’s resources had been channeled back from overseas battlefronts toward tech development in the states.
Unlike the 1970s, the term cognitive evokes an air of sterility. It derives from the medieval Latin cognitio for “knowledge, perception, a judicial examination, trial.” In modern psychology, cognition falls on one side of a dichotomy often drawn between processes for thinking and feeling. This goes back at least as far as the 1920s, when Carl Jung defined the two as distinct types. One can only speculate why Miller and Gazzaniga chose the cognitive half of the mind for the namesake of their field. If I had to guess, it might have something to do with power. Objectivity reigns in the sciences, and to be taken seriously, it was crucial that the biology of mental ephemera be regarded as an objective endeavor.
Subjectively, cognition has the flavor of a process that can be studied by objective methods, whereas emotion has a slippery feel – but we can go one step further in our consideration of their relative objectivity. In the spirit of science, we'll test for ourselves whether the cognitive can be studied more objectively than the emotional with a few experiments. To conclude, we'll consider the implications of the apparent divide between cognition and emotion in naming the field that uses neuroimaging to study human brain function.
The hard sciences of thinking and feeling
To study our own cognitive and emotional processes, we'll use a classic approach from psychology. The goal is to measure a mental function of interest without also measuring the processes involved in viewing a computer screen and making motor responses. This is accomplished by the subtraction method. Researchers design task conditions that differ only in the mental function they seek to isolate, then subtract the corresponding measurements. In psychology, the measurements are typically reaction times, which give a clue to the time it takes the brain to process information. Below, you can apply the subtraction method to isolate the time it takes for a cognitive process:
Tasks adapted from the Open Cognition Lab.
The stimuli in the two conditions differ in one important way. In the first condition, the stimuli are congruent – the words match their font color. In the second, incongruent condition, the words and their font colors are mismatched. You might have noticed that it was more difficult to perform the second condition, and it probably took a bit longer. This is called the Stroop effect after John Ridley Stroop, the inventor of the task. To compute the time it takes to resolve the cognitive interference due to the word-color mismatch, simply subtract the reaction time of condition 2 from that of condition 1. Congratulations – you've isolated a cognitive process! Next, try a variation on the Stroop task:
With a small tweak to the classic Stroop task, it’s possible to isolate an emotional effect. If you're like most individuals who perform this task, then it took longer to perform the second condition, when words were charged with negative emotion. (Note: The trials were not randomized between conditions, and you've been learning along the way, so don't be worried if you got faster instead!) An increase in reaction time for the second condition is because the emotional words capture your attention, distracting you from responding to the font color. If a group of individuals had performed this task in an MRI scanner, the brain structures involved in emotionally-guided attention could be identified by subtracting hemodynamic response signals between the two conditions. Results are shown below for meta-analyses of fMRI studies that did just that. In the middle is a new task which combines features of the color Stroop and the emotional Stroop tasks.
Brain maps show locations of the difference in hemodynamic response for the contrasts shown below, combined across studies by activation likelihood meta-analysis. Color Stroop results are from Nee et al. (2006), Figure 6, which included 11 studies. Emotional Stroop results are from Song et al. (2017), Figure 2, which included 9 studies in the incongruent – congruent contrast and 7 studies in the negative – neutral contrast.
The above experiments go to show that emotional processes can be isolated by the same general scientific method as cognitive processes. Moreover, both emotional and cognitive processes have been mapped consistently to the brain in fMRI meta-analyses, indicating that they are both supported by neurobiological systems conserved across people. These findings leave me unconvinced that the study of emotion is any less objective than the study of cognition.
Furthermore, there’s a meta-scientific reason that cognitive and emotional processes are of equal importance to the field. If you look at what neuroimaging tools have actually been used to investigate over the history of the discipline, the processes are a mix of the cognitive and the emotional. Below is a map of the topic structure of 1,763 studies that used fMRI to study human mental processing. The red and blue papers on the left fall into categories at the two poles of emotion, from positive valence (e.g., happiness) to negative valence (e.g., sadness or fear). The green papers in the upper left corner studied a mix of cognitive processes such as interference and working memory. The purple papers on the bottom right addressed social processes, primarily language, which include components related to emotion (e.g., tone of voice) and cognition (e.g., grammar).
To plot the topic structure of the field, I first mapped citations from one paper to another across 1,763 fMRI studies from the BrainMap database. Topics were detected by an algorithm that maximizes modularity (i.e., how distinct the topics are from one another). The topics were then matched to the domains of the Research Domain Criteria framework based on the words for behaviors, self-report measures, and task paradigms in their full texts. Finally, I applied a threshold to show papers with at least ten links. This sample of papers isn’t comprehensive, but in ongoing analyses with ten times as many, the results are similar.
An inclusive science of human brain function
Cognitive doesn’t quite do justice to the seriousness and the extent of studies on emotional processing carried out with human brain imaging. But what to call the field? Cognitive/affective neuroscience doesn’t exactly roll off the tongue. We might consider splitting the field in two, but as I'll argue, these two sets of processes are more than the sum of their parts. The final point I’d like to make about the thinking-feeling split is one which blows up the preceding discussion altogether. The divide between cognition and emotion is artificial – more or less made-up to make it easier to talk about and study experiences that we perceive as qualitatively distinct. The language takes a complex web of interwoven neurobiological systems and snips it in half.
One illustration of how the thinking-feeling dichotomy breaks down is in the way that humans form memories. If you define memory in terms of information storage and retrieval, it sounds like the poster child of cognition. However, take a moment to think back to the earliest scene in your childhood memory. Have it? Chances are that it was either a time of pure joy – when you got the toy you wanted for your third birthday – or a time of trauma – an injury, a death, a sudden loss. What the brain stows away into long-term memory is a function of emotional state at the time. When this goes wrong following a traumatic event, individuals may experience vivid flashbacks, one of the defining features of post-traumatic stress disorder (PTSD). In addition to re-experiencing events, PTSD is characterized by a host of symptoms that are both cognitive (e.g., distorted thinking patterns) and emotional (e.g., negative mood).
The intermingling of cognitive and emotional processes is one reason why you shouldn’t believe claims that the left brain is responsible for cognition and the right brain for emotion. It’s just so much more complicated than that. An important finding made by imaging the human brain is that processes thought to be purely cognitive involve brain regions formerly thought to be emotional hubs. For instance, basic visual perception is guided by input from the amygdala that signals the salience (i.e., importance) of objects in the visual field. This pathway enables individuals blinded by visual cortex damage to respond to stimuli they can’t otherwise see, a remarkable phenomenon known as blind sight.
At last, let’s return to the question of what to name the field that uses neuroimaging to study human brain function. I apologize for stringing you along this far, because as a lowly grad student, I’m entirely unqualified to make this call. What I will say is that I've taken to calling myself a human neuroscientist. It’s a bit of a cop-out, letting chaos be chaos when it comes to the interdependencies between cognition and emotion – at least for the time being. I anticipate that in the years to come, the most exciting neuroimaging results will point out concrete ways in which mental processes have been over- and under-specified in the language of psychology. Moreover, human neuroscience appropriately emphasizes what I see as two key features of the field:
1. Humans are the subject of the science. We’ve sacrificed the ability to directly measure and manipulate neuronal activity, as is possible with animal models. Instead, we primarily apply noninvasive neuroimaging tools to understand mental functions specific to our species. It’s a rather selfish priority, but better to acknowledge it upfront than to dance around it, in my opinion.
2. As neuroscientists, we’re humans, too. This one is a bit cheeky, but hear me out. We now have technologies that allow us to objectively measure the biological substrates of our human traits. Yet, how we wield those technologies is subjective at every step. As just one example, our tendency to bin mental processes into categories like cognition and emotion arises out of a bias to perceive structure in the world (e.g., check out the closure and categorical perception phenomena). We must be conscious of our biases in order to control for them, restrain our interpretations, and improve our language for human brain systems.
So, there you have it: human neuroscience. The name is grounded in appreciation of the priorities and limitations of the field. Notably, I can’t take any credit for it. There’s a leading journal called Frontiers in Human Neuroscience and an assortment of labs that carry it as their banner. I’m likewise holding up a sign for it, like any activist on any city street in the 1970s.