How rewarding the conservative is killing innovation and discovery in science

There’s a great little trick that audiences can play on someone giving a lecture (try it next time you’re in a lecture). All the audience has to do is pick one side of the auditorium to be the happy side, and the other to be the angry side. If you are on the ‘happy’ side smile each time the lecturer looks over in your direction. If you are on the ‘angry’ side, frown and look unhappy. You will soon notice the speaker gravitating towards the ‘happy side’ of the auditorium.

185472365_7ae7f2303b_zThere is a near universal premise behind this little trick that is also the backbone of behavioural economics – we respond to rewards. Simply rewarding certain actions results in dramatic changes to behaviour. Behavioural economists and psychologists have shown countless times that rewarding or incentivizing behaviour can have profound effects on society, having both good and bad long-term outcomes.

There are many different ways we hear about new science. Conference talks, discussions with colleagues, reading published papers, reviewing papers for journals or even reviewing research grants. When a colleague tells me about a new piece of exciting research, the natural dynamic is that the attention is on them, they are giving the ‘presentation’, and I am the audience. I really only have a few possible response options:

1. Tell them honestly how cool and exciting their finding is.

2. Ask for clarification, then, once I understand, give some form of positive response.

3. Take on the role of skeptic and ask about alternative causes of their finding. To put it bluntly: try and show how their discovery is wrong.

Of course I’m not trying to be negative (I promise), but this scenario poses an interesting situation. As the ‘audience’ listening to my colleagues breakthrough discovery there are two ways to impress them: come up with a more exciting breakthrough, or logically falsify their discovery by giving an alternative explanation of their data. So if I don’t have a competitive discovery of my own, then the only way I can impress my colleague is to be more conservative or skeptical than they are, and tell them I cannot be sure their discovery is what they say it is. If I want to look good or impress, my only real option is to out-conserve them.4046758837_ec664ec1b5_z

Of course being conservative is an important part of science, precision and being sure of our claims is a must. However, the inherent asymmetry of falsification in the scientific method, naturally forces an audience into a position of more conservative = better. So once my colleague has finished telling me about their amazing new research, I have an urge to impress them, as one naturally does in the presence of a person they respect, so I quickly rack my brain until I come up with an alternative explanation of their data. “What about this…” I ask, “Couldn’t this also explain your data? You might need to run further control experiments to exclude this potential confound.” “Oh you are right, good point” my colleague replies with a sigh.

This in isolation is not really a problem, until we realise that what just transpired was analogous to the happy side of the audience smiling at the lecturer. Just like the lecturer and the smiles, by being more conservative than my colleague I was rewarded by their respect. So again, like the lecturer who will return to the happy side of the auditorium, next time I hear about a colleague’s new research I will move to the conservative side of the spectrum to get my reward.

rewarding_conservativeMight incentivizing being conservative, maybe, just maybe, produce an environment in which individuals shy away from risky or ambiguous science? Once the habit of conservatism has formed, it won’t just be applied to the work of others, but also our own. Novel leftfield ideas will be put aside as too crazy, too risky and potential discoveries will be lost. On a mass scale scientific progress will slow and operate in a much more conservative parametric ‘safe-zone’.

Its hard to know what kind of impact incentivizing conservatism might have on a large scale, but for a moment think through all the scenarios in which moving to the conservative side of the spectrum can win you the incentive of respect. Asking questions after a conference talk has the potential for a big reward, as you get the opportunity to impress many people at once. What about when reviewing papers? Or grants? Both provide opportunities to be rewarded for moving to the conservative side of the spectrum. When reviewing a paper, if I want a journal editor to respect me, the best course of action is to come up with an alternative to what the authors are claiming in the manuscript: maximizing conservatism wins me respect. Unnecessarily boosting conservatism like this forces people to use more resources to check all possible alternatives, while promoting conservative safe-science.

What is the effect of incentivizing conservatism year-in and year-out like this? Is this a bad thing? After all we want to be sure about science, especially if there are important or dangerous implications, say like with climate science or life saving medical procedures. However, at the same time we also want to maximise breakthrough discoveries. The discoveries that will most profoundly change our lives are the ones we aren’t expecting and can’t predict and it is precisely these that are lost by the science community becoming more and more conservative.

Black Swans, the now famous term coined by Nassim Taleb. It refers to the huge impact of unexpected rare events.

Black Swans, the now famous term coined by Nassim Taleb. It refers to the huge impact of unexpected rare events.

Do the rewards of discovery (respect etc.) outweigh the rewards for conservative skepticism (also respect)? Yes, probably, but there is a huge difference in difficulty between breakthrough discoveries and being conservative. There is still no real recipe or how-to guide for breakthrough discoveries. Whereas, for doubt and skepticism, simple logic will give you all the alternative possible explanations you need. Which means anyone can be a skeptic anytime it’s easy. However, coming up with new breakthroughs is hard and unpredictable. In other words, being conservative in science is an easier reward than going for a novel discovery.

Is there a way to prevent or counter the conservatism in science? One idea is that by simply acknowledging the nature of the incentivizing system, we should be less influenced by it.

Many venture capitalists invest in countless different ventures, knowing full well that the majority will fail, but they are relying on the radical success of a small minority. Just one huge success can more than make up for all the failures. Nassim Nicholas Taleb coined the phrase “Black Swan” to describe the huge impact of a rare and unexpected event. Black Swan investing, an investing strategy in which you bet on an extreme event, positive or negative, occurring at some stage in the market. Over time this investment strategy can be costly because everyday the rare event doesn’t occur costs you money, but then when a Black Swan event eventually occurs (GFC, a volcano, Google etc.), the win is so great that it dwarfs the accumulated slow loss.

Could we apply this Black Swan strategy to science? What would this look like? What would it involve? One way to apply such a strategy in science would be to fund and green-light high-risk, high-reward research projects, knowing full well that most of these projects will fail. But a small percent, and it only needs to be small, will end up being Black Swan discoveries, that have a huge impact on technology, medicine or our understanding the world around us.

Would such an ‘open-minded’ liberal research strategy by design change the conservative nature of science? Maybe not, but perhaps by explicitly acknowledging that certain science projects are by design high-risk and high failure, the reward incentives might shift away from conservative skepticism. In other words, by shifting the focus or value to discovery as opposed to doubt and conservatism, we might just be able to boost the number of life-changing discoveries.

Agree? Disagree? I’d love to hear from you…

Measuring the mind’s private images

Mental imagery, the voluntary retrieval and representation of sensory information from memory, has a fascinating biography. Historically, mental-imagery research suffered criticism because of methodological constraints caused by imagery’s inherent private nature. Recently, many objective research methods have been introduced that allow a more direct investigation into the mechanisms and neural substrates of mental imagery. These new methods have spurred numerous new discoveries, culminating in a flurry of impactful publications over the past few years.

Although imagery played a distinct role in discussions of mental function for thousands of years, empirical work on imagery did not gain strong traction until the last 30 or 40 years. Despite this recent traction, mental-imagery research has still not enjoyed the same degree of investigative attention that other psychological topics have. For example, this graph, shows that the number of articles published each year tfigure_1hat include the phrase “mental imagery” in the title, compared with those that include “visual attention” or “visual working memory,” is relatively low.

In the 1970s cognitive psychologists started to develop tricky methods to measure and study mental imagery objectively. Some of the early discoveries demonstrated a clear relationship between the content of mental images and the time it took to generate or manipulate them (Kosslyn et al., 1978; Shepard et al., 1971). The larger the imagery manipulation, the longer it took to complete, suggesting a correspondence between imagery and physical space.

More recently, there has been a jump in brain-imaging work investigating mental imagery. A recent trend of analyzing the information content of fMRI patterns (instead of the mean amplitude change) has yielded interesting results. This work is often described as decoding because one of the more popular methods trains an algorithm to decode, or make a prediction about, the experimental condition or task, on the basis of the spatial pattern of the fMRI signal across a brain area.

More recently there has been a jump in brain imaging work investigating mental imagery. A recent trend of analysing the information content of fMRI patterns (instead of the mean amplitude change) has yielded interesting results. This work is often described as ‘decoding’, as one of the more popular methods trains an algorithm to decode, or make a prediction about the experimental condition or task, based on the spatial pattern of the fMRI signal across a brain area (Tong et al., 2012).

Recent work from our lab has demonstrated that imagery can facilitate subsequentperception (Pearson et al., 2008). By separating the period of imagery generation and perception in time, the effects of imagery can be examined without the potential confounds of attention (Carrasco et al., 2004). This work demonstrated that when individuals imagine one of two patterns, that pattern has a much higher probability of being perceptually dominant in a subsequent brief binocular rivalry presentation (Pearson et al., 2008; 2011). In other words, the content of the mental image primes subsequent dominance in binocular rivalry – it changes visual awareness of the rivalry display. Binocular rivalry is a visual phenomenon that occurs when two different visual stimuli are presented one to each eye, such that they are forced to coexist at the same visual location. One pattern tends to be dominant over the other, forcing it out of awareness. Binocular rivalry has been a hugely popular tool to study visual awareness in recent times (Tong et al., 2006). However, this work used rivalry as a tool to measure the sensory strength or ‘visual energy’ of mental imagery, enabling individual episodes of imagery to be assessed in an indirect and objective sensory manner. This discovery is also interesting in its own right, as it demonstrates that what we imagine can literally change how we see the world.

To read more on recent developments in objective methods to measure mental imagery check out the recent paper from which some of the above text was taken:

Pearson, J. (2014). New directions in mental imagery research: the binocular rivalry technique and decoding fMRI patterns. Current Directions in Psychological Science. 23(3), 178-183.

 

Or catch my upcoming tutorial at the 2014 ASSC meeting: Seeing what’s not there and measuring it: Conscious perception without a stimulus

 

 

References

Carrasco, M. et al. (2004). Attention alters appearance. Nature neuroscience, 7(3), 308–313. doi:10.1038/nn1194

Kosslyn, S. M. et al. (1978). Visual images preserve metric spatial information: evidence from studies of image scanning. J Exp Psychol Hum Percept Perform, 4(1), 47–60.

Pearson, J. et al. (2008). The functional impact of mental imagery on conscious perception. Current biology : CB, 18(13), 982–986. doi:10.1016/j.cub.2008.05.048

Pearson, J. et al. (2011). Evaluating the Mind’s Eye: The Metacognition of Visual Imagery. Psychol Sci. doi:10.1177/0956797611417134

Shepard, R. N. et al. (1971). Mental rotation of three-dimensional objects. Science (New York, NY), 171(3972), 701–703.

Tong, F. et al. (2006). Neural bases of binocular rivalry. Trends Cogn Sci, 10(11), 502–511. doi:10.1016/j.tics.2006.09.003

Tong, F. et al. (2012). Decoding Patterns of Human Brain Activity. Annual review of psychology, 63(1), 483–509. doi:10.1146/annurev-psych-120710-100412