Quick takeaway: You can’t design a single-stage incentive and expect a long-term behavior change (enter: gamification). Also, designing behavioral interventions without understanding prior work in psychology is dangerous and possibly unethical.
For years, I’ve been deeply fascinated by behavioral economics. I love the discipline’s emphasis on incentive and reward, and the increasing realization by the “capital-E” Economics community that people are not purely rational. I’ve been in awe of the success of platforms like Freakonomics and the excitement wrought by dozens of million-plus viewer Ted talks about how simple nudges can help people for the better. At the same time – as someone who’s studied psychology most of my life, I’ve struggled with BE’s lack of historical context. Every time I listen to a BE professor or practitioner, I’m amazed at the tunnel vision they have for their field: many fail to incorporate the “soft” understanding of motivation and behavior that had been developed by psychologists and psychiatrists over hundreds of years.
I’ve not however viewed the field’s failings as dangerous until recently. And now – with new research on Indian schoolkids out this week (h/t @dustinshell) – the limitations of BE are starting to become apparent.
Researchers devised an incentive program to see if they could raise attendance at a poor, rural Indian school: attend for 85% of a month and receive a prize – in this case a pencil/eraser set. The students were able to see and touch the prize in advance and the overall results were amazing: the lowest-attendance students increased their participation by 2.3x. After the contest was over, attendance returned to the baseline before the test except for one key group: those who failed to meet the target. Their attendance rate dropped below the baseline after the experiment.
The researchers hypothesized that the reason for this was that the experiment pointed out their failure to attend. Perhaps the experiment also demotivated these students because they were then forced to publicly acknowledge their failure with their classmates. Whereas before poor attendance was a subjective issue, now the failing students didn’t have the special pencil/eraser and everyone could see their “shame”. Perhaps the core reason they don’t attend is more emotional or circumstantial (caring for a sick relative, lack of income, etc.) so the shame they felt at failing was compounded by the demonstrable reward and incentive.
But what about the shame that the researchers should feel for constructing such a poor experiment? Or the collective weight of “not invented here” stupidity that the Behavioral Economics profession continues to foist on the world. A simple reading of the psychological literature would quickly reveal a ton of research on the negative outcomes of failure incentives both in children and adults. There is also plenty of work on the overjustification effect and how this kind of incentivization is unlikely to produce long-term results unless there is an ongoing system of reinforcement and internalization of reward and motivation.
Did any of that stop yet another BE researcher from performing a study on at-risk, poor people in the developing world? No. Did this research somehow slip through the ethics and review process without anyone using common sense to ask “Hey, what do you think will happen to those that fail?” Yes. Did the reporters who covered it ask anyone other than the researchers to contextualize the results? Nope. Of course, these kind of mistakes aren’t just confined to BE – every discipline has its share of ethical research issues.
Ultimately however, I feel the true failure lies in not understanding that BE is a close relative of psychology – not a truly separate discipline. It’s fine to use BE in isolation to conceptualize how rewards might be used to change behavior, but BE alone cannot succeed in driving end user results long-term. We must think of motivation and incentives as a related “system” when designing for behavior change; trying to succeed with only one or the other will not yield optimal results.
Of course, people have to genuinely care about the behavioral change outcomes themselves, not just answering questions about variables in isolation. By default, this means that the “experiment” must run over a period of time, evolve to its participants and have outcomes in mind. This is part of the reason that gamification is so much more effective at changing behavior over a period of time than its antecedents. It’s a systematic approach that considers the user’s environment, motivation and incentive to design for success. Gamification uses behavioral economics, psychology – and even loyalty program design – to coax and nudge users while also teaching them the importance of the behavior. It is essential to get a handle on all these underlying elements if we want to change behavior and change the world.
Of course, the researchers (who were economists, public policy experts and marketers) expressed contrition once the results came in, musing that perhaps they now regret doing the experiment in the first place.
For many reasons, so do I.
Image credit: CC Flickr user Simply CVR
Link goes to Apple, not Indian study.
Thanks for the heads up. Link is fixed.
I agree that manipulating people’s incentives can be dangerous! What the research designers failed to appreciate is that people make decisions in a social and moral context. By adding an economic incentive for attendance, they simultaneously reduced the social and moral incentives (social norms) for going to school. Gamification, perhaps, creates a new “social context” within which we can change engagement and motivational patterns safely.
This is an example of an unethical study.
a) those scientists could have easily figured that out beforehand, as there are enough studies that indicate that this is not working (here are some more examples: http://tinyurl.com/on8rvte)
b) they actually inflicted damage to the kids. Those kids who were already struggling with attending now attend even less; this is an unethical behavior from the scientists
c) they should have used a method such as 5 Whys to figure out why some kids struggle with attendance. They may have figured out beforehand that fucking erasers are the least problems of those poor kids. There may be reasons such as no transportation, not enough food, work that they are forced to help at home, abusive parents/caretakers etc.
Now those fucking idiot scientists even punished them on top of their struggle for survival.
DISGUSTING!!!