How Some Experiments Use Emails to Control for Systemic Bias
Researchers put a twist on a classic experimental design.
How Some Experiments Use Emails to Control for Systemic Bias
Francesco Ciccolella
Every day, people make dozens of choices about how to deploy mental effort and where to focus. Do we give attention to work or play? Should we exercise or zone out in front of the TV?
The decisions we make appear to underrate the importance of attention, according to the results of a study. A group of researchers including Chicago Booth’s Eric Zwick find that we often make irrational decisions, undervaluing how much attention is worth relative to distractions.
“Imagine a spectrum between ‘we’re totally lost and clueless’ on one side and ‘we’re a human computer’ on the other,” Zwick explains. “I’d say we’re maybe one-third of the way toward the human computer.”
The “rational inattention” theory assumes that people fully understand the consequences of, say, scrolling through Instagram instead of hunkering down and writing that report—and that we choose where and how best to put our attention. To test it, the researchers ran three experiments that gave people the opportunity to reduce the demands on their mental bandwidth.
For the first experiment, they asked nearly 1,400 US students enrolled in an online coding class to complete three 15-minute coding lessons each week. The students were rewarded monetarily for completing the lessons, with a randomized range of bonuses between $1 and $5. To help make completing the tasks easier, the researchers offered students a plan-making tool to block out three 15-minute windows on their online calendars.
The researchers—interested in nudges that could help people build good habits—find that students who used the tool were more likely to complete the tasks and get the financial reward, but adoption of the tool depended on the incentives. As the incentives for completing the weekly lessons increased, so did the tool-adoption rate. This makes sense: the more you get out of doing something, the more likely you are to invest in making sure you complete the task. However, more participants should have used the planning tool, according to the theory. The fact that they didn’t led researchers to conclude that participants undervalued it.
In the second experiment, the researchers offered financial bonuses to 2,300 participants to complete an online survey. The catch? The survey wouldn’t be sent to them for anywhere from two days to six weeks. To help participants complete the survey and earn their bonuses of $3–$12, the researchers offered to send a set of three reminder emails, gauging participants’ willingness to pay for them. But as in the first experiment, they find that participants undervalued the reminders.
Can you pay people to concentrate?
The researchers asked participants to complete a series of weekly coding tasks. When offered an incentive, more participants completed the tasks—and used a planning tool to help them focus. But the effect wore off over time.
Could people be prompted to value these sorts of nudges? In a third experiment, the researchers briefly showed participants a series of equations and asked them to state whether most were correct or incorrect, indicating that participants would be rewarded for accuracy. Before doing this a second time, the researchers offered to make the challenge easier for a small fee. Participants were willing to pay some money for an easier task and increased likelihood of earning the bonus, but they weren’t willing to pay as much as the theory had predicted.
The findings can be applied to many aspects of what Zwick calls today’s “distraction economy.” Exercising now could reduce health problems later, and adopting bandwidth enhancements such as automatic reminders could help us complete chores—be they saving money or booking a medical appointment—that we’d rather put off.
“This offers some kind of structure on how we can tame them [the demands on our attention],” says Zwick. “If people aren’t optimally allocating their attention and dealing with all these distractions, we need to think about interventions to help them get closer to what they could ideally be doing.”
Erin T. Bronchetti, Judd B. Kessler, Ellen B. Magenheim, Dmitry Taubinsky, and Eric Zwick, “Is Attention Produced Optimally? Theory and Evidence from Experiments with Bandwidth Enhancements,” Working paper, June 2022.
More from Chicago Booth Review
One Factor You May Be Leaving Out of Your Financial Decision-Making
Do your choices reflect the marginal utility of money for you?
One Factor You May Be Leaving Out of Your Financial Decision-Making
Why Locking In Subscribers Is Bad for Business
Exploiting customers through discounted, auto-renewing promotional offers can backfire.
Why Locking In Subscribers Is Bad for BusinessMore from Chicago Booth
Your Privacy
We want to demonstrate our commitment to your privacy. Please review Chicago Booth's privacy notice, which provides information explaining how and why we collect particular information when you visit our website.