Jul 18, 20206 min read

Messaging Can Help Fight COVID-19: Evidence from an RCT by Esther Duflo and Abhijit Banerjee

There is new evidence, from West Bengal, that messaging by celebrities can act as a ‘nudge’ to help in prevention and early diagnosis of COVID-19. We explain the implications of this study for India.

We are in the midst of an unprecedented health emergency in independent India. The challenges of low human development indicators, poverty, poor public health infrastructure, low levels of education, and population density compound the threat of a highly infectious pandemic.

In this circumstance it is vital that we pick the right policies to curb the disease. ‘Right’ in this context would mean a policy that has the desired impact, does not cause any harm, and does not cost more than the benefit it accrues. The most foolproof way to test a policy on these parameters is to build in a testing mechanism into the design of the policy.

Gathering evidence on the impact of a policy is not as easy as it seems. The easiest thing would be to measure results before and after the implementation of a policy or perhaps compare an area where the policy has been implemented to an area where it has not. The issue, however, is that in doing so one can never be entirely sure if the people who took up the policy, as against those who did not, are also different from one another in other ways. Or if something else happened to the people or area that received the policy at the same time. If any of these is the case then we might be picking up the impact of something else rather than the policy.

In order to deal with these issues econometricians have come up with methods that eliminate selection bias – in simpler words, methods so as to try and narrow down on the impact of the policy itself discounting all other factors that might have simultaneously shaped the outcome being analysed. The most effective of these methods is a Randomised Control Trial (RCT).

The use of RCTs to gather evidence of impact of policies has been pioneered by Nobel laureates Esther Duflo and Abhijit Banerjee, amongst others. Both of them recently designed an RCT alongside other colleagues to measure the impact of

the prevention and mitigation of COVID-19 in West Bengal. The project was implemented by J-PAL South Asia.

The questions they were trying to answer are as follows –

During health crises, like COVID-19, individuals are inundated with messages promoting health-preserving behavior. Does additional light-touch messaging by a credible individual change behavior? Do the features of the message matter?

Before we get into how they sought to answer this, some more details about the context. In India, by the end of March and beginning of April 2020, individuals received messages about distancing and hygiene through television, radio, public signs, local government addresses, and even a short jingle accompanying outgoing mobile phone calls. Besides these official sources, they also received messages via physical conversations, phone calls, WhatsApp, Facebook and Twitter.

In a survey in the Indian state of West Bengal, residents reported that on average in the two previous days, they heard about the importance of social distancing 20.2 times, about washing hands 16.9 times, and about wearing masks 17.2 times.

Absorbing and acting on multiple streams of information may be challenging and despite the ubiquity of messages, or perhaps because of it, compliance with basic recommendations was far from complete.

Non-compliance comes at a huge cost. Given the lack of adequate health infrastructure, particularly in rural India, prevention (handwashing, social distancing and wearing a mask) is the only way to avoid a large scale tragedy. Also, given the absence of widespread testing, it is imperative to set up an effective system of reporting COVID-19 symptoms.

In this context, the economists set out to answer the question stated above.

They designed 2.5 minute long video clips, delivered by Abhijit Banerjee, an author of this study and a public intellectual, who is native to West Bengal and well-known there, especially after his Nobel win.

These video messages were sent via a link embedded in a text message to 25 million subscribers of the phone network Reliance Jio (henceforth called Jio), randomly selected out of 28 million subscribers. The remaining three million received a text containing a link to the government website on COVID-19, similar to the many messages they have already received.

All video messages instructed individuals with cough and fever to contact their local frontline health worker. In addition, there were eight different variants emphasizing one practice (social distancing or hand-washing), one rationale for action (cost to self or cost to everyone including self), and, for some of the messages, a social problem (an explicit statement that ostracism of COVID-19 victims is unacceptable and should be reported to the authorities).

Experimental message treatments were randomized at the PIN code (Postal Index Number) level across 1214 out of the state’s 1264 PIN codes, covering an estimated 88 million people. This is critical to removing bias and narrowing down on the impact of the messaging itself.

A few days after the campaign, they conducted two surveys. First, they surveyed 677 frontline health workers from the sample communities to ask them about reporting of symptoms.

Second, they surveyed a random sample of 1,883 individuals drawn from a publicly available database of former and current local village council (gram panchayat) members. They measured health-preserving behaviors, including whether individuals traveled outside their villages, the number of interactions within two arms’ length (which was how the two meters mentioned in the message was interpreted in the local context), the estimated frequency of hand-washing upon returning home, and the use of masks or similar face covering when leaving home. They also measured the frequency of conversations that respondents had about COVID-19 (which included both face-to-face discussions, phone calls, or chats such as WhatsApp), beliefs about symptomatic and asymptomatic carriers, and knowledge of the virus.

Surveyed individuals included both those who received the message (Jio users) and those who were not messaged directly (non-Jio users).

Results showed that the nudge delivered through the video message by a well-known personality worked.

First, reporting of symptoms to the frontline health workers, critical to the tracking of the epidemic, doubled. Since the measurement was done within five days of the intervention, this almost certainly reflects reporting, rather than an increased level of sickness, given lag times for transmission and the time it takes for someone to become symptomatic.

Second, rates of not leaving the village, washing hands and wearing masks improved significantly in treatment communities. Critically, the effects are of similar magnitudes for behaviors targeted in the videos (not traveling out, washing hands) and those not even mentioned (mask use). Third, distancing went up in the sample where only hygiene was mentioned by more or less the same amount as when distancing was explicitly mentioned. Both these spillovers are consistent with the view that the message acted as a nudge.

Finally, magnitudes of the effects are also similar between Jio users and others, suggesting significant community diffusion of health behaviors. In other words, people were passing on this information to those who did not receive it and they were adopting the behaviour as well.

The conclusion from this study is that even against a background of a high level of messaging, an additional message by a respected public figure can still have large direct and indirect effects.

The policy implications of this study are as follows –

1. There is value in persistent messaging, at least by celebrities. Though there is a lot of existing information, a new message can act as a useful nudge and reminder and this may be particularly important if the conditions change and updating the information is necessary.

2. Given that governments have a fixed budget for messages and since within-community spillovers are large, it is important to disseminate the messages broadly across many communities, rather than instead using the same budget to target more people within a particular community.

3. The exact content of the message may not matter in such cases, since the information is out there and what is needed is a nudge to pay attention to it. Indeed, the literature finds that very long messages hurt comprehension so the message should be short rather than comprehensive.

4. Behavior and knowledge are not one-to-one. We have observed large shifts in distancing, hygiene, and mask-wearing despite little-to-no shifts in knowledge. Emphasis on behavior seems to be paramount.