Growing Movements & Saving Birds With Behavioral Science
How can organizations grow participation in science that protects our world? And how can we design research to study behaviors that matter?
Four days in every February, tens of thousands of people around the world join together in the Great Backyard Bird Count (GBBC). By observing birds and entering “checklists” into the eBird database, people are contributing to the world’s largest biodiversity-related community science project. At the Cornell Lab of Ornithology, scientists hope that the GBBC can inspire first-time bird-watchers to continue bird-watching in ways that advance science and help protect our world.
How do environmental and community science organizations decide what kind of messaging to use? In the book Analytic Activism, David Karpf explains that many organizations brainstorm ideas and just choose whatever sounds best. Another option is to learn from science. For example, Sarah West and Rachel Pateman have published an article on recruiting and retaining participants in citizen science that suggests ideas from social psychology and economics.
Even with academic guidance, how can organizers know which idea among many might work the best for their audience? One powerful option is to conduct a test.
Last spring, students in the Cornell class I teach on the Design and Governance of Digital Experiments worked with the Lab of Ornithology to design an experiment that tested messaging for the Great Backyard Bird Count. Teams of students read scientific literature, proposed possible messages based on the literature, and pitched them to the Lab of Ornithology. The Lab chose the three that they could learn the most from and used them in an experiment that we designed together.
We agreed on three message ideas:
- The control message used the lab’s standard annual email, with the subject “GBBC e-News.” A call to action linked to more information about birds.
- A conservation message, with a headline that “It’s up to us to bring birds back!” The call to action linked to a FeederWatch, a popular donation-based program to spot and save birds
- A intrinsic motivation message (shown above) promised results from the bird count. The call to action linked to eBird, where people can report the birds they spot without paying a donation.
In this experiment, we randomly assigned people who were first-time contributors to the Great Backyard Bird Count (9,608 people) to receive one of these three follow-up emails.
We also designed the study to measure the behaviors that mattered. The behavioral scientist Matt Wallaert calls this starting at the end. Even though we did record whether people opened the email, we cared much more about whether people continued to volunteer. So we also recorded the number of lists of bird-sightings people contributed to eBird and whether they signed up for other programs like FeederWatch (full experiment plan here). This study was reviewed and approved by the Cornell University Institutional Review Board.
What We Discovered
We started by looking at email opens. Among people who received the standard newsletter, 41% opened the email. Across all groups, most of these were people who contributed at least one checklist to the bird count before we sent out the email. While 14% of people who never contributed checklists did open the follow-up email, 100% of those who contributed checklists opened the email (p<0.001).
Changing the headlines did influence reading behavior (full analysis here). The “we still need your help” email subject increased opens by 4.8 percentage points on average (p<0.001) compared to “GBBC eNews.” If scaled up, this could increase the number of email opens for this email by 461 newcomers on average.
Contrary to our expectations, we didn’t find a difference in email opens between the newsletter email subject and the “It’s up to us to bring birds back” email subject.
Did different messaging have downstream effects on someone’s later contributions to community science? While we didn’t observe an effect on whether someone would contribute to eBird, messaging did influence the number of contributions that people made on average. The “we still need your help” email increased the rate of eBird contributions by 2.6 percent compared to the control group (p<0.001). If scaled up, this could increase the sum of submitted lists across all 9,608 first-time GBBC participants by a total of 34, on average.
We didn’t find an effect from the “Bring Birds Back” email on participation in FeederWatch, either because fewer people opened the email, or because the barriers to joining FeederWatch (such as donation cost) were too high to detect a significant effect. The result is similar even after adjusting the model for how many contributions someone made before receiving the message.
What Does This Mean for Community Science?
What do we learn from this messaging experiment?
In this study (which has not yet been peer reviewed and is a pilot test for a larger project), we showed that the messages someone receives early in their volunteer experience can influence their longer-term contributions to community science.
Clear, persuasive messages and straightforward pathways for participation can increase people’s contributions to community science. We found that people volunteered more over time when we gave them information on the outcome of their contributions and when we explained that their help was still needed.
People shared more observations when we gave them information on the outcome of their contributions and explained that help was still needed
We also learned that people who submit bird lists during GBBC are much more likely to engage with content and opportunities later on — but that some of those who never submit a list are also potential long-term contributors. While non-participate might indicate lack of motivation or opportunity for on-going contribution, some non-participants might just have not seen any birds that day. Future initiatives should look separately at first-time contributors as distinct from people who signed up but haven’t managed to contribute yet.
What do we learn from the failure of bird-conservation messaging to increase participation? With just one study, we don’t have enough evidence to rule out conservation as a more generally persuasive messaging approach. First, it’s possible that the people who join the GBBC might be different from the wider population. Those volunteers might uniquely have reasons other than conservation to participate. Perhaps if the GBBC could broaden its recruitment to include more conservation-minded people, those messages might be more effective. Second, it’s possible that our messaging was incorrectly framed. Recent research has shown that poor framing of environmental messaging can lead to under-estimates in the number of people who care about the environment, especially across race, ethnicity, and socio-economic status.
Clear, persuasive messages and straightforward pathways for participation can increase people’s contributions to community science
Even if you’re not in conservation or community science organizations, you can still learn something from the study. By measuring the behaviors that matter to your organization, you can go beyond open rates and view counts to measure impact of your interventions on behavior.
Acknowledgments
Many thanks to Becca Rodomsky-Bish, Max Klein, Tina Phillips, and the students of COMM 4242 for putting together this amazing study!
References
- Karpf, D. (2016). Analytic activism: Digital listening and the new political strategy. Oxford University Press.
- West, S. E., & Pateman, R. M. (2016). Recruiting and retaining participants in citizen science: what can be learned from the volunteering literature?. Citizen Science: Theory and Practice.
- Song, H., Lewis Jr, N. A., Ballew, M. T., Bravo, M., Davydova, J., Gao, H. O., Garcia, R., Hiltner, S., Naiman, S., Pearson, A., Romero-Canyas, R., Schuldt, J. (2020). What counts as an “environmental” issue? Differences in issue conceptualization by race, ethnicity, and socioeconomic status. Journal of Environmental Psychology, 101404.
