Economic Security Project Action first tested four Keep Families Afloat ad concepts with Civis Analytics using their Creative Focus platform to identify the top two performers among a registered voter audience. Those ads were then given to BPI for real-world testing.
BPI constructed a universe of 1.3M voters across OH, GA, PA, and NV based on KFA’s voter file model that predicted support for the Child Tax Credit, targeting voters who were identified as strong CTC supporters and were likely to be either campaign turnout targets or persuasion targets. A randomized-controlled trial was set up in which they were split into 3 treatment groups. Groups 1 and 2 saw different ads highlighting Democratic Senate candidates’ support for the Child Tax Credit and its impact on families, Group 3 saw a non-political placebo message.
Video ads were delivered to these voters across online pre-roll ads shown before news, entertainment, and other video content, as well as on Connected TV apps (e.g. Roku, Tubi, PlutoTV) for 2 weeks, and voters reached with an ad saw it an average of 5.4 times. In the week following the campaign, 2,230 survey responses were collected from those targeted with ads via BPI’s Vantage platform, which delivers surveys in pre-roll video ad slots. (This is a different approach from an in-survey test, where voters see the ad and immediately get asked questions about it; in this case the ad and the survey questions were separated by anywhere from a couple days to a week.
Performance was measured on 2 attributes:
- Trust in candidates to support parents with young children
- Vote choice
The key findings were:
- The top-performing ad drove a 9-point increase in Trust and 15 point increase in Democratic vote choice (Note: Respondents were presented a “Do Not Know” option for vote choice, which is often most utilized in this type of interruptive pulse survey methodology. However, the level of vote choice movement in this test likely reflects both a true persuasion effect as well as strong recall of the ads viewed the week prior.)
- Despite relatively similar attention + engagement performance from the ads, the “Earrings” ad – featuring a heartfelt story of a family in need – had a far greater persuasive impact than the more upbeat and cutting youth-narrated “Only One” ad.
- Ads were most effective in Ohio and Pennsylvania, with smaller effect sizes in Georgia and Nevada. Nevada had a smaller sample size on the survey and more uncertainty about the effect size.
- Ads performed best among Turnout voters, defined as people with a high Dem partisanship, a moderate turnout propensity (30-70%), and who are highly likely to support the Child Tax Credit (80%+ CTC support score on TargetSmart voter file model).
KFA’s voter file model used in this test is publicly available for any organization on TargetSmart. Given these strong results, BPI recommends running this message to strong (80+ partisan score) Democrats for mobilization efforts.