Optimizing The Washington Post retention funnel
Project team
Product Manager, Product Designer (my role), Marketing Director, Engineering: FED/BED/QA, Analyst, Customer Service Director
Background
The self-service cancel flow is a relatively new feature at The Post. At its core, we believe that canceling a subscription should be low-friction if someone is truly ready to end their relationship with us—however we also recognize the need to balance that with profitability and the greater needs of the business.
We’re constantly optimizing ways to surface subscription value in this flow so that subscribers self-defer from canceling—but historically, our strategy has been to simply just kick the can down the road by putting a lower offer in front of them to stay. This left us wondering:
How might we surface value beyond monetary savings at the point of cancellation so that subscribers truly understand the value of their Washington Post subscription?
The control cancel flow already included a rote list of benefits—but there wasn’t much context, so we saw a plethora of opportunity to dive into.
Control user flow:
Hypothesis
Displaying subscription benefits prominently, at the beginning of the cancel flow, will increase discovery and exploration, and lower overall cancellations.
Through this A/B split test, we aimed to answer two questions:
Does moving the benefits page from Step 3 to Step 1 increase the cancel drop-off rate?
Which benefit variant—“Subscriber Features,” or “Exclusive Journalism”—is more enticing, thus resulting in a higher drop-off rate?
Variant A (“Subscriber Benefits”), highlights three key benefits of being a subscriber, with links out to these features to show them more:
Subscriber-only programs—subscribers can attend live programming with government and business leaders
Interests selections page—users can customize content recommendations to surface more of what matters to them
Podcasts landing page—Subscribers often cancel because they don’t have time to read. Listening to the news can sometimes be an alternate, time-saving medium to reading
For Variant B “Exclusive Journalism”, we partnered with newsroom editors to feature three high-converting pieces of content exclusive to the Washington Post. These articles have historically been top-converting from the paywall, so it made sense to try them here as well. In this variant, we include an illustration, headline, and blurb, paired with an explicit call to action to promote interest and curiosity.
Test results
The test ran from April 12, 2022–May 4, 2022. The winning variant rolled out May 5.
Both variants worked well to lead people away from the cancel flow, but the overall winner was Variant B “Exclusive Journalism.”
Key insights
Both variants outperformed the control in terms of leading subscribers away from the cancel flow and encouraging them to reengage with our content and benefits, however Variant B “Exclusive Journalism” was the clear winner.
Only half of those who abandoned the flow to explore benefits/journalism, returned to cancel either that day, or on a different day. We are continuing to monitor this until we reach statistical significance, so that we can truly understand the broader impact of this updated flow on subscriber retention.
What’s next
There’s much experimentation and tinkering to be done on this experience. A few things we’d really like to dig into and optimize in the coming quarters:
Understand which subscriber features they are and are not using, and leverage that information as a retention tactic at the point of cancellation.
Be smarter with presenting low-priced offers as a retention tactic. How might we offer a discounted rate that is customized to the user so that it’s not a blanket discount, but the right offer for them in that moment?
Experiment with emotional appeal by testing pricing transparency tactics. Does disclosing “where the money goes” at the point of cancellation remind subscribers that their subscription funds boots on the ground journalism?