Skip to main content
A customer who got an email and stayed might have stayed anyway. Each email’s Results tells the two apart by comparing it with a holdout group: eligible customers left out at random, who would otherwise have got it. Results live with each email’s customers. On At risk, Ending soon or Cancelled, switch from Customers to Results. Each tab’s Results starts with its own reports (the outreach log and Churn Signals report on At risk, month by month on Ending soon, the survey’s answers on Cancelled), then the email’s holdout comparison:
The Cancellation survey's results, with what the email did and what customers did outlined

Choose the period

  • Decided in: emails sent, or held back, in the last 90 days, 180 days or year.
  • Outcome measured after: how long after each email to look at what the customer did: 30, 60 or 90 days.

What the email did

Sent, delivered, opened, clicked and replied (and, for the Cancellation survey, survey answered), with how many weren’t sent. Delivery and opens come from Postmark’s reports; opens are estimates, since some mail apps load images automatically. Test emails never count.

What customers did

For the emailed and held-back groups: who kept their plan, reactivated, is still paying, or churned, and the share saved, with its MRR. Customers still within the measurement window show as in progress. Above the groups, OneMRR says what the comparison shows:
  • The email made a difference: the emailed group did better, clear at 95% confidence, with about how much MRR stayed because of it.
  • Emailed customers did worse: worth reviewing what the email says.
  • No clear difference yet: the 95% range still includes zero. More customers will narrow it.
  • Too early to compare: each group needs 20 customers whose outcome window has passed.
  • No holdout group: OneMRR shows the emailed group’s outcomes, but says they might have stayed anyway.
To measure properly, set a Holdout group for each email in Setup › Emails. The pill beside the email’s name shows its holdout: No holdout, Holdout off, or the share held back. Replies count when they come back through OneMRR’s reply address, which also forwards them to your reply-to. With white-label sending, replies go straight to you and aren’t counted.

Not emailed any more

People who unsubscribed, whose address bounced, or who marked an email as spam are listed at the bottom of Setup › Emails. OneMRR checks this list before every customer email.