How AI helps us ask for reviews at the right time

In October 2025, a Product team asked Happiness, Automattic’s customer support team, for help with a project. Our part: Happiness Engineers would invite WooPayments users who’d just had a great support experience to share it in a review on WordPress.org (this would be a separate, rarer ask alongside our automated customer satisfaction survey). Because of negativity bias, we’re all more moved to write about a bad experience than a good one, so reviews on high-stakes products (like a payment gateway) can skew toward the bad days and tell other merchants little about the day-to-day reliability of the product.

For Happiness Engineers, side-quests like this are part of being closer to the customer. However, our core responsibility of delivering the best possible experience in a timely fashion has a strong gravitational pull of its own. Asking depended on someone remembering at the right moment, so adoption grew slowly. For that reason, we asked our buddy Odie, the AI assistant that already helps answer our customers’ chats and emails, to help us spot those moments and, more recently, to send some of the requests itself.

Teaching a bot to spot the right moment

We called the bot Positive Experience Classifier, after the question we started with: “was this a positive experience?” That turned out to be too simple a framing. Most support conversations end on a friendly note, so Odie flagged them left and right, but Happiness Engineers often disagreed. Odie was picking up on how the merchant sounded and on the support we delivered; Happiness Engineers were looking at the merchant’s whole situation to judge whether the moment was right. A positive experience isn’t the same as a positive outcome. A merchant whose checkout broke can be genuinely thankful for our help, but asking them to rate the product while they’re still counting lost sales would be tone-deaf. They can still leave a review if they want to; we just won’t add an extra ask after an issue that might have left them with a sour taste.

Building structure around the decision

Reading the room is second nature to Happiness Engineers. On its own, an AI isn’t reliable enough to do it consistently, so we changed the question to “would a review request be welcome right now?” and broke one big judgment call into smaller ones:

The first checks are plain rules, with no AI involved. Then, before judging anything, the bot writes down the facts. That was the step that surprised us most: once they’re on paper, a friendly “thanks!” at the end can’t outweigh a negative outcome established earlier in the thread. The gates are sensible requirements:

• “the refund will be issued” is a promise, not an outcome;
• if the merchant no longer uses the product, that’s no place for a review ask;
• if our last reply says “let me know if that fixes it”,” we can’t ask for a review before they’ve answered, or we look like the left hand doesn’t know what the right is up to.

From suggesting to sending

Until early October 2026, the bot only suggested, and a Happiness Engineer decided whether to send. That helped us ask far more often, from a few requests a week to a few dozen. We’ve yet to hear from a merchant who minded, and over the past year, alongside a lot of product, design, and engineering work, the share of positive reviews has grown considerably.

In the weeks leading up to this post, we laid the groundwork to allow Odie to send some of the requests itself. We reviewed two months of its most confident flags, focusing on the ones Happiness Engineers skipped. Fewer than one in 20 of all its confident flags were real misses, and we adjusted the prompt to catch those cases. As you’re reading this post, the green path in the chart is active: WooPayments tickets that a Happiness Engineer worked on and that Odie is confident about will trigger a mostly templated automated email from Happy Bot, a clearly labeled bot account, name-checking the Happiness Engineer. Everything else still goes through the tried and true manual path. If it holds up, the same approach will expand to other products next.

AI doesn’t replace the judgment and nuance that Happiness Engineers bring to reading the room, but with enough scaffolding around it, Odie can perform a side-quest while the Happiness Engineer stays focused on the customer in front of them. If you’re curious what makes those positive moments happen, we read every positive comment from two months.