How Notion is supercharging its product feedback loop using Enterpret

July 11, 2023

The product operations and user insights team at Notion gathers user insights to represent the voice of the customer, working with product development to prioritize users’ most important problems.

Why Enterpret?

As the user base and company grew, Notion’s manual tagging process (>700 unique tags, tens of thousands of monthly support tickets) was reaching its limits, with two major problems:

Lack of trust in the underlying data
Time-consuming to get insights, with no way to visualize and segment feedback data

Enterpret Benefits

Enterpret has unlocked the following capabilities for the product ops team:

Unified feedback platform and taxonomy across support tickets, Slack, Twitter, and app store reviews
Machine learning identifies feedback reasons and surfaces unknowns, serving as a “second brain” for the team
Self-serve insights via queries and user segmentation in Enterpret

Enterpret Impact

Cut the time spent on the monthly user insights report from 2 weeks to 3 days
Informed a decision to dedicate an engineering team to fix login problems

Over the past few years, Notion has become an essential part of many teams’ workflows. But as Notion’s customer base grew, it became increasingly difficult to stay on top of user feedback coming in through multiple channels.

We recently sat down with a few members of Notion’s Customer Experience and Product Ops teams to learn more about how they’re tackling these issues so they can learn from their customer feedback at scale.

We talked to the team at the forefront of elevating the voice of the customer at Notion: Emma Auscher - Global Head of Customer Experience; Misty Smith - Head of Product Operations; and Maya Bakir - Product Operations Specialist.

Challenge: The limitations of manual tagging and insights

In late 2020, the team recognized that their previous efforts to learn from user feedback were breaking down.

Lack of trust in manual tagging

Notion gets tens of thousands of support tickets coming in monthly, in addition to ever increasing feedback from Twitter, surveys, app store reviews, and community forums. Every support conversation was getting tagged by a support agent, but they needed to choose from a set of over 700 tags.

Accuracy was a big concern: with a distributed team and so many tags to choose from, how do you ensure consistency in how they are applied? The product development team was losing trust in the data’s reliability and usefulness.

Time-consuming manual insights

In addition to waning confidence in the underlying data, it was difficult to get insights out of it. They were dumping feedback records into a database, but there was no easy way to visualize feedback data, identify trends, or slice and dice by different user segments.

Solution: Automating user insights with Enterpret

Notion’s product ops team decided to bring on Enterpret to help them derive more learnings from their customer feedback data. Some of the main benefits they found include:

Having a unified feedback platform
Leveraging machine learning to surface unknowns
Self-service insights

Unified feedback platform

With Enterpret’s integrations to different feedback sources, Notion was able to setup feedback from Intercom (support tickets), app store reviews, and Twitter with just a few clicks.

As Emma Auscher, Notion’s Global Head of Customer Experience (CX) puts it: “Enterpret helps us have a holistic view from our social media coverage, to our support tickets, to every single interaction that we're plugging into it. Beyond just keywords, we can actually understand: what are the broader sentiments? What are our users saying?”

Leveraging machine learning to identify feedback reasons and surface unknowns

After connecting Notion’s feedback sources, Enterpret used natural language processing (NLP) to generate a custom feedback taxonomy for Notion.

According to Misty Smith, Head of Product Operations: “The advantage of Enterpret is that we’re not relying entirely on human categorization. Enterpret is like a second brain that is looking out for themes and trends that I might not be thinking about.”

The real magic happens with ‘reasons’, where Enterpret’s model identifies why the customer gave the feedback; things like: “want to upgrade to a team plan”, “unable to add members to account”, or “keyboard shortcuts don’t work”.

“Reasons get more granular than our internal tags. Our tags capture features or product areas, but not the actual feedback itself,” says Maya Bakir, the product ops specialist responsible for user insights.

Self-service insights: querying and segmenting customer feedback

“Enterpret has made it so much easier to navigate through our feedback,” says Maya. “It has been monumental in terms of being able to visualize trends and filter on our data so easily. That’s not something we were capable of doing before. If I’m curious about a specific trend, I can quickly find it in Enterpret. I can see how it’s changed over time, segment by users on different plan types, and see all the raw feedback records.”

In addition to regular reporting on top trends, “we’re starting to use Enterpret more for ad hoc, one-off questions,” says Misty. “PMs are asking me: Hey, what are the top requests related to search? When do users report issues with real time collaboration? Enterpret makes it easy to quickly search for feedback data to validate a hypothesis or inform product planning."

Supercharging the product feedback loop

Over the past year and a half, Enterpret has enabled the product operations team to work more efficiently and effectively in their partnership with product development.

Cut the time spent on the monthly user insights report from 2 weeks to 3 days

Every month, Maya puts together a report on feedback trends that gets discussed in a voice of the customer (VoC) meeting. Enterpret has enabled a drastic reduction in the time she spends on the report.

“In the past it would take me two weeks. With Enterpret I can get it done in 3 days,” says Maya.

That report, along with inputs from sales, customer success, and product marketing, helps Notion answer questions like: “Do we still agree with where the product roadmap is at? Are we still working on the right things, or is there anything we need to flag from our user base that we need to fix right away or think differently about?”

Using feedback insights to advocate for customer problems

Feedback reasons identified in Enterpret have helped, "drive more productive discussions with product and engineering." Misty shares, “Reasons help make the problem more concrete in people’s brains.”

‘Login problems’ was consistently a top category of feedback that wasn’t getting addressed. The product ops team was able to say, “Look, this month we noticed there’s a specific type of login problem that is surfacing.”

Because the reason Enterpret provided was so specific, it sparked a really good discussion with the product team. Says Misty, “I can draw a dotted line from those discussions to the fact that we now have a whole new engineering team dedicated to fixing login problems.”

Continuing the partnership between Notion and Enterpret

Notion has been a valued partner of Enterpret for over 3 years.

“What makes the difference is the Enterpret team and how responsive they are: proactively working on issues we have, looking at solutions, and always coming to us with proposals on how we can move forward. That made the decision to sign another year’s contract that much easier. It’s a partnership.” - Emma Auscher

What is the team looking forward to over the next year? “I'm hoping that we can empower more of our product managers to go into Enterpret themselves to understand our feedback data,” says Misty, Head of Product Operations.

And for Maya, she plans to continue to partner closely with the team as they iterate on features that make the platform both more powerful and easier to use. “The Enterpret team is always really quick to take action when I have a question or feedback. They are the most responsive, helpful team I've ever worked with. They have helped unblock me in so many ways that I didn't think would be possible.” - Maya Bakir, Product Operations

Want more? Learn how Notion is running their Quarterly Voice of Customer Report and download the Notion Template to get started!

‍

We chose Enterpret for its intuitive UI and seamless data integrations, which enable us to combine data from all sources and generate deep, multi-layered insights. The Enterpret team ensured a smooth onboarding experience, even increasing data ingestion cadence to provide real-time alerts. The insights we've gained are already driving meaningful change across our organization, and their responsive support and collaborative approach have been invaluable.

Nir Ben Ari

Director, Customer Support, Vimeo

Wisdom saves me hours every week. With 'Summarize with Wisdom,' I can condense feedback with a single click, replacing the tedious process of reading through hundreds of tickets. It’s life-changing!

Jil McKinney

Director of Customer Support, Descript

Before Enterpret, organizing research data took an entire day. Now, research synthesis is 83% faster - it takes just 15 minutes to pull the data and another 15 minutes to start synthesizing. Enterpret removes the manual work, allowing me to focus on strategic thinking with a clear mind.

Mike McNasby

User Research Lead, Descript

We are laser-focused on giving customers more than they expect through a hospitality-first, individualized approach to drive retention and loyalty. Enterpret has allowed us to stitch together a full picture of the customer, including feedback and reviews from multiple data points. We now can super-serve our loyal customers in a way that we have never been able to before.

Anna Esrov

Vice President of Customer Experience & Loyalty

Enterpret allowed us to listen to specific issues and come closer to our Members - prioritizing feedback which needed immediate attention, when it came to monitoring reception of new releases: Enterpret picked up insights for new updates and became the eyes of whether new systems and functionality were working well or not.

Louise Sellars

Analyst, Customer Insights

Enterpret is one of the most powerful tools in our toolkit. It's very Member-friendly. We've been able to share how other teams can modify and self-serve in Enterpret. It's bridged a gap to getting access to Member feedback, and I see all our teams finding ways to use Enterpret to answer Member-related questions.

Dina Mohammad-Laity

VP of Data

The big win-win is our VoC program enabled us to leverage our engineering resources to ship significantly awesome and valuable features while minimizing bug fixes and" keep the lights on" work. Magnifying and focusing on the 20% that causes the impact is like finding the needle in a haystack, especially when you have issues coming from all over the place

Abishek Viswanathan

CPO, Apollo.io

Since launching our Voice of Customer program six months ago, our team has dropped our human inquiry rate by over 40%, improved customer satisfaction, and enabled our team to allocate resources to building features that increase LTV and revenue.

Abishek Viswanathan

CPO, Apollo.io

Enterpret's Gong Integration is a game changer on so many levels. The automated labeling of feedback saves dozens of hours per week. This is essential in creating a customer feedback database for analytics.

Michael Bartimer

Revenue Operations Lead

Enterpret has made it so much easier to understand our customer feedback. Every month I put together a Voice of Customer report on feedback trends. Before Enterpret it would take me two weeks - with Enterpret I can get it done in 3 days.

Maya Bakir

Product Operations, Notion

The Enterpret platform is like the hero team of data analysts you always wanted - the ability to consolidate customer feedback from diverse touch points and identify both ongoing and emerging trends to ensure we focus on and build the right things has been amazing. We love the tools and support to help us train the results to our unique business and users and the Enterpret team is outstanding in every way.

Larisa Sheckler

COO, Samsung Food

Enterpret makes it easy to understand and prioritize the most important feedback themes. Having data organized in one place, make it easy to dig into the associated feedback to deeply understand the voice of customer so we can delight users, solve issues, and deliver on the most important requests.

Lauren Cunningham

Head of Support and Ops

With Enterpret powering Voice of Customer we're democratizing feedback and making it accessible for everyone across product, customer success, marketing, and leadership to provide evidence and add credibility to their strategies and roadmaps.

Michael Nguyen

Head of Research Ops and Insights, Figma

Boll & Branch takes pride in being a data driven company and Enterpret is helping us unlock an entirely new source of data. Enterpret quantifies our qualitative data while still keeping customer voice just a click away, adding valuable context and helping us get a more complete view of our customers.

Matheson Kuo

Senior Product Analyst, Boll & Branch

Enterpret has transformed our ability to use feedback to prioritize customers and drive product innovation. By using Enterpret to centralize our data, it saves us time, eliminates manual tagging, and boosts accuracy. We now gain near real-time insights, measure product success, and easily merge feedback categories. Enterpret's generative AI technology has streamlined our processes, improved decision-making, and elevated customer satisfaction

Nathan Yoon

Business Operations, Apollo.io

Enterpret helps us have a holistic view from our social media coverage, to our support tickets, to every single interaction that we're plugging into it. Beyond just keywords, we can actually understand: what are the broader sentiments? What are our users saying?

Emma Auscher

Global VP of Customer Experience, Notion

The advantage of Enterpret is that we’re not relying entirely on human categorization. Enterpret is like a second brain that is looking out for themes and trends that I might not be thinking about.

Misty Smith

Head of Product Operations, Notion

As a PM, I want to prioritize work that benefits as many of our customers as possible. It can be too easy to prioritize based on the loudest customer or the flavor of the moment. Because Enterpret is able to compress information across all of our qualitative feedback sources, I can make decisions that are more likely to result in positive outcomes for the customer and our business.

Duncan Stewart

Product Manager

We use Enterpret for our VoC & Root Cause Elimination Program. It's helping us solve the issues of aggregating disparate sources of feedback (often tens of thousands per month) and distilling it into specific reasons, with trends, so we can see if our product fixes are delivering impact.