In Honeycomb, Boards are a place to save useful graphs you want to retain for later reuse and reference. While useful, team adoption was low. Recognizing boards as a key piece of team activation and onboarding, we set out to make them a little more sticky for our users.
Direction & Facilitation: Sarrah Vesselov UX Research: Jeremy Ford, Mei Luo, Tanya Romankova Product Design: Udie Chima, Jeremy Ford
Problem: Growth Stagnation
Honeycomb is a complex, data-heavy tool with a steep learning curve. It is fast and powerful, providing insights in seconds. However, in 2023, Honeycomb's growth and adoption stagnated. As part of a plan to address this stagnation, we investigated how improving key features could help teams get valuable insights faster.
We began by defining the key actions we felt teams needed to take to become activated (we defined activation as regular use and interaction with Honeycomb. Activation occurs when a team does the following:
- Runs a query
- Creates a Trigger
- Creates a Board
- Shares a board
- Duplicates a board
One feature that had the potential to increase activation was Boards. Boards are a place to save valuable queries for later use and reference. They may contain one query or a collection of queries. Boards serve as a way for teams to share insights from incidents and investigations and are good jumping-off points for new users joining an existing team. However, users were not utilizing boards. Data showed that only 12% of teams were using the feature.
Hypothesis: Introduce Templates
Teams need to get started quickly and derive insights from the product. Introducing Templates for boards would allow teams to get started and engage with boards more frequently, gain insights into their systems more quickly, and learn repeatable ways to derive value from Honeycomb.
Discovery: Talk to our users
We started the discovery process by talking with our users. Through surveys and user interviews, we identified two significant points of friction:
- Teams access all boards created by their organization on the boards' preview page. An individual Board could be set to display here in one of two ways: list or graph. The list view provides high-level details about the board; name, creator, and description. The graph view displayed these as well as the visualization associated with the query. The majority of those we spoke with found the list view unhelpful. The board preview page is a data-heavy view, making it difficult for users to find insights quickly. Our data supported this, showing that less than a quarter of boards used the list view.
- The board preview page is a data-heavy view, making it difficult for users to find insights quickly.
- New teams needed help figuring out where to start. They did not know what boards were for or how to use them.
This last finding resonated with the team. Honeycomb requires data instrumentation to query and provide insights. If you're new to Honeycomb or observability, you may not know what you should instrument or how. We pondered: Can we use board templates to help teams instrument their data and get better insights?
Design & Prototyping
With these insights, product design got to work, exploring different ways to improve the board experience. Key goals of this phase:
- Make it easier for teams to get started with boards by providing a starting point instead of a blank page.
- Improve the information architecture of the board list view - make it easy to parse at a glance.
- Find opportunities to "meet them where they are" by highlighting missing instrumentation.
Initially, our instincts were to remove the list view altogether and default to the graph view on the boards' preview page. Problems arose, however, when investigations revealed that the graph view posed significant performance issues. A single board can have many graphs, each loading and rendering on the board's page (the list of all boards within an organization). There was much back and forth between product, engineering, and design over this issue. Ultimately, we decided that removing the graph view and defaulting to the list view was the right path.
Over the next few weeks, we tested designs and prototypes with users until they felt they had something worth sharing more broadly. We released redesigned boards under a feature flag, and feedback was collected.
Templates
Alongside iterating on the flow and design of boards, we worked on discovery and design for templates. Understanding our users' most common needs was essential—what templates would help kick-start their exploration? What would resonate with their needs?
For this, we reviewed data on the most common queries and boards and surveyed members through our Slack community, "Pollinators." We then narrowed it down to three templates.
- Refinery Overview: An overview of Refinery's sampling operations. Sampling is essential to get right! We assembled a set of queries to give visibility into Refinery so users can adjust their sampling strategy where necessary.
- Service Health: Understand your services' overall health. Quickly identify where the slowest requests and errors occur, allowing users to investigate the root causes.
- Real User Monitoring (RUM): Overview of RUM information. Get a quick view of page performance and how their users interact with their front-end applications.
Outcome
The feedback we received was encouraging and helpful. It helped us confirm, refine, and finally release redesigned boards with the following improvements:
List-view only:
We reduced the amount of information displayed, focusing on key points to make scanning easier.
Search functionality:
We added a search function to the boards list page, allowing users to search by specific names or browse the terms used on the board.
Tabbed access to Templates
We made templates easy to access via tabbed navigation.
Templates with instrumentation detection:
To help teams get started and explore more use cases with Honeycomb, we built in detection for additional instrumentation. Each template contains a set of queries that rely on specific fields in the data. If a team does not have those fields available, we'll highlight what fields they need to instrument and how to add the instrumentation to their applications. A progress indicator gives them an easy visual of how far they have left to go.
Collecting Feedback
We incorporated a simple thumbs-up/down feedback mechanism to gather feedback in-app. Once activated, a small feedback window would pop up for additional details. This helped us to gather user sentiment and context and to quickly iterate and improve on each template.
Empty state illustrations and visual design enhancements:
We seized the opportunity to inform and delight, adding bright and informative empty states. Additionally, we added pattern treatments to the templates, randomizing how they would be applied. The bright and cheery colors brought life and a sense of delight to an otherwise dull arrangement of data.
These changes were well received. Our users loved templates and asked for more use cases in other product areas.
One surprising outcome was that templates helped users to understand how to query their data more quickly. Templates allow users to see their data in a useful context immediately, helping them make connections and understand what is possible.