The first users of every startup’s product are its employees. Starting with a few lines of code, they bring forth the scaffolding that will eventually become the MVP and, with luck, a full blown production product. They’re also the ones who, for the first few years of a startup’s life, are amongst the most active users of the product. But it's easy to get misled. If you're not careful and look at the data, dogfooding can lead to the wrong conclusions.
Eating Your Own Dogfood
Many startups promote “eating their own dogfood” as part of their development process. “Dogfooding,” as it’s commonly known, is the strategy of encouraging employees to become active users of their own product. The benefits of this include more feedback, earlier in the development cycle, and feedback that's based on actual experience, rather than just contrived QA scenarios.
However, it’s easy to get misled by dogfooding if you don’t recognize the important fact that the population of internal users is not representative of your user population as a whole. In general, internal users (even non-technical ones) quickly become expert users. But that’s not the only thing. The manner in which internal users interact with a product differs in some subtle but important ways from how equivalently-skilled external users do. I'll demonstrate this with an example from DataHero.
Connections at DataHero
When we first released DataHero in 2013, users could import data from 6 supported cloud services or upload files from their computer. The interface was a sidebar with square icons for all of the available services. It acted as both a way to navigate data imported from connected services and a way to connect to new ones:
Over time, we added more connections and transitioned to a scrollable interface that included both the icon and name of the services users had connected to and a separate button for adding new services:
Eventually, we wanted to recapture some of the screen real estate used by the sidebar so that we could show larger previews of charts and dashboards. The list of active connections took up the majority of the sidebar, but could we afford to reduce it?
What Did the Team Think?
Since we were big proponents of dogfooding, we started by looking inwards. In early design sessions, the product team was nearly unanimous in the belief that all active connections should remain visible. Being able to quickly navigate between datasets from different services with a single click was something they did frequently and everyone felt it would be frustrating to have to take multiple actions to go between services. The engineers, designers and product managers each had a dozen or more services connected, so that was a lot of real estate.
But what about the rest of the team?
It turned out that sales, marketing, and customer support all felt the same way. Almost everyone at DataHero had 10 or more cloud services connected to their account, and everyone regularly switched between them as part of their work flow.
Had we ended our design process with our dogfooding, we would have kept the sidebar and all of the real estate used by a dozen or more logos and names. But we had data...
What Did the Data Say?
When we looked at the distribution of connected services across all users, we had more than 100 users with 10 or more connected services. For DataHero, the number of connected services was a leading indicator of whether a user would become a paying customer, so at first glance this would argue to keep the sidebar as is.
But when we removed all of the employee user accounts from our data, the story changed dramatically. It turned out that every single user with more than 10 connected services was an employee. In fact, there was only one external user who had more than 5 connected services. The average number of connected services was between 2 and 3.
Why the Disconnect?
When we went back and looked at how DataHero employees were using their accounts, it quickly became clear why everyone had so many connected services:
- Engineers, designers and product managers had almost all of the services connected for testing purposes
- The sales team had all of the services connected because they needed to be able to demo any service during a sales call
- The marketing team had all of the services connected so that they could write blogs and create other marketing content for whichever service they were promoting
- The support team needed all of the services connected so that they could debug customer issues related to any service
The Result
Instead of a sidebar with room for a dozen or more services (that, it turns out, was blank for all of our actual users!), we were able to move the connection icons to the top navigational bar and recapture a huge amount of screen real estate:
Data + Dogfooding = Better Design
Data and dogfooding go hand-in-hand. Dogfooding is an incredibly beneficial strategy for getting early feedback on your product, but it's essential that you use data to fact-check your conclusions and make sure your internal users are truly representative of your user base. Otherwise, you might end up designing for your employees instead of your users.
To help you put this into practice, here are 5 Steps to Use Data to Dogfood Better.