Pantone 50th Anniversary Products
September 17, 2013
55 10
inShare
5
After I wrote a post on six important cohort reports, I received a handful of questions about how to interpret cohort charts effectively. When I review cohort data from companies I work with, I look into three different trends to evaluate a product’s performance.
This is a cohort chart of hypothetical product indicating percentage of monthly active users each week for 12 weeks. There’s great data here and but the amount of data can be overwhelming.
But if we simplify the analysis into three charts we can isolate three key trends that will indicate our product’s performance. These are the On Boarding Trend, the Longitudinal Trend and the Cohort Trend.
The On Boarding Trend, the yellow bottom arrow, indicates the product’s effectiveness in its first month of use and its trend over time which is nothing less than a metric for user on boarding effectiveness. The last cell in each row indicates the monthly active rate for the cohort’s first month as users. In our hypothetical data set, that number grows from 4% in the first row to 41% in the last row, a 10x improvement. The product team has done a marvelous job of improving user on boarding and engaging users from the moment they sign up.
The Longitudinal Trend, the top red arrow, indicates how the activity rate changes as users continue to use the product. The first row is the oldest cohort of users, the ones who signed up most recently. The bottom row is the newest cohort.
In our hypothetical data, there are two important conclusions. First, our user base becomes more active over time and over the past 12 weeks, we don’t see any signs of the activity rate falling, indicating the product keeps the attention of its users. Second, the time required to achieve about 35% activity rate is decreasing rapidly. The first cohort needed 12 weeks to reach that activity rate while the last three cohorts need only one to three weeks. Both of these trends indicate the product satisfies and engages users increasingly well.
A decreasing engagement rate at 12 weeks would indicate problems retaining users and would compel the product team to investigate user behavior for those users.
The Cohort Trend, the left orange arrow, indicates the current contribution to activity of each of the cohorts. In our hypothetical example, each cohort contributes relatively equally to the engagement with the product. If the newer cohorts were contributing less, the data would be a leading indicator the product direction clashes with user needs.
Cohort analysis is quite useful because product teams can test different product features and measure the impact on a user base over time. Different on boarding flows will impact the on-boarding trend. New lifecycle features will alter the course of the Longitudinal Trend. Engagement tactics will change the cohort trends. Tying product changes to behavior informs great product design.
There are many more analyses that can be done using cohorts, but these are the three key analyses I use that are consistently useful to evaluate product performance.
Posted by:Tomasz Tunguz
via The 3 Key Cohort Analyses for Measuring Your Startup's Product Performance | LinkedIn.
