Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chapter 2 - How to keep score (H. Segment, Cohort, A/B Testing, and…
Chapter 2 - How to keep score
H. Segment, Cohort, A/B Testing, and Multivariate Analysis
Segmentation
Definition:
A segment is simply a group that shares some common characteristic.
How to define segments:
According to a range of technical and demographic information, then compare one segment to another.
Example:
Segment A: visitors using the Firefox browser.
Segment B: visitors using Safari browser.
Cohort
Definition: "Longitudinal studies"
Data is collected along the natural lifespan of a customer group such the user in first month and the user in the fifth month.
A/B Testing
Definition: cross-sectional studies
Compare one attribute of a subject's experience, such as, link color and assuming everything is equal
Multivariate Analysis
G. Moving Target P.21
Background:
in the early stage, you don't know how to define the success, it's like you're drawing a line in the sand-not carving it in stone.
What you do:
Adjusting your goal and how you define your key metrics is acceptable.
In the reality:
There is a gulf between what you assume and what users actually do. For example, you might think that people will pay your multiplayer game, only to discover that they're suing you as a photo upload service.
Case Study:
HighScore House Defines an "Active Users"
Know your customer:
A combination research with quantitive data (to monitor the behaviours) and qualitative data(to discover the reasons) is necessary to adjust your key metrics.
C. Vanity v.s. Real Metrics
Ask “What will I do differently based on this information?”
Total signups / Total active users / Percent of users who are active
Avoid these:
Number of hits
Number of page views.
Number of visits.
Number of unique visitors
Number of followers/friends/likes.
Time on site/number of pages.
Emails collected.
Number of downloads
D. Exploratory v.s. Reporting Metrics
Known unknowns：有目的的找出來
Unknown unknowns：探索
F. Correlated v.s. Causal Metrics
分辨真正的因果關係
B. Qualitative v.s. Quantitative Metrics
If quantitative data answers “what” and “how much,” qualitative data answers “why.”
You need to ask specific questions without leading potential customers or skewing their answers.
I. A Lean Analytics Cycle
E. Leading v.v Lagging Metrics
A leading metric is to predict the future.
A lagging metric 讓你知道現有的狀況
A. What Makes a Good Metrics?
Comparative
Understandable
Use ratio or rate
Easier to take action
Comparative
Good for comparing factors that are opposite but you should consider
Change the way you behave
Accounting metrics
Check whether the actual results are convergin on your business
Experiment Metrics
Optimize! Learn sth new? Need to try sth new?
