You're in the demo. Made-up data for a sample product, with problems planted on purpose. Click anything: approve improvements, create issues, check results. Nothing reaches GitHub or Mixpanel.Back to Նշիր
04 · Results

Did it work?

14 days after each release we compare its goal step and the goal itself with the baseline. A change that didn't help is still a result: it tells us what not to try again.

Demo data: connect Mixpanel for real events · Text written by templates, not Claude

Goal since 18 Sept1.8% → 3.7%
Shipped4
Improved1
No change · unclear2 · 0
Measuring1
  1. Larger answer buttons on mobile

    Released 23 Oct · result on 13 Nov · day 14 of 14
    ● Measuring
    MeasureBeforeAfterChangePeople
    Finish the first test (goal step)43.6%41%too early1,702 → 251
    Trial users ask for paid access (goal)3.7%2%too early1,702 → 251
    What we learnedToo early to say. Interim numbers are shown for the days measured so far.
    NextCheck again on 13 Nov 2026.
  2. Show the paywall link on results

    Released 2 Oct · measured 19 Nov over 14 days
    ✓ Improved
    MeasureBeforeAfterChangePeople
    Send a paid-access request (goal step)10.2%23.3%+13.1 pts313 → 163
    Trial users ask for paid access (goal)2%3.7%+1.7 pts1,835 → 937

    90% interval 7.1 to 19.4 pts · P(improved) 100%

    What we learnedImproved: more people now send a paid-access request (10% before, 23% after). "Show the paywall link on results" worked, and we are confident the change caused it.
    NextKeep it.
  3. Hide the timer in practice mode

    Released 25 Sept · measured 19 Nov over 14 days
    ○ No change
    MeasureBeforeAfterChangePeople
    Finish the first test (goal step)43%43.8%+0.8 pts1,811 → 962
    Trial users ask for paid access (goal)2%3.4%+1.4 pts1,811 → 962
    Come back on day 2 (next step, guardrail)47.8%49.4%+1.6 pts

    90% interval -2.5 to 4.0 pts · P(improved) 66% · OVERLAPPING RELEASE: goal credit is split

    What we learnedNo change: about the same share of people finish the first test (43% before, 44% after). "Hide the timer in practice mode" was not the reason people stop at "Finish the first test". We won't suggest it again. Another change went live around the same time, so part of this may come from it.
    NextKeep it or undo it; either is fine.
  4. Shorter signup form

    Released 18 Sept · measured 19 Nov over 14 days
    ○ No change
    MeasureBeforeAfterChangePeople
    Start the first test (goal step)43.9%42.5%−1.5 pts4,205 → 2,167
    Trial users ask for paid access (goal)1.8%2%+0.2 pts1,847 → 920
    Finish the first test (next step, guardrail)43.1%42.6%−0.5 pts

    90% interval -3.6 to 0.7 pts · P(improved) 13% · OVERLAPPING RELEASE: goal credit is split

    What we learnedNo change: about the same share of people start the first test (44% before, 43% after). "Shorter signup form" was not the reason people stop at "Start the first test". We won't suggest it again. Another change went live around the same time, so part of this may come from it.
    NextKeep it or undo it; either is fine.