What we noticed
From Nshir's synthetic data, last 28 days. Each insight names the goal step it affects. Last run 19 Nov 2026, 17:21 UTC.
Demo data: connect Mixpanel for real events · Text written by templates, not Claude
From Nshir's synthetic data, last 28 days. Each insight names the goal step it affects. Last run 19 Nov 2026, 17:21 UTC.
Demo data: connect Mixpanel for real events · Text written by templates, not Claude
In the last 28 days, 288 people reached "Open a paid feature". 66 of them went on to "Send a paid-access request", so 222 stopped here.
Why it matters: This is one of the two biggest losses on the way to "More trial users ask for paid access". Everyone who stops here is lost for the goal.
Possible reasons (not confirmed yet)
Measured numbers
Open a paid feature → Send a paid-access request: 66 of 288 (22.9%) continue, 77.1% drop
How sure we are. Counted from reliable events over one period.
S1 · funnel_drop · sample 288 · 2026-09-29 to 2026-10-27
-- funnel_timelines.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"site_visited",["site_visited","test_started","test_finished","session_started","locked_feature_viewed","paid_access_requested"],"2026-10-27T00:00:00.000Z"]
-- Per-user timelines for the funnel. Cohort: users whose first entry event falls in [$2, $3).
-- $1 product_id, $2 cohort start, $3 cohort end, $4 excluded distinct_ids,
-- $5 entry event, $6 funnel event names, $7 horizon end (events after it are ignored)
with cohort as (
select distinct_id,
min(time) as first_time,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
)
select c.distinct_id, c.platform, c.browser, e.name, e.time
from cohort c
join events e on e.product_id = $1 and e.distinct_id = c.distinct_id
where e.name = any($6::text[]) and e.time >= c.first_time and e.time < $7::timestamptz
order by c.distinct_id, e.time
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
1,096 of the 1,922 people who started did not finish (57%). 61% of them left before question 3. People on phones quit this early more often than people on desktop (38% vs 25%).
Why it matters: People who quit here never reach the steps that lead to "More trial users ask for paid access", so fixing the start of "Finish the first test" lifts every step after it.
Possible reasons (not confirmed yet)
Measured numbers
Start the first test → Finish the first test: 826 of 1922 (43%) continue, 57% drop · 1096 of 1922 starters (57%) left without finishing; 667 (34.7% of starters) left before question 3. Mobile 37.9%, desktop 24.8%.
How sure we are. Counted from reliable events with plenty of people, over one period.
S2 · funnel_drop · sample 1922 · 2026-09-29 to 2026-10-27
-- funnel_timelines.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"site_visited",["site_visited","test_started","test_finished","session_started","locked_feature_viewed","paid_access_requested"],"2026-10-27T00:00:00.000Z"]
-- Per-user timelines for the funnel. Cohort: users whose first entry event falls in [$2, $3).
-- $1 product_id, $2 cohort start, $3 cohort end, $4 excluded distinct_ids,
-- $5 entry event, $6 funnel event names, $7 horizon end (events after it are ignored)
with cohort as (
select distinct_id,
min(time) as first_time,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
)
select c.distinct_id, c.platform, c.browser, e.name, e.time
from cohort c
join events e on e.product_id = $1 and e.distinct_id = c.distinct_id
where e.name = any($6::text[]) and e.time >= c.first_time and e.time < $7::timestamptz
order by c.distinct_id, e.time
S3 · progress_dropoff · sample 1922 · 2026-09-29 to 2026-10-27
-- progress_dropoff.sql -- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"test_started","answer_selected","question","test_finished"] -- Where inside a step users stop. Starters of $5 in [$2, $3); last value of the progress -- property $7 on event $6 after starting; whether they reached $8. -- $1 product_id, $4 excluded distinct_ids with starters as ( select distinct_id, min(time) as t0, (array_agg(props->>'platform' order by time))[1] as platform from events where product_id = $1 and name = $5 and distinct_id <> all($4::text[]) group by distinct_id having min(time) >= $2::timestamptz and min(time) < $3::timestamptz ), progress as ( select s.distinct_id, coalesce(max((e.props->>$7)::int), 0) as last_step from starters s left join events e on e.product_id = $1 and e.distinct_id = s.distinct_id and e.name = $6 and e.time >= s.t0 group by s.distinct_id ), finished as ( select distinct s.distinct_id from starters s join events e on e.product_id = $1 and e.distinct_id = s.distinct_id and e.name = $8 and e.time >= s.t0 ) select s.platform, p.last_step, (f.distinct_id is not null) as finished, count(*)::int as users from starters s join progress p using (distinct_id) left join finished f using (distinct_id) group by 1, 2, 3 order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
Before 22 Oct this happened in 0.4% of attempts. Since then it has happened 35 times out of 172 attempts, to 35 people.
Why it matters: People who hit this error lose their progress on "Finish the first test", and most of them do not try again.
Possible reasons (not confirmed yet)
Measured numbers
exam_submit_failed 0.4% → 20.3% of attempts since 2026-10-22 (35 of 172) vs the 28-day baseline
How sure we are. A clear jump compared with the 28 days before; the cause is not confirmed yet.
S4 · error_spike · sample 172 · 2026-10-22 to 2026-10-27
-- error_rate_daily.sql
-- params: ["nshir","2026-09-01T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"exam_submit_failed","exam_submitted"]
-- Daily failure rate of an action: failures $5 vs successes $6, from $2 to $3.
-- $1 product_id, $4 excluded distinct_ids
select date_trunc('day', time)::date::text as day,
count(*) filter (where name = $5)::int as failures,
count(*) filter (where name = $6)::int as successes,
count(distinct distinct_id) filter (where name = $5)::int as failed_users
from events
where product_id = $1 and name in ($5, $6) and time >= $2::timestamptz and time < $3::timestamptz
and distinct_id <> all($4::text[])
group by 1
order by 1
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
33% of Firefox users on desktop who reached "Visit the site" went on to "Start the first test" (56 of 171), compared with 46% of everyone else.
Why it matters: Firefox users on desktop are 4% of people at this step. At everyone else's rate, about 22 more of them would reach "Start the first test".
Possible reasons (not confirmed yet)
Measured numbers
Visit the site → Start the first test: platform=desktop, browser=firefox 32.7% (56 of 171) vs 45.8% for everyone else
How sure we are. Enough people in this group; checked over one period.
S5 · segment_gap · sample 171 · 2026-09-29 to 2026-10-27
-- segment_step_gap.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"site_visited","test_started",0]
-- Conversion from $5 to $6 per platform and browser. Users whose first $5 is in [$2, $3);
-- converted if $6 happens after it, at least $7 calendar days later, before $3.
-- $1 product_id, $4 excluded distinct_ids
with f as (
select distinct_id, min(time) as t,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
),
c as (
select f.distinct_id
from f
join events e on e.product_id = $1 and e.distinct_id = f.distinct_id and e.name = $6
and e.time >= f.t and e.time < $3::timestamptz
and date_trunc('day', e.time) >= date_trunc('day', f.t) + make_interval(days => $7::int)
group by f.distinct_id
)
select coalesce(f.platform, 'unknown') as platform, coalesce(f.browser, 'unknown') as browser,
count(*)::int as n, count(c.distinct_id)::int as converted
from f left join c using (distinct_id)
group by 1, 2
order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
24% of iPhone users on Safari who reached "Start the first test" went on to "Finish the first test" (144 of 614), compared with 52% of everyone else.
Why it matters: IPhone users on Safari are 32% of people at this step. At everyone else's rate, about 176 more of them would reach "Finish the first test".
Possible reasons (not confirmed yet)
Measured numbers
Start the first test → Finish the first test: platform=ios, browser=safari 23.5% (144 of 614) vs 52.1% for everyone else
How sure we are. Enough people in this group; checked over one period.
S6 · segment_gap · sample 614 · 2026-09-29 to 2026-10-27
-- segment_step_gap.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"test_started","test_finished",0]
-- Conversion from $5 to $6 per platform and browser. Users whose first $5 is in [$2, $3);
-- converted if $6 happens after it, at least $7 calendar days later, before $3.
-- $1 product_id, $4 excluded distinct_ids
with f as (
select distinct_id, min(time) as t,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
),
c as (
select f.distinct_id
from f
join events e on e.product_id = $1 and e.distinct_id = f.distinct_id and e.name = $6
and e.time >= f.t and e.time < $3::timestamptz
and date_trunc('day', e.time) >= date_trunc('day', f.t) + make_interval(days => $7::int)
group by f.distinct_id
)
select coalesce(f.platform, 'unknown') as platform, coalesce(f.browser, 'unknown') as browser,
count(*)::int as n, count(c.distinct_id)::int as converted
from f left join c using (distinct_id)
group by 1, 2
order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
63% of Safari users on desktop who reached "Start the first test" went on to "Finish the first test" (78 of 124), compared with 42% of everyone else.
Why it matters: If we learn what helps them, it may work for everyone.
Possible reasons (not confirmed yet)
Measured numbers
Start the first test → Finish the first test: platform=desktop, browser=safari 62.9% (78 of 124) vs 41.6% for everyone else
How sure we are. Enough people in this group; checked over one period.
S7 · segment_gap · sample 124 · 2026-09-29 to 2026-10-27
-- segment_step_gap.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"test_started","test_finished",0]
-- Conversion from $5 to $6 per platform and browser. Users whose first $5 is in [$2, $3);
-- converted if $6 happens after it, at least $7 calendar days later, before $3.
-- $1 product_id, $4 excluded distinct_ids
with f as (
select distinct_id, min(time) as t,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
),
c as (
select f.distinct_id
from f
join events e on e.product_id = $1 and e.distinct_id = f.distinct_id and e.name = $6
and e.time >= f.t and e.time < $3::timestamptz
and date_trunc('day', e.time) >= date_trunc('day', f.t) + make_interval(days => $7::int)
group by f.distinct_id
)
select coalesce(f.platform, 'unknown') as platform, coalesce(f.browser, 'unknown') as browser,
count(*)::int as n, count(c.distinct_id)::int as converted
from f left join c using (distinct_id)
group by 1, 2
order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
57% of Firefox users on desktop who reached "Finish the first test" went on to "Come back on day 2" (20 of 35), compared with 44% of everyone else. This is based on 35 people, fewer than the 100 we need to be sure.
Why it matters: If we learn what helps them, it may work for everyone. Treat this as a lead to check, not a fact.
Possible reasons (not confirmed yet)
Measured numbers
Finish the first test → Come back on day 2: platform=desktop, browser=firefox 57.1% (20 of 35) vs 44.2% for everyone else
How sure we are. Fewer than 100 people in this group. Below the minimum sample of 100 users, so confidence stays Low.
S8 · segment_gap · sample 35 · 2026-09-29 to 2026-10-27
-- segment_step_gap.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"test_finished","session_started",1]
-- Conversion from $5 to $6 per platform and browser. Users whose first $5 is in [$2, $3);
-- converted if $6 happens after it, at least $7 calendar days later, before $3.
-- $1 product_id, $4 excluded distinct_ids
with f as (
select distinct_id, min(time) as t,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
),
c as (
select f.distinct_id
from f
join events e on e.product_id = $1 and e.distinct_id = f.distinct_id and e.name = $6
and e.time >= f.t and e.time < $3::timestamptz
and date_trunc('day', e.time) >= date_trunc('day', f.t) + make_interval(days => $7::int)
group by f.distinct_id
)
select coalesce(f.platform, 'unknown') as platform, coalesce(f.browser, 'unknown') as browser,
count(*)::int as n, count(c.distinct_id)::int as converted
from f left join c using (distinct_id)
group by 1, 2
order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
13% of iPhone users who reached "Open a paid feature" went on to "Send a paid-access request" (14 of 109), compared with 27% of everyone else.
Why it matters: IPhone users are 36% of people at this step. At everyone else's rate, about 16 more of them would reach "Send a paid-access request".
Possible reasons (not confirmed yet)
Measured numbers
Open a paid feature → Send a paid-access request: platform=ios 12.8% (14 of 109) vs 27.3% for everyone else
How sure we are. Enough people in this group; checked over one period.
S9 · segment_gap · sample 109 · 2026-09-29 to 2026-10-27
-- segment_step_gap.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"locked_feature_viewed","paid_access_requested",0]
-- Conversion from $5 to $6 per platform and browser. Users whose first $5 is in [$2, $3);
-- converted if $6 happens after it, at least $7 calendar days later, before $3.
-- $1 product_id, $4 excluded distinct_ids
with f as (
select distinct_id, min(time) as t,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
),
c as (
select f.distinct_id
from f
join events e on e.product_id = $1 and e.distinct_id = f.distinct_id and e.name = $6
and e.time >= f.t and e.time < $3::timestamptz
and date_trunc('day', e.time) >= date_trunc('day', f.t) + make_interval(days => $7::int)
group by f.distinct_id
)
select coalesce(f.platform, 'unknown') as platform, coalesce(f.browser, 'unknown') as browser,
count(*)::int as n, count(c.distinct_id)::int as converted
from f left join c using (distinct_id)
group by 1, 2
order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
38% of Android users who reached "Open a paid feature" went on to "Send a paid-access request" (36 of 94), compared with 15% of everyone else. This is based on 94 people, fewer than the 100 we need to be sure.
Why it matters: If we learn what helps them, it may work for everyone. Treat this as a lead to check, not a fact.
Possible reasons (not confirmed yet)
Measured numbers
Open a paid feature → Send a paid-access request: platform=android 38.3% (36 of 94) vs 14.8% for everyone else
How sure we are. Fewer than 100 people in this group. Below the minimum sample of 100 users, so confidence stays Low.
S10 · segment_gap · sample 94 · 2026-09-29 to 2026-10-27
-- segment_step_gap.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"locked_feature_viewed","paid_access_requested",0]
-- Conversion from $5 to $6 per platform and browser. Users whose first $5 is in [$2, $3);
-- converted if $6 happens after it, at least $7 calendar days later, before $3.
-- $1 product_id, $4 excluded distinct_ids
with f as (
select distinct_id, min(time) as t,
(array_agg(props->>'platform' order by time))[1] as platform,
(array_agg(props->>'browser' order by time))[1] as browser
from events
where product_id = $1 and name = $5 and distinct_id <> all($4::text[])
group by distinct_id
having min(time) >= $2::timestamptz and min(time) < $3::timestamptz
),
c as (
select f.distinct_id
from f
join events e on e.product_id = $1 and e.distinct_id = f.distinct_id and e.name = $6
and e.time >= f.t and e.time < $3::timestamptz
and date_trunc('day', e.time) >= date_trunc('day', f.t) + make_interval(days => $7::int)
group by f.distinct_id
)
select coalesce(f.platform, 'unknown') as platform, coalesce(f.browser, 'unknown') as browser,
count(*)::int as n, count(c.distinct_id)::int as converted
from f left join c using (distinct_id)
group by 1, 2
order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
In the last 7 days, 40 of 422 people had this recorded twice, less than a second apart.
Why it matters: Every number built on it for "Finish the first test" is off until this is fixed, so other decisions on this step rest on wrong data.
Possible reasons (not confirmed yet)
Measured numbers
answer_selected fired twice within 1 s for 40 of 422 users (9.5%) in 7 days; 244 duplicate events
How sure we are. Counted directly from the raw events.
S11 · double_fire · sample 422 · 2026-10-20 to 2026-10-27
-- double_fire.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"]]
-- Same user, same event within 1 second, in the 7 days before $3.
-- $1 product_id, $4 excluded distinct_ids ($2 unused, kept for a uniform signature)
with e as (
select distinct_id, name, time,
lag(time) over (partition by distinct_id, name order by time) as prev
from events
where product_id = $1 and time >= $3::timestamptz - interval '7 days' and time < $3::timestamptz
and distinct_id <> all($4::text[]) and $2::timestamptz is not null
)
select name,
count(*)::int as events,
count(*) filter (where prev is not null and time - prev < interval '1 second')::int as dup_events,
count(distinct distinct_id)::int as users,
count(distinct distinct_id) filter (where prev is not null and time - prev < interval '1 second')::int as dup_users
from e
group by name
order by name
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
This should be recorded every time it happens, but it has not been recorded once in the last 28 days.
Why it matters: Without it we cannot see what happens after "More trial users ask for paid access", for example how many requests turn into paying users.
Possible reasons (not confirmed yet)
Measured numbers
admin_unlocked is a journey event in the dictionary but fired 0 times in the window
How sure we are. Zero records in the whole period.
S12 · untracked_event · sample 0 · 2026-09-29 to 2026-10-27
-- event_volume.sql
-- params: ["nshir","2026-09-29T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"]]
-- Events per name per day in [$2, $3), plus how many lack each required property.
-- $1 product_id, $4 excluded distinct_ids
select name, date_trunc('day', time)::date::text as day, count(*)::int as events,
count(distinct distinct_id)::int as users
from events
where product_id = $1 and time >= $2::timestamptz and time < $3::timestamptz and distinct_id <> all($4::text[])
group by 1, 2
order by 1, 2
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template
5 of 59 records in the last 7 days are missing the feature.
Why it matters: For those cases we can't tell which feature people were interested in.
Possible reasons (not confirmed yet)
Measured numbers
locked_feature_viewed is missing feature on 5 of 59 events (8.5%) in 7 days
How sure we are. Counted directly from the raw events.
S13 · missing_property · sample 59 · 2026-10-20 to 2026-10-27
-- missing_property.sql
-- params: ["nshir","2026-10-20T00:00:00.000Z","2026-10-27T00:00:00.000Z",["internal_qa_1","internal_qa_2"],"locked_feature_viewed","feature"]
-- Share of $5 events in [$2, $3) without property $6.
-- $1 product_id, $4 excluded distinct_ids
select count(*)::int as events,
count(*) filter (where not (props ? $6) or props->>$6 is null or props->>$6 = '')::int as missing
from events
where product_id = $1 and name = $5 and time >= $2::timestamptz and time < $3::timestamptz
and distinct_id <> all($4::text[])
Run run_9beac44cdca34f40 · context v1 · prompt S3/v2 · model offline-template