You're in the demo. Made-up data for a sample product, with problems planted on purpose. Click anything: approve improvements, create issues, check results. Nothing reaches GitHub or Mixpanel.Back to Նշիր
Engine settings

How the engine is learning

Calibration adjusts the numbers, lessons adjust the reasoning, prompt versions adjust the instructions. Each change is logged and can be rolled back.

Model: offline templates
SHIPPED THAT IMPROVED33%
BRIER SCORE0.44
MEDIAN LIFT ERROR3.5 pts
EFFORT ERROR1.0 d
APPROVAL RATE—
INSIGHT → SHIPPED—

3 measured results. Multipliers stay at 1.0 until 5 results, shrink toward 1.0, stay within 0.5–1.5 and move at most 20% per update.

Calibration

SCOPEMULTIPLIERVALUERESULTSUPDATED
category: uxkI0.870319 Nov 2026, 17:21 UTC
category: uxkC0.926319 Nov 2026, 17:21 UTC
category: uxkE1.375319 Nov 2026, 17:21 UTC

Change log

category: uxkE1.400 → 1.37519 Nov 2026, 17:21 UTC
category: uxkC0.973 → 0.92619 Nov 2026, 17:21 UTC
category: uxkI1.087 → 0.87019 Nov 2026, 17:21 UTC
category: uxkE1.167 → 1.40019 Nov 2026, 17:20 UTC
category: uxkC1.019 → 0.97319 Nov 2026, 17:20 UTC
category: uxkI1.200 → 1.08719 Nov 2026, 17:20 UTC
category: uxkE1.000 → 1.16719 Nov 2026, 17:20 UTC
category: uxkC1.000 → 1.01919 Nov 2026, 17:20 UTC
category: uxkI1.000 → 1.20019 Nov 2026, 17:20 UTC

Factor weights

Live: impact ×0.6, severity ×0.4. A proposal appears after 30 measured results.

Detector precision

DETECTORINSIGHTSLED TO AN IMPROVED RESULT
funnel_drop20
progress_dropoff10
error_spike10
segment_gap60
double_fire10
untracked_event10
missing_property10

Lessons

LESSONSCOPEWEIGHTSTATUS
On "Start the first test", "Shorter signup form" did not move conversion (-1.5 pts, n=6372).step: s21.00active
On "Finish the first test", "Hide the timer in practice mode" did not move conversion (0.8 pts, n=2773).step: s31.00active
On "Send a paid-access request", "Show the paywall link on results" moved conversion (13.1 pts, n=476).step: s61.00active

Prompt versions

STAGEVERSIONSTATUSEVALMODEL
FUNNELv1live—claude-haiku-4-5
PROFILEv1live—claude-haiku-4-5
S10v1retired—claude-opus-5-5
S10v2live—claude-opus-5-5
S11v1live—claude-opus-5-5
S3v1retired—claude-opus-5-5
S3v2live—claude-opus-5-5
S4v1live—claude-opus-5-5
S5v1retired—claude-opus-5-5
S5v2live—claude-opus-5-5
S6v1live—claude-opus-5-5
S7v1retired—claude-haiku-4-5
S7v2live—claude-haiku-4-5
S8v1retired—claude-haiku-4-5
S8v2live—claude-haiku-4-5
S9v1live——
preamblev1retired——
preamblev2live——

A candidate needs an eval pass (pnpm eval <stage>) and one shadow run before approval. Live prompts are never edited in place.

Model calls

STAGECALLSFAILEDAVG LATENCYCOST
S1030324 ms$0.000
S1130322 ms$0.000
S310393 ms$0.000
S410378 ms$0.000
S5120694 ms$0.000
S6120344 ms$0.000
S7120322 ms$0.000