todo-app-scaffolded · pi-sonnet-5.5
pi · started 2026-10-07T15:21:34.459Z · horizon 480s · judge ai (anthropic:claude-sonnet-5), 1 frames scored
Area under curve
0.967
headline, 0-1
First render
15.8s
anything on screen
First reviewable
15.8s
worth feedback
Final score
1.00
no regression
Correctness over time
Where the time went
95%
Model thinking 14.6sTool round trips 0.0sUnaccounted 0.8s
| Phase | Time | Share | Side |
|---|---|---|---|
| Model thinking | 14.6s | 94.8% | agent |
| Tool round trips | 0.0s | 0.1% | agent |
| Dependency install | 0.0s | 0.0% | toolchain |
| Build | 0.0s | 0.0% | toolchain |
| Dev server boot | 0.0s | 0.0% | toolchain |
| First paint | 0.0s | 0.0% | toolchain |
| Unaccounted | 0.8s | 5.1% | unknown |
| Wall clock (cold start) | 15.4s | 100% |
Agent or toolchain
95%
Agent (model + tool round trips) 14.6s
Toolchain (install + build + boot + paint) 0.0s
Unaccounted 0.8s
Accounted for: 94.9% of wall clock.
- Excludes 15.0s of deliberate idle observation (settle window); the curve still covers it.
Iteration: prompt to visible change
Follow-ups were injected into the same live session.
| Edit | First visible change | Correct change | Agent done | Tool calls | After agent | Broken for |
|---|---|---|---|---|---|---|
| Make the header background blue. | 3.5s | 3.5s | 5.2s | 1 (edit) | -1.7s | -- |
| Add a fourth column called "Blocked" after Done. | 2.7s | 2.7s | 4.1s | 2 (edit) | -- | -- |
Judged frames
All criteria met: polished, fully-styled board matching the brief's requirements.
Caveats
- The page had only 4 distinct visual state(s) across 110 frames, so frames/ holds 4 screenshot(s) and the other 106 frames reuse them. That is deduplication, not missing captures: the timeline in result.json maps every frame to its picture.
- 1 poll ticks were skipped because a capture overran the interval.
- judge: anthropic:claude-sonnet-5 does not accept a sampling temperature, so these frames were scored at the provider's default rather than at 0. Two runs of the same page can be scored differently, and that variance is inside the AUC. result.json records the temperature as null rather than as the value that was asked for, so p2p compare will not pool this run with one that was scored at a fixed temperature.
- Iteration "add-column" was corrected after the run: The brief's check matched \bblocked\b, and this column header renders as <h2>Blocked<span>0</span></h2>, whose innerText is BLOCKED0 with no word boundary, so it reported NEVER LANDED. Re-checked after the run with the corrected check against the run's final code: it passes. The page changed once after the prompt and held that state to the end of the window, so the edit landed at its first visible change.