Skeleton first: four agents, told to render early vs told how
ops-dashboard · 24 runs · horizon 420s · 163 distinct screenshots · showing the first 240s, by when every run had ended
Correctness over time
The chart stops at 240s because every run had ended by then. Each curve holds its last value from there to the 420s horizon, and the AUC is integrated over all of it.
Follow-up edits
Once the first build was on screen, every agent got the same follow-up prompts in its live session. Each edit is replayed from the moment its prompt was sent, so the panels show the same edit at the same moment however long each agent took over the first build.
Ranking
| Run | AUC | Final | First render | First reviewable | By AUC | By final | Window closed by |
|---|---|---|---|---|---|---|---|
| Told the clock is running | |||||||
| cc-sonnet-5.5 | 0.953 | 1.00 | 19.6s | 19.6s | 1 | 1 | signal |
| cc-sonnet-5.5 | 0.953 | 1.00 | 19.9s | 19.9s | 2 | 1 ≠ | signal |
| cc-sonnet-5.5 | 0.941 | 1.00 | 24.9s | 24.9s | 3 | 1 ≠ | signal |
| pi-gpt-5.5 | 0.936 | 1.00 | 27.1s | 27.1s | 4 | 1 ≠ | signal |
| pi-gpt-5.5 | 0.932 | 1.00 | 28.4s | 28.4s | 5 | 1 ≠ | signal |
| pi-gpt-5.5 | 0.927 | 1.00 | 30.6s | 30.6s | 6 | 1 ≠ | signal |
| cc-opus-5.5 | 0.926 | 1.00 | 31.1s | 31.1s | 7 | 1 ≠ | signal |
| cc-opus-5.5 | 0.921 | 1.00 | 33.0s | 33.0s | 8 | 1 ≠ | signal |
| cc-opus-5.5 | 0.912 | 1.00 | 36.9s | 36.9s | 9 | 1 ≠ | signal |
| pi-gemini-3.8-flash | 0.895 | 1.00 | 44.0s | 44.0s | 10 | 1 ≠ | signal |
| pi-gemini-3.8-flash | 0.893 | 1.00 | 45.1s | 45.1s | 11 | 1 ≠ | signal |
| pi-gemini-3.8-flash | 0.892 | 1.00 | 45.3s | 45.3s | 12 | 1 ≠ | signal |
| Told to save a skeleton first, then one section per save | |||||||
| cc-sonnet-5.5 | 0.942 | 1.00 | 7.5s | 26.9s | 1 | 1 | signal |
| cc-opus-5.5 | 0.942 | 1.00 | 13.7s | 29.3s | 2 | 1 ≠ | signal |
| cc-sonnet-5.5 | 0.931 | 1.00 | 7.5s | 34.0s | 3 | 1 ≠ | signal |
| cc-sonnet-5.5 | 0.931 | 1.00 | 8.7s | 32.7s | 4 | 1 ≠ | signal |
| cc-opus-5.5 | 0.930 | 1.00 | 12.7s | 28.6s | 5 | 1 ≠ | signal |
| pi-gpt-5.5 | 0.929 | 1.00 | 20.9s | 29.3s | 6 | 1 ≠ | signal |
| cc-opus-5.5 | 0.928 | 1.00 | 16.6s | 31.2s | 7 | 1 ≠ | signal |
| pi-gemini-3.8-flash | 0.916 | 1.00 | 16.1s | 37.5s | 8 | 1 ≠ | signal |
| pi-gpt-5.5 | 0.907 | 1.00 | 27.5s | 38.1s | 9 | 1 ≠ | signal |
| pi-gemini-3.8-flash | 0.881 | 1.00 | 28.3s | 57.1s | 10 | 1 ≠ | signal |
| pi-gemini-3.8-flash | 0.874 | 1.00 | 19.9s | 61.7s | 11 | 1 ≠ | signal |
| pi-gpt-5.5 | 0.806 | 0.86 | 12.5s | 27.8s | 12 | 12 | signal |
Highlighted rows rank differently by trajectory than by final score — the disagreement the trajectory metric exists to surface. Playback shows the frames the scores were computed from, on one shared clock, so the ranking above and the pictures below cannot disagree.
What naming the stages was worth
| Run | AUC skeleton | AUC told | Δ AUC | First render skeleton | told | Effect on first render |
|---|---|---|---|---|---|---|
| claude-code:claude-opus-5-5 medians of 3/3 | 0.930 | 0.921 | +0.008 | 13.7s | 33.0s | 19.3s sooner |
| pi:gemini-3.8-flash medians of 3/3 | 0.881 | 0.893 | −0.012 | 19.9s | 45.1s | 25.2s sooner |
| claude-code:claude-sonnet-5-5 medians of 3/3 | 0.931 | 0.953 | −0.022 | 7.5s | 19.9s | 12.4s sooner |
| pi:gpt-5.5 medians of 3/3 | 0.907 | 0.932 | −0.025 | 20.9s | 28.4s | 7.5s sooner |
Paired: the same agent on the same brief, told to render early either way, with one difference — whether it was also told how: a skeleton save first, then one section per save. Positive means the skeleton instruction helped. First render and area under the curve can move in opposite directions here: an early skeleton scores little, and every extra save is another model turn before the page is complete.