Dense ops dashboard: five agent/model pairs, three vendors

ops-dashboard · 15 runs · horizon 420s · 68 distinct screenshots · showing the first 210s, by when every run had ended

00:00 / 00:00
1 pi-sonnet-5.5 AUC 0.962
pi-sonnet-5.5 at the current time
-- first render 16.1s · reviewable 16.1s report
2 cc-sonnet-5.5 AUC 0.957
cc-sonnet-5.5 at the current time
-- first render 18.0s · reviewable 18.0s report
3 cc-sonnet-5.5 AUC 0.955
cc-sonnet-5.5 at the current time
-- first render 18.9s · reviewable 18.9s report
4 cc-sonnet-5.5 AUC 0.952
cc-sonnet-5.5 at the current time
-- first render 20.0s · reviewable 20.0s report
5 pi-sonnet-5.5 AUC 0.945
pi-sonnet-5.5 at the current time
-- first render 23.3s · reviewable 23.3s report
6 pi-sonnet-5.5 AUC 0.929
pi-sonnet-5.5 at the current time
-- first render 29.6s · reviewable 29.6s report
7 pi-gpt-5.5 AUC 0.929
pi-gpt-5.5 at the current time
-- first render 30.0s · reviewable 30.0s report
8 pi-gpt-5.5 AUC 0.926
pi-gpt-5.5 at the current time
-- first render 30.9s · reviewable 30.9s report
9 cc-opus-5.5 AUC 0.920
cc-opus-5.5 at the current time
-- first render 33.4s · reviewable 33.4s report
10 cc-opus-5.5 AUC 0.917
cc-opus-5.5 at the current time
▼●

-- first render 34.7s · reviewable 34.7s report
11 pi-gpt-5.5 AUC 0.915
pi-gpt-5.5 at the current time
-- first render 35.7s · reviewable 35.7s report
12 cc-opus-5.5 AUC 0.914
cc-opus-5.5 at the current time
-- first render 36.1s · reviewable 36.1s report
13 pi-gemini-3.8-flash AUC 0.909
pi-gemini-3.8-flash at the current time
●▼▼

-- first render 38.2s · reviewable 38.2s report
14 pi-gemini-3.8-flash AUC 0.908
pi-gemini-3.8-flash at the current time
●▼

-- first render 38.6s · reviewable 38.6s report
15 pi-gemini-3.8-flash AUC 0.893
pi-gemini-3.8-flash at the current time
▼

-- first render 45.1s · reviewable 45.1s report

Correctness over time

0.00 0.25 0.50 0.75 1.000s35s70s105s140s175s210s pi-sonnet-5.5cc-sonnet-5.5cc-sonnet-5.5cc-sonnet-5.5pi-sonnet-5.5pi-sonnet-5.5pi-gpt-5.5pi-gpt-5.5cc-opus-5.5cc-opus-5.5pi-gpt-5.5cc-opus-5.5pi-gemini-3.8-fpi-gemini-3.8-fpi-gemini-3.8-f

The chart stops at 210s because every run had ended by then. Each curve holds its last value from there to the 420s horizon, and the AUC is integrated over all of it.

Follow-up edits

Once the first build was on screen, every agent got the same follow-up prompts in its live session. Each edit is replayed from the moment its prompt was sent, so the panels show the same edit at the same moment however long each agent took over the first build.

0.0s
1 pi-sonnet-5.5
pi-sonnet-5.5 during the edit
--
2 cc-sonnet-5.5
cc-sonnet-5.5 during the edit
--
3 cc-sonnet-5.5
cc-sonnet-5.5 during the edit
--
4 cc-sonnet-5.5
cc-sonnet-5.5 during the edit
--
5 pi-sonnet-5.5
pi-sonnet-5.5 during the edit
--
6 pi-sonnet-5.5
pi-sonnet-5.5 during the edit
--
7 pi-gpt-5.5
pi-gpt-5.5 during the edit
--
8 pi-gpt-5.5
pi-gpt-5.5 during the edit
--
9 cc-opus-5.5
cc-opus-5.5 during the edit
--
10 cc-opus-5.5
cc-opus-5.5 during the edit
--
11 pi-gpt-5.5
pi-gpt-5.5 during the edit
--
12 cc-opus-5.5
cc-opus-5.5 during the edit
--
13 pi-gemini-3.8-flash
pi-gemini-3.8-flash during the edit
--
14 pi-gemini-3.8-flash
pi-gemini-3.8-flash during the edit
--
15 pi-gemini-3.8-flash
pi-gemini-3.8-flash during the edit
--

Ranking

RunAUCFinal First renderFirst reviewable By AUCBy finalLater changesWindow closed by
pi-sonnet-5.5 0.9621.00 16.1s16.1s 1 1 nonesignal
cc-sonnet-5.5 0.9571.00 18.0s18.0s 2 1 ≠ nonesignal
cc-sonnet-5.5 0.9551.00 18.9s18.9s 3 1 ≠ nonesignal
cc-sonnet-5.5 0.9521.00 20.0s20.0s 4 1 ≠ nonesignal
pi-sonnet-5.5 0.9451.00 23.3s23.3s 5 1 ≠ nonesignal
pi-sonnet-5.5 0.9291.00 29.6s29.6s 6 1 ≠ nonesignal
pi-gpt-5.5 0.9291.00 30.0s30.0s 7 1 ≠ nonesignal
pi-gpt-5.5 0.9261.00 30.9s30.9s 8 1 ≠ nonesignal
cc-opus-5.5 0.9201.00 33.4s33.4s 9 1 ≠ nonesignal
cc-opus-5.5 0.9171.00 34.7s34.7s 10 1 ≠ ▲0 ●1 ▼1signal
pi-gpt-5.5 0.9151.00 35.7s35.7s 11 1 ≠ nonesignal
cc-opus-5.5 0.9141.00 36.1s36.1s 12 1 ≠ nonesignal
pi-gemini-3.8-flash 0.9091.00 38.2s38.2s 13 1 ≠ ▲0 ●1 ▼2signal
pi-gemini-3.8-flash 0.9081.00 38.6s38.6s 14 1 ≠ ▲0 ●1 ▼1signal
pi-gemini-3.8-flash 0.8931.00 45.1s45.1s 15 1 ≠ ▲0 ●0 ▼1signal

Highlighted rows rank differently by trajectory than by final score — the disagreement the trajectory metric exists to surface. Playback shows the frames the scores were computed from, on one shared clock, so the ranking above and the pictures below cannot disagree. Later changes counts each visible change after the first render, judged against the state it replaced — ▲ better, ● about the same, ▼ worse. It is reported beside the scores and never changes them; a rubric that every version passes cannot say whether a change helped, and this can. Each pair is judged in both orders, and a preference that flips with the order is counted as the same.