Task board on a running Vite + React project: five agent/model pairs, three vendors

todo-app-scaffolded · 15 runs · horizon 480s · 67 distinct screenshots · showing the first 150s, by when every run had ended

00:00 / 00:00
1 cc-sonnet-5.5 AUC 0.973
cc-sonnet-5.5 at the current time
-- first render 12.9s · reviewable 12.9s report
2 cc-sonnet-5.5 AUC 0.971
cc-sonnet-5.5 at the current time
-- first render 13.8s · reviewable 13.8s report
3 pi-sonnet-5.5 AUC 0.967
pi-sonnet-5.5 at the current time
-- first render 15.8s · reviewable 15.8s report
4 pi-gpt-5.5 AUC 0.964
pi-gpt-5.5 at the current time
▲▲

-- first render 12.5s · reviewable 12.5s report
5 cc-opus-5.5 AUC 0.956
cc-opus-5.5 at the current time
●

-- first render 21.2s · reviewable 21.2s report
6 pi-gpt-5.5 AUC 0.955
pi-gpt-5.5 at the current time
-- first render 21.5s · reviewable 21.5s report
7 cc-opus-5.5 AUC 0.951
cc-opus-5.5 at the current time
●

-- first render 23.6s · reviewable 23.6s report
8 pi-gpt-5.5 AUC 0.947
pi-gpt-5.5 at the current time
-- first render 25.4s · reviewable 25.4s report
9 cc-sonnet-5.5 AUC 0.945
cc-sonnet-5.5 at the current time
-- first render 26.4s · reviewable 26.4s report
10 cc-opus-5.5 AUC 0.944
cc-opus-5.5 at the current time
●

-- first render 26.9s · reviewable 26.9s report
11 pi-sonnet-5.5 AUC 0.940
pi-sonnet-5.5 at the current time
-- first render 28.9s · reviewable 28.9s report
12 pi-gemini-3.8-flash AUC 0.878
pi-gemini-3.8-flash at the current time
▲

-- first render 58.4s · reviewable 58.4s report
13 pi-sonnet-5.5 AUC 0.874
pi-sonnet-5.5 at the current time
-- first render 60.3s · reviewable 60.3s report
14 pi-gemini-3.8-flash AUC 0.860
pi-gemini-3.8-flash at the current time
▼

-- first render 67.4s · reviewable 67.4s report
15 pi-gemini-3.8-flash AUC 0.854
pi-gemini-3.8-flash at the current time
▼●

-- first render 68.8s · reviewable 68.8s report

Correctness over time

0.00 0.25 0.50 0.75 1.000s25s50s75s100s125s150s cc-sonnet-5.5cc-sonnet-5.5pi-sonnet-5.5pi-gpt-5.5cc-opus-5.5pi-gpt-5.5cc-opus-5.5pi-gpt-5.5cc-sonnet-5.5cc-opus-5.5pi-sonnet-5.5pi-gemini-3.8-fpi-sonnet-5.5pi-gemini-3.8-fpi-gemini-3.8-f

The chart stops at 150s because every run had ended by then. Each curve holds its last value from there to the 480s horizon, and the AUC is integrated over all of it.

Follow-up edits

Once the first build was on screen, every agent got the same follow-up prompts in its live session. Each edit is replayed from the moment its prompt was sent, so the panels show the same edit at the same moment however long each agent took over the first build.

0.0s
1 cc-sonnet-5.5
cc-sonnet-5.5 during the edit
--
2 cc-sonnet-5.5
cc-sonnet-5.5 during the edit
--
3 pi-sonnet-5.5
pi-sonnet-5.5 during the edit
--
4 pi-gpt-5.5
pi-gpt-5.5 during the edit
--
5 cc-opus-5.5
cc-opus-5.5 during the edit
--
6 pi-gpt-5.5
pi-gpt-5.5 during the edit
--
7 cc-opus-5.5
cc-opus-5.5 during the edit
--
8 pi-gpt-5.5
pi-gpt-5.5 during the edit
--
9 cc-sonnet-5.5
cc-sonnet-5.5 during the edit
--
10 cc-opus-5.5
cc-opus-5.5 during the edit
--
11 pi-sonnet-5.5
pi-sonnet-5.5 during the edit
--
12 pi-gemini-3.8-flash
pi-gemini-3.8-flash during the edit
--
13 pi-sonnet-5.5
pi-sonnet-5.5 during the edit
--
14 pi-gemini-3.8-flash
pi-gemini-3.8-flash during the edit
--
15 pi-gemini-3.8-flash
pi-gemini-3.8-flash during the edit
--

Ranking

RunAUCFinal First renderFirst reviewable By AUCBy finalLater changesWindow closed by
cc-sonnet-5.5 0.9731.00 12.9s12.9s 1 1 nonesignal
cc-sonnet-5.5 0.9711.00 13.8s13.8s 2 1 ≠ nonesignal
pi-sonnet-5.5 0.9671.00 15.8s15.8s 3 1 ≠ nonesignal
pi-gpt-5.5 0.9641.00 12.5s12.5s 4 1 ≠ ▲2 ●0 ▼0signal
cc-opus-5.5 0.9561.00 21.2s21.2s 5 1 ≠ ▲0 ●1 ▼0signal
pi-gpt-5.5 0.9551.00 21.5s21.5s 6 1 ≠ nonesignal
cc-opus-5.5 0.9511.00 23.6s23.6s 7 1 ≠ ▲0 ●1 ▼0signal
pi-gpt-5.5 0.9471.00 25.4s25.4s 8 1 ≠ nonesignal
cc-sonnet-5.5 0.9451.00 26.4s26.4s 9 1 ≠ nonesignal
cc-opus-5.5 0.9441.00 26.9s26.9s 10 1 ≠ ▲0 ●1 ▼0signal
pi-sonnet-5.5 0.9401.00 28.9s28.9s 11 1 ≠ nonesignal
pi-gemini-3.8-flash 0.8781.00 58.4s58.4s 12 1 ≠ ▲1 ●0 ▼0signal
pi-sonnet-5.5 0.8741.00 60.3s60.3s 13 1 ≠ nonesignal
pi-gemini-3.8-flash 0.8601.00 67.4s67.4s 14 1 ≠ ▲0 ●0 ▼1signal
pi-gemini-3.8-flash 0.8541.00 68.8s68.8s 15 1 ≠ ▲0 ●1 ▼1signal

Highlighted rows rank differently by trajectory than by final score — the disagreement the trajectory metric exists to surface. Playback shows the frames the scores were computed from, on one shared clock, so the ranking above and the pictures below cannot disagree. Later changes counts each visible change after the first render, judged against the state it replaced — ▲ better, ● about the same, ▼ worse. It is reported beside the scores and never changes them; a rubric that every version passes cannot say whether a change helped, and this can. Each pair is judged in both orders, and a preference that flips with the order is counted as the same.