0.69
[0.44, 0.86]
Δ +0.12 [−0.19, +0.41]
docs/readme_comprehension
1.00
[0.68, 1.00]
Δ 0.00 [−0.32, +0.32]
first: 0a3e6b0
last: 2a0e7b7
newest-changed: 2a0e7b7
| Commit | Time | Model | Samples | Score (95% CI) | Action |
|---|---|---|---|---|---|
| 2a0e7b7 | 55m | google/gemini-3.5-flash-lite | 8 | 1.00 [0.68, 1.00] | view samples → |
| 0a3e6b0 | 2d | google/gemini-3.5-flash-lite | 8 | 1.00 [0.68, 1.00] | view samples → |
docs/readme_comprehension_blind
0.38
[0.14, 0.69]
Δ +0.25 [−0.17, +0.59]
first: 0a3e6b0
last: 2a0e7b7
newest-changed: 2a0e7b7
| Commit | Time | Model | Samples | Score (95% CI) | Action |
|---|---|---|---|---|---|
| 2a0e7b7 | 55m | google/gemini-3.5-flash-lite | 8 | 0.38 [0.14, 0.69] | view samples → |
| 0a3e6b0 | 2d | google/gemini-3.5-flash-lite | 8 | 0.12 [0.02, 0.47] | view samples → |
Score at main is the pooled Wilson success rate over the newest completed baseline observations at the current commit. An observation is one completed task result on one Run.