Benchmark
Pattern IQ v3
pattern-iq@3.0.0
Pattern IQ v3
pattern-iq@3.0.0
Questions
50
50
Models
7
7
All Models Pass
28
28
All Fail / Disagreement
0 / 22
0 / 22
Observed model totals · sorted by accuracy
gpt-5.6
gpt-5.6
50/50 · 100%
0 failed · 0 errors
Avg latency: 3096 ms
Qwen 32B AWQqwen32b-awq
44/50 · 88%
6 failed · 0 errors
Avg latency: 253 ms
GPT-4o-minigpt-4o-mini
40/50 · 80%
10 failed · 0 errors
Avg latency: 2762 ms
Qwen 2.5 14Bqwen2.5-14b
39/50 · 78%
11 failed · 0 errors
Avg latency: 1537 ms
Gemma 2 9B ITgemma2-9b-it
38/50 · 76%
12 failed · 0 errors
Avg latency: 186 ms
Qwen 2.5 14B AWQqwen2.5-14b-awq
36/50 · 72%
14 failed · 0 errors
Avg latency: 148 ms
Phi-3 Mediumphi3-medium-4k
33/50 · 66%
17 failed · 0 errors
Avg latency: 2367 ms
Question × model matrix (6 shown)
| Question | Qwen 32B AWQ |
|---|---|
| piq07_letters_skip_pattern Find the next letter when alphabet-position increments are +2, +3, +4, +5: A, C, F, J, ? | ✗ FAIL |
| piq19_mixed_odd_even Given: 2, 9, 4, 11, 6, 13, ? The even numbers increase by 2, the odd numbers increase by 2. What is | ✗ FAIL |
| piq35_mixed_code_pattern Using A=1, B=2, ..., Z=26, which letter string is encoded by 3, 1, 20, 20, 1? | ✗ FAIL |
| piq36v2_numeric_grid_offsets A 3×3 grid starts with 2 at top-left. Moving one cell right adds 3; moving one cell down adds 4. Wha | ✗ FAIL |
| piq42_advanced_letter_block Two-letter blocks start at A, C, F, J, with starting-letter increments +2, +3, +4, then +5. Each blo | ✗ FAIL |
| piq47v2_balanced_rows Each row has the same sum. The rows are [2, 5, 7], [3, 4, 7], [6, 1, ?]. What is missing? | ✗ FAIL |
✓ pass · ✗ fail · ERROR execution/evaluation error · — unavailable. Outcomes are observations for the selected runs, not general capability claims.