PolicyBench
GPT-5.6 Sol
OpenAI · AI alone, no tools
United States rank
#1 of 23
US exact
88.7%
United States
United States benchmark
Exact match
88.7%
Bounded score
95.3%
Parse rate
100.0%
Eligibility flags
965/984
Score by programhardest first
| Program | Exact | Within 1% | n |
|---|---|---|---|
| State tax before refundable credits | 64.0% | 68.0% | 100 |
| Federal tax before refundable credits | 75.0% | 84.0% | 100 |
| SNAP | 78.0% | 82.0% | 100 |
| State refundable credits | 84.0% | 84.0% | 100 |
| Federal refundable credits | 94.0% | 96.0% | 100 |
| Payroll tax | 94.0% | 94.0% | 100 |
| Person-level Medicare eligibility | 94.4% | 94.4% | 177 |
| Person-level Medicaid eligibility | 96.6% | 96.6% | 177 |
| SSI | 97.0% | 97.0% | 100 |
| Person-level CHIP eligibility | 98.9% | 98.9% | 177 |
| Reduced-price school meals eligibility | 99.0% | 99.0% | 100 |
| TANF | 99.0% | 99.0% | 100 |
| Free school meals eligibility | 100.0% | 100.0% | 100 |
| Local income tax | 100.0% | 100.0% | 100 |
| Person-level Early Head Start eligibility | 100.0% | 100.0% | 38 |
| Person-level Head Start eligibility | 100.0% | 100.0% | 38 |
| Person-level WIC eligibility | 100.0% | 100.0% | 177 |
| Self-employment tax | 100.0% | 100.0% | 100 |
Hardest casesworst misses on positive references
- SNAP425% offReference $461 · predicted $2,424Inspect household #023
- Federal tax before refundable credits233% offReference $1,346 · predicted $4,485Inspect household #039
- State refundable credits204% offReference $473 · predicted $1,437Inspect household #100
- State tax before refundable credits186% offReference $514 · predicted $1,473Inspect household #039
- Federal tax before refundable credits123% offReference $5,200 · predicted $11,600Inspect household #123
- State refundable credits100% offReference $25 · predicted $0Inspect household #013
Scores are from the frozen manuscript snapshot under the AI-alone condition: one structured response per household, no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology.