PolicyBench
GPT-5.4 mini
OpenAI · AI alone, no tools
United States rank
#20 of 23
US exact
70.5%
United States
United States benchmark
Exact match
70.5%
Bounded score
78.8%
Parse rate
100.0%
Eligibility flags
898/984
Score by programhardest first
| Program | Exact | Within 1% | n |
|---|---|---|---|
| Federal tax before refundable credits | 50.0% | 50.0% | 100 |
| Payroll tax | 54.0% | 65.0% | 100 |
| State tax before refundable credits | 56.0% | 56.0% | 100 |
| SNAP | 74.0% | 74.0% | 100 |
| State refundable credits | 79.0% | 79.0% | 100 |
| Federal refundable credits | 81.0% | 81.0% | 100 |
| Person-level Medicaid eligibility | 83.6% | 83.6% | 177 |
| Person-level Head Start eligibility | 84.2% | 84.2% | 38 |
| Person-level WIC eligibility | 86.4% | 86.4% | 177 |
| Self-employment tax | 90.0% | 95.0% | 100 |
| Person-level CHIP eligibility | 92.1% | 92.1% | 177 |
| Person-level Early Head Start eligibility | 92.1% | 92.1% | 38 |
| Person-level Medicare eligibility | 96.0% | 96.0% | 177 |
| SSI | 97.0% | 97.0% | 100 |
| Free school meals eligibility | 98.0% | 98.0% | 100 |
| Reduced-price school meals eligibility | 99.0% | 99.0% | 100 |
| TANF | 99.0% | 99.0% | 100 |
| Local income tax | 100.0% | 100.0% | 100 |
Hardest casesworst misses on positive references
- SNAP636% offReference $288 · predicted $2,116Inspect household #030
- SNAP307% offReference $288 · predicted $1,172Inspect household #108
- Federal tax before refundable credits100% offReference $2,883 · predicted $0Inspect household #000
- Federal tax before refundable credits100% offReference $22,155 · predicted $0Inspect household #003
- Federal tax before refundable credits100% offReference $3,708 · predicted $0Inspect household #007
- State tax before refundable credits100% offReference $756 · predicted $0Inspect household #007
Scores are from the frozen manuscript snapshot under the AI-alone condition: one structured response per household, no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology.