PolicyBench
GPT-5.4 nano
OpenAI · AI alone, no tools
United States rank
#23 of 23
US exact
62.3%
United States
United States benchmark
Exact match
62.3%
Bounded score
73.4%
Parse rate
100.0%
Eligibility flags
898/984
Score by programhardest first
| Program | Exact | Within 1% | n |
|---|---|---|---|
| Payroll tax | 17.0% | 22.0% | 100 |
| Federal tax before refundable credits | 47.0% | 48.0% | 100 |
| State tax before refundable credits | 52.0% | 52.0% | 100 |
| Person-level Medicaid eligibility | 71.2% | 71.2% | 177 |
| State refundable credits | 79.0% | 79.0% | 100 |
| SNAP | 80.0% | 80.0% | 100 |
| Federal refundable credits | 86.0% | 86.0% | 100 |
| Person-level Medicare eligibility | 91.0% | 91.0% | 177 |
| Self-employment tax | 91.0% | 92.0% | 100 |
| Free school meals eligibility | 92.0% | 92.0% | 100 |
| Person-level Head Start eligibility | 94.7% | 94.7% | 38 |
| Person-level WIC eligibility | 96.0% | 96.0% | 177 |
| SSI | 97.0% | 97.0% | 100 |
| Person-level Early Head Start eligibility | 97.4% | 97.4% | 38 |
| Reduced-price school meals eligibility | 99.0% | 99.0% | 100 |
| TANF | 99.0% | 99.0% | 100 |
| Local income tax | 100.0% | 100.0% | 100 |
| Person-level CHIP eligibility | 100.0% | 100.0% | 177 |
Hardest casesworst misses on positive references
- Federal tax before refundable credits1260% offReference $500 · predicted $6,794Inspect household #046
- State tax before refundable credits959% offReference $7 · predicted $72Inspect household #023
- Payroll tax198% offReference $4,590 · predicted $13,663Inspect household #009
- State tax before refundable credits195% offReference $3,070 · predicted $9,055Inspect household #123
- Federal tax before refundable credits182% offReference $5,200 · predicted $14,677Inspect household #123
- State tax before refundable credits134% offReference $2,087 · predicted $4,875Inspect household #026
Scores are from the frozen manuscript snapshot under the AI-alone condition: one structured response per household, no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology.