PolicyBench
Claude Sonnet 4.6
Anthropic · AI alone, no tools
United States rank
#9 of 23
US exact
77.1%
United States
United States benchmark
Exact match
77.1%
Bounded score
89.9%
Parse rate
100.0%
Eligibility flags
924/984
Score by programhardest first
| Program | Exact | Within 1% | n |
|---|---|---|---|
| Federal tax before refundable credits | 48.0% | 50.0% | 100 |
| State tax before refundable credits | 54.0% | 56.0% | 100 |
| SNAP | 76.0% | 76.0% | 100 |
| State refundable credits | 79.0% | 79.0% | 100 |
| Payroll tax | 83.0% | 87.0% | 100 |
| Person-level Medicaid eligibility | 87.0% | 87.0% | 177 |
| Federal refundable credits | 89.0% | 90.0% | 100 |
| Person-level CHIP eligibility | 89.3% | 89.3% | 177 |
| Person-level Medicare eligibility | 94.4% | 94.4% | 177 |
| Person-level Early Head Start eligibility | 94.7% | 94.7% | 38 |
| SSI | 97.0% | 97.0% | 100 |
| Person-level Head Start eligibility | 97.4% | 97.4% | 38 |
| Free school meals eligibility | 98.0% | 98.0% | 100 |
| Reduced-price school meals eligibility | 98.0% | 98.0% | 100 |
| Self-employment tax | 98.0% | 99.0% | 100 |
| TANF | 99.0% | 99.0% | 100 |
| Person-level WIC eligibility | 99.4% | 99.4% | 177 |
| Local income tax | 100.0% | 100.0% | 100 |
Hardest casesworst misses on positive references
- Federal tax before refundable credits635% offReference $1,910 · predicted $14,034Inspect household #068
- Federal tax before refundable credits493% offReference $500 · predicted $2,963Inspect household #046
- SNAP405% offReference $461 · predicted $2,328Inspect household #023
- State tax before refundable credits260% offReference $1,253 · predicted $4,514Inspect household #068
- Federal tax before refundable credits254% offReference $1,346 · predicted $4,768Inspect household #039
- State refundable credits200% offReference $25 · predicted $75Inspect household #013
Scores are from the frozen manuscript snapshot under the AI-alone condition: one structured response per household, no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology.