PolicyBench
Claude Opus 4.7
Anthropic · AI alone, no tools
United States rank
#7 of 23
US exact
77.4%
United States
United States benchmark
Exact match
77.4%
Bounded score
88.7%
Parse rate
100.0%
Eligibility flags
905/984
Score by programhardest first
| Program | Exact | Within 1% | n |
|---|---|---|---|
| Federal tax before refundable credits | 54.0% | 56.0% | 100 |
| State tax before refundable credits | 57.0% | 61.0% | 100 |
| SNAP | 73.0% | 73.0% | 100 |
| State refundable credits | 75.0% | 75.0% | 100 |
| Federal refundable credits | 81.0% | 82.0% | 100 |
| Payroll tax | 83.0% | 85.0% | 100 |
| Person-level Medicaid eligibility | 86.4% | 86.4% | 177 |
| Person-level Head Start eligibility | 86.8% | 86.8% | 38 |
| Person-level CHIP eligibility | 88.1% | 88.1% | 177 |
| Person-level Medicare eligibility | 89.8% | 89.8% | 177 |
| Person-level WIC eligibility | 95.5% | 95.5% | 177 |
| SSI | 96.0% | 96.0% | 100 |
| Reduced-price school meals eligibility | 98.0% | 98.0% | 100 |
| Free school meals eligibility | 99.0% | 99.0% | 100 |
| TANF | 99.0% | 99.0% | 100 |
| Local income tax | 100.0% | 100.0% | 100 |
| Person-level Early Head Start eligibility | 100.0% | 100.0% | 38 |
| Self-employment tax | 100.0% | 100.0% | 100 |
Hardest casesworst misses on positive references
- State refundable credits5847% offReference $19 · predicted $1,130Inspect household #043
- SNAP1143% offReference $288 · predicted $3,576Inspect household #073
- SNAP845% offReference $288 · predicted $2,718Inspect household #045
- Federal tax before refundable credits834% offReference $500 · predicted $4,669Inspect household #046
- SNAP717% offReference $461 · predicted $3,768Inspect household #023
- SNAP674% offReference $288 · predicted $2,227Inspect household #108
Scores are from the frozen manuscript snapshot under the AI-alone condition: one structured response per household, no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology.