PolicyBench
Gemini 3.1 Pro Preview
Google · AI alone, no tools
United States rank
#6 of 23
US exact
77.9%
United States
United States benchmark
Exact match
77.9%
Bounded score
91.1%
Parse rate
100.0%
Eligibility flags
964/984
Score by programhardest first
| Program | Exact | Within 1% | n |
|---|---|---|---|
| Federal tax before refundable credits | 46.0% | 47.0% | 100 |
| State tax before refundable credits | 57.0% | 58.0% | 100 |
| Payroll tax | 66.0% | 67.0% | 100 |
| SNAP | 78.0% | 78.0% | 100 |
| State refundable credits | 81.0% | 81.0% | 100 |
| Federal refundable credits | 85.0% | 88.0% | 100 |
| Person-level Head Start eligibility | 94.7% | 94.7% | 38 |
| SSI | 95.0% | 95.0% | 100 |
| Person-level Medicaid eligibility | 96.6% | 96.6% | 177 |
| Person-level Medicare eligibility | 96.6% | 96.6% | 177 |
| Person-level CHIP eligibility | 98.3% | 98.3% | 177 |
| Free school meals eligibility | 99.0% | 99.0% | 100 |
| Reduced-price school meals eligibility | 99.0% | 99.0% | 100 |
| TANF | 99.0% | 99.0% | 100 |
| Person-level WIC eligibility | 99.4% | 99.4% | 177 |
| Local income tax | 100.0% | 100.0% | 100 |
| Person-level Early Head Start eligibility | 100.0% | 100.0% | 38 |
| Self-employment tax | 100.0% | 100.0% | 100 |
Hardest casesworst misses on positive references
- Federal tax before refundable credits748% offReference $1,346 · predicted $11,415Inspect household #039
- SNAP657% offReference $461 · predicted $3,492Inspect household #023
- State tax before refundable credits287% offReference $514 · predicted $1,990Inspect household #039
- Federal tax before refundable credits273% offReference $500 · predicted $1,864Inspect household #046
- State tax before refundable credits202% offReference $285 · predicted $860Inspect household #042
- State tax before refundable credits193% offReference $1,028 · predicted $3,013Inspect household #122
Scores are from the frozen manuscript snapshot under the AI-alone condition: one structured response per household, no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology.