PolicyBench
Grok Build 0.1
xAI · AI alone, no tools
United States rank
#12 of 23
US exact
76.1%
United States
United States benchmark
Exact match
76.1%
Bounded score
88.9%
Parse rate
100.0%
Eligibility flags
962/984
Score by programhardest first
| Program | Exact | Within 1% | n |
|---|---|---|---|
| Federal tax before refundable credits | 40.0% | 40.0% | 100 |
| State tax before refundable credits | 47.0% | 49.0% | 100 |
| Payroll tax | 71.0% | 76.0% | 100 |
| SNAP | 78.0% | 80.0% | 100 |
| State refundable credits | 78.0% | 78.0% | 100 |
| Federal refundable credits | 83.0% | 83.0% | 100 |
| Person-level Medicaid eligibility | 94.4% | 94.4% | 177 |
| SSI | 95.0% | 95.0% | 100 |
| Reduced-price school meals eligibility | 97.0% | 97.0% | 100 |
| Person-level CHIP eligibility | 97.7% | 97.7% | 177 |
| Free school meals eligibility | 98.0% | 98.0% | 100 |
| Person-level Medicare eligibility | 98.3% | 98.3% | 177 |
| Self-employment tax | 99.0% | 99.0% | 100 |
| TANF | 99.0% | 99.0% | 100 |
| Local income tax | 100.0% | 100.0% | 100 |
| Person-level Early Head Start eligibility | 100.0% | 100.0% | 38 |
| Person-level Head Start eligibility | 100.0% | 100.0% | 38 |
| Person-level WIC eligibility | 100.0% | 100.0% | 177 |
Hardest casesworst misses on positive references
- State tax before refundable credits2386% offReference $7 · predicted $169Inspect household #023
- State refundable credits726% offReference $19 · predicted $157Inspect household #043
- SNAP680% offReference $461 · predicted $3,600Inspect household #023
- Federal tax before refundable credits554% offReference $1,346 · predicted $8,805Inspect household #039
- State tax before refundable credits292% offReference $514 · predicted $2,019Inspect household #039
- Federal tax before refundable credits223% offReference $1,979 · predicted $6,389Inspect household #042
Scores are from the frozen manuscript snapshot under the AI-alone condition: one structured response per household, no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology.