PolicyBench

GLM-5.2

Z.ai · AI alone, no tools

Current board, snapshot 2026-10-14

United States rank

#42 of 50

US exact

76.6%

United States

United States benchmark

Exact match

76.6%

Bounded score

86.2%

Parse rate

93.0%

Eligibility flags

892/975

Exact match by programall 18 output groups, best first

Bar length is the unweighted exact-match rate on that program's outputs; the right column is the program's share of the headline weight, which follows household dollars, so the bars do not average to the headline.

Reduced-price school meals eligibility94.0%0.1%
TANF94.0%0.3%
Local income tax93.9%0.3%
Person-level CHIP eligibility93.2%0.2%
Free school meals eligibility93.0%0.7%
SSI92.8%2.0%
Person-level WIC eligibility92.7%0.3%
Person-level Medicare eligibility91.3%11%
Self-employment tax91.0%2.1%
Person-level Early Head Start eligibility89.5%3.1%
Person-level Head Start eligibility89.5%1.2%
Person-level Medicaid eligibility87.3%30%
SNAP81.7%4.1%
Federal refundable credits79.0%3.7%
Payroll tax75.0%16%
State refundable credits74.7%0.6%
State tax before refundable credits49.5%6.0%
Federal tax before refundable credits47.6%19%

Table view (exact, within 1%, outputs), hardest first

Program	Exact	Within 1%	n
Federal tax before refundable credits	47.6%	47.6%	84
State tax before refundable credits	49.5%	51.6%	93
State refundable credits	74.7%	74.7%	99
Payroll tax	75.0%	80.2%	96
Federal refundable credits	79.0%	79.0%	100
SNAP	81.7%	82.8%	93
Person-level Medicaid eligibility	87.3%	87.3%	173
Person-level Early Head Start eligibility	89.5%	89.5%	38
Person-level Head Start eligibility	89.5%	89.5%	38
Self-employment tax	91.0%	91.0%	100
Person-level Medicare eligibility	91.3%	91.3%	172
Person-level WIC eligibility	92.7%	92.7%	177
SSI	92.8%	92.8%	97
Free school meals eligibility	93.0%	93.0%	100
Person-level CHIP eligibility	93.2%	93.2%	177
Local income tax	93.9%	93.9%	98
Reduced-price school meals eligibility	94.0%	94.0%	100
TANF	94.0%	94.0%	100

Hardest casesworst misses on positive references

Federal refundable creditsno parseable answer
Reference $12,434
Inspect household #008
Payroll taxno parseable answer
Reference $2,277
Inspect household #008
Self-employment taxno parseable answer
Reference $565
Inspect household #008
State tax before refundable creditsno parseable answer
Reference $265
Inspect household #008
State refundable creditsno parseable answer
Reference $5,842
Inspect household #008
Federal tax before refundable creditsno parseable answer
Reference $0
Inspect household #027

Scores are from the frozen manuscript snapshot under the AI-alone condition: structured responses over the same household facts (whole-scenario or in output subsets, per the serving-configuration table), no tools, graded against PolicyEngine reference outputs. See the leaderboard and paper for methodology. The frozen US annotations cover 8,765 rows whose legacy threshold score is below 1. That universe contains 8,761 of 8,761 exact-match misses and 4 exact hits. Another 2,329 rows with bounded score below 100 were not selected and have no audit annotation.