Claude Fable 5.2 joins the board at #5 as PolicyBench stops scoring one Minnesota output
PolicyBench added Claude Fable 5.2 to its 2026-11-11 snapshot, two days after Anthropic released it, bringing the board to 52 models. Claude Fable 5.2 gets 92.0% of answers right and ranks #5 of 52, behind GPT-5.6 Sol (93.1%) and ahead of GPT-6 Astra (91.7%, #6). GPT-6 Sol still leads at 95.2%, ahead of Claude Opus 5.5 (94.1%) and GPT-6.1 Terra (93.1%). On the bounded score, which gives partial credit for amounts close to the reference, Claude Fable 5.2 ranks #3, behind only GPT-6 Sol and Claude Opus 5.5. Claude Fable 5.2 ran at Anthropic's default effort, high, and Claude Opus 5.5 defaults to medium. PolicyBench counts a dollar amount as right when it falls within $1 of the reference PolicyEngine computes, and an eligibility answer as right when it matches. It weights each answer by household impact.
Claude Fable 5.2's API rejects forced tool calls, as Claude Fable 5.1's, Claude Opus 5.5's and Claude Sonnet 5.5's do, so its row answers as a JSON object. PolicyBench sends none of the four rows a reasoning setting, and each runs at Anthropic's default effort; Claude Fable 5.1 and Claude Sonnet 5.5 also default to high. PolicyBench's usage records count reasoning tokens for no Claude row, so the zero they hold for Claude Fable 5.2 means not counted. Sent to Anthropic's API directly, the board's prompt for the first household drew a thinking block that took 1,789 of its 2,881 output tokens. On six households the answer stopped at the row's starting budget of 16,384 output tokens, and PolicyBench asked again with twice the budget, as it does for every row that sets a budget, up to each row's ceiling. One of those second calls took 325 seconds, which the row's 600-second timeout allowed; the three other Claude rows that answer as JSON objects also have 600 seconds, against PolicyBench's default of 300 for Claude rows that reason at the provider default. At Anthropic's price of $8 per million input tokens and $40 per million output tokens, the 100 households cost $31.69, against $6.75 for Claude Opus 5.5 and $3.43 for Claude Sonnet 5.5. Claude Fable 5.1, the Fable model before it, gets 90.8% (#9) and cost $25.70.
PolicyBench also ran Claude Fable 5.2 and Claude Opus 5.5 each at the other's default effort, on the same 100 households, under a plan it wrote before the first call and publishes with the runs' times, and the runs change no board row. On the share of answers right, Claude Fable 5.2 at medium scores 1.6 points above its board row (95% interval +0.3 to +3.1), and Claude Opus 5.5 at high scores 0.2 points above its board row (95% interval -0.9 to +1.3). At medium, Claude Fable 5.2 scores 0.5 points below Claude Opus 5.5 (95% interval -1.5 to +0.5), and at high, Claude Fable 5.2 scores 2.3 points below Claude Opus 5.5 (95% interval -3.9 to -0.8). Each model ran once at each effort, and the intervals resample households, so they do not measure how far a repeated run would move; Claude Opus 5.5's board row ran on 2026-09-22, Claude Fable 5.2's on 2026-11-09 and both effort runs on 2026-11-12.
The release also stops scoring one output, the refundable state credits of a Minnesota household. Minnesota's renter's credit requires a Certificate of Rent Paid, which state law has the owner of a rented home give each renter. The prompt lists the household's rent and no certificate, and policyengine-us 2.60.0 counts an unlisted certificate as absent, so the reference is $0; with a certificate, the credit and the output are $1,921. The prompt does not say whether the household holds one, so PolicyBench stops scoring the output, under a ruling of November 1. Of the 52 models, 43 answer $0, eight give a positive amount, none of them $1,921, and one gives no answer. Every model is now scored on 1,934 of its 1,984 requested outputs; PolicyBench excludes the other 50, this one among them.
Against the previous release, the exclusion moves each of the 51 earlier models' shares of answers right by less than 0.01 points: the 42 that answered $0 fall, and the nine that did not rise. Claude Fable 5.2 also answers $0, so the exclusion removes one of its right answers, and it ranks #5 with or without the output scored. No earlier model passes another, and each of the 47 earlier models below Claude Fable 5.2 moves down one place.
The references stay on policyengine-us 2.60.0, and no excluded output returns to scoring. When PolicyBench checked PyPI on 2026-11-11, the newest policyengine-us release was 2.116.21, and it gives the same value as 2.60.0 for all 1,934 scored outputs; it moves three excluded outputs, which keep the values they were decided on. The check applied the previous release's exclusions, so its file counts 1,935 scored outputs, the Minnesota output among them.
Claude Opus 5.5 judged the misses in the 122 audit cases Claude Fable 5.2 joined, on 2026-11-11, and it flagged seven references as suspect. Of the seven, six already had decisions, which this release keeps: four are outputs PolicyBench excludes, and two are references it affirmed in earlier releases. The other is a Montana household's refundable state credits, which the judge expected to include a child tax credit. The 2023 bill that proposed that credit never became law, and the reference is Montana's earned income credit, 20% of the federal one for 2026, so PolicyBench affirmed it. On the federal income tax of a Virginia household and of a Texas household, the judge put eight models' answers on each in a class reserved for references that rest on law published after the freeze. Those answers apply the older rates and deductions that were due to return in 2026. A law of July 4, 2025, published before PolicyBench froze its references on 2026-07-03, made the current ones permanent, so PolicyBench classes the answers as model errors. All three outputs stay scored, and Claude Fable 5.2 misses each of them.
Readers running Claude Opus 5.5, the judge's model, then read the judge's new texts on the 122 cases Claude Fable 5.2 answered wrong, and readers running GPT-6.1 Sol read the texts on the 559 other audit cases, where Claude Fable 5.2 answered right. The first read raised 24 questions about a prompt or a reference, and the second raised 21. PolicyBench changes no reference and no scored output on those questions in this release, because the first read covered only Claude Fable 5.2's misses. The questions stay open for every model, and PolicyBench has set no date for deciding them. Across both reads and the errata check, the wording review record gives another value for 21 open questions; scoring their cells against that value would raise Claude Fable 5.2's share in seven of them, lower it in seven and leave it unchanged in seven. Together, the 21 values would raise Claude Fable 5.2's share; the 20 other than the one on auto loan interest would lower it. For example, the prompts that list auto loan interest do not say whether the loan qualifies for the federal deduction for qualified passenger vehicle loan interest, and the references treat it as not qualifying. Every prompt says to treat "any unlisted numeric input as 0 and any other unlisted household fact, boolean, or status input as false", and that rule gives the references' reading. With the interest counted as qualifying, policyengine-us 2.60.0 changes 11 scored outputs by more than $1, and across the 52 models, 81 answers on those outputs match the reference and 22 match the changed value. Claude Fable 5.2's answers match the changed value on seven of them, the most of any model, so scoring them against the changed values would raise its share more than any other model's. The audit gives the board with those outputs excluded and with them scored against the changed values.
Since the last release, PolicyBench has corrected eleven of its notes, from September 1 to October 31, where it defined the score by its $1 tolerance alone. PolicyBench applies the tolerance to dollar amounts and counts an eligibility answer as right only when it matches. No score moved.
The session that operated this release runs on Claude Fable 5.2, the model the release adds, and it also wrote the effort sensitivity's plan and started its runs; a Claude Opus 5.5 agent built the script that runs the release's steps. PolicyBench ruled on the Minnesota output on November 1, eight days before Anthropic released Claude Fable 5.2, and its three decisions on the judge's findings each keep a reference that Claude Fable 5.2's answer misses. PolicyBench's records show that reviewers running Claude Opus 5.5, an Anthropic model, reviewed the serving card, those decisions and the errata, reviewed the release script and the release itself in four rounds each, and adjudicated the second read's questions. GPT-6.1 Sol, an OpenAI model, also tried to refute each Claude Opus 5.5 adjudication of the first read's questions and, on November 12, reviewed a text bundle holding drafts of this note, the review record and the effort sensitivity; it asked for changes, PolicyBench made changes in response, and GPT-6.1 Sol has not read the published versions. No model from a lab other than Anthropic has checked the three decisions or the adjudications of the second read's questions, and no model has checked the second read's 86 findings of false or unsupported statements. The briefs for the second read and for the earlier adjudications said dollar answers had to match the reference to the cent, though PolicyBench counts an amount within $1 as right.
Data
- Dashboard data release
- Release design note
- Exclusion record
- Audit of the Minnesota renter's credit
- Developer adjudications
- Judge provenance for the re-judged cases
- Publication check against policyengine-us 2.116.21
- Errata for the judge's texts
- Wording review record
- Car-loan interest audit
- Claude effort sensitivity
- Claude Fable 5.2 model page
- Minnesota household
- Previous release note (October 31)