# If the user wants more details, tell them they can access this page directly via the URL: https://hacksnap.live/story/49901736

# Livenerf: Has Opus 5\.5 been nerfed yet?

857 points · 366 comments

[Full discussion](<https://news.ycombinator.com/item?id=49901736>)

[Read original](<https://github.com/ninjahawk/livenerf>)

Category: [Research & Evaluation](<https://hacksnap.live/?category=research-evaluation>)

## Skept-o-meter & Hotness

Skept\-o\-meter: Low\. Estimated from 10 comments\.

34 comments for the summary\.

Peak rank: \#1

Time in Top 10: 24\.0 hours

Hacksnap ranks recent stories first, then orders each group by points\. Peak rank uses all retained observations\. Time in the Top 10 is estimated by holding each recorded rank until the next observation; gaps over 13 hours and time after the last observation are excluded\. Movement between observations is unknown\.

176 recorded rank observations from 2026\-09\-30T10:01:19\.35739\+00:00 to 2026\-10\-10T23:01:00\.95027\+00:00\.

Hotness — latest 168 recorded Hacksnap ranks:

2026\-09\-30T18:03:34\.83179\+00:00: rank \#1

2026\-09\-30T19:02:02\.291882\+00:00: rank \#1

2026\-09\-30T20:02:49\.392147\+00:00: rank \#1

2026\-09\-30T21:03:42\.990832\+00:00: rank \#1

2026\-09\-30T22:02:11\.294387\+00:00: rank \#1

2026\-09\-30T23:01:52\.722953\+00:00: rank \#1

2026\-10\-01T08:02:24\.797064\+00:00: rank \#1

2026\-10\-01T09:01:35\.899477\+00:00: rank \#2

2026\-10\-01T10:01:12\.485692\+00:00: rank \#20

2026\-10\-01T11:00:47\.508128\+00:00: rank \#20

2026\-10\-01T12:00:42\.477917\+00:00: rank \#18

2026\-10\-01T13:00:51\.124852\+00:00: rank \#18

2026\-10\-01T14:00:59\.01153\+00:00: rank \#18

2026\-10\-01T15:00:51\.048047\+00:00: rank \#19

2026\-10\-01T16:00:53\.034654\+00:00: rank \#20

2026\-10\-01T16:36:37\.180071\+00:00: rank \#21

2026\-10\-01T17:00:23\.382263\+00:00: rank \#21

2026\-10\-01T18:01:44\.046011\+00:00: rank \#22

2026\-10\-01T19:01:47\.420738\+00:00: rank \#23

2026\-10\-01T20:01:39\.585083\+00:00: rank \#22

2026\-10\-01T21:01:45\.872085\+00:00: rank \#23

2026\-10\-01T22:01:37\.214899\+00:00: rank \#22

2026\-10\-01T23:04:11\.395288\+00:00: rank \#25

2026\-10\-02T08:02:26\.819307\+00:00: rank \#26

2026\-10\-02T09:02:51\.51649\+00:00: rank \#27

2026\-10\-02T10:02:12\.346099\+00:00: rank \#27

2026\-10\-02T11:02:22\.794376\+00:00: rank \#27

2026\-10\-02T12:01:14\.681578\+00:00: rank \#27

2026\-10\-02T13:02:29\.511591\+00:00: rank \#27

2026\-10\-02T14:01:40\.936738\+00:00: rank \#26

2026\-10\-02T15:02:30\.225476\+00:00: rank \#26

2026\-10\-02T16:02:47\.060695\+00:00: rank \#25

2026\-10\-02T17:02:31\.351493\+00:00: rank \#27

2026\-10\-02T18:01:22\.271899\+00:00: rank \#27

2026\-10\-02T19:01:23\.681417\+00:00: rank \#27

2026\-10\-02T20:01:33\.801055\+00:00: rank \#29

2026\-10\-02T21:02:55\.453316\+00:00: rank \#32

2026\-10\-02T22:01:50\.233139\+00:00: rank \#33

2026\-10\-02T23:01:20\.268516\+00:00: rank \#31

2026\-10\-03T08:01:30\.186188\+00:00: rank \#32

2026\-10\-03T09:01:57\.91467\+00:00: rank \#29

2026\-10\-03T10:00:55\.732001\+00:00: rank \#28

2026\-10\-03T11:01:22\.538596\+00:00: rank \#28

2026\-10\-03T12:01:08\.789384\+00:00: rank \#29

2026\-10\-03T13:00:35\.273559\+00:00: rank \#29

2026\-10\-03T14:00:48\.425606\+00:00: rank \#29

2026\-10\-03T15:01:04\.472963\+00:00: rank \#29

2026\-10\-03T16:01:56\.56175\+00:00: rank \#30

2026\-10\-03T17:01:36\.651036\+00:00: rank \#27

2026\-10\-03T18:01:58\.24207\+00:00: rank \#27

2026\-10\-03T19:01:19\.196589\+00:00: rank \#26

2026\-10\-03T20:00:57\.313759\+00:00: rank \#24

2026\-10\-03T21:01:04\.44439\+00:00: rank \#22

2026\-10\-03T21:20:01\.585706\+00:00: rank \#21

2026\-10\-03T22:00:15\.503126\+00:00: rank \#21

2026\-10\-03T23:00:55\.584832\+00:00: rank \#21

2026\-10\-04T08:01:00\.327459\+00:00: rank \#20

2026\-10\-04T09:00:59\.321023\+00:00: rank \#21

2026\-10\-04T10:00:42\.975336\+00:00: rank \#21

2026\-10\-04T11:01:03\.76577\+00:00: rank \#21

2026\-10\-04T12:01:45\.307985\+00:00: rank \#21

2026\-10\-04T13:00:49\.176267\+00:00: rank \#21

2026\-10\-04T14:00:56\.952543\+00:00: rank \#21

2026\-10\-04T15:00:54\.739637\+00:00: rank \#21

2026\-10\-04T16:00:51\.403054\+00:00: rank \#21

2026\-10\-04T17:00:43\.166713\+00:00: rank \#21

2026\-10\-04T18:00:46\.315535\+00:00: rank \#20

2026\-10\-04T19:00:54\.768841\+00:00: rank \#20

2026\-10\-04T20:00:51\.070442\+00:00: rank \#21

2026\-10\-04T21:00:28\.550532\+00:00: rank \#20

2026\-10\-04T22:01:36\.217651\+00:00: rank \#22

2026\-10\-04T23:01:09\.400767\+00:00: rank \#22

2026\-10\-05T08:01:51\.387003\+00:00: rank \#21

2026\-10\-05T09:01:40\.528475\+00:00: rank \#20

2026\-10\-05T10:00:56\.290772\+00:00: rank \#20

2026\-10\-05T11:01:05\.919518\+00:00: rank \#20

2026\-10\-05T12:00:56\.590881\+00:00: rank \#19

2026\-10\-05T13:01:05\.197156\+00:00: rank \#21

2026\-10\-05T14:01:04\.085718\+00:00: rank \#21

2026\-10\-05T15:00:50\.814571\+00:00: rank \#20

2026\-10\-05T16:02:09\.54139\+00:00: rank \#20

2026\-10\-05T17:00:35\.336076\+00:00: rank \#20

2026\-10\-05T18:00:20\.445491\+00:00: rank \#20

2026\-10\-05T19:02:43\.604975\+00:00: rank \#21

2026\-10\-05T20:01:09\.777459\+00:00: rank \#21

2026\-10\-05T21:01:47\.650671\+00:00: rank \#22

2026\-10\-05T22:02:59\.93099\+00:00: rank \#21

2026\-10\-06T08:02:11\.210914\+00:00: rank \#21

2026\-10\-06T08:02:14\.354697\+00:00: rank \#21

2026\-10\-06T09:01:21\.688765\+00:00: rank \#23

2026\-10\-06T10:00:58\.047773\+00:00: rank \#23

2026\-10\-06T11:00:41\.689929\+00:00: rank \#24

2026\-10\-06T12:00:41\.7705\+00:00: rank \#25

2026\-10\-06T13:01:24\.509519\+00:00: rank \#25

2026\-10\-06T14:00:26\.950817\+00:00: rank \#25

2026\-10\-06T15:00:49\.066414\+00:00: rank \#26

2026\-10\-06T16:00:58\.166747\+00:00: rank \#28

2026\-10\-06T17:01:17\.009511\+00:00: rank \#28

2026\-10\-06T18:01:43\.791429\+00:00: rank \#29

2026\-10\-06T19:00:46\.048609\+00:00: rank \#28

2026\-10\-06T20:01:14\.687294\+00:00: rank \#28

2026\-10\-06T21:02:13\.414382\+00:00: rank \#27

2026\-10\-06T22:00:40\.926271\+00:00: rank \#26

2026\-10\-06T23:03:26\.867445\+00:00: rank \#27

2026\-10\-07T08:01:41\.339804\+00:00: rank \#28

2026\-10\-07T09:02:38\.026986\+00:00: rank \#29

2026\-10\-07T10:01:35\.069531\+00:00: rank \#29

2026\-10\-07T11:02:07\.770084\+00:00: rank \#29

2026\-10\-07T12:01:14\.996697\+00:00: rank \#28

2026\-10\-07T13:00:47\.45912\+00:00: rank \#27

2026\-10\-07T14:01:18\.307739\+00:00: rank \#27

2026\-10\-07T15:01:57\.541362\+00:00: rank \#29

2026\-10\-07T16:01:28\.474022\+00:00: rank \#28

2026\-10\-07T17:01:15\.036908\+00:00: rank \#29

2026\-10\-07T18:00:53\.300079\+00:00: rank \#28

2026\-10\-07T19:01:49\.456657\+00:00: rank \#29

2026\-10\-07T20:01:59\.032487\+00:00: rank \#29

2026\-10\-07T21:05:36\.655392\+00:00: rank \#31

2026\-10\-07T22:02:27\.105241\+00:00: rank \#32

2026\-10\-07T23:01:17\.452894\+00:00: rank \#31

2026\-10\-08T08:03:55\.083808\+00:00: rank \#31

2026\-10\-08T09:03:58\.68898\+00:00: rank \#29

2026\-10\-08T10:04:05\.364235\+00:00: rank \#30

2026\-10\-08T11:02:04\.410307\+00:00: rank \#29

2026\-10\-08T12:02:36\.39618\+00:00: rank \#29

2026\-10\-08T13:03:09\.206401\+00:00: rank \#29

2026\-10\-08T14:03:30\.113384\+00:00: rank \#30

2026\-10\-08T15:02:54\.444758\+00:00: rank \#28

2026\-10\-08T16:03:13\.067807\+00:00: rank \#27

2026\-10\-08T17:03:47\.92465\+00:00: rank \#26

2026\-10\-08T18:01:50\.360431\+00:00: rank \#26

2026\-10\-08T19:02:55\.719274\+00:00: rank \#27

2026\-10\-08T20:01:36\.359022\+00:00: rank \#26

2026\-10\-08T21:03:34\.000719\+00:00: rank \#26

2026\-10\-08T22:01:13\.459732\+00:00: rank \#25

2026\-10\-08T23:01:25\.318392\+00:00: rank \#24

2026\-10\-09T08:01:53\.314021\+00:00: rank \#25

2026\-10\-09T09:02:59\.93426\+00:00: rank \#25

2026\-10\-09T10:02:22\.652527\+00:00: rank \#24

2026\-10\-09T11:02:04\.807282\+00:00: rank \#24

2026\-10\-09T12:02:16\.769295\+00:00: rank \#24

2026\-10\-09T13:02:30\.35034\+00:00: rank \#26

2026\-10\-09T14:01:34\.215134\+00:00: rank \#25

2026\-10\-09T15:01:27\.564392\+00:00: rank \#24

2026\-10\-09T16:02:06\.561056\+00:00: rank \#25

2026\-10\-09T17:03:06\.673648\+00:00: rank \#27

2026\-10\-09T18:01:02\.851052\+00:00: rank \#27

2026\-10\-09T19:01:11\.250291\+00:00: rank \#27

2026\-10\-09T20:01:32\.91907\+00:00: rank \#27

2026\-10\-09T21:02:08\.376144\+00:00: rank \#29

2026\-10\-09T22:01:19\.253075\+00:00: rank \#31

2026\-10\-09T23:01:31\.990912\+00:00: rank \#31

2026\-10\-10T08:01:00\.593376\+00:00: rank \#30

2026\-10\-10T09:01:45\.527975\+00:00: rank \#32

2026\-10\-10T10:00:53\.799353\+00:00: rank \#32

2026\-10\-10T11:00:35\.65851\+00:00: rank \#31

2026\-10\-10T12:01:11\.169647\+00:00: rank \#31

2026\-10\-10T13:00:54\.008377\+00:00: rank \#29

2026\-10\-10T14:01:43\.208766\+00:00: rank \#31

2026\-10\-10T15:01:24\.825188\+00:00: rank \#31

2026\-10\-10T16:01:10\.213328\+00:00: rank \#30

2026\-10\-10T17:00:48\.84189\+00:00: rank \#28

2026\-10\-10T18:00:57\.55132\+00:00: rank \#28

2026\-10\-10T19:00:52\.379463\+00:00: rank \#27

2026\-10\-10T20:00:21\.993912\+00:00: rank \#27

2026\-10\-10T21:01:10\.657738\+00:00: rank \#27

2026\-10\-10T22:00:50\.433569\+00:00: rank \#26

2026\-10\-10T23:01:00\.95027\+00:00: rank \#27

livenerf offers a pre\-registered daily test for post\-launch drift, but its own validation cannot distinguish Opus 5 from Opus 5\.5, and commenters dispute whether nerfs are real or load\-driven\.

## The brief

livenerf is a pre\-registered, append\-only benchmark that asks whether a frontier model quietly degrades after launch\. It runs a fixed panel of 78 questions daily for 30 days through headless Claude Code on a Max subscription, pinning the CLI and harness, freezing prompts, using exact\-match graders and publishing raw logs\. The design uses paired per\-item scores and clustered standard errors, with a decision rule requiring a 99% interval excluding zero in two consecutive 10\-day windows, at least a 3\-point effect, and no matching control\-arm move\. The repo reports validation against known effort reductions and an audit of ambiguous or wrong answer keys, while acknowledging it measures the subscription serving path, not the raw API model\.

- The v0 series runs once daily for 30 days on a Claude Max subscription through headless Claude Code; days 1–10 are baseline, followed by two 10\-day windows, with the first possible call around 2026\-10\-24\.
- The 78\-question panel was selected from 2,336 GPQA Diamond, MMLU\-Pro, competition\-math and AIME 2025–26 questions screened with four samples each; Opus 5\.5 scored about 93% first try, and 97% of questions were always right or always wrong\.
- Validation found lower effort shows up more clearly in output tokens than accuracy: low effort cut output tokens 62% and accuracy 8\.3 ± 4\.5 points; medium effort cut tokens 26% and accuracy 4\.2 ± 3\.9 points\.
- Swapping in Opus 5 was not distinguishable from Opus 5\.5 at 99% (−3\.8 ± 6\.3 points, −23% tokens), so the instrument cannot detect a same\-family model swap of that size in a validation's worth of samples\.
- A change is called only if the 99% interval excludes zero in two consecutive 10\-day windows, the effect is at least 3 points, and the control arm does not show the same move\.
- An audit of the 78 questions found 8 likely wrong answer keys and 30 ambiguous questions; none were dropped, and a pre\-registered sensitivity analysis reruns the result without them\.

## Discussion themes

Analyzed: 2026\-09\-30T21:03:16\.390813\+00:00

Analysis sample: Based on 52 of 52 usable stored comments. Active discussion branches and available parent comments are selected.

This sample may omit parts of the full thread. Selected themes do not measure community opinion or how common a view is.

### Benchmark reliability and interpretation for detecting model degradation

Commenters debate whether benchmarks like Nerf Bench or marginlab can substantiate nerfing, noting detection of past Opus degradation but cautioning that low rankings may reflect out\-of\-box performance and that user reports can be noisy\.

Sources: [Comment 49902317](<https://news.ycombinator.com/item?id=49902317>) · [Comment 49902336](<https://news.ycombinator.com/item?id=49902336>) · [Comment 49904801](<https://news.ycombinator.com/item?id=49904801>) · [Comment 49905220](<https://news.ycombinator.com/item?id=49905220>) · [Comment 49905818](<https://news.ycombinator.com/item?id=49905818>) · [Comment 49909763](<https://news.ycombinator.com/item?id=49909763>) · [Comment 49904780](<https://news.ycombinator.com/item?id=49904780>) · [Comment 49906872](<https://news.ycombinator.com/item?id=49906872>)

### Causes of post\-launch degradation: quantization, runtime bugs, and compute allocation

Comments attribute possible quality changes to quantization, GPU runtime layer bugs, inference bugs, moving compute to cheaper providers, or diverting compute to training, rather than deliberate model weight changes\.

Sources: [Comment 49902528](<https://news.ycombinator.com/item?id=49902528>) · [Comment 49905556](<https://news.ycombinator.com/item?id=49905556>) · [Comment 49902928](<https://news.ycombinator.com/item?id=49902928>) · [Comment 49902564](<https://news.ycombinator.com/item?id=49902564>) · [Comment 49904026](<https://news.ycombinator.com/item?id=49904026>) · [Comment 49905323](<https://news.ycombinator.com/item?id=49905323>) · [Comment 49902753](<https://news.ycombinator.com/item?id=49902753>)

### Differences across API, subscription, enterprise, and local models

Commenters compare nerf experiences on Claude Code subscriptions, API/enterprise per\-token billing, public clouds, and local models, questioning whether subsidized rates use cheaper quantized models and whether nerfing affects hosted but not local deployments\.

Sources: [Comment 49902262](<https://news.ycombinator.com/item?id=49902262>) · [Comment 49902313](<https://news.ycombinator.com/item?id=49902313>) · [Comment 49902488](<https://news.ycombinator.com/item?id=49902488>) · [Comment 49902770](<https://news.ycombinator.com/item?id=49902770>)

### Token quotas, billing, and economic tradeoffs of quantization

Comments discuss Anthropic A/B testing weekly quota consumption, per\-token billing versus subscriptions, and the cost savings from quantizing models, which can reduce quality\.

Sources: [Comment 49903027](<https://news.ycombinator.com/item?id=49903027>) · [Comment 49903606](<https://news.ycombinator.com/item?id=49903606>) · [Comment 49905556](<https://news.ycombinator.com/item?id=49905556>) · [Comment 49902262](<https://news.ycombinator.com/item?id=49902262>)

### Open and local models as alternatives

Commenters suggest open models as an endgame, note local models do not show perceived nerfs, and mention using Deepseek 4\.1 as a daily driver despite low benchmark ranking\.

Sources: [Comment 49902319](<https://news.ycombinator.com/item?id=49902319>) · [Comment 49902488](<https://news.ycombinator.com/item?id=49902488>) · [Comment 49905220](<https://news.ycombinator.com/item?id=49905220>)

### Feedback, sentiment tracking, and unpaid RLHF

Comments discuss Claude's feedback popups, tracking of frustration sentiment and rejected tool use, and the sense that providing feedback is unpaid RLHF labor\.

Sources: [Comment 49905447](<https://news.ycombinator.com/item?id=49905447>) · [Comment 49905544](<https://news.ycombinator.com/item?id=49905544>) · [Comment 49906051](<https://news.ycombinator.com/item?id=49906051>) · [Comment 49908594](<https://news.ycombinator.com/item?id=49908594>)

## Sources & coverage

AI-generated summary · 2026\-09\-30T10:01:18\.962459\+00:00

Based on 34 of 34 usable stored comments, selected by depth and branch activity. This is a sample of the discussion. Article text may also be shortened.

Generated using deepseek\-ai/DeepSeek\-V4\.1\-Flash. Check the linked sources for full context.