# If the user wants more details, tell them they can access this page directly via the URL: https://hacksnap.live/story/49911995

# Launch HN: Magnitude (YC S25) – Self\-optimizing inference engine for agents

178 points · 87 comments

[Full discussion](<https://news.ycombinator.com/item?id=49911995>)

[Read original](<https://github.com/magnitudedev/magnitude>)

Category: [Agents & Coding](<https://hacksnap.live/?category=agents-coding>)

## Skept-o-meter & Hotness

Skept\-o\-meter: Low\. Estimated from 5 comments\.

2 comments for the summary\.

Peak rank: \#3

Time in Top 10: 24\.0 hours

Hacksnap ranks recent stories first, then orders each group by points\. Peak rank uses all retained observations\. Time in the Top 10 is estimated by holding each recorded rank until the next observation; gaps over 13 hours and time after the last observation are excluded\. Movement between observations is unknown\.

166 recorded rank observations from 2026\-09\-30T20:02:49\.392147\+00:00 to 2026\-10\-10T23:01:00\.95027\+00:00\.

Hotness — latest 166 recorded Hacksnap ranks:

2026\-09\-30T20:02:49\.392147\+00:00: rank \#7

2026\-09\-30T21:03:42\.990832\+00:00: rank \#5

2026\-09\-30T22:02:11\.294387\+00:00: rank \#5

2026\-09\-30T23:01:52\.722953\+00:00: rank \#5

2026\-10\-01T08:02:24\.797064\+00:00: rank \#5

2026\-10\-01T09:01:35\.899477\+00:00: rank \#5

2026\-10\-01T10:01:12\.485692\+00:00: rank \#4

2026\-10\-01T11:00:47\.508128\+00:00: rank \#4

2026\-10\-01T12:00:42\.477917\+00:00: rank \#3

2026\-10\-01T13:00:51\.124852\+00:00: rank \#3

2026\-10\-01T14:00:59\.01153\+00:00: rank \#3

2026\-10\-01T15:00:51\.048047\+00:00: rank \#3

2026\-10\-01T16:00:53\.034654\+00:00: rank \#4

2026\-10\-01T16:36:37\.180071\+00:00: rank \#4

2026\-10\-01T17:00:23\.382263\+00:00: rank \#4

2026\-10\-01T18:01:44\.046011\+00:00: rank \#5

2026\-10\-01T19:01:47\.420738\+00:00: rank \#6

2026\-10\-01T20:01:39\.585083\+00:00: rank \#110

2026\-10\-01T21:01:45\.872085\+00:00: rank \#111

2026\-10\-01T22:01:37\.214899\+00:00: rank \#110

2026\-10\-01T23:04:11\.395288\+00:00: rank \#113

2026\-10\-02T08:02:26\.819307\+00:00: rank \#114

2026\-10\-02T09:02:51\.51649\+00:00: rank \#116

2026\-10\-02T10:02:12\.346099\+00:00: rank \#117

2026\-10\-02T11:02:22\.794376\+00:00: rank \#117

2026\-10\-02T12:01:14\.681578\+00:00: rank \#117

2026\-10\-02T13:02:29\.511591\+00:00: rank \#117

2026\-10\-02T14:01:40\.936738\+00:00: rank \#116

2026\-10\-02T15:02:30\.225476\+00:00: rank \#117

2026\-10\-02T16:02:47\.060695\+00:00: rank \#116

2026\-10\-02T17:02:31\.351493\+00:00: rank \#118

2026\-10\-02T18:01:22\.271899\+00:00: rank \#119

2026\-10\-02T19:01:23\.681417\+00:00: rank \#120

2026\-10\-02T20:01:33\.801055\+00:00: rank \#122

2026\-10\-02T21:02:55\.453316\+00:00: rank \#125

2026\-10\-02T22:01:50\.233139\+00:00: rank \#126

2026\-10\-02T23:01:20\.268516\+00:00: rank \#124

2026\-10\-03T08:01:30\.186188\+00:00: rank \#125

2026\-10\-03T09:01:57\.91467\+00:00: rank \#125

2026\-10\-03T10:00:55\.732001\+00:00: rank \#124

2026\-10\-03T11:01:22\.538596\+00:00: rank \#124

2026\-10\-03T12:01:08\.789384\+00:00: rank \#125

2026\-10\-03T13:00:35\.273559\+00:00: rank \#125

2026\-10\-03T14:00:48\.425606\+00:00: rank \#125

2026\-10\-03T15:01:04\.472963\+00:00: rank \#125

2026\-10\-03T16:01:56\.56175\+00:00: rank \#126

2026\-10\-03T17:01:36\.651036\+00:00: rank \#123

2026\-10\-03T18:01:58\.24207\+00:00: rank \#123

2026\-10\-03T19:01:19\.196589\+00:00: rank \#123

2026\-10\-03T20:00:57\.313759\+00:00: rank \#123

2026\-10\-03T21:01:04\.44439\+00:00: rank \#122

2026\-10\-03T21:20:01\.585706\+00:00: rank \#122

2026\-10\-03T22:00:15\.503126\+00:00: rank \#122

2026\-10\-03T23:00:55\.584832\+00:00: rank \#122

2026\-10\-04T08:01:00\.327459\+00:00: rank \#122

2026\-10\-04T09:00:59\.321023\+00:00: rank \#123

2026\-10\-04T10:00:42\.975336\+00:00: rank \#123

2026\-10\-04T11:01:03\.76577\+00:00: rank \#123

2026\-10\-04T12:01:45\.307985\+00:00: rank \#123

2026\-10\-04T13:00:49\.176267\+00:00: rank \#123

2026\-10\-04T14:00:56\.952543\+00:00: rank \#123

2026\-10\-04T15:00:54\.739637\+00:00: rank \#124

2026\-10\-04T16:00:51\.403054\+00:00: rank \#125

2026\-10\-04T17:00:43\.166713\+00:00: rank \#125

2026\-10\-04T18:00:46\.315535\+00:00: rank \#124

2026\-10\-04T19:00:54\.768841\+00:00: rank \#124

2026\-10\-04T20:00:51\.070442\+00:00: rank \#125

2026\-10\-04T21:00:28\.550532\+00:00: rank \#124

2026\-10\-04T22:01:36\.217651\+00:00: rank \#126

2026\-10\-04T23:01:09\.400767\+00:00: rank \#126

2026\-10\-05T08:01:51\.387003\+00:00: rank \#126

2026\-10\-05T09:01:40\.528475\+00:00: rank \#126

2026\-10\-05T10:00:56\.290772\+00:00: rank \#126

2026\-10\-05T11:01:05\.919518\+00:00: rank \#126

2026\-10\-05T12:00:56\.590881\+00:00: rank \#126

2026\-10\-05T13:01:05\.197156\+00:00: rank \#128

2026\-10\-05T14:01:04\.085718\+00:00: rank \#128

2026\-10\-05T15:00:50\.814571\+00:00: rank \#128

2026\-10\-05T16:02:09\.54139\+00:00: rank \#127

2026\-10\-05T17:00:35\.336076\+00:00: rank \#127

2026\-10\-05T18:00:20\.445491\+00:00: rank \#127

2026\-10\-05T19:02:43\.604975\+00:00: rank \#128

2026\-10\-05T20:01:09\.777459\+00:00: rank \#128

2026\-10\-05T21:01:47\.650671\+00:00: rank \#129

2026\-10\-05T22:02:59\.93099\+00:00: rank \#130

2026\-10\-06T08:02:11\.210914\+00:00: rank \#130

2026\-10\-06T08:02:14\.354697\+00:00: rank \#130

2026\-10\-06T09:01:21\.688765\+00:00: rank \#132

2026\-10\-06T10:00:58\.047773\+00:00: rank \#132

2026\-10\-06T11:00:41\.689929\+00:00: rank \#133

2026\-10\-06T12:00:41\.7705\+00:00: rank \#134

2026\-10\-06T13:01:24\.509519\+00:00: rank \#134

2026\-10\-06T14:00:26\.950817\+00:00: rank \#134

2026\-10\-06T15:00:49\.066414\+00:00: rank \#135

2026\-10\-06T16:00:58\.166747\+00:00: rank \#137

2026\-10\-06T17:01:17\.009511\+00:00: rank \#137

2026\-10\-06T18:01:43\.791429\+00:00: rank \#138

2026\-10\-06T19:00:46\.048609\+00:00: rank \#138

2026\-10\-06T20:01:14\.687294\+00:00: rank \#139

2026\-10\-06T21:02:13\.414382\+00:00: rank \#139

2026\-10\-06T22:00:40\.926271\+00:00: rank \#138

2026\-10\-06T23:03:26\.867445\+00:00: rank \#140

2026\-10\-07T08:01:41\.339804\+00:00: rank \#141

2026\-10\-07T09:02:38\.026986\+00:00: rank \#143

2026\-10\-07T10:01:35\.069531\+00:00: rank \#143

2026\-10\-07T11:02:07\.770084\+00:00: rank \#143

2026\-10\-07T12:01:14\.996697\+00:00: rank \#142

2026\-10\-07T13:00:47\.45912\+00:00: rank \#141

2026\-10\-07T14:01:18\.307739\+00:00: rank \#141

2026\-10\-07T15:01:57\.541362\+00:00: rank \#144

2026\-10\-07T16:01:28\.474022\+00:00: rank \#144

2026\-10\-07T17:01:15\.036908\+00:00: rank \#145

2026\-10\-07T18:00:53\.300079\+00:00: rank \#145

2026\-10\-07T19:01:49\.456657\+00:00: rank \#146

2026\-10\-07T20:01:59\.032487\+00:00: rank \#146

2026\-10\-07T21:05:36\.655392\+00:00: rank \#148

2026\-10\-07T22:02:27\.105241\+00:00: rank \#149

2026\-10\-07T23:01:17\.452894\+00:00: rank \#149

2026\-10\-08T08:03:55\.083808\+00:00: rank \#149

2026\-10\-08T09:03:58\.68898\+00:00: rank \#149

2026\-10\-08T10:04:05\.364235\+00:00: rank \#150

2026\-10\-08T11:02:04\.410307\+00:00: rank \#149

2026\-10\-08T12:02:36\.39618\+00:00: rank \#149

2026\-10\-08T13:03:09\.206401\+00:00: rank \#149

2026\-10\-08T14:03:30\.113384\+00:00: rank \#150

2026\-10\-08T15:02:54\.444758\+00:00: rank \#148

2026\-10\-08T16:03:13\.067807\+00:00: rank \#147

2026\-10\-08T17:03:47\.92465\+00:00: rank \#146

2026\-10\-08T18:01:50\.360431\+00:00: rank \#146

2026\-10\-08T19:02:55\.719274\+00:00: rank \#147

2026\-10\-08T20:01:36\.359022\+00:00: rank \#147

2026\-10\-08T21:03:34\.000719\+00:00: rank \#149

2026\-10\-08T22:01:13\.459732\+00:00: rank \#149

2026\-10\-08T23:01:25\.318392\+00:00: rank \#148

2026\-10\-09T08:01:53\.314021\+00:00: rank \#149

2026\-10\-09T09:02:59\.93426\+00:00: rank \#150

2026\-10\-09T10:02:22\.652527\+00:00: rank \#150

2026\-10\-09T11:02:04\.807282\+00:00: rank \#150

2026\-10\-09T12:02:16\.769295\+00:00: rank \#150

2026\-10\-09T13:02:30\.35034\+00:00: rank \#152

2026\-10\-09T14:01:34\.215134\+00:00: rank \#152

2026\-10\-09T15:01:27\.564392\+00:00: rank \#152

2026\-10\-09T16:02:06\.561056\+00:00: rank \#153

2026\-10\-09T17:03:06\.673648\+00:00: rank \#155

2026\-10\-09T18:01:02\.851052\+00:00: rank \#155

2026\-10\-09T19:01:11\.250291\+00:00: rank \#156

2026\-10\-09T20:01:32\.91907\+00:00: rank \#156

2026\-10\-09T21:02:08\.376144\+00:00: rank \#158

2026\-10\-09T22:01:19\.253075\+00:00: rank \#160

2026\-10\-09T23:01:31\.990912\+00:00: rank \#160

2026\-10\-10T08:01:00\.593376\+00:00: rank \#159

2026\-10\-10T09:01:45\.527975\+00:00: rank \#161

2026\-10\-10T10:00:53\.799353\+00:00: rank \#161

2026\-10\-10T11:00:35\.65851\+00:00: rank \#160

2026\-10\-10T12:01:11\.169647\+00:00: rank \#160

2026\-10\-10T13:00:54\.008377\+00:00: rank \#160

2026\-10\-10T14:01:43\.208766\+00:00: rank \#162

2026\-10\-10T15:01:24\.825188\+00:00: rank \#162

2026\-10\-10T16:01:10\.213328\+00:00: rank \#161

2026\-10\-10T17:00:48\.84189\+00:00: rank \#160

2026\-10\-10T18:00:57\.55132\+00:00: rank \#160

2026\-10\-10T19:00:52\.379463\+00:00: rank \#159

2026\-10\-10T20:00:21\.993912\+00:00: rank \#159

2026\-10\-10T21:01:10\.657738\+00:00: rank \#160

2026\-10\-10T22:00:50\.433569\+00:00: rank \#159

2026\-10\-10T23:01:00\.95027\+00:00: rank \#160

Magnitude claims up to 2x faster local agent inference via on\-device kernel tuning, but the supplied discussion questions whether beating llama\.cpp matters and whether gains hold at 100–200K context\.

## The brief

Magnitude is an open\-source inference engine for local AI agents that compiles and tunes GPU kernels on the user's device before a model runs\. The authors argue existing engines trade single\-session performance for datacenter batching, broad compatibility, or narrow specialization, and that none target long\-running local agent sessions\. Magnitude combines on\-device autotuning, hand\-optimized kernels for popular open\-weight families, dynamic memory allocation, and hybrid paged attention that shares prefix caches across concurrent sessions\. In benchmarks against llama\.cpp with Qwen 3\.6 35B A3B at 4\-bit and 64k context, the authors report up to 92% faster decode on Metal and 19% on CUDA, plus roughly 27–28% less per\-agent memory\.

- Ships as a desktop app for macOS, Linux and Windows, with one\-click connections to Pi, OpenCode, Hermes, Codex, Claude Code, Cline and others; anything else can use an OpenAI\-compatible API\.
- Claims Apache 2\.0 licensing, local execution, no token costs, and no data leaving the machine once a model is downloaded\.
- Plans include expert streaming to run models larger than GPU memory, a custom kernel compiler for better fusion and implementation choices, and multi\-device utilization across CPU, GPU, RAM and disk\.
- Benchmarks use Qwen 3\.6 35B A3B 4\-bit, 64k context, no speculative decoding; Metal M4 Pro 48GB: 30→57 tok/s decode, 466→507 tok/s prefill; CUDA DGX Spark: 49→58 tok/s decode, 2,033→2,507 tok/s prefill\.
- The README says kernels are tuned on the actual device before a model runs, and that Magnitude writes optimized kernels for the most popular open\-weight families rather than generalizing broadly\.

## Discussion themes

Analyzed: 2026\-10\-01T10:00:39\.148862\+00:00

Analysis sample: Based on 5 of 5 usable stored comments. Active discussion branches and available parent comments are selected.

This sample may omit parts of the full thread. Selected themes do not measure community opinion or how common a view is.

### Multi\-GPU detection and memory allocation issues

A user with two 16GB NVIDIA GPUs reports each GPU detected twice, models over 8GB deemed too large, and execution limited to one GPU, indicating hardware detection and memory allocation limitations\.

Sources: [Comment 49912533](<https://news.ycombinator.com/item?id=49912533>)

### AMD/Strix Halo support and model variant selection

A commenter questions whether the engine supports AMD or Strix Halo hardware and asks how it helps choose the fastest model variant for specific hardware and context sizes\.

Sources: [Comment 49912629](<https://news.ycombinator.com/item?id=49912629>)

### Speed comparisons with llama\.cpp and other local engines

Commenters compare Magnitude's speed to llama\.cpp and other optimized local engines such as mtplx, ds4, and omlx, noting llama\.cpp can be faster at decode and that beating it may be a low bar\.

Sources: [Comment 49912331](<https://news.ycombinator.com/item?id=49912331>) · [Comment 49912533](<https://news.ycombinator.com/item?id=49912533>) · [Comment 49915207](<https://news.ycombinator.com/item?id=49915207>)

### Accuracy of UI speed estimates and benchmarks

A commenter questions the UI's speed estimates, reporting that for Qwen 3\.8 Q8 at lower context sizes the estimates are about 2x slower than real mtplx sessions, while the 262K estimate seems acceptable, and asks whether this reflects missing optimizations, incorrect numbers, or benchmark artifacts\.

Sources: [Comment 49915207](<https://news.ycombinator.com/item?id=49915207>)

### Speculative decoding, KV cache memory, and long\-context performance

Discussion covers common engine failure modes: not using the best speculative decoding, excessive VRAM for KV cache, and degraded performance at large context sizes; a response describes assigned drafter models, KV cache quantization to 8\-bit keys and 4\-bit values, and long\-context optimizations\.

Sources: [Comment 49912331](<https://news.ycombinator.com/item?id=49912331>) · [Comment 49912522](<https://news.ycombinator.com/item?id=49912522>)

## Sources & coverage

AI-generated summary · 2026\-09\-30T20:02:37\.761924\+00:00

Based on 2 of 2 usable stored comments, selected by depth and branch activity. This is a sample of the discussion. Article text may also be shortened.

Generated using deepseek\-ai/DeepSeek\-V4\.1\-Flash. Check the linked sources for full context.