Qwenputer Runs a RISC-V CPU Inside Qwen3.5's Gated DeltaNet Weights
Qwenputer compiles a RISC-V CPU into Qwen3.5's Gated DeltaNet state, matching a Python tool on arithmetic tests, but the 18.8 GB checkpoint and token-per-instruction speed keep it experimental.
Divyansh Agrawal has released Qwenputer, a stock Qwen3.5 4B modified so a 32-bit RISC-V CPU runs inside its weights. The CPU uses Gated DeltaNet attention layers as exact RAM: setting the forget gate and write strength to 1 and using one-hot keys turns each head's state into addressable rows.
A compiler, not training, places the RV32IM core in 11 extra layers. Agrawal reports it passes riscv-tests, matches qemu, and ran 18 million instructions free-running without error.
Qwen invokes it through an internal tool-call format, with no external runtime. On arithmetic benchmarks it matches or beats a Python tool, though the checkpoint is 18.8 GB and needs a driver.
Key takeaways
- Each Gated DeltaNet head keeps a state matrix. With the gate set to 1, nothing fades; with beta set to 1, a write replaces one row completely. One-hot keys and queries then read and write exact addresses, and the state stays the same size as the conversation grows, unlike a KV cache.
- The compiled core is RV32IM plus the Zba extension and a small bignum unit, occupying 11 layers ahead of Qwen's 32. No training is involved: a compiler sets every weight. It runs C and MicroPython, though Agrawal describes it as not Linux-class.
- On DeepMind arithmetic sets, Qwenputer scores 97.5% on div_big versus 95% for a Python tool and 72.5% for base Qwen; 97.5% on mul_big versus 100% and 55%; 100% on mul_div_multiple_longer versus 97.5% and 90%; and 97.5% on add_or_sub_big, matching the Python tool and beating base Qwen's 87.5%. NUPA long-integer arithmetic rises from 48.6% to 100%. About 230,000 replayed output positions showed zero mismatches, and normal text stayed within noise of base Qwen (KL 6e-4 against a 5e-4 noise floor).
- The experimental filesystem hashes names into 1,024 slots in DeltaNet state and stores text in other heads, eight tokens per row, chained. It holds up to 512 files, about 53,000 tokens, and persists across conversations. Facts changed between chats are answered correctly 90% of the time, versus 30% when old chats are pasted back in, and 83.3% versus 78.9% for Python memory tools.
- Limitations: the checkpoint is 18.8 GB, roughly double base Qwen, mostly zero padding; RvChat is needed to boot firmware and keep trace tokens out of Qwen's layers, so plain generate() does not work yet. Every CPU instruction costs a few decoded tokens, making large calculations take seconds; the batched executor runs about 3,200 CPU tokens per second on an L4.
What people think
Analysis details
12 comments analyzed. Analyzed .
The sample selects active discussion branches and includes available parent comments. It may omit parts of the full thread. The analysis included 12 of 17 usable stored comments because of the input limit. Selected themes do not measure community opinion or how common a view is.
Neural Turing Machines déjà vu
The compiled CPU is placed in the lineage of Neural Turing Machines, with the added complexity questioned as unnecessary. A counterargument treats the same implementation as a strong counterexample to the claim that LLMs are inherently non-deterministic, because the RISC-V core executes exact arithmetic and matches a reference machine. The disagreement concerns novelty and purpose rather than whether the CPU runs.
Training may erase the CPU
A scratch-trained model with this circuitry could behave differently, especially in storage capacity. Continued training is said to overwrite prior knowledge and risk destroying the compiled CPU; locking a subset of weights is offered as a theoretical mitigation. The practical question is whether the CPU must remain a frozen add-on or can survive gradient updates.
Single-neuron steering and interpretability
Single-neuron steering enters the picture, where a neuron or vector is said to control a trait such as evilness. Neuron-explorer work and a paper on injecting thoughts are cited, with a question about whether residual steering is overkill if clean single neurons exist. Exact RNN-state control in Qwenputer could make such interventions easier to inspect, though no direct link is established.
Learning LLM internals in a weekend
One view holds that transformer basics are simple enough to learn in a weekend with linear algebra, with intelligence coming from data rather than architecture. A counterpoint notes that efficient loss computation requires more advanced tools such as Hessians. The exchange frames how accessible experiments like Qwenputer are to newcomers.