Show HN: Cactus Needle 3: 8-29MB automation models can match DeepSeek V4 Flash
Needle 3's core pitch is a laddered, non-chat automation model for constrained hardware, but the discussion shows that reliability on ambiguous natural-language commands and clear anti-use cases will determine whether…
Needle 3's core pitch is a laddered, non-chat automation model for constrained hardware, but the discussion shows that reliability on ambiguous natural-language commands and clear anti-use cases will determine whether its small-model claims are trusted.
Cactus Compute's Show HN introduces Needle 3, a non-chat automation model for tiny devices that handles tool calls, structured extraction, and embeddings. It uses one set of weights deployable as 2-20 layer subnetworks, ships as 8-29 MB CQ2 binaries, and claims fine-tuned small subnetworks can match much larger models on narrow tasks.
Key takeaways
- Needle 3 targets tool calls, structured JSON extraction, and text embeddings rather than open-ended chat; it returns an empty tool list when no declared tool fits.
- One weight set supports subnetworks from 2 to 20 layers (25-121M parameters), shipping as 9-29 MB CQ2-bit binaries; claimed Raspberry Pi 5 speeds are 400-4k decode tokens/s and 1-10k prefill tokens/s.
- The article claims a 4-layer subnetwork fine-tuned on a narrow downstream task can match DeepSeek V4 Flash after one epoch, and that fine-tuning on DroidCall lifts subnetworks by 18-36 points.
- Benchmarks claim the 20-layer model scores 86.0 on Mobile Actions through the 2-bit binary, versus LFM2.5 1.2B at 82.4, Qwen3.5 0.8B at 76.0, and Apple's on-device model at 57.6.
- Architecture details include Laddered Simple Attention Networks, Monarch Hadamard MLP, engram fusion, GQA attention, confidence gating, regex triggers, multilingual support, and prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, WASM, and WASI.
What people think
Practical niche and anti-use cases questioned
hirako2000 warned that dubious 'tiny model beats LLMs' claims can obscure real innovation and asked for clear anti-use cases. Tsarp asked for real use cases beyond fictional ones and noted local 8-27B models for privacy. HenryNdubuaku replied that Needle targets hardware that cannot hold 8-27B, such as ARMv7, MIPS32, RISC-V, and watches, and that availability and latency matter more than cost.
Demo misrouting and nonsensical reasoning
IanCal reported ambiguous toilet/wee phrases triggering music, vacuum, toilet, or coffee-maker actions. gs17 found 'it's too cold' turned the thermostat down and 'make it hot' also turned it down with high confidence. poly2it saw 'My car crashed I need help' map to play_music with confidence 1. monster_truck noted a car command did not immediately refuse.
Confidence calibration concerns
gs17 observed low confidence on some bad responses but high confidence on a wrong thermostat action, suggesting a demo threshold. They also reported Celsius/Fahrenheit confusion and unreliable reasoning such as 'turn it up' meaning lower volume.
Developer interest in embedded and admin tool calling
viccis is building an ESP32 Echo replacement and wants to know whether small local models can handle basic tool calling with reasoning, citing a ~36-scenario test suite and poor local Qwen3.8 results versus Gemma4 E4B. rglover suggested using it for admin actions like 'Reboot <server>' from chat.
Author response on tool definitions and reasoning
HenryNdubuaku responded that many demo failures came from playground tool definitions and triggers, which they updated, and argued Needle performs best after tailoring the tool list. He also said the reasoning field mainly provides keywords for the SAN model to ground its tool-call output.
Estimates describe sampled comments, not the whole community.