Raschka's 11-Hour Course Builds Reasoning Models From Scratch
Raschka's 11-hour course makes reasoning-model internals concrete with PyTorch code, though its practical payoff depends on compute access and project work.
Sebastian Raschka released an eight-video, 11-hour course that builds reasoning models from scratch in Python and PyTorch. Starting from a pre-trained Qwen3 0.6B model, it implements evaluation, inference-time scaling, reinforcement learning with verifiable rewards, and distillation, with a companion book.
The lectures begin with environment setup and text generation with KV caching, then build a math-answer verifier for MATH-500 that later supplies the reward signal for RLVR. Later videos cover temperature scaling, top-p sampling, self-consistency, self-refinement, a simplified GRPO training loop, training diagnostics such as clipped policy ratios and KL terms, and distillation from DeepSeek R1-generated answers.
Key takeaways
- Raschka skips training a base LLM because that 'would be a whole other book,' pointing to his earlier Build a Large Language Model (From Scratch) course. The reasoning materials are designed as standalone content, so the prior course is recommended but not strictly required.
- A math-answer verifier built for MATH-500 serves double duty: it evaluates the base model and later supplies the reward signal for reinforcement learning with verifiable rewards.
- Inference-time scaling is taught through temperature scaling, multinomial sampling, top-p filtering, self-consistency by majority vote, and a self-refinement loop that critiques and revises answers.
- The reinforcement learning section implements a simplified GRPO loop, then adds training diagnostics such as entropy, advantage statistics, clipped policy ratios, and KL loss terms; distillation uses DeepSeek R1-generated answers.
- Raschka recommends comfort with Python and PyTorch. The first video covers environment setup, hardware choices, and common issues, and the repository includes hardware and environment recommendations.
What people think
Analysis details
18 comments analyzed. Analyzed .
The sample selects active discussion branches and includes available parent comments. It may omit parts of the full thread. The analysis included 18 of 19 usable stored comments because of the input limit. Selected themes do not measure community opinion or how common a view is.
Who the course actually serves
The course is framed as a way to understand reasoning models by implementing them, but its practical payoff is contested. One view holds that video tutorials cannot substitute for the work of building a model if that is the goal; another argues lectures are a starting point, with homework and projects doing the real teaching. A separate argument suggests the material is mainly for learning how the technology works, not for producing deployable systems.
Compute and data barriers for hobbyists
A recurring concern is that applied ML requires compute, training data and infrastructure that put it out of reach for home hackers beyond toy exercises. Toy models are said to lack the nuance of language, and the field is compared to biology, physics or law, where institutional resources matter. The counterpoint is that smaller-scale research domains, such as drug discovery, can be pursued with publicly accessible data and GPUs.
Top-down resources and competing courses
Recommendations point to Sebastian Raschka's book and Karpathy's videos as alternative or complementary paths. The argument for top-down teaching is that theoretical foundations first can be discouraging, while building an LLM from scratch provides a concrete endpoint. A request for a non-coding version of the same material suggests demand for conceptual overviews alongside hands-on courses.
Human versus LLM data efficiency
The claim that language models need massive data to capture nuance drew a challenge: humans learn from far fewer books and interactions. The response is that models are expected to answer plausibly across nearly any topic, which requires broad factual coverage; hallucinated Python functions still look syntactically plausible. The disagreement leaves the fundamental difference between human and machine learning unresolved.
Course economics and personal brand
One critique frames the video course as a monetisation strategy for a personal brand, aimed at people who feel they should learn the material but may not finish. The counterargument is that lectures are a standard teaching format and still require learners to complete projects. The tension is between educational access and commercial incentives in independent AI content.
Interview prep as a use case
A sardonic suggestion that build-your-own-model guides are useful for interview preparation drew a request for specifics about which interviews ask for this.