A first-principles book on LLMs Two volumes available now
Most engineers can call an LLM API. Almost none can explain what happens after the request leaves their machine. They patch around strange behavior with prompt tricks they found by accident. Then ship those tricks into production with nothing to fall back on when something breaks.
This book is the answer.
What you will learn across both volumes
Decoders generate one token at a time, and every new token correlates with everything before it through attention. Repeat the instruction and the model sees it once near the start, where attention weights run hot, and once at the end, where recency dominates. The signal stacks. Position is not metadata. Position is the math.
The model never sees individual letters. Byte-Pair Encoding compresses the word into about two integer token IDs. Counting characters inside a single learned 4096-dimensional vector is asking the model to do something it cannot do. Spell it out and you change the tokenization. Now the letters are separately addressable.
Temperature divides the model's raw logits before the softmax. Set it to 2 and the distribution flattens. Set it to 0.5 and it sharpens. Top-k truncates to the k highest probabilities, a hard cliff. Top-p adds candidates until cumulative probability reaches p, a soft cutoff that adapts to the distribution. Stop guessing. Start predicting which one will fail when your prompt shifts.
Full fine-tuning updates every parameter. Billions of degrees of freedom for a task that needs a few thousand. The actual weight update lives in a low-rank subspace. LoRA decomposes it into two small matrices with about 0.1 percent of the parameters. Optimizer state, gradients, and activation memory all shrink by that ratio. Not just the parameter count. The whole memory budget collapses.
Knowing the trick is folklore. Knowing the mechanism is the difference between shipping and explaining. Between a hack and a system. Between building once and building reliably.
For the engineer who has shipped with the API and now wants to read a transformer paper without bouncing off Section 3. Builds the model from scratch in PyTorch.
For the engineer who has been told to fine-tune for the company's actual domain and is six tabs deep in Hugging Face docs without a clear path. Four end-to-end applied projects.
Seven diagrams. The book has hundreds.
Each transformation, each operation, each decoder pass is drawn out. Color-coded inputs, computed values, results, and aggregations. Step cards that walk you through the math one tensor at a time. Code with line-by-line annotations. This is not a book you skim. It is a book you build along with.
Most LLM books assume you remember your linear algebra. This one teaches every operation right where it appears, with worked numerical examples, color-coded tensors, and step-by-step breakdowns. From dot products in Chapter 5 to backpropagation in Chapter 8 to cross-entropy loss in Chapter 16, the math is built up where you need it, not gated behind a prerequisites chapter you have to suffer through first.
Multiply position by position, add the results, that is it. Chapter 5 builds attention from this single operation. Color-coded so you see exactly which numbers came from where.
Attention is just a weighted sum where the weights come from a softmax. Both operations are walked through with concrete numbers, before any matrix notation appears.
The loss function that trains every modern LLM. You see it as a sum, you see it as a probability, you see why it punishes confidence in wrong answers. No "the loss is calculated" hand-waving.
Backpropagation, AdamW, weight decay. The intuition first, then the equations, then the PyTorch code that implements them. Three passes through the same idea so it actually sticks.
What you actually need to start: Python (loops, classes, NumPy basics), high-school math (multiplication, exponents, basic graphs), and curiosity. That is it. Calculus shows up briefly for backprop, and Chapter 8 walks through the chain rule from scratch when you need it.
Not "what is attention?" trivia. Real scenarios where you have to debug, decide, or defend a choice in front of a hiring manager. The kind of questions FAANG teams, AI labs, and AI-first startups actually ask. Sample of what is inside, with chapter references so you know where the full answer lives.
Same total compute as single-head attention, but each head can specialize on different relationships (syntactic, positional, semantic, long-range vs. local). Splitting d_model means head outputs concatenate back to d_model with no extra projection cost. Multiple full-dimension heads would multiply both compute and parameters with no upside, because the heads would learn redundant patterns. The split forces specialization.
As d_k grows, the variance of the dot products grows with it. Without scaling, the softmax inputs blow up, the distribution saturates into a near one-hot, and the gradient through the non-attended entries goes to zero. Training stalls. Dividing by √dk keeps the dot products in a regime where the softmax stays smooth and gradients flow to every key position.
The intrinsic dimension of the task-specific weight update sets the minimum rank. Probe-train on a small subset of your data across ranks 4, 8, 16, 32. The smallest rank where validation loss plateaus is your answer. If even rank 32 underfits, the task likely requires moving general capabilities, not specializing them. At that point you reach for QLoRA at higher rank or full fine-tuning.
Two compounding effects. First, attention is a softmax over the full context, so the relevant chunk competes against 49 distractors and its weight drops by roughly an order of magnitude. Second, decoders under-attend to the middle of long contexts (lost-in-the-middle, Liu et al. 2023), and 50 chunks puts the relevant one in the middle band ~80% of the time. Fix: re-rank to top-5, place the best chunk at the start or end of context, and use a cross-encoder reranker before the LLM call.
(1) Train and inference tokenizer mismatch. Different special tokens, different pad token, different BOS/EOS handling. The model sees an out-of-distribution prefix at generation time and produces noise. (2) Sampling parameters wrong for the task. Temperature too high for code generation, top-p too restrictive for creative tasks, or no special-token suppression. (3) Position encoding mismatch. Trained on max sequence length 2048, inferring at 4096 with no rope-scaling. The model has never seen positions that high.
Argmax over logits is theoretically deterministic. In practice, GPU floating-point operations are non-associative, so reduction order changes per batch shape and per kernel. Different batching across requests, different cuBLAS algorithm selection, or speculative decoding can all shift the rounding by a few ULPs, occasionally flipping the argmax when two top candidates are within that delta. Determinism in production needs greedy decoding plus pinned kernels plus batch-size locking.
Each chapter ends with 10 to 15 questions like these. Some test the mechanism, some test the trade-off, some test debugging instincts. Across the two volumes, you finish with 150+ scenario-based questions you can actually use to prepare for an LLM-engineer or AI-engineer interview.
Read the original transformer paper end to end and recognize every component on the page.
Explain attention, residuals, and layer norm without reaching for an analogy you do not trust.
Build and train a complete GPT from scratch in PyTorch.
Pick a fine-tuning approach (full FT vs LoRA vs QLoRA vs RLHF) and defend the choice.
Walk through four end-to-end fine-tuning projects on real domains.
Optimize inference for production: KV cache, quantization, batching, speculative decoding.
Build the function-calling foundation that AI agents and agentic AI workflows depend on.
Predict when a prompt trick will fail. Predict why a fine-tune will work. Stop guessing.
Volume I covers foundations and the complete transformer architecture in nine chapters: tokenization, embeddings, positional encoding, attention, residuals, layer norm, the feed-forward network, and a complete GPT built from scratch in PyTorch. Volume II covers what happens after a model is trained: inference internals, KV cache, quantization, batching, RLHF, instruction tuning, LoRA, QLoRA, and four end-to-end applied projects on real domains.
If you want to understand how transformers work and read research papers without bouncing, start with Volume I. If you need to ship a fine-tuned LLM into production for a specific domain like legal, SQL, function-calling, or classification, Volume II is the faster path to the answers you need.
Comfort with Python (loops, classes, NumPy basics) and a working understanding of neural networks at a high level. You do not need a graduate degree, you do not need to have read the original transformer paper, and you do not need linear algebra beyond matrix multiplication and dot products. Every concept is built up from intuition first, math second.
PyTorch throughout. Volume I builds a complete GPT from scratch in PyTorch in Chapter 9. Volume II uses PyTorch with Hugging Face Transformers and PEFT for the four applied fine-tuning projects.
The mechanisms taught in the book are what every modern LLM is built on: decoder architecture, tokenization, attention, KV cache, RLHF, LoRA. Once you understand these, every commercial model you use, including GPT-4, Claude, Gemini, and the Llama family, becomes legible. The book teaches the substrate, not the API surface.
Most build-from-scratch books stop at training a small GPT. Inside Large Language Models keeps going. After Volume I builds and trains the model, Volume II walks through production inference, optimization, RLHF, and four applied fine-tuning projects on real domains: contract classification, legal QLoRA, text-to-SQL, and function calling. You finish with a model in production, not just a model on disk.
Chapter 9 of Volume I is exactly this exercise. By the end you will have written every component of a GPT in PyTorch and trained it on real text. Every line is annotated. Every tensor shape is shown.
Chapter 17 of Volume II covers function calling end to end: the LLM-side capability that AI agents and agentic AI workflows depend on. Tool use, structured output, the model-side mechanics that let an LLM decide which function to call and what arguments to pass. The book does not cover specific agent frameworks like LangChain, AutoGPT, or CrewAI. It teaches the LLM-side foundation that every agent framework builds on. Once you understand function calling at the level this book teaches, every agent framework becomes a thin wrapper around what you already know.
Two volumes. Available now.
Every week without this knowledge is another week of folklore engineering. Another shipping decision made on guesswork. Another LLM bug nobody on your team can debug.
If you have ever found a prompt trick that just worked and wished you knew why, these books are written for you. Not someday. Today.