Bibliography¶
Compiled on 2026-10-11 from the study plan. It lists every book and paper the plan cites, in the chapters and appendices alike, plus the long-form online articles it uses as reference texts (section 5). Courses, documentation, tools and datasets are not listed.
How to read this list¶
- Ch. is the first chapter that cites the work; "A" is Appendix A, the core book shelf.
- Year is the year of the edition or version the plan cites. For papers it is the first public release (usually arXiv). When the formal venue appeared in a later year, the venue column says so.
- Publisher / venue is the publisher for books and the venue for papers. "arXiv" means the work is a preprint only.
- Free online? has three values:
- Free means the full text is legally and freely available from the official source.
- Partial means part of it is free, such as code, notebooks, chapter summaries or sample chapters.
- Paid means it must be bought or borrowed. Paid, hard to get marks a paid book that is also difficult to obtain: no open-access edition and no free copy through the usual routes.
- Papers with more than three authors show the first author followed by "et al.".
- Books that exist only as a website, or whose official edition is a website, carry the link to the official site in the title. All other works are cited through the official page given in the chapter that uses them.
1. Books¶
1.1 Free (full text)¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 1 | Precalculus (2nd ed.) | Jay Abramson et al. | 2021 | OpenStax (Rice University) | Free |
| 1 | College Algebra (2nd ed.) | Jay Abramson et al. | 2021 | OpenStax (Rice University) | Free |
| 1 | Precalculus (3rd corrected ed.) | Carl Stitz, Jeff Zeager | 2013 | Self-published (CC BY-NC-SA) | Free |
| 1 | Paul's Online Math Notes (Algebra, Calculus I–III) | Paul Dawkins | n.d. (continuously updated) | Lamar University | Free |
| 2 | Calculus Vol. 1–3 | Gilbert Strang, Edwin Herman et al. | 2016 | OpenStax (Rice University) | Free |
| 2 | Calculus (3rd ed.) | Gilbert Strang | 2017 | Wellesley-Cambridge Press (free via MIT OpenCourseWare) | Free |
| 2 | Mathematics for Machine Learning | Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong | 2020 | Cambridge University Press | Free |
| 3 | Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares | Stephen Boyd, Lieven Vandenberghe | 2018 | Cambridge University Press | Free |
| 3 | Linear Algebra (4th ed.) | Jim Hefferon | 2020 | Self-published (Saint Michael's College) | Free |
| 3 | Linear Algebra Done Right (4th ed.) | Sheldon Axler | 2024 | Springer (open access, CC BY-NC) | Free |
| 4 | Introduction to Probability (2nd ed.) | Joseph K. Blitzstein, Jessica Hwang | 2019 | Chapman & Hall/CRC | Free |
| 4 | Probabilistic Machine Learning: An Introduction | Kevin P. Murphy | 2022 | MIT Press | Free |
| 4 | Information Theory, Inference, and Learning Algorithms | David J. C. MacKay | 2003 | Cambridge University Press | Free (on-screen reading) |
| 4 | Pattern Recognition and Machine Learning | Christopher M. Bishop | 2006 | Springer | Free |
| 5 | Deep Learning | Ian Goodfellow, Yoshua Bengio, Aaron Courville | 2016 | MIT Press | Free |
| 5 | Dive into Deep Learning | Aston Zhang, Zachary C. Lipton, Mu Li, Alexander J. Smola | 2023 | Cambridge University Press | Free |
| 5 | Convex Optimization | Stephen Boyd, Lieven Vandenberghe | 2004 | Cambridge University Press | Free |
| 6 | Think Python (3rd ed.) | Allen B. Downey | 2024 | O'Reilly | Free |
| 6 | Automate the Boring Stuff with Python (3rd ed.) | Al Sweigart | 2025 | No Starch Press | Free (web) |
| 6 | Python Data Science Handbook (1st ed. online; 2nd ed. in print) | Jake VanderPlas | 2016 (1st ed.); 2022 (2nd ed.) | O'Reilly | Free (1st ed. online) |
| 6 | Pro Git (2nd ed.) | Scott Chacon, Ben Straub | 2014 | Apress | Free |
| 7 | Machine Learning Engineering Open Book | Stas Bekman | continuously updated | Self-published (GitHub) | Free |
| 7 | The Ultra-Scale Playbook: Training LLMs on GPU Clusters | Nouamane Tazi, Ferdinand Mom et al. | 2025 | Hugging Face | Free |
| 8 | An Introduction to Statistical Learning with Applications in Python | Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor | 2023 | Springer | Free |
| 8 | An Introduction to Statistical Learning with Applications in R (2nd ed.) | Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani | 2021 | Springer | Free |
| 8 | The Elements of Statistical Learning (2nd ed.) | Trevor Hastie, Robert Tibshirani, Jerome Friedman | 2009 | Springer | Free |
| 8 | The Hundred-Page Machine Learning Book | Andriy Burkov | 2019 | Self-published (Andriy Burkov) | Free ("read first, buy later") |
| 8 | Understanding Deep Learning | Simon J. D. Prince | 2023 | MIT Press | Free |
| 9 | Deep Learning: Foundations and Concepts | Christopher M. Bishop, Hugh Bishop | 2024 | Springer | Free (read online) |
| 10 | Neural Networks and Deep Learning | Michael Nielsen | 2015 | Determination Press | Free |
| 11 | Speech and Language Processing (3rd ed. draft, Aug 2026 release) | Dan Jurafsky, James H. Martin | 2026 (draft) | Stanford (draft of the Pearson edition) | Free |
| 11 | Introduction to Natural Language Processing | Jacob Eisenstein | 2019 | MIT Press | Free (draft PDF) |
| 15 | The Smol Training Playbook: The Secrets to Building World-Class LLMs | Loubna Ben Allal et al. | 2025 | Hugging Face | Free |
| 16 | How to Scale Your Model: A Systems View of LLMs on TPUs | Jacob Austin et al. | 2025 | Google DeepMind | Free |
| 17 | Reinforcement Learning from Human Feedback | Nathan Lambert | 2026 | Manning (print); free online edition | Free (online), paid print |
| 19 | Reinforcement Learning: An Introduction (2nd ed.) | Richard S. Sutton, Andrew G. Barto | 2018 | MIT Press | Free |
| 12 | Super Study Guide: Transformers & Large Language Models (optional) | Afshine Amidi, Shervine Amidi | n.d. (not stated in the plan) | Self-published | Free (read online; paper and PDF editions are sold) |
1.2 Partly free (code, notebooks, summaries or samples only)¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 3 | Introduction to Linear Algebra (6th ed.) | Gilbert Strang | 2023 | Wellesley-Cambridge Press | Partial (sample sections and selected solutions) |
| 7 | AI Engineering | Chip Huyen | 2025 | O'Reilly | Partial (chapter summaries) |
| 8 | Hands-On Machine Learning with Scikit-Learn and PyTorch (1st ed.) | Aurélien Géron | 2025 | O'Reilly | Partial (notebooks free) |
| 9 | Deep Learning for Coders with fastai and PyTorch | Jeremy Howard, Sylvain Gugger | 2020 | O'Reilly | Partial (notebooks free) |
| 12 | Build a Large Language Model (From Scratch) | Sebastian Raschka | 2024 | Manning | Partial (code and companion videos free) |
| 13 | Hands-On Large Language Models | Jay Alammar, Maarten Grootendorst | 2024 | O'Reilly | Partial (notebooks free) |
| 19 | Build a Reasoning Model (From Scratch) | Sebastian Raschka | 2026 | Manning | Partial (code free) |
| 22 | Designing Machine Learning Systems | Chip Huyen | 2022 | O'Reilly | Partial (chapter summaries) |
1.3 Paid¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 1 | Precalculus: Mathematics for Calculus (8th ed.) | James Stewart, Lothar Redlin, Saleem Watson | 2024 (as cited in the plan) | Cengage | Paid |
| 2 | Calculus: Early Transcendentals (9th ed.) | James Stewart, Daniel Clegg, Saleem Watson | 2020 | Cengage | Paid |
| 3 | Linear Algebra and Its Applications (6th ed.) | David C. Lay, Steven R. Lay, Judi J. McDonald | 2020 | Pearson | Paid |
| 3 | Linear Algebra and Learning from Data | Gilbert Strang | 2019 | Wellesley-Cambridge Press | Paid |
| 4 | Introduction to Probability (2nd ed.) | Dimitri P. Bertsekas, John N. Tsitsiklis | 2008 | Athena Scientific | Paid |
| 4 | All of Statistics: A Concise Course in Statistical Inference | Larry Wasserman | 2004 | Springer | Paid |
| 6 | Fluent Python (2nd ed.) | Luciano Ramalho | 2022 | O'Reilly | Paid |
| 7 | Programming Massively Parallel Processors (4th ed.) | Wen-mei W. Hwu, David B. Kirk, Izzat El Hajj | 2022 | Elsevier (Morgan Kaufmann) | Paid |
| A | Large Language Models: A Deep Dive: Bridging Theory and Practice | Uday Kamath, Kevin Keenan, Garrett Somers, Sarah Sorenson | 2024 | Springer | Paid, hard to get (see verification note 6) |
2. Papers¶
2.1 Foundations: mathematics, optimization and deep learning (Ch. 2–10)¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 2 | The Matrix Calculus You Need For Deep Learning | Terence Parr, Jeremy Howard | 2018 | arXiv | Free |
| 2 | Automatic Differentiation in Machine Learning: a Survey | Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey A. Radul, Jeffrey M. Siskind | 2018 (arXiv 2015) | Journal of Machine Learning Research (JMLR) | Free |
| 5 | An overview of gradient descent optimization algorithms | Sebastian Ruder | 2016 | arXiv | Free |
| 5 | Adam: A Method for Stochastic Optimization | Diederik P. Kingma, Jimmy Ba | 2015 | ICLR | Free |
| 5 | Decoupled Weight Decay Regularization | Ilya Loshchilov, Frank Hutter | 2019 | ICLR | Free |
| 5 | What Every Computer Scientist Should Know About Floating-Point Arithmetic | David Goldberg | 1991 | ACM Computing Surveys | Free (Oracle copy of the article) |
| 5 | Mixed Precision Training | Paulius Micikevicius et al. | 2018 (arXiv 2017) | ICLR | Free |
| 8 | A Few Useful Things to Know About Machine Learning | Pedro Domingos | 2012 | Communications of the ACM | Free (author's copy) |
| 9 | Dropout: A Simple Way to Prevent Neural Networks from Overfitting | Nitish Srivastava et al. | 2014 | JMLR | Free |
| 9 | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift | Sergey Ioffe, Christian Szegedy | 2015 | ICML | Free |
| 9 | Layer Normalization | Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton | 2016 | arXiv | Free |
| 9 | Deep Residual Learning for Image Recognition | Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun | 2016 | CVPR | Free |
| 10 | Learning representations by back-propagating errors | David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams | 1986 | Nature | Paid |
| 10 | Understanding the difficulty of training deep feedforward neural networks | Xavier Glorot, Yoshua Bengio | 2010 | AISTATS (PMLR) | Free |
| 10 | Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification | Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun | 2015 | ICCV | Free |
2.2 NLP before the Transformer, and the Transformer (Ch. 11–14)¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 11 | A Neural Probabilistic Language Model | Yoshua Bengio, Réjean Ducharme, Pascal Vincent, Christian Jauvin | 2003 | JMLR | Free |
| 11 | Efficient Estimation of Word Representations in Vector Space | Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean | 2013 | arXiv (ICLR workshop) | Free |
| 11 | Distributed Representations of Words and Phrases and their Compositionality | Tomas Mikolov et al. | 2013 | NeurIPS | Free |
| 11 | GloVe: Global Vectors for Word Representation | Jeffrey Pennington, Richard Socher, Christopher D. Manning | 2014 | EMNLP | Free |
| 11 | Sequence to Sequence Learning with Neural Networks | Ilya Sutskever, Oriol Vinyals, Quoc V. Le | 2014 | NeurIPS | Free |
| 11 | Neural Machine Translation by Jointly Learning to Align and Translate | Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio | 2015 (arXiv 2014) | ICLR | Free |
| 11 | Deep contextualized word representations (ELMo) | Matthew E. Peters et al. | 2018 | NAACL | Free |
| 12 | Attention Is All You Need | Ashish Vaswani et al. | 2017 | NeurIPS | Free |
| 12 | Formal Algorithms for Transformers | Mary Phuong, Marcus Hutter | 2022 | arXiv | Free |
| 12 | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding | Jacob Devlin et al. | 2019 (arXiv 2018) | NAACL | Free |
| 12 | RoFormer: Enhanced Transformer with Rotary Position Embedding | Jianlin Su et al. | 2021 | arXiv (later in Neurocomputing, 2024) | Free |
| 12 | On Layer Normalization in the Transformer Architecture | Ruibin Xiong et al. | 2020 | ICML | Free |
| 12 | Root Mean Square Layer Normalization | Biao Zhang, Rico Sennrich | 2019 | NeurIPS | Free |
| 12 | DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model | DeepSeek-AI | 2024 | arXiv | Free |
| 13 | Neural Machine Translation of Rare Words with Subword Units | Rico Sennrich, Barry Haddow, Alexandra Birch | 2016 (arXiv 2015) | ACL | Free |
| 13 | Language Models are Unsupervised Multitask Learners (GPT-2) | Alec Radford et al. | 2019 | OpenAI technical report | Free |
| 13 | Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates | Taku Kudo | 2018 | ACL | Free |
| 13 | SentencePiece: A simple and language independent subword tokenizer and detokenizer | Taku Kudo, John Richardson | 2018 | EMNLP (system demonstrations) | Free |
| 13 | Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation | Yonghui Wu et al. | 2016 | arXiv | Free |
| 13 | Using the Output Embedding to Improve Language Models | Ofir Press, Lior Wolf | 2017 (arXiv 2016) | EACL | Free |
| 13 | Language Model Tokenizers Introduce Unfairness Between Languages | Aleksandar Petrov et al. | 2023 | NeurIPS | Free |
| 13 | ByT5: Towards a token-free future with pre-trained byte-to-byte models | Linting Xue et al. | 2022 | TACL | Free |
| 14 | Language Models are Few-Shot Learners (GPT-3) | Tom B. Brown et al. | 2020 | NeurIPS | Free |
| 14 | The Curious Case of Neural Text Degeneration | Ari Holtzman et al. | 2020 (arXiv 2019) | ICLR | Free |
| 14 | TinyStories: How Small Can Language Models Be and Still Speak Coherent English? | Ronen Eldan, Yuanzhi Li | 2023 | arXiv | Free |
| 14 | A Mathematical Framework for Transformer Circuits | Nelson Elhage, Neel Nanda, Catherine Olsson et al. | 2021 | Transformer Circuits Thread (Anthropic) | Free |
2.3 Pretraining and systems (Ch. 15–16)¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 15 | Scaling Laws for Neural Language Models | Jared Kaplan et al. | 2020 | arXiv | Free |
| 15 | Training Compute-Optimal Large Language Models (Chinchilla) | Jordan Hoffmann et al. | 2022 | arXiv (NeurIPS 2022) | Free |
| 15 | The Pile: An 800GB Dataset of Diverse Text for Language Modeling | Leo Gao et al. | 2020 | arXiv | Free |
| 15 | Deduplicating Training Data Makes Language Models Better | Katherine Lee et al. | 2022 (arXiv 2021) | ACL | Free |
| 15 | The RefinedWeb Dataset for Falcon LLM | Guilherme Penedo et al. | 2023 | NeurIPS (Datasets & Benchmarks) | Free |
| 15 | The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale | Guilherme Penedo et al. | 2024 | NeurIPS (Datasets & Benchmarks) | Free |
| 15 | FineWeb2: One Pipeline to Scale Them All | Guilherme Penedo et al. | 2025 | arXiv | Free |
| 15 | Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research | Luca Soldaini et al. | 2024 | ACL | Free |
| 15 | OLMo: Accelerating the Science of Language Models | Dirk Groeneveld et al. | 2024 | ACL | Free |
| 15 | 2 OLMo 2 Furious | Team OLMo | 2024 | arXiv | Free |
| 15 | The Llama 3 Herd of Models | Meta (Llama Team) | 2024 | arXiv | Free |
| 15 | DeepSeek-V3 Technical Report | DeepSeek-AI | 2024 | arXiv | Free |
| 16 | ZeRO: Memory Optimizations Toward Training Trillion Parameter Models | Samyam Rajbhandari et al. | 2020 (arXiv 2019) | SC20 (IEEE/ACM) | Free |
| 16 | PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel | Yanli Zhao et al. | 2023 | Proceedings of the VLDB Endowment | Free |
| 16 | Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism | Mohammad Shoeybi et al. | 2019 | arXiv | Free |
| 16 | Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM | Deepak Narayanan et al. | 2021 | SC21 (IEEE/ACM) | Free |
| 16 | Reducing Activation Recomputation in Large Transformer Models | Vijay Korthikanti et al. | 2023 (arXiv 2022) | MLSys | Free |
| 16 | FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness | Tri Dao et al. | 2022 | NeurIPS | Free |
| 16 | FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning | Tri Dao | 2024 (arXiv 2023) | ICLR | Free |
| 16 | FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision | Jay Shah et al. | 2024 | NeurIPS | Free |
2.4 Post-training and reasoning (Ch. 17–19)¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 17 | LoRA: Low-Rank Adaptation of Large Language Models | Edward J. Hu et al. | 2022 (arXiv 2021) | ICLR | Free |
| 17 | QLoRA: Efficient Finetuning of Quantized LLMs | Tim Dettmers et al. | 2023 | NeurIPS | Free |
| 17 | LoRA Learns Less and Forgets Less | Dan Biderman et al. | 2024 | TMLR | Free |
| 17 | Finetuned Language Models Are Zero-Shot Learners (FLAN) | Jason Wei et al. | 2022 (arXiv 2021) | ICLR | Free |
| 17 | LIMA: Less Is More for Alignment | Chunting Zhou et al. | 2023 | NeurIPS | Free |
| 18 | Training language models to follow instructions with human feedback (InstructGPT) | Long Ouyang et al. | 2022 | NeurIPS | Free |
| 18 | Direct Preference Optimization: Your Language Model is Secretly a Reward Model | Rafael Rafailov et al. | 2023 | NeurIPS | Free |
| 18 | Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback | Yuntao Bai et al. | 2022 | arXiv (Anthropic) | Free |
| 18 | Constitutional AI: Harmlessness from AI Feedback | Yuntao Bai et al. | 2022 | arXiv (Anthropic) | Free |
| 18 | Tülu 3: Pushing Frontiers in Open Language Model Post-Training | Nathan Lambert et al. | 2024 | arXiv (Ai2) | Free |
| 18 | Proximal Policy Optimization Algorithms | John Schulman et al. | 2017 | arXiv | Free |
| 19 | Chain-of-Thought Prompting Elicits Reasoning in Large Language Models | Jason Wei et al. | 2022 | NeurIPS | Free |
| 19 | Self-Consistency Improves Chain of Thought Reasoning in Language Models | Xuezhi Wang et al. | 2023 (arXiv 2022) | ICLR | Free |
| 19 | DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models | Zhihong Shao et al. | 2024 | arXiv | Free |
| 19 | DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning | DeepSeek-AI | 2025 | arXiv (also published in Nature, 2025) | Free |
| 19 | Let's Verify Step by Step | Hunter Lightman et al. | 2024 (arXiv 2023) | ICLR (OpenAI) | Free |
| 19 | Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters | Charlie Snell et al. | 2024 | arXiv (ICLR 2025) | Free |
2.5 Evaluation, serving and operations (Ch. 20–24)¶
| Ch. | Title | Authors | Year | Publisher / venue | Free online? |
|---|---|---|---|---|---|
| 20 | Lessons from the Trenches on Reproducible Evaluation of Language Models | Stella Biderman et al. | 2024 | arXiv | Free |
| 20 | Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations | Evan Miller | 2024 | arXiv | Free |
| 20 | Measuring Massive Multitask Language Understanding (MMLU) | Dan Hendrycks et al. | 2021 (arXiv 2020) | ICLR | Free |
| 20 | Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge | Peter Clark et al. | 2018 | arXiv | Free |
| 20 | HellaSwag: Can a Machine Really Finish Your Sentence? | Rowan Zellers et al. | 2019 | ACL | Free |
| 20 | Training Verifiers to Solve Math Word Problems (GSM8K) | Karl Cobbe et al. | 2021 | arXiv | Free |
| 20 | Evaluating Large Language Models Trained on Code (HumanEval) | Mark Chen et al. | 2021 | arXiv | Free |
| 20 | GPQA: A Graduate-Level Google-Proof Q&A Benchmark | David Rein et al. | 2024 (arXiv 2023) | COLM | Free |
| 20 | Instruction-Following Evaluation for Large Language Models (IFEval) | Jeffrey Zhou et al. | 2023 | arXiv | Free |
| 20 | SWE-bench: Can Language Models Resolve Real-World GitHub Issues? | Carlos E. Jimenez et al. | 2024 (arXiv 2023) | ICLR | Free |
| 20 | Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena | Lianmin Zheng et al. | 2023 | NeurIPS | Free |
| 20 | Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference | Wei-Lin Chiang et al. | 2024 | ICML | Free |
| 20 | Proving Test Set Contamination in Black Box Language Models | Yonatan Oren et al. | 2024 (arXiv 2023) | ICLR | Free |
| 20 | Holistic Evaluation of Language Models (HELM) | Percy Liang et al. | 2023 (arXiv 2022) | TMLR | Free |
| 21 | Efficiently Scaling Transformer Inference | Reiner Pope et al. | 2023 (arXiv 2022) | MLSys | Free |
| 21 | LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale | Tim Dettmers et al. | 2022 | NeurIPS | Free |
| 21 | GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers | Elias Frantar et al. | 2023 (arXiv 2022) | ICLR | Free |
| 21 | AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration | Ji Lin et al. | 2024 (arXiv 2023) | MLSys | Free |
| 21 | SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models | Guangxuan Xiao et al. | 2023 (arXiv 2022) | ICML | Free |
| 21 | Fast Inference from Transformers via Speculative Decoding | Yaniv Leviathan, Matan Kalman, Yossi Matias | 2023 (arXiv 2022) | ICML | Free |
| 21 | Accelerating Large Language Model Decoding with Speculative Sampling | Charlie Chen et al. | 2023 | arXiv (DeepMind) | Free |
| 21 | Efficient Memory Management for Large Language Model Serving with PagedAttention | Woosuk Kwon et al. | 2023 | SOSP | Free |
| 21 | SGLang: Efficient Execution of Structured Language Model Programs | Lianmin Zheng et al. | 2024 (arXiv 2023) | NeurIPS | Free |
| 22 | Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks | Patrick Lewis et al. | 2020 | NeurIPS | Free |
| 22 | Retrieval-Augmented Generation for Large Language Models: A Survey | Yunfan Gao et al. | 2023 | arXiv | Free |
| 22 | ReAct: Synergizing Reasoning and Acting in Language Models | Shunyu Yao et al. | 2023 (arXiv 2022) | ICLR | Free |
| 22 | Model Cards for Model Reporting | Margaret Mitchell et al. | 2019 (arXiv 2018) | FAT* (ACM) | Free |
| 23 | DataComp-LM: In search of the next generation of training sets for language models | Jeffrey Li et al. | 2024 | NeurIPS (Datasets & Benchmarks) | Free |
| 24 | How to Read a Paper | S. Keshav | 2007 | ACM SIGCOMM Computer Communication Review | Free |
| 24 | Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) | Joelle Pineau et al. | 2021 (arXiv 2020) | JMLR | Free |
| 24 | Troubling Trends in Machine Learning Scholarship | Zachary C. Lipton, Jacob Steinhardt | 2018 | arXiv (ICML workshop) | Free |
3. Totals¶
| Group | Count |
|---|---|
| Books, free | 37 |
| Books, partly free | 8 |
| Books, paid | 9 |
| Books | 54 |
| Papers, free | 108 |
| Papers, paid | 1 (Rumelhart et al., 1986) |
| Papers | 109 |
4. Caveats¶
The plan states the title and the edition for most of these works. It does not always state the year, the publisher or the authors in full. I filled those gaps from memory and did not check them against the web. Check these entries before relying on them:
- Stewart, Redlin, Watson, Precalculus 8th ed. The year 2024 is copied from the plan. I could not confirm it.
- Stitz and Zeager, Precalculus. The plan says 2013 (3rd corrected ed.). A local copy was named 2012.
- VanderPlas, Python Data Science Handbook. The plan gives 2022 for the 2nd edition. Please confirm that edition exists in print.
- Downey, Think Python 3rd ed. I listed O'Reilly as publisher, which is not in the plan.
- Burkov, Hefferon, Bekman, Amidi & Amidi. I listed these as self-published. I did not check that.
- Kamath et al., Large Language Models: A Deep Dive: Bridging Theory and Practice. Marked paid and hard to get (Uday Kamath, Kevin Keenan, Garrett Somers and Sarah Sorenson; Springer, 2024). Only the Springer preview chapters are free, and I found no sign that the full text is open access.
- Jurafsky and Martin, SLP3. The plan says "August 2026 release". The year 2026 is the draft date, not a print edition.
- DeepSeek-R1. The Nature publication in 2025 is from memory.
- Papers with a later formal venue (RoFormer, Chinchilla, Snell et al., FineWeb2, OLMo 2). I kept the arXiv year and flagged the venue only where I was fairly sure.
- Garbled or paraphrased titles. A few titles are the full official titles rather than the shortened ones in the plan, for example GSM8K, IFEval, ARC and MMLU. They are worth a spot check.
5. Long-form online articles and guides¶
These are not books or papers, but the plan uses them as reference texts: long guides, playbooks and essays that are read like a chapter. All are free to read online. The link goes to the official page the plan cites. "Year" is the year the plan gives; "updated" means the plan says the page was revised later.
5.1 Training, scaling and systems¶
| Ch. | Title | Authors | Year | Publisher / venue |
|---|---|---|---|---|
| 7, 15, 16 | Transformer Math 101 | Quentin Anthony, Stella Biderman, Hailey Schoelkopf | 2023 | EleutherAI blog |
| 7, 16 | Making Deep Learning Go Brrrr From First Principles | Horace He | 2022 | Personal site |
| 7 | Which GPU(s) to Get for Deep Learning | Tim Dettmers | 2023 | Personal blog |
| 9 | Deep Learning Tuning Playbook | Varun Godbole, George E. Dahl, Justin Gilmer, Christopher J. Shallue, Zachary Nado | 2023 | Google Research (GitHub) |
| 9 | A Recipe for Training Neural Networks | Andrej Karpathy | 2019 | Personal blog |
| 16 | How to Train Really Large Models on Many GPUs? | Lilian Weng | 2021 | Lil'Log |
| 23 | Beating GPT-2 for <<$100: the nanochat journey | Andrej Karpathy | current | GitHub discussion |
| 23 | I trained a ChatGPT clone for £75 | Sam Dobson | 2025 | Personal blog |
| 23 | SmolLM3: smol, multilingual, long-context reasoner | Hugging Face | 2025 | Hugging Face blog |
5.2 Architecture and model internals¶
| Ch. | Title | Authors | Year | Publisher / venue |
|---|---|---|---|---|
| 11 | The Unreasonable Effectiveness of Recurrent Neural Networks | Andrej Karpathy | 2015 | Personal blog |
| 11 | Understanding LSTM Networks | Christopher Olah | 2015 | colah's blog |
| 12 | The Illustrated Transformer | Jay Alammar | 2018 | Personal blog |
| 12 | The Annotated Transformer | Sasha Rush et al. | 2018, updated 2022 | Harvard NLP |
| 12 | The Big LLM Architecture Comparison | Sebastian Raschka | 2025, updated 2026 | Ahead of AI (newsletter) |
| 14 | How to generate text: using different decoding methods for language generation with Transformers | Patrick von Platen | 2020 | Hugging Face blog |
5.3 Post-training, reasoning and evaluation¶
| Ch. | Title | Authors | Year | Publisher / venue |
|---|---|---|---|---|
| 17 | LoRA Without Regret | John Schulman, Thinking Machines Lab | 2025 | Thinking Machines blog |
| 18 | Illustrating Reinforcement Learning from Human Feedback (RLHF) | Nathan Lambert, Louis Castricato, Leandro von Werra, Alex Havrilla | 2022 | Hugging Face blog |
| 18 | Reward Hacking in Reinforcement Learning | Lilian Weng | 2024 | Lil'Log |
| 19 | Understanding Reasoning LLMs | Sebastian Raschka | 2025 | Ahead of AI (newsletter) |
| 20 | The LLM Evaluation Guidebook | Clémentine Fourrier et al. | not stated | Hugging Face (OpenEvals) |
| 20, 22 | Your AI Product Needs Evals | Hamel Husain | 2024 | Personal blog |
5.4 Inference, serving and applications¶
| Ch. | Title | Authors | Year | Publisher / venue |
|---|---|---|---|---|
| 21 | Transformer Inference Arithmetic | kipply | 2022 | Personal blog |
| 21 | Understanding and Coding the KV Cache in LLMs from Scratch | Sebastian Raschka | 2025 | Ahead of AI (newsletter) |
| 21 | Optimizing your LLM in Production | Patrick von Platen | 2023 | Hugging Face blog |
| 22 | Patterns for Building LLM-based Systems & Products | Eugene Yan | 2023 | Personal blog |
| 22 | What We've Learned From a Year of Building with LLMs | Eugene Yan, Bryan Bischof, Charles Frye, Hamel Husain, Jason Liu, Shreya Shankar | 2024 | applied-llms.org |
| 22 | Building Effective Agents | Erik Schluntz, Barry Zhang | 2024 | Anthropic |
| 22 | Effective context engineering for AI agents | Anthropic Applied AI team | 2025 | Anthropic |
| 22 | OWASP Top 10 for LLM Applications 2025 | OWASP | 2025 | OWASP |
| 22 | The Open Source AI Definition 1.0 | Open Source Initiative | 2024 | OSI |
5.5 Mathematical background and perspective¶
| Ch. | Title | Authors | Year | Publisher / venue |
|---|---|---|---|---|
| 4 | Visual Information Theory | Christopher Olah | 2015 | colah's blog |
| 5 | Why Momentum Really Works | Gabriel Goh | 2017 | Distill |
| 5 | The Bitter Lesson | Rich Sutton | 2019 | Personal site |
| 5 | What Is Bfloat16 Arithmetic? | Nicholas J. Higham | 2020 | Personal blog |
| 10 | Calculus on Computational Graphs: Backpropagation | Christopher Olah | 2015 | colah's blog |
| 24 | Open challenges in LLM research | Chip Huyen | 2023 | Personal blog |
6. Still not included¶
About 60 further items are not listed anywhere in this file: documentation pages, tools and libraries, courses and lecture series (the complete ones are on the standalone courses and books page), cheatsheets, datasets, newsletters and shorter blog posts.