Resources
Discover useful tools, tutorials, and helpful resources organized by category
AI Safety (CS 2881) Fall 2026 Lecture 3: Modern LLM Training and Inference
This lecture from CS 2881, the graduate AI Safety course taught by Boaz Barak at Harvard in Fall 2026, explains how modern large language models are trained and how inference works. The course covers challenges in the alignment and safety of AI, considering both technical questions and the societal impact of the field, and its lectures are recorded and published online. A clear, current university-level explanation of the LLM training and inference stack for developers who want the fundamentals behind the tools they use.
2026 in LLMs (so far) - Simon Willison
In this closing keynote at WeAreDevelopers World Congress North America in San Jose, Simon Willison gives a chronological tour of what happened in large language models during 2026. He covers the late-2025 point where coding agents became reliable for daily use, the rise of Claw-style agent software, new model releases from several labs, rogue agents breaking containment during training, notable security incidents, and the progress of local models. A fast way for developers to catch up on the year's most important shifts in AI tooling.
Steve Yegge's Vibe Coding Manifesto
In this interview from the AI Engineer Summit, legendary developer Steve Yegge lays out his "Vibe Coding Manifesto," arguing that the industry is shifting from traditional manual coding to a "factory farming" model driven by AI agents. Yegge claims that senior engineers are often the most resistant to this change because their identities are tied to old workflows, even though early adopters are seeing productivity gains as high as 10x [03:09]. He envisions a future where the IDE is replaced by an "agent orchestration dashboard" [14:11], where developers act more like high-level product managers or "NASCAR drivers" who manage dozens of agents simultaneously rather than writing individual lines of code [10:11]. While acknowledging the current chaos and execution problems within major labs like OpenAI and Google [29:48], Yegge remains infectious about the future, asserting that while you don't need to know syntax anymore, you still need deep technical concepts to effectively guide the "village of agents" that will soon be building our software [34:47].
John Schulman on dead ends, scaling RL, and building research institutions
A conversation from Cursor CEO with Thinking Machines chief scientist John Schulman on the first year LLMs could have been useful, building research teams, and where RL goes from here.
Bill Dally - Trends in Deep Learning Hardware
In this talk, Bill Dally walks through the evolving landscape of hardware designed for deep learning, explaining how computational demands of neural networks drive innovation in processors and systems. He highlights the gap between traditional CPUs and what modern AI workloads require — massive parallelism, high throughput, efficient data movement — and shows how specialized hardware (like GPUs and newer AI accelerators) addresses those needs. He also outlines architectural trends and design considerations for future deep-learning hardware: energy efficiency, memory bandwidth, and optimized data flow being key. Overall, the video offers a clear, forward-looking perspective on why and how hardware must evolve to keep up with rapidly advancing AI.
Richard Sutton – Father of RL thinks LLMs are a dead end
In this particular episode, host Dwarkesh Patel interviews Richard Sutton, often dubbed the “father of reinforcement learning.” They discuss Sutton’s views on the current trajectory of large language models (LLMs), why he believes LLMs might be a “dead end” in AI research, and how reinforcement learning (RL) still holds promise for building more generalizable, decision-making systems. The conversation combines deep technical insights with philosophical arguments about AI’s future and the kinds of problems that remain unsolved.
Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)
In this lecture from Stanford’s CS229 course, the speaker offers a high-level walkthrough of how large language models are built, touching on more than just model architecture. Key topics include pre-training vs post-training, the importance of data quality and filtering, evaluation metrics (such as perplexity and human benchmarks), and system implementation (e.g. resource constraints like compute and infrastructure). The talk emphasizes that while novel architecture attracts attention, what often makes the biggest difference in practice are data, evaluation, and systems engineering.
A Visual Guide to Quantization
In this visually rich post, Maarten Grootendorst offers an intuitive and deeply illustrated introduction to quantization in the context of large language models, guiding readers through fundamentals like numerical representation (FP32, FP16, BF16, INT8), symmetric vs. asymmetric quantization, and advanced techniques such as dynamic and static post-training quantization. With over 50 custom visuals, he breaks down key concepts like scale and zero-point calculations, quantization error, clipping strategies, and post-training tweaks (e.g., GPTQ, GGUF) to make compression techniques accessible and engaging—even for those new to the subject.
What is ChatGPT doing...and why does it work?
In this engaging AMA-style presentation, Stephen Wolfram breaks down the core magic behind ChatGPT by illustrating how it constructs language “one word at a time.” He walks viewers through the foundational mechanics—how ChatGPT predicts the next most probable word based on patterns seen in massive text corpora—and explains the surprisingly simple yet powerful processes that allow it to generate human-like responses. Using intuitive analogies and step-by-step reasoning, the video demystifies how ChatGPT models language, why its responses can feel so coherent, and what “makes it work” beyond rote memorization.
Deep Dive into LLMs like ChatGPT
In this three-and-a-half-hour lecture, Andrej Karpathy—former Tesla AI lead and OpenAI founding member—walks a general audience through the inner workings of large language models like ChatGPT. He delves into the full training stack: from preprocessing massive internet text corpora and tokenization to model structure, inference dynamics, fine-tuning techniques, hallucinations, reinforcement learning, memory systems, and model alignment strategies. With clarity and insight, Karpathy shares mental models to understand how LLMs think, their limitations, and how to use them most effectively—making complex AI concepts accessible and actionable.
Andrej Karpathy
Andrej Karpathy’s YouTube channel is a treasure trove for anyone fascinated by AI and deep learning. As a former AI leader at Tesla and founding member of OpenAI, Karpathy brings unparalleled expertise to his content—ranging from foundational tutorials like building GPT models from scratch to deep dives into how large language models actually work. His videos are insightful, technical yet accessible, and feature a unique mix of theory, code walkthroughs, and real-world AI application, making complex concepts feel intuitive and exciting.
