Resources
Discover useful tools, tutorials, and helpful resources organized by category
Steve Yegge's Vibe Coding Manifesto
In this interview from the AI Engineer Summit, legendary developer Steve Yegge lays out his "Vibe Coding Manifesto," arguing that the industry is shifting from traditional manual coding to a "factory farming" model driven by AI agents. Yegge claims that senior engineers are often the most resistant to this change because their identities are tied to old workflows, even though early adopters are seeing productivity gains as high as 10x [03:09]. He envisions a future where the IDE is replaced by an "agent orchestration dashboard" [14:11], where developers act more like high-level product managers or "NASCAR drivers" who manage dozens of agents simultaneously rather than writing individual lines of code [10:11]. While acknowledging the current chaos and execution problems within major labs like OpenAI and Google [29:48], Yegge remains infectious about the future, asserting that while you don't need to know syntax anymore, you still need deep technical concepts to effectively guide the "village of agents" that will soon be building our software [34:47].
John Schulman on dead ends, scaling RL, and building research institutions
A conversation from Cursor CEO with Thinking Machines chief scientist John Schulman on the first year LLMs could have been useful, building research teams, and where RL goes from here.
Bill Dally - Trends in Deep Learning Hardware
In this talk, Bill Dally walks through the evolving landscape of hardware designed for deep learning, explaining how computational demands of neural networks drive innovation in processors and systems. He highlights the gap between traditional CPUs and what modern AI workloads require — massive parallelism, high throughput, efficient data movement — and shows how specialized hardware (like GPUs and newer AI accelerators) addresses those needs. He also outlines architectural trends and design considerations for future deep-learning hardware: energy efficiency, memory bandwidth, and optimized data flow being key. Overall, the video offers a clear, forward-looking perspective on why and how hardware must evolve to keep up with rapidly advancing AI.
Richard Sutton – Father of RL thinks LLMs are a dead end
In this particular episode, host Dwarkesh Patel interviews Richard Sutton, often dubbed the “father of reinforcement learning.” They discuss Sutton’s views on the current trajectory of large language models (LLMs), why he believes LLMs might be a “dead end” in AI research, and how reinforcement learning (RL) still holds promise for building more generalizable, decision-making systems. The conversation combines deep technical insights with philosophical arguments about AI’s future and the kinds of problems that remain unsolved.
Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)
In this lecture from Stanford’s CS229 course, the speaker offers a high-level walkthrough of how large language models are built, touching on more than just model architecture. Key topics include pre-training vs post-training, the importance of data quality and filtering, evaluation metrics (such as perplexity and human benchmarks), and system implementation (e.g. resource constraints like compute and infrastructure). The talk emphasizes that while novel architecture attracts attention, what often makes the biggest difference in practice are data, evaluation, and systems engineering.
A Visual Guide to Quantization
In this visually rich post, Maarten Grootendorst offers an intuitive and deeply illustrated introduction to quantization in the context of large language models, guiding readers through fundamentals like numerical representation (FP32, FP16, BF16, INT8), symmetric vs. asymmetric quantization, and advanced techniques such as dynamic and static post-training quantization. With over 50 custom visuals, he breaks down key concepts like scale and zero-point calculations, quantization error, clipping strategies, and post-training tweaks (e.g., GPTQ, GGUF) to make compression techniques accessible and engaging—even for those new to the subject.
What is ChatGPT doing...and why does it work?
In this engaging AMA-style presentation, Stephen Wolfram breaks down the core magic behind ChatGPT by illustrating how it constructs language “one word at a time.” He walks viewers through the foundational mechanics—how ChatGPT predicts the next most probable word based on patterns seen in massive text corpora—and explains the surprisingly simple yet powerful processes that allow it to generate human-like responses. Using intuitive analogies and step-by-step reasoning, the video demystifies how ChatGPT models language, why its responses can feel so coherent, and what “makes it work” beyond rote memorization.
Deep Dive into LLMs like ChatGPT
In this three-and-a-half-hour lecture, Andrej Karpathy—former Tesla AI lead and OpenAI founding member—walks a general audience through the inner workings of large language models like ChatGPT. He delves into the full training stack: from preprocessing massive internet text corpora and tokenization to model structure, inference dynamics, fine-tuning techniques, hallucinations, reinforcement learning, memory systems, and model alignment strategies. With clarity and insight, Karpathy shares mental models to understand how LLMs think, their limitations, and how to use them most effectively—making complex AI concepts accessible and actionable.
Andrej Karpathy
Andrej Karpathy’s YouTube channel is a treasure trove for anyone fascinated by AI and deep learning. As a former AI leader at Tesla and founding member of OpenAI, Karpathy brings unparalleled expertise to his content—ranging from foundational tutorials like building GPT models from scratch to deep dives into how large language models actually work. His videos are insightful, technical yet accessible, and feature a unique mix of theory, code walkthroughs, and real-world AI application, making complex concepts feel intuitive and exciting.
