Resources
Discover useful tools, tutorials, and helpful resources organized by category
Blog Hamel's Blog
Hamel's Blog is written by Hamel Husain, a machine learning engineer with more than 20 years of experience who has worked at Airbnb and GitHub. The notes cover applied AI engineering, machine learning and data science, with a strong focus on evals: how teams debug, analyze and measure LLM systems. Long-form posts include A Field Guide to Rapidly Improving AI Products, Stop Saying RAG Is Dead, Selecting The Right AI Evals Tool and "It's Hard to Eval" Is a Product Smell, and the blog stays current with AI Evals: Everything You Need to Know and Claude's new auto eval tool from September 2026. A practical source for engineers who ship LLM features and need to know whether they work.
Course Stanford CS224N: Natural Language Processing with Deep Learning
CS224N is Stanford's course on natural language processing with deep learning, taught in Winter 2026 by Diyi Yang and Yejin Choi. It starts with word vectors, neural networks and backpropagation, then moves through language models, transformers and self-attention, pretraining, post-training, prompting and parameter-efficient fine-tuning. The current schedule also covers agents, tool use and retrieval-augmented generation, benchmarking and evaluation, reasoning and interpretability. Slides, notes and assignments are posted on the course site, and lecture videos from the 2024 offering are free on YouTube. A solid path for developers who know Python and basic machine learning and want a structured route from embeddings to modern LLM practice.
Paper False Frontiers: Diagnosing and Mitigating Co-Cheating in Self-Evolving Search Agents
The paper by Meijia Chen, Hao Li and colleagues studies self-evolving search agents, where a proposer writes questions and a solver answers them in a closed training loop. It names a failure mode called co-cheating: the two models increasingly agree on shared errors, so the internal reward rises while pseudo-label correctness stagnates or declines. The authors first try multi-sample verification, then propose CrossFit, which splits the source documents into two groups and scores questions from one group with an auxiliary solver trained only on the other. With Qwen3.5-4B and Qwen3.5-9B, CrossFit cuts false-agreement mass from 6.1% and 8.8% to 3.0% and 3.7%, and improves average results on seven search benchmarks by 8.8 and 8.4 points over standard coupled self-evolution. Useful for anyone building self-improving agents who needs to know when the training signal can be trusted.
Book Deep Learning
Deep Learning is the MIT Press textbook by Ian Goodfellow, Yoshua Bengio and Aaron Courville. The authors describe it as a resource intended to help students and practitioners enter the field of machine learning in general and deep learning in particular. The complete book is available online for free and the authors state that it will remain so; the site also provides the table of contents, exercises, lectures, errata and the notation used in the book. It is worth a place on the shelf of anyone who wants the mathematical background behind the libraries they use.
Book Dive into Deep Learning
Dive into Deep Learning is an interactive deep learning book with code, math and discussions, written by Aston Zhang, Zack C. Lipton, Mu Li and Alex J. Smola with additional chapter authors. It is implemented in PyTorch, NumPy/MXNet, JAX and TensorFlow, and each section is an executable Jupyter notebook, so readers can change the code and tune hyperparameters to see the effect at once. Later additions cover reinforcement learning, Gaussian processes, hyperparameter optimization and transformers for vision and large-scale pretraining. The site says the book has been adopted at 500 universities from 70 countries. A fit for developers who want theory and runnable code side by side.
Blog Chip Huyen's Blog
Chip Huyen's blog covers AI system design and what it takes to bring AI into production. Huyen previously worked on ML tooling at NVIDIA, Snorkel AI and Netflix, taught ML Systems at Stanford, and wrote the books Designing Machine Learning Systems and AI Engineering. The posts are long, structured write-ups such as Building A Generative AI Platform, which starts from the simplest architecture and adds components step by step, Agents, and Common pitfalls when building generative AI applications. A good fit for engineers who build products on top of foundation models and want architecture-level guidance.
Blog colah's blog
colah's blog is the personal site of Christopher Olah, who works on reverse engineering artificial neural networks into human understandable algorithms and is a co-founder of Anthropic and of the journal Distill. The index groups his essays by theme: neural networks in general, recurrent networks, convolutional networks and visualizing neural networks, with pieces such as Understanding LSTM Networks, Calculus on Computational Graphs: Backpropagation, Understanding Convolutions and Neural Networks, Manifolds, and Topology. It also links his later interpretability work, including the Circuits thread on Distill and Transformer Circuits. A strong starting point for developers who want intuition for how networks compute, not just how to train them.
Video AI Safety (CS 2881) Fall 2026 Lecture 3: Modern LLM Training and Inference
This lecture from CS 2881, the graduate AI Safety course taught by Boaz Barak at Harvard in Fall 2026, explains how modern large language models are trained and how inference works. The course covers challenges in the alignment and safety of AI, considering both technical questions and the societal impact of the field, and its lectures are recorded and published online. A clear, current university-level explanation of the LLM training and inference stack for developers who want the fundamentals behind the tools they use.
Video 2026 in LLMs (so far) - Simon Willison
In this closing keynote at WeAreDevelopers World Congress North America in San Jose, Simon Willison gives a chronological tour of what happened in large language models during 2026. He covers the late-2025 point where coding agents became reliable for daily use, the rise of Claw-style agent software, new model releases from several labs, rogue agents breaking containment during training, notable security incidents, and the progress of local models. A fast way for developers to catch up on the year's most important shifts in AI tooling.
Podcast Latent Space: The AI Engineer Podcast
Latent Space is a podcast and newsletter for AI engineers hosted by swyx and Alessio, describing itself as covering how leading labs build agents, models, infrastructure and AI for science. Episodes are long technical conversations with people building the field, and past guests include Greg Brockman, Andrej Karpathy, George Hotz, Simon Willison and Soumith Chintala. It has more than 200,000 subscribers and suits developers who build on top of LLMs and want to hear how practitioners make engineering decisions.
Paper Context Language Models
The paper by Rulin Shao, Mike Lewis, Luke Zettlemoyer, Pang Wei Koh and colleagues introduces Context Language Models, which manage their own context by treating it as a file they can update, so the model decides which information is worth keeping. The authors report 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, show that the approach works in multi-agent systems, and cover both in-context and parametric learning of context-management strategies. They also present an online reinforcement learning method and Suffix Cache Reuse, which cuts server-side compute by 35% compared with standard methods. A good read for anyone working on long-horizon agents and context engineering.
Paper RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
The paper from a Google team looks at methods that automatically evolve an LLM agent's harness, meaning the prompts, control flow, tools, memory and context management around a frozen model, and shows that this recursive self-improvement tends to overfit the tasks it trains on. RRSI adds regularization to the loop: the proposer works with an annealed budget on how many edits a candidate can bundle and favours unexplored directions, while a critic screens benchmark-specific proposals and a pruner removes changes that are too small, too costly or no longer useful. Across eight coding, agentic workspace and engineering design benchmarks it gains up to 14.1 points in-distribution and up to 4.7 points out-of-distribution, with a harness that uses 30% fewer policy tokens. Useful reading for anyone tuning agent scaffolding automatically.
Course Stanford CS336: Language Modeling from Scratch
CS336 is a Stanford course taught by Tatsunori Hashimoto and Percy Liang in which students build their own language model end to end. It covers the entire development process: collecting and cleaning pre-training data, constructing the transformer, training the model, and evaluating it before deployment. The lecture videos are public on YouTube and the course materials are on the course site. One of the most thorough free paths for engineers who want to understand LLMs by building one rather than only calling an API.
Blog Eugene Yan
Eugene Yan, a Member of Technical Staff at Anthropic, writes about building AI-powered products: recommendation systems, large language models, evaluation, engineering practices and leadership. His posts are practical and grounded in shipping real systems, with recent pieces on patterns for building cybersecurity evals, using LLMs to secure source code, how to work and compound with AI, and product evals in three simple steps. A strong resource for engineers who need to move LLM features from prototype to a reliable product.
Blog Ahead of AI
Ahead of AI is the newsletter of Sebastian Raschka, PhD, focused on machine learning and LLM research for researchers and practitioners. Posts walk through new model architectures and research papers in depth, with recent issues covering LLM architecture developments such as KV sharing, mHC and compressed attention, controlling reasoning effort, local coding agents, building an AI text detector from scratch, and a curated list of the 2026 LLM research papers. It is written by the author of Build a Large Language Model (From Scratch) and suits developers who want to keep up with research without reading every paper themselves.
Book Neural Networks and Deep Learning
Neural Networks and Deep Learning is a practical and approachable introduction to how modern neural networks work, written with a strong focus on intuition rather than heavy theory. The book walks readers through the core ideas behind deep learning — including perceptrons, backpropagation, gradient descent, overfitting, convolutional networks, and recurrent networks — while gradually building real implementations from scratch in Python. Instead of treating AI as a black box, it explains why neural networks learn, how training actually happens, and what design choices improve performance, making it especially valuable for programmers and self-learners who want to understand the foundations behind today’s AI systems without needing an advanced mathematics background.
Paper LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
The paper LeWorldModel introduces a new way to train AI “world models” directly from raw pixels in a simple and stable manner, addressing a key problem in prior approaches where models collapse or require complex tricks to work. It proposes a streamlined architecture using just two loss terms—one to predict future states in a latent space and another to keep those representations well-structured—eliminating the need for heavy tuning or pre-trained components. Despite being lightweight (about 15M parameters and trainable on a single GPU), the model achieves competitive performance across various control tasks while planning much faster than larger foundation-model-based systems, and it also learns meaningful physical structure, allowing it to detect unrealistic or physically implausible scenarios.
Book Language Models Interview Handbook
This book is a practical guide to understanding and preparing for large language model (LLM) engineering roles, focusing not just on theory but on how LLMs work as complete systems in real-world applications. It walks through the full stack—from tokenization, embeddings, and attention to retrieval (RAG), evaluation, deployment, and optimization—while emphasizing how these components interact to deliver reliable, scalable AI products. Rather than memorizing concepts, the book trains readers to think like engineers: framing problems, making trade-offs between cost, latency, and accuracy, and explaining design decisions clearly in interviews. It also covers common patterns like hybrid search, reranking, and context management, along with career guidance such as resume building and communicating impact, ultimately aiming to help readers bridge the gap between LLM theory and production-ready systems.
Paper Attention Residuals
The paper introduces Attention Residuals, a new approach to replacing the traditional residual connections used in Transformer models, which typically add layer outputs together in a fixed and uniform way. Instead of this rigid accumulation, the authors propose using an attention mechanism across layers, allowing each layer to selectively retrieve and combine information from earlier layers based on the input. This helps address issues like information dilution and inefficient scaling in deep networks, enabling models to better preserve useful representations as they grow deeper. To make the method practical, the paper also introduces a more efficient block-based variant, and experiments show that this approach can improve performance and training behavior with minimal overhead, making it a promising drop-in improvement for modern large language models.
Book Understanding Deep Learning
Understanding Deep Learning by Simon J. D. Prince is a comprehensive and accessible introduction to the core ideas behind modern deep learning. The book focuses on explaining why neural networks work rather than just how to implement them, gradually building from basic concepts like supervised learning and simple neural networks to more advanced topics such as deep architectures and generative models. With clear explanations and visual intuition, it aims to help readers develop a solid conceptual understanding of deep learning systems and the principles that power many of today’s AI applications.
Paper Multi-agent cooperation through in-context co-player inference
This paper explores how cooperation can emerge among self-interested agents in multi-agent reinforcement learning without hardcoded assumptions about how other agents learn; instead of using specialized meta-learning or explicit timescale separations, the authors show that training sequence-model agents against a diverse mix of co-players naturally leads them to infer and adapt to their partners’ strategies within an episode, producing in-context best-response behaviors that, under mutual pressure to shape each other’s learning dynamics, lead to cooperative outcomes, suggesting a scalable way to achieve cooperation using standard decentralized RL with sequence models.
Blog Zartbot
The zartbot/blog repository functions as a technical knowledge base and discussion forum primarily focused on cloud-native networking, high-performance computing (HPC), and system-level optimization. The issues section serves as a collection of long-form articles and engineering deep dives where the author, zartbot (a notable engineer in the networking and AI infrastructure space), explores advanced topics such as SD-WAN architecture, RDMA/RoCE virtualization, BGP and cloud-scale routing protocols, and GPU networking for AI training. By leveraging GitHub Issues as a blogging platform, the repository facilitates technical discourse on specialized subjects like NetDAM (Network Direct Attached Memory) and the trade-offs of the CAP theorem in modern distributed systems, catering to a professional audience of network architects and system programmers.
Paper Deep Delta Learning
The paper introduces Deep Delta Learning (DDL), a new neural network architecture that extends traditional deep residual connections by replacing the fixed identity shortcut with a learnable, data-dependent geometric transformation called the Delta Operator. By parameterizing this operator with a direction vector and a gating scalar, the model can smoothly interpolate between identity mapping, projection, and reflection behaviors, giving the network fine-grained control over how information is erased and written at each layer. This enables it to represent more complex, non-monotonic dynamics than standard residual networks while retaining stable training, effectively broadening the expressive power of deep models without losing the benefits of residual learning.
Paper mHC: Manifold-Constrained Hyper-Connections
The paper "mHC: Manifold-Constrained Hyper-Connections" by DeepSeek-AI introduces a novel architectural framework designed to overcome the training instabilities inherent in high-complexity "Hyper-Connections." While traditional residual streams rely on identity mapping to maintain signal stability, expanding these connections often leads to unbounded signal amplification that causes models to crash during scaling. To solve this, the authors propose mHC, which utilizes manifold projection and doubly stochastic matrices to ensure feature preservation across layers, effectively restoring stability without sacrificing topological diversity. Empirical tests on models up to 27B parameters demonstrate that mHC achieves superior performance on benchmarks like BigBench Hard with minimal computational overhead, offering a more robust path for scaling future foundational models.
Video Steve Yegge's Vibe Coding Manifesto
In this interview from the AI Engineer Summit, legendary developer Steve Yegge lays out his "Vibe Coding Manifesto," arguing that the industry is shifting from traditional manual coding to a "factory farming" model driven by AI agents. Yegge claims that senior engineers are often the most resistant to this change because their identities are tied to old workflows, even though early adopters are seeing productivity gains as high as 10x [03:09]. He envisions a future where the IDE is replaced by an "agent orchestration dashboard" [14:11], where developers act more like high-level product managers or "NASCAR drivers" who manage dozens of agents simultaneously rather than writing individual lines of code [10:11]. While acknowledging the current chaos and execution problems within major labs like OpenAI and Google [29:48], Yegge remains infectious about the future, asserting that while you don't need to know syntax anymore, you still need deep technical concepts to effectively guide the "village of agents" that will soon be building our software [34:47].
Video John Schulman on dead ends, scaling RL, and building research institutions
A conversation from Cursor CEO with Thinking Machines chief scientist John Schulman on the first year LLMs could have been useful, building research teams, and where RL goes from here.
Video Bill Dally - Trends in Deep Learning Hardware
In this talk, Bill Dally walks through the evolving landscape of hardware designed for deep learning, explaining how computational demands of neural networks drive innovation in processors and systems. He highlights the gap between traditional CPUs and what modern AI workloads require — massive parallelism, high throughput, efficient data movement — and shows how specialized hardware (like GPUs and newer AI accelerators) addresses those needs. He also outlines architectural trends and design considerations for future deep-learning hardware: energy efficiency, memory bandwidth, and optimized data flow being key. Overall, the video offers a clear, forward-looking perspective on why and how hardware must evolve to keep up with rapidly advancing AI.
Paper Detect Anything via Next Point Prediction
In this work, the authors propose a model called Rex‑Omni — a 3-billion-parameter multimodal large language model (MLLM) — which reframes a wide range of visual-perception tasks (object detection, spatial grounding, key-pointing, OCR) as a “next-point prediction” problem. Instead of the traditional regression-based coordinate prediction widely used in detectors, Rex-Omni quantises spatial coordinates into discrete tokens and asks the model to predict the next token (or point) in a sequence, allowing detection and localization to be treated in the same token-prediction paradigm as language modelling. The paper argues that this unification leads to both stronger language-vision alignment and improved localization performance. They support their claims with evaluations on standard benchmarks (such as COCO) and show that Rex-Omni outperforms prior methods when given the same or less supervision. Beyond just detection, the design opens up a path toward more general “detect-anything” models where a single architecture handles many tasks.
Paper Recurrent Neural Network Regularization
The paper Recurrent Neural Network Regularization by Zaremba, Sutskever, and Vinyals (2014) introduces a novel approach to applying dropout in Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units. Traditional dropout techniques, effective in feedforward networks, often fail in RNNs due to the recurrent connections amplifying noise, which can hinder learning. The authors propose a method to apply dropout selectively within LSTMs, demonstrating its efficacy in reducing overfitting across various tasks, including language modeling, speech recognition, image caption generation, and machine translation.
Paper DeepSeek-OCR: Contexts Optical Compression
The paper introduces DeepSeek-OCR, a vision-language model designed to go beyond traditional OCR by extracting text from images and understanding visual context such as layout, semantics, and multimodal relationships. It presents a mixed-expert architecture where only relevant sub-modules activate, enabling efficient processing of high-resolution document images and complex visuals. The authors describe their training system, datasets spanning screenshots, PDFs, and scene text, and evaluate the model on benchmark tasks where it demonstrates strong performance in text extraction, document understanding and layout reasoning. The paper concludes that the unified vision + language approach offers improved accuracy and flexibility for modern OCR applications.
Github MiniMind
MiniMind is an ultra-lightweight, open-source large language model project that boasts a version as small as 1/7,000 the size of GPT‑3 — making it feasible to train on a standard single consumer GPU. It includes not just the model architecture (built from scratch in native PyTorch, without relying on high-level libraries) but also full pipelines for mixed-expert modules (MoE), data cleaning, pre-training, supervised fine-tuning (SFT), LoRA tuning, direct preference optimization (DPO), and model distillation. With a vision toward both researchers and beginners, MiniMind also offers a vision-multimodal variant (MiniMind-V) and emphasizes hands-on learning and democratizing access to large-model creation.
Video Richard Sutton – Father of RL thinks LLMs are a dead end
In this particular episode, host Dwarkesh Patel interviews Richard Sutton, often dubbed the “father of reinforcement learning.” They discuss Sutton’s views on the current trajectory of large language models (LLMs), why he believes LLMs might be a “dead end” in AI research, and how reinforcement learning (RL) still holds promise for building more generalizable, decision-making systems. The conversation combines deep technical insights with philosophical arguments about AI’s future and the kinds of problems that remain unsolved.
Podcast Dwarkesh Podcast
Dwarkesh Podcast, hosted by Dwarkesh Patel, features deeply researched interviews and conversations on artificial intelligence, robotics, cognition, technology, and future thinking. The show often invites experts and thinkers—such as Andrej Karpathy and Richard Sutton—to discuss their perspectives on AGI, reinforcement learning, scaling, and paradigm shifts in AI. Episodes blend technical depth with broader vision and speculation, making it appealing to listeners who want to explore both practical and philosophical dimensions of the AI frontier.
Github Nanochat
nanochat is a minimalist, full-stack implementation of a ChatGPT-style large language model built to be accessible and hackable. The repository combines all phases—tokenization, pretraining, fine-tuning, evaluation, inference, and a simple web UI—into a clean, dependency-light codebase. Karpathy designed it so that with a single script (e.g. speedrun.sh), one can train a conversational model on an 8× H100 node for about four hours (costing ~$100) and immediately interact with it via the web.
Paper POISONING ATTACKS ON LLMS REQUIRE A NEAR-CONSTANT NUMBER OF POISON SAMPLES
In this paper, the authors challenge the prevailing assumption that poisoning large language models (LLMs) becomes harder as model size and training data scale up. Through large-scale experiments on models ranging from 600 million to 13 billion parameters, they show that injecting only a fixed, near-constant number of malicious training samples (e.g. ~250) can successfully implant backdoors regardless of how much clean data the model sees. They further demonstrate that this phenomenon holds both in pretraining and fine-tuning settings, suggesting that backdoor attacks could be more broadly feasible than previously thought. The findings raise serious implications about how we assess and defend against data poisoning in future, larger models.
Paper The Annotated Transformer
The Transformer uses self-attention and feed-forward layers in stacked encoder and decoder blocks to process sequences, unlike previous models relying on RNNs or convolutions, achieving parallel processing and reducing computational complexity.
Paper Video models are zero-shot learners and reasoners
The authors explore how a modern generative video model, Veo 3, demonstrates a surprising level of zero-shot capability across many visual tasks it never explicitly trained for. Tasks include segmentation, edge detection, image editing, understanding physical dynamics, affordance recognition, and even reasoning puzzles like maze solving and symmetry completion. They argue that these emergent abilities suggest video models are evolving toward generalist vision foundation models, much like how LLMs transformed language understanding. Their experiments support a hierarchy of visual competence — from perception to modeling to manipulation to reasoning — showing that Veo 3 can “see, model, act, and reason” with visual content in ways that hint at future unified vision systems.
Blog Lil’Log
Lil’Log (lilianweng.github.io) is the personal blog of Lilian Weng, where she documents her explorations in AI, machine learning, and deep learning research. She publishes well-researched, thoughtful essays on topics like large language models, diffusion models, generative AI, hallucination in LLMs, reinforcement learning, model inference, and more. The posts often include background theory, recent progress, tradeoffs, and references to academic papers — making it a bridge between cutting-edge research and accessible discussion for technical audiences.
Paper DeepSeek-R1: incentivizes reasoning in LLMs through pure reinforcement learning
DeepSeek-R1 demonstrates that large language models can be trained to develop advanced reasoning abilities using reinforcement learning (RL) rather than depending on human-annotated reasoning traces. By building on DeepSeek-V3 Base and using a method called Group Relative Policy Optimization (GRPO), the authors use only final prediction correctness as the reward signal—without instructing the specific reasoning path. The result is that the model learns emergent behaviors like self-reflection, verification steps, and dynamic adaptation of strategy. DeepSeek-R1 outperforms models trained with supervised learning using human demonstrations in tasks involving mathematics, coding, and STEM reasoning. The supplementary materials (ESM1) provide detailed experimental setups, dataset details (including how they handle decontamination of evaluation data), and additional evaluation metrics that support the main claims.
Blog Shunyu Yao’s site
Shunyu Yao’s site showcases his work as a researcher at OpenAI, focusing on “agents” — systems that perceive, reason, and take actions in digital environments. The site features his selected publications, including papers such as ReAct, Tree of Thoughts, Deep Research, and SWE-agent, along with links to repo, talks, benchmarks, and blog posts. It reflects an overarching interest in how language models can be structured not just to predict text, but to act, interact, evaluate, and generalize — moving toward agents that integrate reasoning and action. The writing and work are forward-looking, engaging questions about what constitutes evaluation, how digital environments should be designed, and what the “second half” of AI might look like.
Course Prompt Engineering Guide
Prompting Guide is a comprehensive online resource created by DAIR.AI that gathers up-to-date theory and best practices for designing effective prompts when working with large language models (LLMs). It covers the basics of prompting—such as instructions, context, input data, and output formats—while providing more advanced techniques like function calling, context caching, synthetic dataset generation, and handling model risk (bias, factuality, etc.). Users can find example prompts (for tasks like summarization, reasoning, QA, coding), model-specific guidance, tools, notebooks, and research readings, all intended to help developers, researchers, and AI practitioners get better results and understand LLM capabilities more deeply.
Video Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)
In this lecture from Stanford’s CS229 course, the speaker offers a high-level walkthrough of how large language models are built, touching on more than just model architecture. Key topics include pre-training vs post-training, the importance of data quality and filtering, evaluation metrics (such as perplexity and human benchmarks), and system implementation (e.g. resource constraints like compute and infrastructure). The talk emphasizes that while novel architecture attracts attention, what often makes the biggest difference in practice are data, evaluation, and systems engineering.
Book Build a Large Language Model (From Scratch)
In Build a Large Language Model (From Scratch), Sebastian Raschka leads readers on a hands-on journey through designing, coding, training, and fine-tuning an entire large language model using publicly available tools. Beginning with planning and preparing your data, the book walks you through implementing the model architecture, loading pretrained weights, and adapting the model for tasks like text classification or instruction-following. Even if you’re working on a laptop, it shows how you can build a GPT-style LLM, while also discussing scaling options. With clear explanations, diagrams, code examples, and practical exercises, the book aims to demystify what goes on “inside” generative AI and help you really understand what makes these models work.
Blog Anthropic Engineering Blog
At Anthropic, engineering centers on building AI systems that are reliable, interpretable, and steerable—especially through things like agents and tool integrations. Their team publishes about how they enable Claude (their AI model) to use tools well, optimize performance, and evaluate tools rigorously. They also talk about internal research efforts like scaling interpretability, making features behaving predictably, and improving developer workflows. Their engineering blog shows not just what they build, but how they think about safety, feedback loops, and tooling to support clean, robust, agent-based AI systems.
Blog Thinking Machines Blog
Thinking Machines Lab is an AI research & product company founded in 2025 by Mira Murati. Their blog—titled “Connectionism”—serves as a public window into their work, promising technical posts, research, and code releases. Their mission is to make frontier AI systems easier to understand, customize, and reliably determinable (e.g. tackling nondeterminism in inference), while emphasizing human-AI collaboration, model intelligence, infrastructure quality, and safety.
Podcast a16z Podcast
The a16z Podcast, produced by the venture capital firm Andreessen Horowitz (a16z), explores the intersection of technology, culture, and the future, with a recurring theme that “software eats the world.” Featuring conversations with industry experts, business leaders, and innovative thinkers, the show offers rich insights into emerging trends, startup dynamics, and the evolving digital landscape. Multiple episodes drop each week, making it a timely resource for anyone curious about where tech is headed.
Paper Mixtral of Experts
Mixtral 8x7B introduces a Sparse Mixture of Experts (SMoE) language model built on the Mistral 7B architecture, but enhanced with eight feedforward “experts” per layer. A lightweight router activates just two of these experts per token during inference, giving each token access to a massive 47 billion parameters while only actively using 13 billion—striking an effective balance between capacity and efficiency (47B vs 13B). Trained with a context size of 32K tokens, Mixtral outperforms or matches the performance of Llama 2 70B and GPT-3.5 across benchmarks—especially excelling in math, code generation, and multilingual tasks. Its instruction-tuned variant, Mixtral 8x7B – Instruct, surpasses leading models like GPT-3.5 Turbo, Claude-2.1, Gemini Pro, and Llama 2 70B-chat on human evaluation tests. Both models are released under an Apache 2.0 license and accompanied by open-source tooling integrations.
Paper Scaling Laws for Neural Language Models
The paper empirically uncovers robust power-law relationships between a language model’s performance (measured via cross-entropy loss) and three core factors: model size, dataset size, and compute budget. Remarkably, these trends hold across more than seven orders of magnitude, while architectural variations like layer depth or width exert minimal impact. The authors further derive simple formulas that predict overfitting behavior and optimal compute allocation—revealing that larger models are significantly more sample-efficient, and that achieving compute-efficient training often means using very large models trained on modest data volumes and stopping well before full convergence.
Paper Attention Is All You Need
The paper introduces the Transformer, an innovative neural network architecture that relies entirely on attention mechanisms—discarding both recurrence and convolution—delivering faster training and superior performance. Tested on machine translation tasks, the model achieves a remarkable 28.4 BLEU score on English-to-German and sets a new state-of-the-art 41.8 BLEU on English-to-French translation, all while training with significantly less compute time. The authors also demonstrate the model’s versatility by applying it successfully to constituency parsing, showcasing its efficiency, parallelism, and adaptability across diverse tasks.
Video A Visual Guide to Quantization
In this visually rich post, Maarten Grootendorst offers an intuitive and deeply illustrated introduction to quantization in the context of large language models, guiding readers through fundamentals like numerical representation (FP32, FP16, BF16, INT8), symmetric vs. asymmetric quantization, and advanced techniques such as dynamic and static post-training quantization. With over 50 custom visuals, he breaks down key concepts like scale and zero-point calculations, quantization error, clipping strategies, and post-training tweaks (e.g., GPTQ, GGUF) to make compression techniques accessible and engaging—even for those new to the subject.
Course Neural Networks by 3Blue1Brown
Grant Sanderson’s (3Blue1Brown) Neural Networks playlist offers an elegant and visual exploration of how neural networks learn—from the basics of node-to-node processing and gradient descent to backpropagation and feature learning—using rich animations and intuitive storytelling that make complex concepts feel both beautiful and approachable.
Course Stanford CS231N Deep Learning for Computer Vision
This free YouTube playlist compiles the latest edition of Stanford’s legendary CS 231N: Deep Learning for Computer Vision course, presenting a thorough, lecture-by-lecture exploration of how neural networks interpret and generate images. From core concepts like convolutional layers and optimization techniques to modern techniques in object detection, generative modeling, and explainability, the course is tailored for both students and professionals looking to deepen their understanding of cutting-edge computer vision methods.
Blog Simon Willison’s Weblog
Simon Willison—co-creator of the Django web framework and creator of the open-source tool Datasette—has been blogging since 2002 about web development, data journalism, AI tooling, and LLMs. His site features fast, insightful entries—including project demos like EmbeddingGemma and semantic tools like the “Semantic Galaxy”—plus thoughtful reflections on topics like prompt engineering and AI agents. Willison also publishes both free and paid newsletters, maintains all his posts on a custom Django-powered site, and continues contributing to open-source and tech communities.
Blog Andrej Karpathy
Andrej Karpathy’s Blog is the personal website of Andrej Karpathy, an AI researcher and educator known for his work in deep learning and large language models. The blog features a mix of technical content, personal reflections, and educational resources. Notably, Karpathy offers a self-hosted, minimalist blog experience, emphasizing simplicity and directness in his writing. His posts cover topics such as the philosophy of technology, personal anecdotes, and insights into AI development. Additionally, Karpathy has launched Eureka Labs, an AI+Education company, and shares educational videos on his YouTube channel, focusing on deep learning and large language models.
Video What is ChatGPT doing...and why does it work?
In this engaging AMA-style presentation, Stephen Wolfram breaks down the core magic behind ChatGPT by illustrating how it constructs language “one word at a time.” He walks viewers through the foundational mechanics—how ChatGPT predicts the next most probable word based on patterns seen in massive text corpora—and explains the surprisingly simple yet powerful processes that allow it to generate human-like responses. Using intuitive analogies and step-by-step reasoning, the video demystifies how ChatGPT models language, why its responses can feel so coherent, and what “makes it work” beyond rote memorization.
Video Deep Dive into LLMs like ChatGPT
In this three-and-a-half-hour lecture, Andrej Karpathy—former Tesla AI lead and OpenAI founding member—walks a general audience through the inner workings of large language models like ChatGPT. He delves into the full training stack: from preprocessing massive internet text corpora and tokenization to model structure, inference dynamics, fine-tuning techniques, hallucinations, reinforcement learning, memory systems, and model alignment strategies. With clarity and insight, Karpathy shares mental models to understand how LLMs think, their limitations, and how to use them most effectively—making complex AI concepts accessible and actionable.
Video Andrej Karpathy
Andrej Karpathy’s YouTube channel is a treasure trove for anyone fascinated by AI and deep learning. As a former AI leader at Tesla and founding member of OpenAI, Karpathy brings unparalleled expertise to his content—ranging from foundational tutorials like building GPT models from scratch to deep dives into how large language models actually work. His videos are insightful, technical yet accessible, and feature a unique mix of theory, code walkthroughs, and real-world AI application, making complex concepts feel intuitive and exciting.
Course Practical Deep Learning
A free course designed for people with some coding experience, who want to learn how to apply deep learning and machine learning to practical problems.
Course DeepLearning.ai
DeepLearning.AI is a pioneering education technology company founded by AI innovator Andrew Ng in 2017. Its mission is to equip a global audience—from aspiring AI professionals to non-technical learners—with high-quality, hands-on training in artificial intelligence. Through widely respected programs on Coursera like the Deep Learning Specialization and AI for Everyone, DeepLearning.AI has helped millions build tangible machine learning skills, supported by a collaborative community and a growing ecosystem of AI tools and partnerships.
