ABOUT
HOW IT WORKS
FOLLOW US
FEEDBACK
Discover useful tools, tutorials, and helpful resources organized by category
MiniMind is an ultra-lightweight, open-source large language model project that boasts a version as small as 1/7,000 the size of GPT‑3 — making it feasible to train on a standard single consumer GPU. It includes not just the model architecture (built from scratch in native PyTorch, without relying on high-level libraries) but also full pipelines for mixed-expert modules (MoE), data cleaning, pre-training, supervised fine-tuning (SFT), LoRA tuning, direct preference optimization (DPO), and model distillation. With a vision toward both researchers and beginners, MiniMind also offers a vision-multimodal variant (MiniMind-V) and emphasizes hands-on learning and democratizing access to large-model creation.
nanochat is a minimalist, full-stack implementation of a ChatGPT-style large language model built to be accessible and hackable. The repository combines all phases—tokenization, pretraining, fine-tuning, evaluation, inference, and a simple web UI—into a clean, dependency-light codebase. Karpathy designed it so that with a single script (e.g. speedrun.sh), one can train a conversational model on an 8× H100 node for about four hours (costing ~$100) and immediately interact with it via the web.