Stanford CS229 I Machine Learning I Building Large Language Models (LLMs)

In this lecture from Stanford’s CS229 course, the speaker offers a high-level walkthrough of how large language models are built, touching on more than just model architecture. Key topics include pre-training vs post-training, the importance of data quality and filtering, evaluation metrics (such as perplexity and human benchmarks), and system implementation (e.g. resource constraints like compute and infrastructure). The talk emphasizes that while novel architecture attracts attention, what often makes the biggest difference in practice are data, evaluation, and systems engineering.