Study notes for Stanford CS329A — Self-Improving AI Agents (Autumn 2025), in English and Chinese.
The whole course is that one loop: generate → verify → amortize. Spend compute at inference to produce many candidates, keep the ones that survive checking, fold them back into the weights so the next round is cheaper. Verification is the bottleneck, and a verifier you can fool turns the loop into reward hacking. Every lecture below sits somewhere on that circuit.
Nine lectures. Each one is written from the actual lecture recording plus a close reading of every assigned paper, not from abstracts. Where the lecture and the paper disagree, the notes say so.
Open the reader → onehr.github.io/cs329-notes — all nine lectures in one page. Use the EN / 中 button to switch language.
Prefer it offline? study-companion.html is the same reader as a single self-contained file — download it, open it, no server and no network needed.
| Lecture | Topic | Papers covered |
|---|---|---|
| 01 | Course Overview | — |
| 02 | Test-Time Compute Scaling | Large Language Monkeys, Monkey Power Laws, Snell et al., Archon |
| 03 | Robust Verification | Cobbe et al., Let's Verify Step by Step, Math-Shepherd, Weaver |
| 04 | Tool Use and Feedback | ReAct, RLEF, Constitutional AI |
| 05 | Reasoning and Planning | LATS, SWiRL, SPRINT, ADaPT, AB-MCTS |
| 06 | Train-Time Scaling and RL | STaR, DeepSeekMath (GRPO), DAPO |
| 07 | Search and Deep Research | AlphaCode, AlphaCode 2, Search-o1 |
| 08 | Agentic Evaluation | METR, GDPval, DeepScholar-Bench |
| 09 | Future Directions | Multiagent Finetuning, DeepSeekMath-V2, Absolute Zero, Intelligence per Watt |
Every lecture note has the same shape:
- Background you need — every term defined on first use, no assumed background
- The papers, one section each, with the figures that carry the argument
- Where the lecture and the paper differ — a table, because they often do
- What carries over — the parts that are still true outside this course
companion/
L01-course-overview.md # English
L01-course-overview-zh.md # 中文
... # 9 lectures × 2 languages
figures/L02 … L09/ # figures referenced by both editions
index.html # the hosted reader (650 KB, figures load as you scroll)
study-companion.html # the offline reader (11 MB, everything inlined)
assets/hero-{light,dark}.{png,svg} # the banner above; build/make_hero.py draws all four
fetch_papers.sh # downloads all 39 papers
fetch_transcripts.sh # downloads the 9 lecture transcripts
The markdown files are readable on their own in any editor that renders images (VS Code, Obsidian, Typora). Both HTML readers are built from those same files — index.html points at companion/figures/, study-companion.html carries the figures inside it.
Papers and transcripts are not in this repo — they belong to their authors. The scripts fetch them:
bash fetch_papers.sh # 39 papers → papers/<lecture>/
bash fetch_papers.sh --list # print the manifest without downloading
bash fetch_transcripts.sh # transcripts → course-transcripts/Both scripts skip what you already have, so re-running them is cheap. Transcripts
need yt-dlp and jq (brew install yt-dlp jq); papers need only curl.
The video Part numbers do not match the syllabus lecture numbers. The recorded series has 9 parts; the syllabus has more lectures. Between Part 6 and Part 7 there is a lecture on open-ended evolution (AlphaEvolve, Darwin-Gödel Machine, ADAS, AI Scientist) that was never recorded. These notes follow the video Part numbers, so they line up with the recordings you can actually watch. Lecture 7 ends with a note explaining this.
Some papers were updated after the lecture was given. METR now reports o3 at about 110 minutes where the lecture cited Claude 3.7 Sonnet at 59 minutes; DeepScholar-Bench's ceiling moved from the 19% quoted in class to a 31% geometric mean. Each lecture's difference table records this, with the paper version the note was checked against.
Figures are cropped from the papers and are reproduced here for study, with the source paper and figure number in every caption. Copyright stays with the original authors. One figure in Lecture 9 (az-learnability-reward.png) was drawn for these notes and is labelled as such.
If you are an author and want a figure removed, open an issue.
Notes and the reader: CC BY 4.0. Scripts: MIT. See LICENSE.
Papers, transcripts and figures are not covered — they belong to their authors. This is an independent set of study notes, not affiliated with or endorsed by Stanford.