Harvard CS2680:
A Free Course on AI Agents and the Systems Under Them
One agentic system, followed from the agent loop all the way down to the GPU. The slides, readings and assignments are public. Here is what the course covers, how it is built, and how to work through it on your own.
Updated: October 26, 2026 · Term: Fall 2026 · Cost to follow: free, bring your own compute
At a glance
- Course: CS2680 Modern AI Systems: Agents and System Optimizations, Harvard John A. Paulson School of Engineering and Applied Sciences, Fall 2026.
- Taught by: Juncheng Yang, with four teaching fellows, supported by Harvard's MaDSys (Measurement and Design of Systems) research group.
- The arc: use a coding agent, build your own, optimize it, then serve the model yourself and optimize the whole stack underneath it.
- Level: graduate. It assumes CS61, at least one graduate systems course, and comfort with Python and PyTorch.
- Free to follow: slides, readings, reference notes and full assignment specs are on the public site. Compute, grading and staff support are for enrolled Harvard students only.
What it teaches
Most AI courses stop at either the model or the application. CS2680 follows a single agentic system through every layer between them. In the course's own words, students "first use a modern coding agent (Claude Code), then design your own agent using provided API access, measure it, and optimize it. Then you serve an open-weight model yourself and take responsibility for the layers underneath: batching and scheduling, KV-cache and prefix reuse, routing, quantization, and speculative decoding."
How coding agents work, context management and tool design, multi-agent architecture, memory and reliability, evaluation, cost, and agent safety.
GPUs, batching, paging and parallelism, speculative decoding, scheduling and prefix caching, pruning and quantization, KV-cache design, routing and load balancing.
Five assignments, one system
The assignments are the heart of the course and its most unusual design choice. They are not five separate exercises: each one takes apart and replaces a layer of the same agent. "Mads" stands for Modern Agent Design and Systems.
Mads-Lens: watch the agent work
Use Claude Code to build a web UI for Claude Code itself: submit a prompt, watch every tool call stream live, see subagents branch into a tree, and see what each run cost.
Estimated effort: 4 to 6 hours · Weight: 10%
Mads-Loop: build the loop
Write your own agent loop around a model API: tool schemas, context management, retries, termination, and graceful failure when output is malformed or tools break.
Estimated effort: 10 to 12 hours · Weight: 10%
Mads-Opt: make every token count
With the model still a black-box API, cut tokens and calls, pick cheaper models where they suffice, overlap tools and restructure context, holding task success fixed.
Estimated effort: 10 to 15 hours · Weight: 20%
Mads-Serve: own the serving engine
Drop the frontier API and serve an open model yourself on one 32 GB GPU, run your agent against it, and make it faster by changing the engine or the agent.
Estimated effort: 10 to 14 hours · Weight: 20%
Mads-Stack: optimize the full stack
Everything is in scope. Profile the whole path, write a performance hypothesis first, then optimize across agent, model and serving layers and measure what mattered.
Estimated effort: 12 to 18 hours · Weight: 10%
Assignments 3 and 5 are submitted as published blog posts about the optimization journey, and a public leaderboard ranks the serving work on tasks solved and end-to-end time. A final project (20%) lets students choose their own problem. There is no exam. Full specs are on the assignments page.
The schedule
Lectures run Monday and Wednesday from September 2 to December 7, 2026. The topics, with the course's tentative dates:
| Date | Part I: LLMs and agents |
|---|---|
| Sep 2 | Course overview and logistics |
| Sep 9 | Using agents I: how Claude Code works |
| Sep 14 | Using agents II: the 100x engineer with multiple agents |
| Sep 16 | Agent design I: context management and tool design |
| Sep 23 | Agent design II: multi-agent architecture, coordination and communication |
| Sep 28 | Agent design III: memory and reliability |
| Sep 30 | Agent design IV: evaluation |
| Oct 5 | Guest lecture (Snowflake): developing and evaluating agents in the enterprise |
| Oct 7 | Advanced agent design I: cost and performance |
| Oct 14 | Advanced agent design II: agent safety and self-evolving harnesses |
| Date | Part II: Systems for LLMs and agents |
|---|---|
| Oct 19 | Efficient LLM serving I: introduction to LLMs |
| Oct 21 | Efficient LLM serving II: introduction to GPUs |
| Oct 28 | Efficient LLM serving III: batching, paging and parallelism |
| Nov 2 | Speculative decoding |
| Nov 4 | Efficient agentic systems: scheduling and prefix caching |
| Nov 9 | Guest lecture (Meta): why infrastructure sets the pace of AI |
| Nov 11 | Pruning and quantization |
| Nov 16 | KV-cache optimization and model architectures |
| Nov 23 | Efficient agentic systems: routing and load balancing |
| Dec 7 | Final project posters and live demos |
Student-led sessions on paper discussions and assignment results run alongside these. Slide PDFs are linked from each lecture on the course site.
The reading list
Each lecture links to readings, and most of them are freely available papers. Ten that anchor the course:
- Attention Is All You Need: the Transformer.
- ReAct: reasoning and acting in a loop, the basis of tool-using agents.
- MemGPT: memory management for long-running agents.
- PagedAttention (vLLM): paging the KV cache to serve more requests.
- SGLang: structured programs and prefix reuse in serving.
- Speculative decoding: drafting tokens cheaply and verifying them in parallel.
- Grouped-query attention: shrinking the KV cache with shared key and value heads.
- AWQ: activation-aware weight quantization.
- Mixtral of Experts: sparse mixture-of-experts models.
- The Llama 3 Herd of Models: a modern open-weight model, end to end.
No textbook is required. For background, the course points to four free books:
- Operating Systems: Three Easy Pieces (computer systems)
- Machine Learning Systems (ML systems)
- Foundations of Large Language Models (large language models)
- The Little Book of Deep Learning (deep learning)
Two of the course's own reference notes are worth reading on their own: Agent evaluation and benchmarks and Cost-effective agents.
How to follow it on your own
- Lecture slides (PDF) and weekly readings
- Full specs for all five assignments
- Reference notes and computing setup guides
- The class blog and the leaderboard
- A Claude Code subscription for assignment 1
- Model API access for assignments 2 and 3
- A GPU with 32 GB of memory, or a cloud equivalent, for assignments 4 and 5
- Your own feedback loop: no grading or staff help for non-students
The staff are clear that they "only respond to emails from Harvard students", and auditing in person comes without computing resources. So treat it the way it is offered: as an open syllabus. A realistic pace for a working engineer is one assignment every two to three weeks, using the course's effort estimates of roughly 46 to 65 hours in total for the five. Doing assignments 3 and 5 as public blog posts, as enrolled students do, is a good way to get feedback.
Expect to teach yourself. The course says so directly: it "will not teach you every piece of knowledge you need for the assignments" and focuses instead on "the skills to learn with". AI use is encouraged, as long as it is disclosed.
Our take
This is the course we would point an engineer to who wants to work on AI products and not just call an API. Its best idea is structural: by making students replace each layer of one system in turn, it forces them to see how the agent loop, the token bill and the serving engine affect each other. That connection shows up across our own writing, from the cost of every token in AARRR for AI products to serving and startup costs in Ship the Kernels With the Weights and What a 4GB Model Is For.
It is not a beginner's course. Without systems background and working PyTorch, start with the background books first. With them, it is one of the most practical free syllabi on modern AI systems we have seen.
Source: CS2680 course site, its assignments page and linked reference pages, as of October 26, 2026. Schedule dates are the course's tentative plan and may change. This guide is independent and not affiliated with Harvard. Spot something out of date? Tell us.