CS 489/698 is an introduction to natural language processing. There is no exam; the course is assignment- and project-based.
Instructors: Freda Shi and Victor Zhong
CS 489: Two assignments (25% each) and a course project, the Interstellar Autocomplete Challenge (50%). The project has an initial proposal (due February 2), a midterm check-in (due February 23), and a final submission (due during the final exam period).
CS 698: Two assignments (25% each), the Interstellar Autocomplete Challenge (same requirements as CS 489, worth 25%), and an NLP-related research project (25%). The research project has a proposal/check-in (5%, due February 23) and a final submission (20%, due during the final exam period).
All projects must be done individually.
Two parallel sections with identical content. You may attend either section, even if registered in the other.
| Week | Monday | Wednesday | Instructor |
|---|---|---|---|
| 1 | Jan 5: Introduction and Fundamentals. Slides | Jan 7: Words: Definition, Tokenization, Morphology. Slides. Readings: Sennrich et al. (2016); Kudo (2018) | Freda |
| 2 | Jan 12: Lexical Semantics and Word Embeddings. Slides. Reading: SLP 3, Chapter 6 | Jan 14: Building a Text Classifier. Slides. Reading: SLP 3, Chapter 6 | Freda |
| 3 | Jan 19: Common Neural Architectures. Slides | Jan 21: Common Neural Architectures (cont.); Assignment 1 out: A1 Description; A1 Data. A1 Kaggle: Task 1; Task 2.1; Task 2.2; Task 3 | Freda |
| 4 | Jan 26: No class (campus closed) | Jan 28: Language Modeling. Slides. Reading: SLP 3, Chapter 3 | Freda |
| 5 | Feb 2: Language Modeling (cont.); Project Initial Proposal due. Readings: Holtzman et al. (2020); HuggingFace LLM Course Ch. 7.6: Training GPT-2 | Feb 4: Neural Language Models and Language Model Analysis. Slides. Readings: Devlin et al. (2019); HuggingFace Masked Language Modeling Tutorial | Freda |
| 6 | Feb 9: Syntax and Context-Free Grammars. Slides. Assignment 1 due. Readings: SLP 3, Chapter 18; SLP 3, Chapter 19 | Feb 11: Language Grounding and Multimodal Language Models. Slides. Assignment 2 out (Feb 13): Decipher PCFG from Transformers: A2 Description; A2 Material. A2 Kaggle: Task 1; Task 2; Task 3 | Freda |
| 7 | No class (reading week) | ||
| 8 | Feb 23: Pretraining. Slides. Midterm Project Check-In due (extended to Mar 1). Reading: Radford et al. (2018) | Feb 25: Instruction Fine-tuning. Slides. Readings: Raffel et al. (2019); Brown et al. (2020); Zhou et al. (2023) | Victor |
| 9 | Mar 2: Reinforcement Learning and Alignment. Slides. Readings: Schulman et al. (2017); Ouyang et al. (2022); Rafailov et al. (2023); Shao et al. (2024) | Mar 4: Test-time Scaling and Inference Compute; Assignment 2 due (Mar 6). Slides. Readings: Wei et al. (2022); Yao et al. (2022); Shinn et al. (2023); Chen et al. (2026) | Victor |
| 10 | Mar 9: Retrieval Augmented Generation. Slides. Readings: Lewis et al. (2020); Karpukhin et al. (2020); Shi et al. (2023) | Mar 11: Advanced Topic: Mixture of Experts. Slides. Readings: Shazeer et al. (2017); Fedus et al. (2021); Zoph et al. (2022); Dai et al. (2024) | Victor |
| 11 | Mar 16: Advanced Topic: Physical Devices and Compute. Slides. Readings: Cunha (2024); Sekar and Subbu (2026); Armbruster (2024); Dao et al. (2022) | Mar 18: Advanced Topic: Agents. Slides. Readings: Green (2017); Jimenez et al. (2024); Xie et al. (2024); Pascanu et al. (2017); Wang et al. (2025) | Victor |
| 12 | Mar 23: History of NLP. Slides | Mar 25: Open QA & High-Level Project Discussions | Victor |
| 14 | Final Projects submission due (1 project for undergrads, 2 for grads) |