AI Prelim

[ckwolfe]

Federated AI Prelim document in full Markdown format with all specified links added:

---

Federated AI Prelim

Revision Dates:

• 4/15/2019, revised 5/7/2020
• Revised 6/30/2021
• Revised 8/14/2023
• Revised 8/15/3024

AI faculty are offering a Federated AI Prelim comprising the following areas and associated faculty members:

• Vision (Darrell, Efros, Malik, Kanazawa)
• Robotics (Abbeel, Levine, Dragan, Goldberg)
• Deep Learning (Abbeel, Levine, Darrell, Canny, Sahai)
• Natural Language (Klein, DeNero)
• AI-Systems (Gonzalez, Keutzer, Stoica, Zaharia)

(2024 examiners in bold).

This is distinct from the “ML Prelim” that has been supported by several ML/Stats faculty and from the earlier CS188/CS189-based AI Prelim offered prior to 2019.

AI faculty currently serving on the Grad Matters committee have responsibility for overseeing the Federated AI Prelim and assigning/scheduling AI faculty (in coordination with department staff) to proctor exams and approve final outcomes. (Darrell is currently serving.)

Two or more faculty serve for each sub-exam; at least one is primarily in the area, and the second may be from a different area and primarily observe and assess performance.

Students choose two of the topic areas listed above. Achieving a high score in an associated course is expected to be predictive of a successful outcome in the exam. Each exam is a 30-minute oral exam. During the first 5 minutes, the student may describe relevant coursework, projects, or research they have completed in the area.

---

Syllabi and Resources

Vision

• Course: CS280
• Textbook: Foundations of Computer Vision by Torralba et al.
• Topics Covered:
  • Key papers and methods, including:
    • Burt and Adelson, Gaussian & Laplacian Pyramids (1983)
    • Canny Edge Detector (1986)
    • Heeger & Bergen ('95)
    • Olshausen & Field
    • Fukushima (1980)
    • LeCun, LeNet-5 (1998)
    • AlexNet, ResNet, FCNs, Pix2Pix, Lowe’s Recognizing Panoramas, Phototourism, Video Google with spatial verification, Video Textures, Gatys (Style Transfer), DUSt3R
  • Additional Resources: Recent talks and papers discussed in weekly “compvision” meetings and historically important CV papers.
• Recommendation: It is “highly recommended” that students have taken or TAed CS280 before attempting the exam.

Robotics (Revised 8/1/2023)

• Topics:
  • MDPs (exact/approximate solving), LQR, Optimization (constrained/unconstrained), Optimization-based Control, Robot Motion Planning, Robot Grasping and Manipulation, Bayes Filters, Kalman Filters, Particle Filters, POMDPs, Model-free RL (on/off-policy), Model-based RL, Inverse RL, human-robot interaction (assistance, communication, coordination)
• Study Links:
  • CS287 Fall 2019
    • Lecture materials until/including the 10/22 lecture (15 lectures)
    • Foundations study handout: Study Handout (up to and including p.16)
  • CS285 Fall 2021
    • Lecture materials (Lectures 1 - 12, 20)
  • Robot Motion Planning:
    • Planning Algorithms by Steve Lavalle (Cambridge U Press, 2006):
      • Chapters: 1 (Overview), 4.1-4.3 (C-Space: Topology, Manifolds, Quaternions, Minkowski Sums), 5.1-5.6 (Sampling-Based Motion Planning, PRM, and RRT), 11.1-11.6 (Sensing: Information Space, Kalman Filter), and 12.1, 12.3.5, 12.5 (Uncertainty: POMDP, SLAM, Sensorless Manipulation)
  • Human-Robot Interaction (CS287H):
    • Reward learning from diverse human inputs: Reward rational choice
    • Intent inference/expression: Intent
    • MDP/POMDP formulations for HRI: Interaction as MDPs/POMDPs
• Amount of Study Material: Approximately 33 lectures.
• Exam Format: Combination of questions that ask students to derive algorithms, illustrate how algorithms work through examples, and determine which algorithms are relevant for a given problem. Strong performance requires clear, effective explanations on a tablet (if over Zoom) or whiteboard.

Deep Learning

• Expectation:
  Students should have a solid knowledge of current deep learning foundations and techniques. The field is vast; the best reference for this is the most recent offerings of 182/282A:
  • CS182 Spring 2023
  • Lecture Videos (Spring 2023)
  • Lecture Videos (Fall 2023)
• Core Knowledge:
  Students should understand:
  • Modern optimization-driven deep learning, including self-supervision, data augmentation, and random initializations.
  • Major families of deep networks: convolutional nets, recurrent nets, transformer-based nets.
  • Generative models, including auto-regressive generation, VAEs, GANs, and diffusion models.
  • Basics of vision and NLP domains, with a strong iterative, adaptive approach despite incomplete understanding.
• Classics in Deep Learning:
  • AlexNet (Krizhevsky, Sutskever, Hinton, 2012)
  • LeNet (LeCun et al., 1989)
  • Attention is All You Need (Vaswani et al., 2017)
  • ResNet (He et al., 2015)
  • LSTM (Hochreiter and Schmidhuber, 1997)
  • Word2vec (Mikolov et al., 2013)

CycleGAN (Zhu et al., 2017)

• StyleGAN v2 (Karras et al., 2019)
• GradCAM (Selvaraju et al., 2016)
• Exam Format:
  • Fall 2024: An interactive multi-part question about deep learning.
  • Fall 2019: Knowledge questions with two randomly drawn lectures and a paper for explanation and deeper exploration.

Natural Language

• Topics Covered:
  • Language Modeling (including Neural LMs), Machine Translation (including Statistical and Neural MT), Speech, Parsing (including Neural Parsing), and Semantics.
  • CS288 Course Site - 18 lectures on these topics.

AI Systems

• Administered by: Joseph Gonzalez and Matei Zaharia
• Prelim Reading List: Provided for 2024.

For further questions, please contact Joseph E. Gonzalez or Matei Zaharia.