Computer Science @ The University of Alabama

CS 665: Intelligent Robotics

Graduate seminar & lab on AI methods for autonomous robots. Topics include probabilistic graphical models, probabilistic logic learning, natural-language grounding, deep learning for perception, task & motion planning, and integrated reasoning–acting systems.

  • Hands-on labs with NAO (humanoid) and Sawyer (collaborative) robots.
  • State-of-the-art readings on scene understanding, commonsense reasoning, and long-horizon planning.
  • Team research project culminating in a demo and paper-style report.

Learning Outcomes

  1. Model uncertainty in robotics using Bayes nets, MDPs/POMDPs, and factor graphs.
  2. Build perception pipelines (detection/segmentation/pose) with modern deep networks.
  3. Ground language to perception and actions for instruction following.
  4. Compose task & motion plans under dynamics, kinematics, and safety constraints.
  5. Execute a research-grade project and communicate results clearly.

Prerequisites

  • Graduate standing; solid Python proficiency.
  • Recommended: Intro AI/ML, linear algebra, probability, and basic ROS experience.

Assessment

  • Reading quizzes & participation — 10%
  • Homework (probabilistic models, perception, planning) — 25%
  • Labs (NAO/Sawyer) — 20%
  • Midterm (concepts + short problems) — 15%
  • Course Project (proposal 5%, milestone 5%, final paper 10%, demo 10%) — 30%

Texts & Resources

  • S. Thrun, W. Burgard, D. Fox, Probabilistic Robotics (selected chapters)
  • L. Kaelbling et al., task & motion planning readings (papers provided)
  • ROS/MoveIt & NAOqi API docs (links on course page)

Weekly Outline (14 weeks)

W1 Course intro; robot architectures; ROS refresher; probability quickstart
W2 Bayes nets, factor graphs; inference; SLAM overview
W3 Decision-making: MDPs/POMDPs; belief tracking; planning under uncertainty
W4 Deep perception I: detection/segmentation; datasets; evaluation
W5 Deep perception II: 3D vision, pose estimation; point clouds
W6 Language grounding; semantic parsing; instruction following
W7 Task planning (HTN/STRIPS); symbolic-geometric integration
W8 Motion planning (RRT*, CHOMP, TrajOpt); constraints; MoveIt labs
W9 Manipulation & contact; grasping; controllers (impedance/adm.)
W10 Human–robot interaction; safety; collaborative strategies
W11 Learning for planning (imitation/RL in robotics)
W12 Long-horizon autonomy; memory; mapping for semantics
W13 Project clinic; reproducibility; ablations & evaluation
W14 Project demos & final presentations

Offerings

CS 484/584: Reinforcement Learning

Undergraduate/graduate course on sequential decision making and learning to act. Emphasis on core theory (MDPs, value functions, policy optimization) and practical deep RL implementations for control, games, and robotics.

Learning Outcomes

  1. Formulate tasks as finite and continuous MDPs; compare model-free vs. model-based methods.
  2. Implement TD learning, Monte-Carlo, Q-learning, and policy gradient algorithms.
  3. Stabilize deep RL with replay, target networks, entropy regularization, and advantage estimation.
  4. Tune and evaluate RL systems with appropriate metrics and ablations.
  5. Apply RL to simulated control (Gymnasium/Isaac Gym) and (optionally) real robots.

Prerequisites

  • Data structures & algorithms; probability/statistics; strong Python/NumPy/PyTorch skills.
  • For 584: additional mathematical maturity and a research-oriented final project.

Assessment

  • Problem Sets (theory) — 20%
  • Programming Assignments (DQN, PPO, etc.) — 35%
  • Midterm Exam — 15%
  • Final Project (proposal 5%, milestone 5%, report 10%, presentation 10%) — 30%

Topics (12–14 weeks)

W1 Intro to RL; bandits; exploration–exploitation; regret basics
W2 MDPs; dynamic programming; value/policy iteration
W3 Monte-Carlo & TD learning; n-step returns; eligibility traces
W4 Q-learning, SARSA; function approximation
W5 Deep Q-Networks; replay buffers; target networks; stability
W6 Policy gradients; REINFORCE; variance reduction; baselines
W7 Actor–critic; Advantage methods; TRPO/PPO
W8 Deterministic PG; DDPG/TD3; continuous control
W9 Model-based RL; planning; world models
W10 Exploration strategies; curiosity; entropy regularization
W11 Imitation learning; offline RL; safety & constraints
W12 Multi-agent RL; partial observability; generalization
W13 Scaling RL; reproducibility; experiment design
W14 Project presentations

Tools

  • Python, PyTorch, Gymnasium (or Isaac Gym/Robotics), Weights & Biases (optional)
  • Compute: local GPU lab or cloud (student credits where available)

Offerings

Open Online Courses

Freely available self-paced modules with videos, labs, and code notebooks.

Computer Vision Foundations (Open)

  • Image formation, filtering, features, geometry, and modern CNNs for recognition.
  • Includes Python notebooks and small projects (classification, keypoint detection, pose).

Spring 2020 materials (archived)

Reinforcement Learning Mini-Course (Open)

  • Short modules on bandits, value-based RL, policy gradients, and continuous control.
  • Starter code for DQN and PPO; experiment tracking templates.

Status: Under preparation.

Robotics Perception & Planning Lab Pack (Open)

  • ROS/MoveIt starter packages, perception nodes, and planning exercises.
  • Designed for simulation-first workflows (Gazebo/Isaac Sim).

Status: Under preparation.