01
Rigid-Body Math & ROS2 First Contact
2D/3D rotations · quaternions · axis-angle · Euler failure modes · SO(3) · ROS2 architecture · DDS layer · why ROS1 is dead.
Lab 01
rclpy publisher + rotating quaternion in rviz2
Forty-eight weeks of intentional work on intelligent robotics and autonomy. ROS2 deeply, Gazebo and Isaac Sim, Nav2 and MoveIt2, sensor fusion and SLAM, learned policies — Diffusion Policy, ACT, OpenVLA — sim-to-real with domain randomization, language-conditioned manipulation, a documented safety case, and a chaos drill graded live. The longest track in Crunch Labs. Free, forever.
§ I · The Program
Crunch Robotics is the intelligent-robotics and autonomy specialization of the Code Crunch academy — a full mastery-tier year of work, the longest track in the Labs catalog. It is built for engineers who refuse to settle for a Python notebook simulator and want, instead, the full autonomy stack of a real mobile manipulator: math, ROS2, perception, sensor fusion, SLAM, planning, control, learned policies, sim-to-real, multi-robot coordination, AI-powered task execution, safety cases, and on-call operations.
This is a mastery track. It is deliberately longer than any other Crunch Labs course because robotics is genuinely a year of intentional work and no shorter curriculum produces the engineer we want to graduate. C24 is also distinct from C7 (Crunch Wire). C7 owns the single device — firmware, peripherals, the embedded fleet. C24 owns the multi-actuator autonomous robot — wheels and arms, perception and policy, the safety case, the language-conditioned task. Many engineers do both. C7 first if you want to ship a connected product; C24 if you want to build a robot that thinks.
"This is the longest track in Crunch Labs. It is also the most demanding. Robotics is genuinely a year of intentional work, and no shorter course produces the engineer described above. We refuse to oversell."— Crunch Robotics, course README
§ II · Who It's For
C24 is built for four overlapping personas. The hard floor is fluent Python (C1), shipped firmware or applied ML (C7 or C5), and a willingness to read a control-theory textbook without flinching.
Finished C7 or has shipped firmware in industry. Knows the motor controller, the CAN bus, the RTOS deadline. Wants to climb the stack — from “this device runs” to “this fleet of robots decides what to do.”
Finished C5 (AI/DS) and probably C23 (Agents). Trains and orchestrates models. Tired of agents that only manipulate text. Wants the model to touch the world — to drive, grasp, place, and recover.
Designed the chassis, wired the harness, sized the motors. The robot moves. Now it has to think. Wants the software discipline — ROS2, sensor fusion, motion planning, learned policies — without the marketing fluff.
Has shipped at scale. Knows microservices, observability, on-call. Has an offer (or a tab open) at a robotics startup and needs the discipline-specific vocabulary — perception to policy, safety case, fleet ops — fast.
§ III · Six Phases
The arc of the program runs through six eight-week phases — six because mastery requires it. Each phase ends with an integration milestone reviewed against a rubric before you advance.
Rigid-body math, SE(3), quaternions, ROS2 deeply — nodes, topics, services, actions, lifecycle, QoS, DDS. URDF and xacro. The first simulated robot in Gz Sim. The first SLAM with slam_toolbox. A clean TF tree you can defend.
IMU calibration and Allan variance. EKF, UKF, particle filters, factor graphs (GTSAM). Classical OpenCV. Learned 2D perception on TensorRT. RGB-D pipelines. Open3D and point clouds. A fused perception node inside a 30 ms cycle on a Jetson Orin Nano.
Nav2 architecture and behavior trees. A*, RRT*, lattice planners. PID, LQR, MPC. Manipulator kinematics, Jacobians, and MoveIt2 first contact. The hazard log opens. Every controller declares its fail-safe.
Grasp taxonomies and Contact-GraspNet. Behavior Cloning, DAgger, PPO, SAC. Diffusion Policy and the Action Chunking Transformer. Generalist policies — Octo, OpenVLA — fine-tuned for your task. A learned policy with a documented classical fallback.
Gz Sim and Isaac Sim compared. Domain randomization. Distributed SLAM and Open-RMF fleet coordination. Vision-language models as policies (OpenVLA, RT-X). Grounded planners and structured tool use. Edge ML latency budgets on Orin.
One integrated mobile manipulator. One portfolio-quality safety case (ISO 13482 / ISO 10218). Two chaos drills, live-graded. One mock robotics-startup interview. One polished portfolio. One panel defense at week 48.
§ IV · The Curriculum
Each entry corresponds to a folder in the GitHub repository with lecture notes, exercises, challenges, a quiz, homework, and a mini-project. Detailed acceptance criteria live in the syllabus.
2D/3D rotations · quaternions · axis-angle · Euler failure modes · SO(3) · ROS2 architecture · DDS layer · why ROS1 is dead.
rclpy publisher + rotating quaternion in rviz2
Homogeneous transforms · the SE(3) group · twists · adjoints · the tf2 tree · static vs. dynamic broadcasters · extrapolation exceptions.
Four-link manipulator tf2 tree with one dynamic joint
URDF schema · xacro macros · joint types · collision vs. visual meshes · inertials · Gz Sim plugins for diff-drive, IMU, LiDAR.
Diff-drive robot with LiDAR + IMU, spawned in Gz Sim
Services and actions · the managed-node lifecycle · single- vs. multi-threaded executors · callback groups · composition.
Cancellable Spin90Degrees action server (IMU-closed)
Reliability · durability · history · deadline · liveliness · CycloneDDS vs. Fast-DDS · the silent QoS-mismatch failure mode.
QoS audit + a written QoS-mismatch postmortem
Diff-drive forward and inverse · unicycle · bicycle · Ackermann · mecanum · wheel-encoder odometry · drift growth · PlotJuggler.
Hand-written diff-drive odometry · drift on a 10 m square
Occupancy grids · scan matching · loop closure · the slam_toolbox architecture · mapping vs. localization vs. lifelong modes.
Map a multi-room Gz Sim world · save and re-localize
Launch-file composition · parameter discipline · namespaces · remapping · the bring-up package pattern · TF-tree defense.
Defend your TF, QoS, odometry, and map to a panel
Accelerometer and gyro models · bias and scale factor · noise · Allan variance · static and dynamic calibration · integration drift.
Allan-variance plot · bias correction · drift quantified
Kalman recap · EKF for nonlinear motion · the robot_localization package · covariance bookkeeping · process-noise tuning.
Fused wheel-odom + IMU · drift drop documented
UKF intuition · particle filters · AMCL · introduction to GTSAM factor graphs · the filter-vs.-smoother mental model.
AMCL run + a two-pose GTSAM factor graph by hand
Image formation · pinhole camera · intrinsics and distortion · checkerboard calibration · ORB · Lucas-Kanade · stereo · RANSAC.
Camera calibration + optical-flow velocity estimate
YOLOv8/v10 · DETR · SAM2 · Depth-Anything v2 · TensorRT · ONNX Runtime · ROS2 inference-node patterns · nsys profiling.
YOLOv8n → TensorRT FP16 → ROS2 node at 30 FPS on Orin
Stereo geometry · disparity to depth · structured light vs. ToF · RealSense / OAK-D pipelines · depth filtering · RGB-D fusion.
Synchronized RGB+depth+IMU into a point cloud · Foxglove
Voxel grids · passthrough · statistical outlier removal · RANSAC ground · Euclidean clustering · point-to-plane ICP · drift on KITTI.
ICP registration on 100 LiDAR scans · drift reported
End-to-end perception graph · timing diagrams · the latency budget · midterm architecture review.
A 30 ms fused perception node defended to a panel
Planner · controller · behavior · smoother · recovery · lifecycle manager · 2D/3D costmaps · the BT-driven navigation pattern.
Custom Nav2 behavior plugin — pause-on-operator-hold
Graph search · D* Lite · state lattices · RRT, RRT*, BIT* · admissible heuristics · replanning under dynamic obstacles.
A* from scratch vs. NavFn vs. SMAC Hybrid-A*
BTs vs. state machines · BT.CPP · sequence, fallback, parallel · decorators · conditions · ticking · Groot 2 visualization.
Patrol-with-yield BT + recovery to the charging station
PID anatomy · tuning by feel · integrator wind-up · derivative kick · feedforward terms · regulation vs. tracking.
Yaw-rate PID + feedforward · step-response analysis
State-space form · controllability · observability · the LQR cost · the algebraic Riccati equation · LQR–LQE duality.
Numerical LQR for diff-drive · curved-path tracking vs. PID
Receding horizon · QP solvers (OSQP, acados, do-mpc) · constraint handling · the MPC tuning trade-offs · latency-aware control.
Kinematic-bicycle MPC tracking a figure-8 at 1 m/s
6-DOF forward kinematics · DH parameters · closed-form, numerical, IKFast · the Jacobian · MoveIt2 architecture · move_group.
UR5/MyCobot in MoveIt2 · pose-goal plan-and-execute
Nav2 + MoveIt2 in one launch graph · namespace discipline · hazard log · fail-safe categories · software vs. hardware E-stop.
Drive-reach-return + a 200 ms latched E-stop
Force closure · form closure · analytic grasp planners · ACRONYM and GraspNet-1Billion · the gripper frame · antipodal scoring.
Antipodal grasp candidates on a tabletop cloud · top-10
Architecture · training vs. runtime data · the segmentation-aware head · transparent and reflective object failure modes.
Contact-GraspNet → MoveIt2 · three objects picked in sim
Behavior cloning · covariate shift · DAgger · demonstration collection (teleop, scripted) · the diffusion-of-error problem.
50 demos · MLP BC · one round of DAgger · success delta
Policy gradients recap · PPO · SAC · reward shaping · Gymnasium · Isaac Lab · reward hacking · the sim-throughput axiom.
PPO reach task · 100 parallel envs · 90% success in 30 min
Chi et al. architecture · action-chunk prediction · receding-horizon execution · observation encoders · CFG-style conditioning.
Diffusion Policy on week-27 demos · vs. BC and BC+DAgger
Zhao et al. architecture · chunked action prediction · temporal ensembling · observation tokenization · deployment latency.
ACT trained · Orin latency · vs. Diffusion at fixed budget
Open-X Embodiment · OpenVLA · cross-embodiment data · prompt-as-task · limits of zero-shot transfer · fine-tuning realities.
OpenVLA fine-tune · zero-shot vs. fine-tuned · failure modes
Learned policy + classical fallback · safety scaffolds · trajectory clamping · predictive safety filters · the architecture review.
Safety-wrapped policy + BT fallback · panel-defended
Gz Sim Garden/Harmonic · Isaac Sim · Bullet, ODE, PhysX, MuJoCo · ROS2 bridges · throughput vs. fidelity trade-offs.
Same robot in Gz and Isaac · contact + sensor delta
Visual and dynamics randomization · sensor noise injection · the Sadeghi-Levine and Tobin et al. recipes · the gap-closure metric.
Randomized PPO · held-out real-style eval · gap closed
Distributed SLAM · multi-robot Cartographer · Kimera-Multi · map merging · namespacing · ROS2 discovery domains.
Two-robot mapping in Gz Sim with a merged shared map
Task allocation (auction-, market-, optimization-based) · Open-RMF stack · fleet adapters · narrow-passage conflict resolution.
Two robots into Open-RMF · five tasks · reallocation drill
RT-2 · RT-X · OpenVLA · PaLI-X for robotics · grounding language to actions · the VLA-as-policy pattern · edge latency reality.
Fine-tuned OpenVLA wired into the manipulator · 3 instructions
LLM-as-planner · SayCan-style grounded planning · structured tool use · the planner-plus-skill-library architecture · safety in language.
Local-LLM planner (“clear the table”) · grammar-constrained
TensorRT advanced (FP16, INT8, sparsity) · ONNX Runtime · distillation · QAT · mixed precision · the latency budget as artifact.
Integrated graph profiled on Orin · INT8 quant · Gantt chart
Capstone spec unsealed · pre-flight checks · chaos-drill template · safety-case template · the kickoff ritual.
Happy-path language-conditioned pick-and-place in sim
Hardware bring-up checklist · sim-production-grade checklist · ISO 13482 · ISO 10218 · FMEA · hazard log · residual risk.
Portfolio-quality safety case (8–15 pages)
Sim-to-hardware (Path A) or sim production hardening (Path B) · real-sensor noise · real-actuator latency · the first integration day.
20 m trajectory under the stack · 60 s cold-boot
Prometheus · OpenTelemetry · Foxglove · OTA for robots · the operator dashboard · remote teleop assist.
Foxglove dashboard + one-click teleop takeover
Capstone-specific fine-tuning · eval-set curation · the twenty-instructions evaluation suite · per-instruction reporting.
Twenty-instruction eval · fine-tune · success-rate delta
Robotics-startup system design · technical interview (coding, math, sensors) · the five-projects résumé conversation.
Two mock interviews — system design + technical
Chaos engineering for robots · two intentional failures · postmortem template · operator-detectable events · time-to-recover.
LiDAR-dropout + planner-deadlock drills · 2 postmortems
The robotics-startup loop · the five-minute capstone pitch · portfolio polish (READMEs, video, Mermaid diagrams, safety appendix).
Full-loop mock interview + three flagship project polish
Final defense · panel reviews the safety case, two videos, the chaos-drill postmortems, and the integrated repo · live Q&A.
Autonomous mobile manipulator with language-conditioned pick-and-place
§ V · The Toolchain
Every primary tool below is open-source, free for learning use, or has a documented free tier. Vendor stacks (Isaac Sim, Jetson, TensorRT, ODrive) are taught as the production-scale path — never as the only path.
§ VI · Skills You Will Carry
By the end of Week 48, you are able to do each of the following — credibly, on a real (or honestly-documented simulated) robot, in front of a real reviewer.
§ VII · The Capstone
Weeks 41–48 are reserved for a single substantial integrated system — the kind a real robotics-startup team would scope across a half. Architecture diagram, two videos, safety case, two chaos-drill postmortems, operator-dashboard recording, polished portfolio, live defense.
Capstone Brief
Build (or simulate) a wheeled-base plus 6-DOF-arm robot that takes a natural-language instruction — "bring me the red cup from the left bench" — and executes it via a perception → planner → controller → policy stack. The autonomy runs on a Jetson Orin (Path A) or in Gz / Isaac Sim against a documented hardware target (Path B). Telemetry streams to an operator dashboard. The robot passes a documented safety case for shared-space operation and survives two chaos drills.
§ VIII · Getting Started
The setup is intentionally lightweight. If you can install Ubuntu (or run WSL2 / a Linux VM) and a terminal command, you can begin Week 1 today. The hardware kit ships separately, and a simulation-only path clears the capstone bar.
# 1. Clone the curriculum repository git clone https://github.com/CODE-CRUNCH-WORLDWIDE/C24-CRUNCH-ROBOTICS.git cd C24-CRUNCH-ROBOTICS # 2. Install ROS2 (Humble / Iron / Jazzy — match your cohort) sudo apt install ros-jazzy-desktop ros-jazzy-nav2-bringup \ ros-jazzy-moveit ros-jazzy-slam-toolbox # Ubuntu 24.04 # 3. Choose your simulator (Gz Sim by default; Isaac Sim from Phase 4 onward) sudo apt install ros-jazzy-ros-gz # free, ROS-native # or follow docs/isaac-sim-setup.md for the NVIDIA path # 4. Open Week 1 README and begin $EDITOR curriculum/week-01-rigid-body-math-and-ros2-intro/README.md
Need the hardware tier list, the simulation-only path, or the cloud-GPU budget? See the README.
§ IX · Frequently Asked
Two paths are supported and both clear the capstone bar. Path A (recommended if budget allows): a TurtleBot 4 Lite (~USD 1,200) or DIY differential-drive base, a Jetson Orin Nano (8 GB; upgrade to NX 16 GB or AGX Orin for the learned-policy phase), a RealSense D435i or OAK-D Lite depth camera, and a 6-DOF arm (PiArm, MyCobot 280 Pi, or a used UR-style arm) added from Phase 4. Path B (simulation-only, the affordable path): a laptop with a discrete NVIDIA GPU (≥ 8 GB VRAM recommended; Apple Silicon acceptable for CPU-bound labs). The capstone is fully gradable in simulation — the rubric scores autonomy-stack quality, safety-case construction, and chaos-drill recovery, not whether a real robot was bought. Both paths share roughly USD 25/month of cloud GPU credit for the four weeks of policy training.
Helpful, not strict. C7 is the recommended pre-track because the hardware-bring-up weeks assume you have brought up an MCU, calibrated a sensor, and debugged a real-time loop. If you have strong applied firmware or robotics-adjacent industry experience instead (you have shipped a motor-controlled product, calibrated an IMU on hardware, debugged CAN at 3 AM), you can come in without C7. Plan for two to three extra weeks of self-driven C++ and embedded I/O shoring up before Week 1 if you choose that route.
Both, eventually. Gz Sim is the default through Phase 3 — it is free, ROS-native, and runs on modest hardware. From Phase 4 onward, Isaac Sim (free tier) becomes the recommended simulator for GPU-parallel policy training and the higher-fidelity sim-to-real labs. If you cannot run Isaac Sim locally, Gz Sim is supported throughout with a small expressiveness penalty in the learned-policy weeks; we document the substitution in each affected lab.
Because a robot that does not have a safety case is a robot you cannot ship near people. The capstone safety case is framed against ISO 13482 (personal-care robots) and ISO 10218 (industrial manipulators), and it is graded as a portfolio-quality artifact — hazard log, FMEA, mitigations checklist, validation plan, residual-risk acceptance. It is also the artifact that, in our experience, wins the second-round interview at a robotics startup. The chaos-drill postmortems are the artifact that wins the third.
The course is written to be distribution-agnostic across the currently-supported LTS and rolling distributions: Humble, Iron, and Jazzy. Each cohort fixes a single distribution at kickoff (typically the latest LTS available on the supported Ubuntu version). Lab files are tested on all three; vendored package versions in each weekly resources.md declare what is known-good. ROS1 is not supported and is not taught.
Yes. Robotics is the engineering discipline where math, perception, planning, control, learned policies, sim-to-real, multi-robot coordination, fleet ops, and safety cases all have to coexist in one stack. Each Phase is eight weeks because we have tried shorter and watched it fail to produce the engineer described in the README. Plan for 36 hours per week of focused work. Part-time at 18 hours per week is supported and runs about ninety calendar weeks; we will not pretend otherwise. The track is the longest in Crunch Labs because the discipline demands it.
§ X · Begin
Open the repository. Read Week 1. The bench, and the simulator, are yours.