Twenty-four weeks on the modern AI engineering stack. The agent loop as a discipline. Retrieval as engineering. Memory as a context-budget cache. Multi-agent orchestration graphs. Local inference on Ollama, vLLM, and NeMo. Tool surfaces over the Model Context Protocol. Open-Claude-compatible (OpenClaw) runtimes. Eval-in-prod. Free, forever.
§ I · The Program
Crunch Agents is the AI agent systems engineering specialization of the Code Crunch academy — a 2026 redesign for the Crunch Labs tier. It is built for engineers who want to ship LLM-backed products: not notebook demos, but agentic systems with versioned prompts, hybrid retrieval over a private corpus, memory tiers that survive a forty-turn conversation, multi-agent supervisor graphs, and a tool surface served over the Model Context Protocol.
This track is deliberately distinct from C5 (Crunch AI · Data Science). C5 owns classical machine learning through deep learning — feature engineering, gradient boosting, CNNs, transformers as architectures, PyTorch fundamentals. C23 picks up where C5 leaves the room: LLMs as components in a system, not as research artifacts. The discipline here is systems engineering on top of models you mostly do not train. By Week 24 you have shipped a multi-agent research assistant served from your own vLLM cluster, instrumented with OpenTelemetry Gen-AI conventions, evaluated with Ragas and a calibrated LLM-as-judge, and survived a chaos drill in which a GPU node dies, a tool is prompt-injected, and a vector index corrupts — all in the same week.
"If C5 makes you a model builder, C23 makes you the engineer who keeps an agentic product alive in production."— Crunch Agents, course README
§ II · Who It's For
Agents is opinionated about its audience. C1 (Code Crunch Convos) is the floor — you should be able to read Python comfortably, run Docker without flinching, and survive a Linux shell.
Has shipped Flask, FastAPI, or Django. Has called the OpenAI API. The gap between demo and production is fog — wants the durable patterns: retrieval as engineering, eval as engineering, the agent loop with rigor.
Came up on scikit-learn, XGBoost, PyTorch. Can train, calibrate, fight a leakage bug. Lacks muscle memory for RAG, MCP, vLLM, agent graphs, tool-use safety, and the operating model of an LLM-backed product.
Runs platforms — K8s, Prometheus, on-call, autoscaling. The new pager is GPU pools and token budgets. Wants vLLM topology, NeMo Inference, continuous batching, KV-cache economics, and a real runbook for an agentic system.
Has a Ph.D. or near-Ph.D. and reads a paper a week. The applied side — testable prompts, observability, retrieval pipelines, cost engineering, the messy reality of shipping — is the gap. C23 closes it without insulting the background.
§ III · Four Phases
The arc of the program is composed in four phases — six weeks each — each building on the last like floors of a building.
LLMs as system components. Tokens, context, sampling, structured output. Prompt-as-code with promptfoo and Langfuse. Tool calling across vendors. The agent loop (ReAct, plan-and-execute, reflection). Local inference bring-up on Ollama, llama.cpp, and vLLM.
Embeddings, chunking, hybrid search, reranking. Vector stores in production — pgvector, Qdrant, Weaviate. Memory tiers: episodic, semantic, procedural. Multimodal RAG. Evaluation as engineering with Ragas and a calibrated LLM-as-judge.
LangGraph state graphs. Mastra and durable execution. The Model Context Protocol in depth, plus the open-Claude-compatible OpenClaw runtime family. Fine-tuning with LoRA / QLoRA. Safety, prompt injection, threat modeling. Observability via OTel Gen-AI conventions.
vLLM clusters with continuous batching. NeMo Inference and the NVIDIA stack. Cost engineering, model routing, semantic cache. Two capstone sprints. Week 24 chaos drill: GPU node loss, prompt-injection on a tool, vector-index corruption. Postmortem.
§ IV · The Curriculum
Each entry corresponds to a folder in the GitHub repository with lecture notes, exercises, challenges, a quiz, homework, and a mini-project. Detailed acceptance criteria live in the syllabus.
Tokenizer → context → forward pass → sampling · decoder-only transformer at a systems level · open-weights (Llama 4, Qwen 3, Mistral, Gemma 3, DeepSeek) vs closed-weights · reading a model card without falling for the benchmark.
Build llmpick — multi-model recommender
BPE · SentencePiece · tiktoken · the price of long context · KV cache · temperature, top-p, top-k, min-p · structured output with outlines, guidance, xgrammar · grammar-constrained decoding.
Tokenization explorer + grammar-constrained JSON
The prompt is code · system/user/assistant roles · few-shot, CoT (honest), self-consistency · jailbreak surface · versioning with promptfoo and Langfuse · spec-then-implement with Claude Code and Cursor.
Promptfoo regression suite — 6 versions, 30 golden examples
Function calling across vendors · open-source equivalents on Llama and Qwen · MCP as the cross-vendor protocol (stdio, SSE, streamable HTTP) · JSON-mode vs grammar-constrained · tool-use as RCE.
4-tool agent on Claude + local Qwen 2.5 7B
ReAct, plan-and-execute, reflection, self-critique · the "infinite tool-call" failure · step / token / time / cost budgets · claude-agent-sdk, OpenAI Agents SDK, AWS Strands, Google ADK at survey depth.
~150-line ReAct loop, no framework, vs claude-agent-sdk
Ollama for iteration · llama.cpp for portability · vLLM for throughput · SGLang, TensorRT-LLM, TGI · quantization (GGUF, AWQ, GPTQ, bitsandbytes) · speculative decoding · Apple MLX.
Same 7B on Ollama, llama.cpp, vLLM — measure tokens/sec, p95, VRAM
Open families — BGE, GTE, jina-embeddings-v3, nomic-embed, E5-Mistral · vendor embeddings · the MTEB leaderboard, read skeptically · ANN indexes (HNSW, IVF, ScaNN).
Three open + one vendor embedding over a legal corpus in pgvector
Token-window, semantic-paragraph, recursive, sliding-window, late chunking · Unstructured, MinerU, LlamaParse, PyMuPDF · OCR (Tesseract, Surya) · table extraction · chunk size as a hyperparameter.
Chunking A/B harness — MRR, Recall@5, faithfulness delta
BM25 + dense · reciprocal rank fusion · bge-reranker-v2, Cohere, ColBERT late-interaction · text-to-SQL · query rewriting · HyDE.
Cumulative lift: dense → +BM25 → +RRF → +reranker → +HyDE
pgvector, Qdrant, Weaviate, Milvus, Chroma · filtered ANN · backup, replication, rebuild · GraphRAG and knowledge-graph hybrids · agentic-RAG patterns.
Same pipeline across pgvector, Qdrant, Weaviate — ingest, query, recovery
Three tiers — episodic, semantic, procedural · summarization (rolling, hierarchical, map-reduce) · eviction (LRU, salience-weighted) · MemGPT / Letta· the "lost in the middle" effect.
40-turn chat with three-tier memory vs no-memory baseline
VLMs — LLaVA, Qwen2-VL, Phi-Vision, InternVL · CLIP / SigLIP · Whisper, Piper, XTTS · SDXL / Flux · Ragas as standard · DeepEval, promptfoo, TruLens · calibrated LLM-as-judge.
Full Ragas suite — faithfulness, context recall, precision, relevancy
State graphs, conditional edges, checkpoints, persistence · supervisor, swarm, hierarchical patterns · honest critique of AutoGen and CrewAI · when a state machine beats an agent.
Re-implement Week 5 as a LangGraph with SQLite checkpointer
Mastra (TypeScript-first) · Inngest for event-driven durable execution · Trigger.dev · Temporal at scale · Python-first vs TS-first agent stacks, honestly.
Supervisor in Mastra + LangGraph · Inngest-triggered research run
MCP server / client, transport modes, tool / resource / prompt primitives · writing MCP servers in Python and TS · the open OpenClaw family — open MCP gateways, self-hosted MCP servers, community Claude-compatible loops · MCP security review.
Two MCP servers (FS + private-corpus search) over stdio and HTTP
When not to fine-tune (almost always) · LoRA, QLoRA, DoRA · SFT, DPO, ORPO, KTO · NeMo Framework, Axolotl, Unsloth · RLHF/RLAIF at survey depth.
Unsloth SFT of Qwen2.5-7B on a 500-example custom DSL
Direct vs indirect prompt injection · jailbreak surface · output filtering (regex, classifier, judge) · Llama Guard, OpenAI Moderation, Perspective · the OWASP LLM Top 10 · tool-use threat modeling.
Red-team your Week 15 MCP agent — 25 attacks, 3 defenses
OpenTelemetry Gen-AI semantic conventions · Langfuse (self-hosted), Arize Phoenix, LangSmith, Helicone · token accounting per route/user/model · latency SLOs · eval-on-traces.
Three dashboards: token-by-route, p95-by-step, retrieval over time
Continuous batching · paged attention · prefix caching · tensor / pipeline parallel · multi-replica vLLM behind LiteLLM · speculative decoding · OpenAI-compatible API.
vLLM + LiteLLM on H100 — concurrency 1/8/32/128 + break-even memo
NVIDIA NeMo Inference for production serving · TensorRT-LLM kernels · Triton Inference Server · NeMo Guardrails as policy · honest NeMo vs vLLM comparison.
Qwen2.5-14B on NeMo + Triton — benchmark vs vLLM, add a guardrail
Routing — small for easy, big for hard · semantic cache (GPTCache or self-built pgvector) · prompt compression (LLMLingua) · batched inference · per-feature cost accounting · speculative decoding as a cost lever.
Router: Qwen2.5-7B local vs Claude vendor + 0.92-cosine semantic cache
Architecture review · hybrid retrieval over a 10 GB private corpus · three-tier memory wiring · Mermaid architecture diagram · 6-page architecture document.
Retrieval + memory layers of the agentic research assistant
Supervisor + retrieval + code + writing agents · MCP tool surface · vLLM cluster deploy · Ragas + calibrated LLM-as-judge on 100-question gold set · OTel traces to Langfuse + Phoenix · cost-tracked routing.
End-to-end system shipped as deploy URL or docker compose image
Eval-in-prod with shadow traffic · blue/green model deploys · canary by user cohort · the on-call runbook for an agentic system · the chaos drill · the postmortem · interview prep.
GPU node loss · prompt-injection on a tool · vector-index corruption
§ V · The Toolchain
Every primary tool below is open-source or has an open-weights, self-hosted path. Vendor APIs (OpenAI, Anthropic, Gemini, Bedrock) are taught as the production scale path and the frontier-capability path — never as the only path.
§ VI · Skills You Will Carry
By the end of Week 24, you are able to do each of the following — credibly, on a real system, in front of a real reviewer.
§ VII · The Capstone
Weeks 22–24 are reserved for a single substantial system — the kind a real product team would scope across a quarter. Architecture document, live deploy, video walkthrough, chaos-drill postmortem.
Capstone Brief
A multi-agent system — supervisor plus retrieval, code, and writing agents — operating over a 10 GB private corpus with hybrid retrieval, three-tier memory, an MCP tool surface, and self-hosted serving on a vLLM cluster with continuous batching. Cost-tracked routing between a local 7B/13B and a frontier vendor model. Full eval suite. OTel tracing. Red-team report. Chaos-drill postmortem.
§ VIII · Getting Started
The setup is intentionally lightweight. If you have a Linux, macOS, or WSL2 laptop and can run a terminal command, you can begin Week 1 today. No GPU required on day one.
# 1. Clone the curriculum repository git clone https://github.com/CODE-CRUNCH-WORLDWIDE/C23-CRUNCH-AGENTS.git cd C23-CRUNCH-AGENTS # 2. Pull a local model (Apple Silicon / Linux / WSL2) ollama pull llama3.1 # or qwen2.5:7b, mistral, gemma3 # 3. Set up a Python venv with uv (fast, reproducible) uv venv && source .venv/bin/activate uv pip install -r requirements.txt # 4. Open Week 1 README and begin $EDITOR curriculum/week-01-llm-as-system-component/README.md
Need the hardware budget, rented-GPU recipes, or CPU-only fallbacks? See the README.
§ IX · Frequently Asked
Not on day one. A 16 GB laptop carries you through Week 6, and Apple Silicon via Metal carries you further. From Week 7 onward a 24 GB GPU (RTX 3090 / 4090, used or new, or a rented A10 / L4 at ~$0.50–$1.00 per hour) unlocks the local-embedding and reranker labs. Two specific labs (vLLM cluster, NeMo Inference) benefit from H100 access on rented spot instances. Every lab has a CPU-only fallback. Total compute budget across the course on rented GPUs is roughly $30.
§ X · Begin
Open the repository. Read Week 1. The agent loop is yours.