Why inference matters
The fastest-growing category in AI. Closed vs open, and why you're early.
Explore→The three layers
Runtime, infrastructure, tooling — and how they work together.
Explore→Prefill & decode
Two phases, two bottlenecks. TTFT, TPS, and the KV cache.
Explore→Optimisation tricks
Quantization, speculation, caching, parallelism. 30–50% gains.
Explore→GPUs & engines
Hopper, Blackwell, vLLM, SGLang, TensorRT-LLM. Pick your stack.
Explore→Models & modalities
LLMs, VLMs, embeddings, ASR, TTS. Architecture and attention.
Explore→Ship it
Containers, autoscaling, cold starts, multi-cloud. Production ops.
Explore→Cheat sheet
Metrics, GPU specs, when-to-use guides. Quick lookup.
Explore→Interview prep
Common Q's, tradeoffs, system design. Numbers to know.
Explore→Flashcards
29+ terms. Flip to learn. TTFT, KV cache, speculative decoding.
Explore→Revision Quiz
52 questions: transformers, inference, quantization, LoRA, sampling.
Explore→The inference stack
In 30 seconds
Compute-bound. Ingest prompt, build KV cache. Sets TTFT.
Memory-bound. Generate tokens one by one. Sets TPS.