Why inference matters
Inference is the most valuable category in AI. The discipline is still young — and that's the opportunity.
Then vs now
Nov 2022 (ChatGPT launch): A few hundred inference specialists, mostly at frontier labs. Training looked so hard that closed models would dominate; everyone else would rent AI via API.
Today: 2M+ open models on Hugging Face. Every engineer can deploy their own. DeepSeek V3 closed the intelligence gap. Open models match closed within months.
Closed vs open
Closed: Proprietary weights (GPT-5, Claude). Easier to start, vendor lock-in risk.
Open: Weights public (Llama, DeepSeek). 80%+ cheaper at scale, full control, fine-tune for your domain.
Why switch to open?
- Latency — APIs optimise throughput; open lets you tune for real-time
- Availability — Four nines possible vs two nines
- Cost — Often 80%+ cheaper at scale
Who's building
Cursor, Clay, Gamma, Mercor (AI-native). Notion, Superhuman (deep integration). World Labs, Writer, Mirage (train + ship). OpenEvidence, Abridge, Ambience (healthcare). Enterprise moving past pilots.
Test your understanding
Why does the author say inference engineering is 'still in its infancy' despite inference being the most valuable AI category?