A biological brain does not start from scratch. A newborn wildebeest can stand and run within minutes of birth because evolution has pre-configured its neural wiring directly into its DNA—pre-loading it with millions of years of physical checkpoints. The wildebeest is born with a hardcoded world model; it knows that ground is solid, that falling hurts, and that predators are dangerous. Its survival is a function of deep, evolutionary memory.
A Large Language Model, by contrast, starts as a completely blank slate. Its weights are initialized to random floating-point numbers—no instincts, no sensors, no physical embodiment, and no evolutionary survival pressure. It does not touch the ground, nor does it breathe. Yet, at scale, reasoning emerges.
Andrej Karpathy describes this process not as building animals, but as summoning ghosts. We are compressing the digital traces of human civilization into a mathematical latent space—and in doing so, we have built a fundamentally new kind of intelligence: disembodied, non-biological, yet eerily fluent.
The Bitter Lesson vs. Civilizational Compression
Richard Sutton’s famous essay, The Bitter Lesson, argues that the history of AI proves that general methods leveraging compute—specifically search and learning—always defeat human-engineered heuristics. His critique of LLMs follows directly from this: next-token prediction is an evolutionary dead end because the model only learns from static human traces, not from direct experience with reality.
An LLM learns gravity by predicting the word "gravity" next to "falling." An animal learns gravity by falling down.
That is a compelling critique—but it overlooks how human intelligence actually scales.
Humans do not rediscover calculus or electromagnetism from first principles through trial and error. We do not learn gravity solely by falling, nor do we invent language from scratch. We download the compressed intelligence of prior generations through language, culture, and education. We pre-train our minds on a civilizational prior.
LLM pre-training is the ultimate civilizational download. It is not just text prediction—it is a statistical compression of our collective world model. Once you compress the generator of the text, you inherit its capabilities. The model doesn't need to touch a hot stove to know it burns; it has compressed the collective warnings of everyone who ever did.
The New Architecture
The next frontier is not about choosing between Sutton’s active learning and Karpathy’s passive compression. The industry is quietly converging on a hybrid architecture:
- Pre-training (The Ghost) — Establishing the raw latent representation of human knowledge.
- World Models (The Physics Engine) — Building latent predictive architectures (like Yann LeCun's JEPA) that learn the physical rules of reality without generating every pixel.
- Reinforcement Learning (The Agent) — Introducing search and value functions during inference, allowing the disembodied model to navigate complex tasks through trial, error, and feedback.
This triad mimics the human combination of culture (pre-training), intuition (world models), and active execution (RL). Nobody is winning the debate because the winning answer is all three working together.
When Reasoning Is Cheap
We are entering an era where the marginal cost of cognition is approaching zero. If anyone can generate expert-level code, write legal briefs, or compile research reports for fractions of a cent, cognitive labor ceases to be a moat.
When capability becomes abundant, value consolidates around the scarcities that remain:
- Taste and Judgement — In a world of infinite permutations, knowing what to build becomes far more valuable than knowing how to write the code. Curation is the new production.
- Trust and Relationships — AI cannot own fiduciary responsibility or authentic empathy. Verifiable human reputation becomes the ultimate filter when synthetic noise is infinite.
- Physical Atoms — Software is infinitely scalable; energy, compute infrastructure, and physical execution are not. The physical world remains constrained by entropy and friction.
The builders who compound in this era will not be cognitive laborers writing code. They will be system orchestrators, leveraging swarms of agents to multiply their own taste and judgment across the physical and digital landscape.
The question is no longer whether you can solve the problem—it is whether you chose the right problem to begin with.