Emergence
Higher-level behavior that arises from lower-level structure. The thing everyone argues about when they should be looking underneath.
Emergence is when a system's higher-level behavior can't be straightforwardly predicted from its parts. It's the phase transition, the sudden capability, the "where did that come from?" moment. And it's where most analysis goes wrong — because emergence is dramatic, and drama attracts attention away from the substrate that produced it.
The problem with emergence
Emergence isn't mysterious. It just looks that way when you're watching the wrong level.
When grokking "suddenly" happens in a neural network, that's emergence — test accuracy jumps from chance to perfect in a few steps. But it's only sudden if you're watching test accuracy. Watch the weight structure and you see a smooth, predictable reorganization that was happening for thousands of steps before the accuracy metric caught up. 00. — This is the central finding of my grokking work: the "phase transition" is a measurement artifact.
The pattern repeats everywhere:
- A team "suddenly" becomes dysfunctional — but the communication patterns degraded weeks ago
- A market "suddenly" crashes — but the leverage ratios were building for months
- An LLM "suddenly" shows reasoning — but the training data statistics shifted gradually
In each case, the emergence is real. But the surprise is an artifact of measuring at the wrong level.
Analyzing at the wrong level
The most common mistake in analyzing complex systems is picking a level and staying there. You see the emergent behavior, you describe it, you taxonomize it, and you build theories about it — all without ever looking at what produced it.
This is comfortable because emergent properties are usually what we care about. We care about whether the model can reason, whether the team is productive, whether the system detects threats. But caring about a level doesn't make it the right level to analyze.
A useful diagnostic: if your explanation of a phenomenon is purely descriptive — "it has property X, it does behavior Y" — you're probably at the wrong level. Explanations that actually work reference the mechanism that produces the phenomenon, not just the phenomenon itself.
In philosophy of mind: The entire "does the LLM understand?" debate is stuck at the emergence level. Both sides describe behavioral outputs and argue about whether those outputs constitute understanding. The deflationary move is to drop down a level: what internal computation would understanding require? What would you look for in the substrate? My Bearer Problem paper makes this move explicitly.
In defense: Counting sensors gives you a description of a detection system. Analyzing the coupling structure between those sensors — the Jacobian rank — gives you an explanation of its capability. The number is emergence; the coupling is substrate. Perihelion formalizes this.
00. — Philosophers distinguish weak emergence (deducible from lower-level laws, just hard to compute) from strong emergence (genuinely novel, can't be derived from substrate). Almost everything interesting in practice is weak emergence. This site is entirely about weak emergence: cases where looking at the substrate dissolves the apparent mystery. 00. — ML has a particular version of the wrong-level problem: evaluating models by benchmarks (emergence) instead of understanding training dynamics (substrate). Scaling laws are a partial fix but still correlational. The deeper question is always: what changed in the learned representation, and why?Earning the right to talk about emergence
I'm not anti-emergence. Emergent descriptions are useful — often essential. You can't navigate the world by thinking about atoms. But there's an order of operations: understand the substrate first, then describe the emergence. When you do it in that order, the emergent description is grounded. When you skip the substrate, the emergent description is just pattern-matching on vibes.