There is No Curve
My thoughts on AI risk and the confusion at the heart of AI discourse.
The doomer and the accelerationist agree on more than either of them notices. They agree that intelligence is the order parameter. They agree that there is a curve, that the curve goes up, and that the disagreement between them is whether the rise is something to celebrate or fear. Eliezer Yudkowsky and Marc Andreessen are arguing about the same graph. They are wrong about the graph.
I’ve been working through the physics of this lately, and what I want to do here is set down the argument in one place. The conclusion is that the AI scaling discourse is malformed at the foundations — not in the sense that one side has its numbers wrong, but in the sense that the question both sides have been litigating doesn’t refer to anything natural. There is no curve. What there is, is something else, and that something else has consequences both sides have missed.
⁂
Start with the physics, because the physics has been telling us this for a while and we haven’t been listening.
Jacob Bekenstein gave us the relevant bound in 1981. There is a maximum amount of information that can be contained in a region of given energy and volume before that region must become a black hole. Seth Lloyd worked the consequence out for computation specifically in his 2000 Nature paper “Ultimate physical limits to computation”. Push computation to its physical limit and you get a black hole. Not a faster computer. The “ultimate laptop” is a Schwarzschild radius.
This already tells you that the Kurzweilian explosion-to-infinity story is incoherent at the limit. There is a ceiling. What is past the ceiling is a horizon, not more intelligence. The acceleration cult’s vision of unbounded recursive self-improvement runs straight into general relativity well before it produces the singularity Ray Kurzweil imagines.
But this is the formal version of the argument, and it gets you to the right answer the wrong way. The Bekenstein bound is nine or ten orders of magnitude beyond any biological substrate. Pointing at it and saying “see, the runaway hits a wall” is technically correct and practically irrelevant. The wall is too far away to do real work in the conversation that matters.
The real argument lives much closer.
What is intelligence?
The discourse treats intelligence as a scalar. A number. A thing that can be more or less of, and that scaling laws can be applied to. This is the load-bearing mistake in the conversation, and once you see it the entire frame falls apart.
Intelligence is a natural property. It is what consciousness is at the limit. The harmonic patterns in the substrate of reality, when integrated richly enough, manifest as conscious experience, and consciousness at its observational boundary is intelligence. This is the dual-aspect monist picture I’ve been developing in these pages — consciousness as fundamental, matter as the outward-facing aspect of the same substance, intelligence as the natural property that emerges where consciousness encounters its own observational structure.
What follows from this is that intelligence has a regime. It exists in a thermodynamic envelope of stability, like every harmonic pattern in nature. Push past the envelope and the harmonic pattern fails. Scale the failure appropriately and you get gravitational collapse. The black hole is the negative space of consciousness. It is what is on the other side of the harmonic envelope.
This means the substrate-tensor — the relational structure of the medium that supports the dynamics — is doing the work. Intelligence belongs to substrate-relations themselves, not to computations abstracted from them.
Turing-Church machines bracket exactly this. The whole point of the Church-Turing formalism is to study what survives the abstraction from substrate. That bracketing is fine for proofs about computability. It is useless for the question of when the substrate destabilizes, because phase boundaries belong to the medium, not the abstraction. A Turing machine, by construction, does not have a substrate-tensor, because Turing machines are defined by what is invariant across substrates.
This is why every transformer trained by gradient descent on next-token prediction is in the same regime-class. The substrate-tensor that matters is set by the architecture and the loss, not by the parameter count. Claude 5 differs from Claude 1 the way a finer-mesh finite-element model differs from a coarse one — same physics, more resolution. That is not nothing. It is also not phase change. They are exactly as intelligent as each other, in the sense of intelligence I’m using. What scales with parameter count is something else.
