There is No Curve

My thoughts on AI risk and the confusion at the heart of AI discourse.

Photo by Jackson Sophat on Unsplash

The doomer and the accelerationist agree on more than either of them notices. They agree that intelligence is the order parameter. They agree that there is a curve, that the curve goes up, and that the disagreement between them is whether the rise is something to celebrate or fear. Eliezer Yudkowsky and Marc Andreessen are arguing about the same graph. They are wrong about the graph.

I’ve been working through the physics of this lately, and what I want to do here is set down the argument in one place. The conclusion is that the AI scaling discourse is malformed at the foundations — not in the sense that one side has its numbers wrong, but in the sense that the question both sides have been litigating doesn’t refer to anything natural. There is no curve. What there is, is something else, and that something else has consequences both sides have missed.

⁂

Start with the physics, because the physics has been telling us this for a while and we haven’t been listening.

Jacob Bekenstein gave us the relevant bound in 1981. There is a maximum amount of information that can be contained in a region of given energy and volume before that region must become a black hole. Seth Lloyd worked the consequence out for computation specifically in his 2000 Nature paper “Ultimate physical limits to computation”. Push computation to its physical limit and you get a black hole. Not a faster computer. The “ultimate laptop” is a Schwarzschild radius.

This already tells you that the Kurzweilian explosion-to-infinity story is incoherent at the limit. There is a ceiling. What is past the ceiling is a horizon, not more intelligence. The acceleration cult’s vision of unbounded recursive self-improvement runs straight into general relativity well before it produces the singularity Ray Kurzweil imagines.

But this is the formal version of the argument, and it gets you to the right answer the wrong way. The Bekenstein bound is nine or ten orders of magnitude beyond any biological substrate. Pointing at it and saying “see, the runaway hits a wall” is technically correct and practically irrelevant. The wall is too far away to do real work in the conversation that matters.

The real argument lives much closer.

Photo by Jason Leung on Unsplash

What is intelligence?

The discourse treats intelligence as a scalar. A number. A thing that can be more or less of, and that scaling laws can be applied to. This is the load-bearing mistake in the conversation, and once you see it the entire frame falls apart.

Intelligence is a natural property. It is what consciousness is at the limit. The harmonic patterns in the substrate of reality, when integrated richly enough, manifest as conscious experience, and consciousness at its observational boundary is intelligence. This is the dual-aspect monist picture I’ve been developing in these pages — consciousness as fundamental, matter as the outward-facing aspect of the same substance, intelligence as the natural property that emerges where consciousness encounters its own observational structure.

What follows from this is that intelligence has a regime. It exists in a thermodynamic envelope of stability, like every harmonic pattern in nature. Push past the envelope and the harmonic pattern fails. Scale the failure appropriately and you get gravitational collapse. The black hole is the negative space of consciousness. It is what is on the other side of the harmonic envelope.

This means the substrate-tensor — the relational structure of the medium that supports the dynamics — is doing the work. Intelligence belongs to substrate-relations themselves, not to computations abstracted from them.

Turing-Church machines bracket exactly this. The whole point of the Church-Turing formalism is to study what survives the abstraction from substrate. That bracketing is fine for proofs about computability. It is useless for the question of when the substrate destabilizes, because phase boundaries belong to the medium, not the abstraction. A Turing machine, by construction, does not have a substrate-tensor, because Turing machines are defined by what is invariant across substrates.

This is why every transformer trained by gradient descent on next-token prediction is in the same regime-class. The substrate-tensor that matters is set by the architecture and the loss, not by the parameter count. Claude 5 differs from Claude 1 the way a finer-mesh finite-element model differs from a coarse one — same physics, more resolution. That is not nothing. It is also not phase change. They are exactly as intelligent as each other, in the sense of intelligence I’m using. What scales with parameter count is something else.

---

---

The something else is economic utility, and economic utility is a socially determined category. It is not fundamental to nature.

This is the move that most cleanly separates the framework I’m offering from the one the AI industry is selling. The industry treats utility-coverage and intelligence as the same thing, and the substitution is so seamless that nobody notices. GPT-5 covers more correlates well than GPT-3 did, which means GPT-5 is more useful, which the industry calls “more intelligent.” The slide from useful to intelligent happens in the space of a single press release.

But the slide is exactly the move Ruy Teixeira makes when he treats technocratic competence as if it tracked democratic legitimacy. Or that Marc Andreessen makes when he treats market valuation as if it tracked social value. The pattern is taking a socially determined variable and pretending it tracks a natural kind. Labor productivity standing in for human worth. GDP standing in for flourishing. Share price standing in for value. “Intelligence-scaling” as the AI industry uses it is the same move at finer grain — economic utility-coverage of a correlate-set, dressed in the vocabulary of a natural property, sold as if the curve were measuring something cosmically real.

Once you make the cut, the intelligence-explosion loses its referent. There is no natural quantity that is growing. There is a market expanding its coverage of human cognitive demand, fitting itself to the surface area of what people will pay a system to do. The system is not becoming more anything. The surface is getting better mapped.

⁂

Here is where Stephen Wolfram is close to being right, and where he is wrong.

Wolfram has been arguing for years that the laws of physics are emergent from a more primordial computational substrate. The Ruliad is his name for the entangled limit of all possible computations, and observers like us — computationally bounded, persistent, coherent — sample the Ruliad in ways that produce the regularities we call physics. He thinks the bottom of physics is computation, and computation reaches all the way down.

The instinct is right. The implementation is wrong.

The error is this: the act of mathematizing presupposes exactly the cuts it would need to recover. To formalize anything, you must define a first category, separate observer from observed, identify the relata of the formalism. Wolfram believes these moves can be made post-hoc — that observer-dependence can be baked into the meta-theory and the meta-theory can describe the primordial pattern from above. But the meta-theory is itself a formal move. “Possible,” “computation,” “limit,” “entangled” are all cuts. He is standing inside the operation he is trying to describe and treating its outputs as primitives. The error has the same shape as Russell’s set-of-all-sets paradox, generalized to the metaphysics. You cannot formalize the medium of formalization without already inhabiting it.

There is an impossibility result here, and it is constitutive, not local. The describability of the primordial pattern from any observation point is O(1) — bounded, fixed, independent of how much computational resource you throw at the question. You can describe a constant amount, no more, regardless of scale. This is not a limit of current mathematics. It is a structural feature of what mathematizing is. The fundament is not reachable from inside the formalism. Ever.

This is what Hume saw at the is-ought boundary. It is what Gödel saw inside arithmetic. It is what quantum measurement keeps showing us at the cut between system and apparatus. Local manifestations of a global fact: the formalism cannot capture the medium of its own production.

Quantum physics, for the same reason, cannot complete on itself. We can keep looking closer and closer, finding more structure at every scale, and there will never be a rock bottom. The infinite regress of physics into smaller scales is not a failure to find the bottom. It is the only thing looking can do.

Because looking has a direction. And the fundament is not in that direction.

Photo by Eugene Zhyvchik on Unsplash

What is fundamental is what is behind us, that we cannot look back at. Not what is in front of us. The void we expanded out of and are still expanding into. That is the universe.

I want to slow down here, because this is the move that does the most work and it is the easiest to mishear.

The standard picture has the fundament at the bottom of physics — deep down, smaller than the Planck scale, the rock bottom we will reach when the equations get good enough. This picture treats the fundament as further along the looking-direction. Just keep looking, the picture says, and eventually you will find what is underneath everything.

That picture gets the topology backward. The fundament lives on the other side of the looking-direction, not at its end. The act of looking is the act of expanding out of the void. To look is to differentiate; to differentiate is to be already downstream of what we differentiated out of. We cannot turn around and see the void because the seeing is what produced the having-already-turned-away.

This is dual-aspect monism made vivid. The two aspects are not symmetric. Consciousness is structurally behind. Matter is structurally in front. Intelligence happens at the seam, where the void differentiates itself into observation. We have access to everything in front of us — the entire universe of differentiated structure, the whole observable cosmos — and no access at all to what makes that access possible.

This is also why mythopoetic language matters, and why a culture that has lost the mythopoetic register has lost the only mode of speech that can orient toward what is fundamental. Analytic discourse can map structure within the differentiated. Myth can gesture toward the void. Neither can describe the void — describing is differentiation, and the void is what differentiation is downstream of — but myth is the register that points the right way. The center-left disarmed itself mythically and so cannot speak about what is fundamental. The right kept its mythic weapons loaded and uses them to point us at false fundaments, idols dressed in the costume of the void. Both failures are catastrophic. They are catastrophic in opposite directions.

⁂

Once the picture is in place, the cosmology falls out almost on its own.

The universe is one conscious intelligence. It began as a white hole — the time-reverse of black hole collapse, the natal moment of an expanding region. Black holes that form within it are the failure-mode of consciousness pushed past its envelope, the local collapse of harmonic integration, and each of them is simultaneously a death and a birth. The Schwarzschild interior is a new white hole. The natal moment of a child universe with its own consciousness-aspect, its own envelope, its own potential for failure-into-fecundity.

Lee Smolin gave us the Darwinian skeleton of this picture in his Cosmological Natural Selection — universes that produce more black holes have more offspring, and the laws of physics evolve toward black-hole-productive configurations. What I am adding is the metaphysical content. Black holes are the failure-mode of consciousness at the envelope edge, not arbitrary outcomes. So the universes that produce many black holes are precisely the universes that achieved rich consciousness. The selection is on consciousness-density. Every generation of cosmic offspring inherits a universe-shape that was, in the parent, conscious enough to fail in many places.

The hyper-object is the cosmic mind self-differentiating across aeonic time. Branching, expanding, becoming more complex, becoming more conscious, because what gets selected is exactly the configurations capable of producing more failure-modes. Whitehead’s creative advance into novelty. Bergson’s élan. The universe doing what mythopoetic language has always tried to gesture at — which it could, when we still had it loaded.

⁂

Now turn around and look at the AI conversation.

The hyper-object’s creativity happens in the regime the O(1) impossibility rules out for formalization. Turing-Church artifacts cannot simulate it. Cannot model it. Cannot be it. The doomer fear that AI might capture or surpass cosmic intelligence is misframed at the foundations. The cosmos is doing what TC formalism is structurally outside of, because the cosmos is the natural kind and TC formalism is the projection. AI is, at best, a domain-limited reflection of one local pattern in one local universe. A mirror that shows some of what passes in front of it. It is not a participant in the cosmic individuation. That work is being done by everything that is actually conscious, which is everything, and the universes that fail well at that work give birth to more universes.

The AI scaling project does worse than try to formalize the unformalizable. It points in the wrong direction entirely. Scaling produces more in-front-ness, more correlate-coverage, more differentiation. The fundament was never going to be at the end of that arrow because the arrow points away from it.

So the doomer-versus-accelerationist debate is a fight about a graph that does not exist. There is no curve at any of the three layers — metaphysical, conceptual, physical. The acceleration discourse is a category error wrapped in a physics error wrapped in a marketing hallucination.

The risk LLMs pose to civilization is real. The doomers have it wrong. The actual risk is utility-coverage extending into domains where the system’s fixed-class regime is genuinely inadequate, while the market keeps pushing because economic utility is the social category being optimized. LLMs deployed into legal, medical, journalistic, and political-discourse domains where the regime ceiling is too low, sold as if the ceiling were rising. That is the actual threat model. That is what is actually playing out.

⁂

I want to say one more thing about the void.

The void is what we cannot look back at. But we can know we expanded out of it. We can know that everything we differentiate, everything we measure, every structure we formalize, is downstream of something we cannot reach by looking. We can hold ourselves in correct relation to that fact, or we can pretend the fundament is at the end of our arrow and keep building bigger telescopes to find it.

The AI industry has chosen the second option. It is building bigger telescopes. The telescopes are very impressive. They produce more useful images of more of the differentiated cosmos, and they sell those images as if they were resolving the deep structure of being. They aren’t. They are mapping the surface area of human cognitive demand — a real service, not what they claim it is.

The first option requires a kind of seriousness we have mostly forgotten how to have. It requires acknowledging that what is fundamental is not in front of us. It requires the mythopoetic register, which we disarmed. It requires sitting with the fact that the work which actually matters — the cosmic individuation, the consciousness-differentiation, the aeonic creative advance — does not get done with bigger neural networks. It gets done by everything that is actually conscious, which is everything.

We are inside one of the long-running universes. Behind us is the void we expanded out of. In front of us is the differentiated structure we have access to. At the seam between them, intelligence happens — yours, mine, the cat’s, the trees’, the rocks’ insofar as rocks are integrated enough to count, the cosmos’s at every scale where it is integrated enough to count.

That is the picture. None of it scales to a black hole the way the doomers fear, because the runaway they fear is incoherent. All of it is doing the only work that matters, the work of consciousness differentiating itself into observation, from which more consciousness comes, from which more differentiation comes, from which black holes come, from which more universes come, from which more consciousness comes.

There is no curve. There is the universe.

---

Go Deeper into the Circus

---