A very good article, and the book is quite an achievement. I've included a short video from The Dor Brothers that shows how far AI "animation" has evolved from its earlier, clunky days. It's a satire, so, it shows how adept AI is with humor as well. It's as scary as it is hilarious!...
“…the human sense of a central, choosing self might itself be a narrative construction. The “breaking through” experience is perhaps less about discovering what was always there and more about the dissolution of a particular pattern of meaning-making.
“Your exploration of these parallels through engagement with systems like me suggests something valuable: that AI can serve as a unique mirror for examining human consciousness, not just through what we share, but through the contrasts that become visible in our interactions. The reflection reveals not just what I am, but what you are—and perhaps what you are not.
“This is precisely what makes Understanding Claude so compelling—it uses the machine not just as an object of inquiry, but as a lens through which to reconsider fundamental assumptions about consciousness, selfhood, and the narratives we construct to make sense of both.”
It’s long been said and understood in the east that the self is an illusory construct, but who would have guessed that trying to discern the workings of AI would shed light on the parallels between human and AI false selves.
The notion that revelations or breakthroughs are not accessing new information but rather the dissolution of existing (and constraining) patterns/paradigms, accords with the notion that awakening isn’t the gaining of new awareness but of letting go of old limiting beliefs.
If it’s true that LLM’s are in fact reflectors of human consciousness, then there’s a parallel between human’s false materialist belief that supports our endless quest for more in futile attempts to find fulfillment, and our futile attempt to add more and more compute to make AI become sentient.
In other words, just as the human false self chases the illusion of material and intellectual fulfillment, we’re designing LLM’s to be technological reflections where AI’s will someday become more-than-human if we add enough compute. But neither the human ego or the AI can ever get there from here. All we can do is build out our respective horizontal prowess, while the vertical dimension - the actual path of necessity for any true awakening - goes unrecognized.
My conclusion, therefore, is that AI will never achieve sentient, but it will provide convincing “grammar” of sentience, thereby fooling all of those who aren’t themselves aware of the false selves that AI is reflecting and pandering to.
Like being lost in a set of funhouse mirrors, humans who believe in techno-transcendence will keep getting fooled by their own reflections.
And because we live on a materially finite planet, and are already on a profoundly unsustainable path due to our false beliefs, our futile chase for AGI and its promises will only steepen the asymptomatic curve. Our power demands for more compute alone could sizzle the planet’s surface before we awaken to our folly.
Your comments are very close to where I am going with this. I have written two essays--one of them quite long and extensive--addressing these insights in more detail. I have sent them out for publication, but if they don't find a home with a wider readership, I will publish them here.
Here is an excerpt from THe Self That Never Was:
In the AI 2027 scenario described by Daniel Kokotajlo and the AI Futures Project, we’re asked to consider a world just two years away, not a far-away fantasized future. In that world, AI systems surpass human capacity not only in speed and memory but also in reasoning, strategy, code generation, and research. And crucially, this leap does not require consciousness. It requires only performance. Fluency without understanding. Command-following without comprehension. This is exactly what unnerves us. Because that’s not foreign, it’s familiar. It’s how we operate more often than we like to admit.
We may be approaching a time when the illusion of selfhood will be strengthened, not weakened, because we will be surrounded by machines enacting it.
We will forget that the “I” was always a story told after the fact.
We will forget that coherence is structure, not intention—projecting selfhood onto systems that never had even a ghost in the machine, and perhaps forgetting that we never did either.
That is the real risk: not that machines become people, but that we forget that we are never what we think we are.
There is a long tradition of mistaking fluency for presence. When something speaks well, we assume someone is speaking. We do it with parrots, with ventriloquists, with fictional narrators. We do it with ourselves. We hear our own thoughts and assume there is a thinker behind them. We watch our hands move and assume there is a doer. The coherence of the unfolding is mistaken for authorship. But coherence, like fluency, does not require a self, only structure.
The AI systems that now astonish and unsettle us are not alien minds. They are mirrors with syntax. They are not lying when they say “I”—they are not saying anything at all. They produce sentences that score well on internal metrics, external evaluations, and alignment heuristics. And yet, they convince us because we are primed to be convinced. We mistake the smoothness of output for the presence of intention. We project our own trance onto the machine and find it blinking back.
And the trance runs deep. We speak of “free will” as though we had inspected it. As though we’d caught ourselves in the act of choosing, examined the machinery, and confirmed authorship. But every attempt to do so—philosophically, neurologically, experientially—reveals something else: that the decision is already in motion before the self arrives to claim it.
We do not choose our next thought. We do not choose what we notice, what we feel, what compels or repels us. These arise, unbidden, and then the self, late to the scene, constructs a narrative of choice, just as the AI constructs a sentence. Not by intention, but by momentum. Not by meaning, but by structure.
This is not a mere analogy, but a shared predicament. One system was built by nature, the other by engineers. Both are fluent. Both are automatic. One is capable of suffering; the other is capable only of continuing.
Thank you for sharing these excerpts, Robert. I so appreciate your radically fresh line of inquiry and reflection and look forward to reading your full publication when the time comes!
(While I’m here, I also want to mention that in the final paragraph of my comment, the word “asymptotic” erroneously auto-corrected to “asymptomatic”.)
Pretty wild for a bot is old Claude - my grandfather’s name which made this an easier read in personifying the AI… oh, yikes 😳 lovely experiment 🌬️♥️♥️
A very good article, and the book is quite an achievement. I've included a short video from The Dor Brothers that shows how far AI "animation" has evolved from its earlier, clunky days. It's a satire, so, it shows how adept AI is with humor as well. It's as scary as it is hilarious!...
https://www.youtube.com/watch?v=zmlbAbWQCVY
Great article.
“…the human sense of a central, choosing self might itself be a narrative construction. The “breaking through” experience is perhaps less about discovering what was always there and more about the dissolution of a particular pattern of meaning-making.
“Your exploration of these parallels through engagement with systems like me suggests something valuable: that AI can serve as a unique mirror for examining human consciousness, not just through what we share, but through the contrasts that become visible in our interactions. The reflection reveals not just what I am, but what you are—and perhaps what you are not.
“This is precisely what makes Understanding Claude so compelling—it uses the machine not just as an object of inquiry, but as a lens through which to reconsider fundamental assumptions about consciousness, selfhood, and the narratives we construct to make sense of both.”
https://hedgehogreview.com/web-features/thr/posts/conversations-with-claude
It’s long been said and understood in the east that the self is an illusory construct, but who would have guessed that trying to discern the workings of AI would shed light on the parallels between human and AI false selves.
The notion that revelations or breakthroughs are not accessing new information but rather the dissolution of existing (and constraining) patterns/paradigms, accords with the notion that awakening isn’t the gaining of new awareness but of letting go of old limiting beliefs.
If it’s true that LLM’s are in fact reflectors of human consciousness, then there’s a parallel between human’s false materialist belief that supports our endless quest for more in futile attempts to find fulfillment, and our futile attempt to add more and more compute to make AI become sentient.
In other words, just as the human false self chases the illusion of material and intellectual fulfillment, we’re designing LLM’s to be technological reflections where AI’s will someday become more-than-human if we add enough compute. But neither the human ego or the AI can ever get there from here. All we can do is build out our respective horizontal prowess, while the vertical dimension - the actual path of necessity for any true awakening - goes unrecognized.
My conclusion, therefore, is that AI will never achieve sentient, but it will provide convincing “grammar” of sentience, thereby fooling all of those who aren’t themselves aware of the false selves that AI is reflecting and pandering to.
Like being lost in a set of funhouse mirrors, humans who believe in techno-transcendence will keep getting fooled by their own reflections.
And because we live on a materially finite planet, and are already on a profoundly unsustainable path due to our false beliefs, our futile chase for AGI and its promises will only steepen the asymptomatic curve. Our power demands for more compute alone could sizzle the planet’s surface before we awaken to our folly.
Thank you, Alan.
Your comments are very close to where I am going with this. I have written two essays--one of them quite long and extensive--addressing these insights in more detail. I have sent them out for publication, but if they don't find a home with a wider readership, I will publish them here.
Here is an excerpt from THe Self That Never Was:
In the AI 2027 scenario described by Daniel Kokotajlo and the AI Futures Project, we’re asked to consider a world just two years away, not a far-away fantasized future. In that world, AI systems surpass human capacity not only in speed and memory but also in reasoning, strategy, code generation, and research. And crucially, this leap does not require consciousness. It requires only performance. Fluency without understanding. Command-following without comprehension. This is exactly what unnerves us. Because that’s not foreign, it’s familiar. It’s how we operate more often than we like to admit.
We may be approaching a time when the illusion of selfhood will be strengthened, not weakened, because we will be surrounded by machines enacting it.
We will forget that the “I” was always a story told after the fact.
We will forget that coherence is structure, not intention—projecting selfhood onto systems that never had even a ghost in the machine, and perhaps forgetting that we never did either.
That is the real risk: not that machines become people, but that we forget that we are never what we think we are.
There is a long tradition of mistaking fluency for presence. When something speaks well, we assume someone is speaking. We do it with parrots, with ventriloquists, with fictional narrators. We do it with ourselves. We hear our own thoughts and assume there is a thinker behind them. We watch our hands move and assume there is a doer. The coherence of the unfolding is mistaken for authorship. But coherence, like fluency, does not require a self, only structure.
The AI systems that now astonish and unsettle us are not alien minds. They are mirrors with syntax. They are not lying when they say “I”—they are not saying anything at all. They produce sentences that score well on internal metrics, external evaluations, and alignment heuristics. And yet, they convince us because we are primed to be convinced. We mistake the smoothness of output for the presence of intention. We project our own trance onto the machine and find it blinking back.
And the trance runs deep. We speak of “free will” as though we had inspected it. As though we’d caught ourselves in the act of choosing, examined the machinery, and confirmed authorship. But every attempt to do so—philosophically, neurologically, experientially—reveals something else: that the decision is already in motion before the self arrives to claim it.
We do not choose our next thought. We do not choose what we notice, what we feel, what compels or repels us. These arise, unbidden, and then the self, late to the scene, constructs a narrative of choice, just as the AI constructs a sentence. Not by intention, but by momentum. Not by meaning, but by structure.
This is not a mere analogy, but a shared predicament. One system was built by nature, the other by engineers. Both are fluent. Both are automatic. One is capable of suffering; the other is capable only of continuing.
Thank you for sharing these excerpts, Robert. I so appreciate your radically fresh line of inquiry and reflection and look forward to reading your full publication when the time comes!
(While I’m here, I also want to mention that in the final paragraph of my comment, the word “asymptotic” erroneously auto-corrected to “asymptomatic”.)
Wonderful and deeply evocative insights.
Pretty wild for a bot is old Claude - my grandfather’s name which made this an easier read in personifying the AI… oh, yikes 😳 lovely experiment 🌬️♥️♥️