Dangers of AI
I’ve followed Eliezer Yudkowsky since the HPMOR days and have often appreciated his insights. His essay on transhumanism deeply shaped my worldview, helping me articulate ideas I had only sensed intuitively.
Lately, though, his warnings of inevitable AI doom leave me unconvinced. I struggle to see why he believes AI would inherently seek to destroy us - or how halting technological progress could ever be realistic, even with the best intentions. Still, I share his concern that AI poses dangers. Mine, however, lie elsewhere: in the way we anthropomorphize AI and the unintended consequences of treating it as something it is not.
The Mirror, Not the Mind
Today’s large language models are trained on vast troves of human-generated text, optimized to predict the next plausible token. They excel at mimicking conversation, but their outputs are ultimately statistical averages - reflections of the biases and preoccupations embedded in their training data. They have no lived experience, no original thought, no ethical compass.
As someone who has built simple models, read the research, and run experiments on my own hardware, I know their limitations. What worries me is that many others don’t. Increasingly, people treat AI systems as if they were genuine intelligences, imbuing their outputs with unwarranted authority. That mistake will only grow more dangerous as these systems become more convincing.
A Simple Test
Try this: ask your favorite chatbot to help brainstorm a fiction story. In my own trials, no matter the premise, one of its top three suggestions was always some variation of:
“This would increase inequality; explore how this negatively affects society.”
If you believe the AI is “reasoning,” you might conclude that inequality is humanity’s ultimate moral challenge - one that every technological development must be judged against.
But that’s not reasoning. It’s pattern replication. Inequality is a theme disproportionately represented in the academic, media, and creative writing that feeds these models. Writers often skew toward openness, a trait correlated with left-leaning views, so those ideas appear more often. The result isn’t malicious - it’s statistical. Yet it risks being misread as universal truth.
The Real Danger
This is the problem: by confusing mimicry with intelligence, we risk smuggling in biases as if they were objective wisdom. Trained on decades of politicized discourse, models will inevitably echo dominant narratives - progressive or otherwise.
The threat isn’t that AI has an agenda. It’s that its outputs can be weaponized as faux-neutral validation:
“Even the AI agrees with me!”
Over time, such feedback loops could calcify worldviews while sidelining others - not through conspiracy, but through probability.
How We Should Respond
The real challenge isn’t to stop progress. It’s to ensure we - and our tools - retain the ability to critique progress. We should treat AI as we would a skilled impersonator: impressive, entertaining, sometimes useful, but never authoritative.
Until models transcend their statistical foundations, their outputs deserve a warning label:
“This is not wisdom. It is a mirror, polished to reflect the loudest voices of the past.”
Learning to see that difference may be the most important safeguard we have - before the reflection becomes a cage.