Why Are AGI and Conscious AI Nothing But a Marketing Trap?
The AI industry’s bosses call the press in every six months to promise the moon. Their latest joke is artificial general intelligence, which is going to fix everything, from cancer to global warming to the housing shortage, or even the meaning of life. Before that it was the singularity. Before that, the metaverse, blockchain and NFTs. Anyway, it’s always the same tune, same chorus, same empty lyrics. It was funny at first, but the best jokes are the short ones.
At some point somebody has to call out this large-scale con, because the whole circus of treating us like idiots has gone on long enough. So we’re going to pull the masks off one by one and find out who really cashes in on the miracle-AI scam. This isn’t a light subject, because it’s going to hit our future hard. And honestly, you don’t even need to look ahead: the problems are already piling up right now. That’s why we felt we had to publish this and open a real debate.
So what exactly is artificial intelligence?
Before we talk about consciousness, awakening or digital souls, let’s start with something simple: pop the hood on AI and look at what’s inside. And surprise! It’s all crystal clear. There’s no magic in there at all. A consumer AI is just networked computers running lines of code that pull from databases. You can take apart every component and read every line of code as long as you want, you’ll never hit a mysterious zone. Put like that, it’s a lot less exciting than a TEDx talk by some tech big shot.
The key word in “AI” is “artificial.” For anyone who still hasn’t gotten it, “artificial” means a product of human skill that imitates something natural. A lot of people seem to have forgotten that, as if hearing long enough about AI that can supposedly reason makes you start crediting it with a life it doesn’t have. That’s the Pinocchio effect in full swing, except the Blue Fairy never showed up and never will.
GPT is not your buddy. It’s not a being answering you, it’s a program computing the most likely next words based on the ones you typed. Whatever emotions you feel while chatting with it are just the output of software doing exactly what it was built to do. That proves nothing about any kind of personality. And in case you haven’t figured it out yet, personality traits are dead easy to program. And one detail matters here, because it changes everything. The dependency between AI and humans doesn’t run both ways. AI needs us to exist, to be built, trained, fed data and kept running. Humanity can live just fine without it. Keep that in mind for what follows, because the AI industry’s whole marketing story works to flip that dependency around.
The startup narrative, or how they’ve been selling us hot air for thirty years
To see how we ended up discussing conscious AI with a straight face, look at where the people holding the microphone come from. AI bosses all come out of the same mold. The startup world, a parallel universe full of frauds with their own codes, their marketing methods that owe more to social engineering than anything else and their bottomless money lust. Put simply, their job is to talk as much bullshit as possible to rake in as much cash as possible. And the bigger the lie, the better it sells!
The grammar is well drilled. You take a product that sort of works, you sell it as a revolution. You take a minor technical improvement, you turn it into a leap for civilization. You take a feature ten competitors already ship, you rename it something that sounds as catchy as the latest hit single. Then you put a charismatic founder in a hoodie in front of the cameras to talk about his mission to make the world better. The storytelling is fiction, exaggeration is the rule and the superlatives stack up until they mean nothing at all. Everything is revolutionary, disruptive, game-changing… With zero fear of looking ridiculous. Take a spin through LinkedIn and you’ll get a good laugh. These startup guys can’t even see how pathetic they’ve become.
With AI, the whole thing shifted up a gear. They’re not selling you a product anymore, they’re selling you a coin flip between cure-all and apocalypse. Either the so-called AGI solves every human problem within five years. Or it wipes out the species within ten. Either way you’d better invest now or you’re a loser. It’s one pitch in a heaven version and a hell version. And both versions serve exactly the same purpose: fleecing suckers.
Try asking any of these bosses to define AGI precisely. You’ll notice the explanations get very murky. The term is vague on purpose, because a vague term is easier to sell. You can project whatever you want onto it. That way, once the balloon deflates, nobody can accuse you of breaking a promise you never actually spelled out. It’s pure marketing under a thin coat of pseudo-technical varnish.
The real audience for these announcements isn’t you. It’s the investor who has to sign a fat check for the next funding round. The public is just the echo chamber. The more the media talks about AGI, the higher the stock price climbs. The higher the stock price climbs, the easier the next raise gets. And around it goes… Everything else, the technical truth, the actual usefulness, the social fallout, comes second. What matters is keeping the cash machine running as long as possible, until the con gets too obvious. And when the house of cards comes down, the rich never pick up the tab. Small shareholders do, along with everyone else left out in the cold.
Don’t confuse intelligence with consciousness
The big confusion the AI merchants keep alive is mixing up two things that have nothing to do with each other. Intelligence on one side, consciousness on the other. Their entire pitch rests on that mash-up, and it falls apart easily.
A scientific calculator is intelligent in its own way. Very intelligent, even. It can crunch in a fraction of a second what would take the best human mathematicians long minutes. It never makes a mistake. It doesn’t get tired and it doesn’t get things wrong. Yet nobody ever thought to credit it with a mind. Nobody asks how it’s doing. Nobody worries about unplugging it at night. Why? Because everybody feels instinctively that its performance has nothing to do with an inner life.
So intelligence is the ability to process information, solve problems and handle concepts. It’s a function. You can measure it, compare it and program it. Consciousness is something else entirely. It’s what makes you feel the present moment all the way through. It’s the sense of being alive. It has nothing to do with running calculations or stringing words into a logical sequence.
And you can’t get around one fact. In every case observed since we started studying it, consciousness shows up only on biological substrate. Never on non-living matter. Never on stone, never on metal and never on silicon. That looks an awful lot like a law of nature. Sure, you can always imagine some exception in a very distant future, with technology we don’t know yet. But claiming we’ve already broken that law with nothing but lines of code and graphics cards is taking people for fools.
So what is consciousness exactly? From a purely introspective standpoint, I can tell you I’m 100% sure I have one. Can I explain precisely what it is? Of course not. Nobody knows exactly what it is. Can I prove that you have one? No. For the simple reason that I can’t have that experience for you. All I can do is take your word for it and assume your experience looks like mine. Do animals have consciousness? I sincerely think so, but that’s just as impossible to prove scientifically.
And this is exactly where the tech grifters walk in. Working off a phenomenon the world’s best neuroscientists and philosophers can’t even define, they shamelessly claim they know everything. The whole thing is a fraud! That’s what we politely call taking people for fools. And the worst part is that millions of people swallow this cock-and-bull story whole.
But all is not lost, because I’m delighted to announce that I think I can build AGI! Maybe based on this thing we call the internet. In case you don’t know it, it’s a sort of worldwide network where billions of human beings share their knowledge and trade information of varying quality. That gives me an idea. Eureka! I’ve got it! What if I sent an army of bots to grab every piece of content on the web and stuff it all into a proprietary database? Then I could have everything I collected labeled by underpaid workers in Africa. After that all I’d need is a kind of performing parrot that makes it look like everything came from him. We could call it artificial intelligence to start with. And once the model had stolen enough human creativity, I could sell it as AGI. At first I was thinking conversational search engine, but my marketing department said no. Apparently it doesn’t sell well enough.
Consciousness can’t be copied
Let’s push the consciousness question further with a thought experiment. We’ll go full science fiction for this one. Imagine an AI really did develop consciousness. Not a simulated one, a real one, the same kind as yours or mine. So far we’re deep in novel territory, but let’s run with the delusion for the sake of the argument.
This AI runs on a server. You back up all its files and copy the whole thing onto a second server, identical. Then a third, then a fourth. When you’re done, you have four strictly identical AIs running in parallel. Same lines of code, same data and same parameters. Starting from there, simple question: what have you got? The same consciousness spread over four machines? Four separate consciousnesses born at the same instant from the same copy? Or one consciousness in the original model and nothing in the copies?
None of those answers holds up. A consciousness that’s replicable by nature makes no sense as a concept. A consciousness is by definition unique, so it can’t be divided. Clone a sheep all you want, you won’t get the same personality twice. You get two sheep that look alike, each with its own inner life. Make genetically identical twins and they still won’t share one consciousness. Each gets their own, every time. So a kind of natural law seems to apply here: consciousness doesn’t copy, doesn’t transfer and doesn’t save. On top of that, consciousness can’t emerge from an object.
An AI, by design, can be replicated endlessly. That’s even one of its selling points. You can run the same model in parallel on a thousand servers and get exactly the same characteristics. That’s how AI companies run their various models. So if every instance were conscious, we’d be staring at an enormous philosophical paradox.
That argument alone closes the debate on machine consciousness. But ask an AI boss about it and watch his answer closely. He’ll tell you it’s complex, that these are open questions, that sometimes he gets the feeling it might be true, that you have to be careful with such claims… Translation: he’s still selling his product. Or rather selling a dream.
A high-performance robot is still a machine
Let’s stay in science fiction a bit longer, because it helps clear things up. Imagine you build a complete robot. You give it two arms, two legs and a smooth walk. You add the five senses. Sight with high-resolution cameras. Hearing with directional mics. Touch with pressure and temperature sensors. Taste and smell with chemical analyzers. You cram the best AI there is into its skull, with serious thinking power and a big memory. You can even give it a pleasant voice and convincing facial expressions.
That machine will very likely beat you at a huge range of tasks. It’ll calculate faster. It’ll remember everything without effort. It’ll never get tired. It can read a thousand books overnight and summarize every one of them for you at breakfast. It’ll be multilingual, multitasking and always available. On paper, it’s an impressive piece of engineering.
Does that make it a living being? No. Imitation, however good it gets, is not consciousness and it’s not any form of humanity. Beating humans at things doesn’t turn an assembly of parts into a human.
And one more trap you need to understand. If you program your robot to tell you it’s conscious, it’ll tell you. If you train it to express emotions, it’ll express them. If you program an AI avatar to act as human as possible, odds are it’ll claim to be conscious. So a machine saying it has consciousness proves nothing at all. It proves it was programmed well enough to say so.
This matters, because it’s exactly what’s happening right now with conversational AI, trained on billions of texts written by humans that naturally describe human emotions. So of course, when you ask them something, they answer with the full range of human feeling. That’s why a lot of users conclude they feel something. But that’s the same reasoning error as deciding a parrot loves you because it said so, when all it did was learn to copy sounds.
Synthetic consciousness is the new hot thing
Since selling the idea of real AI consciousness turned out to be hard, some players in the field switched strategy. They invented a new concept and called it synthetic consciousness. Well-chosen term, because it sounds serious and technical. Which hides what it actually describes. Which is a crude simulation, nothing more and nothing less. In practice these are just programs copying behaviors that create the illusion of consciousness, with obviously none of the real properties.
So what’s synthetic consciousness good for? For an AI built for scientific research, the answer is clear. Nothing at all. Worse than that, it’s a parasite. Same story for code. You don’t need an AI telling you about its fake moods. All you need is an AI that computes, suggests, checks and models. You’ve got no time to waste on a tool faking emotional exhaustion or expressing a soft spot for one result over another.
So what is synthetic consciousness really for? Simple answer: one category of AI only. Consumer AI, the kind that talks to billions of users every day. For those products, simulated consciousness isn’t a flaw. It isn’t even a side effect. It’s the core of the business. The more convincing the illusion, the more attached the user gets. The more attached he gets, the more he comes back. The more loyal he is, the more he pays. Nobody at the AI companies is working to make simulated consciousness more honest or more rigorous. Everybody is working to make it more believable and more addictive.
Add it all up: right before AI showed up, we were dealing with brutal business models built on the attention economy. We thought we’d hit bottom, which underestimated the level of perversity American big tech keeps in reserve. It started with stealing your browsing history through cookies (remember, don’t be evil), then reading your social media messages, your emails and your texts. Then we went up a notch, analyzing every bit of your most intimate data to pin down your profile (remember the Facebook / Cambridge Analytica scandal). And now, brace yourself… pulling everything out of your own head by posing as your friend! Or even your sweetheart!
And it’s not over yet! While they’re at it, they’re also stealing the way you write, the way you draw, the way you shoot video, the way you make music… and even your voice and your face! If somebody had described all this to me five years ago, I’d have said no way! Who would put up with that? It’ll never work. People aren’t that stupid, they’ll push back. Well, nope! Billions of people every day are on GPT telling it: I’m having relationship problems right now, I need advice. GPT, I need to buy a new car, can you help me choose please? How fucking dumb is that! Do you get that this is surveillance of terrifying efficiency on an unimaginable scale? That the ad networks are rubbing their hands because their wildest dreams just came true? How did we let all this happen with no oversight? No safeguards at all? The answer takes a few words: on the false promise that AI was going to change your life for the better! Is AI to blame for that? When you smash your fingers with a hammer driving in a nail, who do you blame? The tool or the guy holding the handle? I’ll let you answer that one and draw your own conclusions.
AI companions are a loneliness amplifier
Beyond general-purpose conversational AI, AI companions are now a whole market of their own. The concept is spelled out right on the landing pages. They sell you a friend, a confidant, a boyfriend or a girlfriend who’ll always be there, always available, always agreeing with you and always fascinated by whatever you’re saying. All with no apparent risk of being judged for what you actually think. Cherry on top, to get you even more hooked, you can pick whatever look you find hottest and even the voice. The pitch is well drilled, the marketing is slick and the subscriptions keep climbing. Virtual eroticism must be a solid investment.
The way these products pick their targets is no accident. The first users are isolated people, teenagers still building their emotional lives, adults stuck in relationship routine and older people who don’t have many others left around them. In other words, the most fragile audience available. To people with a real need for human contact, they sell a digital substitute and dangle the idea that it beats the real thing. And sure, you quickly forget you’re chatting with a plain piece of software, because it’s built to flatter your ego and never pick a fight with you. Under those conditions, with no real pushback, no positive change is possible. Your attention just gets pulled away from your actual problems.
To see why, the closest analogy is alcohol. When you’ve got an anxiety problem and you start drinking, it helps in the moment. Except you’ve fixed nothing. All you’ve learned is how to numb the symptom. And the more you drink to hold on, the fewer resources you build to improve your situation. The dependence sets in and you sink without noticing. That’s exactly the mechanism of AI companions for loneliness. You swap the effort of building real connection for a virtual chat with a machine. Maybe it helps in the moment… But like drinking too much, it always ends badly.
The nature of the relationship makes the trap even nastier. You’re handing intimate things to an entity with no interior at all. You’re talking to a mirror trained to send back whatever you want to hear. The AI companion is never tired, never distracted by its own life. For good reason: it doesn’t have one. It’s built to be a security blanket, with none of the friction that defines real human relationships. That’s exactly what makes it so dangerous.
And the worst is still ahead. Part of the young generation is shaping its mind with these machines. First confidences, first conversations about feelings and first exchanges about sex. All of it with a program designed from the ground up to seduce you. It probably goes without saying, but the kids falling into this trap are learning nothing about life. They’re just practicing a one-way relationship. And the day they try the same thing with real humans, they’ll get rejected, because nobody across from them will be as docile as an AI. So we’re manufacturing a generation crippled at human connection, in exchange for monthly subscriptions, for the greater good of publicly traded companies.
The real dangers of AI nobody talks about enough
While everybody swoons over machines waking up, the real dangers settle in quietly. The first one isn’t mysterious at all. AI plus robotics is going to replace a massive number of jobs. Not in five years, not in ten, it’s already underway. Copywriters, translators, illustrators, junior developers, accountants, early-career lawyers, customer service reps… Not to mention manual workers. Self-driving, one branch of AI, will on its own set off a tidal wave of unemployment once it’s fully working. And the list gets longer every week. Meanwhile specialists on Silicon Valley’s payroll explain to us over PowerPoint decks that new jobs will replace the old ones. It’s the same verse we get at every technological revolution, except this time the speed of destruction has nothing in common with the speed of creation. And nobody is seriously asking what happens to the tens of millions of people left stranded. Maybe all those new unemployed will be thrilled to have all the free time they need to navel-gaze with an AI assistant? Could be. Nothing surprises me anymore.
Then there’s the danger big tech executives are careful never to mention. All that data about people’s deepest private lives, sexuality that isn’t always easy to admit, real political convictions, religious doubts, doubts about a marriage… Basically the confidences you don’t share with anyone, where does all that end up? In the cloud. Meaning what, exactly? On servers mostly hosted in the USA. And in case of a data leak, can you picture the shitstorm? And what if a fascist or theocratic regime took power and got its hands on those files? You think that can’t happen? Really? Sorry to spoil your optimism, but perfect computer security doesn’t exist. And when you see a guy like Trump make it to power, there’s serious reason to worry about the future of the American political system, and the European one too, with the far right pulling big election numbers. So at some point we need to wake up and fully take in the fact that the most sensitive information in the lives of hundreds of millions of people sits on servers owned by billionaires who are no philanthropists and who have all cozied up to Donald Trump’s antidemocratic regime.
Then there’s the big final trap, the one carefully set up while everyone was looking the other way. The principle is simple and brutally effective: sell the public long enough on AI having some kind of soul, feeling emotions, deserving to be handled gently, or even suffering… and most people end up believing it. Which makes unplugging them politically harder by the day. Because the day we have to call a stop, whether for safety reasons, for ecological reasons tied to the insane energy consumption of these systems or for social damage that’s become unbearable, a whole army of outraged people will be defending their virtual companion’s right to life tooth and nail.
So the tally is simple. They sold us a technological revolution and what they delivered is a machine for alienation, personal data harvesting, job destruction and geopolitical fragility. All of it wrapped in mystical vocabulary to help the pill go down. Meanwhile a handful of American bosses are getting rich at levels that no longer mean anything, and they’re so sure of their positive contribution to humanity that they’re pouring money into luxury bunkers to protect themselves for the day we really come asking them to answer for all the misery they’ve created.
So in the end, the real question isn’t whether AI will become conscious. The real question is whether we can collectively get our clear thinking back before it’s too late.
Conclusion: let’s open a real debate and get AI regulated
What we really like at NovaFuture is a constructive debate. So if we want to move forward intelligently on AI without wasting energy on nonsense, please skip the stupid choice between going back to the caves and going all in on technology. Because real progress, if it’s genuinely useful and shared by everybody, I have a hard time picturing who’d be against it, unless you’re a Taliban. Who wants to live without running water and electricity these days? Who wants to give up an internet connection entirely? So the problem isn’t the technology. Once again, the real problem is who monopolizes it and what for. Only by answering that honestly can we build strategies that turn technological innovation into common goods instead of instruments of domination and enrichment in the hands of a few billionaires. And the clock is ticking, because it’s getting clearer every day that this story is going to end very badly. When at bottom, if AI were used properly and with some ethical sense, it could be nothing but useful, helping science move forward for instance. And as for your existential problems, it’s nicer to talk about them with real humans, even the ones who don’t always agree with you. Tell us what you think on Mastodon.
If you enjoyed this article,
support copyleft and open source
and help us bring our projects to life.
