Meta claims it is “building the future of human connection.” But unless you’re living under a rock, and maybe even if you are and can get bars, most of us know—or should—that they ain’t doing it out of the goodness of their hearts.
So when the CEO drops a manifesto about the future of AI, it deserves a little extra scrutiny.
Because Meta is really an ad company. In fact, they’re poised to overtake Google this year as the world’s largest digital advertising company, and already earns nearly $200 billion in annual advertising revenue.
Yet the manifesto spent exactly zero word count explaining how or why one of the world’s most profitable ad companies fits into the future of AI other than at the altruistic level. The Zuckster’s special glasses and all-knowing social media platforms will make AI the servant that really knows you, and will really help you be everything you can be.
Colour me skeptical.
AI with intentions
In the manifesto, titled Personal Superintelligence, Zuckles argues, broadly, that the future of AI shouldn’t simply be a handful of giant systems deciding what intelligence looks like for everyone. It should be personal. Everyone should have access to their own superintelligence: an AI that understands you and can supercharge your capabilities, knowledge and productivity.
I almost hate that, because I agree—AI could be a natural extension of my own knowledge and abilities that allows me to do more, be more, just generally dabble and even possibly excel at things that previously required expertise I don’t have.
But Zuckferatu’s vision goes further. Your AI will know you intimately (you are your Insta!), see what you see (invasive glasses anyone?), and understand what you care about. It could organize your life, answer questions, make decisions, choose your outfits and your words. It would know an extraordinary amount about you based on how you interact with Meta platforms, and use that familiarity to optimize your life.
Who will your AI work for?
For me, AI shaping its suggestions and perception of who I am and what I want based on my social media sounds dystopian. And anyway, is most people’s social media use an accurate or useful reflection of who they are and want to be? I see mostly primping and performance, the face we want the world to see. What happens when personalized AI is built around the digital projection of yourself, instead of the real you?
Especially if it is engineered around my second concern—as a digital ad whisperer.
Meta is an advertising company first and foremost. Its platforms are built around understanding and exploiting human behaviour to capture attention, maintain engagement and sell advertisers access to it. Whatever else Zuckletoes wants Meta to be, advertising is its true nature. And in any case, Meta has repeatedly been accused of prioritizing engagement despite its consequences—why would that change now?
Now imagine that machine integrated with the personal AI system Zuckzilla wants to be one of the architects of: your personalized superintelligence, only now with a motive that isn’t necessarily in your best interests.
Advertising is not the enemy (for once)
I’m not opposed to advertising—I’m a copywriter, and I’m all for the art of persuasion.
Nor am I against a personal AI that recommends commercial products to me—in fact, I want it. If I need help choosing a camera, a restaurant, a car… whatever, I do want it to be very good at recommending specific products and businesses.
It’s the commercialization of influence that worries me. Because your relationship with personal AI is fundamentally different from your relationship with an ad. You know an advertisement is trying to persuade you—that’s how it works. You can pay attention to it, ignore it, question it, compare it with something else.
A personal AI, on the other hand, is supposed to be acting on my behalf. I’m telling it things I wouldn’t tell an advertiser. I’m allowing it to learn my habits, weaknesses, ambitions and preferences. And then I start to trust it, and rely on it, and probably shift a whole lot of choice and decision-making onto it.
And that’s the point where I want AI to be able to answer one simple question:
Whose interests are you representing?
The answer better be “yours and only yours.” Otherwise it’s just an advertising platform with an extraordinarily intimate—and influential—user interface.
One rule to rule them all
If we’re really heading toward Meta’s world of democratized personal superintelligence — an additional intelligence sitting alongside our own and helping us navigate our lives — then we should set the rules about that relationship. Not them.
Here’s what I’d start with:
Commercial relationships can never influence what my AI concludes is best for me.
That doesn’t mean AI can’t contain advertising, or that companies can’t advertise to me, or that AI can’t recommend a brand. It just means the recommendation itself cannot be for sale. If I ask my AI to assess, recommend or help me make a decision, money paid by a third party cannot secretly alter what it concludes.
In other words, you can’t just buy my personal AI’s influence on my judgement, and by extension, the trust I put into it.
Build the AI you want to see
We’re already surrounded by subtle influence all the time. Culture, brands, other people…. And algorithms already influence what gets put in front of us.
But a commercialized personal AI would be a level of engagement the most unscrupulous advertisers can only dream of. As envisioned by Zuckwad, it would become intimately intertwined with who you are and how you act, a trusted personal influencer for almost every decision you’ll ever need to make. In other words, it will have your trust, and by extension a lot of power over your judgement and actions: are you really ready to let Meta sell that leverage to the highest bidder?
So as long as we’re debating the future of AI, let’s not forget a line item that should be as important as preventing job apocalypses and robot takeovers: AI’s judgement about what is best for me must never be for sale.
And we should do it now. Because when the CEO of an advertising-dependent business doesn’t even bring it up in a manifesto on AI, we already know where this is headed.