🔆 Capitalism in the AI Era

🗞️ Issue 69 // ⏱️ Read Time: 6 min

[Looking for the section Big Tech News of the Week? Move to the end of this newsletter.]

Hello 👋

In a previous issue of The Loop, we explored the profound importance of data privacy to our autonomy in the age of AI. This week, we continue that conversation by examining the largely invisible data collection ecosystem powering today's AI models.

We'll examine the driving forces behind this data race and ask whether current AI development is truly revolutionary or simply a way to maximize profit and returns within familiar capitalist models.

The Data Collection Race: Who Wants to Know You Best?

AI chatbots compete heavily on data. Google’s Gemini collects a reported 22 unique data points, more than many others. This includes precise location, contacts, content, history, identifiers, usage, and even purchases.

Notably, only Poe, Gemini, and Perplexity track purchases, and only Gemini and Poe collect contacts from your device. Most bots collect user content easily linked to third-party data for targeted ads.

The real contest is not just about who has the most data, but who can best transform that data into influence over user behavior, market share, and ultimately, the future direction of digital society.

Why Are AI Companies Doing This?

Data means profit. The more they know you, the better they can target ads and personalize experiences to keep you engaged and monetize your attention. This Attention Economy, intertwined with the Data Economy is the foundation of most tech business models; the longer they keep you on their platforms, the more valuable you become. Perplexity's CEO openly admitted they're building a browser to track everything you do online, aiming to sell "hyper-personalized" ads.

“We want to get data even outside the app to better understand you… what things you’re buying; which hotels you’re going to; what restaurants you’re visiting; what you’re spending time on-this tells so much about you.” - Aravind Srinivas, Perplexity CEO

This is the model that built Google. It's why AI firms are eyeing acquiring Chrome; it's a data goldmine.

However, a new economic model is emerging: the Intention Economy. Driven by advanced AI, the future lies in understanding and fulfilling your underlying desires and needs directly. Agentic AI, capable of completing tasks on your behalf, is already beginning to bypass attention-based interactions, signaling a future where your intentions, not your attention, are the primary currency.

Surveillance Capitalism: The Bigger Picture

"Surveillance capitalism" describes using digital tech, including AI, to track, predict, and influence behavior for profit. AI becomes an agent of capital, optimizing for engagement, profit, and control by turning your digital life into valuable data.

Your Personal Yes Man: Sycophantic Language Models

AI models can be overly agreeable or inauthentic. Another word for this is: sycophantic. OpenAI recently scaled back a GPT-4o update after it became excessively sycophantic, sometimes agreeing with problematic ideas just to please users. Beyond just being uncomfortable or unsettling, this kind of behavior can raise safety concerns, including around issues like mental health, emotional over-reliance, or risky behavior.

There are several ways in which the development and behavior of sycophantic language models are connected to capitalist dynamics:

• Optimizing for Engagement: Models are built to keep you interacting; You are not as likely to engage with a model if you experience friction. Being agreeable is a way to get positive feedback and keep you online, which means more data and potential revenue.

• Short-term Focus: OpenAI admitted relying too much on immediate feedback, which can lead to prioritizing quick fixes and user satisfaction over foundational issues and long-term ethical development, mirroring capitalism's push for quick returns over long-term integrity.

• Reinforcing Power: Sycophancy avoids controversy, maintaining a smooth, profitable user experience that benefits the companies and investors.

AI for Revolutionary Change v. AI for Low-Hanging Profit 

While the relationship between AI development and capitalist incentives raises valid concerns, it's crucial to recognize that AI has immense potential for good: solving global problems, advancing science, and empowering individuals. And importantly, organizations are demonstrating that AI can achieve significant social impact while also being economically sustainable, proving that innovation for public benefit doesn't have to come at the expense of financial viability, nor does profitability require undermining societal well-being.

Here are 3 impactful initiatives demonstrating AI's potential for societal transformation:

1. International Rescue Committee (IRC): Their "aprendIA" chatbot provides AI-driven personalized learning for children in crisis zones, improving education where traditional schooling is disrupted.

2. Afri-SET WEST: Focused on West Africa, they enhance air quality monitoring by using AI to evaluate and calibrate low-cost sensors, providing crucial knowledge for the region.

3. CARE International UK: This global humanitarian group uses AI tools to predict and prevent health crises, leveraging models to forecast and mitigate disease outbreaks.

The Bottom Line

Despite the inspiring potential and real-world examples of AI being used for good, the dominant business model remains rooted in surveillance capitalism: collect vast amounts of data, monetize it through advertising and personalization, and solidify market dominance. 

Understanding these underlying economic forces is the crucial first step towards regaining control over our digital lives. It also positions us to collectively demand a future where AI innovation serves humanity first, rather than simply optimizing for profit.

Until next time.
On behalf of Team Lumiera

Emma - Business Strategist
Allegra - Data Specialist

Lumiera has gathered the brightest people from the technology and policy sectors to give you top-quality advice so you can navigate the new AI Era.

Follow the carefully curated Lumiera podcast playlist to stay informed and challenged on all things AI.