🔆 The AI Cost Iceberg: Total Cost of Ownership (part 1 of 2)

🗞️ Issue 58 // ⏱️ Read Time: 8 min

Hello 👋

When people talk about the cost of AI, there's often a focus on the headline-grabbing training costs of large models like Deepseek-R1 and GPT-4. However, the true cost of AI adoption is far more nuanced and complex. This week, we're breaking down the real numbers behind AI development, training, and implementation to help decision-makers better understand the financial impact of AI.

AI Total Cost of Ownership

For most organizations, the cost of AI systems doesn't end when a foundational model is released into general availability; that's where it begins. Total cost of ownership (TCO) in the AI context goes beyond the traditional definition of purchase and deployment costs. It encompasses the entire journey of AI adoption: from initial acquisition and customization, through ongoing operations and maintenance, to the eventual evolution or retirement of the system. 

Unlike traditional IT systems, AI TCO is characterized by unique challenges:

• the need for advanced hardware infrastructure

• specialized expertise

• continuous model updates

• data quality management, and

• cultural transformation.

As organizations rush to adopt AI, understanding these comprehensive costs becomes crucial for sustainable implementation and long-term success. 

Today, we’ll discuss the cost considerations of build vs. buy, pay-as-you-go, and customization. Other key components like Design Thinking and Cost of Culture will be covered in next week’s newsletter, making this newsletter Part 1 out of 2. Let’s get into it!

The Build v. Buy Decision

One of the major strategic decisions organizations face when implementing AI is whether to build custom solutions or buy existing ones. This choice fundamentally shapes the TCO. In 2025, most organizations are purchasing AI in the form of closed-source foundation models and either using them off the shelf or customizing them to their needs. While buying existing solutions typically has lower upfront costs, organizations must carefully consider whether or not to rely on closed-source models. These models can offer stronger performance but also create dependency on vendor pricing and policies.

OpenAI, Anthropic, and Google lead the pack, but a few others are not far behind, including Meta’s Llama as an open-source option. Survey results indicate that very few (<5%) stick to one model vendor, and the average company has 2.3 model vendors in production, in an attempt to stay flexible.

The hybrid approach of using multiple vendors simultaneously does reduce dependency risks and allows you to leverage different models' strengths. However, it also introduces its own costs in terms of integration complexity and vendor management.

Regardless of whether you build or buy, you need to budget for integration, customization, maintenance, monitoring, and regular updates to keep systems performing optimally.

Pay-as-you-Go

Cloud computing popularized and expanded the use of the Pay-As-You-Go (PAYG) pricing model in the technology sector. AI API providers have widely adopted this model, and many companies are opting for cloud-based AI services to avoid large hardware investments.

Anthropic’s Claude 3.5 Sonnet API

Anthropic offers a PAYG model for its Claude API:

• Claude 3.5 Sonnet:  $3 / 1M input tokens and $15 / 1M output tokens

• Claude 3.5 Haiku: $.80 / 1M input tokens and $4 / 1M output tokens

Additional services like latency optimization for faster responses are priced separately.

Try this calculator to compare costs for sample text/code across models. 

The PAYG model allows businesses to scale their AI usage as needed without large upfront investments. However, it's crucial to carefully monitor and optimize usage to prevent unexpected cost escalations as applications grow.

Customization Costs

Foundation models like GPT-4 and Claude are trained on massive datasets collected from across the internet. This broad training helps them recognize patterns in everything from scientific papers to social media posts. While this extensive training makes these models versatile, the real costs begin when organizations need to adapt and customize them for specific business needs.

There are a few ways to customize a model for your given use case:

a) Prompt Engineering involves crafting specific inputs to guide the model's output without modifying its parameters. Costs include:

1. Skilled prompt engineers

2. Continuous optimization

3. Documentation management

4. Staff training

5. Regular updates as models evolve
   

b) Retrieval Augmented Generation (RAG) enhances LLM responses by referencing external, authoritative knowledge bases. Costs include:

1. Document processing infrastructure

2. Vector database management

3. Knowledge base maintenance

4. Continuous data updates

5. Integration with existing systems

6. Additional compute costs for retrieval
   

c) Fine-tuning involves adjusting a pre-trained model's parameters using task-specific data. Costs include:

1. Computing resources for model adaptation

2. Data cleaning and preparation (a top cost driver)

3. Testing and validation cycles

4. Performance monitoring

5. Regular updates and optimization

6. Subject matter expert time (10.4% of total costs)

📣 Recent research shows that data preparation alone accounts for 13.2% of AI implementation costs, with companies often underestimating the complexity of making these powerful but general-purpose models work for specialized tasks.

Beyond the Tech

Realizing the potential of AI requires a clear understanding of the full investment required. Organizations that thrive in this new era take a structured, deliberate approach to AI adoption - one that acknowledges not just the obvious technical costs, but the essential investments in design, culture, and human capability. 

Success requires sustained investment in design thinking and cultural change. In Part 2 of this series, we’ll go further into the costs associated with organizational readiness. Stay tuned!

Until next time.
On behalf of Team Lumiera

Emma - Business Strategist
Allegra - Data Specialist

Lumiera has gathered the brightest people from the technology and policy sectors to give you top-quality advice so you can navigate the new AI Era.

Follow the carefully curated Lumiera podcast playlist to stay informed and challenged on all things AI.