Note · AI Implementation Strategies

Before You Commit YourCompany to One AI Model

David He, FounderNovember 22, 20268 min read

Switching AI models is easy and rarely done. Eight questions to answer before you commit your company to one model, with sources for every number.

The same handful of questions come up whenever I talk AI with people inside larger companies. How stuck are we once we build on a model? If prices keep falling, who gets the savings? Why does approval take so long? Will today's leader still be the leader in three years?

They're good questions, and the quick answers that get passed around tend to skip the parts that cost money later. So here are the answers I'd give a team that's about to make the call, with sources for every number that isn't mine. For context, I've been writing software for about 19 years, and these days I build AI agents and voice agents for businesses at NexAI Advisors.

How locked in are you once you build on one model?

At the API level, less than people fear. Moving a chat request from one provider to another can be a config change. The lock-in hides in how the model behaves.

Here's my own example. In August my OpenRouter credits ran out, and I pointed my knowledge system from an OpenAI model to Gemini. Three environment variables, no code. One of my three jobs then failed on every run. It sent a strict JSON schema with bounds like minItems and pattern, which OpenAI accepted and Gemini refused. Each job sent a different shape of request, and I had only tested one of the three. When I wrote about it at the time, the line I landed on was this: a gateway abstracts your bill. It does not abstract your integration.

What turns a scary migration into a boring one is an eval suite. That's a fixed set of test cases, built from real or synthetic data, scored the same way every time. When you want to try a new model, you run the suite and compare scores. If they match, you switch. If they don't, you know exactly which cases to look at.

Size the suite to the cost of a wrong answer. Write down the five mistakes that would hurt you most, and make sure each one has a test. Include every distinct shape of request you send, which is the part I skipped.

You'll need the suite even if you never change vendors. Providers retire models on a schedule, and requests to a retired model fail. Anthropic and OpenAI both publish deprecation pages. On October 14, 2026, OpenAI told developers that certain GPT-4 series models will be removed from its API on January 14, 2027. Every forced upgrade is a small migration. With evals in place, it's a routine afternoon.

Where does the real stickiness live?

Mostly in the workspace around the model. That means the connectors to your email and files, the company knowledge you've loaded, the agents people have built, and the habits of everyone who uses the tool daily. None of that moves with an API key.

The data backs this up. Menlo Ventures surveyed builders in mid-2025 and found that only 11% had switched model providers in the past year. 66% had upgraded to a newer model from the provider they already used, and 23% made no change at all. Switching is technically easy and rarely done.

Before you roll a workspace tool out to the whole company, ask the vendor how you'd export what people build in it: conversation history, custom instructions, agents and knowledge bases. If the answer is that you can't, put that in the business case, along with the time it would take to retrain everyone on something else.

As model prices fall, who keeps the savings?

The price of a given level of capability has dropped fast. In 2024, Andreessen Horowitz estimated that an LLM of equivalent performance was getting about 10x cheaper every year, while noting that its method was far from perfect.

You'd expect AI bills to shrink as a result. They mostly haven't. Menlo found that builders don't stay on older models to bank the savings. They move to the newest, most capable one. Enterprise LLM spend in their estimate went from $3.5 billion in November 2024 to $8.4 billion by mid-2025.

So the savings get spent on more capability. My view is that the savings go to whoever can switch. A provider keeps its margin while it has the best model for your job. A software company keeps it where it owns the workflow and the customer. A customer captures it when moving is cheap. If you want more of the value, make yourself the party that can move, which brings you back to evals and a tested fallback.

How do software companies charge for AI features?

Three patterns show up in public pricing.

Per seat, where the AI comes bundled into the license or sold as a per-user add-on. Intercom's Copilot, for example, is $29 per agent per month billed annually.

Seat plus usage. Intercom charges per teammate and then $0.99 per outcome for its Fin AI agent. An outcome counts when a customer confirms the issue is resolved, doesn't ask for more help, or Fin completes a workflow.

Credits, where customers prepay for units of work and each unit is priced to cover the model cost plus a margin.

Whichever you pick, know what one request costs you to serve. Most companies can't tell you that number, and it's the one that decides whether your heaviest users are your best customers or your biggest losses.

Why does approving an AI model take months?

The approval path depends on what data the model will touch, so start there.

In regulated industries, a common gate is the BAA, or business associate agreement. It's the HIPAA contract, and federal rules require one when a vendor creates, receives, maintains or transmits protected health information on behalf of a covered entity like a health plan or a provider (45 CFR 160.103). Many companies go further and make a signed BAA company policy for every AI vendor before anyone can use the model, whatever the data. Negotiating one takes time, which is a big reason approvals run for months, and a big reason companies stay with a vendor once it's approved. Where no BAA is involved, expect a data processing agreement, a security review, and a negotiation over how long the vendor keeps your prompts and outputs.

Two details catch teams out.

First, coverage is per product and per feature. The large providers offer BAAs for some enterprise and API offerings, exclude their consumer plans, and in some cases exclude specific features. "We have a BAA with them" doesn't mean every button in their app is covered. Ask for the eligible-services list and check your use case against it.

Second, the fastest route is often a contract you already have. Claude is sold through AWS, Google Cloud and Microsoft Foundry, Azure's model catalog. OpenAI models are sold through Microsoft Azure, and Gemini through Google Cloud. If your company already has a cloud agreement with security sign-off, adding a model inside it can be a much shorter review than onboarding a new vendor.

Whoever starts the request, the review moves faster when compliance gets three answers on day one: what data goes in, where it's stored and for how long, and what happens to it if you turn the tool off.

Do end customers care which model is underneath?

Most of them care whether the answer is right and the product is easy to use. That's why more products route between models behind the scenes.

Regulated buyers are the exception. Security questionnaires and data processing agreements usually ask you to list your subprocessors, meaning the outside companies that handle your customers' data, and your model provider is one of them. Hide the model in the interface if you like, but expect enterprise customers to find it in the paperwork, and to ask what happens if you change it.

How many model providers should you run?

My recommendation is one primary and one tested fallback, chosen per job. The best model for code may not be the best one for long documents, or for cheap high-volume sorting.

Signing up five providers because you might need them someday mostly buys you five security reviews. Running one with no fallback turns an outage, a price change or a retirement notice into an emergency. Two, with the same evals run against both, gets you most of the protection for a fraction of the paperwork.

Will today's leader still lead in three years?

Nobody knows, and the record says the lead moves quickly. Menlo estimated that OpenAI's share of enterprise LLM usage fell from 50% in 2023 to 25% by mid-2025, while Anthropic and Google gained.

Availability can also change overnight for reasons unrelated to quality. In June 2026, the US government issued an export control directive barring any foreign national, inside or outside the United States, from using Anthropic's Fable 5 and Mythos 5. Anthropic had no reliable way to sort its users by nationality, so it disabled both models for every customer that evening. Its other models stayed up. Teams built around Fable 5 lost it with no warning. A team with a tested fallback could switch that same night.

It isn't only about one vendor. On February 4, 2025, Australia ordered all DeepSeek products, applications and web services removed from government systems and devices on security grounds, effective immediately.

Whoever wins, what protects you is being able to leave. Keep a tested fallback. Own your evals. Keep your prompts and company knowledge in a form you could take with you. Then re-run the comparison every quarter, so the next switch is a decision you make on purpose.

The short version

Find out what data the model touches before anything else. Use the cloud contract you already have where you can. Build evals before you need them, and run them against a second provider. Ask how you'd export what your people build. Know your cost per request.

What's the one thing that would make it hardest for your team to change AI tools tomorrow?

More notesnewest first

Working on something like this?

Bring the app or the process to a free 15-minute call. I will tell you what I would look at first, and whether I am the right person for it.