Model providers and API styles
Several labs and clouds sell access to models. Learn the shapes so you can switch.
Words on this page (11)
- retrieval
- : Finding the most relevant pieces of text for a query.
- API
- : Application programming interface: a way for one program to call another, usually over HTTP.
- tool
- : A function the model can ask your program to run, described by a name, description and argument schema.
- structured output
- : Model output that follows a schema, such as valid JSON with fixed fields.
- batch
- : A group of examples processed together in one step.
- prompt
- : The input text you give a model, including instructions, examples and data.
- eval
- : A repeatable test of AI output quality, run before and after every change.
- tool use
- : The model asks your program to run a function and reads the result.
- system prompt
- : Standing instructions placed at the start of every call that set the model's role and rules.
- JSON
- : A text format for structured data: keys and values in braces, like {"name": "Ada"}.
- token
- : A piece of text the model reads and writes, usually a word or part of a word. About 4 characters of English on average.
Labs
Anthropic (Claude), OpenAI (GPT and o-series reasoning models), Google (Gemini, and the open Gemma), Meta (Llama, and now the Muse models), Mistral, Cohere (enterprise and focus), DeepSeek, Alibaba (Qwen), xAI (Grok) and others. Each releases families in sizes from small and cheap to large and strong. See the model atlas for current names.
Where to call them
Direct from the lab's , through a cloud such as AWS Bedrock, Google Vertex AI or Azure, or through a router such as OpenRouter that offers one API for many models. Clouds add enterprise controls and billing. Routers add flexibility and fallback.
API shapes
Most offer chat-style message APIs, calling, , streaming, vision input and discounts. Many accept the OpenAI-compatible format, so one client library works across them with a different base URL. Local runners such as Ollama and LM Studio also expose that format.
Staying portable
Keep model names and in config, wrap calls in one function, and run your set on any candidate. Provider-specific features such as caching and extended thinking differ, so isolate them in one place.
A worked example: one function
callModel(prompt, options) is the only place in the code that talks to a provider. It reads the model name from config, logs usage and handles retries. Switching providers means changing config and running the test set, not hunting through the code.
Where to call from
Direct from the lab for the newest features. Through a cloud provider when your company already uses it and needs its controls. Through a router when you want to try many models quickly. Prices, limits and features change often, so check the vendor pages.
What actually differs between providers
Many things look the same: messages in, a message out. The differences are in the details. The way are written, how the is passed, the names of usage fields, the error codes, the rate limits and the rules for caching all vary. An OpenAI-compatible endpoint covers the common core. Features beyond it, such as caching or extended thinking, usually need provider-specific code.
A second example: a switch that breaks
You change the base URL and model name, and your app runs. But your prompt asked for and the new model sometimes adds a sentence before it, so your parser fails. Invented numbers for illustration: your test set passed 94 of 100 before and passes 81 of 100 now. Without that test, you would find out from angry users. With it, you adjust the prompt, rerun and decide whether the new model is worth it.
Trade-offs of each route
Calling a lab directly gives the newest features and the fewest middlemen. A cloud adds company controls, but a model may arrive there later and some features may differ. A router makes trying models easy, but it puts another company between you and the model. That is one more point of failure and one more place your data passes through.
What goes wrong
A model name in your code is retired, and calls start to fail with a not-found error. Keep names in config and read the provider's deprecation notices.
Costs differ for the same text, because each provider splits text into its own way. Compare the logged token counts, not your guess.
A feature you relied on does not exist on the second provider. Isolate such features in one place.
Try it yourself
Take the 20-line script from the calling lesson. Change only the base URL, the key and the model name, and point it at a second provider that accepts the same format. Send five prompts to both. Expect the answers to differ in wording and the token counts to differ for identical input.
Common mistake
Using one provider's special features everywhere. Isolate them so they are easy to swap.
A small startup builds on one model provider. When that provider has a bad day, so does the startup. Another startup wrapped model calls in one function and keeps a second provider ready. A test run with the other provider takes ten minutes. Portability is cheap insurance.
Check yourself
Why wrap model calls in one function?
Not quite. Portability is a single point of change.
What does an OpenAI-compatible endpoint let you do?
Not quite. The request format is shared.
Your only provider has an outage. What would have made recovery easy?
Not quite. Keeping a tested fallback behind one function turns an outage into a config change.
You got 0 of 3 right.
Go deeper
- ToolOpenRouterTool
OpenRouter · One API and model comparison across providers.
- DocsAmazon BedrockDocs
AWS · Managed access to several providers' models.
- DocsVertex AIDocs
Google Cloud · Google's platform for Gemini and others.
- Repo
BerriAI · A library that calls many providers through one interface.
- DocsGemini APIDocs
Google · Models, quickstarts and pricing for Gemini.
- Docs
Anthropic · The reference for Anthropic's chat API.
- ToolMistral AITool
Mistral · Open and hosted models from a European lab.
Spotted a mistake or something unclear? Report it on GitHub.