The map of the AI tooling stack
AI tools stack in layers. Knowing the layers tells you what a new tool replaces and what it does not.
The layers of the AI stack. Click a layer
Applications
What people use: chat apps and coding agents.
When you need it: You need one when you want the result, not the plumbing.
Claude Code, Codex, Cursor, ChatGPT, your own product
Words on this page (18)
- tool
- : A function the model can ask your program to run, described by a name, description and argument schema.
- weights
- : The learned numbers that define a trained model. A model whose weights are published is called open-weight.
- inference
- : Running a trained model to produce output, as opposed to training it.
- retrieval
- : Finding the most relevant pieces of text for a query.
- embedding
- : A list of numbers that represents the meaning of a piece of text or an image, so similar things sit close together.
- vector
- : An ordered list of numbers. Embeddings and model activations are vectors.
- prompt
- : The input text you give a model, including instructions, examples and data.
- agent
- : A model run in a loop that chooses its own tools and steps until the task is done.
- MCP
- : Model Context Protocol: a standard for plugging tools and data into AI apps.
- AGENTS.md
- : A shared convention for a repo file with instructions for coding agents.
- repo
- : Repository: a project folder tracked by git, usually hosted on GitHub.
- A2A
- : Agent2Agent: a protocol for agents from different makers to find each other and exchange tasks.
- eval
- : A repeatable test of AI output quality, run before and after every change.
- observability
- : Being able to see what your system did, through logs, traces and metrics.
- API
- : Application programming interface: a way for one program to call another, usually over HTTP.
- trace
- : The full record of one run: inputs, model calls, tool calls, outputs, timing and cost.
- chunk
- : A small piece of a document, sized for indexing and retrieval.
- vector database
- : A database that stores embeddings and finds the nearest ones quickly.
The layers
Models: the trained networks you call, from labs like Anthropic, OpenAI and Google, or families you host. : the software that runs a model fast, such as vLLM for servers and llama.cpp or Ollama for local use. Data and : models, databases such as Chroma or pgvector, and search. Orchestration: libraries that manage , and loops, such as LangGraph, Pydantic AI and the OpenAI and Claude SDKs. Protocols: for tools, for instructions, for agent-to-agent calls. and : promptfoo, Langfuse, Phoenix and Inspect. Applications: chat apps and coding agents like Claude Code, Codex and Cursor.
How to use the map
When a tool shows up, find its layer. Many new releases are one layer repackaged. Learn the layer's job first, then pick the tool. For most projects you need a model, a way to call it, some retrieval or tools, and evals. Everything else is optional until a problem shows up.
A worked example: a stack for a support bot
Model: a mid-size model. Data: documents embedded and stored in pgvector. Orchestration: a short script with one tool-calling loop. Protocol: none yet. Evals and : a test set in promptfoo and traces in Langfuse. Application: a web chat page. Six layers, each with a clear job. Nothing exotic.
Reading a new tool
Which aisle is it in? Does it replace something I use, or add a layer I do not have? What problem of mine does it solve today? If you cannot answer the third question, bookmark it and move on.
How the layers connect
Follow one question through the support bot.
- A visitor types into the chat page (application).
- A short script (orchestration) receives it.
- The script turns the question into an embedding, a list of numbers that captures meaning, and finds similar document in the database (data and retrieval).
- It sends the question and those chunks to a model API.
- The model runs on someone's inference software and returns an answer.
- The script logs the full exchange (observability), and a nightly test run checks quality (evals).
Each hop has one job. When something breaks, you ask which hop it was.
A second example: a stack with fewer layers
A script reads one text file each morning and emails a three-line summary. It needs a model and a call. There is no , no framework and no protocol. Adding them would add code to maintain and nothing else. A small stack is a good sign, not a gap.
What goes wrong
One product covers two layers, so you cannot tell what it replaces. Symptom: a tool you cannot place on the map. Read its docs for the one job it does best.
You skip evals because everything seems to work. Later you change the prompt and cannot tell if answers improved. The symptom is arguments based on feelings. A small test set fixes it.
You pick a layer's tool before understanding the layer. Switching later costs more than learning first.
Try it yourself
Take a project idea and draw the six layers as boxes on paper. Mark each as "needed today", "later" or "never". Write one tool name under every "needed today" box. Expect to mark at least two or three layers as "later". Those are the tools you should not install yet.
Common mistake
Adopting tools for layers you do not need yet.
A hardware store is organized by aisle: fasteners, paint, plumbing. A new gadget does not confuse you because you know which aisle it belongs in. The AI tooling market is overwhelming until you learn the aisles: models, serving, data, orchestration, protocols, testing and applications.
Check yourself
A new tool claims to replace everything. What do you do first?
Not quite. Placing it in a layer shows what job it does and what it leaves to others.
What is the minimum stack for most AI features?
Not quite. Add other layers only when a concrete problem requires them.
Your team wants a vector database for 20,000 chunks. What should you ask?
Not quite. Match infrastructure to the size of the problem.
You got 0 of 3 right.
Go deeper
- Repo
Shubham Saboo · Runnable examples across agents, RAG and multi-agent apps.
- Repo
EthicalML · Curated list of articles, courses, papers and tools, updated monthly.
- Article
Louis-François Bouchard · A broad guide organized by learning style.
- Tool
roadmap.sh · A community roadmap with a clickable map of skills.
Spotted a mistake or something unclear? Report it on GitHub.