Open-weight models
Some labs publish the trained weights, so anyone can download and run the model.
Words on this page (13)
- weights
- : The learned numbers that define a trained model. A model whose weights are published is called open-weight.
- open source
- : Software whose code is public and can be reused under a license.
- training
- : Adjusting a model's parameters on data so its predictions improve.
- fine-tuning
- : Further training on a smaller, targeted set of examples to change behavior or style.
- Hugging Face
- : A hub for sharing open models, datasets and demos, and the maker of widely used libraries.
- tool
- : A function the model can ask your program to run, described by a name, description and argument schema.
- API
- : Application programming interface: a way for one program to call another, usually over HTTP.
- token
- : A piece of text the model reads and writes, usually a word or part of a word. About 4 characters of English on average.
- GPU
- : Graphics processing unit. A chip with thousands of cores, ideal for the matrix math in AI.
- parameter
- : One learned number inside a model. Modern models have billions of them. Training adjusts them.
- quantization
- : Storing weights with fewer bits (for example 4 instead of 16) to save memory at a small quality cost.
- precision
- : Of the cases a system flagged, the share that were truly positive.
- prompt
- : The input text you give a model, including instructions, examples and data.
What it means
means the numbers that define the trained model are available. It is not always : licenses vary, and the data and code are often not released. You can run an open-weight model on your own machine or server, it and inspect it. Llama (Meta), Mistral, Qwen (Alibaba) and DeepSeek are well-known families. hosts close to a million models and the to use them.
Trade-offs
Open weights give privacy, control, predictable cost at scale and the ability to customize. You take on serving, updates and safety. Frontier closed models are usually somewhat ahead on the hardest tasks, and the gap has varied over time. Smaller open models run on a laptop and are good enough for many tasks like classification and extraction.
Open weights versus open source
"Open weights" means you can download the trained numbers. "Open source" in the strict sense would also include the training data and code. Most releases are the first kind. Always read the license: some forbid commercial use or have size limits.
A worked example: choosing
You need to classify 5 million support tickets a month. A closed at $0.50 per million with 300 tokens per ticket costs about 1.5 billion tokens × $0.50/million = $750 a month. A small open model on one rented may cost around $400 a month and keep data in-house, but you must run it and keep it updated. If volume is small, the API is easier. Here the saving is only $350 a month, which an hour or two of engineer time a week could use up, so hosting pays off only when volume is much larger or privacy matters.
What you actually download
An open-weight release is a large file of numbers plus a program that runs them. The file size is roughly the number of times the bytes stored for each one. Invented numbers: a model with 7 billion parameters stored at 2 bytes each is 14 GB. (storing each number with less ) at half a byte each cuts that to 3.5 GB, at some cost in quality. The model also needs extra memory while it works, so a file that fits on your disk may not fit in memory.
A second example: when the API wins
A team builds a feature that gets 200 requests a day with no privacy rule. Suppose each request uses 1,000 tokens. That is 200 × 1,000 × 30 = 6 million tokens a month. At the invented price of $0.50 per million tokens, the API costs about $3. A rented GPU is billed by the hour whether it is busy or idle, so it would cost far more. At low volume the API is the clear choice, and you can use a stronger model than your own hardware could run.
What goes wrong
The symptom is a model that runs but crawls, because it did not fit in fast memory and spilled to slower storage. Another is answers that look worse than the online demo. Common causes are heavy quantization or the wrong chat format for that model. To confirm, run the same 20 on the local model and on a hosted one and compare the results side by side. A license surprise is a third failure, so read the terms before you build.
Try it yourself: run one locally
Install Ollama and download a small model. Ask it five questions you have already asked a hosted assistant, including one hard one. Note how fast it answers and whether your computer slows down. Expect simple questions to work well and the hard one to be weaker or slower. That is the real shape of the trade: control and privacy on one side, raw capability on the other.
Common mistake
Assuming open means free. You pay in hardware, time and maintenance.
A hospital cannot send patient notes to an outside service. It downloads an open-weight model and runs it on its own servers. The notes never leave the building. The model is a bit weaker than the best closed one, but for summarizing notes it is good enough, and the privacy problem disappears.
Check yourself
What do open-weight models usually let you do?
Not quite. The weights are published. Licenses and data availability vary.
When is an open model a good choice?
Not quite. Running it yourself keeps data in-house, at the cost of operating it.
You classify 5 million tickets a month and have no privacy concern. What should you compare first?
Not quite. At volume, total cost including your own time decides, not size or fame.
You got 0 of 3 right.
Go deeper
- ToolHugging Face HubTool
Hugging Face · The main place to find, compare and download open models.
- Repo
Ollama · Run open models locally with one command.
- Repo
ggml · Efficient local inference in C and C++.
- Article
Mark Zuckerberg · Meta's argument for open weights.
- ToolGemmaTool
Google DeepMind · Google's lightweight open models.
- CourseBuilding with Llama 4Course
DeepLearning.AI and Meta · Short course on using Llama models.
Spotted a mistake or something unclear? Report it on GitHub.