Metadata and dynamic filters
Store facts about each chunk next to its vector, and filter on them before or during search.
Words on this page (7)
- chunk
- : A small piece of a document, sized for indexing and retrieval.
- vector
- : An ordered list of numbers. Embeddings and model activations are vectors.
- metadata
- : Extra labels stored with a document or chunk, such as its title, date or country, which let you filter results.
- policy
- : A rule for choosing actions. In a language model, the model itself is the policy.
- structured output
- : Model output that follows a schema, such as valid JSON with fixed fields.
- prompt
- : The input text you give a model, including instructions, examples and data.
- JSON
- : A text format for structured data: keys and values in braces, like {"name": "Ada"}.
The idea
Every carries : source document, author, date, product, language and who is allowed to see it. A query like "what changed in version 3?" should only search chunks tagged version 3. Filtering before the search narrows the candidates and removes whole classes of wrong answers.
Dynamic filters
Let the model extract filters from the question. "Show me refund for EU customers last year" becomes a semantic query "refund policy" plus filters region = EU and date within the last year. The model returns the filter as that your code validates and passes to the database. Never let the model write raw database queries without validation.
Why it matters for safety
Permissions belong in metadata filters applied by your code, not in the . If a user may not read a document, its chunks must never reach the prompt. A line like "do not reveal confidential data" is not access control.
A worked example: extracting filters
Question: "What was the warranty on the 2023 Model X in Spain?" The system asks a model to return a small : {query: "warranty terms", year: 2023, product: "Model X", country: "ES"}. Your code checks the values against allowed lists, applies them as filters, and then runs the semantic search on the query text.
How a filtered search runs
A filtered query has two parts: the meaning ("warranty terms") and the conditions (year = 2023, country = ES). The database can apply the conditions before the vector search, so only matching chunks compete. It can also apply them after, trimming the nearest results. Filtering first is safer because the shortlist cannot be emptied by the filter. Check how your database does it.
A second example: a filter that hurts
A user asks "What is the warranty on Model X?" and says nothing about a year. If the model guesses year = 2024, every chunk from other years is excluded, including the one with the answer. Filters should come only from what the user actually said or what your app knows, such as their country on file. When a value is missing, leave that filter out or search broadly and show the year in the answer.
What goes wrong
Messy metadata causes silent misses. One document is tagged "US", another "USA", a third "United States". A filter for "US" skips the other two. A document with no date tag fails every date filter. The symptom is "I cannot find that" for a document you know exists. Confirm by repeating the search without the filter. If it appears, the tags are the problem.
Also watch for filters that are too strict. Three filters at once may leave no chunk to match.
Try it yourself
Tag 20 short chunks with a country and a year, using a few spellings on purpose. Run one query with no filter and one with country = Canada. Count the results. Expect the filtered search to remove the wrong-country chunks, and expect it to also drop any chunk tagged "CA" or "Canada " (with a stray space). Fix the tags to one allowed list and rerun.
Common mistake
Relying on "do not reveal confidential data" in the prompt. A prompt is not a lock.
A user asks "What is our refund policy?" The index holds policies for ten countries and five years. Without filters, retrieval may return Germany's 2019 policy for a customer in Canada. With metadata, the search first narrows to country = Canada and the latest year, then looks for the best passage.
Check yourself
Where should access permissions be enforced in RAG?
Not quite. Chunks the user cannot see must never reach the prompt at all.
What can a model extract from a question for filtering?
Not quite. Validate the structure before using it.
A Canadian customer gets a policy from another country. What is the likely cause?
Not quite. Metadata filters narrow the candidates to the right region before searching.
You got 0 of 3 right.
Go deeper
- ArticleMetadata filteringArticle
Pinecone · Filter syntax and performance notes.
- DocsQdrant filteringDocs
Qdrant · Payload filters combined with vector search.
- ArticleContextual RetrievalArticle
Anthropic · Related technique for giving chunks context.
Spotted a mistake or something unclear? Report it on GitHub.