How Keel does RAG: hybrid search, local embeddings and answers with sources

Keel turns each document into overlapping chunks, embeds them where Keel runs, stores them in Postgres with pgvector and full-text search, and answers from a hybrid search merged by reciprocal rank fusion. The agent is told to search first, answer only from the results, cite each claim, and say it cannot find the answer when nothing matches.

From a document to a cited answer

1. Parse. Text is extracted from PDF, Word, HTML, markdown and plain text. Scans go through text recognition first.
2. Chunk. The text is split recursively into passages of about 800 characters with 120 characters of overlap, set per knowledge base.
3. Embed. Each chunk is turned into a vector by a model that runs next to Keel, not at a third party.
4. Index. The vector goes into pgvector and the text goes into a full-text index, in the same Postgres database.
5. Search and merge. A question runs both searches at once and the two ranked lists are merged into one.

Everything after that is the agent reading the merged passages and writing from them, which is the easy part once the passages are right.

Why 800 characters, and why the overlap

Small enough to be specific, large enough to keep a clause together with its heading. The overlap exists because answers rarely respect a boundary: a sentence split across two chunks would otherwise be half missing from both.

Both numbers belong to the knowledge base, not the code. A contract set and a support ticket archive have different shapes, so they can use different sizes without anyone editing the pipeline.

Hybrid search, merged by rank

Two searches run on every question. One compares meaning through the vectors, which finds a paraphrase like contractors leaving instead of notice period. The other compares words through a full-text index, which finds an exact product code, an invoice number or a surname that embeddings blur.

The lists are merged with reciprocal rank fusion, which scores a passage by where it appears in each list rather than by the raw scores, because a cosine distance and a text rank are not comparable numbers. The practical effect: a passage that is second in both lists beats a passage that is first in one and missing from the other, which is usually the right answer.

The rules the agent runs under

• When the question is about the organisation, search the knowledge base first.
• Answer only from the passages that came back, not from what the model remembers.
• Cite each claim inline with the passage number, like [1] or [2], and list the sources at the end.
• If the passages do not contain the answer, say you could not find it. Do not fill the gap.
• Search only the knowledge bases the person is allowed to use.

These are instructions in the agent plus scoping in the query, and every search is a tool call. It is recorded like any other action: who asked, what was searched, which passages came back and what the agent did with them. When an answer is questioned a month later, the evidence is still there.

Why the embedding model runs next to the database

Embeddings are computed on the machine that runs Keel, with a small model through FastEmbed. Documents are never sent to a third party just to be indexed. That matters for data residency, for regulated customers and for installs with no internet at all. It also means reindexing a large archive does not create a per document bill.

Getting documents in, and keeping them current

Documents arrive by upload, from a connected drive or wiki, or by pointing Keel at a website. Website import reads the sitemap or a list of pages, so a company's public help pages can become a knowledge base in minutes rather than a migration project.

Every document carries a status: ready with a chunk count, or failed with the reason. Reindexing replaces a document's chunks in one transaction, so a half processed file never serves answers, and a deleted document stops appearing immediately.

What this looks like to a customer

A help desk that answers with citations. The same pipeline behind a support inbox: cited answers, AI triage on the rest.

The document assistant product page

FAQ

Infrenic builds AI agents on your data and tools, ships them to production and runs them, all on Keel. Send us a problem and see it working in 48 hours.