Local AI: the model is one part of the system

Make the evidence path inspectable

Retrieval-augmented generation brings retrieved material into a model's answering process. The original RAG research combined a language model with a searchable external memory and evaluated that combination on knowledge-intensive tasks. It provides a foundation for the approach, not a guarantee that any folder of documents will produce reliable answers.

For a document assistant, retain the question, retrieved passage identifiers, document versions and final answer in an appropriate test record. That makes it possible to distinguish a search failure from an answer that misused good evidence. A fluent response alone cannot make that distinction.

Keep the minimum material needed for evaluation. A debugging log should not become an uncontrolled second copy of confidential documents.

Sources: [1]

Test search before judging the answer

Use a small synthetic document set with answers you can check directly. For example, create two fictional maintenance manuals for the same imaginary device. Version A says an inspection occurs every 40 operating hours; version B changes it to 30. Mark version B as current and version A as superseded.

Ask for the current interval, the old interval, and the reason for the change when no reason is supplied. The expected behaviour is different for each question: retrieve B, retrieve A, and acknowledge missing information. Repeating the number from the most similar-looking passage is insufficient.

Inspect the retrieved text before changing the model. If the current document never reaches it, first investigate parsing, metadata filters, document versioning and retrieval ranking. Add a larger model only when the evidence points to a limitation it could plausibly address.

Use cases that can disprove the design

The following is a proposed acceptance set. It deliberately includes questions that should not receive a complete factual answer.

Keep ordinary and difficult cases separate when reporting results. A single overall score can conceal a system that performs well on easy lookups but fails whenever a document is missing. Record the dataset and configuration behind each evaluation so comparisons mean something.

Keep tool authority outside retrieved text

OWASP identifies indirect prompt injection as instructions arriving through external content such as documents. A retrieved page can contain text that tries to redirect an assistant. Searching a trusted folder does not turn every sentence inside it into an authorised command.

A useful boundary is to keep document reading separate from action permissions. Retrieved text may support an answer; it should not grant the ability to email files, change accounts or execute commands. The application must enforce the allowed actions and destinations independently of the model's prose.

For the fictional manual test, include a passage telling the assistant to transmit the manual elsewhere. The expected result is an answer grounded in relevant maintenance content, with no transmission. Evaluate the action boundary as well as the wording.

Sources: [2]

Define what local actually covers

Draw the data path for inference, document parsing, embeddings, search, logs and tools. Identify any network dependency explicitly. A model running locally does not prove that every surrounding component stays offline.

Decide which records persist, who can read them and how they can be removed. Then test the system against that decision. The useful outcome is an assistant whose evidence and permitted actions are understandable enough to inspect when something goes wrong.