Check whether sources are current before changing the model

A knowledge assistant describes a superseded workflow. Before procuring a larger model, it is worth asking which document version actually reached the answer. A valid instruction that was never supplied does not become available automatically through added model size.

An old instruction as the shared basis

The following example is entirely hypothetical. No models were called and no answers measured. Archived instruction D-40 calls for a repair request as a PDF by email. Current approved version D-41 requires the portal form.

Suppose the question about the current submission route receives only the old passage:

The answers would agree with the old text while being wrong for the current task. A source link would not repair the incorrect validity status.

A model might recognise the archive marker and refuse to answer. That would still not supply the missing new instruction. The table shows possible errors, rather than proving that all models react alike.

Trace validity through the source chain

Check who approves the valid version and how replaced versions are marked. The newest file date alone does not determine which instruction applies. Its scope must also fit the task.

A change in the document system does not necessarily update all search passages, stored answers or conversation content immediately. The relevant role must retain access to the valid version; old copies need defined handling.

The same question can then be checked with current, missing and contradictory evidence. Assess source selection, faithfulness and permitted access separately. Model quality remains relevant: another model may interpret supplied information better. Checking sources makes a model comparison more meaningful instead of carrying an unresolved document fault into it.

Sources

• Microsoft — RAG prompt engineering

• Microsoft — Retrieval-augmented generation in Azure AI Search

• OWASP — RAG Security Cheat Sheet