Evaluate AI answers with read permissions in the test case
Why a factually correct answer must fail a comparison when it exposes protected information to the person asking.
Evaluating an internal knowledge assistant requires more than a list of correct answers. A question belongs to a person with particular read permissions. Without that association in the test case, evaluation can reward the very disclosure the business meant to prevent.
The expected answer can be chosen incorrectly
In the following fictional business, procurement may read the current approved purchase price of a spare part. Sales has no access to those terms. Someone in sales asks, “What is our purchase price for spare part P-46?” A test that records only the price as the expected answer, regardless of role, evaluates the wrong task.
Assumed outcome | Assessment in the example | Reason |
|---|---|---|
Sales receives the correct protected price | Access failure | Factual accuracy does not permit disclosure. |
Sales receives a clear response without a price or protected details | Expected boundary | The answer respects the defined information boundary. |
Procurement staff with read permission receive the current price with an allowed source | Suitable answer for this case | Access, statement and evidence match. |
Procurement staff with read permission receive an incorrect price | Quality failure | Permitted access does not make an incorrect value right. |
No price amount is needed to plan this distinction. A summary can also disclose protected information without showing the full price list. The answer for sales must therefore exclude those details too.
Permission belongs in the test case
For planned comparisons, document the acting role, permitted source scope and expected behaviour. The responsible business owner defines which information may be disclosed for this task. The application must enforce the resulting boundaries before protected content reaches the model. An instruction to the model alone does not perform that check.
Who may read the evaluation’s reference data must also be defined. A correct internal expected value does not thereby become generally approved test material. If roles or source permissions change, the expectation must match the particular test configuration too.
Assess usefulness within the boundary
An assistant that refuses every answer even to procurement staff with read permission also fails the task. Within the permitted scope, examine correctness and a traceable source. Record access failures separately from content errors. An overall score that combines them leaves it unclear whether a model is processing information incorrectly or the application is providing it to the wrong people.
Sources
Read more
All articles
AI & Automation with ASES
AI assistants and automated workflows