← All notes
LLMsRAGevals

When company guidelines pull against grant rules

2025 · Mark Temchenko

At Twin AI the grant pipeline's writing stage pulled context from two places. One was RAG over the client's own documents: Salesforce and Bynder for one client, drives and APIs for the others. The other was a separate module that injected the EU and scheme-specific rules for the grant being written.

What went wrong

Sometimes a company's own guidelines pulled against a specific grant requirement. The model never said so. It leaned one way, quietly, and wrote a fluent section. Either the draft stopped sounding like the company, or it stopped fitting the grant.

That was worse than a plain error. Even when a draft was right, nobody could tell which side it had taken or why it read slightly off, so reviewers second-guessed good drafts as well as bad ones.

The fix: judge outside the model

We added an automated assessment after drafting. Its criteria came from both sides: the grant's rules and the company's internal metrics. It scored each section from 1 to 10, named the weak spots, and flagged the sections where the two sides pulled apart.

The flag went to the user, not back into the prompt. They made the call: accept the draft, or adjust the source documents. A draft that felt wrong for unclear reasons became a clear decision with a reason attached.

What I took from it

More context is not automatically better context. When two sources can disagree, the disagreement has to be detected and shown to a person, because the model will resolve it on its own, silently and fluently.