The Internal Q&A Agent Is Worth Building Only If It Refuses to Answer
A knowledge agent's value comes from grounding and refusal, not fluency. Building one mostly reveals that your documented knowledge has holes filled by two or three long-serving people.
Stephen Saracco, MRICSAsset PerformanceWhere it pays
Lease and contract term lookups. Policy and standard-operating-procedure answers across a distributed hospitality group. Tender and RFP answer libraries. Equipment and specification questions for facility teams. Onboarding questions from new staff.
The non-negotiable mechanic: every answer cites its source document and section, and the agent is instructed to say "not in the corpus" and route to a human rather than infer. Refusal rate is a metric to monitor, not a defect to engineer away.
Do not over-build the infrastructure
Below a few hundred documents, well-structured folders of plain text beat a vector database on both cost and accuracy, and stay debuggable by a non-technical owner. Do not buy retrieval infrastructure for two hundred PDFs.
Corpus hygiene is the actual project. Three versions of the same policy in three folders produce confidently contradictory answers. One owner per folder, a review date on every document, and a deletion policy.
The failure nobody sees coming
No corpus owner. The agent is accurate at launch and quietly wrong by month four, with no signal that anything changed. Document decay is invisible in a chat interface.
Measure two things: questions answered with a citation, and questions escalated. A rising escalation rate is telling you exactly where your documentation is missing.
Pick one recurring internal question. Gather every document that claims to answer it into a single folder and count how many contradict each other.
Over 20 years of international real estate and facilities strategy across the U.S. and EMEA.
Meet the team →Questions we get asked
What is the real benefit of an internal knowledge agent?
Usually not speed. It is the gap analysis — you discover which policies were never written down and which operations depend on one person's memory. That is a succession-risk finding delivered by a technology project.
Do I need a vector database for company documents?
Below a few hundred documents, no. Well-organised folders of plain text outperform retrieval infrastructure on cost and accuracy and remain debuggable by someone non-technical.
How do I stop an AI assistant inventing answers?
Ground every answer in a cited source document and section, and instruct the agent to escalate rather than infer when the answer is not in the corpus. Treat a high refusal rate in week one as correct configuration.
Related insight
The AI Assistant That Does Everything Does Nothing
An agent's reliability collapses as you add systems it must be simultaneously correct about. The only agents that reach production…
AutomationYour Software Is Fine. The Gaps Between It Cost You Forty Hours a Week.
The recoverable waste in most operating companies sits in the transfers between systems, not inside any one of them. No vendor is …
AutomationAutomate a Broken Process and You Buy a Faster Broken Process
Automation makes a process permanent. Any step you encode without questioning becomes structurally harder to remove than it was th…
Where is your business still running manually?
We review your processes, identify the highest-impact automation opportunities and estimate their financial impact — before anything is built.