Build around company knowledge when it improves a valuable, recurring decision and you can maintain the information and the rules for using it. Test that advantage against an existing tool or ordinary search before building a product. Proprietary documents alone are not a business case. The case depends on a useful result, clear access rights, an accountable owner and sustainable operating costs.
Key takeaways
- Choose a recurring decision where company context changes the useful answer.
- Compare against an affordable baseline using cases the demonstration did not include.
- Budget for maintaining sources, permissions and evaluation after launch.
Ivan Nosov, Co-founder & COO, ACW Soft Solutions. Prepared as an editorial Q&A draft for founder review, not as a transcript of an interview.
Which customer decision depends on information a public model cannot supply?
Look for a decision whose correct answer depends on current, authorized company information. A public model may explain how a service contract usually works. It cannot establish a particular customer's active entitlement without access to the relevant agreement and business records. The opportunity is to make that specific decision easier to reach and verify.
Consider a hypothetical specialist distributor answering whether a replacement component is covered by an account's service agreement. The answer may depend on the equipment record, contract revision and an approved exception. The product's value comes from assembling the applicable evidence and showing what remains uncertain. A conversational interface is useful only if it improves that job.
Retrieval-augmented generation, or RAG, supplies retrieved information as context for a model's response. Microsoft describes grounding answers in proprietary content while highlighting retrieval and access-control challenges. That explains a technical option, not a commercial advantage. First demonstrate that access to the right company context changes an outcome the customer values.
Sources[1]
Who owns the answer when experts or documents disagree?
Assign an accountable business owner for the decision and source owners for the information supporting it. A system cannot turn unresolved disagreement into an authoritative policy simply by selecting the most plausible paragraph. Define which records take precedence, who can approve an exception and when the assistant must refer the case to a person.
In the distributor example, a sales note might promise coverage that the signed agreement does not include. The product should expose that conflict to the authorized owner. It should not silently treat whichever document was uploaded most recently as the final rule. Record the resolution so the next similar case has an approved basis.
Also establish who is entitled to see each piece of evidence. OWASP's guidance on vector and embedding weaknesses describes risks from mismatched access controls and conflicting information across sources. A technically retrievable document is not automatically appropriate for every account. If owners cannot agree on the decision rule or the audience, narrow the use case before building around that knowledge.
Sources[2]
What is the cheapest useful test before building?
Test the decision with a small, deliberately varied set of cases before investing in a broad integration. Include common requests, incomplete information, changed agreements and cases where the correct result is escalation. Have an authorized reviewer establish the expected outcome and its supporting evidence before examining the AI answer.
Compare the proposed approach with how the work is done today and with a simpler alternative, such as better search. Use an approved environment and synthetic or appropriately authorized material. Keep some cases out of the prompt-development process so the team can see whether the approach works beyond the examples used to tune it.
Judge the result on task completion, decision accuracy, evidence quality, review time and consequences of error. A response can quote the correct document while applying it to the wrong account. A cautious refusal can also waste time on a question that the source answers clearly. The test should reveal both. It provides evidence for the next investment decision, not proof that the product is ready for unrestricted use.
When should the company buy an existing tool instead?
Buy or configure an existing tool when the business need is broadly shared and the available product meets the important workflow, access and operating requirements. Building becomes more plausible when a distinctive decision process cannot be supported adequately through configuration or integration, and the company is willing to own that gap over time.
Evaluate both options with the same cases. Check the user experience, source permissions, update behavior, evaluation options, export paths and support responsibilities. A compelling demonstration does not show how an account loses access or how a corrected agreement replaces an old one. Include those events in the comparison.
A hybrid approach may fit: use a managed retrieval service while building the application-specific workflow and approval logic. Keep the boundary explicit so the company knows what its provider operates and what its own team must maintain. Choose custom development for a demonstrated requirement or advantage, not because owning every component sounds more strategic.
Which cost is easiest to underestimate?
The recurring work required to keep an answer useful is easy to miss in a prototype budget. Documents change, customers gain and lose access, source systems move, and reviewers discover exceptions. Someone must decide what to update, verify the change and investigate answers that no longer match the business.
Use a simple capacity calculation before a revenue forecast. In a hypothetical workflow, 120 cases a month at 15 minutes each require 30 hours. If an assisted process needs six minutes of review per case, that is 12 hours. Add eight hours for source maintenance and exception handling, and the monthly workload becomes 20 hours. The net capacity released is ten hours, before counting implementation effort and software costs.
These are illustrative assumptions, not ACW results or a vendor benchmark. Replace them with observations from the pilot. Time released becomes financial value only if it reduces a real cost, enables additional work or improves an outcome worth paying for. Test the downside too: if review time doubles or content changes more often, the expected advantage may disappear.
What evidence should make a founder stop after a promising demonstration?
Stop or narrow the initiative when the product does not improve the selected decision enough to justify its operating commitment. A successful demo shows that an answer can be produced under chosen conditions. It does not establish that customers will use the result, that the underlying sources stay current or that the team can handle exceptions.
Define the conditions in advance. Examples include unresolved access rights, persistent disagreement among source owners, review effort that removes the capacity benefit, or failures on a critical case the team cannot contain. An existing search tool delivering a comparable result at a lower total burden is also a reason to reconsider a custom build.
Preserve useful learning even if the product stops. Cleaning up source ownership, clarifying an approval rule or improving ordinary search may still help the business. Revisit the AI product when a specific condition changes: a better-maintained source, a stronger recurring demand or a capability that closes the measured gap. Set that condition clearly so a new model announcement does not restart the project without a business reason.
What should a founder do next?
- Name the recurring decision, its user and the observable benefit of a better answer.
- Assign source owners and resolve the authority and access rules.
- Compare an existing tool, a configured service and a custom approach on the same unseen cases.
- Estimate maintenance and review effort, then set conditions for continuing, narrowing or stopping.
Reader questions
Does RAG make answers reliable automatically?
No. Retrieval can supply relevant context, but the system can retrieve the wrong source, miss a permission boundary or misapply correct information. Evaluate the decision and its evidence, not just the presence of a citation.
Is a private document collection a defensible product advantage?
Only when the company can use it legitimately to deliver something customers value and can keep that advantage useful. A collection that is outdated, poorly governed or easy to replace may create maintenance work without a durable benefit.
Sources and further reading
- Retrieval-augmented generation in Azure AI Search (opens in a new tab). Microsoft Learn · Accessed 6 Nov 2026
- LLM08:2025 Vector and Embedding Weaknesses (opens in a new tab). OWASP · Accessed 6 Nov 2026
About the author. Ivan Nosov, Co-founder & COO, ACW Soft Solutions. Ivan Nosov is the COO and co-founder of ACW Soft Solutions, where he helps companies build, launch and scale digital products. His focus is the business challenge behind a product: whether a company needs an MVP, a modernized platform, a better user experience, AI or automated processes, the right first step is sometimes defining what to build. He works with founders and CEOs, product and operations leaders, SaaS companies and technology partners. View profile
