The usual answer to “how do we build a chatbot with our own data?” is a short recipe. Collect your documents, split them into passages, and store the passages in a searchable index. When someone asks a question, give the language model the most relevant passages and let it write the answer. The technique is called retrieval-augmented generation, or RAG. With a small set of clean documents, a demonstration can be ready in a few days. This guide covers the information and operating rules a company needs before building one.
The demonstration is built on a handful of clean documents and answers the questions its builders thought to ask. A company’s real information is different. It is spread across files, shared drives, websites, and business systems. Some of it is out of date, some is meant only for certain people, and some contradicts the rest. The people who work with it every day sort through this without thinking about it. A chatbot needs those decisions made explicit.
For many company chatbots, connecting the model is easier than preparing the information it will use. Most of the work is getting your information into a state a chatbot can use. That means deciding which source is correct for which question, connecting the systems that hold live answers, and controlling who can see what. It also means deciding when the chatbot should hand over to a person, and testing it on the questions people actually ask. If you know this before the project starts, you know what to prepare and why the work takes longer than the demonstration suggested.
Your documents were written for people who already know the context
Most company documents assume the reader already knows a lot. An experienced employee knows that the 2024 price list was replaced in January and that the policy page overrides what a sales representative wrote in an email. They know the draft procedure in the shared folder was never approved, and that the internal pricing sheet should never be quoted to a customer. Those details may be missing from the documents themselves.
A chatbot given passages without version, ownership, or access information has little basis for choosing between them. When it searches your documents, it finds the passages that look most relevant, and a passage from an outdated price list can look just as relevant as one from the current list. The model may then write a confident answer from an outdated passage. It doesn’t know what your employees know, and without a source reference, the customer cannot check where the answer came from. Building the chatbot is largely the work of writing that knowledge down in a form the system can apply.
Decide which source wins when answers conflict
When two sources disagree, a person picks one and moves on, usually without noticing that a decision was made. A chatbot can’t do that reliably. Given the old price list and the new one, it may choose either, or blend them into an answer that matches neither. The company has to decide in advance which source wins, and the decision has to be built into the system.
The rules are rarely complicated, but someone has to state them. The current version of a document replaces earlier versions. An official policy outranks a presentation, a marketing FAQ, or a note in the shared wiki. An agreement with a specific customer overrides the general price list for that customer, and only for that customer. A record in the company’s system outranks a PDF exported from it last quarter. Drafts, archives, and superseded files stay out of the chatbot, or are marked so that no answer is ever based on them.
These rules depend on information that documents often do not carry in a usable form: which document is official, which version is current, who it applies to, and when it stops being valid. Part of every chatbot project is adding it, so the system can apply the rules before the model writes anything.
Which document wins?
A question typed into a company chatbot: “What discount do we get on orders over 500 units?” This fictional example takes place on 1 April 2026, while the agreement is valid. The search found four documents. Switch the rules on and off to compare illustrative answers.
Who is asking
Source rules
Leak
Chatbot
Orders over 500 units get a 15% discount until 30 June 2026.
Based on: Kestrel Supply agreement
This is another customer's negotiated price, shown to a stranger. It is also not a price the company offers the visitor.
What the search found
A. Price list 20258%
Official price list · January 2025
“Orders over 500 units: 8% discount.”
Found, but not used
B. Price list 202610%
Official price list · January 2026
“Orders over 500 units: 10% discount.”
Found, but not used
C. Sales team wiki pagearound 12%
Informal internal note · edited March 2024
“Bulk orders usually get around 12%.”
Found, but not used
D. Kestrel Supply agreement15%
Signed customer agreement · valid until 30 June 2026
“Orders over 500 units: 15% discount.”
Used in the answer
Each rule is simple. The chatbot can apply them only if someone has written them down and recorded, for every document, whether it is official, which version it is, and who may see it.
Some conflicts appear only over time. A document that was correct at launch becomes wrong when a price changes, a policy is revised, or a product is discontinued. This is easy to miss, because a wrong answer drawn from an outdated document sounds exactly as confident as a right one.
The fix is mostly organizational. Every source the chatbot uses needs a named owner who keeps it current. The events that change the information, such as a new price list or a revised policy, should trigger the update; an annual review is too slow. Where the date matters, the answer can show the date of its source so the reader can judge how fresh it is. Answers drawn from a neglected document go out of date with it, however well the other sources are kept.
Many chatbot answers are not in documents at all
“Where is my order?” “How many days of leave do I have left?” “Is this part in stock at the Izmir warehouse?” “When does my contract renew?” These questions need current, account-specific answers that a general document collection usually cannot provide. The answers live in business systems such as order management, the HR platform, the inventory database, and the CRM. Some change by the hour; others depend on the person asking.
A chatbot built only on documents handles these questions badly. It may send the customer somewhere else to look. Or it may find a document that looks related, such as a general description of delivery times, and present that as the answer.
To answer them properly, the chatbot has to be connected to the systems that hold the answers. When a customer asks about an order, the chatbot identifies the customer, looks up that customer’s orders through the order system’s interface, and answers from the live record. That is different work from indexing documents. Each system needs a connection, a clear definition of what the chatbot may read, and handling for the cases where the system is slow, unavailable, or returns something unexpected. In projects like this, most of the effort goes into these connections rather than the model, a pattern described in You’re not doing AI. You’re doing integration.
A useful exercise before any project begins is to take a sample of real questions and mark where each answer lives: in a document, in a system, or in someone’s head. Count how many fall into each group before deciding what to build. A document chatbot can handle the first group. The second tells you which system connections the project needs, and the third shows what has to be written down or handed to a person.
The chatbot has to know who is asking
The same question can have different correct answers for different people. A customer and a dealer asking about prices should see different numbers. Employees in Turkey and Germany asking about leave are covered by different rules. A manager may see salary bands for their team; an individual employee should not.
In a well-run company, the systems people use already enforce these boundaries. Shared folders have permissions, the HR platform shows each employee only their own record, and customer accounts are kept separate. When documents are copied into a chatbot’s search index, the project must check that the connector preserves and enforces those permissions. Without that enforcement, the index contains everything it was given, and anyone who asks the right question can receive an answer built from any of it. The OWASP security project lists this among the main risks of applications built on retrieval: weak access controls on stored material can lead the model to retrieve and disclose personal data, proprietary information, or content meant for another group of users. OWASP
The chatbot therefore has to identify the person asking and limit its search to the documents and records that person may see, before the model sees anything. A prompt asking the model to keep a secret is not a reliable access control. Enforce permissions in retrieval and system connections, before restricted information reaches the model.
In practice, every document and record the chatbot uses has to carry information about who may see it, and that information has to stay in step with the company’s existing permissions as people join, leave, and change roles. For a public website chatbot answering from public pages, the requirement is light: keep internal material out entirely. For an internal assistant serving several departments, or a customer chatbot that looks up individual accounts, access control is one of the largest parts of the project.
A good chatbot knows when to stop
A well-built chatbot declines some questions. Some need a person under the company’s escalation policy: a complaint that could become a legal dispute, a refund request beyond the stated policy, a medical or safety question, a message from someone clearly upset. Others cannot be answered from the information available, and a confident guess is worse than no answer. Deciding which questions fall into these groups is part of designing the chatbot.
The chatbot can answer when its sources cover the question, ask for clarification when details are missing, or hand over when it cannot answer reliably. For factual answers, show the supporting source and, where relevant, its date. “I’m not certain about this. Let me connect you with our billing team” is a better reply than a confident wrong answer, for the customer and for the company.
Some of these decisions are rules the company writes in advance: topics the chatbot must never answer, situations that always trigger a handover, and limits on what it may promise. Others depend on the evidence found for each question. The chatbot has to check whether any source covers it, whether the sources agree, and whether the person asking may see the answer. The project has to design and test both.
The handover needs design too. A customer who has already explained the problem to the chatbot should not have to explain it again. The person taking over should receive the conversation, the sources the chatbot found, and the reason it stopped. And someone has to be there to pick it up, during working hours or with a clear message about when a reply will come. Otherwise the customer is passed from a chatbot to an inbox nobody reads.
Test the chatbot with your real questions, before launch and after
A chatbot that performs well on the questions its builders thought of can still fail on the questions people actually ask. Real questions are phrased unexpectedly, combine two issues, use internal abbreviations or product nicknames, contain spelling mistakes, and ask about situations no document anticipated. Real questions from its intended users are an essential part of testing whether a chatbot is ready. Add deliberate attempts to retrieve restricted information or make it ignore its rules, since ordinary questions may never expose those failures.
Start by collecting them. Support emails, ticket histories, chat logs, and the questions sales and HR teams answer every week are the best sources. For a narrowly scoped pilot, one to two hundred questions can be a practical starting set, not a launch threshold. Include the awkward ones: the question that needs a live system, the one that should be refused, the one where two documents disagree, the one only certain people may ask. For each question, someone who knows the business writes down the correct answer, where it comes from, and who may receive it.
This list does two jobs. First, it helps define the chatbot’s scope. Questions that appear often and matter belong in it, and information that answers none of them may not need to be included. Second, it provides regression tests. Keep a separate set of questions out of day-to-day tuning so the launch review also tests cases the builders have not optimized for. Before launch, you check every version of the chatbot against it: whether the answer is correct, whether it came from the right source, whether restricted information stayed restricted, and whether the chatbot stopped when it should have. An average score won’t show everything. A chatbot can answer 95% of questions well and still reveal one customer’s price to another, and the average hides that. The broader case for evaluating on your own conditions is made in Perfect on paper, wrong in practice.
Showing sources makes this work much easier. When every answer lists the documents or records it relied on, a reviewer can investigate whether the failure came from outdated content, retrieval that missed the right passage, or a model that misread it. Retrieval logs help distinguish these cases. A citation alone does not prove that the source supports the answer. Sources also let users check an answer themselves, which matters most when the answer has consequences.
Testing does not end at launch. Documents change, new products appear, and people ask questions nobody expected. Someone should regularly review the conversations that ended in a handover, the answers users marked as unhelpful, and the questions the chatbot couldn’t answer, and add the important ones to the test set. Over time, the test set becomes an accurate description of what the chatbot is expected to know, and the fastest way to check any change, from a revised document to a new model.
Where will the work be in your chatbot project?
Four questions to help you start scoping the work.
1 of 4
Where do most answers live?
Next question
Start with the questions and the rules
A company that wants a chatbot built on its own data usually starts by gathering documents. A spreadsheet is a better starting point. Collect a hundred real questions. For each one, write down where the correct answer lives, which source wins if there is more than one, who may receive it, and whether the chatbot should answer it at all.
That spreadsheet is uncomfortable to fill in. It exposes the price list nobody updated, the policy that exists in three versions, the answer that lives only in one person’s head, and the system nobody has an interface for. Without it, a customer finds those gaps later, in a chatbot answer that sounded completely sure of itself.
The spreadsheet gives the team a concrete starting specification. You can choose the model that writes the answers and replace it later. Your company still has to own the source rules and acceptance criteria, then test whether the whole system follows them.
