The answer to most work questions at a company is already written down somewhere: in a policy, a how-to guide, a contract, a wiki page or an email thread from a year ago. The problem is that finding it takes ages. People dig through folders, message a colleague, wait for a reply, or go from memory and sometimes get it wrong.
An AI knowledge base solves exactly that: an employee asks a question in plain words and gets a short answer with a link to the document it came from. Let's look at how it works, how it differs from regular search and from ChatGPT, what to do about your documents and access rights, and how to run a pilot.
Why a company needs an AI knowledge base
Internal knowledge is usually scattered. Policies sit in a shared folder, how-tos in a wiki, supplier terms in contracts, and half of the practical know-how lives in chats and email. Each place makes sense on its own, but together they become a maze.
Typical questions that eat up time:
- What is the approval process for a business trip, and who signs off?
- What are the payment terms in our contract with a specific supplier?
- How do we process a return if the customer lost the receipt?
- Where is the power of attorney template, and which version is current?
- How do I set up access for a new hire in the accounting system?
These questions usually land on experienced colleagues, whose day keeps getting interrupted. New hires have it even harder: they don't know where to look and hesitate to ask. An AI assistant for employees takes repetitive questions off experienced staff and speeds up onboarding.
This is not the same as a customer-facing assistant on your website. A customer assistant answers from public information about products and services, while an internal one works with confidential documents, so access rights and data protection matter far more.
How it works, in plain terms
Technically, the approach is called RAG, retrieval-augmented generation: the answer is generated from documents found by search. It sounds complicated, but step by step it is easy to follow.
- Documents are split into passages. Policies, guides, contracts and wiki pages are broken into meaningful chunks: a section, a clause, a paragraph. Each passage keeps track of which document and version it came from and who can see it.
- Passages are indexed by meaning. The system captures not just the words but the meaning of the text. So the question "how many days do we have for a return" finds the clause that says "claims must be filed within 14 calendar days."
- The question retrieves matching passages. When an employee asks something, search picks the few passages closest in meaning from documents that person has access to.
- The model answers only from them. The AI model receives the question and the retrieved passages with an instruction: answer only from this text, and if the answer isn't there, say so.
- The answer comes with a source link. The employee sees a short answer and a link to the document and clause, so they can open the original and check if needed.
The key point of this setup: the model isn't recalling, it's reading your documents. It works like an attentive colleague who was handed the right pages and asked to sum them up.
How it differs from regular search and from ChatGPT
Regular document search looks for matching words. If the policy says "business travel" and the employee searches for "work trip," they may find nothing. And if they do find something, they get a list of twenty files in which the answer still has to be dug out.
Simply asking ChatGPT doesn't work either. A public model knows nothing about your policies, prices or contracts. When it lacks data, it may confidently produce a plausible but invented answer. On top of that, uploading internal documents to a public service is often forbidden by security policy.
| Keyword search | AI answer with sources | |
|---|---|---|
| How to phrase the question | Guess the words in the document | In plain words, as to a colleague |
| What the employee gets | A list of files | A short answer and a link to the clause |
| Synonyms and paraphrases | Poorly | Understands meaning |
| Answer spanning several documents | Assemble by hand | Combined into one answer |
| Risk of error | Didn't find what was needed | Inaccurate summary, hence the source link |
| Verification | Read whole files | Open the source from the link |
Smart search across policies doesn't replace regular search: many people like having both. But for "how do I do this" questions, an answer with a source saves more time than a list of files.
Source quality: it won't work without order in your documents
AI for internal documents won't fix chaos in the documents themselves. If a folder holds three versions of a policy, the assistant may quote the old one. If two documents contradict each other, the answer depends on which passage was found first.
An assistant answers only as well as the documents it was given.
So documents need sorting before launch. Not rewriting everything, just establishing basic order:
- Remove what's outdated. Old policy versions go to an archive that search doesn't cover.
- Resolve contradictions. If two documents describe the same process differently, decide which one applies.
- Assign section owners. Every section of the knowledge base needs someone responsible for keeping it current: HR, procurement, sales, IT.
- Agree on updates. When a policy changes, the new version enters the knowledge base and the old one is archived. Ideally this happens automatically from wherever the documents are stored.
Access rights: everyone sees only their own
A knowledge base will hold documents of different sensitivity: general guides, supplier contracts, HR files, financial policies. A warehouse employee must not get an answer from a document about executive pay, even if they ask directly.
The rule is simple: answers are built only from documents the employee can access. Permissions are checked at the search stage, before any passages reach the model. If permissions are checked only after generation, restricted information can leak into the summary.
It is better not to set up permissions from scratch but to inherit them from the systems where documents already live: the intranet, file storage, the CRM or the ERP. Then a change in access in one place is reflected in the assistant right away.
How to stop the assistant from making things up
You can't eliminate AI errors entirely, but you can make them rare and visible. Several techniques are used together.
- Permission to say "I don't know." The model gets an explicit instruction: if the retrieved passages don't contain the answer, say so and suggest who to contact. An honest "I don't know" beats a confident invention.
- Quotes and links. Every answer comes with a link to the document and clause, sometimes with an exact quote. An employee can verify it in seconds.
- Topic limits. An assistant for internal documents doesn't chat about the weather or write poems. It politely declines questions outside the knowledge base.
- Feedback. "Helpful" and "Wrong" buttons under each answer. Flagged errors are reviewed by the section owner.
- Regular checks. A benchmark set of questions is rerun after every change to documents or settings.
Where to put the assistant
A knowledge base chatbot is useful where employees already spend their working day. A separate site you have to go out of your way to open gets used less.
- Web interface. A dedicated page or a section of the intranet with a chat and question history. Convenient for office staff and for long answers with links.
- Telegram or another messenger. Good for employees on the road, in the warehouse or in retail. Sign-in is tied to the employee's account so access rights still apply.
- Inside the CRM or intranet. An assistant panel right in the work system: a manager asks about contract terms without switching windows.
The same assistant can run in several channels at once: search and answers work the same way, only the interface changes.
Data and GDPR: cloud or local model
Internal documents often contain personal data: names, contact details, ID numbers in contracts, HR information. Processing that data is governed by the GDPR, and sending it to an external service, especially one outside the EU, brings additional requirements.
There are two main approaches:
- Cloud model. A fast start and strong models, but document passages go to the provider. You need to know where the data is processed, what the provider contract says, and whether personal data can be sent there.
- Local model. Runs on your own servers or in a European cloud, and data never leaves the company's perimeter. It requires hardware and maintenance, and answer quality has to be tested on your documents.
A mixed option is also possible: personal data is masked before it reaches the model, and sensitive sections are processed only locally. The specific setup should be agreed with your lawyer and security team before development starts, not after.
How to measure quality
"It seems to answer fine" is a poor metric. The quality of an AI knowledge base has to be measured, and that doesn't take complex math.
- A benchmark set of questions. Real employee questions from chats and email, each with the correct answer and source document. Include questions the documents can't answer: the correct response to those is "I don't know."
- Share of correct answers. How many questions from the set the assistant got right, how many times it honestly declined, and how often it gave the correct source link.
- Search time. How many minutes an employee spends on a typical question before and after launch.
- Usage. How many people use the assistant regularly, and how many questions have moved out of group chats.
The benchmark set is run before launch, after every major change and regularly in production. That shows whether things actually improved, not just whether they feel better.
The pilot: where to start
Rolling out an assistant to the whole company at once is risky and expensive. It's better to start with one department that has lots of questions and documents in reasonable shape.
- Choose a department and typical questions. For example, HR questions, internal systems support or contract work. Collect the questions employees ask most often.
- Sort the documents. Select the relevant sources, remove outdated ones, assign section owners.
- Benchmark questions and baseline measurements. Record how people find answers today and how long it takes.
- Build a prototype. Document indexing, permission-aware search, and answers with links in one channel.
- Test with a small group. A few weeks of use, collecting feedback, reviewing errors and fixing documents.
- Decide on scaling. Compare metrics against the baseline and decide whether to add other departments and channels.
We build a pilot like this as a small web service integrated with the systems where your documents live. The approach is the same as in our other projects as an AI and web development agency: first we understand the processes, then we design and build. You can see examples in our projects.
Readiness checklist
Before the project starts, make sure you have:
- A clear group of users and a list of their frequent questions.
- Documents where the answers to those questions are actually written down.
- Section owners ready to take responsibility for keeping content current.
- An understanding of where documents are stored and how access rights work there.
- A decision from your lawyer and security team on personal data and the choice of model.
- A channel where employees find it convenient to ask questions.
- A benchmark question set and baseline measurements of search time.
If the answer to most of these is yes, you can start the pilot. If not, spend the first weeks on documents and permissions: order in your sources pays off in answer quality.
FAQ
How is an AI knowledge base different from ChatGPT?
ChatGPT knows nothing about your policies, prices or contracts and, when it lacks data, may confidently invent an answer. A corporate AI knowledge base first finds passages in your documents, answers only from them and shows a link to the source, and if the documents don't contain the answer, it says so honestly.
Do we need to rewrite our documents before launch?
You don't need to rewrite everything. It is enough to remove clearly outdated versions, resolve contradictions in key policies and assign section owners. The rest gets sorted during the pilot: the assistant itself shows where the documents are weak.
Will employees see documents they don't have access to?
They shouldn't. Permissions are checked at the search stage: passages from restricted sections never reach the model if the employee has no rights to them. Permissions are usually inherited from the systems where the documents already live.
Can we do without cloud AI?
Yes, the model can be deployed on your own servers. That makes data control simpler but requires hardware and maintenance, and answer quality has to be tested on your documents. The choice between cloud and local is made after assessing which data will go into the knowledge base.
How do we know the assistant answers well?
Build a benchmark set of questions with correct answers and sources, and run it before launch and after every change. Track the share of correct answers, the share of honest "I don't know" responses, the accuracy of source links and the time employees spend searching.




