The lecture
On Monday I sat in a lecture on my master’s, part of the Digital4Business programme, given by Yuriy Perezhohin. The topic was “grounding and RAG”, and within twenty minutes I was thinking less about the lecture and more about the businesses I mentor.
The lecture opened with a simple experiment. A small AI model was asked three everyday consumer questions. It said flight delay compensation in the EU is €240 for every hour over three hours; the rule is a fixed €250. It said you have 30 to 60 days to return something bought online; the law says 14. It said the legal guarantee on a laptop is one to three years; it’s at least two.
Every answer was fluent, confident and wrong, and there was nothing to open and check.
That’s the problem with how most people use AI today. A model answers from memory: whatever it absorbed during training, up to a cut-off date. It has never read your procedures, your contracts, your price list or your staff handbook. When it doesn’t know something, it doesn’t say so. It writes whatever sounds most likely. And likely isn’t the same as true.
RAG in plain English
RAG stands for retrieval-augmented generation. The idea is simple: instead of letting the AI answer from memory, you make it look things up first.
Think of it as the difference between a memory test and an open-book exam. In a memory test, you write what you remember and hope it’s right. In an open-book exam, you find the right page, read it, and answer from it, and anyone marking your paper can check the page you used.
RAG works in three steps:
- Index your documents, once. Your documents are cut into short sections, and each section is turned into a list of numbers that represents its meaning. Sections about similar things end up with similar numbers, even when they use different words.
- Retrieve, for every question. The question is turned into numbers the same way, and the system finds the sections of your documents closest in meaning.
- Generate, with citations. Those sections are given to the AI along with the question. It answers from them and points to the section it used.
Nothing in the AI itself changes. There’s no training, no expensive customisation. Update a document, and the next answer uses the new version.
What could a small business do with it?
The more I thought about it, the more uses I could see. Almost every small business I work with runs on documents of one kind or another.
Building standards and safety. A contractor’s method statements, risk assessments and safety statement have to keep up with building regulations and safety legislation. RAG can check each document against the current rules and flag what doesn’t match: “your working-at-height procedure doesn’t mention X, which the regulations require.”
HR policies. Staff have questions about leave, expenses, sick pay and grievance procedures every week. A RAG assistant answers from your actual handbook and links to the section, so the answer is your policy, not the AI’s idea of a typical one. The same method can check the handbook itself against changes in employment law.
Business policies and customer service. Returns, warranties, delivery terms and pricing, answered from your own policies. This is also where the biggest warning applies. In 2024, a Canadian tribunal ruled that Air Canada was responsible for a wrong refund policy its website chatbot had invented. The airline argued the chatbot was responsible for its own words, and the tribunal disagreed. If your AI tells a customer something, you said it.
Medical devices and other regulated sectors. Quality procedures checked against the regulations before an audit. This is the one I tested, and I’ll come to it below.
Staff training. This is the use I’d suggest most small businesses try first. Picture a new salesperson with a question: “A customer wants 15% off for a bulk order. What can I offer?” The training assistant answers from your sales policy, quotes the discount rules, and links to the section so they can read it in full. It can also quiz them: generate questions from your policies and check their answers against them. Training is internal, a manager can see the answers, and nobody outside the company relies on it, so the risk is low compared with putting a chatbot in front of customers. It’s the safest place to learn how these tools behave.
So I tested it
I spent years in the medical device sector, and if there’s one thing I remember, it’s audits. Weeks of reading procedures against regulations, line by line, hoping nothing had been missed. So that’s where I tested it.
I wanted the test to be honest, so I built a fictional company where I knew every answer in advance.
The test set. Slieve Bloom Medical Ltd is invented. It makes the DrainSure single-use wound drainage system, a Class IIa medical device sold in the EU. I wrote eight realistic quality procedures for it: document control, corrective and preventive action (CAPA), complaints, supplier control, training, management review, design and change control, and post-market surveillance and vigilance. Then I hid 15 mistakes in them. They came in three types:
- Missing: no procedure for something the regulation requires, such as UDI (unique device identification) or clinical evaluation.
- Wrong: a procedure that states an incorrect rule, such as serious incidents reported “within 30 days” when the EU Medical Device Regulation (MDR) requires 15 at most, or a reference to a standard that was replaced years ago.
- Conflicting: two procedures that contradict each other: one says staff must be trained before a revised procedure takes effect, another says within 30 days after.
The checklist was the list of quality system requirements in Article 10(9) of the MDR: thirteen items, free to use.
The tools. All free:
- The AI model: Qwen2.5 Instruct, an open model, in two sizes: 7 billion parameters (4.7 GB) and 14 billion (9 GB). Both are small enough to run on an ordinary laptop, the smaller one on 8 GB of memory and the larger on 16 GB.
- Ollama, a free program that runs models like these locally.
- The search: all-MiniLM-L6-v2, a small free model that turns each section into 384 numbers representing its meaning.
- Where I ran it: a free Google Colab notebook, so anyone can repeat it in a browser.
The software cost: €0.
The twist: reverse RAG. Normally, a person asks the question and the system finds the documents. I turned it round: each requirement in the regulation became the question. The system found the company’s procedure sections closest to it and asked the AI for a verdict: COVERED, WEAK or MISSING, with a reason and a citation. That’s what an auditor does, clause by clause.
Round one: the honest baseline
The first run scored 6 out of 13 verdicts correct. When I checked the reasons by hand, it had genuinely identified only 2 of the 15 hidden mistakes.
Every error went the same way: it said COVERED when the answer was WEAK or MISSING. It never raised a false alarm; it only missed problems. For audit preparation, that’s the dangerous direction: a tool that tells a non-compliant company it’s mostly fine.
When I read the report to see why, the main cause surprised me. Often the AI never saw the problem. The search kept handing it the Purpose, Scope and Responsibilities sections of each procedure. Those sections mention the topic by name, but say nothing about what actually happens. The section with the wrong 30-day deadline was never retrieved. The AI reasonably concluded vigilance was covered, because every page it was shown mentioned vigilance.
The other cause was knowledge. The AI read “ISO 14971:2007” and called it fine, because it didn’t know the 2019 edition replaced it. A model that only reads your documents can’t know the rules you’re checking them against.
Round two: one change at a time
The lecture’s most useful lesson was to change one thing at a time and re-run the same test, because a fix for one question can break another. So I tried three fixes, separately and together, on the 7B model:
| Run | What changed | Verdicts right (of 13) | Verdict and reason right (of 13) |
|---|---|---|---|
| 1 | Baseline | 6 | 5 |
| 2 | Give the AI whole procedures, not sections | 4 | 4 |
| 3 | A stricter instruction: check every part of the requirement | 7 | 5 |
| 4 | Give the AI the relevant regulation rules too | 10 | 7 |
| 5 | Stricter instruction + regulation | 5 | 4 |
| 6 | All three together | 6 | 5 |
Three things stood out.
Giving the AI the rules was the single biggest improvement. With the relevant MDR articles alongside the procedures, it correctly flagged the outdated ISO 14971 edition, the missing clinical follow-up plan, and even something that wasn’t on my checklist: no person responsible for regulatory compliance (a PRRC) named anywhere.
More text made the model more confident, not more careful. Handing it whole procedures made the score worse. A complete-looking document on the right topic, with neat numbered steps, convinced it everything was covered.
Combining fixes made things worse. The stricter instruction plus the rules dropped to 5. My reading is that a small model has limited capacity for instructions: one extra job helps, three at once and it loses the thread.
My scoring was flawed, and I had to fix it. In run 4, the model said WEAK to almost everything. My answer key accepted WEAK for most items, so a tool that answered WEAK to everything, without reading a word, would have scored 11 out of 13. So I added a second score: a verdict only counts if the reason also names the actual planted problem. Test the test, not just the tool.
Round three: check like an auditor
The table showed something else: the requirement-by-requirement check is good at finding what’s missing, but poor at finding what’s wrong, because the search has to land on the exact sentence with the error.
A good auditor doesn’t only go clause by clause. They also read each procedure on its own merits, and look for places where procedures disagree. So I added two more passes, each asking the AI one short, simple question at a time, which suits small models much better than one long, complicated instruction:
- Pass 2, section versus rules: every one of the 59 substantive procedure sections was compared with the three closest regulation rules. “Does this section contradict any of these rules?”
- Pass 3, section versus section: the 20 most similar pairs of sections from different procedures (out of 1,522 possible pairs) were compared. “Do these two give conflicting instructions?”
Pass 2 went straight to the problems. Pass 3 had an interesting result with the 7B model: the two contradicting training sections were the second most similar pair in the whole system (0.82 similarity), so the search found them perfectly, but the model read “before” and “within 30 days after” and decided they didn’t conflict. The search did its job; the model didn’t.
So, finally, I tried the 14B model.
The results
Hand-checked against the 15 hidden mistakes:
| Baseline (7B) | 7B, all three passes | 14B, all three passes | |
|---|---|---|---|
| Hidden mistakes found | 2 of 15 | 8 of 15 (+1 partial) | 9 of 15 |
| “Wrong” and “conflicting” mistakes found | 0 of 5 | 3 of 5 | 5 of 5 |
| False alarms from passes 2 and 3 | n/a | 3 of 8 flags | 1 of 7 flags |
The 14B model caught every factual error and contradiction I planted: the outdated directive, the outdated standard, the 30-day deadline, and the training contradiction. On the wrong type of post-market report, it gave exactly my reason: the document it describes “applies only to Class I devices, and DrainSure is a Class IIa device.”
But it wasn’t better at everything. On the broad requirement check, it played safe and called all 13 requirements WEAK, including UDI, where there’s no procedure at all. If everything is “weak”, a quality manager can’t tell what to fix first.
Never found, by any setting: the CAPA procedure with no check that actions worked; supplier approval based on a certificate alone; design changes with no assessment of whether they need notifying to the notified body; and a management review that ignores post-market data. These are all judgement calls: not whether a process exists, but whether it’s good enough. That’s still a job for an experienced person.
Warnings
If you take one thing from this post, take these.
It will sound right when it’s wrong. Every wrong answer in this test was fluent and confident. One stated, falsely, that the procedures “do not specify the timelines” for incident reporting. They did; they specified the wrong one.
The search decides what the AI can see. Most of my early failures happened before the AI read anything. If you buy a “chat with your documents” tool, ask how it finds the sections, and test it with questions where you know the answer is buried in the middle of a document.
More isn’t better. More text and more instructions both made a small model worse. Every change needs testing.
Rules leak. Given a set of regulation extracts, the model sometimes applied them where they didn’t belong, demanding field safety procedures inside a CAPA procedure, for example. Irrelevant context doesn’t just get ignored; it gets used.
Test before you trust, and test the test. Without known answers, I’d have had a confident report saying a non-compliant company was mostly fine. And my own first scoring method could be fooled by a model that said “weak” to everything.
Check the citations. A right answer for the wrong reason is a warning sign, not a success.
False alarms cost time; misses cost more. A false alarm takes a person a few minutes to dismiss. A missed problem costs you at the audit, or in front of a customer.
A person signs off. Every report this tool produces is headed “Draft for review by a qualified person.” That isn’t a disclaimer; it’s the design. For anything that commits the business (refunds, contracts, compliance, safety), a person decides.
You are responsible for what it says. If an AI answers your customers, the Air Canada case says those answers are yours.
Your documents need an owner. RAG is only as good as the documents it reads. Out-of-date procedures give out-of-date answers, with a citation that makes them look authoritative. And think about who may ask what: an HR assistant shouldn’t reveal salaries to everyone.
Know where your data goes. Running a model on your own laptop or server means nothing leaves the building. Free cloud notebooks like Colab are fine for testing with invented data like mine, but not for your real procedures or staff records.
Be honest about how you tested. I designed my fixes after seeing how the tool failed on this test, so the improvement partly reflects that. The fair next step is a second fictional company with different hidden mistakes. And the regulation notes I gave the AI are my own paraphrase. For real use, work from the official text.
How a small business could start
- Start with training, not customers. Pick one area, such as your sales policy or staff handbook, and let staff ask it questions with a manager watching the answers.
- Put your documents in order. Clear headings, current versions, plain text or Word rather than scans.
- Write ten questions you know the answers to, including two whose answers aren’t in your documents. If it invents answers to those, you’ve learned something important.
- For checking against rules, give it the rules, and check section by section, not just requirement by requirement.
- Read the citations, keep a person in the loop, and re-test after every change.
Try it yourself
Everything is on GitHub: the code, the eight fictional procedures, the 15 hidden mistakes, the answer key and the scoring: http://github.com/Declan444/qms-gap-rag. Look at the readme file and one-click Google Colab notebook runs the whole test in your browser, with nothing to install.
Next, I want to try it on a second fictional company with different hidden mistakes, to see whether the improvements hold up. I’ll publish those results whether they’re good or bad.
If you run a small business and you’re wondering whether something like this could work on your own documents, I’d be glad to talk it through: services@dlconsultancy.ie
With thanks to Yuriy Perezhohin and the Digital4Business programme for the lecture that started this.