How it works
What happens in a review?
A review runs in six steps. Each has set inputs, checks that run in code, and a point where a reviewer decides whether to carry on.
-
Step
Profile the organisation
A short, guided conversation establishes what kind of organisation this is: its sector, its size, where it operates, who regulates it and what it has outsourced. You confirm each fact, and the confirmed profile decides which assessment criteria apply.
Nothing about the organisation is assumed from its documents.
-
Step
Stage the documents
Upload policies, standards, procedures, questionnaires and evidence. Each document is given a role, a version and a scope, so a draft is never treated as current policy and last year's version is never treated as this year's. Scanned PDFs are read with optical character recognition in your deployment's region.
Word and PDF, including scans, and Excel questionnaires.
-
Step
Map
Keelhold reads every document and maps it to the areas it covers, with a coverage rating for each area and a note of what is out of scope. A reviewer corrects the map before anything is assessed, and can set a document aside.
A person checks the map first.
-
Step
Assess
Each area's criteria are assessed against the mapped documents. A rating has to quote the passages it rests on, and code checks that every quotation is in the source. Conflicts between documents are listed, and a cross-check looks for disagreements between areas.
Quotations are checked by code, not by the model.
-
Step
Decide
Reviewers accept, correct or reject each result under their own sign-in. Running a review and reviewing its results are separate roles. Every decision, and every change to who is on the engagement, is kept in its history.
Nothing counts until a decision is recorded.
-
Step
Report
Findings carry a severity worked out from likelihood and impact. Code builds the reports from accepted results only: a Word report for the reader and an Excel workbook for the detail, with the quotations and decisions behind every finding.
Reports are built from accepted results.
Limits
What will it not do?
These are rules in the code and in the instructions the model is given.
- Give credit for a control because a policy says it should exist.
- Treat a service as available because a catalogue lists it.
- Treat a self-assessment as evidence that something is done.
- Approve policy wording, adopt a set of questions, accept a risk or release a report.
- Present a draft or an old version as current policy.
- Give a rating without a quotation that code has found in the source.
- Read a document that was not staged for the run.
- Offer legal, audit or certification assurance. Qualified people do that, and Keelhold gives them better material to do it with.
What you need
What do you need to start?
The documents, in Word or PDF, including scanned PDFs, and any questionnaires in Excel. Keelhold imports a workbook's questions and answers as they are.
A reviewer with the authority to decide. Keelhold takes away the reading and the cross-referencing, not the judgement.
Somewhere to run it. Our hosted service, or your own Azure subscription if your data has to stay in your own estate.
Models
Which model does it use?
Whichever the deployment is set up with, through Microsoft Foundry. Every run records the exact model profile: provider, model, version, deployment and reasoning settings. We test Keelhold against a synthetic estate of invented organisations and documents, and we repeat that testing before a change of model, prompt or workflow is released.
Different models give different wording and sometimes different ratings, so two deployments on different models will not always agree. What holds in every deployment is that each rating is traceable to quoted evidence and a recorded decision.