Skip to content

2026-05-27 · 6 min

Evidence or it didn't happen

A finding without the API call that produced it is an opinion. How kaveo records, links, and cites its evidence.

Every security scanner produces findings. Very few produce evidence. The difference sounds academic until you're in an audit, a vendor review, or an incident retro — the three rooms where "the dashboard said so" is not an acceptable sentence.

A finding is a claim#

When kaveo reports S3 bucket allows public read, that is a claim about the world at a moment in time. Claims invite questions. Which API call established it? What did the response actually say? Was that before or after Tuesday's change window? Can you show me?

Most tools answer with a severity color. The finding exists because the tool says it exists, and when an engineer disputes it — "we fixed that," "that's the marketing bucket, it's supposed to be public" — the conversation has nowhere to go but opinion.

Store the evidence, not just the verdict#

kaveo's detectors work only from what a scan has already collected and recorded. What each collector reads becomes a row in an append-only evidence ledger: one row per collector run by default, and one per API call and page for collectors that report each call, such as IAM. Each row records the call, for example page 2 of iam:GetAccountAuthorizationDetails, and a SHA-256 hash of the resources it returned, not the payload itself. A database trigger rejects any edit to those rows.

A finding about a collected resource links to the stored observation behind it, and the API returns that link for every finding. An assessor can check the source without trusting our UI at all.

Every claim cites a stored observationSection through kaveo, one level per stage, inside your perimeter. Accounts: three AWS accounts. Collection: 44 AWS collectors make read calls only. Evidence: what each collector reads becomes an observation in an append-only ledger, one per collector run, or one per API call and page for collectors such as IAM. Each observation records the call and a SHA-256 hash of what it returned, not the payload. A database trigger rejects edits and direct deletes. Findings: 93 deterministic AWS detectors; the same state always gives the same findings, and each finding cites the observations behind it. Citation gate: every claim from the AI layer is checked before anything is stored. AI layer: five stages, with an offline stub by default. Four claims rise from the AI layer. Two cite observation ids they were given; they pass the gate and are stored as AI output together with its citations. One claim has no citation and is dropped at the gate. One cites an id it was not given and is dropped. If no claim survives, nothing is stored. Note 1: grounding proves each cited observation exists and was supplied, not that the model read it correctly.YOUR PERIMETERACCOUNTSAWS accountAWS accountAWS accountCOLLECTION44 AWS COLLECTORS · READ CALLS ONLYEVIDENCEeach row: call + SHA-256 of its resultobsFINDINGSfindingfindingfinding93 AWS DETECTORS · SAME IN → SAME OUTstored AI outputWITH ITS CITATIONSif no claim survives, nothing is storedCITATION GATEno citation → droppedcites an id it wasnot given → droppedEVERY CLAIM CHECKED1AI LAYERFIVE STAGES · OFFLINE STUB BY DEFAULTEvery claim cites a stored observationSection through kaveo, one level per stage, inside your perimeter. Accounts: three AWS accounts. Collection: 44 AWS collectors make read calls only. Evidence: what each collector reads becomes an observation in an append-only ledger, one per collector run, or one per API call and page for collectors such as IAM. Each observation records the call and a SHA-256 hash of what it returned, not the payload. A database trigger rejects edits and direct deletes. Findings: 93 deterministic AWS detectors; the same state always gives the same findings, and each finding cites the observations behind it. Citation gate: every claim from the AI layer is checked before anything is stored. AI layer: five stages, with an offline stub by default. Four claims rise from the AI layer. Two cite observation ids they were given; they pass the gate and are stored as AI output together with its citations. One claim has no citation and is dropped at the gate. One cites an id it was not given and is dropped. If no claim survives, nothing is stored. Note 1: grounding proves each cited observation exists and was supplied, not that the model read it correctly.YOUR PERIMETERaccountaccountaccount44 AWS COLLECTORS · READ-ONLYobsfinding93 AWS DETECTORSSAME STATE, SAME FINDINGSstored AI outputWITH ITS CITATIONSif no claim survives,nothing is storedno citation → droppedcites an id it wasnot given → droppedEVERY CLAIM CHECKED1FIVE STAGESOFFLINE STUB BY DEFAULTAWS ACCOUNTSCOLLECTIONEVIDENCEcall + SHA-256 per rowFINDINGSCITATION GATEAI LAYER

Three rooms where this changes the conversation:

  • The audit. The assessor asks how you know encryption was enforced in March. You hand them March's scan and the observations behind each finding, not March's screenshots.
  • The dispute. An engineer says the finding is stale. The scan the evidence belongs to settles it in thirty seconds, in either direction.
  • The retro. You need to know when the bucket went public. Each scan's observations are a timeline, not an argument.

Determinism makes evidence possible#

Evidence-backed findings only work because detection is deterministic. Each detector is a pure function of what the scan collected: same state in, same findings out, every time. A probabilistic detector can't produce this kind of evidence — you can't replay a hunch.

kaveo can use a language model for one kind of job: explaining findings and drafting remediation guidance in plain English. It is off by default, it never creates a finding, and any claim it makes that doesn't cite a stored observation is dropped before it is saved. Reasoning on top, determinism underneath.

If a security product's claims can't survive the question "show me," the product is a dashboard. Evidence or it didn't happen.

All posts