Skip to content

2026-05-27 · 6 min

Evidence or it didn't happen

A finding without the API response that produced it is an opinion. How kaveo stores, links, and replays its evidence.

Every security scanner produces findings. Very few produce evidence. The difference sounds academic until you're in an audit, a vendor review, or an incident retro — the three rooms where "the dashboard said so" is not an acceptable sentence.

A finding is a claim

When kaveo reports S3 bucket allows public read, that is a claim about the world at a moment in time. Claims invite questions. Which API call established it? What did the response actually say? Was that before or after Tuesday's change window? Can you show me?

Most tools answer with a severity color. The finding exists because the tool says it exists, and when an engineer disputes it — "we fixed that," "that's the marketing bucket, it's supposed to be public" — the conversation has nowhere to go but opinion.

Store the response, not just the verdict

kaveo's rule engine works exclusively from configuration snapshots it has already stored. When a rule fires, the finding records which snapshots it evaluated, and each snapshot is the raw AWS API response — GetBucketAcl, GetBucketPolicyStatus — with its timestamp and request context.

The console renders this as a trace link on every finding. Click it and you see the exact JSON that made the rule fire. Export it and you have an audit packet an assessor can verify without trusting our UI at all.

Three rooms where this changes the conversation:

  • The audit. The assessor asks how you know encryption was enforced in March. You hand them March's responses, not March's screenshots.
  • The dispute. An engineer says the finding is stale. The evidence timestamp settles it in thirty seconds, in either direction.
  • The retro. You need to know when the bucket went public. Snapshots over time are a timeline, not an argument.

Determinism makes evidence possible

Evidence-backed findings only work because detection is deterministic. Rules are reviewable YAML: same snapshot in, same finding out, every time. A probabilistic detector can't produce this kind of evidence — you can't replay a hunch.

We do use language models in the product, for one job: explaining findings and drafting remediation guidance in plain English. The explanation layer never gates, scores, or suppresses a finding. Reasoning on top, determinism underneath.

If a security product's claims can't survive the question "show me," the product is a dashboard. Evidence or it didn't happen.