Verifying an evidence bundle¶

Someone sent you a file and says: this is what the model did. This page is how you check it, without trusting them and without buying anything.

Verifying a bundle is free and needs no licence key. The first check below needs no Mokili at all: fifteen lines of Python.

What a bundle is¶

One JSON file, mokili.evidence/1. It holds the four records of one run — the model, the scenario, the run and its result — the files the model needs (an FMU, a source file, a library's licence), and the digest of each.

{
  "schema": "mokili.evidence/1",
  "model": { … }, "scenario": { … }, "run": { … }, "result": { … },
  "digests": {"model": "sha256:2f48…", "scenario": "sha256:982a…",
              "run": "sha256:3dd7…", "result": "sha256:5428…"},
  "artifacts": {"sha256:098b…": "<the file, in base64>"},
  "assurance": {"level": "unsigned_record", "statement": "…"}
}

A record is named by the SHA-256 of its content. The scenario names the model by that digest, the run names the scenario and the result, and so on: change one character anywhere and a name no longer matches what it names.

Check it yourself¶

The digest of a record is the SHA-256 of its JSON written with its keys sorted, no spaces, in UTF-8. The digest of a file is the SHA-256 of its bytes.

import base64, hashlib, json, sys

bundle = json.load(open(sys.argv[1]))

def digest(record):
    text = json.dumps(record, sort_keys=True, separators=(",", ":"), ensure_ascii=False)
    return "sha256:" + hashlib.sha256(text.encode("utf-8")).hexdigest()

for kind, named in bundle["digests"].items():
    print(kind, "matches" if digest(bundle[kind]) == named else "DOES NOT MATCH")
for named, content in bundle["artifacts"].items():
    found = "sha256:" + hashlib.sha256(base64.b64decode(content)).hexdigest()
    print("file", named[:19], "matches" if found == named else "DOES NOT MATCH")

Then read what the records say, and check that they name each other: scenario.model is the model's digest, run.scenario the scenario's, run.result the result's.

In another language, write numbers as Python does (the shortest text that reads back as the same number, 1e-05 and not 0.00001): the digest is of the text.

Check it with Mokili¶

lakisa verify-bundle --bundle b.json

{"verdict": "verified", "scope": "bundle_integrity", "executed": false}

It does what the script above does, and also checks each record against its published format. executed is false: nothing of the model is run.

What this proves, and what it does not¶

It proves	It does not prove
The model, the scenario, the run and the result belong together.	Who produced the bundle, or when.
None of them, and none of the files, was edited after the export.	That the model describes the real system.
Which engine, in which version, on which platform, produced the result — as the run records it.	That running it again gives the same result.

A bundle says so itself: assurance.level is unsigned_record.

Who produced it: the seal¶

A bundle can be sealed in a Litatoli evidence log: an entry naming the bundle's digest, chained to the entries before it and signed with Ed25519. Ask the sender for the log and for their public key, by a channel you trust.

lakisa verify-seal --bundle b.json --log-file evidence.jsonl --expected-pubkey <hex>

`verdict`	Meaning
`sealed`	The log holds an entry naming this exact bundle, and the whole chain verifies.
`broken`	The log names the bundle, but its chain does not verify: the log was edited, or signed with another key.

Without the public key the chain can only be found consistent, and anyone could have written it: verify-seal refuses (seal_unpinned) unless you ask for that weaker check with --allow-unpinned, which answers signer: "not established". seal_not_found means no entry of the log names this bundle: it was never sealed there, or it changed since.

That it gives the same result again¶

Running the scenario again is another matter than reading the file: it runs the model. It needs Mokili and a licence; an auditor's is free and limited in time. Write to support@mokili.dev.

lakisa reproduce --bundle b.json

`verdict`	Meaning
`reproduced`	The rerun produced a result with the same digest.
`mismatch`	Another result: `result_differs` in the same environment, `environment_differs` when a version or the platform is not the one recorded.
`not_reproducible`	The scenario had no seed: a rerun is another sample, not a check.

The formats¶

Every record says which format it follows, "schema": "mokili.model/1", and a record written today stays readable: see Record versions. The formats are published as JSON Schema, under the Apache License 2.0, so that any tool can read and write them.