Artifacts
Methodology, metric, and reproduction boundary.
Benchmark methodology
LawVM benchmarks replay output against real-world publication surfaces. For Finland, that means comparing replayed point-in-time text against the Finlex updated/derived comparison surface.
Some divergences enter candidate review when primary-source checks support further investigation beyond the comparison surface. The residual taxonomy classifies each mismatch so that evaluation is not a single number but a typed evidence surface.
August 2026 report
| Metric | Value |
|---|---|
| Scope | Project’s Finnish law replay benchmark |
| Comparison | Declared Finlex updated/derived surface |
| Mean normalized text edit distance | 0.23% |
| Provenance | Project-reported development result |
Reported in August 2026. The 0.23% measures distance to the declared comparison surface. Correctness and proof coverage require source review, residual classification, and separate denominators.
Evidence package status
The Finnish evidence work exposes replay-vs-Finlex comparison-surface divergences for triage. The project reported 22 candidates that passed its internal review threshold for external review. These remain candidates pending confirmation by a competent authority.
The internal review dataset records statutes, affected provisions, source evidence, comparison role, and disposition. Individual review packets are not part of this website; the public surface reports aggregate status and review boundaries only.
Run the current benchmark
uv sync
uv run lawvm bench --mode official_consolidation --corpus data/finland/bench_core.csv --no-save
This runs the included Finnish benchmark manifest without writing benchmark history or run artifacts. A newly acquired archive can differ as publisher-controlled comparison surfaces change.
The source archive (finlex.farchive) is built from Finlex open data batch downloads. The acquisition scripts and benchmark tooling are in the repository.