Methodology v1 · July 17, 2026
“The PDF looks right” is not a reliability test.
A production PDF workflow must make the source values, field positions, missing records, layout changes, and repeat behavior inspectable. This page publishes the checklist and public fixture we use to evaluate that workflow — and the correction rates our own detector produced on real traffic, including the caveats that make those numbers look better than they are.
Scope: this is a reproducible product fixture, not an independent multi-vendor benchmark and not a claim of universal accuracy. It intentionally avoids fabricated success rates. Use the files below to inspect the workflow or repeat the test yourself.
Acceptance criteria
Five things every claim should let you verify.
A public five-row CSV is downloadable.
A result cannot be audited if the expected values are hidden.
The public fixture uses the original W-9 PDF.
Rebuilding a form can change layout, fonts, pagination, or legal text.
Detected fields and spreadsheet mapping are shown before export.
A plausible-looking value in the wrong box is still a failed fill.
One selected spreadsheet row produces one PDF in the ZIP.
Batch completeness matters as much as the accuracy of one document.
Reviewed field layouts can be saved as exact templates.
A production workflow should not ask a model to rediscover stable geometry every time.
Measured, not claimed
What real users had to correct.
Benchmarks are easy to publish and hard to trust. The number that actually describes a detector is how much work it leaves behind, so we record every field a user deletes, adds, moves or resizes before they download — and publish it, including the parts that look bad.
of detected fields were deleted
125 of 5,674 — approximates false positives
were added by hand
67 fields — approximates missed fields
were moved or resized
200 fields — approximates placement error
documents needed no correction at all
Every field kept exactly as detected
- The sample is small. 43 documents and 5,674 fields, not a benchmark suite.
- It only counts people who finished. Corrections are recorded at download, so anyone who gave up before exporting is missing from this sample — which flatters the result.
- A deletion is not proof of an error. Users also delete fields they simply do not need, and they cannot correct a mistake they never noticed.
- Every document here was flat or scanned — the hard case. PDFs with embedded form fields are read directly and never reach the detector.
The model behind those results
- mAP@50 (golden set, 1093 images)
- 86.7
- mAP@75
- 77.1
- mAR
- 83.6
- Detection time per page (CPU)
- 10–22s
These are the metrics of the checkpoint currently serving production, not a best-ever run. Measured 10 August 2026.
How to repeat the test
Use the same acceptance criteria with any tool. Record observable errors rather than compressing everything into a single vague “accuracy” number.
- 1
Download the blank PDF and source CSV before opening the product.
- 2
Write down the expected value and target field for every populated cell.
- 3
Run detection and preserve screenshots of the field boxes and mapping.
- 4
Export all five outputs and check missing, duplicate, misplaced, clipped, and altered values.
- 5
Repeat one record using a saved template and verify that field geometry is reused rather than rediscovered.
Discovery needs review
ML field detection is probabilistic. Confidence and previews must remain visible.
Repetition should be exact
After review, stable forms should use stored geometry instead of repeated interpretation.
Business meaning stays human
Correct placement does not validate tax, legal, medical, or operational decisions.
Use the fixture, not our adjectives.
The input files and workflow recording are public.