Detection: precision, recall, F
A detector is scored like an object detector. Each predicted region is compared with the ground-truth regions; a prediction matches a ground-truth region when their intersection over union (IoU) reaches a threshold, almost always 0.5. Then:
- Precision = matched predictions ÷ all predictions. Low precision means boxes on things that are not text.
- Recall = matched ground truth ÷ all ground truth. Low recall means missed text.
- F-measure (the ICDAR competitions say H-mean) is their harmonic mean. This is the number papers report.
Four details decide whether two F-measures are comparable:
- Matching rule. One-to-one (ICDAR 2015 and later) or DetEval's one-to-many and many-to-one (ICDAR 2013). Under DetEval a word split into two boxes can still count.
- Region shape. Axis-aligned boxes, quadrilaterals or polygons. Polygon IoU on curved text punishes box detectors regardless of how well they "saw" the text.
- Don't-care regions. Illegible text marked
###is ignored when matched and ignored when missed, but how a paper treats a prediction that half-overlaps one varies. - Test resolution and scales. Single-scale at a fixed long side is the norm; multi-scale testing adds points and is not always disclosed.
Tightness-aware IoU (TIoU, 2019) additionally penalises boxes that are loose or cut text off; a few papers report it next to the standard metric.
End-to-end spotting
When a system detects and reads, the ICDAR protocol counts a detection as correct only if the transcription also matches (usually after case-folding and with a small vocabulary or none). The "strong / weak / generic" lexicon settings of ICDAR 2015 describe how much help the recognizer gets: a 100-word list per image, a 1,000-word list, or a 90k-word dictionary.
Recognition: CER and WER
Recognizers are scored with edit distance. Character error rate is the number of insertions, deletions and substitutions needed to turn the output into the ground truth, divided by the ground-truth length; word error rate is the same over words. "Accuracy" without a definition usually means one minus one of these, or the fraction of words read exactly right, and the three numbers can differ by a factor of two on the same output. Ask which.
The 2026 benchmark landscape
Scene text detection still uses the datasets on the datasets page. Document OCR, where most commercial interest now sits, has moved to whole-page benchmarks that score structure as well as characters:
- OCRBench v2: a broad test of vision-language models on text-centric tasks (recognition, key information extraction, reasoning), with a leaderboard refreshed periodically, most recently March 2026.
- OmniDocBench (CVPR 2025): document parsing across page types, scored with TEDS for tables and CDM for formulas plus text edit distance; version 1.7 (April 2026) added new leaderboards and skills-based evaluation.
- olmOCR-bench: unit-test-style checks on real PDFs across eight categories (old scans, tiny text, handwriting, arXiv math, multi-column layouts and more).
- IDP Leaderboard: aggregates OlmOCR, OmniDocBench and its own IDP Core (key information extraction, OCR, tables, visual question answering).
- OCRBench leaderboard on Hugging Face: the original OCRBench scores for multimodal models.
How to read a vendor benchmark
- Is the test set public, and could the vendor have trained on it?
- Which metric, exactly? "Accuracy" is not a metric until it is defined.
- Were competitors run with their recommended settings and current versions?
- Does the document mix look like yours? Receipts, scans and street scenes are different problems.
- Is speed measured on the same hardware, including preprocessing and post-processing?
Numbers on this site follow the same rules: every result names its dataset, metric and source, and unverified table values are marked until someone checks the PDF. See the models table.