Seven datasets account for nearly every number in scene text detection papers. Each was built to expose a different weakness, so a detector’s profile across them says more than any single score.
| Dataset | Year | Images (train / test) | Unit and shape | Script | Made to test |
|---|---|---|---|---|---|
| ICDAR 2013 | 2013 | 229 / 233 | word, axis-aligned box | English | clean horizontal text |
| ICDAR 2015 | 2015 | 1,000 / 500 | word, quadrilateral | English | small, blurred, rotated incidental text |
| Total-Text | 2017 | 1,255 / 300 | word, polygon | English | curved text |
| SCUT-CTW1500 | 2017 | 1,000 / 500 | line, 14-point polygon | English, Chinese | long curved lines |
| COCO-Text | 2016 | 63,686 (COCO images) | word, axis-aligned box | mostly English | incidental text at scale |
| SynthText | 2016 | ~800,000 synthetic | word and character boxes | English | pretraining, not evaluation |
| ICDAR MLT 2019 | 2019 | 10,000 / 10,000 | word, quadrilateral | 10 languages | multilingual detection |
Not in the table but common in papers: MSRA-TD500 (2012; 300 train / 200 test, line-level rotated boxes, English and Chinese, the set DBNet’s abstract reports on) and the ICDAR 2017 RCTW Chinese set.
Protocols
- DetEval (ICDAR 2013 competition) accepts one-to-many and many-to-one matches with area thresholds, so a detector that splits a word into two boxes can still score.
- ICDAR 2015 protocol requires one-to-one matching at IoU ≥ 0.5 and ignores “###” don’t-care regions. Most papers after 2015 use this on every dataset, including polygon IoU for Total-Text and CTW1500.
- TIoU (tightness-aware IoU, 2019) penalises loose and truncated boxes; a few papers report it alongside the standard metric.
Two results on the same dataset under different protocols can differ by several points. Tables that do not name the protocol are the first thing to distrust.
How training usually goes
- Pretrain on SynthText (or SynthText-MLT for multilingual work) for a few epochs.
- Fine-tune on the target benchmark’s training split, often with the training splits of ICDAR 2015, ICDAR 2017 MLT and Total-Text pooled together.
- Test single-scale at a fixed long side (1,280 to 2,240 px depending on the dataset) unless the paper says multi-scale.
Step 2 is where comparability goes wrong: “trained on ICDAR 2015” and “trained on ICDAR 2015 + MLT” produce different numbers for the same architecture.
Getting the files
The ICDAR sets require an account on the Robust Reading Competition site (rrc.cvc.uab.es); Total-Text and CTW1500 are on GitHub; COCO-Text v2 is on its project page; SynthText is a 41 GB archive from the generator’s repository. Links and citations are on each dataset page.
Pitfalls we have seen
- Training on ICDAR 2011 and testing on ICDAR 2013 (they overlap).
- Reporting Total-Text results on the pre-correction annotations without saying so.
- Comparing word-level detectors on CTW1500 (line-level) without merging.
- Quoting COCO-Text numbers without the version.
- Using SynthText-pretrained weights as if they were a fair “no real data” baseline for curved text.