CRAFT (Baek et al., CVPR 2019) looks at text one character at a time. The network outputs two heat maps at half resolution: a region score that peaks at the centre of each character, and an affinity score that peaks between characters that belong to the same word. Thresholding the two maps and taking connected components gives word instances; because the grouping happens at character level, the resulting polygon follows curved and deformed text naturally.
The training trick
Real scene-text datasets are labelled at word level. CRAFT is trained on SynthText (which has character boxes) and then on real data with pseudo character labels: an interim model splits each word crop into characters, and a confidence score derived from how well the split matches the known character count weights the loss. This weakly supervised loop is why CRAFT could use character-level supervision without anyone annotating characters by hand.
Where it fits
- Strong on curved text (Total-Text, CTW1500) years before that was common.
- The polygons are word-level; a separate step groups words into lines when you need lines.
- Slower than the segmentation detectors that followed (DBNet, FAST) because of the two-map post-processing at half resolution.
- It is the default detector in EasyOCR, so many people run CRAFT without knowing it. See the Python guide.
Citation
@inproceedings{baek2019craft,
title = {Character Region Awareness for Text Detection},
author = {Baek, Youngmin and Lee, Bado and Han, Dongyoon and Yun, Sangdoo and Lee, Hwalsuk},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2019}
}