Text detection and recognition in natural scene images

Abstract

Text detection and recognition in natural scene images has applications in computer vision systems such as license plate detection, automatic street sign translation, image retrieval and help for visually impaired people. Scene text, however, may have complex background, image blur, partially occluded text, variations in font-styles, image noise and varying illumination. Hence scene text recognition is a challenging computer vision problem. This work addresses the problem of dictionary driven end-to-end scene text recognition, which is divided into a text detection problem and a text recognition problem. For text detection an AdaBoost sliding window classifier is used to detect text in multiple scales. The effectiveness of several feature-sets for this classifier are compared and evaluated. A modified Local Ternary Pattern (LTP) feature-set is found as most effective for text detection. In a post-processing stage Maximally Stable Extremal Regions (MSER) are detected and labeled as text or non-text. Text regions are grouped to textlines. Textlines are split into words by a word-splitting method build upon k-means and linear Support Vector Machines (SVM). For text recognition a deep Convolutional Neural Network (CNN) trained with backpropagation is used as one-dimensional sliding window classifier. To avoid overfitting the network is regularized by Dropout. Recognition-responses are used in a Viterbi-style algorithm to find the most plausible word in a dictionary. The influence of training set size and size of convolutional layers is evaluated. The system presented outperforms state of the art methods on the ICDAR 2003 and 2011 dataset in the text-detection (F-score: 74.2% / 76.7%), dictionary-driven cropped-word recognition (F-score: 87.1% / 87.1%) and dictionary-driven end-to-end recognition (F-score: 72.6% / 72.3%) tasks.