Improved Detection and Interpretation of Multilingual Signboards in Natural Scene for Visually Impaired People

Abstract

Visually impaired individuals face numerous challenges in localization, navigation, reading content, and communicating with others. Unlike sighted individuals who rely on vision in the physical environment to reach their destinations, visually impaired individuals require assistive technologies and smartphone-based applications that employ computer vision to overcome the issues of obstacle detection, read signboards, and navigate in natural scenes. Signboards with text information serve as crucial indicators for navigation and recognition in daily life. However, detecting and interpreting signboard text in natural scenes poses difficulties for visually impaired individuals. Researchers have made efforts to address issues such as text detection on signboards, reducing blurriness, removing backgrounds, improving lighting conditions, and enhancing accuracy and response time of recognition. The primary challenges lie in the detection and interpretation of multilingual and diverse content on signboards in real-time. The proposed system presents a methodology for real-time detection and interpretation of diverse content on signboards, including text in English and Urdu along with phone numbers, for visually impaired individuals. An Android-based application is developed to improve text detection and recognition methods for multilingual texts, providing vital navigation information to visually impaired individuals in real-time. The methodology and application are evaluated, and the results demonstrate promising outcomes.