Multi-class Vehicle Detection and Automatic License Plate Recognition Based on YOLO in Latin American Context

Abstract

In Latin America, and many other countries around the globe, serious problems exist regarding the high level of traffic that generates congestion on avenues and streets, with poor road planning being one of the main causes, plus the excess of buses, mini-buses, taxis, and other vehicles that cause obstructions. Therefore, it would be very useful to know the flow of existing vehicles in each area to know and segment which roads certain vehicles should transit, thus generating greater control. This research proposes a methodology for the detection and multi-classification of vehicles in eight classes: cars, buses, trucks, combis (micro-buses), moto-taxis (auto-rickshaws), taxis, motorcycles, and bicycles; to later carry out the detection of the vehicle license plates and do the recognition of the characters on them; using Deep Learning techniques, specifically YOLOv3 and LeNet. The proposed methodology consists of four stages: Vehicle Detection, License Plate Detection, Character Segmentation, and Character Recognition. We also introduce a novel open-access dataset, LAT-VEDA, which contains more than 22 000 images divided into 8 classes. Good results were obtained in each one of the four stages of the system in comparison with the state of the art. Achieving the best mAP of 1.0 in the Vehicle License Plate Detection stage and having the lowest performance in the Vehicle Detection stage with a mAP of 0.68. This approach may be used by the Government to support the management of public transport, giving greater control and information about the flow of vehicles by area, in addition to the fact that the license plate recognition system can help in the management of the control of public policies and regulations. © 2021, Springer Nature Switzerland AG.