An Understanding of Vehicle Number Plate Detection Mechanism using Computer Vision

AI and computer vision is playing a major role in transportation evolution. Autonomous vehicle technology, fuel injection systems, self-driving cars, digital scanning, sensing and processing technologies, intelligent transportation systems, etc., are transforming many aspects of human life.

The role of transportation technologies is massive in making bustling cities livable, workable, manageable, safe, and sustainable. Such technologies can be used in different applications in urban areas for sustainable urban mobility. One of the vital application areas is vehicle number plate detection, which captures the information from the number plates via the video or photo footage of cameras and identifies vehicles.

In today’s post, I will discuss the novel system of vehicle number plate recognition, its use-case, significance, how it works, techniques, etc. Let’s unfold this further and see how computer vision can help recognize vehicle number plates seamlessly.

Defining the Vehicle Number Plate Detection 

Number plate detection (NPD) is one such innovative mechanism that identifies a specific part on the vehicle license plate and understands the characters using advanced technologies. It is one of the key functions of intelligent transportation systems. 

Known as Registration Plate Recognition (RPR), or License Plate Recognition (LPR), the latest system of Vehicle Number Plate Recognition (NPR) uses AI, ML, deep learning, and computer vision-powered approaches to read such license plates on vehicles without human interaction. 

This advanced system of number plate identification captures an image from a moving vehicle, searches for a number plate, and extracts relevant alphanumerics with the aid of the optical character recognition (OCR) mechanism. It translates the characters into readable digital text, which can be used for various applications such as preventing car theft, automated toll tax collection, traffic control, etc.

Authorities can install a number plate recognition mechanism on CCTV in many regions to identify vehicles and control other facets of inspection, traffic management applications, and safety. This automated system helps them track the actual entry/exit time of a vehicle and the amount of time it spends in a particular region.

Thus, the system can open a wide range of advantages to parking systems along with space management, governance, and vehicle traffic controls in urban areas. 

Popular Use Cases of the Automated Vehicle Number Plate Detection System Include the following:

• Effective traffic law enforcement
• Vehicle theft prevention
• Journey time analysis
• Automatic parking management (Smart Parking)
• Traffic management and traffic optimization
• Automated collection of tolls and toll booth records 
• To assist visitor management systems in recognizing guest vehicles
• Security monitoring and border crossings
• A perfect solution while dealing with the new normality distances and touchless sanitary regulations.

About Vehicle Registration and Number Plate Format

Every vehicle has its unique identity, which is represented by its license plate. Vehicle registration is mandatory as it is proof of vehicle ownership acknowledged by the government. The vehicle registration certificate becomes essential for any legal actions to be pursued upon the vehicle.

Talking about the vehicle number plate format, it comprises a numeric or alphanumeric code, which helps identify the vehicle. It has an official registration number, a numeric or alphanumeric character in different colors, fonts, and sizes depending on the country and its rules. 

For example, the current UK number plate format consists of two letters, two numbers, then a space, and further three letters (e.g. – XX20 XXX), introduced in 2001 for all new vehicles registered.

1. The first two letters are memory tags representing vehicle registration location, whereas the former represents the region and the latter represents a DVLA local office. 
2. The following two numbers are age identifiers, which show the year and six-month period in which the vehicle was registered for the first time. This age identifier changes every six months – March and September.  
3. And the last three letters followed by a space are random letters. 

The vehicle number plate format has changed several times and is pliable in the future as per country regulations. Also, different countries have their own set of representations for the number plate format. Using various algorithms of the number plate recognition system, authorities can extract precise vehicle information. Some of the useful algorithms are as discussed below:

1. Optical character recognition to identify characters
2. Plate localization to find the plate in image
3. Plate orientation and sizing to adjust the dimensions 
4. Character segmentation to find specific characters on the plates
5. Normalization to adjust the brightness as well as the contrast of an image
6. Syntactical analysis to check characters and positions against the country-specific rules

The software solves many critical problems such as blurry images, poor file resolution, different fonts, an object obscuring part of the plate, etc. Now, let us understand how number plate detection works.

How Does Vehicle Number Plate Recognition System Work?

The sole purpose of the automated system is to distinguish vehicles by recognizing number plates. This number plate recognition process is divided into three parts: Identification of vehicle number plate, Plate character recognition and segmentation, and OCR calculation to translate characters into encoded content. 

The automatic license plate recognition system consists of digital image capture units (like high-speed cameras with IR filters), application software (like video analytics software), processors capable of performing various object and character recognition, different algorithms to capture information from fast-moving vehicles, and an alert capability to notify operators.  The system works like this:

1. Moving vehicle number plate identification – It first detects the vehicles and then captures, normalizes, and enhances the image of vehicle number plates using a series of image manipulation techniques
2. Number plate character identification – Then OCR extracts accurate information such as the alphanumerics of the number plates.
3. Number plate character translation into encoded content – At last, the software verifies the sequence of those alphanumeric characters, converts them into text format, and stores the database.

Computer Vision-powered Approaches Used In Vehicle Number Plate Recognition

In urban areas, the real traffic environment is complicated. Usually, authorities can not recognize images from the license plate in such surroundings as the images are too vague or of lower quality, i.e., lower than 32X100 pixels and the rotation angle is beyond −10° to 10° or have uneven illumination that affects the recognition results.

Under these conditions, the main objective of the license plate recognition software is to – 

• Detect the License plate location from the vehicle object and 
• Extract text from the detected number plate. 

For seamless vehicle identification, it is mandatory to recognize vehicle number plates precisely. Incorporating innovative computer vision-powered approaches can help achieve the stated goals. Let’s understand the same in detail. 

1 Approaches To Detect the License Plate Location From the Vehicle Object

The foremost step in extracting vehicle identity is to detect the license plate from the vehicular image using machine learning models.

In order to detect the license plate from vehicle images, several machine learning models like deep learning architectures based upon convolutional neural networks can be utilized. This problem statement of object detection is indeed a fast-paced progressing research topic in modern AI. Here’s a list of considerable popular approaches:

1.1 Object detection using Haar-cascade

A Haar-cascade classifier is a machine learning object detection program that identifies objects from images and videos. Typically, it slides a square window over an image and, at each location, asks a machine learning algorithm –  “is this a number plate?”

1.2 Single Shot MultiBox Detector (SSD)

The SSD algorithm is used to detect objects and segment images, i.e., locate license plates in images using a single deep neural network. It is a technique to create a bounding box (an area) for each segment within an image, allowing exact object detection and segmentation. 

1.3 Histogram of Oriented Gradients (HOG)

HOG is widely used for object detection tasks and focuses on extracting features from images to obtain the object’s shape and structure. For the regions of the image, its feature descriptor generates histograms using the magnitude and orientations of the gradient.

1.4 Region-Based Convolutional Neural Networks (R-CNN) 

The key concept behind R-CNN  is region proposals which are used to localize the objects within an image-based upon their pixel intensities for better detection. The R-CNN consists of a family of approaches differing in the region proposal methods, speed, and performance –

• R-CNN
• Fast R-CNN
• Faster R-CNN
• Mask R-CNN

 1.5 You Only Look Once (YOLO)

YOLO architecture proposes the use of end-to-end neural networks to predict object class probabilities and detect the bounding boxes all at once. This approach potentially beats other real-time object detection techniques by a large margin. State of the art for YOLO has seen many versions and improvements with time, listing here different versions of YOLO proposed till date:

• YOLO
• YOLO v2
• YOLO v3
• YOLO v4
• YOLOR (You Only Learn One Representation)

As an outcome of the object detection (here object being the number plate), using any of the techniques as mentioned above, we would obtain the bounding box images of the license plates detected for each supplied vehicle image. Note that we can use pre-trained license plate recognition models or make a training model from real-time data using the above-mentioned algorithmic approaches. 

Some of the available datasets for Automatic License Plate Detection are: 

• Indian License Plates Dataset
• License Plate images Dataset of 500 vehicles captured from rear view
• Dataset of South America Mercosur license plates with images and labels
• Chinese City Parking Dataset

2 Text Extraction From The Detected Number Plate

Once the vehicle license plate is detected, the subsequent step is to infer the text in the license plate and segment the alphanumeric characters from it to know the vehicle identity. 

Text extraction is one of the most important processes for the automatic identification of license plates because if it fails, the vehicle recognition will not be correct. In order to achieve this objective, OCR can be utilized to decipher the text and employ natural language processing that recognizes, translates, and interprets the text effectively. 

The OCR algorithms can be based upon traditional image processing techniques or deep learning techniques. The conventional machine learning-based approaches include image pre-processing and cleansing to eliminate noise from the picture, followed by contour detection to detect text segmentation lines. 

Various machine learning-based algorithms can then be applied to extract the text from the segmented image. On the other hand, deep learning techniques provide superior results by combining computer vision-based approaches to natural language processing. OCR algorithms predict the bounding box from the extracted textual regions, and such approaches can be broadly categorized into two stages: 

2.1 Textual Region Proposal 

This is the first stage in OCR wherein the textual regions from the input license plate image are recognized by convolutional neural networks, and the detected regions are enclosed with the bounding boxes. This typically marks an object detection problem, and object detection models can be taken to advantage here.

2.2 Natural Language Processing 

The transformer-based language processing, Recurrent Neural Networks, or simply Convolution-based algorithms can be applied further to extract meaningful characters from the extracted text to make a proper interpretation. As an outcome of Optical Character Recognition, the actual license key number can be identified to serve the purpose of vehicle identification.

Market Dynamics of Vehicle Number Plate Detection

Massive growth in the automobile sector, advancements in the processing technologies, development of integrated unmanned aerial vehicles (UAVs), infrastructure growth in emerging economies, increasing allocation of government funds for intelligent transportation systems, etc., are expected to propel the vehicle number plate recognition market growth globally.

The adoption rate of the license number plate recognition mechanism has also increased to locate vehicles – in the defaulter’s list for toll collection purposes, have violated traffic laws, have expired insurance policies, or been stolen. 

The USA and other European countries are actively taking advantage of the computer vision-powered mechanism for security and traffic enforcement applications. The market size was USD 2.3 billion in 2020, which is expected to reach USD 3.8 billion by 2025, at a CAGR of 10.0% during the forecast period.

Apart from this, the license plate recognition system allows direct connectivity, identifies the full string, automates tasks, and inspects international plate syntax to define the country of origin. It also offers cost-effective, flexible, and scalable storage solutions. With automated systems like parking payment systems, booking systems, electronic toll collection, etc., the AI-enabled tool enhances the experience of end-users. This ultimately drives the growth of the license plate recognition system market. 

Conclusion

Vehicle number plate recognition has become a mature technology and is broadly used in various applications serving vehicle detection, localization, and recognition. This computer vision technology captures photographic surveillance and owes the capacity to transform the optical data from the images to identifiable digital information in real-time scenarios. Indeed, this technology provides an easy-to-understand, cost-effective, better, faster, touchless, and frictionless vehicular identification and parking service. 

From automating tasks to managing space, improving the mobility of people and goods, reducing traffic congestion, and managing incidents effectively, the vehicle number plate identification system offers numerous benefits. Get in touch with our experts to implement the vehicle number plate recognition system and enhance the urban mobility experience.

Author's Bio: