License Plate Recognition for Parking Control System by Mathematical Morphology

Javier Ortiz and Alberto Gómez

1.Introduction

Number plate recognition is playing an important role in the image processing field.The fundamental issue in number recognition is the shape analysis.Many methods on the shape analysis have been seen in the past ten years,such as methods using morphological functions[1],methods based on gradient propagation[2],and methods by comparing weighted shape graphs[3].Many other techniques,such as the Fourier description,template matching[4],mathematical morphology,and neural network[5],are also used for the shape analysis.All these methods require a precise angle of image acquisition.The algorithm proposed in this paper is designed to recognize the license plates of vehicles in real time by means of mathematical morphology and specifically,using skeletons which are added in a database and not angle sensitive.

This paper is organized as follows.Section 2 presents the processes of plate localization and extraction.Section 3 deals with the character recognition based on skeletons of mathematical morphology and a pattern-learning algorithm.Section 4 discuses the experimental results.Section 5 concludes this work.Paper ends with acknowledgments and references.

2.License Plate Extraction

The first stage of the process is to locate the license plate area in an input photo.In order to do so,we need to emphasize regions of high spatial frequency that correspond to edges(image enhancement).In the European Union,the characters of license plates are black on a white or yellow background,as shown in Fig.1.

We apply the same technique proposed by Karim et al.in [5]to locate the plate area,using a new morphological enhancement,which consists in a top-hat contrast operation[6]applied to the grayscale image of the car.

The white top-hat(WTH)is added to the original image to enhance bright objects and the black top-hat(BTH)is subtracted from the resulting image to enhance dark objects.We denote this top-hat operator by kTH:

where f is the original image,and φ and γ represent the closing and opening operators,respectively.

The next in the process is the binarization and an example result is shown in Fig.2(b).Finally,we use a morphological operation of area opening to select the plate area(by sizes)and delete the rest of false positives(lights,car logos,etc.).The resulting image of area opening will be the marker that identifies the plate license area or region of interest(ROI),as shown in Fig.2(c).This operation includes removing all connected components whose area in number of pixels is lower than a threshold value λ:

where λ is the specified area and Xi,and i∈I denotes the connected components of X(the binarized image).

Fig.1.Example of license plates in the European Union:(a)Italy,(b)Spain,(c)UK,(d)Netherlands,(e)Portugal,and(f)France.

Fig.2.Plate selection in Spanish vehicle:(a)original image,(b)binarized image of top-hat contrast,(c)marker image for processing(ROI),area opening of license plate,and(d)license plate after erasing anything that does not correspond with characters.

Fig.3.Homotopic marking of license plate:morphological skeleton of characters.

Once we have identified the ROI,the next step is to identify numbers and letters in the license plate[7]-[9].We use color information to eliminate the areas with at least 50% of high saturation pixels.This area corresponds to the blue European Union logo.Next,the morphological operation of opening allows us to eliminate the small spots between the numbers of the plate[10].In Fig.2(d),we can see the result of this stage of the algorithm.

3.Character Segmentation and Recognition

The license plate is segmented to obtain the characters individually.Number recognition is based on the morphological hit/miss transformation of skeletons(medial axis transform).We use the mathematical procedure called homotopic marking,which means reducing an image to the smallest possible information while preserving the homotopy of the input image[7].We avoid the typical mistakes in classic pattern recognition[11],[12],because our skeletons are not angle sensitive.Small variations of these skeletons will be dynamically added to a database.

Usually,a homotopic marking does not preserve enough information about the shape of the input pattern.A solution to this problem is to predefine a set of pixels that should belong to the skeleton.By definition,these pixels are non-deletable and cannot be thinned when performing homotopic thinnings until idempotence.As a result the obtained skeleton is anchored to these predefined pixels.They are therefore called “anchor points”.A classical set of anchor points consists of maximal discs centers or equivalently the skeleton by opening.When processing a binary set,the anchor points should be selected in accordance with the connectivity used for the foreground pixel.The skeleton gets a simpler structure by using the ultimate eroded set or,equivalently,the regional maxima of the distance as the anchor points[6]:

where B is the elementary square 8-connected(3 pixels×3 pixels); λ,B is a square of pixels(λ=2)5 pixels×5 pixels,(λ=3)7 pixels×7 pixels,and(λ=4)9 pixels×9 pixels; ελrepresents the operation of erosion; “” means the relative complement.

Note that(3)corresponds to the union of the WTH of successive erosions of the original set.Fig.3 shows the morphological skeleton(medial axis)of plate characters.The identification of critical points to shape representation,and the identification of further points necessary to preserve homotopy are guaranteed.

The final step of the algorithm is template matching.In our case,data mining uses pattern recognition.The skeleton image is compared with the ones in the database and the best similarity is measured.To measure the similarity and find the best match,a statistical method,correlation,is used[13].The correlation is an effective technique for image recognition,which was developed by Horowitz[14].

The capability of this operator in detecting patterns with specific geometric properties has reduced the algorithm complexity.Since there are two signals:the unknown input skeleton and known database skeleton in this system,the particular and efficient cross-correlation is used.Let f1(j,k)and f2(j,k)be two discrete skeletons denoting the skeleton to be searched and the template,respectively.The normalized cross-correlation used in our algorithm is defined as:for m=1,2,··,M and n=1,2,··,N,where M and N are the size(height×width)of skeletons.In our case,the database works with 33 sets of alphanumeric skeletons of characters(23 alphabetic and 10 numeric)with the size of M=36 pixels and N=18 pixels.These sets contain hundreds of pattern variations of valid skeletons characters.

4.Experimental Results of the Algorithm

Experiments have been performed to test the proposed system,which was designed in software Matlab 7(see Fig.4).The system received 125 “red,green,blue”(RGB)images sized 1400 pixels ×1200 pixels as the input.These images were acquired from a car parking company in Alicante(Spain),at different lightning conditions and different angle positions of cars.As shown in Fig.5,it is an Alicante car parking entrance with license plate control.The parking locates in a tourist area,with a high percentage of foreigners cars.The test lasted for two months.

Table 1 gives the experiment results,which reports a high accuracy rate about 95% for extraction plates and 91%for the correct identification of the plate numbers.This percentage is the same for the different European license plates.Our plate recognition achieves a complexity reduction in comparison with other methods.In order to speed up morphological operations,the logarithmic decomposition of structuring elements is used[15].Experiments indicate that our new algorithms are more than 20 times faster for pixel operations.Moreover,they also perform basic morphological transformations faster than other known software.

Fig.4.License plate recognition system.

Fig.5.Alicante car parking entrance with license plate control.

Table 1:Experimental results of our algorithm.

5.Conclusions

This paper describes an algorithm that realizes the extraction of license plate numbers of cars by means of mathematical morphological.First of all,we extracted the plate location.Next,the plate characters were separated individually.Then,we obtained the character skeletons,and finally,we applied a template matching of character skeletons for the recognition of license plate numbers by machine learning.The main advantage of our algorithm is the high accuracy of the technique that works independently of the color,size,location,and angle of the number plates.

Now,we work in the use of fuzzy morphological operators to make our approach more powerful and to reduce the processing time of the algorithm.And our system can be easily updated for multinational car license plates.

Acknowledgment

We would like to thank the University of Alicante project PPC–1928273/A and the parking company GrupoCivica-Parkings in Alicante(Spain)for giving technical support to do this work.

[1]D.Sinha and C.Giardina,“Discrete black and white object recognition via morphological functions,” IEEE Trans.on Patterns Analysis and Machine Intelligence,vol.12,no.3,pp.275-293,1990.

[2]J.Ben-Arie,K.Rao,and Z.Wang,“Robust shape description and recognition by gradient propagation,” in Proc.of Int.Conf.on Image Processing,Washington,DC,1995,pp.368-371.

[3]K.Kupeev and H.Wolfson,“A new method of estimating shape similarity,” Pattern Recognition Letter,vol.17,no.8,pp.873-887,1996.

[4]M.Aksoy,G.Cagil,and A.Turker,“Number-plate recognition using inductive learning,” Robotics and Autonomous Systems,vol.33,no.2-3,pp.149-153,2000.

[5]H.Karim and Y.Zhang,“Vehicle number plate recognition using mathematical morphological and neural networks,”WSEAS Trans.on Computers,vol.10,no.7,pp.781-790,2008.

[6]S.Lee,“Off-line recognition of totally unconstrained handwritten numerals using multiple cluster neural network,” IEEE Trans.of Pattern Analysis and MachineIntelligence,vol.18,no.6,pp.648-652,1996.

[7]P.Soille,Morphological Image Analysis,Principles and Applications,Berlin:Springer-Verlag,2003,ch 3.

[8]J.Serra,Image Analysis and Mathematical Morphology,London:Academic Press,1988,pp.21-34.

[9]J.Serra,Image Analysis and Mathematical Morphology,vol II:Theorical Advances,London:Academic Press,1988,pp.23-45.

[10]F.Ortiz,A.Naranjo,and A.Gómez,“Authomatic vehicle number recognition by mathematical morphology and data mining,” in Proc.of Int.Conf.on Machine Learning and Data Mining,New York,2011,pp.77-81.

[11]H.Choi,“A study on the extraction and recognition of a car number plate by image processing,” Journal of the Korea Institute of Telematics and Electronics,vol.24,no.2,pp.309-312,1987.

[12]H.Sulehria and Y.Zhang,“Extraction of vehicle number plates using mathematical morphological techniques,” in Proc.of the 8th WSEAS Int.Conf.on Automation and Information,Cairo,2007,pp.258-261.

[13]H.Kim,“Recognition of a car number plate by a neural networ,” in Proc.of the Korea Information Science Society Fall Conf.,Seoul,1991,pp.259-262.

[14]L.Zheng and X.He,“Number plate recognition based on support vector machines,” presented at IEEE Int.Conf.on Video and Signal Based Surveillance,Washington,DC,November 22-24,2006.

[15]R.Boomgaard and R.Balen,“Methods for fast morphological image transforms using bitmapped binay images,” Graphical Models and Image Processing,vol.54,no.3,pp.252-258,1992.