Identifying a car by its License Plate (LP) is a critical task in many applications, such as travel time estimation, vehicle re-identification, automatic toll collection, etc. Therefore, matching them must be as accurate as possible. This research proposes a novel deep neural network based method and its learning strategy for LP matching (LPM), which is originally from the cognitive-psychology-inspired unified objective function. The proposed method uses a deep Convolutional Neural Network (CNN) model to extract effective visual signature of the LP image. It exploits the multi-task learning approach to optimize the model, which combines two different tasks: (a) parallel letters recognition to transcribe the image-text contents and (b) image classification to classify the distinct LPs. Moreover, it takes profit from the use of image augmentation techniques. The proposed method is evaluated on three datasets of different characteristics. One of them was collected in this research and will be released publicly. The obtained results show that the proposed method performs better than the state-of-the art methods based on the commonly used evaluation metrics and computation time.