Cross domain meta-network for sketch face recognition

Because of the large modal difference between sketch image and optical image, and the problem that traditional deep learning methods are easy to overfit in the case of a small amount of training data, the Cross Domain Meta-Network for sketch face recognition method is proposed. This method first designs a meta-learning training strategy to solve the small sample problem, and then proposes entropy average loss and cross domain adaptive loss to reduce the modal difference between the sketch domain and the optical domain. The experimental results on UoM-SGFS and PRIP-VSGC sketch face data sets show that this method and other sketch face recognition methods.


Introduction
Sketch face recognition [1] is the process of matching face photos with sketch images. Because of its important role in criminal investigations, it has received great attention in recent years [2].
To solve the problem of modal differences, the algorithms currently designed for sketch face recognition can be divided into traditional sketch face recognition methods and deep learning-based methods. The traditional sketch face recognition methods can be roughly divided into three categories [3]: feature-based methods, the method based on synthesis and the method based on common subspace projection. Feature-based methods aim to represent face images with local feature descriptors. However, this type of method usually results in the loss of details in the face image. The synthesis-based method solves the problem of face photos (photos) by converting photos (sketches) into sketches (photos), and then using the facial recognizer to match the synthesized sketches (photos) with the original sketches (photos). However sketching face synthesis [4] is more challenging. The method based on subspace projection aims to project facial images of different modalities into a common subspace to reduce the influence of modal differences on recognition performance. However, this type of method may lose important information in the original image.
To solve the above problems, a Cross Domain Meta-Network for sketch face recognition is proposed here. In addition, the entropy mean loss and cross domain adaptive loss are proposed to reduce the modal difference between the sketch domain and the optical domain and improve the recognition ability of the model.

Cross domain meta-network
In this section, the cross domain adaptive entropy element network is introduced in detail. Figure 1 shows the training process of a single training episode and a single test episode.

Training episode
Testing episode The training process of a single training episode and a single test episode. First, the feature extractor extracts the features in the training episode, and then learns the metric relationship between the two in the embedding space, and finally classifies the image through the meta-classifier and calculates the entropy mean loss.

Mean entropy loss
Randomly sample K<N classes from the training set The mean entropy loss is obtained:

Cross domain adaptive loss
Aiming at the data offset between the training data and the test data, a co-modal adversarial loss is proposed:

Implementation details
This paper uses UoM-SGFS and PRIP-VSGC datasets to evaluate the effectiveness of this method. This article sets up two data sets based on the above two data sets. In the first data set (S1), 450 pairs of sketch-photos as the training set, and 150 pairs of sketch-photos are used as the test set. In the second data set (S2), the data set setting is the same as S1. This paper uses the MTCNN method to locate and align the images. Before the training process, all images are cropped to a standard size of 256×256. In the training process, the feature extractor uses the resnet-18 network pre-trained by the ImageNet dataset. The Adam optimizer with parameters 1 0.5 β = and 2 0.999 β = are used to optimize the entire network model, and the initial learning rate is set to 0.0001.

Comparative experiments
In order to prove the effectiveness of the method in this paper, we compare with other advanced sketch face recognition methods, including LGMS [5], D-RS [6], D-RS+CBR [7], DEEPS [8], JAN [9] and CDAN [10]. The Table 1 and Fig. 2~3 show the experimental results of these methods and the method in this paper. It can be found that the method in this paper is better than these methods.  Aiming at the problem of the small amount of existing sketch face data and the large modal gap between sketch image and optical image, this paper proposes a sketch face recognition method based on a Cross Domain Meta-Network. This method trains the network by designing a meta-learning method based on the distance measurement and improves the learning level from data to the task, thereby improving the generalization ability of the network. In addition, by introducing average loss and cross domain adaptive loss into the cross domain adaptive entropy element network, the modal difference between the sketch domain and the optical domain is reduced. Experiments on the UoM-SGFS database and PRIP-VSGC database show that this method can effectively improve the effect of sketch face recognition.