SlideShare a Scribd company logo

Face Recognition Methods based on Convolutional Neural Networks

•
•
Elaheh Rashedi
Elaheh Rashedi

Convolutional neural networks (CNN) have improved the state of the art in many applications, especially the face recognition area. In this work, we present a review on latest face verification techniques based on Convolutional Neural Networks. In addition, we give a comparison on these techniques regarding their architecture, depth level, number of parameters in the network, and the obtained accuracy in identification and/or verification. Furthermore, as the availability of large-scale training dataset has significantly affected the performance of CNN based recognition methods, we present a preface to the most common large-scale face datasets, and then we describe some of the successful automatic data collection procedures.

Related slideshows

Facial emotion detection on babies' emotional face using Deep Learning.
Facial emotion detection on babies' emotional face using Deep Learning.Facial emotion detection on babies' emotional face using Deep Learning.
Facial emotion detection on babies' emotional face using Deep Learning.
 
Face Detection and Recognition System
Face Detection and Recognition SystemFace Detection and Recognition System
Face Detection and Recognition System
 
Attendance system based on face recognition using python by Raihan Sikdar
Attendance system based on face recognition using python by Raihan SikdarAttendance system based on face recognition using python by Raihan Sikdar
Attendance system based on face recognition using python by Raihan Sikdar
 
1 of 24
Download to read offline
Face Recognition Methods based on Convolutional Neural Networks Presenter: Elaheh Rashedi Advisor: Prof. Xuewen Chen Winter 2016 1
Contents ■ Background – Neural network – Convolutional neural network – General CNN-based face recognition schema ■ Face recognition models based on CNN – DeepFace model – Web-scaled DeepFace model – DeepID model series – FaceNet model – VGG model – Lightened CNN Model ■ CNN training and testing dataset – CASIA-WebFace, MegaFace, IJB-A, VGG, … ■ Summary ■ Future Work 2
Neural Network Fig 1. (a) General structure of a Neural Network, (b) an example of a neuron 3 (a) (b)
Convolutional Neural Networks ■ CNN – A kind of neural network where the input is image – Contains less fully connectivity between neurons ■ CNN layers – Input layer – Convolutional layer – Pooling layer – Fully connected layer – Loss layer 4
Convolutional Neural Networks (cont. …) Fig 2. General CNN structure in face recognition problems 5
General CNN-based face recognition schema ■ Common steps: – Face detection  Viola-Jones, Cascade CNN, … – Pre-processing  Geometric & lighting normalization – CNN training  Supervised vs. unsupervised – Face identification  Classification problem – Metric learning  Joint-Bayesian, Cosine similarity, Triplet Similarity, Energy-based similarity, … – Face Verification ■ There are various ways to perform each step! 6
7 Fig3.Generalfacerecognitionpipeline
Face recognition models based on CNN ■ CNN based models are different – Architecture of CNN – Depth of neural network – Number of parameters – Scale of training dataset – Similarity metric – Alignment vs. non-alignment pre-processing – … 8
DeepFace Model ■ First CNN-based face recognition method (2014) – By Facebook research group ■ Includes 4 main steps – Detection – 3D Alignment – Feature representation – Classification ■ Similarity metric learning – Siamese energy based neural network 9
DeepFace Model (cont. …) 10 Fig 4. DeepFace [1] Fig 5. Siamese network [2]
Web-scaled DeepFace Model ■ Based on DeepFace (2015) – Higher identification acc. – Lower verification acc. – Lower feature vector dimension ■ Applies bootstrapping on large training dataset – Select harder recognition cases – Ignore easy recognition cases ■ Claim – High dimensional feature vectors do not necessarily result in better accuracy! 11
DeepID model series ■ Inspired by DeepFace ■ Model Series – DeepID (2014) – DeepID2 (2014) – DeepID2+ (2015) – DeepID3 (2015) 12
DeepID model ■ CNN structure – 4 convolutional layers – 3 max-pooling layers – 1 fully connected layer ■ Alignment – Center of two eyes, two corners of mouth, nose tip. ■ Multiple patches – Extract different features for different part of face ■ Similarity metric learning – Joint-Bayesian 13
DeepID2 Model 14 Fig 6. DeepID2 [3] Fig 7. DeepID [4]
DeepID2+ Model ■ Based on DeepID2 – More deep network – Uses supervisory signals – Uses fully connected layers ■ Fully connected layers – Early feature extraction ■ Claim – Deep CNN based networks are more robust to corruption of image! 15Fig 8. DeepID2+ [5]
DeepID3 Model ■ DeepID3 – More Deep than DeepID2+ – Less deep than FaceNet & VGG ■ Includes – Very deep neural networks – 15 feature extraction layers – Early fully connected layers ■ Similarity metric learning – Joint-Bayesian 16Fig 9. DeepID3 [6]
FaceNet Model ■ FaceNet model: by Google research group – Same framework for identification and verification – Very deep network – No alignment – Efficient representation of features ■ Similarity metric learning – Triplet loss 17Fig 10. Triplet loss learning [7]
VGG Model ■ VGG model: by Visual Geometry Group – Inspired by the very deep FaceNet network – Very deep CNN – 36 level of feature extraction ■ Similarity metric – Triplet loss ■ Contributions – Automatic collection of large face dataset – Publically available pre-trained CNN model 18
Lightened CNN Model ■ Shallow network – 4 convolutional layers – 4M parameters ■ Less computational intensive – 9 times less than VGG model 19
CNN Training and Testing Datasets 20
Summary ■ The size of training dataset ■ Alignment ■ Joint-Bayesian similarity ■ Multiple patches ■ Verification performance on video ■ Identification performance on larger number of identities 21
Future Work ■ Using face verification to outperform face detection ■ Designing a similarity metric learning ■ Designing a face tracking method based on face verification 22
References 1. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014. 2. S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, pp. 539–546, 2005. 3. Y. Sun, Y. Chen, X. Wang, and X. Tang, “Deep learning face representation by joint identification verification,” in Advances in Neural Information Processing Systems, pp. 1988–1996, 2014. 4. Y. Sun, X. Wang, and X. Tang, “Deep learning face representation from predicting 10,000 classes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1891–1898, 2014. 5. Y. Sun, D. Liang, X.Wang, and X. Tang, “Deepid3: Face recognition with very deep neural networks,” arXiv preprint arXiv:1502.00873, 2015. 6. Y. Sun, X.Wang, and X. Tang, “Deeply learned face representations are sparse, selective, and robust,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2892–2900, 2015. 7. F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. 23
Thank You! 24

More Related Content

Face Recognition Methods based on Convolutional Neural Networks

  • 1. Face Recognition Methods based on Convolutional Neural Networks Presenter: Elaheh Rashedi Advisor: Prof. Xuewen Chen Winter 2016 1
  • 2. Contents ■ Background – Neural network – Convolutional neural network – General CNN-based face recognition schema ■ Face recognition models based on CNN – DeepFace model – Web-scaled DeepFace model – DeepID model series – FaceNet model – VGG model – Lightened CNN Model ■ CNN training and testing dataset – CASIA-WebFace, MegaFace, IJB-A, VGG, … ■ Summary ■ Future Work 2
  • 3. Neural Network Fig 1. (a) General structure of a Neural Network, (b) an example of a neuron 3 (a) (b)
  • 4. Convolutional Neural Networks ■ CNN – A kind of neural network where the input is image – Contains less fully connectivity between neurons ■ CNN layers – Input layer – Convolutional layer – Pooling layer – Fully connected layer – Loss layer 4
  • 5. Convolutional Neural Networks (cont. …) Fig 2. General CNN structure in face recognition problems 5
  • 6. General CNN-based face recognition schema ■ Common steps: – Face detection  Viola-Jones, Cascade CNN, … – Pre-processing  Geometric & lighting normalization – CNN training  Supervised vs. unsupervised – Face identification  Classification problem – Metric learning  Joint-Bayesian, Cosine similarity, Triplet Similarity, Energy-based similarity, … – Face Verification ■ There are various ways to perform each step! 6
  • 8. Face recognition models based on CNN ■ CNN based models are different – Architecture of CNN – Depth of neural network – Number of parameters – Scale of training dataset – Similarity metric – Alignment vs. non-alignment pre-processing – … 8
  • 9. DeepFace Model ■ First CNN-based face recognition method (2014) – By Facebook research group ■ Includes 4 main steps – Detection – 3D Alignment – Feature representation – Classification ■ Similarity metric learning – Siamese energy based neural network 9
  • 10. DeepFace Model (cont. …) 10 Fig 4. DeepFace [1] Fig 5. Siamese network [2]
  • 11. Web-scaled DeepFace Model ■ Based on DeepFace (2015) – Higher identification acc. – Lower verification acc. – Lower feature vector dimension ■ Applies bootstrapping on large training dataset – Select harder recognition cases – Ignore easy recognition cases ■ Claim – High dimensional feature vectors do not necessarily result in better accuracy! 11
  • 12. DeepID model series ■ Inspired by DeepFace ■ Model Series – DeepID (2014) – DeepID2 (2014) – DeepID2+ (2015) – DeepID3 (2015) 12
  • 13. DeepID model ■ CNN structure – 4 convolutional layers – 3 max-pooling layers – 1 fully connected layer ■ Alignment – Center of two eyes, two corners of mouth, nose tip. ■ Multiple patches – Extract different features for different part of face ■ Similarity metric learning – Joint-Bayesian 13
  • 14. DeepID2 Model 14 Fig 6. DeepID2 [3] Fig 7. DeepID [4]
  • 15. DeepID2+ Model ■ Based on DeepID2 – More deep network – Uses supervisory signals – Uses fully connected layers ■ Fully connected layers – Early feature extraction ■ Claim – Deep CNN based networks are more robust to corruption of image! 15Fig 8. DeepID2+ [5]
  • 16. DeepID3 Model ■ DeepID3 – More Deep than DeepID2+ – Less deep than FaceNet & VGG ■ Includes – Very deep neural networks – 15 feature extraction layers – Early fully connected layers ■ Similarity metric learning – Joint-Bayesian 16Fig 9. DeepID3 [6]
  • 17. FaceNet Model ■ FaceNet model: by Google research group – Same framework for identification and verification – Very deep network – No alignment – Efficient representation of features ■ Similarity metric learning – Triplet loss 17Fig 10. Triplet loss learning [7]
  • 18. VGG Model ■ VGG model: by Visual Geometry Group – Inspired by the very deep FaceNet network – Very deep CNN – 36 level of feature extraction ■ Similarity metric – Triplet loss ■ Contributions – Automatic collection of large face dataset – Publically available pre-trained CNN model 18
  • 19. Lightened CNN Model ■ Shallow network – 4 convolutional layers – 4M parameters ■ Less computational intensive – 9 times less than VGG model 19
  • 20. CNN Training and Testing Datasets 20
  • 21. Summary ■ The size of training dataset ■ Alignment ■ Joint-Bayesian similarity ■ Multiple patches ■ Verification performance on video ■ Identification performance on larger number of identities 21
  • 22. Future Work ■ Using face verification to outperform face detection ■ Designing a similarity metric learning ■ Designing a face tracking method based on face verification 22
  • 23. References 1. Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to human-level performance in face verification,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014. 2. S. Chopra, R. Hadsell, and Y. LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, pp. 539–546, 2005. 3. Y. Sun, Y. Chen, X. Wang, and X. Tang, “Deep learning face representation by joint identification verification,” in Advances in Neural Information Processing Systems, pp. 1988–1996, 2014. 4. Y. Sun, X. Wang, and X. Tang, “Deep learning face representation from predicting 10,000 classes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1891–1898, 2014. 5. Y. Sun, D. Liang, X.Wang, and X. Tang, “Deepid3: Face recognition with very deep neural networks,” arXiv preprint arXiv:1502.00873, 2015. 6. Y. Sun, X.Wang, and X. Tang, “Deeply learned face representations are sparse, selective, and robust,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2892–2900, 2015. 7. F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. 23