An Overview of Face Image Forgery Detection

Defen      He; Shuai      Liu; Xin      Jin; Shanshan      Huang; Qian      Jiang

doi:10.2174/2665997202666220818122419

Abstract

With the development of face forgery techniques, the spread and malicious abuse of forged images have become a thought-provoking problem, and the face forgery detection technique has also attracted people's attention. Academia has carried out in-depth research and discussion on detection techniques. This review discussed different face forgery methods and detection techniques. Four categories of detection methods are introduced: 1) detection method based on spatial domain, 2) detection method based on the frequency domain, 3) detection method based on biological information, and 4) detection method of multiple feature domains. This paper discussed each detection method's evolution and development in recent years. We paid special attention to the detection method of multiple feature domains and focused on the progress that has been made and the challenges it faced. In addition, this paper discussed open issues and future development trends that should be paid attention to in this field.

Keywords: Deepfake detection, face manipulation, neural network, media forensics, image processing, multiple feature domains.

Graphical Abstract

[1]
D.P. Kingma,  and M. Welling, "Auto-encoding variational bayes", arXiv preprint arXiv: 1312.6114, 2013.
[2]
A.S. Chivukula,  and W. Liu, "Adversarial deep learning models with multiple adversaries", IEEE Trans. Knowl. Data Eng., vol. 31, no. 6, pp. 1066-1079, 2018.
 [http://dx.doi.org/10.1109/TKDE.2018.2851247]
[3]
A. Chivukula, X. Yang, W. Liu, T. Zhu,  and W. Zhou, "Game the- oretical adversarial deep learning with variational adversaries", IEEE Trans. Knowl. Data Eng., vol. 33, no. 11, pp. 3568-3581, 2021.
[4]
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville,  and Y. Bengio, "Generative adversarial nets", Adv. Neural Inf. Process. Syst., pp. 2672-2680, 2014.
[5]
M.D. Ansari, E. Rashid, S.S. Skandha,  and S.K. Gupta, "A comprehensive analysis of image forensics techniques: Challenges and future direction", Rec. Pat. Eng., pp. 458-467, 2020.
[6]
F. Matern, C. Riess,  and M. Stamminger, “Exploiting visual artifacts to expose deepfakes and face manipulations”, 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), 07-11 January 2019., Waikoloa, HI, USA, IEEE, 2019, pp. 83-92.
 [http://dx.doi.org/10.1109/WACVW.2019.00020]
[7]
L. Li, J. Bao, T. Zhang, H. Yang, D. Chen, F. Wen,  and B. Guo, "Face x-ray for more general face forgery detection", Proceedings of the IEEE/CVF conference on computer vision and patternrecognition, 13-19 June 2020, Seattle, WA, USA, pp. 5001-5010, 2020.
 [http://dx.doi.org/10.1109/CVPR42600.2020.00505]
[8]
D. Afchar, V. Nozick, J. Yamagishi,  and I. Echizen, "Mesonet: A compact facial video forgery detection network", IEEE international workshop on information forensics and security (WIFS), 11-13 December 2018, Hong Kong, China, pp. 1-7, 2019.
[9]
K. Xu, M. Qin, F. Sun, Y. Wang, Y-K. Chen,  and F. Ren, "Learning in the frequency domain", In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition June, 2020, Seattle, WA, 2020, pp. 1740-1749 
[10]
X. Zhang, S. Karaman,  and S-F. Chang, Detecting and simulating artifacts in GAN fake images2019 IEEE International Workshop on Information Forensics and Security (WIFS) 09-12 December 2019, Delft, Netherlands, IEEE, 2019, pp. 1-6.
 [http://dx.doi.org/10.1109/WIFS47025.2019.9035107]
[11]
R. Durall, M. Keuper,  and J. Keuper, "Watch your up-convolution: Cnn based generative deep neural networks are failing to reproduce spectral distributions", In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 13-19 June 2020, Seattle, WA, USA, 2020, pp. 7890-7899 
 [http://dx.doi.org/10.1109/CVPR42600.2020.00791]
[12]
J. Frank, T. Eisenhofer, L. Schönherr, A. Fischer, D. Kolossa,  and T. Holz, "Leveraging frequency analysis for deep fake image recognition", In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, 2020, pp. 3247-3258 
[13]
Y. Li, M-C. Chang,  and S Lyu, "In ictu oculi: Exposing ai generated fake face videos by detecting eye blinking", arXiv preprint arXiv:1806.02877, 2018.
[14]
U-A. Ciftci, I. Demir,  and L. Yin, "FakeCatcher: Detection of synthetic portrait videos using biological signals", EEE Trans. Pattern Anal. Mach. Intell., 2020.
 [http://dx.doi.org/10.1109/TPAMI.2020.3009287] [PMID: 32750816]
[15]
H. Qi, Q. Guo, F. Juefei-Xu, X. Xie, L. Ma, W. Feng, Y. Liu,  and J. Zhao, "Deeprhythm: Exposing deepfakes with attentional visual heartbeat rhythms", Proceedings of the 28th ACM International Conference on Multimedia, pp. 4318-4327, 2020.
[16]
J. Hernandez-Ortega, R. Tolosana, J. Fierrez,  and A Morales, "Deepfakeson-phys: Deepfakes detection based on heart rate estimation", arXiv preprint arXiv:2010.00400, 2020.
[17]
J. Hu, X. Liao, W. Wang,  and Z. Qin, "Detecting compressed deepfake videos in social networks using frame-temporality two-stream convolutional network", IEEE Trans. Circ. Syst. Video Tech., vol. 32, no. 3, pp. 1089-1102, 2021.
 [http://dx.doi.org/10.1109/TCSVT.2021.3074259]
[18]
S. Chen, T-P. Yao, Y. Chen, S-H. Ding, J-L. Li,  and R.G. Ji, "Local relation learning for face forgery detection", In: The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21), 2021, pp. 1081-1088.
[19]
H. Liu, X-D. Li, W-B. Zhou, Y-F. Chen, Y. He, H. Xue, W-M. Zhang,  and N-H. Yu, Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 20-25 June 2021, Nashville, TN, USA, 2021, pp. 772-781.
 [http://dx.doi.org/10.1109/CVPR46437.2021.00083]
[20]
T. Karras, S. Laine,  and T. Aila, "A style-based generator architecture for generative adversarial networks", In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15-20 June 2019, Long Beach, CA, USA, IEEE, 2019, pp. 4401-4410 
 [http://dx.doi.org/10.1109/CVPR.2019.00453]
[21]
A. Radford, L. Metz,  and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks", In 9th Conference on Artificial Intelligence and Robotics and 2nd Asia-Pacific International Symposium, 10-10 December 2018, Kish Island, Iran, IEEE, 2018. 
[22]
M. Mirza,  and S. Osindero, "Conditional generative adversarial nets", Comput. Sci., pp. 2672-2680, 2014.
[23]
X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever,  and P. Abbeel, "InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets", In NIPS’16: Proceedings of the 30th International Conference on Neural Information Processing Systems, December 2016, pp. 2180-2188 
[24]
H. Zhang, StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks. 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 5908-5916.
 [http://dx.doi.org/10.1109/ICCV.2017.629]
[25]
M. Tahmid, S. Alam,  and M.k. Akram, "Comparative analysis of generative adversarial networks and their variants", 23rd International Conference on Computer and Information Technique (ICCIT), pp. 1-6, 2020.
 [http://dx.doi.org/10.1109/ICCIT51783.2020.9392660]
[26]
Y. Choi, M. Choi, M. Kim, J. Ha, S. Kim,  and J. Choo, "StarGAN: Unified genera- tive adversarial networks for multi-domain image-to-image translation", Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8789-8797, 2018.
 [http://dx.doi.org/10.1109/CVPR.2018.00916]
[27]
M. Liu, Y. Ding, M. Xia, X. Liu, E. Ding, W. Zuo,  and S. Wen, "STGAN: A unified se- lective transfer network for arbitrary image attribute editing", Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3673-3682, 2019.
 [http://dx.doi.org/10.1109/CVPR.2019.00379]
[28]
J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt,  and M. Nießner, "Face2face: Real– time face capture and reenactment of RGB videos", Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2387-2395, 2016.
 [http://dx.doi.org/10.1145/2929464.2929475]
[29]
J. Thies, M. Zollhöfer,  and M. Nießner, "Deferred neural rendering: Image synthesis using neural textures", ACM Trans. Graph., vol. 38, no. 66, pp. 1-12, 2019.
 [http://dx.doi.org/10.1145/3306346.3323035]
[30]
Y. Kawai, M. Seo,  and Y-W. Chen, "Automatic generation of facial expression using generative adversarial nets", IEEE 7th Global Conference on Consumer Electronics (GCCE), pp. 278-280, 2018.
 [http://dx.doi.org/10.1109/GCCE.2018.8574866]
[31]
A. Pumarola, A. Agudo, A-M. Martinez, A. Sanfeliu,  and F-M. Noguer, "GANimation: One-shot anatomically consistent facial animation", Int. J. Comput. Vis., vol. 128, no. 3, pp. 698-713, 2020.
 [http://dx.doi.org/10.1007/s11263-019-01210-3]
[32]
H. Li, B. Li, S. Tan,  and J. Huang, "Identification of deep network generated images using disparities in color components", Signal Processing, vol. 174, p. 107616, 2020.
 [http://dx.doi.org/10.1016/j.sigpro.2020.107616]
[33]
L. Nataraj, T.M. Mohammed, B.S. Manjunath, S. Chandrasekaran, A. Flenner, J.H. Bappy,  and A.K. Roy-Chowdhury, "Detecting GAN generated fake images using co-occurrence matrices", Electronic Imaging, vol. 2019, no. 5, pp. 532-1-532-7, 2019.
 [http://dx.doi.org/10.2352/ISSN.2470-1173.2019.5.MWSF-532]
[34]
G. Wang, J. Zhou,  and Y Wu, "Exposing deep-faked videos by anomalous co-motion pattern detection", arXiv preprint arXiv:2008.04848, 2020.
[35]
R. Andreas, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies,  and N. Matthias, "Faceforensics: A large-scale video dataset for forgery detection in human faces", arXiv preprint arXiv, 2018.
[36]
A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies,  and M. Nießner, "Faceforensics++: Learning to detect manipulated facial images", Proceedings of the IEEE International Conference on Computer Vision, pp. 1-11, 2019.
[37]
Y. Li, X. Yang, P. Sun, H. Qi,  and S. Lyu, "Celeb-df: A large-scale challenging dataset for deep- fake forensics", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3207-3216, 2020.
[38]
B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang,  and C.C Ferrer, "The deepfake detection challenge dataset", arXiv preprint arXiv: 2006.07397, 2020.
[39]
J. Yang, A. Li, S. Xiao, W. Lu,  and X. Gao, "MTD-Net: Learning to detect deepfakes images by multi-scale texture difference", IEEE Trans. Inf. Forensics Security, vol. 16, pp. 4234-4245, 2021.
 [http://dx.doi.org/10.1109/TIFS.2021.3102487]
[40]
K. He, X. Zhang, S. Ren,  and J. Sun, "Deep residual learning for image recognition", 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
 [http://dx.doi.org/10.1109/CVPR.2016.90]
[41]
O.D. Lima, S. Franklin, S. Basu, B. Karwoski,  and A. George, "Deepfake detection using spatiotemporal convolutional networks", arXiv preprint arXiv, p. 2006.14749, 2020.
[42]
E. Sabir, J. Cheng,  and A. Jaiswal, "W. AbdAlmageed, I. Masi, and P. Natarajan, “Recurrent convolutional strategies for face manipulation detection in videos", CVPR Workshop, 2019.
[43]
S-Y. Wang, O. Wang, R. Zhang, A. Owens,  and A.A. Efros, CNN-generated images are surprisingly easy to spot… for now. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 2020, pp. 8692-8701.
 [http://dx.doi.org/10.1109/CVPR42600.2020.00872]
[44]
T. Karras, T. Aila, S. Laine,  and J. Lehtinen, "Progressive growing of gans for improved quality, stability, and variation", Neural and Evolutionary Computing, ariv, 2017.
[45]
J. Liu, K. Zhu, W. Lu, X. Luo,  and X. Zhao, "A Lightweight 3D convolutional neural network for deepfake detection", Int. J. Intell. Syst., pp. 1-15, 2021.
 [http://dx.doi.org/10.1002/int.22499]
[46]
Z. Liu, X. Qi,  and P.H.S. Torr, "Global texture enhancement for fake face detection in the wild", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8060-8069, 2020.
 [http://dx.doi.org/10.1109/CVPR42600.2020.00808]
[47]
B. Han, X. Han, H. Zhang, J. Li,  and X. Cao, "Fighting fake news: Two stream network for deepfake detection via learnable SRM", IEEE Trans. Biometrics Behav. Identity Sci., vol. 3, no. 3, pp. 320-331, 2021.
[48]
H. Dang, F. Liu, J. Stehouwer, X. Liu,  and A.K. Jain, "On the detection of digital face manipulation", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition, pp. 5781-5790, 2020.
 [http://dx.doi.org/10.1109/CVPR42600.2020.00582]
[49]
H. Zhao, W. Zhou, D. Chen, T. Wei, W. Zhang,  and N. Yu, "Multi-attentional deepfake detection", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2185-2194, 2021.
[50]
F. Chollet, "Xception: Deep learning with depthwise separable convolutions", 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800-1807, 2017.
 [http://dx.doi.org/10.1109/CVPR.2017.195]
[51]
X. Wu, Z. Xie, Y.T. Gao,  and Y. Xiao, "Sstnet: Detecting manipulated faces through spatial, steganalysis and temporal features", ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2952-2956, 2020.
 [http://dx.doi.org/10.1109/ICASSP40776.2020.9053969]
[52]
X. Zhu, H. Wang, H. Fei, Z. Lei,  and S-Z. Li, "Face forgery detection by 3D decomposition", Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2929-2939, 2020.
[53]
X. Yang, Y. Li,  and S. Lyu, "Exposing deep fakes using inconsistent head poses", In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 2019, pp. 8261-8265 
 [http://dx.doi.org/10.1109/ICASSP.2019.8683164]
[54]
J.H. Bappy, C. Simons, L. Nataraj, B.S. Manjunath,  and A.K. Roy-Chowdhury, "Hybrid LSTM and encoder-decoder architecture for detection of image forgeries", IEEE Trans. Image Process., vol. 28, no. 7, pp. 3286-3300, 2019.
 [http://dx.doi.org/10.1109/TIP.2019.2895466] [PMID:  30703026]
[55]
I. Masi, A. Killekar, R.M. Mascarenhas, S.P. Gurudatt,  and W. AbdAlmageed, "Two-branch recurrent network for isolating deepfakes in videos", European Conference on Computer Vision, pp. 667-684, 2020.
[56]
J. Li, H. Xie, J. Li, Z. Wang,  and Y. Zhang, "Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection", In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 20-25 June 2021, Nashville, TN, USA, IEEE, 2021, pp. 6458-6467 
 [http://dx.doi.org/10.1109/CVPR46437.2021.00639]
[57]
Y. Luo, Y. Zhang, J. Yan,  and W. Liu, Generalizing face forgery detection with high-frequency features. Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20-25 June 2021, Nashville, TN, USA, IEEE, 2021, pp. 16317-16326.
 [http://dx.doi.org/10.1109/CVPR46437.2021.01605]

Rights & Permissions Print Cite

Article Metrics

36

3

DOI https://dx.doi.org/10.2174/2665997202666220818122419	Print ISSN 2665-9972
Publisher Name Bentham Science Publisher	Online ISSN 2665-9964

Current Chinese Computer Science

An Overview of Face Image Forgery Detection

Abstract

Graphical Abstract

Related Journals

Related Books