| 引用本文: | 刘洪瑞,李硕士,朱新山,孙浩,张军.风格感知和多尺度注意力的人脸图像修复[J].哈尔滨工业大学学报,2022,54(5):49.DOI:10.11918/202010013 |
| LIU Hongrui,LI Shuoshi,ZHU Xinshan,SUN Hao,ZHANG Jun.Style-aware and multi-scale attention for face image completion[J].Journal of Harbin Institute of Technology,2022,54(5):49.DOI:10.11918/202010013 |
|
| 摘要: |
| 人脸图像修复是计算机视觉领域中重建人脸图像的一项重要图像处理技术。现有人脸图像修复技术存在修复结果全局语义不合理的问题,这主要是由于现有技术的特征长程迁移能力不足,无法将破损图像中已知区域的信息合理地迁移到被遮蔽区域上。为此,本文在生成式对抗网络（generative adversarial network,GAN）框架下,构建了一种融合风格感知和多尺度注意力的编解码人脸图像修复模型。风格感知模块用于提取图像的全局语义信息,并利用提取的信息对编码逐级地进行渲染,以实现对修复过程的全局性调节；利用多尺度注意力模块对多尺度特征进行补丁块提取,并通过共享注意力得分和提取补丁块的矩阵乘法进行多尺度特征的长程迁移。在公开数据集CelebA-HQ上的实验结果表明:风格感知模块和多尺度注意力模块极大地增强了修复网络的特征长程迁移能力。相较于现有先进的人脸图像修复方案,本文所提出的模型在多种评价指标上均有显著的提升；修复结果的全局语义更加合理,并且在暗光条件下的修复效果更加自然。 |
| 关键词: 人脸图像修复 生成对抗网络 风格感知 多尺度注意力 长程迁移 |
| DOI：10.11918/202010013 |
| 分类号:TN911.73 |
| 文献标识码:A |
| 基金项目:国家自然科学基金(2,3);CCF信息系统开放课题(CCFIS2018G02G04);北大方正集团有限公司数字出版技术国家重点实验室开放课题(Cndplab-2019-Z001) |
|
| Style-aware and multi-scale attention for face image completion |
|
LIU Hongrui1,2,LI Shuoshi1,ZHU Xinshan1,2,SUN Hao1,ZHANG Jun1
|
|
(1.School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China; 2.State Key Laboratory of Digital Publishing Technology, Beijing 100871, China)
|
| Abstract: |
| Face image completion is an important image processing technique for reconstructing face images in the field of computer vision. The existing face image completion methods have the problem of unreasonable global semantics, which is mainly due to the lack of long-range transfer capability of the existing techniques that they are unable to reasonably transfer information from known regions in a broken image to occluded regions. To overcome the problem, a novel encoder-decoder face image completion network integrating style-aware and multi-scale attention was proposed under the framework of generative adversarial network (GAN). Specifically, the style-aware module was used to extract the global semantic information of an image, and the extracted information was employed to globally adjust the completion processing by rendering the encoding of the image level by level. The multi-scale attention module extracted patches of multi-scale features and performed a long-range transfer via matrix multiplication between a shared attention score and the extracted patches. Experimental results from the public dataset CelebA-HQ show that the style-aware module and the multi-scale attention module greatly enhanced the long-range transfer capability of the completion network. Compared with the existing state-of-the-art face image completion methods, the proposed model had significant improvement in various evaluation metrics. Meanwhile, the global semantics of the completion results were more reasonable and the completion effect was more natural under low lighting conditions. |
| Key words: face image completion generative adversarial network (GAN) style-aware multi-scale attention long-range transfer |
