风格感知和多尺度注意力的人脸图像修复
CSTR:
作者:
作者单位:

(1.天津大学 电气自动化与信息工程学院,天津 300072;2.数字出版技术国家重点实验室,北京 100871)

作者简介:

刘洪瑞(1996—),男,硕士研究生

通讯作者:

中图分类号:

TN911.73

基金项目:

国家自然科学基金(2,3);CCF信息系统开放课题(CCFIS2018G02G04);北大方正集团有限公司数字出版技术国家重点实验室开放课题(Cndplab-2019-Z001)


Style-aware and multi-scale attention for face image completion
Author:
Affiliation:

(1.School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China; 2.State Key Laboratory of Digital Publishing Technology, Beijing 100871, China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    人脸图像修复是计算机视觉领域中重建人脸图像的一项重要图像处理技术。现有人脸图像修复技术存在修复结果全局语义不合理的问题,这主要是由于现有技术的特征长程迁移能力不足,无法将破损图像中已知区域的信息合理地迁移到被遮蔽区域上。为此,本文在生成式对抗网络(generative adversarial network,GAN)框架下,构建了一种融合风格感知和多尺度注意力的编解码人脸图像修复模型。风格感知模块用于提取图像的全局语义信息,并利用提取的信息对编码逐级地进行渲染,以实现对修复过程的全局性调节;利用多尺度注意力模块对多尺度特征进行补丁块提取,并通过共享注意力得分和提取补丁块的矩阵乘法进行多尺度特征的长程迁移。在公开数据集CelebA-HQ上的实验结果表明:风格感知模块和多尺度注意力模块极大地增强了修复网络的特征长程迁移能力。相较于现有先进的人脸图像修复方案,本文所提出的模型在多种评价指标上均有显著的提升;修复结果的全局语义更加合理,并且在暗光条件下的修复效果更加自然。

    Abstract:

    Face image completion is an important image processing technique for reconstructing face images in the field of computer vision. The existing face image completion methods have the problem of unreasonable global semantics, which is mainly due to the lack of long-range transfer capability of the existing techniques that they are unable to reasonably transfer information from known regions in a broken image to occluded regions. To overcome the problem, a novel encoder-decoder face image completion network integrating style-aware and multi-scale attention was proposed under the framework of generative adversarial network (GAN). Specifically, the style-aware module was used to extract the global semantic information of an image, and the extracted information was employed to globally adjust the completion processing by rendering the encoding of the image level by level. The multi-scale attention module extracted patches of multi-scale features and performed a long-range transfer via matrix multiplication between a shared attention score and the extracted patches. Experimental results from the public dataset CelebA-HQ show that the style-aware module and the multi-scale attention module greatly enhanced the long-range transfer capability of the completion network. Compared with the existing state-of-the-art face image completion methods, the proposed model had significant improvement in various evaluation metrics. Meanwhile, the global semantics of the completion results were more reasonable and the completion effect was more natural under low lighting conditions.

    参考文献
    相似文献
    引证文献
引用本文

刘洪瑞,李硕士,朱新山,孙浩,张军.风格感知和多尺度注意力的人脸图像修复[J].哈尔滨工业大学学报,2022,54(5):49. DOI:10.11918/202010013

复制
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2020-10-08
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2022-04-25
  • 出版日期:
文章二维码