面向图像修复的桥式注意力取证网络
CSTR:
作者:
作者单位:

(天津大学 电气自动化与信息工程学院,天津 300072)

作者简介:

张澜(2000―),男,硕士研究生;朱新山(1977—),男,教授,博士生导师

通讯作者:

薛俊韬,xuejt@tju.edu.cn

中图分类号:

TN911.73

基金项目:

国家自然科学基金(2,3)


Bridge-type attention forensics network for image inpainting
Author:
Affiliation:

(School of Electrical and Information Engineering, Tianjin University, Tianjin 300072,China)

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    为提升多媒体信息的可靠性,减轻图像伪造事件对于社会造成的负面影响,亟需发展图像修复取证技术,检测并定位图像的篡改区域。本研究提出了一种面向图像修复的桥式注意力取证网络,该网络直接接收篡改后的图像,端到端的输出图像中被篡改的区域,网络采用编码器-解码器架构作为基础框架。首先,编码器选用Swin Transformer和RepVGG两个主干网络以提取多域修复特征。然后,使用桥式注意力模块连接两个主干网络的同级阶段,来增加编码器在局部和全局维度上的建模能力。最后,在编码器和解码器中间搭建了语义对齐融合模块,消除了两个主干网络提取的特征之间的语义不一致,有助于提高网络的取证性能。在不同修复取证数据集上的实验结果表明,所提出的模型与其他主流取证模型相比,能够更准确地对修复区域进行定位。特别是在有挑战性的DeepFillV2数据集和Diffusion数据集上,所提出的BAFNet分别取得了91.37%和82.34%的IoU分数,相比于主流的取证网络MVSS-Net, IoU指标分别提升了8.77%和10.46%。另外,综合多个实验结果,BAFNet在取证性能和模型复杂度之间取得了很好的平衡。

    Abstract:

    To enhance the reliability of multimedia information and mitigate the negative impact of image forgery events on society, there is an urgent need to develop image inpainting forensics to detect and locate tampered regions of images. This paper proposes a bridge-type attention forensics network (BAFNet) for image inpainting. The network receives tampered images directly and outputs the tampered regions end-to-end. The network adopts an encoder-decoder architecture as the basic framework. Firstly, the encoder selects two backbones, Swin Transformer and RepVGG, to extract multi-domain inpainting features. Then, a bridge-type attention module is used to connect the same-level stages of the two backbones, enhancing the encoder’s modeling capability in both local and global dimensions. Finally, a semantic alignment fusion module is built between the encoder and the decoder to eliminate semantic inconsistencies between the features extracted by the two backbones, thereby improving the forensic performance of the network. Experimental results on different inpainting forensic datasets demonstrate that the proposed model, compared with other mainstream forensic models, can more accurately locate the inpainting areas. In particular, on the challenging DeepFillV2 dataset and Diffusion dataset, the proposed BAFNet achieves IoU scores of 91.37% and 82.34%, respectively, which improves the IoU metrics by 8.77% and 10.46% compared to the mainstream forensic network MVSS-Net. In addition, combining the results of several experiments, BAFNet achieves a good balance between forensic performance and model complexity.

    参考文献
    相似文献
    引证文献
引用本文

张澜,朱新山,王泽平,薛俊韬.面向图像修复的桥式注意力取证网络[J].哈尔滨工业大学学报,2025,57(4):62. DOI:10.11918/202404005

复制
相关视频

分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2024-04-01
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2025-04-07
  • 出版日期:
文章二维码