基于傅里叶卷积和多特征调制的图像修复网络

孙刘杰, 刘倩倩, 庞茂然

包装工程(技术栏目) ›› 2023 ›› Issue (21) : 286-293.

PDF(5127 KB)
PDF(5127 KB)
包装工程(技术栏目) ›› 2023 ›› Issue (21) : 286-293. DOI: 10.19554/j.cnki.1001-3563.2023.21.036

基于傅里叶卷积和多特征调制的图像修复网络

  • 孙刘杰, 刘倩倩, 庞茂然
作者信息 +

Image Inpainting Network Based on Fourier Convolution and Multi-feature Modulation

  • SUN Liu-jie, LIU Qian-qian, PANG Mao-ran
Author information +
文章历史 +

摘要

目的 解决大面积破损难以修复且修复过程中感受野、特征空间信息利用不足,导致修复后的孔洞区域与背景之间出现结构、纹理、风格不一致的问题。方法 基于傅里叶卷积和多特征调制的修复网络FFC-MFMGAN,傅里叶卷积在网络的浅层便具有较大的感受野,尤其是在宽掩码时能够跳过掩码区域,捕获到有效特征,多特征调制生成网络能够分别利用完整区域的信息和随机样式操纵,增强与未受损区域的语义连贯性,以及大空洞率下修复的多样性。结果 在Place 2数据集上,将文中方法与其他图像修复方法进行了对比实验,经过测试,各类指标均得到明显改善,峰值信噪比提高了1.4%,结构相似性提高了4.5%,平均绝对误差降低了12.6%,基于学习的感知图像块相似性降低了9.1%。结论 FFC-MFMGAN网络能够较好地修复大面积不规则孔洞,同时增强修复图像的全局结构性和清晰度,对实际包装印刷图像的缺陷修复也有一定参考价值。

Abstract

The work aims to address the problem that large areas of damage are difficult to repair, the perceptual field and feature space information are under-utilized in the repair process, and the structure, texture and style between the repaired cavity areas and the background are inconsistent. An inpainting network FFC-MFMGAN based on Fourier convolution and multi-feature modulation was proposed. Fourier convolution had a large perceptual field in the shallow layer of the network, especially in wide masks, which could skip the mask zone to capture the effective features especially when the mask was wide. The generative network based on the multi-feature modulation was able to enhance the semantic coherence with undamaged regions and the diversity of restoration at large void rates with information from intact regions and random pattern manipulation, respectively. Experiments were conducted to compare the proposed method with other state-of-the-art image restoration methods on the Place 2 dataset, and the following categories were tested to show significant improvements, including a 1.4% improvement in PSNR, a 4.5% improvement in SSIM, a 12.6% reduction in MAE, and a 9.1% reduction in LPIPS. The FFC-MFMGAN network can better repair large irregular holes, while enhancing the global structure and clarity of the repaired images, which is also a reference value for the repair of defects in actual packaging printing images.

引用本文

导出引用
孙刘杰, 刘倩倩, 庞茂然. 基于傅里叶卷积和多特征调制的图像修复网络[J]. 包装工程(技术栏目). 2023(21): 286-293 https://doi.org/10.19554/j.cnki.1001-3563.2023.21.036
SUN Liu-jie, LIU Qian-qian, PANG Mao-ran. Image Inpainting Network Based on Fourier Convolution and Multi-feature Modulation[J]. Packaging Engineering. 2023(21): 286-293 https://doi.org/10.19554/j.cnki.1001-3563.2023.21.036

基金

上海市科学技术委员会科研计划(18060502500);上海市自然科学基金面上项目(19ZR1435900)

PDF(5127 KB)

Accesses

Citation

Detail

段落导航
相关文章

/