目的 逆半色调方法是实现数字化文件管理和高精度图像识别的关键技术,通过现有的逆半色调方法恢复得到的图像存在图像内容缺失、图像细节再现模糊等问题。针对现有方法的不足,提出多尺度多层次信息自适应融合的逆半色调方法。方法 首先,提出多尺度自适应深度网络,实现多尺度信息的提取;然后,采用稠密残差块与注意力机制相结合的形式实现图像细节信息的有效提取;最后,构建多信息自适应融合网络,将不同阶段恢复得到的图像内容信息与细节信息有效融合,从而得到高质量的逆半色调图像。实验在Set14、Urban100、Microsoft COCO等3个数据集上与最新的5种方法进行比较。结果 实验结果表明,与现有方法相比,在客观评价方面,其峰值信噪比平均值提高了0.05’5.51 dB,结构相似度平均值提高了0’0.1;在主观评价方面,运用此方法得到的逆半色调图像去除半色调噪点更为彻底,恢复出的图像细节更好,在视觉上与原始图像更为相近。同时,对于处理256像素×256像素的图像,所提出的网络在GPU上的平均运行时间为0.13 s。结论 所提出的多尺度多层次信息自适应融合模型可以得到更高质量的逆半色调图像。
Abstract
Inverse halftoning method is a key technique to restore inverse halftone images, which has been used for digital file management and high precision image recognition. The existing inverse halftone methods have some problems such as missing image content, fuzzy reproduction of image details. To relieve the shortcomings of existing methods, the work aims to propose an inverse halftoning method based on multi-scale and multi-level information adaptive fusion. Firstly, the effective multi-scale adaptive deep convolutional network was proposed to extract the multi-scale information. Then, the detailed information of the halftone image were extracted effectively by the combination of dense residual blocks and attention mechanism. Finally, the multi-information adaptive fusion deep convolutional network was constructed to obtain high quality inverse halftone images by fusing the content information and detailed information recovered from different stages effectively. Extensive experiments were conducted on three public datasets, such as Set14, Urban100 and Microsoft COCO3, and compared with the 5 state-of-the-art methods. The experimental results show that in terms of objective evaluation, the mean peak signal-to-noise ratio increased by 0.05-5.51 dB, and the mean structural similarity increased by 0-0.1. In terms of subjective evaluation, the inverse halftone image obtained by the proposed method was more similar to the original image visually, the halftone noise was removed more thoroughly and the image details were better restored. Moreover, the average runtime running on a GPU of the proposed network was 0.13 s for an image with the size of 256 pixel×256 pixel, which meant that the proposed network could meet the requirements of practical applications. The proposed multi-scale and multi-level information adaptive fusion model can achieve the inverse halftone image of better quality.
关键词
逆半色调方法 /
多尺度自适应深度网络 /
多信息自适应融合深度网络 /
半色调图像
Key words
inverse halftoning method /
multi-scale adaptive deep network /
multi-information adaptive fusion deep network /
halftone image
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] ASHOUR A S, GUO Y H, HAWAS A R, et al.Optimised Halftoning and Inverse Halftoning of Dermoscopic Images for Supporting Teledermoscopy System[J]. IET Image Processing, 2019, 13(3): 529-536.
[2] KODGE S, CHAUDHARY H, SHARAD M.Low Power Image Acquisition Scheme Using On-Pixel Event Driven Halftoning[C]//IEEE Computer Society Annual Symposium on VLSI, 2017: 260-265.
[3] KANG H R.Digital Color Halftoning[M]. Bellingham: SPIE Press, 1999: 29-38.
[4] GUO J M, SANKARASRINIVASAN S. Digital Halftone Database (DHD): A Comprehensive Analysis on Halftone Types[C]//Asia-Pacific Signal and Information Processing Association Annual Summit and Conference. IEEE, 2018: 1091-1099.
[5] FREITAS P, FARIAS M, ARAUJO A.Fast Inverse Halftoning Algorithm for Ordered Dithered Images[C]//24th SIBGRAPI Conference on Graphics, Patterns and Images. IEEE, 2011: 250-257.
[6] UNAL G B, CETIN A E.Restoration of Error-Diffused Images Using Projection onto Convex Sets[J]. IEEE Transactions on Image Processing, 2001, 10(12): 1836-1841.
[7] MESE M, VAIDYANATHAN P P.Look-up Table (LUT) Method for Inverse Halftoning[J]. IEEE Transactions on Image Processing, 2001, 10(10): 1566-1578.
[8] ZHANG Y, ZHANG E H, CHEN W J, et al.Sparsity-Based Inverse Halftoning via Semi-Coupled Multi-Dictionary Learning and Structural Clustering[J]. Engineering Applications of Artificial Intelligence, 2018, 72: 43-53.
[9] HUANG W B, SU A W Y, KUO Y H. Neural Network Based Method for Image Halftoning and Inverse Halftoning[J]. Expert Systems with Applications, 2008, 34(4): 2491-2501.
[10] SHAO L H, ZHANG E H, LI M.An Efficient Convolutional Neural Network Model Combined with Attention Mechanism for Inverse Halftoning[J]. Electronics, 2021, 10(13): 1574.
[11] 李梅, 张二虎. 融合注意力机制的多尺度深度网络的逆半调方法[J]. 包装工程, 2022, 43(11): 283-291.
LI M, ZHANG E H.A Multi-Scale Deep Network Combined with Attention Mechanism for Inverse Halftoning[J]. Packaging Engineering, 2022, 43(11): 283-291.
[12] LI M, LIU Q.An Inverse Halftoning Method Based on Supervised Deep Convolutional Neural Network[J]. IET Image Processing, 2024, 18(4): 961-971.
[13] LI F Y, YANG J, SANG G L.MSPRL: Multiscale Progressively Residual Learning Network for Image Inverse Halftoning[J]. Journal of Image and Graphics, 2024, 29(4): 953-965.
[14] XIA M H, WONG T T.Deep Inverse Halftoning via Progressively Residual Learning[C]//Asian Conference on Computer Vision. Cham: Springer International Publishing, 2018: 523-539.
[15] SON C H.Inverse Halftoning through Structure-Aware Deep Convolutional Neural Networks[J]. Signal Processing, 2020, 173: 107591.
[16] YUAN J, PAN C, ZHENG Y, et al.Gradient-Guided Residual Learning for Inverse Halftoning and Image Expanding[J]. IEEE Access, 2019, 8: 50995-51007.
[17] LI M, ZHANG E H, WU L L, et al.A Multistage and Multiresolution Deep Convolutional Neural Network for Inverse Halftoning[J]. Expert Systems with Applications, 2022, 191: 116358.
[18] ZHANG E H, LI M, ZHANG Q, et al.An Inverse Halftoning Method for Various Types of Halftone Images Based on Multi-Scale Generative Adversarial Network[J]. Engineering Applications of Artificial Intelligence, 2023, 117: 105600.
[19] ZHANG Y L, TIAN Y P, KONG Y, et al.Residual Dense Network for Image Super-Resolution[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 2472-2481.
[20] SIMONYAN K, ZISSERMAN A. Very Deep Convolutional Networks for Large-Scale Image Recognition [EB/OL]. (2014-09-04)[2024-06-23]. https://arxiv.org/abs/1409.1556.
[21] ZHOU B, LAPEDRIZA A, XIAO J, et al.Learning Deep Features for Scene Recognition Using Places Database[J]. Advances in Neural Information Processing Systems, 2014, 27: 487-495.
[22] EVERINGHAM M, VAN GOOL L, Williams C, et al.The Pascal Visual Object Classes Challenge: A Retrospective[J]. International Journal of Computer Vision, 2015, 111(1): 98-136.
[23] DONG C, LOY C C, TANG X.Accelerating the Super-Resolution Convolutional Neural Network[C]//Computer Vision - ECCV 2016. Cham: Springer International Publishing, 2016: 391-407.
[24] ZEYDE R, ELAD M, PROTTER M.On Single Image Scale-up Using Sparse-Representations[C]//International Conference on Curves and Surfaces. Springer Berlin Heidelberg, 2010: 711-730.
[25] HUANG J B, SINGH A, AHUJA N.Single Image Super-Resolution from Transformed Self-Exemplars[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015: 5197-5206.
[26] LIN T Y, MAIRE M, BELONGIE S, et al.Microsoft COCO: Common Objects in Context[C]//Computer vision-ECCV. Cham: Springer International Publishing, 2014: 740-755.
[27] AMATO G, CARRARA F, FALCHI F, et al.Deep Learning for Decentralized Parking Lot Occupancy Detection[J]. Expert Systems with Applications, 2017, 72: 327-334.
[28] FLOYD R W, STEINBERG L.An Adaptive Algorithm for Spatial Grayscale[J]. In Proceedings of the Society of Information Display, 1976, 17(2): 75-77.
[29] LEDIG C, THEIS L, HUSZÁR F, et al. Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017: 4681-4690.
基金
山西省基础研究计划（202403021222305）; 山西省高等学校科技创新项目（2022L484）; 山西省高等学校教学改革创新项目（J20241298）; 博士科研启动项目（YQ-2023039）
PDF(12976 KB)
渝公网安备 50010702501716号 渝ICP备15012534号-2
