Underwater Image Color Correction and Enhancement Based on Improved Cycle-consistent Generative Adversarial Networks
-
摘要: 针对水下观测图像的颜色失真和散射模糊问题, 提出一种基于改进循环一致性生成对抗网络(Cycle-consistent generative adversarial networks, CycleGAN)的水下图像颜色校正与增强算法. 为了利用CycleGAN学习水下降质图像到空气中图像的映射关系, 对传统CycleGAN的损失函数进行了改进, 提出了基于图像强边缘结构相似度(Strong edge and structure similarity, SESS)损失函数的SESS-CycleGAN, SESS-CycleGAN可以在保留原水下图像的边缘结构信息的前提下实现水下降质图像的颜色校正和对比度增强. 为了确保增强后图像和真实脱水图像颜色的一致性, 建立了SESS-CycleGAN和正向生成网络G相结合的网络结构; 并提出了两阶段学习策略, 即先利用非成对训练集以弱监督方式进行SESS-CycleGAN学习, 然后再利用少量成对训练集以强监督方式进行正向生成网络G的监督式学习. 实验结果表明: 本文算法在校正水下图像颜色失真的同时还增强了图像对比度, 且较好地实现了增强后图像和真实脱水图像视觉颜色的一致性.
-
关键词:
- 水下图像 /
- 深度学习 /
- 循环一致性生成对抗网络 /
- 颜色校正 /
- 图像增强
Abstract: Aiming at the problem of color distortion and scattering blurry effect of underwater observation images, an color correction and enhancement algorithm of underwater images is proposed based on improved cycle-consistent generative adversarial networks (CycleGAN). Firstly, in order to learn the mapping relationship between underwater degraded images and in-air images using CycleGAN, the CycleGAN is improved by introducing a new loss function based on strong edge and structure similarity (SESS). The new SESS-CycleGAN can realize color correction and contrast enhancement of underwater degraded images while retaining their edge and structure information. Secondly, for ensuring the color consistency between the enhanced images and the their corresponding in-air images, a dual-level network structure combining SESS-CycleGAN with a forward generation network G is constructed, and a two-stage learning strategy is proposed. In the two-stage learning process, the SESS-CycleGAN is first trained in a weakly supervised manner with a unpaired training set composed of underwater images and in-air images, and then the forward generation network G of SESS-CycleGAN is trained again in a strong supervised manner with a small number of paired training set. Experimental results show that the proposed algorithm can enhance image contrast while correcting color distortion of underwater images, and has better visual color consistency with real images in air. -
表 1 增强图像的指标对比
Table 1 Comparison of enhanced images
方法 信息熵 UCIQE PSNR (dB) 原始图像 6.546 0.437 — DCP 6.940 0.540 19.537 GW 6.621 0.493 16.491 CycleGAN 7.592 0.613 19.690 SSIM-CycleGAN 7.598 0.541 20.670 本文方法 7.714 0.631 24.594 
下载: 导出CSV
-
[1] Pinto A M, Matos, Anibal C. M. A hybrid imaging system for underwater robot ic applications. Information Fusion, 2020, 55: 16-29. doi: 10.1016/j.inffus.2019.07.014 [2] Chadebecq F, Vasconcelos F, Lacher R. Refractive Two-View Recon- struction for Underwater 3D Vision. International Journal of Computer Vision, 2020, 128(5): 1101–1117. doi: 10.1007/s11263-019-01218-9 [3] Gallimore E, Terrill E, Pietruszka A, Gee J, Nager A, Hess R. Magnetic survey and autonomous target reacquisition with a scalar magnetometer on a small AUV. Journal of field robotics, 2020, 37(7): 1246-1266. [4] Ahn J, Yasukawa S, Sonoda T. An Optical Image Transmission System for Deep Sea Creature Sampling Missions Using Autonomous Underwater Vehicle. IEEE Journal of oceanic Eengineering. 2020, 45(2): 350-361. doi: 10.1109/JOE.2018.2872500 [5] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353. doi: 10.1109/TPAMI.2010.168 [6] Wen H C, Tian Y H, Huang T J, Gao W. Single underwater image enhancement with a new optical model. In: Proceedings of the IEEE International Symposium on Circuits and Systems (ISC 2013). Beijing, China: IEEE, 2013. 753−756 [7] 杨爱萍, 曲畅, 王建, 张莉云. 基于水下成像模型的图像清晰化算法. 电子与信息学报, 2018, 40(2): 298-305.Yang A P, Qu C, Wang J, Zhang L Y. Underwater image visibility restoration based on underwater imaging model. Journal of Electronics & Information Technology, 2018, 40(2): 298-305. [8] Li C Y, Guo J C. Underwater image enhancement by dehazing and color correction. Journal of Electronic Imaging, 2015, 24(3): Article No. 033023 [9] Emberton S, Chittka L, Cavallaro A. Underwater image and video dehazing with pure haze region segmentation. Computer Vision and Image Understanding, 2018, 168: 145-156. doi: 10.1016/j.cviu.2017.08.003 [10] Liu Y C, Chan W H, Chen Y Q. Automatic white balance for digital still camera. IEEE Transactions on Consumer Electronics, 1995, 41(3): 460-466. doi: 10.1109/30.468045 [11] Ebner M. Color Constancy. New York: John Wiley & Sons, 2007. [12] Iqbal K, Odetayo M O, James A E, Salam R A, Talib A Z H. Enhancing the low quality images using unsupervised colour correction method. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics. Istanbul, Turkey: IEEE, 2010. 1703−1709 [13] Hitam M S, Awalludin E A, Yussof W N J H W, Bachok Z. Mixture contrast limited adaptive histogram equalization for underwater image enhancement. In: Proceedings of the IEEE International Conference on Computer Applications Technology (ICCAT). Sousse, Tunisia: IEEE, 2013. 1−5 [14] Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS). Montreal, Canada: ACM, 2014. 2672−2680 [15] Zhu J Y, Park T, Isola P, Efros A A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2242−2251 [16] Li, J, Skinner K A, Eustice R M, Johnson-Roberson M. WaterGAN: Unsupervised generative network to enable realtime color correction of monocular underwater images. IEEE Robotics and Automation Letters, 2017, 3(1): 387–394. [17] Li C Y, Guo J C, Guo C L. Emerging from Water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal Processing Letters, 2018, 25 (3), 323-327. doi: 10.1109/LSP.2018.2792050 [18] Chen B, Zhang X, Wang R T, Li Z, Deng W. Detect concrete cracks based on OTSU algorithm with differential image. The Journal of Engineering. 2019, 23: 9088-9091. [19] Li C Y, Guo C L, Ren W Q, Cong R M, Hou J H, Kwang S, et al. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 2019, 29: 4376−4389 [20] Ahmad S, Abdul G, Nor A M I. Underwater image quality enhancement through integrated color model with Rayleigh distribution. Applied Soft Computing, 2015, 27: 219-230. doi: 10.1016/j.asoc.2014.11.020 [21] Yang M, Sowmya A. An underwater color image quality evaluation metric. IEEE Transactions on Image Processing, 2015, 24(12): 6062-6071. doi: 10.1109/TIP.2015.2491020 -
图(10) / 表(1)
计量
- 文章访问数: 3432
- HTML全文浏览量: 1704
- PDF下载量: 410
- 被引次数: 0
