Journal of Applied Research in Electrical Engineering

Journal of Applied Research in Electrical Engineering

Deep Learning based Underwater Object Detection and Recognition for Multi-Robot Systems

Document Type : Research article

Authors
Chabahar Maritime University, Chabahar, Iran
Abstract
This paper introduces an innovative method for underwater object detection and recognition using a deep learning framework and autonomous underwater vehicles. The proposed method uses a deep learning framework with a network of underwater mobile robots operating within three-dimensional environments. To navigate these environments effectively, a random decentralized algorithm guides autonomous underwater vehicles (AUVs) in detecting objects within confined three-dimensional spaces. The deployment strategy involves utilizing the vertices of a cubic grid to optimize the positioning of AUVs across the search area. To enhance the object detection capabilities of mobile underwater robots within these environments and address the complexities of similar objects concealed in their surroundings, deep learning based on the YOLOv6 framework is refined to achieve improved performance and efficiency metrics. Specifically, the proposed Camouflaged Sea Edge and Attention Module (CSEA) is integrated into the YOLO framework, resulting in a substantial (8%) enhancement in mean average precision (mAP) performance, with a negligible impact (approximately 3%) on efficiency when evaluated on the publicly available TrashCan benchmark dataset. Additionally, to accelerate the training convergence and to better detect small objects within the TrashCan dataset, a distinct loss function based on Distance Intersection over Union (DIoU) is employed in the final proposed model. Several ablation studies are conducted to compare various YOLO models and evaluate the performance of the proposed CSEA module on the TrashCan dataset.

Highlights

  • Introduced a deep learning system for AUV-based underwater object detection via a decentralized 3D search.
  • Proposed the CSEA module to enhance camouflaged object detection. Integrated CSEA with YOLOv6 for 8% detection gain.
  • Used DIoU loss and validated on TrashCan benchmark with extensive quantitative and qualitative ablation studies.

Keywords
Subjects

  1. Nazarzehi and A. V. Savkin, “Decentralized three-dimensional formation building algorithms for a team of nonholonomic mobile agents,” International Journal of Control, Automation and Systems, vol. 17, pp. 1283–1292, 2019.
  2. Li, G. Zhang, C. Jiang, and W. Zhang, “A survey of maritime unmanned search system: theory, applications and future directions,” Ocean Engineering, vol. 285, p. 115359, 2023.
  3. Zou, K. Chen, Z. Shi, Y. Guo, and J. Ye, “Object detection in 20 years: A survey,” Proceedings of the IEEE, 2023.
  4. Raisi and J. Zelek,” Visual place recognition from end-to-end semantic scene text features,” Frontiers in Robotics and AI, vol. 11, p.1424883 2024.
  5. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image segmentation using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3523–3542, 2021.
  6. Rawat and Z. Wang, “Deep convolutional neural networks for image classification: A comprehensive review,” Neural computation, vol. 29, no. 9, pp. 2352–2449, 2017.
  7. Ghafari, A. Amirkhani, E. Rashno, and S. Ghanbari, “Smart AI-based Video Encoding for Fixed Background Video Streaming Applications,” Journal of Applied Research in Electrical Engineering, 2023 vol. 2, no. 1, pp. 37-44.
  8. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in Proc. IEEE Conf. on Comp. Vision and Pattern Recognition, pp. 779– 788, 2016.
  9. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, “SSD: Single shot multibox detector,” in
  1. Eur. Conf. on Comp. Vision, pp. 21–37, Springer, 2016.
  1. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” Advances in Neural Information Processing Systems, vol. 28, 2015.
  2. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969, 2017.
  3. C. Corrigan, Z. Y. Tay, and D. Konovessis, “Real-time instance segmentation for detection of underwater litter as a plastic source,” Journal of Marine Science and Engineering, vol. 11, no. 8, p. 1532, 2023.
  4. Dai, H. Liu, P. Song, and M. Liu, “A gated cross-domain collaborative network for underwater object detection,” arXiv preprint arXiv:2306.14141, 2023.
  5. Deng, D. Ergu, F. Liu, B. Ma, and Y. Cai, “An embeddable algorithm for automatic garbage detection based on complex marine environment,” Sensors, vol. 21, no. 19, p. 6391, 2021.
  6. Ma et al., “Mldet: Towards efficient and accurate deep learning method for marine litter detection,” Ocean & Coastal Management, vol. 243, p. 106765, 2023.
  7. Nazarzehi, A. V. Savkin and A. Baranzadeh, "Distributed 3D dynamic search coverage for mobile wireless sensor networks," IEEE Communications Letters, vol. 19, no. 4, pp.633-636, April 2015.
  8. Nazarzehi and A. V. Savkin. “Distributed self-deployment of mobile wireless 3D robotic sensor networks for complete sensing coverage and forming specific shapes,” Robotica, vol. 36, no. 1, pp. 1-18, 2018.
  9. Nazarzehi and A. Baranzadeh, "A distributed bio-inspired algorithm for search of moving targets in three-dimensional spaces," in 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO), Zhuhai, China, 2015, pp. 2507-2512.
  10. Li, L. Li, H. Jiang, K. Weng, Y. Geng, L. Li, Z. Ke, Q. Li, M. Cheng, W. Nie, et al., “Yolov6: A single-stage object detection framework for industrial applications,” arXiv preprint arXiv:2209.02976, 2022.
  11. Hong, M. Fulton, and J. Sattar, “Trashcan: A semantically segmented dataset towards visual detection of marine debris,” arXiv preprint arXiv:2007.08097, 2020.
  12. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” in Proc. Adv. in Neural Info. Process. Sys., pp. 91– 99, 2015.
  13. He, G. Gkioxari, P. Dollar, and R. Girshick, “Mask R-CNN,” in Proc. IEEE Int. Conf. on Comp. Vision, pp. 2961– 2969, 2017.
  14. -I. Watanabe, Y. Shao, and N. Miura, “Underwater and airborne monitoring of marine ecosystems and debris,” Journal of Applied Remote Sensing, vol. 13, no. 4, Art. no. 044509, 2019.
  15. Valdenegro-Toro, “Submerged marine debris detection with autonomous underwater vehicles,” in 2016 International Conference on Robotics and Automation for Humanitarian Applications (RAHA), pp. 1–7, IEEE, 2016.
  16. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “Yolox: Exceeding yolo series in 2021,” arXiv preprint arXiv:2107.08430, 2021
  17. Redmon and A. Farhadi, “YOLOv3: An incremental improvement,” arXiv preprint arXiv:1804.02767, 2018.
  18. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770– 778, 2016.
  19. -Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2117–2125, 2017.
  20. Jocher, K. Nishimura, T. Mineeva, and R. Vilarino, “Yolov5 (2020),”˜ GitHub repository. [Online]. Available: https://github.com/ultralytics/yolov5, 2020.
  21. -Y. Wang, I.-H. Yeh, and H.-Y. M. Liao, “You only learn one representation: Unified network for multiple tasks,” arXiv preprint arXiv:2105.04206, 2021.
  22. Jocher, A. Chaurasia, and J. Qiu, “Ultralytics yolov8,” [Online]. Available: https://github.com/ultralytics/ultralytics, 2023.
  23. Terven and D. Cordova-Esparza, “A comprehensive review of yolo: From YOLOv1 to YOLOv8 and beyond,” arXiv preprint arXiv:2304.00501, 2023.
  24. -Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, “Microsoft coco: Com-´ mon objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740–755, Springer, 2014.
  25. -P. Fan, G.-P. Ji, G. Sun, M.-M. Cheng, J. Shen, and L. Shao, “Camouflaged object detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2777–2787, 2020.
  26. Mei, G.-P. Ji, Z. Wei, X. Yang, X. Wei, and D.-P. Fan, “Camouflaged object segmentation with distraction mining,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8772–8781, 2021.
  27. Sun, S. Wang, C. Chen, and T.-Z. Xiang, “Boundary-guided camouflaged object detection,” arXiv preprint arXiv:2207.00794, 2022.
  28. Dosovitskiy et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
  29. Raisi and J. Zelek, “Investigation of deep learning optimization algorithms in scene text detection,” International Journal of Industrial Electronics Control and Optimization, vol. 6, no. 3, pp. 171–182, 2023.
  30. -Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, “Focal loss for dense object detection,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 2980– 2988, 2017.
  31. Zhang, Y. Wang, F. Dayoub, and N. Sunderhauf, “Varifocalnet: An iou-aware dense object detector,” in CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8510–8519, 2020.
  32. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-iou loss: Faster and better learning for bounding box regression,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 12993–13000, 2020.
  33. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 658–666, 2019.
  34. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017.
  35. H. Lucas et. al., “Gelatinous zooplankton biomass in the global oceans: geographic variation and environmental drivers,” Global Ecology and Biogeography, vol. 23, no. 7, pp. 701–714, 2014.
Volume 4, Issue 1
Winter and Spring 2025
January 2025
Pages 24-33

  • Receive Date 22 October 2024
  • Revise Date 12 May 2025
  • Accept Date 12 July 2025