Real-time jellyfish classification and detection algorithm based on improved YOLOv4-tiny and improved underwater image enhancement algorithm

The outbreak of jellyfish blooms poses a serious threat to human life and marine ecology. Therefore, jellyfish detection techniques have earned great interest. This paper investigates the jellyfish detection and classification algorithm based on optical images and deep learning theory. Firstly, we create a dataset comprising 11,926 images. A MSRCR underwater image enhancement algorithm with fusion is proposed. Finally, an improved YOLOv4-tiny algorithm is proposed by incorporating a CBMA module and optimizing the training method. The results demonstrate that the detection accuracy of the improved algorithm can reach 95.01%, the detection speed is 223FPS, both of which are better than the compared algorithms such as YOLOV4. In summary, our method can accurately and quickly detect jellyfish. The research in this paper lays the foundation for the development of an underwater jellyfish real-time monitoring system.

www.nature.com/scientificreports/An innovative target monitoring technique is the Convolutional Neural Network (CNN), which has progressively become a novel monitoring approach to study marine life.The CNN-based target identification algorithm is primarily split into two groups: regression-based algorithms like YOLOv4-tiny and Faster R-CNN, which are based on region recommendations.
The research first employs the YOLOv4-tiny since the Faster R-CNN has the issue of low real-time.However, when the YOLOv4-tiny algorithm is used, there is a problem with poor accuracy.
The organization of the rest of this article is as follows."Related research and contributions" section introduce the related works on jellyfish detection and their limitations."Dataset preparation and preprocessing" section presents the establishment of the dataset and underwater image enhancement algorithm.Section "Improved YOLOv4-Tiny Jellyfish Detection Algorithm" describes the jellyfish classification method based on the improved YOLOv4-tiny algorithm."Experiment and result analysis" section shows experimental results to validate the effectiveness and robustness.The conclusion is provided in "Conclusions" section.

Related research and contributions
This section introduces the existing work related to the detection and monitoring of jellyfish and then presents the paper's main contributions.

Related research literature.
Underwater image processing is a necessary means to improve detection accuracy.Therefore, in this paper, we first introduce recent research on underwater image processing.In 2018, Lu et al. proposed the guided image filtering for contrast enhancement method to improve the quality of underwater images.However, the guided filter they used could only be applied to grayscale images, and therefore performed poorly in color restoration 12 .
In 2021, Liu studied underwater image restoration algorithms based on the dark channel prior method, which improved the restoration effect to some extent.However, due to the large number of parameters, the algorithm's robustness was poor 13 .
In 2022, Li et al. proposed the Dark Channel and MSRCR Algorithm Combined method to achieve underwater image dehazing and enhancement, but the method had poor scene applicability 14 .
In 2022, Zhou et al. proposed an algorithm for automatic color correction of underwater images, which solved the color cast caused by the attenuation difference of different color channels in underwater images and could adapt to various underwater environments 15 .
In 2023, Zhou et al. further proposed the multi-interval sub-histogram perspective equalization method for underwater enhancement, which achieved contrast enhancement of underwater images through adaptive interval partitioning and histogram equalization, with excellent image restoration effects 16 .
Furthermore, we introduce related research on jellyfish detection technology.Due to the harm and research value of jellyfish, researchers have long used various technologies, including acoustic, optical, and remote sensing, to search for jellyfish.In 1994, Davis et al. designed a submarine plankton video recording system.Rich visual information, quick recording, and the capacity to capture in-depth jellyfish movement are all benefits of the technology 17 .
In 2006, Houghton et al. recorded jellyfish movement characteristics and distribution using aerial photography technology.However, this approach could only observe large-sized jellyfish near the sea's surface 18 .
In 2015, Donghoon Kim et al. developed an autonomous jellyfish detection and cleaning system.At the same time, the team also proposed a jellyfish detection algorithm based on drone photography.However, it cannot identify jellyfish 19 .
In 2016, Seonghun Kim et al. investigated jellyfish's spatial and vertical distribution by acoustic and optical methods, but this method had limitations in monitoring tiny jellyfish 20 .Hangeun Kim et al. put forward a drone detection system for jellyfish.The design captured the movements of jellyfish on the sea surface and recognized them through deep learning.However, it was limited to the Aurelia aurita 21 .
In 2017, Jungmo Koo and colleagues developed a system seeking out jellyfish distribution by crewless aerial vehicles.They employed a deep neural network that demonstrated high precision and fast speed in accurately identifying jellyfish.Nonetheless, it is noteworthy that this approach can solely discriminate a singular jellyfish species 22 .Martin-Abadal et al. used neural network to design a Jelly monitoring system for the automatic detection and quantification of various jellyfish types, as well as enabling long-term monitoring of their presence.However, the system's applicability is constrained 23 .
In 2018, French et al. implemented underwater imaging technology and neural network to monitor and classify jellyfish.The accuracy of classification reached up to 90%, suggesting that the system can serve as an effective tool for predicting jellyfish outbreaks.However, the system is limited to detect individual jellyfish outbreaks.
In 2020, a novel technique for the automated detection and quantification of jellyfish was developed by Martin Vodopiveca et al.This approach enables the continuous monitoring of jellyfish, while simultaneously assessing the accuracy of manual counting 24 .Through the use of optical imaging and automated image analysis, the algorithm demonstrates the feasibility of identifying jellyfish.However, the current implementation remains limited to offline recognition and is not yet capable of real-time monitoring.
In 2021, Chang Qiuyue et al. of Yanshan University proposed an improved YOLOv3 algorithm, which can achieve real-time detection and identify seven jellyfish species.But its speed and accuracy need to be improved 25 .
In the past, although a series of studies have been carried out on jellyfish detection using acoustics and optics combined with deep learning theory, the research on jellyfish detection is still in the primary stage.So, further study and improvement are needed to improve detection accuracy, speed, and species identification.

Contributions.
The main contributions of this paper are as follows: Underwater image preprocessing.First, we use MSRCR combined with an underwater image fusion method to address the issue of color deterioration and blurring of underwater images acquired by optical equipment.The MSRCR, known for its ability to enhance color in input images 26 , is employed as the initial technique in our new algorithm.The second method focus on image denoising and contrast enhancement [27][28][29] .Subsequently, the output images generated by these two methods are merged using an underwater fusion algorithm, resulting in a final image with vibrant color, sharp contrast, and distinct texture 30 .For a more comprehensive understanding of the process, please refer to Fig. 2, which illustrates the detailed steps of this new algorithm.
In our study, we employ five different algorithms to process the original image.These algorithms include the dark channel prior defogging 13 , contrast enhancement proposed by Lu et al. 12 , MSRCR 26 , underwater image fusion, and our suggested improved underwater image enhancement algorithm.The resulting images are Table 1.Dataset distribution after augmentation.

Dataset settings Images number
The training set 9594 The verification set 1067 The www.nature.com/scientificreports/subsequently evaluated using four evaluation parameters: Entropy 29 , UCIQE 31 , UIQM 32 and EOG 33 .Figure 3 visually presents the effects achieved by applying five algorithms.The evaluation results are given in Table 2.
It can be seen from Table 2 that the Entropy, UCIQE, and EOG reach their maximum when the improved algorithm is applied.Moreover, the improved algorithm can also fulfill the demands of exhibiting the target items and increasing the color of underwater images.Above all, the improved algorithm presented in this section demonstrates excellent performance and could be applied to enhance the optical images utilized in the jellyfish detection system.Figure 4 shows the results processed by five algorithms.Some conclusions can be derived from Fig. 4a-f: The dark channel prior defogging algorithm produces limited color recovery and discernible changes in image texture details.The contrast enhancement method primarily focuses on recovering image texture details, with subpar color recovery.The MSRCR algorithm performs well in image color recovery, but compromises the texture information.While the contrast of the picture backdrop is somewhat increased, the color recovery impact of the fusion algorithm is inferior to that of MSRCR.The new algorithm, on the other hand, produces the most satisfactory overall result.It achieves a mild yet effective color recovery, exact image texture details, and high contrast.

Improved YOLOv4-Tiny Jellyfish Detection Algorithm
The YOLO algorithms have gained widespread popularity in target detection applications.Among them, the YOLOv4-tiny algorithm has fast detection speed, relatively high detection accuracy, a simple network model, and low hardware needs [34][35][36] .The YOLOv4-tiny has a significantly faster recognition speed than YOLOv4, but its accuracy has declined 37 .Therefore, this work will adopt two ways to enhance the YOLOv4-tiny algorithm's accuracy and make it compliant with the criteria of jellyfish detection accuracy and speed.Specific improvements are: (1) Add the attention mechanism module to improve the feature extraction ability of the network and strengthen its recognition of obscured and tiny targets.Add CBAM.CBAM is an attention mechanism that combines space and channel 38 .Compared with the mechanism that only focuses on one channel attention mechanism, the hybrid attention mechanism can achieve better results.Therefore, the hybrid attention mechanism is introduced to make the neural network concentrate more on the target areas that contain essential information and suppress irrelevant information, thereby improving accuracy.The YOLOv4-tiny obtains feature information through the neural network, and there is no feature extraction step.As a result, it is simple to overlook tiny targets and obscured objects.In the paper, the CBAM is added after upsampling.The attention mechanism can weight the feature data of the target objects with dynamic weight coefficients, thus improving the network's ability to pay attention to the target objects, solving the problem of small targets and occluded objects being ignored.Figure 5 depicts the network topology for adding the CBAM.Improvements in training methods.To improve the detection accuracy further, we will introduce the mosaic data enhancement, cosine annealing learning rate and label smoothing in this section.
Mosaic is a form of data enhancement used before model training.Its purpose is to merge four random images into a single new image, thereby enriching the background of the detection target.This process enhances the variety and informational content of the input images, while also reducing overfitting.The steps involved in the mosaic data enhancement are as follows: 1. Read four random images; 2. Crop, zoom, flip, and color gamut changes for four images, respectively; 3. The images from the second step are stitched to obtain images in the specified size range.
The mosaic enhanced images are shown in Fig. 6.Cosine annealing learning rate can reduces the learning rate using a cosine function.Initially, the model enters the training state with a gradually decreasing function value.This faster decrement leads to accelerated convergence of the learning rate.Subsequently, the learning rate gradually decreases again to prevent overshooting the optimal point.This approach often yields favorable results.
The majority of jellyfish have long tentacles and umbrella-shaped heads, with striking similarities.Because of this, manual labeling will inevitably result in mistakes that will impact on the final predictions.Label smoothing prevents over-trust by assuming that labels may be incorrect during training.In this chapter, label smoothing is introduced to improve accuracy.The smoothing coefficient is 0.01.

Comprehensively improved algorithm.
Combining the improved network and training method, a comprehensively improved algorithm is obtained, and the structure is depicted in Fig. 7.

Experiment and result analysis
Experimental process.The network is trained separately with the original data and the enhanced data in "Dataset preparation" section.The train, valid and test sets is set as Table 1.

Parameter settings
The hyperparameter settings are shown in Table 3.

Algorithm comparison.
To demonstrate the effectiveness and superiority of our proposed method, we compared it with several classical and state-of-the-art methods, including YOLOV4, YOLOV5, YOLOV6, YOLOV7, YOLOV8, and our methods.In order to compare the different algorithms more effectively and intuitively, we compared their complexity and accuracy, and the results are shown in Table 4. Layers, parameter quantity, and FPS reflect the complexity of the algorithm.Lower values of layers and parameter quantity indicate simpler architecture with fewer generated parameters and lower complexity, while higher FPS indicates faster processing speed.mAP and F1 reflect the accuracy of the algorithm, with higher values indicating better performance.As shown in Table 4, our proposed algorithm maintains a lightweight structure while achieving high detection performance.
Figure 8 shows the results of different methods trained on the same dataset in the comparative experiment of jellyfish detection.From the Fig. 8, it can be analyzed that regardless of the method used, both false positives and false negatives occurred in the detection of multiple jellyfish images, which proves that jellyfish detection is a challenging task.Among the detection results, the proposed algorithm has the highest confidence but with significant false negatives.YOLOv7 detected the most jellyfish and maintained a high level of confidence.YOLOv5, YOLOv6, and YOLOv8 did not perform well in jellyfish detection.Therefore, we can consider the YOLOv7 method as a deadline for jellyfish detection, while other methods still need improvement.

Ablation experiment process.
The experimental data are quantitatively analyzed in this part using five evaluation indices, and the findings are as follows: 1. Average precision (AP) value analysis Tables 5 and 6 are the results of the above seven experimental methods.Tables 5 and 6 indicate that the utilization of data enhancement has resulted in improved Average Precision (AP) values for most jellyfish species.In addition, the comprehensively improved algorithm achieves a detection accuracy over 95% for most jellyfish types.The mean average precision (mAP) of the seven algorithms is listed in Table 6.
It can be seen from Table 7 that, except for the original YOLOv4-tiny algorithm, the mAP of the other six algorithms is higher than the values without data enhancement, proving the effectiveness of data enhancement.Comparing the mAP values obtained by various methods, it can be seen that the mAP of the YOLOv4tiny algorithm after data enhancement is 1% lower than that of the YOLOv4 algorithm, this indicates a reduction in the feature extraction capability of the YOLOv4 network due to its simplified structure.Table 6 further demonstrates that the improved network structure leads to a 1.59% increase in mAP compared to YOLOv4-tiny and a 0.59% increase compared to YOLOv4, Among the different enhancements, mosaic enhancement produces the most significant effect, surpassing the impact of cosine annealing learning rate and label smoothing.
The mAP can reach 95.01%utilizing data enhancement and the improved algorithm, which is 2.55% higher than the original YOLOv4-tiny algorithm, illustrating that the improved algorithm has the highest detection accuracy.The bold data in the table is the mAP value of the comprehensively improved algorithm.

FPS analysis
Table 8 displays the FPS values for the seven algorithms.From Table 8, we can see that the detection speed of the YOLOv4-tiny algorithm can reach 248 FPS, which is nearly five times higher than that of the YOLOv4 algorithm with only 43.9 FPS.When the average accuracy is considered, it is clear that YOLOv4-tiny will sacrifice a small amount of precision to increase detection speed.The detection speed of the comprehensively improved algorithm can reach 223 FPS, which is a little different from the original YOLOv4-tiny algorithm.www.nature.com/scientificreports/As a result, the comprehensively improved algorithm improves the detection accuracy while sacrificing a bit of the detection speed.

Precision analysis
Table 9 shows the average precision of each algorithm.Table 9 shows that the accuracy of the comprehensively improved algorithm after the data enhancement can reach 92.56%, which can satisfy the detection criteria.algorithm achieves confidence levels of 0.99, 0.99, and 0.95, respectively, with more accurate box positioning compared to other methods.Hence, the improved algorithm exhibits the best detection performance.
Figure 11 shows the jellyfish video image captured in the field.The FPS value is displayed in the upper left corner, while the name and number above the bounding box indicate the accuracy of jellyfish species identification.The average FPS of the entire video is approximately 20, which is due to the large image resolution of 2448 * 2018.
Based on the analysis of experimental data, visualization effects, and performance metrics such as mAP, FPS, precision, recall, and F1-score, it is evident that the improved algorithm proposed in this paper outperforms the other algorithms.The improved algorithm achieves the best results in terms of mAP, FPS, precision, recall, and F1-score, indicating its superior detection capabilities.The experimental analysis also demonstrates that the improved algorithm produces the best detection effects for jellyfish examples.
The high FPS value indicates the algorithm's ability to perform rapid detection, which is crucial for realtime applications.The high F1-score suggests that the network structure is stable, and the algorithm achieves a balanced performance in terms of precision and recall.
Overall, the comprehensively improved algorithm presented in this paper enhances the accuracy of jellyfish detection while ensuring fast and efficient identification.It meets the requirements for rapid and accurate identification of jellyfish.

Conclusions
This study addresses the demand for jellyfish detection by taking several important steps.Firstly, a new dataset containing a large number of images from seven jellyfish species is established, including both publicly available data and data collected in the laboratory.This dataset serves as a valuable resource for further research and development in the field of jellyfish detection.Next, to improve the quality of underwater images and enhance jellyfish detection, this paper proposes a MSRCR underwater image enhancement algorithm with fusion, and demonstrates the effectiveness and superiority of the proposed method through various objective image evaluation parameters.Futhermore, an improved YOLOv4-tiny jellyfish detection algorithm is proposed.This algorithm combines mosaic data augmentation, cosine annealing, and label smoothing methods for weight training, and incorporates CBAM modules to improve feature extraction capabilities, achieving both accuracy and real-time performance in jellyfish detection.Multiple evaluation results from YOLOv4 series ablation experiments and YOLO series comparative experiments demonstrate the superiority and practicality of the proposed algorithm, meeting the requirements for real-time and accurate detection of jellyfish.
While the proposed algorithm shows promising results, there are still challenges to overcome.These include slow processing speed for high-resolution videos, difficulties in handling multiple overlapping jellyfish scenes, and potential missed detections.Future research efforts will focus on improving the algorithm's ability to handle jellyfish overlap and increasing the processing speed for high-resolution images.
Overall, this study provides a template for jellyfish detection, and our proposed algorithm demonstrates good robustness and detection performance, with certain application and reference value in practical engineering detection.It highlights the research potential of the YOLO-tiny series method in jellyfish detection and sets the stage for future advancements in the field.

Figure 2 .
Figure 2. Flow chart of improved underwater image enhancement algorithm.

( 2 )
The mosaic data enhancement is used at the network's input when training the network.To enhance the overall detection impact, two training techniques are simultaneously introduced: label smoothing and the cosine annealing learning rate.

Figure 5 .
Figure 5. Network structure for adding the CBAM.

Figure 10 .
Figure 10.Results of ablation experiment for P. punctata jellyfish.(a) YOLOv4-tiny; (b) improved network structure; (c) improved network structure and mosaic enhancement; (d) improved network structure and cosine annealing learning rate; (e) improved network structure and label smoothing; (f) ours.

Figure 11 .
Figure 11.Video detection results of A. aurita jellyfish by comprehensively improved algorithm.

Table 2 .
Average evaluation results.

Table 4 .
Comparison and evaluation results of algorithms.

Table 5 .
The AP results of seven algorithms with original dataset.

Table 6 .
The AP results of seven algorithms with enhanced dataset.

Table 7 .
The mAP values of seven algorithms.

Table 8 .
The FPS of seven algorithms.

Table 9 .
The precision of seven algorithms.