Introduction

Drones offer a variety of advantages that make them a powerful tool for wildlife ecologists1,2. In the past, it has been challenging to obtain data of animal counts spatially and temporally because aircraft missions and satellite images are expensive, and ground-based surveys in many cases are restrictive in terms of accessibility to sites, the areas that could be covered, and the low cost-effectiveness ratio. More recently, drones have emerged as a highly cost-effective tool that allows researchers to reduce survey costs, notably increasing the amount of high-quality information3,4,5. Additionally, drones can offer a non-invasive technique that reduces disturbance in comparison with traditional approaches6,7. For these reasons, this technology is being increasingly adopted by wildlife ecologists.

Studies about species detection, abundance, distribution, behavior, and reproduction of terrestrial and marine vertebrates have been growing in the past 2 decades using drone technology3,4,8,9. In particular, studies using drones have been carried out in terrestrial mammals, mostly on large herbivores10,11,12,13,14,15,16,17,18. Most of these studies have been conducted in African ecosystems, like the savannas; however, studies on terrestrial mammalian herbivores in the wild are still lacking in Arctic and sub-Arctic ecosystems. These are logistically and financially challenging regions19, where the survey area needed to be covered is usually very large, and weather conditions make occupied survey flights dangerous20 and satellites unreliable21. Fortunately, drones can ameliorate all three of those problems, and could be used for conservation, research, and monitoring in these challenging environments.

One species of conservation interest is caribou (Rangifer tarandus) where population declines appear associated with human activities along its distributional range22. Available methodologies used to monitor caribou populations such as collaring or monitoring with occupied aircrafts, although useful, could be disruptive to individuals, financially challenging, and certainly logistically intensive23. The use of alternative methodologies, such as drones, to study caribou in sub-Arctic habitats of northern North America, has been assessed by Patterson et al.24. These authors evaluated the use of drones as a methodology to manually detect and count surrogate caribou in natural habitats. However, to our knowledge drone technology has not been empirically evaluated with wild caribou. Additionally, the generation and management of large amounts of raw data and manual counting from imagery are still time consuming, inefficient, and error-prone, decreasing the benefits of the technology25,26. Therefore, efficient and cost-effective approaches to detect and count wild caribou could be advantageous, because its broad distribution requires access to remote locations and to cover extensive (in the range of millions of km2) sections of land27. This situation imposes a challenge for researchers in their efforts to acquire and rapidly analyze data. As a result, monitoring that incorporates drones also requires the development of automated procedures to provide accurate and timely information for wildlife managers23.

One approach to facilitate detection and counting of individuals from aerial imagery is machine learning, and in particular the development of convolutional neural networks (CNNs), which are highly successful for computer vision tasks. CNNs are a type of deep neural network useful for image classification and recognition, which are composed by two elements: feature extraction and classification. The purpose of feature extraction is to produce feature maps, which is carried out by processes called convolutions28. Convolutions consists of applying of a filter that slides over an input image (or other feature map) combining the input value and the filter value to produce another feature map. The process is repeated multiple times with different layers of convolutional filters resulting in multiple layers of feature maps of progressively smaller sizes, where the final layer is a vector of single values, as opposed to tensors of feature maps29. Then, the classification part takes this final layer and adds a small number of fully connected layers, similar to a regular feed forward neural network28,30. The end of the classification part is a loss function, typically softmax for classification tasks, which provides a predicted probability for each of the target objects. Applications of CNNs to drone imagery have been growing during the past decade31. For instance, in koalas (Phascolarctus cinereus)32, cetaceans33, olive ridley sea turtles (Lepidochelys olivacea)34, kiang (Equus kiang)35, birds36,37,38,39,40 and a set of African terrestrial mammals41,42,43. Depending on the quality of the imagery and the amount of training data, evidence shows that precision and accuracy of detections using CNNs can be high, in some cases better than human counts25. As a result, there are opportunities to develop CNNs for a host of different wildlife surveys, including methods to count large mammals in remote locations, such as the challenge caribou pose.

The objectives of this study were to train a CNN to detect and classify caribou from large-scale drone imagery, as most modern CNN architectures are not capable of dealing with huge input images (e.g., mosaics exceeding sizes of 50 k by 50 k pixels). Our aim was to develop an efficient and cost-effective approach to provide accurate and timely information for researchers and wildlife managers. Additionally, in studies where automatic detection and classification algorithms are developed, manual classification is employed for two reasons: first, to train and develop algorithms and secondly for validation44. Both processes could be carried out by expert and/or naïve observers (besides citizen science ventures). In this study, we use an expert observer (who was involved in the field data collection) and a team of qualified naïve observers (some of which are experienced image analysts in other contexts) to manually classify detections of different types of caribou (see “Manual counts” section for caribou type details). The experienced observer classifications are used to train and test the CNN model. In addition, annotations of naïve observers are used to mimic a lifelike scenario, where a qualified team of volunteers is employed to generate training data for algorithms in large-scale contexts, from detections and classifications of terrestrial mammals in drone imagery. Thus, our second objective was to compare the CNN model's detections and classifications to the detections and classifications provided by our team of naïve observers. Finally, we discuss the limitations and what is needed to scale up our approach for large-scale studies required to address populations of large terrestrial mammals.

Methods

Study area

We conducted drone surveys on 18 July 2016 within the braided delta of the Mast River in Wapusk National Park, Manitoba, Canada (Supplementary Fig. S1.1 online). The study area where imagery was collected is 1 km2 (Supplementary Information). It consists primarily of small dwarf willow-dominated (Salix sp.) islands (approximately 1–300 m2), open graminoid meadows, and river habitat. For an in-depth geophysical and biological description of the study area see45,46,47,48. The Cape Churchill sub-population present in the study area was estimated in 2937 individuals in 2007 and is part of the Eastern Migratory caribou population, recently designated as Endangered27.

Drone surveys

During drone surveys of nesting common eiders (Somateria mollissima)49, a herd of caribou moved into the study area and remained bedded down or mostly sedentary for several hours. We opportunistically collected imagery of the entire herd during our eider survey. Flights were performed with small fixed-wing drone (Trimble UX5), which contained a 16.1 MP optical camera in the nadir position. Images of caribou were collected during four flights between 09:08 and 12:41, at altitudes of 120 m (2 flights) and 75 m (2 flights) above ground level (AGL). Following surveys, individual images from each flight were stitched together using Pix4D v. 3.1.22 to create four georeferenced orthomosaic TIFF images (ground sampling distance: 3.7 at 120 m and 2.4 cm at 75 m), which were subsequently used to perform manual and automated counts. For further details such as payload, sensor, data collection, data post-processing, permits, regulations, training, and quality reports of this study, see the Drone Reporting Protocol in the Supplementary Information50.

Methods were planned in accordance with the relevant guidelines and regulations. The study was designed considering the potential impacts on target and non-target species. Thus, we flew no less than 75 m above ground level to reduce disturbance on caribou and other biodiversity, as well as were the lowest altitude threshold the Trimble fixed-wing drone could fly. Also, according to national regulations for drone operations, 122 m is the maximum height that we were authorized to fly, so to stay below this threshold we restricted our maximum altitude to 120 m. Data collection and field procedures were authorized by Canadian Wildlife Service Research and Collection Permit 16-MB-SC001, Wapusk National Park WAP-2005-18760 and WAP-2015-18846, UND’s Institutional Animal Care and Use Committee #A3917-01 (protocol 1505-2), UND’s Unmanned Aircraft System Research Compliance Committee reviewed human privacy and data management projects (approved April 10, 2015), and a Special Flight Operations Certificate (File: 5812-11-302, ATS: 17-18-00008,40, RDIMS: 13138829).

Manual counts

Manual counts of caribou on each of the four mosaics, were performed by six observers. One (AFB) is an experienced observer who participated in the field work activities and is acquainted with the behavior and spatial distribution of this caribou herd. The rest of the five observers (naïve observers) lacked experience with the species, although some are experienced image analysts in other settings. All naïve observers were specifically trained in the counting procedure. To perform the identification and classification, all observers used the platform Open UAS Repository—OUR (https://digitalag.org/our/). OUR is a web-based infrastructure that allows users to upload, share, and collaborate in the analysis of large-scale UAS imagery. Mosaics were uploaded to this repository for the observers to search and count caribou individuals with the aid of a 50 × 50 m grid overlay across the image.

The counting procedure involved the identification of three types of targets: “adult caribou”, “calf caribou”, and “ghost caribou” that are the product of the image mosaic processing. Although adult caribou were dominant in the images and their body size was variable, calves (smaller individuals) could be distinguished based on their size (Fig. 1a). “Ghosts”, however, could be of either size and appeared as blurred or even transparent in the images (Fig. 1a). Because individuals move during image collection, they become visible in multiple images as “ghosts” that appear in one image but from an overlapping image they are not present, which causes a challenge for the mosaicking process and ultimately the automated recognition algorithms. Thus, we decided to include this category in the classification in order to account for this potential source of error.

Figure 1
figure 1

(A) Examples of adult (red circle), calf (blue circle), and ghost (yellow circle) caribou that observers classified at the Open UAS Repository—OUR (https://digitalag.org/our/). (B) Adults (red rectangles), calves (blue rectangles), and ghosts (yellow rectangles) after the classification in the repository. Figure was assembled using Affinity Designer v 1.9.1.979 (https://affinity.serif.com/).

Full size image

During the classification process, the observer used the labeling tool of the Open UAS Repository to draw a rectangle or bounding box surrounding each individual identified (Fig. 1b). Each rectangle contains the actual caribou or ghost including all the pixels and the least possible amount of background (Fig. 1b). After the process of labeling, each classified image box was logged into a text file containing information of the type of label and a list of vertex coordinates (pixel coordinates) of the rectangles for all classified caribou. In addition to labeling, observers were asked to record the time they spent in processing each image, for further comparisons with the automatic detection algorithm.

Automatic detections: faster-RCNN

We trained a Faster-RCNN (faster region-based convolutional neural network) model51 with the goal to be independent of the image resolutions of the sensors or flight characteristics, such as altitude, that impact pixel sizes and ultimately ground sampling distance. To utilize the captured large-scale mosaics as training data, these mosaic images were cut into smaller pieces or “tiles” to train and test the model52,53. Tiling is a useful method when computer memory limits the training of large data sets. Thus, the four orthomosaic files (120 m AGL: 845 MB and 980 MB, 75 m AGL: 1.77 GB and 1.88 GB), were tiled to 1000 pixels × 1000 pixels images. Occasionally, as a product of the tiling process, individual caribou (adults, calves, or ghosts) could be cut off and split into two consecutive tiles. To avoid losing or double counting split animals, tiles were overlapped by 100 pixels on the right and lower borders, so that if an animal is located on an edge, it was counted in the following tile (see Supplementary Fig. S2.1 online). Finally, to evaluate the performance of the Faster-RCNN independent of the ground sampling distance, the tiles of the four orthomosaics were shuffled for training.

A total of 2,148 tiles were produced and split into a training data set of 1607 tiles (75%; Mosaic 1 at 120 m AGL: 271 tiles, Mosaic 2 at 120 m AGL: 288 tiles, Mosaic 3 at 75 m AGL: 548 tiles, and Mosaic 4 at 75 m AGL: 500 tiles) and a testing data set of 541 tiles (25%, including tiles with no caribou on them as negative examples; Mosaic 1 at 120 m AGL: 80 tiles, Mosaic 2 at 120 m AGL: 105 tiles, Mosaic 3 at 75 m AGL: 173 tiles, Mosaic 4 at 75 m AGL: 183 tiles; Fig. 2). The training data set was employed to train the Faster R-CNN model (Fig. 2) using TensorFlow54. One NVIDIA GPU and one Intel(R) Xeon(R) CPU at 2.2 GHz were used in the training/testing of the model. The training took a week and performed 20,000 epochs of backpropagation using stochastic gradient decent. During the model training, and at fixed intervals (30 training epochs), the model was assessed using the testing data set. When the training learning curve of the model55,56 was flat, the threshold was reached and the training was concluded (see Fig. 2). The model was compared against our experienced observer to determine its performance. Using this approach, we assumed that the experienced observer did not miss any individual and correctly classified all the adults, calves, and ghosts.

Figure 2
figure 2

Flowchart of the steps followed to train and test the Faster R-CNN model from the expert observer counts. Results of the trained detection model were then used for comparisons with the experienced observer counts.

Full size image

The performance of the Faster-RCNN model was evaluated estimating accuracy, precision, and recall. Accuracy was defined as the proportion of true positives in relation to the experienced observer. Precision was defined as the proportion of true positives predicted by the model that were actually correct. Recall was defined as the proportion of true positives in relation to all the relevant elements. Accuracy, precision, and recall were estimated as follows:

$$ Accuracy = \frac{true\; positives}{{true\; positives + false\; positives + false \;negatives}} $$
(1)
$$ Precision = \frac{true \;positives}{{true \;positives + false \;positives}} $$
(2)
$$ Recall = \frac{true\; positives}{{true \;positives + false \;negatives}} $$
(3)

Accuracy, precision and recall were estimated for caribou as a species, and for the caribou categories: adults, calves, and ghosts, and for each of the four mosaic at 120 m and 75 m AGL. Using the testing data set and comparing the tiles classified by the Faster-RCNN model with those classified by our experienced observer, we proceeded to estimate true positives, false positives, and false negatives as follows. First, we counted all the individual caribou identified by the Faster-RCNN model that matched our experienced observer and classified them as true positives. Then, in relation to our experienced observer, we counted the caribou that were missed as false negatives, and those that were misclassified (birds, rocks, trees, and trunks classified as caribou) as false positives (Fig. 3). Second, we counted all the caribou categories separately: adults, calves, and ghosts identified by the Faster-RCNN model that matched our experienced observer and classified them as true positives. Then, we estimated the adults, calves, and ghosts missed by our model as false negatives, and those that were misclassified (calves classified as adults or ghosts, adults as calves or ghosts, ghosts as adults or calves; birds, rocks, trees, and trunks classified as any of these caribou categories) as false positives (Fig. 3). To estimate accuracy, we did not use true negatives because our approach did not consider classifying the absence of caribou in the images, and Faster-RCNN is an object detection model which only draws bounding boxes (detections) on classes of interest. For such models, true negatives are typically not calculated.

Figure 3
figure 3

Example of comparison of experienced observer counts (A, C) with the Faster-RCNN (B) and naïve observers (D). Blue arrows indicate misclassifications and yellow arrows missed individuals. Note that at the Faster-RCNN image, detection of an adult and a ghost are overlapped. Counted adults are the true positives, misclassifications are false positives, and missed individuals are false negatives. Figure was assembled using Affinity Designer v 1.9.1.979 (https://affinity.serif.com/).

Full size image

Comparison between the Faster-RCNN model and naïve observers’ classifications

Comparisons between the Faster-RCNN model counts and naïve observers counts were established in relation to our experienced observer. First, from each tile used for testing, we counted all the individual caribou predicted by the Faster-RCNN model output that matched the experienced observer (true positives). Then, we counted the caribou that were missed (false negatives) and those that were misclassified (rocks, birds, trees, and tree trunks; Fig. 3). Secondly, we repeated this procedure incorporating the categories adults, calves, and ghosts. To estimate accuracy, precision, and recall we employed Eqs. 1–3, respectively. In addition, we estimated the percentage difference (i.e., the proportion of detections and classifications in relation to the true positives) to evaluate how the Faster-RCNN model and naïve observers counts compared with the experienced observer. A graphical comparison of the counts in each tile of the experienced observer with the Faster-RCNN model was implemented, and Pearson correlation coefficients estimated. Also, we assessed the amount of time allocated to image classification by all the observers and the time the Faster-RCNN took to produce the outputs, to compare how time-consuming both approaches are. Finally, we evaluated the proportion of misclassifications and missed individuals of the Faster-RCNN model and naïve observers in relation to the true positives for each of the mosaics and caribou class (adults, calves, and ghosts).

Results

Faster-RCNN performance

Overall accuracy of the Faster-RCNN model was 80%, precision 90% and recall 88%. The model performed better at higher altitudes, with accuracies between 80 and 88% at 120 m AGL, and between 75 and 76% at 75 m AGL (Table 1). When the performance of the Faster-RCNN model was analyzed by caribou classes and altitudes, the model was more efficient in detecting adults and calves than ghosts at both 120 m AGL mosaics (Table 2). At 75 m AGL, accuracy and precision of adults were higher than calves and ghosts in the first mosaic, although recall of adults was lower than calves and ghosts. In the second 75 m mosaic, the highest accuracy was detected in calves, followed by adults and decreasing markedly in ghosts (Table 2). However, precision for ghosts was higher than adults, although lower than calves in this mosaic, and recall was higher in adults, followed by calves and ghosts (Table 2).

Table 1 Comparison between the Faster-RCNN model, expert observer (AFB), and naïve observers counts (raw counts, and true positives: TP, false positives: FP, false negatives: FN, accuracy, precision, recall, and percentage difference: % diff.) of caribou for each of the mosaics at 120 m and 75 m AGL in Wapusk National Park, Manitoba, Canada.
Full size table
Table 2 Comparison between the Faster-RCNN model, expert observer (AFB), and naïve observers counts (raw counts, true positives: TP, false positives: FP, false negatives: FN, accuracy, precision, recall, and percentage difference: % diff.) of adults, calves, and ghosts caribou for each of the mosaics at 120 m and 75 m AGL in Wapusk National Park, Manitoba, Canada.
Full size table

Faster-RCNN versus manual counts

The counts of the Faster-RCNN and experienced observer per tile for each of the mosaics showed high correlation at 120 m and 75 m (Fig. 4). Overall, most of the detections of the Faster-RCNN model seem overestimated in relation to the experienced observer (Fig. 4), and the percentage difference between both confirms an overestimation between a minimum of + 8.2% (at 120 m) and maximum of + 63.5% (at 75 m, Table 1).

Figure 4
figure 4

Comparison between (A) the individual caribou counts and (B) caribou adults, calves, and ghosts of the experienced observer and the Faster-RCNN per tile for each mosaic. The diagonal 1:1 line indicates a perfect fit of the experienced observer and Faster-RCNN counts. Plots were created using R v 4.1.3 (R Core Team, 2022) and figure assembled using Affinity Designer v 1.9.1.979 (https://affinity.serif.com/).

Full size image

Faster-RCNN and the experienced observer counts per tile and per caribou class for each of the mosaics, showed in general high correlations for adults, calves, and ghosts (Fig. 4). In most cases, the Faster-RCNN overestimated the count of adults, calves, and ghosts (Fig. 4, Table 2). The percentage difference for adults ranged between + 19.5% (at 75 m) and + 36.2% (at 75 m). For calves, this percentage oscillated between − 7.4% (at 120 m) and + 52.6% (75 m), and for ghosts, between -12.0% (120 m) and + 140.0% (120 m) (Table 2).

Overall, naïve observers showed relatively high accuracy and precision in each of the mosaics (Table 1); when adults, calves, and ghosts were analyzed independently, accuracy and precision ranges were more variable, especially because ghosts were detected and classified with higher error (Table 2). Percentage difference at 120 m ranged between − 9.7% and + 63.6%, and at 75 m varied between − 12.9% and + 38.2% (Table 1). Adults percentage difference varied between − 12.9% at 75 m and + 1.6% at 120 m, while calves varied between − 26.9% at 120 m and + 18.2% at 75 m. Ghost was the most variable category with percentage differences between − 10.0% at 75 m and − 63.6% at 120 m.

In relation to time spent by the human observers for the classification of the images, it took on average 121.6 ± 45.9 min for the first 75 m mosaic, 103.8 ± 66.1 min for the second 75 m mosaic, 59.2 ± 20.3 min for the third mosaic at 120 m, and 51.9 ± 42.5 min for the last 120 m mosaic. Not considering the training phase of the Faster-RCNN model that took 1 week, the time that it took the model to process the testing set was 19.8 min.

Misclassifications and missing individuals

In the first 120 m mosaic, the Faster-RCNN missed less individuals and proportions of adults, calves, and ghosts than the naïve observers (Table 3, Supplementary Fig. S2.2 online). In the second 120 m mosaic, however, naïve observers detected more ghosts than the Faster-RCNN model, although calves and adults were better detected by the model (Table 3, Supplementary Fig. S2.2 online). In the first mosaic at 75 m, the Faster-RCNN model did not miss calves and ghosts and missed only one adult (Table 3, Supplementary Fig. S2.2 online). In the second mosaic at 75 m, the Faster-RCNN model did not miss adults, detected more calves and a much lower proportion of ghosts than observers (Table 3, Supplementary Fig. S2.2 online).

Table 3 Number and proportion of caribou adults, calves, and ghosts that were missed by the Faster-RCNN and the pool of naïve observers (mean ± SD) per mosaic.
Full size table

Faster-RCNN classified caribou better than the pool of observers in the first 120 m mosaic. Adults classified as ghosts, was the misclassification with the highest proportion for both, the Faster-RCNN model and observers (Table 4, Supplementary Fig. S2.3 online). In the second 120 m mosaic, Faster-RCNN classified equally or better than the pool of naïve observers, except for ghosts classified as adults that was higher for the model (Table 4, Supplementary Fig. S2.3 online). At 75 m, misclassifications of the Faster-RCNN model were overall lower than the observer’s misclassifications (Table 4, Supplementary Fig. S2.3 online). However, in the first mosaic the model misclassified more calves as adults and ghosts as calves. In the second 75 m mosaic, the model misclassified more calves as ghosts, and ghosts as adults (Table 4, Supplementary Fig. S2.3 online).

Table 4 Counts and proportion of misclassifications of adults, calves, and ghosts of the Faster-RCNN and naïve observers (mean ± SD) per mosaic.
Full size table

Naïve observers did not misclassify caribou as objects i.e., trees, trunks, rocks, or birds. However, the Faster-RCNN model did misclassify caribou as objects in all the mosaics. At 120 m, 9 objects were misclassified as adults, 1 as a calf, and 1 as a ghost. At 75 m, 22 objects were misclassified as adults, 5 as calves, and 4 as ghosts.

Discussion

To the best of our knowledge, we present one of the first attempts to employ automated detection of large mammals from drone-based imagery in North America. We have developed a method for training Faster-RCNN models given large-scale mosaic imagery to classify and count caribou adults, calves, and ghosts independent of the altitude and ground sampling distance of the collected imagery. After having compared the image detections and classifications of the Faster-RCNN model with those of an experienced observer on the same images, we noticed that the automatic detection and classification performance could be promising for future implementations. When the analysis was performed by mosaic and caribou class, the Faster-RCNN model performance was also promising, in some cases it accomplished better outcomes than the naïve observers. However, ghost was the category which detection and classification were both challenging by the Faster-RCNN and naïve observers. Adults and calves in some cases were better detected and classified by the Faster-RCNN model than the naïve observers. However, a high proportion of adults were misclassified as calves in all the mosaics, mostly by some naïve observers rather than the Faster-RCNN model. This study suggests that there is a need to improve consistency among observers to better classify groups, required to train models accurately in large-scale studies. These types of studies are also challenged by double counting of individuals, a problem that needs to be overcome. Our study found that having trained the model from images with different ground sampling distances, detection and classification of caribou is satisfactory, opening new promising avenues for the implementation of large-scale studies and monitoring.

In applied contexts, the benefits of drone technology could be challenged by the high amount of information collected by the sensors, which artificial intelligence is attempting to unravel. Given these extensive data sets, especially from highly mobile species as the one analyzed in this study, practitioners could be benefitted by using teams of human observers as ground truth for labeling and model training. In this scenario, a minimum level of consistency is desirable for successfully training algorithms from multiple operators57, because accuracy of a model could be undermined by the high uncertainty of the observer annotations58. Our study found that detection and classification varied among observers, which opens questions on how to minimize this variability for further implementation of artificial intelligence in large-scale applied settings. Further, we propose the following actions when designing surveys to improve the quality of training data: (a) the expansion of the training phase and regular assessments of observers’ asymptotic learning curves; (b) working as a group to allow for collective and common learning conducive to a more standardized experience; (c) to provide observers with the same set of tools, such as screens sizes and resolutions, similar timelines with appropriate resting times, and other standard working conditions like light, temperature, and ventilation; and finally (d) aid observers with alternative tools such as GPS or radio tracked caribou to verify true positive detections. This way we might be able to account for inter-observer variation and train models from multiple observers in large-scale situations.

One benefit of mosaicking is the potential reduction of double counting when animals move across the landscape, since overlapped areas in the imagery are excluded8,59; but the process is not perfect, and ghosts emerge are one drawback that we aimed to account for with our classification system. It is noteworthy that our Faster-RCNN model had issues detecting ghosts in two mosaics, one at 120 m and one at 75 m AGL, which similarly happened with the naïve observers. The model had also difficulties to correctly classify ghosts that were mostly confounded with adults, although did better than some naïve observers. Considering that the movements of the caribou herd analyzed in this study were relatively slow, the question of if mosaicking reduce double counting would work with highly mobile species, arise and might be further assessed. To avoid the presence of ghosts in mosaics, it could be useful to use the original raw images or strip sampling as an alternative59, although additional efforts should be allocated to reduce the number of double-counts. For instance, flying more than one drone simultaneously, similar to employ multiple field observers, could reduce double counts, although it could be costly. It may also be helpful to incorporate object tracking components from video footage into the CNN analysis method, to reduce double counts of the same individuals. In any case, it is important that flight plans consider minimizing behavioral responses to reduce the chances that animals do not move in reaction to the aircraft59,60. Moreover, if we could design surveys to reduce double counting close to zero, we could be able to explore the use of hierarchical models to detect and classify individuals. It has been proposed that N-mixture models is a good method to estimate abundance from aerial imagery59, although these models are very sensitive to violations of assumptions, i.e., observers do not double count individuals, counts are independent binomial random variables, detection probabilities are constant, abundance is random and independent, and the population is closed61,62. Other approaches like a modification of the Horvitz–Thompson method63 that combines generalized linear and additive models for overall probability of detection, false detection, and duplicate detection have been proposed as better alternatives to N-mixture models64. This could be a promising avenue to couple models that account for imperfect detection to train neural networks, especially in contexts where data sets are becoming large and difficult to process by human observers.

An algorithm able to learn and classify animals from imagery taken at different altitudes or ground sampling distances could be an advantage for generalizable models65, especially useful for researchers and practitioners. Our Faster-RCNN model could be able to satisfactorily detect and classify caribou at both altitudes with different ground sampling distances. This might open further avenues to overcome difficulties that prevent combining different sources of data, especially when dealing with broadly distributed species. For example, different types of airspaces that constrain flights at certain altitudes could vary between countries and regions. Additionally, access to standardized platforms and sensors for long term and large-scale studies is a challenge, which could be overcome with algorithms like ours, potentially independent of ground sampling distances. Some successful examples of this approach are present in the literature for the detection of conifers66, crops67, and large mammals15. To achieve that, inter alia, we needed to assess what are the limits for algorithms to be trained with a range of ground sampling distances, able to accurately classify targets; in addition to evaluations under different whether conditions, backgrounds, and species. Ultimately, we could be able to generalize and optimize resources and data, to leverage the application of this technology for studying and managing wildlife.

To successfully apply drone technology to large-scale studies of large mammals, we need to scale up flights to larger landscapes rather than smaller areas. However, there are still technical and logistic limitations related to the use of beyond visual line of sight platforms (BVLOS) that facilitate larger areas surveys. A few examples of BVLOS use have been carried out in wildlife ecology, mostly in marine settings. For instance, Hodgson et al.68 assessed the detection probability of a BVLOS drone platform to detect humpback whales (Megaptera novaeangliae) in Australia. Similarly, Ferguson et al.69 evaluated the performance of images taken from a drone platform in relation to direct surveys and imagery from manned aircraft, to detect and count marine mammals in Alaska; BVLOS platforms are promising although still expensive and less efficient than human observers onboard occupied aircrafts, authors concluded. Isolated marine biodiversity such as marine mammals, seabirds, and tundra communities have successfully been surveyed on King George island in Antarctica using BVLOS technology70. Nevertheless, a bigger problem BVLOS surveys have, would be the sheer amount of data collected, and concordantly with our findings, manual counts of wildlife are not scalable due to time restrictions71.