Economical crowdsourcing for camera trap image classification

Using machine learning (ML) to automate camera trap (CT) image processing is advantageous for time-sensitive applications. However, little is currently known about the factors influencing such processing. Here, we evaluate the influence of occlusion, distance, vegetation type, size class, height, subject orientation towards the CT, species, time-of-day, colour, and analyst performance on wildlife/human detection and classification in CT images from western Tanzania. Additionally, we compared the detection and classification performance of analyst and ML approaches. We obtained wildlife data through pre-existing CT images and human data using voluntary participants for CT experiments. We evaluated the analyst and ML approaches at the detection and classification level. Factors such as distance and occlusion, coupled with increased vegetation density, present the most significant effect on DP and CC. Overall, the results indicate a significantly higher detection probability (DP), 81.1%, and correct classification (CC) of 76.6% for the analyst approach when compared to ML which detected 41.1% and classified 47.5% of wildlife within CT images. However, both methods presented similar probabilities for daylight CT images, 69.4% (ML) and 71.8% (analysts), and dusk CT images, 17.6% (ML) and 16.2% (analysts), when detecting humans. Given that users carefully follow provided recommendations, we expect DP and CC to increase. In turn, the ML approach to CT image processing would be an excellent provision to support time-sensitive threat monitoring for biodiversity conservation.

As human activities in natural areas increase, understanding human–wildlife interactions is crucial. Big data approaches, like large‐scale camera trap studies, are becoming more relevant for studying these interactions. In addition, open‐source object detection models are rapidly improving and have great potential to enhance the image processing of camera trap data from human and wildlife activities. In this study, we evaluate the performance of the open‐source object detection model MegaDetector in cross‐regional monitoring using camera traps. The performance at detecting and counting humans, animals and vehicles is evaluated by comparing the detection results with manual classifications of more than 300 000 camera trap images from three study regions. Moreover, we investigate structural patterns of misclassification and evaluate the results of the detection model for typical temporal analyses conducted in ecological research. Overall, the accuracy of the detection model was very high with 96.0% accuracy for animals, 93.8% for persons and 99.3% for vehicles. Results reveal systematic patterns in misclassifications that can be automatically identified and removed. In addition, we show that the detection model can be readily used to count people and animals on images with underestimating persons by −0.05, vehicles by −0.01 and animals by −0.01 counts per image. Most importantly, the temporal pattern in a long‐term time series of manually classified human and wildlife activities was highly correlated with classification results of the detection model (Pearson's r = 0.996, p < 0.001) and diurnal kernel densities of activities were almost equivalent for manual and automated classification. The results thus prove the overall applicability of the detection model in the image classification process of cross‐regional camera trap studies without further manual intervention. Besides the great acceleration in processing speed, the model is also suitable for long‐term monitoring and allows reproducibility in scientific studies while complying with privacy regulations.

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] Camera traps are a popular tool to sample animal populations because they are noninvasive, detect a variety of species, and can record many thousands of animal detections per deployment. Cameras are typically set to take bursts of multiple images for each detection, and are deployed in arrays of dozens or hundreds of sites, often resulting in millions of images per study. The task of converting images to animal detection records from such large image collections is daunting, and made worse by situations that generate copious empty pictures from false triggers (e.g. camera malfunction or moving vegetation) or pictures of humans. We offer the first widely available computer vision tool for processing camera trap images. Our results show that the tool is accurate and results in substantial time savings for processing large image datasets, thus improving our ability to monitor wildlife across large scales with camera traps. In this dissertation, we have developed new image/video processing and computer vision algorithms for efficient and accurate object detection and sequence-level classiffication from natural scene camera-trap images. This work addresses the following five major tasks: (1) Human-animal detection. We develop a fast and accurate scheme for human-animal detection from highly cluttered camera-trap images using joint background modeling and deep learning classification. Specifically, first, We develop an effective background modeling and subtraction scheme to generate region proposals for the foreground objects. We then develop a cross-frame image patch verification to reduce the number of foreground object proposals. Finally, We perform complexity-accuracy analysis of deep convolutional neural networks (DCNN) to develop a fast deep learning classification scheme to classify these region proposals into three categories: human, animals, and background patches. The optimized DCNN is able to maintain high level of accuracy while reducing the computational complexity by 14 times. Our experimental results demonstrate that the proposed method outperforms existing methods on the camera-trap dataset. (2) Object segmentation from natural scene. We first design and train a fast DCNN for animal-human-background object classification, which is used to analyze the input image to generate multi-layer feature maps, representing the responses of different image regions to the animal-human-background classifier. From these feature maps, we construct the so-called deep objectness graph for accurate animal-human object segmentation with graph cut. The segmented object regions from each image in the sequence are then verfied and fused in the temporal domain using background modeling. Our experimental results demonstrate that our proposed method outperforms existing state-of-the-art methods on the camera-trap dataset with highly cluttered natural scenes. (3) DCNN domain background modeling. We replaced the background model with a new more efficient deep learning based model. The input frames are segmented into regions through the deep objectness graph then the region boundaries of the input frames are multiplied by each other to obtain the regions of movement patches. We construct the background representation using the temporal information of the co-located patches. We propose to fuse the subtraction and foreground/background pixel classiffcation of two representation : a) chromaticity and b) deep pixel information. (4) Sequence-level object classiffcation. We proposed a new method for sequence-level video recognition with application to animal species recognition from camera trap images. First, using background modeling and cross-frame patch verification, we developed a scheme to generate candidate object regions or object proposals in the spatiotemporal domain. Second, we develop a dynamic programming optimization approach to identify the best temporal subset of object proposals. Third, we aggregate and fuse the features of these selected object proposals for efficient sequence-level animal species classification.

Camera trap ecology can be viewed as the combination of three steps: data collection, data processing (photo classification), and data analyses. Careful application of these steps can yield valuable insights into ecological parameters. There have been highly successful citizen science projects which crowdsourced at least one of the first two steps, saving substantial time and resources for researchers. However, we believe there is potential to take citizen science camera trapping from “citizens as sensors” to having active participants in all phases of research, which could benefit both researchers and citizen scientists. To that end, we implemented our pilot project – MammalWeb – to integrate all three phases of camera trapping into a complete citizen science web platform. Through a partnership between Durham University and the Durham Wildlife Trust, we recruited citizen scientists from the public to deploy and monitor camera traps across the north east of England. They were trained in camera trapping methodology and asked to employ the same sampling protocol. To integrate camera trap ecology into education, computer science students at Durham University help develop the backend technology for MammalWeb, and secondary school students are leading multidisciplinary projects with teachers to create outreach material for ecological curricula and the public. As of February 2016, over 50 citizen scientists are monitoring camera traps at more than 130 sites across northeast England. They have uploaded over 43,000 images to MammalWeb, of which more than 28,000 have been classified by more than 100 registered users. An algorithm adapted from past work was developed by computer science students at Durham University to calculate consensus identifications of animals from the crowdsourced data, on which a user-facing dashboard is being created for participants to explore and interrogate the database. Our collaboration with secondary schools has engaged students in becoming seed ecological ambassadors, and created multimedia projects to share ecological knowledge with their communities. In this talk, I will describe: The MammalWeb user experience How our collaboration with educational institutions has produced actively involved citizen scientists in camera trap monitoring, and How participants help develop MammalWeb into a modular and generalisable citizen science web platform for big data ecology. This platform can then be deployed by other organisations to crowdsource their research. Lastly, I will report key challenges faced by this approach, plus the potential for future work on integrating state of the art technology into citizen science ecology.

This paper presents the evaluation of 36 convolutional neural network (CNN) models, which were trained on the same dataset (ImageNet). The aim of this research was to evaluate the performance of pre-trained models on the binary classification of images in a “real-world” application. The classification of wildlife images was the use case, in particular, those of the Eurasian lynx (lat. “Lynx lynx”), which were collected by camera traps in various locations in Croatia. The collected images varied greatly in terms of image quality, while the dataset itself was highly imbalanced in terms of the percentage of images that depicted lynxes.

Automated monitoring technologies can increase the efficiency of ecological data collection and support data‐driven conservation. Camera traps coupled with infrared light barriers can be used to monitor temperate‐zone bat assemblages at underground hibernacula, where thousands of individuals of multiple species can aggregate in winter. However, the broad‐scale adoption of such photo‐monitoring techniques is limited by the time‐consuming bottleneck of manual image processing. Here, we present BatNet, an open‐source, deep learning‐based tool for automated identification of 13 European bat species from camera trap images. BatNet includes a user‐friendly graphical interface, where it can be retrained to identify new bat species or to create site‐specific models to improve detection accuracy at new sites. Model accuracy was evaluated on images from both trained and untrained sites, and in an ecological context, where community‐ and species‐level metrics (species diversity, relative abundance, and species‐level activity patterns) were compared between human experts and BatNet. At trained sites, model performance was high across all species (F1‐score: 0.98–1). At untrained sites, overall classification accuracy remained high (96.7–98.2%), when camera placement was comparable to the training images (<3 m from the entrance; <45° angle relative to the opening). For atypical camera placements (>3 m or >45° angle), retraining the detector model with 500 site‐specific annotations achieved an accuracy of over 95% at all sites. In the ecological case study, all investigated metrics were nearly identical between human experts and BatNet. Finally, we exemplify the ability to retrain BatNet to identify a new bat species, achieving an F1‐score of 0.99 while maintaining high classification accuracy for all original species. BatNet can be implemented directly to scale up the deployment of camera traps in Europe and enhance bat population monitoring. Moreover, the pretrained model can serve as a baseline for transfer learning to automatize the image‐based identification of bat species worldwide.

Pangolins in global camera trap data: Implications for ecological monitoring

Citizen Science (CS) is a research approach that has become popular in recent years and offers innovative potential for dialect research in ornithology. As the scepticism about CS data is still widespread, we analysed the development of a 3-year CS project based on the song of the Common Nightingale (Luscinia megarhynchos) to share best practices and lessons learned. We focused on the data scope, individual engagement, spatial distribution and species misidentifications from recordings generated before (2018, 2019) and during the COVID-19 outbreak (2020) with a smartphone using the ‘Naturblick’ app. The number of nightingale song recordings and individual engagement increased steadily and peaked in the season during the pandemic. 13,991 nightingale song recordings were generated by anonymous (64%) and non-anonymous participants (36%). As the project developed, the spatial distribution of recordings expanded (from Berlin based to nationwide). The rates of species misidentifications were low, decreased in the course of the project (10–1%) and were mainly affected by vocal similarities with other bird species. This study further showed that community engagement and data quality were not directly affected by dissemination activities, but that the former was influenced by external factors and the latter benefited from the app. We conclude that CS projects using smartphone apps with an integrated pattern recognition algorithm are well suited to support bioacoustic research in ornithology. Based on our findings, we recommend setting up CS projects over the long term to build an engaged community which generates high data quality for robust scientific conclusions.

As camera trapping has become a standard practice in wildlife ecology, developing techniques to extract additional information from images will increase the utility of generated data. Despite rapid advancements in camera trapping practices, methods for estimating animal size or distance from the camera using captured images have not been standardized. Deriving animal sizes directly from images creates opportunities to collect wildlife metrics such as growth rates or changes in body condition. Distances to animals may be used to quantify important aspects of sampling design such as the effective area sampled or distribution of animals in the camera's field‐of‐view. We present a method of using pixel measurements in an image to estimate animal size or distance from the camera using a conceptual model in photogrammetry known as the ‘pinhole camera model’. We evaluated the performance of this approach both using stationary three‐dimensional animal targets and in a field setting using live captive reindeer Rangifer tarandus ranging in size and distance from the camera. We found total mean relative error of estimated animal sizes or distances from the cameras in our simulation was −3.0% and 3.3% and in our field setting was −8.6% and 10.5%, respectively. In our simulation, mean relative error of size or distance estimates were not statistically different between image settings within camera models, between camera models or between the measured dimension used in calculations. We provide recommendations for applying the pinhole camera model in a wildlife camera trapping context. Our approach of using the pinhole camera model to estimate animal size or distance from the camera produced robust estimates using a single image while remaining easy to implement and generalizable to different camera trap models and installations, thus enhancing its utility for a variety of camera trap applications and expanding opportunities to use camera trap images in novel ways.

Camera trapping studies often focus on estimating population density, which is critical for managing wild populations. Density estimators typically require unique markers such as stripe patterns to identify individuals but most animals do not have such markings. The spatial capture model (SC model; Chandler & Royle, 2013) estimates density without individual identification but lacks sufficient field testing. Here, both the SC model and calibrated capture rate indices were compared against ten sessions of live trapping data on kangaroo rats (Dipodomys spp). These camera and live trapping data were combined in a joint-likelihood model to further compare the two methods. From these comparisons, the factors governing the SC model’s success were scrutinized. Additionally, a method for estimating missed captures was developed and tested here. Regressions comparing live trapping density to the SC model density and capture rate were significant only for the capture rate comparison. Missed image rate had a significant relationship with ambient nighttime temperatures but only marginally improved the capture rate index calibration. Results showed the SC model was highly sensitive to deviations from its movement model, producing potentially misleading results. The model may be effective only when movement assumptions hold. Several factors such as camera coverage area, microhabitat, and burrow locations could be incorporated into the SC model density estimation process to improve precision and inference.

SummaryCamera trapping, paired with analytical methods for estimating occupancy, abundance and other ecological parameters, can yield information with direct consequences for wildlife management and conservation. Although ecological information is the primary target of most camera trap studies, detectability influences every aspect from design to interpretation.Concepts and methods of time‐to‐event analysis are directly applicable to camera trapping, yet this statistical field has thus far been ignored as a way to analyse photographic capture data. To illustrate the use of time‐to‐event statistics and to better understand how photographic evidence accumulates, we explored patterns in two related measures of detectability: detection probability and time to detection. We analysed camera trap data for three sympatric carnivores (snow leopard, red fox and stone marten) in the mountains of northern Pakistan and tested predictions about patterns in detectability across species, sites and time.We found species‐specific differences in the magnitude of detectability and the factors influencing it, reinforcing the need to consider determinants of detectability in study design and to account for them during analysis. Photographic detectability of snow leopard was noticeably lower than that of red fox, but comparable to detectability of stone marten. Site‐specific attributes such as the presence of carnivore sign (snow leopard), terrain (snow leopard and red fox) and application of lures (red fox) influenced detectability. For the most part, detection probability was constant over time.Species‐specific differences in factors determining detectability make camera trap studies targeting multiple species particularly vulnerable to misinterpretation if the hierarchical origin of the data is ignored. Investigators should consider not only the magnitude of detectability, but also the shape of the curve describing the cumulative process of photographic detection, as this has consequences for both determining survey effort and the selection of analytical models. Weighted time‐to‐event analysis can complement occupancy analysis and other hierarchical methods by providing additional tools for exploring camera trap data and testing hypotheses regarding the temporal aspect of photographic evidence accumulation.

Camera traps are widely used to monitor the biodiversity and population density of animal species. Camera trap images are usually taken in bursts, and the animal counting problem for a sequence of camera trap images is also an important part of evaluating animal population density. In this paper, two new animal counting methods based on Microsoft MegaDetector V 4 have been proposed. FilterDetector uses different filters with bounding box ensemble algorithms to achieve more accurate bounding box detection. DLEDetector is an ensemble method that uses two base deep learning models to correct and enhance the detection result of MegaDetector. Our experimental results in iWildCam 2022 competition test dataset show that both methods outperformed the best method in iWildCam 2021 and the baseline method based on MegaDetector V 4 in iWildCam 2022 competition by 9.09% and 6.44%, respectively, and ranked first and third in the competition.

Birds are important indicators for monitoring both biodiversity and habitat health; they also play a crucial role in ecosystem management. Declines in bird populations can result in reduced ecosystem services, including seed dispersal, pollination and pest control. Accurate and long-term monitoring of birds to identify species of concern while measuring the success of conservation interventions is essential for ecologists. However, monitoring is time-consuming, costly and often difficult to manage over long durations and at meaningfully large spatial scales. Technology such as camera traps, acoustic monitors and drones provide methods for non-invasive monitoring. There are two main problems with using camera traps for monitoring: (a) cameras generate many images, making it difficult to process and analyse the data in a timely manner; and (b) the high proportion of false positives hinders the processing and analysis for reporting. In this paper, we outline an approach for overcoming these issues by utilising deep learning for real-time classification of bird species and automated removal of false positives in camera trap data. Images are classified in real-time using a Faster-RCNN architecture. Images are transmitted over 3/4G cameras and processed using Graphical Processing Units (GPUs) to provide conservationists with key detection metrics, thereby removing the requirement for manual observations. Our models achieved an average sensitivity of 88.79%, a specificity of 98.16% and accuracy of 96.71%. This demonstrates the effectiveness of using deep learning for automatic bird monitoring.

Feral cats are a key threat to many Australian native fauna, with camera traps increasingly used to identify individuals for evaluation of management actions. However, observer bias and camera trap settings can affect individual identification rates. We compared feral cat individual identification by two observers with extremes of experience. Arrays of 39–50 camera traps were deployed continuously for 22 months at four sites in the Western Division of New South Wales. Where possible, feral cats were individually identified from phenotypic characteristics by an expert and naïve lay observer. We obtained 10 465 feral cat images, with 72 cats individually identified across the sites. The experienced observer attributed more feral cat events to a known individual compared with the lay observer (21.3 vs 12.9%, respectively). Forty three percent of cat images were similarly tagged by both observers. Daytime events yielded higher identification rates and match success (28.1 vs 19.5 and 17.9 vs 11.8% for day vs night events for the expert and lay observer, respectively). Lack of congruence between observers, combined with a small number of events where cats could be individually identified, and differences in identification accuracy over time and between sites, makes estimation of detection probabilities and errors difficult.

ABSTRACTThe Grévy's zebra, an endangered species native to Kenya and southern Ethiopia, has been the target of sustained conservation efforts in recent years. Accurately monitoring Grévy's zebra populations is essential for ecologists to evaluate ongoing conservation initiatives. Recently, in both 2016 and 2018, a full census of the Grévy's zebra population was enabled by the Great Grévy's Rally (GGR), a citizen science event that combines teams of volunteers to capture data with computer vision algorithms that help experts estimate the number of individuals in the population. A complementary, scalable, cost‐effective and long‐term Grévy's population monitoring approach involves deploying a network of camera traps, which we have done at the Mpala Research Centre in Laikipia County, Kenya. In both scenarios, a substantial majority of the images of zebras are not usable for individual identification due to ‘in‐the‐wild’ imaging conditions—occlusions from vegetation or other animals, oblique views, low image quality and animals that appear in the far background and are thus too small to identify. Camera trap images, without an intelligent human photographer to select the framing and focus on the animals of interest, are of even poorer quality, with high rates of occlusion and high spatiotemporal similarity within image bursts. We employ an image filtering pipeline incorporating animal detection, species identification, viewpoint estimation, quality evaluation and temporal subsampling to compensate for these factors and obtain individual crops from camera trap and GGR images of suitable quality for re‐ID. We then employ the local clusterings and their alternatives (LCA) algorithm, a hybrid computer vision and graph clustering method for animal re‐ID, on the resulting high‐quality crops. Our method processed images taken during GGR‐16 and GGR‐18 in Meru County, Kenya, into 4142 highly comparable annotations, requiring only 120 contrastive same‐vs‐different‐individual decisions from a human reviewer to produce a population estimate of 349 individuals (within 4.6 of the ground truth count in Meru County). Our method also efficiently processed 8.9M unlabelled camera trap images from 70 camera traps at Mpala over 2 years into 685 encounters of 173 unique individuals, requiring only 331 contrastive decisions from a human reviewer.