Automated deep learning analysis of purple martin videos depicting incubation and provisioning

Deep learning models have been developed to automatically analyze video clips of purple martin nesting behavior. Two separate models have been constructed, one for incubation and one for provisioning. The incubation model is a simple two class model that analyzes the videos to determine if an adult purple martin is incubating the eggs/young nestlings or not. The model is a Keras/Tensor Flow convolutional neural network (CNN) trained with 12 thousand still images and achieves a validation data set accuracy of 99.5% percent on still images. A comparison of the results of the automated video analysis with sample validation videos viewed manually shows good agreement; the model approaches human accuracy. Some conclusions from the incubation analysis will be discussed. The provisioning analysis requires a much more complex 3 class model which must distinguish between zero, one parent or both parents on the nest. With training sets including 26 thousand images the CNN model demonstrates a validation set accuracy of 99% on the still images. However, the actual video analysis presents difficulties. Several different CNN models have been tried but results were similar. The best results to date on analyzing the videos for provisioning events have been 88% accuracy with 10% false positives. A discussion of the conclusions from the provisioning model and model analysis will be presented.