Video-Based Recognition of Aquatic Invasive Species Larvae Using Attention-LSTM Transformer

EasyChair Preprint no. 10957

Abstract

We present a video classification model for recognition of invasive and non-invasive larvae from water sample videos. Aquatic species like zebra and quagga mussels are invasive in United States waterways and cause ecological and environmental damage. In addition, there is a need for automated systems to detect and classify invasive and non-invasive species using a video-based system without any human supervision.

Many recent video recognition models are transformer-based and use a combination of spatial and temporal attention, often with large-scale pre-training. We present a model with a CNN-based patch encoder and transformer blocks consisting of temporal attention with LSTM that is end-to-end trainable and effective without pre-training. Based on detailed experiments, the Attention-LSTM model significantly improves over state-of-the-art video classification models, classifying invasive and non-invasive species with $99\%$ balanced accuracy.

Keyphrases: Attention-LSTM, invasive species, recognition, transformer, video recognition, zebra mussel

@Booklet{EasyChair:10957,
  author = {Shaif Chowdhury and Sadia Nasrin Tisha and Monica McGarrity and Greg Hamerly},
  title = {Video-Based Recognition of Aquatic Invasive Species Larvae Using Attention-LSTM Transformer},
  howpublished = {EasyChair Preprint no. 10957},

  year = {EasyChair, 2023}}