An Automated, Real-Time Identification and Monitoring Instrument for Reef Fish Communities

This award is for the development of an integrated instrument for automated visual monitoring of coral reef fish communities in the field. An underwater image feed from multiple cameras stationed on a reef will be combined with a suite of software algorithms to identify each fish to species. In the ideal instrument, the output would be a series of identifications - a time series - that describes the population- and community-level dynamics of the reef's fish. In reality the output will likely also include images for testing and validation, and those representing identification uncertainties for later offline analysis. A lab-based development system will be succeeded in year two by a prototype that will be deployed first in a large, realistic aquarium, and then, in year three, in the field on a real reef. The proposed instrument represents an advance in several areas over previous efforts to monitor macroscopic organisms, and fish in particular. First, the use of more than one camera to provide multiple views of the scene allows the disambiguation of individuals that would otherwise be overlapping in a single camera's view, and correction for pose and location within the target volume. This in turn allows for much greater flexibility in placement and operation of the instrument compared to previous systems, which only worked in limited circumstances, or required the subject organisms to be channeled into unnatural structures that constrained pose. The proposed system will also be able to handle multiple organisms per image. Another advance is the use of a multitude of shape, pattern and color features for species discrimination, via a classifier system that is organized hierarchically in a manner similar to a traditional taxonomic key and designed to handle dozens or even hundreds of species.

Coral reefs and the complex communities they support are in decline worldwide, and the need to monitor those communities for signs of decline or collapse is urgent. Traditional methods (employing human divers) are too expensive and labor-intensive, and the data they return is too scarce. This camera-based instrument will allow a much finer level of resolution in data collection, and it will provide the fundamental technology for a worldwide system of reef monitoring stations. Furthermore, it is expected that the techniques developed and refined during this project, which include both the hardware and software, will be of general use in developing similar systems targeted towards other species in other habitats, (one example is migratory ducks and geese). It will therefore contribute to the goals of a number of recent global and US initiatives (e.g., NEON, GEOSS) to set up long-term monitoring systems for the environment, and indeed will be one of the first to automate real-time data collection on the abundance and dynamics of macroscopic organisms. In addition, undergrads will be involved through NSF programs for undergrad research; and visitors to the New York Aquarium will observe the aquarium phase of the project.