Computer Recognizes Whale Tails
by Annette Kik, Eric Pauwels and Elena Ranguelova
How many whales are there in the ocean and how do they migrate? To answer these questions it is important to identify individual animals. Until now, biologists have tried to search by hand through vast numbers of photographs. In the European EUROPHLUKES project that ended in 2004, CWI researchers developed a method for semi-automatic pattern recognition of whale tails and dorsal fins. This is a first step towards the automatic photo-identification of individual animals.
Governments and organizations such as the International Whaling Commission want to know more about whale populations in order to protect biodiversity and to make informed choices about possible hunting within certain limits. Using parameters such as the number and age of female animals, scientists can estimate mathematically how the population will evolve. Identification is an important tool in collecting these data for stock management. One of the most convenient ways is photo-identification. It is less intrusive than harpooning whales for a sample of DNA, and is more extensive because of the potentially large collections of photographs from biologists, sailors and tourists.
The EUROPHLUKES project commenced in 2001. Its brief was to develop a photo-ID system and database for cetaceans - whales, dolphins and porpoises. The objective was to be able to identify if a particular cetacean had already been photographed, and if so, where and when. The network comprises more than forty partners and participants, mostly marine biologists. It is coordinated by the Universiteit Leiden and is funded by the Fifth Framework of the European Union. To deal with specific computer vision problems, researchers from CWI’s Signals and Images group, a member of ERCIM’s WG on Image and Video Understanding, were invited to the team.
Watershed Method
Human beings can easily identify individual whales, due to the unique spots and scars on the animals’ skin and the shape and indentations of their tails or dorsal fins. However, they can only compare a few pictures at a time, whereas photographic collections are growing rapidly. A computer on the other hand, can quickly compare thousands of pictures in databases but it has great difficulties in spotting similarities. For instance, a tail can look different when it is turned, waves can occlude specific marks and the picture quality can vary enormously. In a black and white picture it is not always easy to distinguish between tail and water. In addition, a computer is not intelligent so it cannot immediately recognize the most important marks: it has to compare all features, big and small.
To recognize individual characteristics semi-automatically, CWI researchers combined and applied several mathematical techniques: image segmentation, contour and feature extraction and finally, comparison of these data with an image database. First, the grey-level of the image is represented as a three-dimensional picture: white is high, darker is lower. At the edge between sea and tail the difference in grey-scale will be large, or in other words, the gradient will be high. The picture of this gradient can be viewed as a ‘topological surface’: It has mountains, plains and valleys. In the so-called watershed method, virtual water floods this surface. When the water is so high that two lakes in valleys are about to merge, a virtual watershed is placed. This procedure has been programmed in MATLAB, a technical computing language, allowing the computer to robustly identify regions of similar grey scales and thus extract the contours of the tail or dorsal fin.
Spots and Scars
To compare spots and scars on a tail photographed from nearby with one that is photographed from a larger distance or a different viewpoint, the researchers attach a virtual grid to the tail that is always the same, or in other words ‘invariant under affine transformations’. This grid is used as a coordinate system. To define it, the user specifies three anatomical points in the tail: the middle notch of the fluke and both its tips. Assuming that the tail is not too flexible, parallelism and relative distances can be used to divide the tail into a large number of small regions (eg thirty). The proportion of spots to background in each of these regions can then be represented in a 30-dimensional vector, which can be compared with other vectors in the database of identified animals.
The performance can be improved by combining the above feature vector with a more detailed mathematical description of specific, salient spots and scars on the animal. With morphological processing these marks can be found - both their centre of gravity with respect to the grid and a computed approximating ellipse. Using these data, the computer gives a top list of potential matches with pictures from the database. The user can then pick the actual match from this shortlist or confirm that there is no matching animal in the database. This method makes it possible to compare pictures with larger cetacean databases.
Links:
http://www.europhlukes.net
http://homepages.cwi.nl/~ely/projects.htm
http://homepages.cwi.nl/~pauwels/PNA4.1.html
Please contact:
Eric Pauwels, CWI, The Netherlands
Tel: +31 20 592 4225
E-mail: Eric.Pauwels
cwi.nl
