What if someone told you that pigeons can tell the difference between artwork painted by Picasso and Monet? Or that they can discriminate between different facial expressions in humans and have been able to identify malignant versus benign breast cancer. You may be surprised and like me, want to find out how this humble bird can achieve these challenging tasks. Pigeons have good visual memory. However, researchers do not select birds that have specific degrees in arts or radiography to help distinguish between artwork and tumours, they have to provide the pigeons with examples of the data they ultimately had to categorise. The pigeons were given time to learn the differences between the data through several training trials in which they were rewarded for making the correct decision and then were given the opportunity to test what they had learned on data that they had not seen before. This type of task is known as classification since the data is being separated into categories.
There are many tasks in the research of animal behaviour that require the discrimination of data. Since we can’t all keep and train pigeons to carry out tasks such as identifying a bear from a deer in camera traps we need a faster way to carry out these tasks. This is where machine learning comes in. Machine learning is a branch of artificial intelligence that can be applied to solve many different tasks such as speech recognition, driving cars, facial expression recognition and is even behind netflix algorithms that tailor the content they suggest based on our watching data. Put simply, they are computer algorithms (mathematical procedures) that learn patterns in data they are provided with to make predictions about new data. They come in different flavours, for example there are algorithms that classify data, placing labels on them in the same way pigeons classified paintings. Others cluster data ‘deciding’’ a place to separate the input data based on their similarities and differences, for example, when a dog scent marks or not based on the accelerometer data placed on the dog’s pelvis. Or regression where you would like to predict an outcome based on data you have by exploring the strength of the mathematical relationship between the input and output data. For example, predicting the diving behaviour of seabirds just from features of GPS data such as the altitude and coverage data (the number of signals present and missed over specific time).
Identifying the difference between an image of a dog or cat may seem like an easy task for us since we use a holistic way of identifying these animals but computer algorithms ‘see’ this data differently and extract information (features) from the data. These could be pixel colours, edges of the image such as the side of the face, or more abstract data structures that we can only dream about. Data can be provided as raw data and the algorithm extracts what it deems important information and carry out the task without your input. These methods are termed unsupervised and you have little knowledge what it is about the data the algorithm learns structure from but are useful in carrying out a task quickly and using patterns in data we might not notice. If you had more ideas about the informative parts of the data you would like to evaluate, use a supervised approach. Here you can select and label which parts of the data go into the algorithm and this technique lends to certain research questions that have more context since you are interested in what the model learns from rather than just its output. For example we know these fur seals and these sea lions are foraging and grooming because of these informative features of accelerometer data.
Once we have decided what type of machine learning algorithm to use, like the pigeon, we provide it with some example data to learn from. The pigeon required 144 trials over the length of 15 days with food reinforcement when they made the correct distinction between malignant and benign tumours. Similarly the algorithm will need several iterations of learning the pattern in the data, hopefully hours and not days!, before it is applied to new data that it has not hasn’t seen before. The pigeon reached an accuracy of 85% when it was asked to label the tumours of new x-rays. Models that reach an accuracy similar to the pigeon on test data are deemed good and you can then use it to start exploring your data. However if the researchers also wanted the pigeon to help diagnose other cancers, they would be at a loss since the pigeon learned that only one type of cancer exists. This is an example of over-fitting, algorithms can become too restricted if they are provided with training data with little variation. Models are only as good as the example data they are provided with. A model that applies what it has learned well to new data is one that has been trained on data that reflects its diversity. If you wanted a model to identify a dog, you wouldn’t provide it with examples of just Labradors but rather an example of all dog breeds.
Why use machine learning to study animal behaviour? Machine learning is being used and applied in studies of animal behaviour and cognition. Often in these studies decisions have to be made about how to capture the behaviour of interest, describe it and analyse it. Some behaviours, such as the motion of mouse whiskers, may be imperceivable or at least challenging for a human observer to notice. I doubt that a person would be able to identify the different species of flying insect pests from a video of them but a group of researchers were able to train an model to do so which would help in targeting methods to protect crops. Machine learning also enables the automation of time consuming tasks such as identifying species from video data, can enable researchers to collect and analyse more data in shorter time frames and this might be useful if they want to respond to population declines rapidly with conservation efforts. Or labelling the behaviour of animals in videos automatically, reducing human error which is inevitable after staring at a screen for hours on end. We must not forget that these models learn from the data we provide them and do not see data in its context the same way that we do. Therefore, they can not replace how we as humans approach and solve problems but there is no doubt that they offer exciting new ways to delve deeper into phenomena in the animal kingdom.
Next time you come across the humble pigeon strolling across the pavement just take a minute to consider what it notices and might remember in its environment. Maybe your local pigeon even recognises you.
Bidder et al (2020). Monitoring canid scent marking in space and time using a biologging and machine learning approach. Scientific Reports. https://doi.org/10.1038/s41598-019-57198-w
Browning et al (2017). Predicting animal behaviour using deep learning: GPS data
alone accurately predict diving in seabirds. Methods in Ecology and Evolution. DOI: 10.1111/2041-210X.12926
Jan-Harm et al (2020). WhiskEras: A New Algorithm for Accurate Whisker Tracking. Frontiers in Cellular Neuroscience. https://doi.org/10.3389/fncel.2020.588445
Janelle Shane (2019). You Look Like a Thing and I Love You. https://www.janelleshane.com/book-you-look-like-a-thing
Kirkeby et al (2021). Advances in automatic identification of flying insects using optical sensors and machine learning. Scientific Reports. https://doi.org/10.1038/s41598-021-81005-0
Ladds et al (2016). Seeing It All: Evaluating Supervised Machine Learning Methods for the Classification of Diverse Otariid Behaviours. PlosOne.
Levenson et al (2015). Pigeons (Columba livia) as Trainable Observers of Pathology and Radiology Breast Cancer Images. PlosOne https://doi.org/10.1371/journal.pone.0141357
Watanabe et al (1995). Pigeons’ discrimination of paintings by Monet and Picasso. Journal of the Experimental Analysis of Behaviour. 10.1901/jeab.1995.63-165