Take a step back and look into the future

Hello friends, this is Yifan again. As the end of the summer draws near, my summer research is also coming to a conclusion. The work I did over the summer was very different from what I expected. Since this is a wrap up post for an ongoing project, let us first go through what exactly I did this summer.

The above is the product flow graph for our MDP project. All the blue blocks and paths are what I worked on this summer. In previous posts I wrote about progress and accomplishments on everything except the bird detection algorithm.

In my second blog post, I talked about using a single HMM (hidden Markov model) to differentiate between a bird and a non-bird. One problem was that HMM classification takes a long time. Running HMM classification on a 30-minute long recording takes about 2 minute. Considering the fact that we need to analyze data much longer than that, we need to pre-process the recording, and ditch the less interesting parts. This way, we are only putting the interesting parts of the recording into the HMM classifier.

This figure is the runtime profile of running HMM on a full 30-minute long recording. The classification took about 2 minutes. After splitting out the interesting parts of the recording, we are only running classification on these short clips, hence reduces the runtime by a very large factor (see figure below).

One thing you might have noticed in these two graphs is that the runtime for wav_parse is also extremely long. Since there is almost no way to get around parsing the wav file itself, the time consumed here will always be a bottleneck for our algorithm. Instead of a better parsing function, I did the mature thing by blaming it all on python’s inherent performance issues. Jokes aside, I think eventually someone will need to deal with this problem, but I think optimization can wait for now.

This figure is the raw classification output using a model trained by 5 samples of a matching bird call. If the model thinks a window in the recording matches the model, it marks that window as 0, otherwise 1. Basically this mess tells us that in these 23 clips, only clip 9 and 10 does not contain the bird used to train the model.

One might ask, why don’t you have a plot or graph for this result? Don’t yell at me yet, I have my reasons… I literally have more than a hundred clips from one 30-minute recording. It’s easier for me to quickly go through the result if they are clustered together in a text file.

Although me and my mentor Stanislav had decided on trying out HMM to do the bird detection. The results aren’t very optimistic. There is the possibility that HMM is not a very good choice for this purpose after all, which means I might need to do more research to find a better solution for bird detection. Luckily, since songbird is an ongoing project, I will get my full team back again in September. Over this summer, I believed I have made some valuable contributions to this project, and hopefully that can help us achieve our initial goals and plans for this product.

This summer has been a wonderful time to me. I would like to thank all my mentors and fellows for their help along the way, it really meant a lot to me. Looking into the future, I definitely believe this project has more potential than just classifying birds, but for now I am ready to enjoy the rest of the summer in order to work harder when I come back to Ann Arbor in fall.

Listening In On Our Backyards

Acoustic Wildlife Recording promotes Citizen Science!

Here at Backyard Brains, we are all about citizen science, or the idea that the scientific community benefits from the collaboration with members of the general public for collecting and analyzing information about the natural world. Very DIY, very much the “for everyone” in our slogan. In 2017, Backyard Brains partnered with the University of Michigan’s Multidisciplinary Design Project (MDP) to focus neuroscience education on another kind of brain: birds! With the help of BYB, a team of undergraduate engineering students worked to develop a new kind of “Backyard Brain.” The idea was this: Create a low-cost device that could be deployed in backyards that would identify and record birdsongs!  This could be used to help track and log bird populations across the country, which is an important index of environmental health. Development of this project continued over the course of our 2017 summer fellowship , and that progress is detailed in Zach’s summer blog posts.  BYB and MDP will team up again for the project this year, with a new team and a new, expanded goal. But first, how did such a project come to mind? Naturally, it is the technological next step of a classic, “analog,” cataloging method…

The Audubon Christmas Bird Count

The National Audubon Society‘s annual “Christmas Bird Count” is perhaps the greatest example of democratized citizen science. Since 1900, volunteers have braved harsh, wintry conditions to help count and identify bird populations in their hometowns, as seen in Audubon’s photo above. These volunteers, from all across the country, then send in their findings, thus informing a national bird census.

The data gathered by initiatives like the Christmas Bird Count and Birdsong Identification project is incredibly important. Bird populations are very sensitive to environmental changes, making them a strong indicator of environmental health, stability, and possible effects of climate change. In this way, bird population trends can also be a lens to see our own world through.

This is the kind of citizen science that has inspired us, and others, to come up with devices which could help perform this task. Our work began in this field last year with the development of a “Birdsong Identification” device. The aim was to create a low cost, easily-distributed listening device which could be deployed to identify songbirds, and Zach’s project this summer started to do just that.

Birds, Rain, Wind, and More

The newest iteration of this project doesn’t stop at birdsongs. For 2018, the BYB-MDP partnership is looking to expand the reach of the project to create an acoustic environmental recorder that can also be listening for rainfall, wind, bats, coyotes, and other wildlife! There is a lot of information to be gleaned by turning an ear on our wilderness. Birdsongs are still on the menu, but with a new team (see above) and a new direction, the goal is to create a low-cost device which can be deployed and modified by both students and scientists to focus on whatever environmental indices interest them most!