Backyard Brains Logo

Neuroscience for Everyone!

+1 (855) GET-SPIKES (855-438-7745)


items ()

Octopus Wrestling and Computer Vision

 

Hello again my faithful viewers, and thanks for tuning in for another exciting octopus themed blog post. As always I am your host Ilya Chugunov, but today I’ve come with sad news; all good things must come to an end, and this marks the end of my summer research here with Backyard Brains. Now’s a time to grab a hot cocoa and reminisce on what we’ve learned and talk about what there’s still left to do.

First and foremost, if you haven’t already had the chance to look at my previous blog posts, you can see them here:

Octopus Learning and Behavior

Studying Aggressive Behavior in Octopodes

Now let’s recap and break this down into some conversational dialogue.

First, we found out, rather accidentally, that if left together our Bimacs will wrestle each other to assert dominance. This gave us the idea of using computer vision to gather data for analysis with the hope that we could identify some interesting features within their behaviors.

First I built my acrylic setup to record the octopuses doing their thing, making sure to have even lighting and a stable setup for my Go Pro so that the code didn’t just explode from all the variability.


The first, and most classic, behavior found in our trials with the Bimacs were “bouts”, which were little sumo-wrestling fights where each octopus tried to push the other around as far as possible; these were common when both the octopuses were excited and lasted about 5 seconds each.

The second curious behavior found was the “poke”, where one octopus wanted to provoke a real fight, but the other just wasn’t feeling it. The more excited octopus would waltz up to the lazy-bones and just briefly tap him with an arm before jetting off across the chamber.

I noticed that in both the bouts and the pokes, right as the distance between the two octopuses closed, and they made contact, the angle between them rapidly decreased too. They would approach each other sideways (almost backwards at times), then rapidly spin around right as they got close to poke/fight. In the poke behavior, the offending octopus would then spin back around and jet off, while in the bout behavior they’d just stay locked face to face.

Another notable thing our octopus do in their fighting ritual is change colors. As I assume you already know, these guys are covered in chromatophores and seem to flash bright black as they go on the offensive (Can you tell who the attacker is in that picture above?).

The poke behaviour elicited the same response, twice! The first bump was the attacked octopus darkening as the poking octopus approached it and second was the poking octopus turning a dark brown as he squirted away.

“But Ilya, how in the world do you process so much video? And how do you know when the fight starts in the first place?”
Why thanks for the question, hypothetical reader. I use a mess of MOG (Mixture of Gaussians) background subtraction, erosion, and band-pass filtering combined with the OpenCV convex hull functions to find the general outlines of the octopus, and then I check if they’re two separate blobs, or one combined megaoctopus. If they’re 2 blobs, they’re not in contact, and vice versa, so now it’s easy to define first contact and a bout vs a poke (long contact, short contact).

 

Using a simple windowing function and a pretty boring logistic regression, we can take a bunch of our video clips of octopus fights where we’ve already classified when a fight occurs, and from them predict a point of contact in a new video we feed into the algorithm. This is where the concept of machine learning starts to play into the project, letting a program learn from previous octopus video to predict what will happen in new octopus video.

I’ve compiled my research results and created a poster which I presented at a University of Michigan symposium.

What’s next?

For me, Canada. Heading up to Montreal next week.

In general, my code is up on my GitHub and is completely open source, so anyone is welcome to make changes to it, take it in whatever directions they want; you don’t even have to use it on octopus if you don’t want. 

Now for some musings…

I’m excited about computer vision. Historically, behavioral studies involve a lot of humans watching animals, recording specific events (like eating a certain food and when), or interpreting their behavior. This is not only time consuming, but also unscientific. In these studies, there needs to be redundancy. Multiple people need to record the events. Then, that data needs to be interpreted statistically to ensure that, on average, the interpretations are consistent between different observers. As you can see already, it is challenging. Computer Vision programs are changing this!
 
By taking the humans out of the equation, you remove chances for bias, for missed behaviors or interaction, or fudged results. Computer vision techniques can be used to comb over hundreds of hours of video footage, quickly providing researchers with quantifiable results. There are certainly still some behavioral studies that require human discretion, for instance, was a touch affectionate or aggressive, but for many researchers, computer vision is the future.

I think there’s a lot still to be done with computer vision and behavioral analysis, and this summer research was just me dipping my foot into the pool. There is much more data we can draw from the same video I was working with, tentacle position and length, how curled the octopus’s arms were, maybe even their heartrate could be extrapolated with enough clever coding. As I continue onward in whatever field of STEM I find myself in next, I hope to keep throwing computational power at problems that don’t seem like they even need a computer, because who knows, maybe they do.

I’ll leave you with some boring philosophy. No one, not a single scientist, knows for certain what the next big thing is going to be. No one knows when or where the next technological revolution is going to be, no one knows if the next world-changing invention is going to be made in a million dollar Elon Musk laboratory, or at 3am by a hungover student in their dorm room. So just know that when you read a blog post like this, about an 11 week undergrad project, even it has the chance to be something big; not all scientific breakthroughs are made by bearded dudes in lab coats, they could be made by you.


Finalizing a No-Harm, Dragonfly Visual Neuron Recording Prep

Welp, it’s my last day of work here at Backyard Brains! It’s been a fun 11 weeks with my fellow interns, but all things must end. Last week we wrapped up all the TED filming for our mini series episodes. I had a great time, and I’m really looking forward to seeing the final result. 

The dragonfly project ended in a good place; we have a good amount of data from the final setup and succeeded in developing a replicable, recoverable prep. I take a dragonfly that has been in the fridge for a few hours and carefully restrain its wings back with a “helping hands” clamp covered in cloth. This prevents damage to the wings. Then I wrap the dragonfly with a cloth, leaving only its head exposed; this is so the dragonfly doesn’t move and pull out the electrode wires during recording. The cloth is taped and pinned into the clamp’s cloth to hold it in place. Then, I still use silly putty to place and hold the electrode’s stick in place so the wires don’t come out when I prepare the recording electrodes and move the Dragonfly later.We modified one of the Backyard Brains Micromanipulator electrodes so that instead of a grounding pin, we use a reference electrode. Then, onto the dragonfly, I place the two electrode wires on either side of a single, exposed ventral nerve cord.I also made a few new stimuli, all on generic size paper. One had a fake plastic fly glued to the middle, and the other four I drew various sizes of dots in the center: 3mm, 7mm, 2.3cm, and 9cm in diameter.

I waved these papers by hand left and right, up and down, and even switched them out in the same recording to compare size preferences, not just direction. Besides just seeing a reaction, I’m interested in seeing the directionality of response.

This indicates that there are certain neurons within the dragonfly’s nervous system, like the target-selective descending neurons (TSDNs), that help the dragonfly differentiate, in an almost mechanical way, what direction a target is moving. This has the advantage of removing some “post processing” of the information, allowing the dragonfly to react quicker and hunt its prey more efficiently. I had success in seeing this kind of evoked response in my trials, which was a great success for the project.

As you can see in the results above, as I improved my prep and experimented with new electrodes, I began to see better results. By the end, I was seeing responses in most of my preps. I began to observe a directional bias more frequently and began seeing more evidence of a size discriminate response. By the time we presented our projects via a poster presentation on August 2nd, I had totaled my data into success rates of getting certain kinds of signals using this final prep I developed, giving students who repeat this experiment an idea of how difficult or easy it will be to see different responses.

Further, we are hoping to publish these results, but in order to do so, the stimuli cannot be moved by hand; the human error of timing the event markers in Spike Recorder with the movement of the stimulus is not accurate or consistent enough for a peer-reviewed journal. We built a servo-motor rig that moves the paper back and forth while simultaneously sending the event markers to the software. The rig has a lot of problems, and I ran out of time to work on it, so if my project is continued next summer, the rig should be the focus to really iron out the automation and precision of stimulus delivery.

That’s all from me! Thanks for reading. Dragonfly girl, signing off.


SpikerBox Assists in Ugandan “Teacher Tools Workshop”

This post comes from our friend Dr. Thomas Tagoe, who was gracious enough to allow us to host his writing on our blog! Dr. Tagoe assisted with a recent neuroscience “Teacher Training Workshop” where dozens of teachers learned strategies for teaching neuroscience to their students. Included in the workshop was the Backyard Brains Neuron SpikerBox! We’re always excited to see teachers using our gear to bring live, hands-on neuroscience to students around the world!

This article, along with other articles written by Dr. Tagoe can be found at (http://ghscientific.com/neuroscientists-go-back-school/ ). Dr. Tagoe writes that this website is used to “capture all our public, STEM engagement activities and to promote the works of others working to enhance the uptake of STEM within Ghana.”

NEUROSCIENCE TEACHING TOOLS WORKSHOP – ENTEBBE, UGANDA

“When teaching is learner-centered, the role of the teacher changes, …. They are no longer the main performer, the one with the most lines, or the one working harder than everyone else to make it all happen”. The day I read this, I knew I will never be the same teacher again. It wasn’t just the text, but the environment within which it was read and it just so happens that I had to travel over 5,000 km to undergo this transformation.

On Sunday 6th June at exactly 3.06am, I touched down in Uganda – (sorry Nigeria, I had really wanted you to be my first destination in Africa but it was not meant to be).  The airport was simple, the drive to the hotel uneventful and my sleep was dreamless. Daylight brought with it activity of the calm type; a visit to the local church with colleagues, banter with the hotel staff as we sought to identify the English name for a local bird species, the Kahloree also known as the Marabou Stork.

Image of the Marabou Stork on the banks of Lake Victoria

Evening was met with conversations on the banks of the Lake Victoria which the locals insisted was a beach. I didn’t blame them; any land locked country will mistake the vast nature of this lake for the sea. The day was pleasant in every sense of the word and I will come to appreciate these events on my first day in Uganda as the calm before the storm, after all, I was not in Uganda for pleasure.


Hello Uganda

I was in Entebbe – Uganda to join 25 other neuroscientists from across Africa for an annual teaching tools workshop. In its 10th year, this week-long workshop brought together neuroscientists from Ghana, Nigeria, South Africa, Zambia, Malawi, Zimbabwe, Senegal, Botswana, Ethiopia, and Cameroon. The purpose of the workshop was to enhance the pedagogical skills of participants in the field of neuroscience and introduce tools to enhance the delivery of content.

The neuroscience teaching tools workshop was the brainchild of Professors Sharon Juliano, Janis Weeks, John Martin, Nilesh Patel and Evelyn Sernagor. Over the past 10 years, these 5 highly accomplished neuroscientists from the USA, UK and Kenya have been sharing their wealth of knowledge and teaching experience with fellow neuroscientists on a continent where the study and understanding of brain structure and function is still a young field.

Professor’s Sharon Juliano and Evelyn Sernagor with a workshop participant

Over the years, many others have come and gone in support of their efforts but these five remained a constant. This year, the team was supported by Drs. Sadiq Yusuf, Musa Mubandla, Rufus Akinyemi, David Fotsing and Professor Melissa Coleman. With funding from the International Brain Research Organisation (IBRO), the International Society of Neuroscience (ISN) and the Grass Foundation, the10th Teaching Tools Workshop soon proved to be the most successful yet.

What happens at the Teaching Tools Workshop?

The workshop began with a session on how to be an effective teacher with particular emphasis on learner-centered approaches to delivering a quality lesson. Having already been prescribed reading from the book “Learner-Centered Teaching” by Maryellen Weimer, we were in a position to discuss these teaching strategies in light of our various teaching environments and experiences. This session was soon followed by back to back content specific sessions on Introduction to the brain and neurotransmission and Principles of Electrophysiology. With that, day 1 came to an end as participants and facilitators reflected on the day’s content over cocktails.

Ongoing workshop session

The next four days saw much of the same, – minus the cocktails. We received a mixture of content specific lectures from facilitators on subjects such as The olfactory system, Higher order cortical function, Motor systems and The thalamus. Everyone present took inspiration from the teaching methods which the facilitators employed. In particular, the neuroanatomists among us who felt they had been short-changed by their own lecturers, now believed they too had been short changing their students – content was so well broken-down. Content sessions were further interspersed with pedagogy sessions on Presentation skills, Classroom management strategies and Assessment strategies. My personal favourite was the use of color coded flashcards as a low tech alternative to Clickers.

Tools for learner centered teaching

All these sessions were built around getting students to own the learning process. Prof. Janis Weeks shared how at that start of her course, students are involved in decided the weighting of course activities towards their final grade. A session dedicated to Online learning saw Dr. Sadiq Yusuf share low cost alternatives to digital tools for enhancing communication and assessment. Due to all these tools, a large portion of group discussions during the week were centered around how these concepts could be best utilised within the diverse contexts we found ourselves.  The highlight of the week was undoubtedly the Electrophysiology lab session led by Prof. Melissa Coleman which saw participants get familiar with the Spikerbox.

Professor Melissa Coleman overseeing demonstrations with the Spiker Box

For the non-electrophysiologists among us, visualizing electrical activity was a novel and enjoyable experience. For the electrophysiologists such as myself, it was a reminder of the simplicity which underlies a very powerful research tool. Either way, you can never get tired of seeing a cockroach leg dance to the beat of Michael Jackson’s “Beat It”.


The teaching tools legacy

No academic activity is complete without time allocated for merry-making. It was a time to reflect and affirm commitment towards neuroscience excellence in Africa. It was also a time to create a roadmap for how best we could contribute towards the advancement of neuroscience on the continent.

Section of TTW participants at the farewell dinner

I am proud to say that I belong to the 10th cohort of the neuroscience teaching tools workshop. Barely a month after our first meeting, members of this cohort have already started implementing changes in their lecture halls. I have personally had the opportunity to use the SpikerBox in public engagement activities to raise awareness around neuroscience.

Junior High School Students been introduced to the magic of the SpikerBox during a National Science Fair in Ghana

To crown it all off, the Ghana Neuroscience Society is creating a program which incorporates content from the Teaching Tools Workshop in an effort to reach a wider audience with the excellent pedagogy strategies shared. This will be in line with the society’s mission to improve access and excellence in neuroscience. Without a doubt, it’s a good time to be a neuroscientist in Africa and I look forward to meeting the 11th cohort of the Neuroscience Teaching Tools Workshop.