Dreaming With Your Eyes Open: Can Tiny ML Help You Deal with FOMO?
—Written by Ariyana Miri—
How much do you notice throughout your day? Are you paying attention to your surroundings? Are you present in every moment of your life? Odds are, the answer is no. As most adults know, the more we grow up, the more we seem to live on autopilot. We’ve got places to be and things to do, and no time to sit around marveling at the world around us.
This is the problem I’m hoping to solve with the help of Tiny ML and a 3D printer.
The concept behind FOMO glasses is to capture an image of what you would be seeing every time you blink. If you do the math (which I did), the average human spends anywhere from 2.7 to 4 years blinking. That’s years of your awake life spent with your eyes closed! This makes blinking an ideal metric for recording ALL the snapshots of your day you’re missing out on.
The goal of this project is to create a wearable prototype that is successfully able to take a photo every time a person blinks. This is done by using EOG signals recorded from the eye to detect when a blink has occurred. EOG signals, otherwise known as electrooculograms, measure the corneo-retinal standing potential between the front and back of the human eye.
After some thoughtful trial and error in the placement of the electrodes, I was able to receive a strong and unique blinking signal distinguishable from other eye movements. After that, I tested several popular filters to better enhance the signal and eliminate noise. It’s not that our eyes are lazy or inert, or that their signal is weak! The struggle with collecting data from the eye is due to how active that region of the body is. Even in resting states, we are constantly moving our eyes in rapid succession, causing large signals to appear in the EOG data. This is why thoughtful electrode placement and data processing are vital to an effective model.
Now that there has been good progress made on data collection and filtering, it is time for the AI/machine learning portion of the project. All of the fellows projects this year are working with small circuits running machine learning models, which is why we refer to it as “Tiny ML”.
For my project, I will be using a neural network to classify what eye movements are blinks, and what aren’t. As of now, I am using a classic neural network with default pooling layers and a spectrogram as a pre-processing block, though this is likely to change as I learn more about my data and continue to fine tune the model.
After this, it will be time for the real challenge: building the prototype. This will involve switching from wet electrodes to dry ones that can be attached to the glasses, designing and 3D printing the actual glasses, and mounting the Arduino Nano and camera onto the prototype in an appealing way. People should want to wear these, after all!
As of now, some designs I have include: A typical set of glasses with a camera on the right side, tech looking glasses that are flush to the face and stretch evenly around the eyes, and a single monocle-like design that can be clipped onto an existing pair of glasses. The more optimal design will become more apparent as the project progresses and the models are tested.
The concept of the FOMO glasses was introduced to me by Backyard Brains, and I immediately wanted to head the project. Not just because the concept itself is tons of fun, but for the implications the success of the project would have! Building something that works solely on eye movements is another testament to the capabilities of BCI (brain-computer interface) as it becomes more a part of our daily lives. We are already starting to be seen as part of our machines, with our phones acting as extensions of ourselves without which we feel vulnerable and incomplete.
However, the interfacing speed between us is still somewhat slow (for example, having to take your phone out, open up an app, and press a button in order to try and capture a moment before it’s too late, oftentimes missing the moment entirely in the process). BCI’s work to make processes like this more efficient and enjoyable. There is also a vast range of applications in regards to accessibility, especially for those who have limitations regarding mobility. In many cases, the eyes are one of, if not the only thing someone in these situations can move easily, so the more we can record and read signals from the eyes, the more applications we can make as tools for communication, entertainment, expressions of creativity…
Another cool part about working with Backyard Brains is getting to work with the teachers hoping to bring our projects into the classroom. These meetings have opened my eyes to applications of my project I hadn’t even considered. One teacher has a physiology and anatomy class where she hopes to use my project to help students understand the eyes’ blind spots and where our brain fills in information that isn’t there. She is hoping that by wearing the glasses and later having to recall information they believe they saw, students will be able to better understand how much information passes through their eyes that they’re not consciously aware of.
Another teacher had a similar idea in mind for her psychology class, where she hopes through a similar process that students will see how unreliable their memories are, either by failing to recall something they saw, or by recalling something that wasn’t there. These psychological reflections on the project are one’s I hadn’t considered before speaking to the teachers, and am excited to explore more in depth as the project continues to advance.
A student at Florida Southern College double majoring in computer science and communications with concentrations: artificial intelligence and machine learning, web and cloud computing, and digital media. In the future, I’d like to fuse these areas of study into making awesome things that people can enjoy and understand, improving the overall quality of life through technology, and aiding in the process of further integrating technology into our lives in a meaningful way. When not working, I like to engage in hobbies such as: painting, playing the piano, rock climbing, virtual reality and video games, and binge watching anything science fiction related.
I found Backyard Brains after the neuroscience professor at my school sent the application to the computer science department groupchat (it’s a small school). I had been in the process of looking at internships when I saw the application. Up until that point, most of the opportunities I’d seen were working for large companies to make some aspect of their business more efficient while working in a large group. While there is great value in this kind of work, it wasn’t what I was looking for. I wanted to work somewhere that promoted creativity. This is why I am very grateful to be a part of the Backyard Brains fellowship program this summer, and look forward to seeing everyone’s projects come to life.