Pick a Card Project Update: Deploying Hardware for TinyML Model

— Written by Nour Chahine —

Before I start discussing my updates, I have an announcement to make: I solemnly swear that I will solely be working with SSVEPs for the remainder of my project!

Over the past few weeks, I worked on feeding different forms of data into the neural network. I mainly applied two approaches: the power spectral density vector method and the notch filter knockout vector method. Both methods rely on quantifying the EEG response relative to the flashing frequencies of the cards.

Power Spectral Density Vector

The power spectral density vector method involves computing the power spectral density of the EEG signal, and then making a vector of the total power of the signal at each of the target frequencies.

In the plot above, we can see the average power spectral density plot of my subject and Fellow Ariyana looking at a card flashing at 7.5 Hz (in red), overlayed with the individual epochs. Notice the peak at 7.5 Hz?

To quantify this information, I first compute the total power at each target frequency + 0.1 Hz: p_f1, p_f2, p_f3, and p_f4. Then I divide by the total power to normalize. 

In this example, the target frequencies are f₁ = 4.3 Hz, f₂ = 6.66 Hz, f₃ = 7.5 Hz, and f₄ = 8.75 Hz. The resulting power spectral density vector obtained {0.15, 0.05, 0.61, 0.17} shows that p_f3 has the greatest value. This means that the subject was most likely looking at the card flashing at f₃= 7.5 Hz

Notch Filter Knockout Vector

The notch filter knockout vector conveys information similar to that of the PSD vector, but it is obtained differently. A notch filter is a type of band-stop filter, which is a filter that attenuates frequencies within a specific range while passing all other frequencies unaltered. If I apply a notch filter of 5 Hz to a signal, then only the 5 Hz components of my signal are attenuated, while the others remain unchanged. How can this be useful?

The working principle here is that I obtain the total power of the signal, P_total. Then, I apply a notch filter to the signal at each of the target frequencies. Afterwards, I compute the power of the notched signals at each notch frequency, P_f1 P_f2 P_f3 and P_f4. By subtracting the notched signal power from the total power, you can determine how much the notched frequency contributed to the power. In other words, you can tell how much f₁, f₂, f₃, and f₄ each contribute to the total power of the signal. This information can be represented in a vector similar to the PSD vector:

The same data from the PSD example generates the following notch filter knockout vector:

We can deduce that the frequency f₃ = 7.5 Hz contributed the most to the total power of the signal. Therefore, the subject was most likely looking at the card flashing at frequency f_3. 

Both vectors convey similar information, but do they train the machine learning model in a similar manner?

That is what I tested next. Using Edge Impulse, I fed PSD vectors and notch knockout vectors that I computed from all of my collected data so far into a neural network. The table below highlights the training and testing accuracies obtained using the PSD vector and the Notch Knockout Vector as a model input.

Evidently, the PSD vector performed better. The Notch knockout vector is still useful for fast computations, though. 

I am now at the stage of hardware deployment. The goal is to deploy the best working model onto the Wio Terminal so that it is able to perform live classification when hooked up to an EEG channel. Before deploying the ML model, I started by hardcoding the classification process to verify that live classification can happen to begin with. I used the notch knockout method in combination with simple thresholding, and the code has been working for test signals from the function generator. 

Hopefully, it should also work with EEG signals. After I test this, I will deploy the ML model in combination with a web app I am developing on the side. Excited to get to the finish line!