Auditory Attention Decoding Using Mobile EEG in Real-Life Environments

Auditory attention decoding is a process of identifying how the brain focuses on specific sounds in complex environments. A recent study used mobile EEG to examine how neural speech tracking works in both controlled laboratory settings and real-world scenarios. By comparing different listening conditions, the researchers explored how factors like background noise, motion, and task complexity influence our ability to track and attend to speech.

This blog provides an overview of the study’s findings and implications for understanding auditory processing in everyday life through auditory attention decoding.

Setting up Lab and Beyond-the-Lab conditions

In the experiment, researchers defined 2 identical lab sessions. In between they added a real-world, beyond-the-lab condition. Goal is to investigate the robustness of EEG-based speech tracking and how fatigue might influence the results.

The beyond-the-lab condition gives us insight into real-world effects to the study paradigm. In all three conditions, scientists introduced both single- and dual-speaker paradigms.

Café background is only present in the Lab1 and Lab2 conditions. For the beyond-the-lab condition, the participant is either sitting in a quite hallway or walking down a street. They also included 2 different streets: one was less busy than the other. This way it’s possible to explore if neural tracking systematically differs in quiet compared to busy streets.

After every narrative, participants had to answer 4 multiple-choice questions on the topic. This was to confirm they paid attention to the task. They also gave two subjective ratings on the block: their level of fatigue and exhaustion (1-7).

During the sessions, many data types were collected. Smarting PRO device was collecting EEG data.

For playing audio stimuli beyond the lab, authors used Portable hearing lab (PHL), and Neurobs Presentation to design the experiment. Periphery physiology was also monitored through a chest belt (breathing, ECG), but this data isn’t analyzed in the current study.

Steps to investigate EEG-based auditory attention decoding

Human brain tracks low-frequency fluctuations of speech. Because of this, there are methods to predict neural response to known stimuli (encoding), or reconstruct stimuli features from neural activity (decoding).

In this study, authors trained decoders to reconstruct both attended and ignored speech. To be precise, they tried to reconstruct the envelope of the audio stream.

They used 2 metrics to investigate the performance. First one is Pearson correlation between the reconstructed and original speech. Second one is the percentage of trials in which the reconstruction correlated more with attended, than ignored speech. So to put it simply – decoding accuracy.

One last interesting analysis they included in the study was to compare decoding results with 2 differently processed data. One is a less-elaborate processing pipeline. It includes basic filtering and removing bad trials in the data. Second pipeline includes ICA analysis and removing components that reflect non-EEG activity.

Questionnaire results

Comparing two lab conditions, it seems participants had no trouble following the narrative. As they managed to answer most of the questions. But during walking in the beyond-the-lab condition, that changes, especially in the dual-speaker case. It seems that environmental distractions had a great impact on their attention during the listening task.

In the two lab conditions, participants generally reported moderate tiredness and exhaustion. The highest levels are observed in dual-speaker scenarios with background noise.

In beyond-the-lab case, participants rated walking with dual-speaker condition as the most exhausting. Interestingly though, tiredness is lower during walking conditions. This could have to do with them being outside in fresh air and feeling more relaxed.

Neural tracking accuracy in case of subject-independent analysis

Let’s first take a look at the results for two lab conditions:

From the results, we can also see that neural tracking is better in case of single-speaker than dual. Also, we can see better neural tracking in case of attended speaker than the one they had to ignore. Decoding accuracy was unexpectedly better in dual-speaker conditions with cafeteria noise compared to those without. This could be because background noise made the attended speaker more perceptually distinct.

Now, for the beyond-the-lab condition:

Authors found significantly better neural tracking for single-speaker than dual conditions.

Walking conditions led to reduced neural tracking. Researchers think this might have to do with cognitive distractions from environmental stimuli. Also, the dual-task nature of walking and attentive listening, which can cause cognitive-motor interference.

It’s interesting to note that authors found the less-elaborate processing pipeline to be comparable with ICA pipeline. That is, the results they got were comparable. So they used the processing pipeline without ICA cleaning.

Final thoughts on real-world applications of EEG-based auditory attention decoding

Despite lower tracking performance in real-world conditions, EEG-based auditory attention decoding is possible. This paper demonstrates the robustness of neural speech tracking, even in dynamic settings. Future studies could focus on disentangling the effects of motor, perceptual, and cognitive demands. This could improve auditory attention decoding robustness in mobile scenarios.

You can read the original work here – https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2024.1483024/full