Unlocking the Secrets of Multitasking: AI EEG analysis

  • neuroergonomics
  • AI

Multitasking analyzed by AI EEG approach.

In today’s fast-paced world, we’re all familiar with the hustle and bustle of juggling multiple tasks at once. Whether it’s answering emails while on a conference call or cooking dinner while helping the kids with homework, multitasking has become second nature to us.

But decoding mental workload (MWL) with EEG showed to be a hard task, even for AI and machine learning algorithms. Why is it so? Does multitasking, and engaging different brain areas at the same time confuse fancy AI algorithms? Should we ask AI EEG analysis different questions? In this paper we tried to test it.

Using MATB-II: The Multitasking Playground

The participants stepped into a virtual world where they were tasked with monitoring systems, steering targets, communicating with air traffic control, and managing resources—all at the same time. This virtual world was created using the Multi-Attribute Task Battery-II (MATB-II) from NASA. The tool simulates real-world multitasking conditions, providing a playground for researchers to study MWL and task performance.

It integrates four subtasks: System Monitoring (SYSM), Tracking (TRCK), Communication (COMM), and Resource Management (RMAN), each simulating different aspects of piloting activities such as:

  • SYSM: monitoring and reacting to system anomalies
  • TRCK: steering a target with a joystick
  • COMM: listening and responding to air traffic control voice commands
  • RMAN: managing resources during emergencies

The Experiment: Participants, EEG, and NASA-TLX

We recruited 50 participants, ranging from 18 to 39 years old, with zero experience in our task. After a quick practice session, they dove into two experimental sessions, each lasting just over 50 minutes, separated by a brief 10-minute rest. Each session included blocks of varying MATB-II subtask combinations to present different levels of task demand:

  • Passive watching (PW): no activity
  • Low Load (LL): all subtasks active except TRCK
  • Medium Load (ML): all subtasks active, increased task demands
  • Hard Load (HL): nearly double the task demands of Medium Load
MATB II task in progress. The brain data recorded with mbt Smarting wireless EEG system
(a) MATB-II on the task display; (b) Experiment in progress.

Utilizing portable EEG recording devices, we captured participants’ neural activity as they navigated through varying combinations of MATB-II subtasks.

To get the full picture, we also asked participants to rate their perceived workload using the NASA Task Load Index (NASA-TLX). This subjective measure, combined with error rates, gave us a comprehensive understanding of their mental workload.

Block durations and arrangements varied to ensure equal exposure to each task load across the sessions. There were four possible configurations of the blocks, and each session could have been in one of the configurations.

MATB II block configurations
Potential configurations for block arrangement. There were four distinct ways each session could be structured. Block types with different task loads are presented with different colors. The instances of administering the NASA-TLX are shown (following each 5 min. block).

Decoding Brain Patterns with AI EEG analysis

To analyze the EEG data obtained from our experiments with AI, we turned to Convolutional Neural Networks (CNNs). Our AI EEG architecture was tailored to two primary objectives. In one setting, the network was trained to classify EEG segments based on task load levels (PW, LL, ML and HL). In another setting, the same network was trained to detect the presence of an individual MATB-II subtasks.

Only the basic EEG preprocessing (filtering and re-referencing) was applied without any artifact removal.

Insights and implications of the study

The NASA-TLX scores revealed statistically significant differences in subjective MWL across different task levels. Surprisingly, participants showed impressive adaptability to increased loads. The increased task load from low load to high load led only to a marginal increase in error rate.

The CNN performance was remarkably good when classifying between Passive Watching (PW), Low Load (LL) and higher load levels (ML or HL). However, it stumbled when it came to distinguishing between medium and high task loads (ML and HL), and saw them as a single class.

confusion matrix for applying convolutional neural networks to MATB II task
Confusion matrix for block TL classification (true class label in vertical axis). Brighter cell color indicates a higher cell value and vice versa.

But here’s the kicker: our CNN excelled at detecting specific subtasks with remarkable accuracy. This opens up exciting possibilities for applications in Brain-Computer Interfaces (BCIs), where knowing what task someone is engaged in is crucial.

F1 score 0.87 0.88 0.87 0.86
Precision 0.87 0.88 0.90 0.85
Recall 0.87 0.88 0.84 0.88
Accuracy 0.87 0.87 0.90 0.84
Results for overall detection and each of the tasks individually

What’s Next? The Future of Multitasking Research

In this study, we have opened a new window into how our brains manage multitasking and mental workload through EEG. We have highlighted the challenges and potentials of AI EEG algorithms in estimating MWL and differentiating human activity.

Looking ahead, the study paves the way for two exciting research paths. One route would be introducing tasks that exceed participants’ cognitive limits, observing how it affects brain patterns. Another approach could involve modifying how task load is adjusted—rather than augmenting the quantity of tasks in multitasking, one could vary the complexity of individual tasks. Testing the same CNN model to these different datasets could reveal further insights into our brain’s management of varying mental demands, potentially transforming how EEG is used to assess our cognitive workload.

The full paper can be found at:

Brain Sciences | Free Full-Text | Mental Workload Classification and Tasks Detection in Multitasking: Deep Learning Insights from EEG Study (mdpi.com)

Recommended reading