Fast and Simple EEG + Eye-Tracking Setup Using a Mobile Phone

  • out-of-lab
  • eye-tracking
  • multimodal
  • how-to
  • synchronization
  • LSL

A very fast setup is important for any kind of research experiment. In this blog, we’ll cover how to use EEG + eye-tracking with just an Android phone.

With this mobile setup, researchers can collect brain activity and ET data quickly in many settings, from classrooms to outdoor environments.

This flexibility allows for more realistic experiments, giving better insights into real-world interactions. The combined EEG and ET data can transform many fields. Marketers can understand consumer behavior better, educators can create personalized learning experiences, and clinicians can track neurological conditions more effectively.

Our EEG systems can be mounted in under 10 mins (we’re talking 64 channels!). We’ll show you how easy it can be to set up an ET system as well – without any calibration steps needed!

(Rather short) list of equipment

Communication between PupilLabs eye-wear and mbt’s Smarting PRO line devices is made possible using our SmartingPro Android app and PupilLabs’ Neon Companion app, installed on the same smartphone.

Install the 2 apps on the same Android phone and then simply follow the setup below.

Let’s make a quick overview of what the person would be wearing:

  1. EEG cap + amplifier (in the example image, we have the 64-channel system)
  2. PupilLabs eye-tracking system – we used NEON eye-glasses
  3. Android phone with installed apps
eeg+eye-tracking setup overview on a participant
(Left) Entire setup placed on our volunteer (right) Layout of the equipment.

Neon glasses start working the second you put them on and plug them in the phone. No calibration is required before or during a recording. This is because PupilLabs uses a neural network – NeonNet to estimate the gaze data.

How does the synchronization work?

SmartingPro app can send triggers to the Companion app over the HTTP protocol. These triggers are then included in the EEG recording as well, in the form of an LSL (labstreaminglayer) stream.

We have the EEG cap setup + the glasses are placed on the participant. You will also need to connect your phone to a Wi-Fi Network.

In the Companion app, you can check out the gaze data before starting any recording. You can also check out the camera view pointed at the eyes.

gaze data and eye-cam data in Companion app

Open the Companion app and click on the Stream button. Remember the noted IP address in the window that pops up.

Stream button in Companion app that allows communication between apps.

You can then close the window, and start recording eye-tracking data.

Now, let’s switch to SmartingPro app. In the configuration screen, you’ll need to enable RESTApi toggle in Event marker types menu. This is how the communication between 2 systems is activated.

enabling RESTApi trigger in SmartingProApp

Go to signals view and let’s start recording EEG data as well. You’ll notice the RESTApi LSL stream in the list. Make sure that you include it in the recording.

Also, remember the IP address we mentioned? Here, you’ll need to type it in to secure the communication.

Starting EEG recording and trigger presentation in SmartingProApp

EEG recording will initiate a triggering sequence, which enables later synchronization between 2 data recordings.

Once you are done with the experiment, end both EEG and ET recordings.

Trigger sequence in eye-tracking data

What does the trigger sequence imply in this synchronization protocol?

Let’s have a look at the recorded ET data for easier explanation. We’ll open the data on Pupil Cloud – the web-based ET platform for data logistics, visualization and analysis.

You can choose to sync ET data to the cloud automatically, right after the recording ending.

Viewing recorded eye-tracking data and triggers using PupilLabs Cloud platform

Below video frames, you’ll notice the Events label. Throughout the video, these Events are marked with a diamond shape. If we put our cursor on the first diamond shape, we’ll see the event label is “EEG.start”. At this point, we started recording in the SmartingPro app and the initial trigger was captured.

We can also see the timestamp of the trigger is included in the label.

After that, SmartingPro app keeps sending triggers each second, to maintain precise synchronization over time.

When you stop the EEG recording, you’ll get an “EEG.stop” trigger in ET data.

 

Let’s open a recorded EEG file and take a look at the triggers there.

Viewing recorded EEG data with included synchronization triggers

We see the initial “EEG.start” trigger at the beginning of the recording, and then the rest of the synchronization triggers..

If you wish to do joint processing of 2 data types, for example in Matlab or Python, you can! PupilLabs gives you access to raw recordings too, so you don’t have to use their cloud platform.

If you are interested in using this solution for your experiments, you can ask us anything! Just contact us through our support page.

Recommended reading