Exploring Neural Coordination – alpha activity and N400 responses in a natural conversation

  • hyperscanning
  • auditory attention
  • LSL
  • synchronization
  • neural oscillations

This study uses a hyperscanning EEG setup to explore neural coordination between speaker and listener while they converse. It takes a look into the well known N400 response to different speech conditions, as well as alpha activity during speech preparation.

During a conversation, the listener has to understand not just the words, but also the tone and the way the sentences are put together. The listener predicts what the speaker will say next, making the conversation run smoother.

However, what happens when the speaker says something that doesn’t follow the rules of language?

And how does the speaker’s brain activity alter once they are prompted to use unexpected words in a sentence?

Hyperscanning experiment setup

In this experiment, a speaker and a listener are set to sit across each other. For the purpose of the study, they formed 9 pairs.

The speaker had a task to come up with an end of a presented sentence. The twist was, in some trials, the speaker had to fill in the blank with an unexpected ending.

Here’s an example: end the sentence “I took my dog for a _____”. Most of us would probably end it with the word “walk”. However, sometimes the speaker in the study would get prompted to end the sentence with something unexpected.

That’s how the authors defined 2 conditions – expected and unexpected. For both conditions, they had 40 trials i.e. 40 different sentences.

one trial of the study showing the unexpected prompt to the speaker
Representation of one trial, for the unexpected condition. The speaker first gets the prompt with unexpected/expected (humorous/serious) instruction. Then, they see the sentence and have 4 seconds to come up with the ending. Once the sentence turns green, they read out the full sentence.

Analyzing Hyperscanning EEG data

To measure brain responses of the pair, scientists used Smarting portable EEG systems. EEG data from both participants and the experiment markers were all wirelessly synchronized. They also synched audio input from a microphone, to find the exact time the word of interest was spoken out loud.

When it comes to analyzing speaker data, authors were particularly interested in the alpha power levels.

They used a wavelet transformation to decompose EEG into frequencies (centered on 2, 6, 8, 12, 20 and 40 Hz). These frequencies represent the predominant frequency bands of EEG – delta, theta, lower alpha, upper alpha, beta, and gamma.

For the listener, scientists looked into the event related potential – ERP response to the said word. Main point of focus was the negative peak at ~400ms after the word. This N400 peak is reported to be sensitive to violations of semantic meaning of a word.

P600 peak is also of interest as it appears to be sensitive to violations in syntax, according to previous research.

Higher neural processing in both speaker and listener, in the “Unexpected word” condition

Let’s take a look at the difference in wavelet energy between 2 conditions on speakers’ data.

topographic plot representing the difference between 2 conditions on all wavelet frequencies

Wavelet energy between 2 conditions is averaged across parietal electrodes. We can see that the unexpected – expected difference is most prominent in the lower alpha band. This finding supports the notion that the “unexpected” condition is more engaging. It requires more resources to come up with a word that doesn’t fit into the previous sentence.

waveform representing the wavelet power changes over time
Waveforms in the image show changes of wavelet energy over time, for 2 conditions. Across speakers, we observe the biggest difference at around 400ms (marked). Boxplot on the right shows the distribution of the difference over all speakers. These values significantly differ from zero (p=0.018).

Now, let’s switch to the time domain to interpret listener’s ERP. The response was averaged across central-parietal electrodes. From the averaged waveform, we can find the minimum (270ms after word onset) and maximum peak (670ms after word onset).

N400 and P600 components in 2 conditions
Authors looked at the difference between 2 conditions for both min and max peaks. Significant difference is seen only for the min peak – take a look at the boxplots on the right. You can also see the difference in a topographic form at the bottom of the image. At 270ms, the head plot shows negative values in the central-parietal region. Compared to that, at 670ms, we see that this difference is not prominent between 2 conditions.

The minimum peak is referred to as N400 in the study. Authors note that N400 appears earlier and slightly right-lateralized on the topoplot. However, listeners’ N400 response to spontaneous unexpected words appears qualitatively similar to the N400 effect in traditional experiments. We can observe a more negative N400 peak in case of unexpected words, aligning with previous studies.

Speaker’s alpha activity and listener’s N400 response correlate positively

Scientists examined correlation between all the extracted EEG parameters, from speaker and listener. Only the correlation between lower-alpha power and N400 amplitude showed to be significant.

Decrease in the speaker’s lower alpha during speech preparation is linked to more pronounced N400 in listeners. You can see the correlation distribution over pairs in the boxplot below, as well as an example of this relation (r = 0.24).

Relationships between speaker alpha and the listener ERP. The speaker’s preparatory alpha band amplitudes were correlated with the listener’s min ERP amplitudes. The distribution of correlations across subject pairs is indicated in the boxplot (left). These coefficients significantly differed from zero (p=0.0009).    (Right) Reduced wavelet amplitudes within the speaker during preparation appear to correspond to a greater N400 amplitude within the listener (r=0.24).

This research marks a significant step in understanding the neural dynamics of natural speech processing within a hyperscanning context, and opens new doors for complex social studies. You can find the original study on this link.

Recommended reading