Current Trends in Hyperscanning research – EEG technology promises studying more than 2 subjects simultaneously

Hyperscanning research represents a transformative approach in neuroscience. It enables simultaneous recording of brain activity from multiple individuals engaged in social interactions.

This technique serves for studying complex neural underpinnings of social behaviours by capturing the dynamic interplay between brains during real-time interactions.

It holds relevance in exploring how humans synchronize their cognitive functions and behaviours, offering profound insights into the fundamentals of human connection and communication.

 

Trends in Hyperscanning research

Recent review done by Hakim et al. (2023)  highlights the significant growth in hyperscanning research, noting an upward trend in publications since 2014, peaking at 46 in 2022.

This increase is attributed to advances in computational capabilities and the availability of more cost-effective neuroimaging technologies.

Among the four main neuroimaging modalities used in hyperscanning research—fNIRS, fMRI, EEG, and MEG—the more economical options, EEG and fNIRS, have become increasingly popular.  Until recently fNIRS was mostly used, but the mobile EEG shows steeper increase in research due to its advantages over other neuroimaging technics in .

Importance of Using EEG for Hyperscanning Research

EEG is particularly valuable for hyperscanning research because of its ability to capture rapid fluctuations in brain activity at millisecond resolution. This is essential for understanding the fast-paced dynamics of social interactions.

Advantages of using EEG for hyperscanning include:

1. High Temporal Resolution: EEG’s strength lies in its ability to provide real-time monitoring of brain processes. This is crucial for capturing the transient yet significant moments of synchronization between individuals engaged in social tasks.
2. Flexibility and Mobility: EEG equipment is portable and can be used in more naturalistic settings outside the lab, which is advantageous for studies aiming to observe real-world social dynamics.
3. Scalability: EEG allows for the simultaneous recording of multiple participants, making it suitable for studying complex social interactions involving more than two individuals.
4. Cost-Effectiveness: Compared to other neuroimaging techniques like fMRI or MEG, EEG is less expensive and more accessible, allowing broader application in diverse research settings.

Mostly Studied Experimental Paradigm Tasks in Hyperscanning Research

As summarized in the recent work of Grasso-Cladera et al. (2024) Most experimental tasks utilized in hyperscanning research involve real interactions among participants, which include:

• Conversational Tasks: Engaging participants in natural dialogues to examine social communication dynamics.
• Game-Playing: Involving competitive or cooperative games that require strategic thinking and interaction.
• Simulation Activities: Utilizing virtual reality or simulators (like flight simulators) to create immersive environments where participants can interact within simulated yet realistic contexts.
• Problem-Solving Tasks: Tasks that require collaborative efforts to solve puzzles or challenges, reflecting joint cognitive efforts.

These tasks are selected to ensure ecological validity, meaning they mimic real-world situations and thus, are highly relevant to participants’ everyday experiences. This approach aligns with the aims of social neuroscience, focusing on authentic social interactions. The studies often employ both simulated environments and real-world settings, enhancing the relevance and applicability of the findings to natural human behaviours.

The design philosophy behind these tasks is known as scalable experimental designs (SED), which emphasizes studying cognitive phenomena as they occur naturally, without overly contrived or artificial setups. This methodological approach helps in capturing the complexity of human cognition more effectively, providing a robust framework for understanding how cognitive processes are embedded in real-world social interactions.

What is Studied in Hyperscanning Research

Recent review paper by Arthur Czesmuski et al. (2020) nicely summarized what is most commonly studied using EEG hyperscanning methods. These include:

1. interpersonal coordination and synchronization
2. Music: A Resonance of Minds
3. Emotion and Affect
4. Cooperation and Competition
5. Games and Decision Making
6. Action Representation and Joint Attention
7. Speech and Communication
8. Intervention Methods

Interpersonal coordination and synchronization

Hyperscanning research reveals that interpersonal synchronization involves a sophisticated interplay of neural activities. These activities occur across different brain regions and frequencies and are significantly influenced by social and environmental factors. This body of work underscores the potential of hyperscanning research in understanding the neural basis of social interactions and coordination, although it calls for more extensive studies to confirm and elaborate these findings further.

This approach provides real-time insights into the reciprocal neural processes that facilitate synchronized movements between individuals, even within milliseconds.

Research employs a range of experimental paradigms, from simple tasks like button pressing to more complex and natural settings like playing music together, to probe the neural underpinnings of these interactions.

Studies often manipulate variables such as visual contact, feedback, and synchronization mode to explore different aspects of action coordination.

Findings have highlighted significant neural couplings, particularly in frontal and parietal regions, which correlate with improved performance and stronger social connections between participants.

This synchronization is not limited to behaviour but extends to neural oscillations across various frequency bands, indicating a complex network of brain areas working in concert during coordinated activities.

In more complex or naturalistic tasks, such as playing music or mimicking finger movements, increased synchronization has been observed, including changes in theta and beta band activities, suggesting that these neural interactions may facilitate not just simple but also complex coordinated behaviours.

The influence of social factors, like the closeness of relationships, has also been shown to affect synchronization, with substances like oxytocin enhancing these effects, indicating a biological underpinning to social connectivity and synchronization.

Music: A Resonance of Minds

Musical performances create ideal conditions for hyperscanning experiments, which examine both intrapersonal action coordination and interpersonal synchronization.

For instance, Sänger et al. (2012) studied the brain activity of musical duets to understand how assigned roles like leader and follower affect neural synchronization, revealing that these roles can induce asymmetric brain synchronization patterns.

Studies like those by Babiloni et al. (2012) and Müller et al. (2013) showed the role of neural synchronization in fostering empathy among musicians and enhancing connection during improvisation.

Research highlighted by Acquadro et al. (2016) explores these dynamics, utilizing various experimental setups to probe different aspects of musical interaction.

Experiments also extend to settings without predefined roles, like improvisational music, which show that even in less structured environments, brain synchronization is crucial for coordination. Novembre et al. (2016) manipulated musical familiarity and observed how it affects synchronization during joint piano playing, indicating that alpha oscillations help regulate the neural balance between self and other integration.

Further studies performed by Ana Zamm et al. (2021) have examined into how these synchronization patterns not only depend on shared perceptions but also the roles individuals play during musical tasks.

Overall, these findings underscore the utility of hyperscanning research in understanding complex interpersonal dynamics in musical settings, illustrating how such environments can serve as powerful models for studying neural mechanisms of social interaction and coordination.

Emotion and Affect: The Silent Communicators in Social Interaction

Hyperscanning research highlights the significant role of emotional regulation and affect in social interactions. Studies have utilized various setups, such as facial communication, mother-infant interactions, and goal-oriented tasks, to explore how emotions coordinate between individuals.

 

For example, synchronized brain activity has been observed in tasks requiring cooperation, especially in brain regions associated with social cognition and emotional processing.

Significant findings have emerged from comparing interactions involving high emotional impact, where greater inter-brain synchronization is noted in scenarios such as romantic relationships or mother-infant interactions.

These relationships demonstrate that emotional closeness can enhance neural connectivity, suggesting that emotions significantly modulate brain-to-brain synchronization.

Furthermore, the research underscores the potential for hyperscanning to investigate broader types of relationships and emotional dynamics, including those not yet extensively studied, such as siblings or employer-employee interactions.

The overall findings indicate that the emotional component in interactions not only enhances synchronization but may also differ based on the nature of the relationship and the emotional context of the interaction, paving the way for further studies into the neural bases of human emotional connections.

Cooperation and Competition: The Two Sides of Social Engagement

Hyperscanning studies have explored how brain activity differs during cooperative and competitive scenarios using tasks like the Prisoner’s Dilemma, Jenga, and other games.

These studies reveal that the prefrontal cortex (PFC) and other brain regions show varied activation depending on whether participants are cooperating or competing.

For example, the medial prefrontal cortex (mPFC) is active in all conditions, while the anterior cingulate cortex (ACC) is specifically activated during competitive actions. These findings suggest that cooperation often requires synchronized brain activity, particularly in the dorsomedial prefrontal cortex (dmPFC), which is essential for planning and motor execution.

Conversely, competition might activate brain areas involved in theory of mind, as participants anticipate opponents’ actions.

Overall, hyperscanning research has demonstrated that the nature of the task significantly affects inter-brain synchronization, potentially influencing how cooperative and competitive behaviours are neurologically orchestrated in more naturalistic settings like sports.

Games and Decision Making: Strategic Minds at Play

Research in games and decision-making has identified critical brain regions involved in interactive decision-making, such as the medial frontal cortex (MFC), superior temporal sulcus (STS), and temporoparietal junction (TPJ).

Studies highlight that cooperative behaviours activate brain areas linked to reward processing, notably the anterior cingulate cortex (ACC) and cingulate motor area (CMA), which are particularly active in leadership roles and competitive situations.

Early investigations using game theory illuminated how the brain processes social interactions, with cooperative decisions activating reward circuits, contrasting with non-cooperative behaviours.

Further research using EEG and fMRI hyperscanning has validated these findings, showing enhanced activity in the ACC and related areas during social cooperation and competition.

Additional studies have explored the role of social variables such as gender and deception, revealing that these factors significantly affect brain activity during interactive games, influencing areas like the TPJ in deceptive contexts.

Overall, this body of work underscores the complex interplay of neural regions during decision-making in games, emphasizing the need for future research to dissect the nuances of these interactions further, including the impact of non-verbal cues like facial expressions and gestures.

Action Representation and Joint Attention: The Choreography of Social Cognition

Hyperscanning research has significantly advanced our understanding of joint attention and action coordination in social interactions.

Studies have focused on how individuals synchronize their neural activities to effectively coordinate actions, such as through mutual gaze or joint musical performances, highlighting the importance of regions like the right inferior frontal gyrus (IFG). This synchronization is crucial for shared attention and the integration of actions and intentions between individuals.

Experiments have demonstrated that joint attention tasks increase neural connectivity, particularly involving the IFG, which plays a pivotal role in processing shared intentions and facilitating social communication. For instance, the synchronization of eye-blinks and neural activity in the IFG during joint tasks suggests that these mechanisms can be learned and retained in social memory, enhancing group coordination.

Further research using diverse neuroimaging methods and experimental designs continues to explore the dynamics of joint attention, emphasizing the role of neural synchronization in social interactions.

This ongoing investigation aims to deepen our understanding of how individuals integrate and respond to each other’s actions and intentions in various social contexts.

Speech and Communication: The Neural Basis of Human Interaction

Hyperscanning research has shown that verbal communication is crucial in understanding social interactions, revealing that face-to-face dialogue significantly increases inter-brain synchrony compared to non-interactive communication.

Study of Bridwell et al. (2018) This study used a hyperscanning EEG setup to explore neural coordination between speaker and listener during the conversation. It investigated well known N400 response to different speech conditions, as well as alpha activity during speech preparation.

 

Kawasaki et al. (2013) confirms that synchrony in human communication involves specific neural pathways, particularly in the theta and alpha bands across temporal and lateral-parietal regions.

Additionally, hyperscanning research by Pérez et al. (2017) points out that while low-level auditory processes mediate speech-to-brain synchronization, it is the interaction itself that drives inter-brain coherence.

Studies involving educational settings have also indicated synchronized brain activity between teachers and students during effective communication, highlighting the potential of hyperscanning to explore dynamic communicative interactions.

However, challenges remain in filtering out artifacts from speech to refine this research method further.

Intervention Methods: Modulating the Social Brain

Intervention methods in hyperscanning research manipulate neural activity directly to study changes in social interaction behaviours.

Studies have employed various techniques such as EEG with oxytocin administration and transcranial alternating current stimulation (tACS) to observe their effects on synchronization tasks.

For instance, hyperscanning research by Mu et al. (2016) found that oxytocin enhanced synchronization in social interactions but not when interacting with computers. Similarly, Novembre et al. (2017) observed increased synchronization with in-phase tACS at 20 Hz, whereas Szymanski et al. (2017) did not find significant effects from similar stimulation settings.

These studies highlight the potential of using physiological and pharmacological interventions to modulate neural activity and examine their impact on behaviour. However, results vary, indicating a need for more extensive research to deepen our understanding of these interventions and their effects on inter-brain synchronization in different social contexts.

Types of Hyperscanning analysis

Analysing hyperscanning data presents unique challenges, requiring adaptation of traditional intra-brain analyses to inter-brain contexts or the development of new analytical methods.

Challenges include distinguishing between inter-brain relationships caused by identical stimuli versus genuine between-brain interactions. Techniques such as partial correlations, permutation analysis with randomly paired participants, and comparison against actual pairs are used to isolate the effects of true interaction.

Overall, hyperscanning analysis involves complex methodologies, including correlation studies, graph theory, and information flow analysis, each contributing to a deeper understanding of how brains synchronize and interact in real-time during joint activities. All this was nicely summarized in the work of Arthur Czesmuski et al. (2020).

Coupling/Connectivity measures in Hyperscanning Research

In EEG hyperscanning research, common methods like Phase-Locking Value (PLV), Phase Lag Index (PLI), and phase coherence are used to estimate the connectivity between brains. These techniques, developed for single-brain studies, are adapted to measure how neural activities from different brains synchronize:

1. PLV assesses if two signals are phase-locked within a time window, indicating perfect synchronization at a value of 1 and no synchronization at 0. It’s used extensively in studies involving coordinated physical tasks and communication.
2. PLI addresses the common source problem that PLV faces by ignoring zero-phase lagging signals, making it a robust alternative for studying interactions like music playing and verbal tasks.

Both PLV and PLI aim to detect synchronization between different brain signals during various interactive tasks. Despite their technical differences, both are effective in hyperscanning setups where brain sources are distinct.

Phase coherence, another key method, measures the consistency of phase relations between signals, offering a nuanced view of inter-brain neural oscillations. It’s applied in diverse contexts, from musical coordination to social interactions.

These connectivity measures are crucial for interpreting how closely brain activities align in real-time interactions, underscoring the need for precise terminology and methodology in hyperscanning research to accurately reflect inter-brain dynamics.

Correlation and Dependence Analysis

In EEG hyperscanning research, correlation and dependence analysis are crucial methods for measuring synchrony between brains. These techniques assess the correlation between EEG signals from two different brains to understand how neural activities synchronize during various interactions.

Key Applications in EEG Studies:

1. Coordination of Speech Rhythm: Kawasaki et al. (2013) utilized correlation between different EEG frequency bands (theta and alpha) to study how speech rhythms are synchronized between individuals.
2. Social Interaction Analysis: Studies like those by Kinreich et al. (2017) explored interactions between strangers and couples by analysing correlations in alpha, beta, and gamma bands to discern differences in neural synchronization based on the nature of the relationship.
3. Educational Dynamics: Dikker et al. (2017) and Bevilacqua et al. (2019) applied total independence analysis to EEG data to predict classroom dynamics and student engagement, illustrating how brain synchronization can reflect group social interactions in learning environments.

These correlation measures provide insights into how brain activities align during tasks that require cooperation, communication, or even competitive interactions.

By analysing the EEG signals’ synchrony, researchers can infer the degree of neural connectivity and coordination between participants, enhancing our understanding of collective brain functions in various social contexts.

Graph Theory measures

Graph theory measures applied to EEG hyperscanning data provide valuable insights into the structural and functional connectivity between brains during social interactions. These measures are particularly useful in mapping and quantifying the complexity of between-brain networks:

1. Decision-Making and Coordination: Studies like De Vico Fallani et al. (2010) have utilized graph theory to analyse network links and modularity during tasks such as decision-making and musical coordination. These analyses reveal how network structures vary depending on the social and cognitive demands of the task.
2. Musical Performance: In research involving guitar players, measures of small-worldness and network topology have shown that neural connections between brains become more structured and regular at higher frequencies during musical performance (Müller et al., 2013). Directionality within these networks has also helped identify leader-follower dynamics among musicians (Sänger et al., 2013).
3. Combat and Cooperation: Another application is in combat cooperation tasks, where the dimensionality of between-brain networks has been investigated to understand collaborative strategies and dynamics (Dodel et al., 2011).

Graph theory metrics like modularity, small-worldness, and directionality enhance the understanding of how brains synchronize and interact, revealing the underlying neural coordination in real-time social scenarios. The application of these measures in EEG hyperscanning research is proving essential in advancing our understanding of complex interbrain relations, with potential for more extensive application as data-driven methods evolve.

Information Flow

In EEG hyperscanning research, analysing the flow of information between brains involves identifying causal relationships, primarily using methods like Granger Causality and Partial Directed Coherence (PDC). These techniques help determine the direction and strength of neural interactions, particularly during cooperative or altruistic behaviours:

1. Pilot Cooperation: Studies by Astolfi et al. (2012) utilized these methods to reveal that causal links between the brains of cooperating pilots are stronger when the level of cooperative behavior increases.
2. Decision-Making Tasks: Research such as that by Ciaramidaro et al. (2018) demonstrated increased causal links between participants engaged in cooperative decision-making, indicating a directional flow of neural information related to collaborative strategies.

These methods are crucial for distinguishing between mere neural synchrony, which could be due to similar sensory inputs, and actual informational flow that underpins cooperative interactions. The challenge remains in precisely identifying the neural substrates that facilitate these information exchanges, which is essential for a deeper understanding of interbrain dynamics.

Overall, the application of information flow analysis in EEG hyperscanning research is a significant step toward elucidating the complex neural mechanisms of social interaction, highlighting the need for continued exploration and refinement of these techniques in cognitive neuroscience.

Over Two Heads: The Power of Group Dynamics

Challenges in recording more than two participants in Hyperscanning Research

In the field of hyperscanning research, over 95% of studies have focused on just two subjects. This was not just due to preference but it was largely driven by the challenges associated with recording EEG data from multiple subjects simultaneously.

First, the logistical and technical demands increase significantly with each additional participant. Coordinating multiple EEG systems to operate without interference required careful planning and sophisticated equipment. Each system needed calibration and needed to be monitored individually to ensure consistent data quality across all participants, a task that grows exponentially complex as more individuals join. The image below graphically depicts what was needed in the past for successfully performing the EEG hyperscanning research.

When looking in what was needed, in terms of the equipment for performing hyperscanning research for only two subject, increasing to multiple subject seemed as the mission impossible (see video bellow).

Wireless and mobile EEG as an answer to multisubject recording challenge in hyperscanning research

mBrainTrain is dedicated to simplifying EEG recordings outside the lab and strictly controlled laboratory settings. We focus intensively on ensuring robust wireless connectivity for seamless data capture. Here’s how we address common challenges:

• Advanced Technology: Our Smarting PRO Line devices leverage Bluetooth 5.0, significantly enhancing data throughput compared to earlier versions.
• Mobile Integration: We facilitate recordings directly to mobile phones, ensuring that subjects are always within range by carrying their device.
• Remote Connection: Our EEG systems are designed without physical buttons, enabling remote access and control, which eliminates the need for direct physical interaction with the subject.
• Data Security: Data is securely stored on internal memory or an SD card, preserving it even if the connection drops.
• Efficient Data Streaming: We transmit only a critical subset of data wirelessly to conserve bandwidth. The complete data set is safely stored on the device and can be synchronized post-experiment.

These features make the Smarting PRO Line exceptionally resilient under diverse conditions and keep improving to further alleviate the burden on researchers.

Compared to traditional wired EEG setups, our wireless systems do not significantly increase the complexity of setting up studies with more participants, as illustrated in the figure below.

Multisubject Hyperscanning Studies

Hyperscanning research in naturalistic settings has recently expanded to include multi-subject studies, enhancing the ecological validity of findings on human interaction.

These studies typically involve groups rather than individuals to better mimic real-life social dynamics. For example, hyperscanning experiments involve participants playing a sorting card game to study group interactions (a work conducted by Shields et al. 2023).

Multi-subject setups are used to understand complex social behaviours that emerge only in group settings. These include classrooms or musical performances, where interactions are less predictable and more dynamic.

This approach allows researchers to explore how group dynamics influence individual behaviours and overall group performance. For instance, studies have shown that group interactions can enhance task performance and that roles within groups can become fluid, reflecting the continuous interaction required in natural social settings.

Concluding Remarks

In the expansive field of hyperscanning research, more than 95% of studies have focused on just two subjects. This limitation primarily arises from the complex technical and logistical challenges involved in recording EEG data from multiple participants simultaneously. Managing multiple EEG systems to operate concurrently without interference requires meticulous coordination, sophisticated equipment, and individual calibration and monitoring to ensure uniform data quality. These demands multiply with each additional participant, making larger group studies particularly daunting.

However, the advent of wireless and mobile EEG technologies is reshaping this landscape. Systems like mBrainTrain’s Smarting PRO Line, which utilize advanced Bluetooth connectivity and mobile integration, are simplifying the setup for multisubject recordings. These systems eliminate the need for direct physical interaction with subjects and ensure data integrity even if the wireless connection is lost, thus addressing major hurdles in scaling up hyperscanning studies. This technological evolution is crucial, especially as the field moves towards including more than two subjects in research to better capture the dynamics of real-world social interactions.