Revolutionizing Design: The Power of BCI in Creative Processes

  • BCI
  • VR
  • multimodal

In this innovative study by Qi Yang and team from Cornell University, researchers created an advanced tool. It combines Brain-Computer Interface (BCI) technology with Virtual Reality (VR). This tool boosts metacognitive monitoring in architectural design.

BCI VR EEG

Their project, ‘Multi-Self,’ uses real-time emotional feedback that could help designers handle the uncertainty and emotional changes common in the creative process.

Problem and Motivation Behind the Study

The researchers tackled a major challenge: designers’ difficulty in self-monitoring and managing cognitive processes during tasks. Traditional methods often fail to guide designers through the complexities of creative professions.

‘Multi-Self’ aims to bridge this gap. It provides real-time, visual biofeedback on emotional responses, using BCI to track and display these within a VR environment.

BCI VR feedback
A – Emotion coordinate system; B – Real-time BCI feedback presentation in VR; and (C) Neurological            feedback pop-up panel

This method is especially useful in architectural design, where decisions have significant, complex impacts.

Experiment Setup: BCI and VR

The study involved 24 participants who used the ‘Multi-Self’ tool to design an interior lobby space in VR.

The tool combined EEG data analysis with VR technology to project designers’ emotional states (valence and arousal) directly into their visual field. The diagram of the “Multi-Self” design tool is presented in the image bellow.

BCI EEG VR system diagram

This setup let participants see their changing feelings about design choices in real-time. It helped them make decisions. The EEG data were collected with a non-invasive, gel-based mBrainTrain Smarting Pro EEG headset. The VR environment was displayed through an HTC Vive head-mounted display. The full Experiment setup can be seen in the figure Bellow.

BCI VR experiment setup
Experiment setup and sensors used in the study

Experimental Procedure

The experiment was structured into four main sessions, each conducted individually. The experiment procedure can be seen in the image bellow.

EEG VR Experiment procedure
Overview of the experimental procedure: (A) Participants view fifty panoramic environments and report their emotional reactions; (B) Participants practice manipulating design options with an Xbox controller; (C) Participants confirm their feelings about design changes using the ‘Multi-Self’ tool; (D) Participants choose three final designs continuously with real-time emotional feedback.

Initially, participants completed a pre-experiment survey and were introduced to the SAM scales for assessing emotional responses. Equipped with EEG and VR gear, participants engaged in a training session. This involved viewing VR images of architectural environments for 10 seconds each while EEG data captured their emotional responses, which they reported using the SAM scale.

Following training, participants moved to a practice session where they familiarized themselves with VR controls using an Xbox adaptive controller, practicing changing aspects of a virtual lobby. Concurrently, researchers processed the EEG data to refine the machine-learning models.

In the validation session, participants tested all available design options, creating their preferred designs. The BCI system analysed brain activities in real-time, providing feedback on the accuracy of emotional predictions through pop-up queries. This session alternated between direct feedback and independent participant reporting on emotions.

The final design session allowed participants to freely develop three lobby designs based on ongoing BCI feedback. This culminated in a debriefing interview to gather qualitative data and understand the emotional impact of the feedback.

Outcome measures

Outcome measures focused on the accuracy and user perception of the BCI system, employing three metrics: offline validation accuracy, online user agreement, and consistency between self-reports and unseen predictions. Additionally, the User Experience Questionnaire assessed the tool’s impact on design processes and user satisfaction.

Study Results and Insights

The results of the study were promising, though mixed in terms of feedback accuracy. While some designers found the real-time feedback helpful in reducing uncertainty and fostering creativity, others were less convinced by the accuracy of the emotional predictions provided by the BCI.

However, a significant majority valued the tool’s role in deepening engagement with the design process. It encouraged exploration of various design alternatives.

The tool’s capability to provide emotional feedback in real-time introduced a new method to support metacognitive monitoring and manage design-related uncertainty.

Future Implications and Applications

The ‘Multi-Self’ tool introduces new ways to use BCI and VR in creative fields. It helps manage emotions and thoughts during design, potentially changing how professionals work.

Future research might broaden the tool’s uses, enhance its emotional accuracy, and test it in educational settings. This could make it a standard in creative industries.

Enhancing Creativity with BCI

This study exemplifies how integrating BCI with VR can provide benefits in creative professions, particularly in architecture.

The ‘Multi-Self’ tool not only aids in emotional and cognitive management but also encourages a more reflective and informed design process.

As BCI technology evolves, its integration into professional practices promises to enhance creativity and efficiency, paving the way for more innovative and responsive design solutions.

The full text of this exciting article can be found here: https://www.sciencedirect.com/science/article/abs/pii/S1071581924000132

Recommended reading