The challenge: Brain-computer interfaces that work for everyone

• AR
• Health
• Machine learning
• Research
• Technology

Share on Twitter : The challenge: Brain-computer interfaces that work for everyone (New window) Share on Linkedin : The challenge: Brain-computer interfaces that work for everyone (New window)

Lately, we’re hearing more and more about brain-computer interfaces or BCIs. Could you tell us more about what they are exactly?

Fabien Lotte: Brain-computer interfaces, or BCIs, are systems that translate the measurement of brain activity into messages or commands for an interactive application. Brain activity is usually measured using an electroencephalogram, or EEG, with electrodes placed on the scalp. A typical example is a BCI system that allows a user to move a cursor on a screen to the left or right simply by imagining movements of the left or right hand, respectively, which are identified in the user’s EEG signals.

Could we say, as it’s sometimes believed, it’s now possible to read people’s minds?

Hakim Si-Mohammed: We often hear about controlling computers “through mind reading” when talking about brain-computer interfaces. This is sometimes seen as exciting for some users, but others are more apprehensive. To answer the question, the concept of mind reading, as is commonly understood, has no accepted or recognized definition from a scientific point of view. So we’re still very far from the concept of “reading minds.” In more concrete terms, what we’re seeking to do through BCIs is to associate identified and very specific electrical brain patterns or signals with messages or commands in a way that a machine (computer) can process them to trigger programmed actions. There is a lot of research and development currently trying to synthesize the voice of people who are completely paralyzed through the analysis of their brain activity, but we’re still rather far from reading minds as imagined by some.

Our prototype makes it possible to turn a connected lamp or outlet on or off, or to control a more complex object, simply by focusing on a target.

What do you think are the main challenges related to the development of this BCI technology? And what are or will be its main applications?

Fabien Lotte: The historical applications of BCI technology, which motivated its creation, are in the medical sector and in the control of assistive technologies, in particular for people with severe physical disabilities, to control wheelchairs or text editors, for example, hands-free with just their brain activity. More recently – and still in the medical sector – BCIs have shown a lot of promise in physical or cognitive rehabilitation, for example following a stroke. After a stroke, some patients are paralyzed in one arm (due to a brain lesion caused by the stroke) and therefore aren’t able to easily follow classic rehabilitation methods. This is where BCIs intervene, because they can detect from the EEG signals that patients want to move, even if they are not physically able to do so. Thanks to this detection, their paralyzed arm could be moved artificially, for example using a tailor-made and customized robotic prosthesis after a calibration phase. In this way, thanks to feedback provided by the prosthesis, the BCIs would be able to tell patients when they have managed to use the area of the brain affected by the stroke and encourage them to do so again. This will then stimulate neuroplasticity in this area and help recovery. In terms of wider consumer applications, BCIs can also be used as a new hands-free control, directly via the brain, for entertainment purposes such as video games. Finally, BCIs are also of particular interest when it comes to the assessment of mental states through brain signals, such as mental load (“mental efforts”), attention or certain emotions, for example. This can be very useful for the evaluation (e.g. does this aircraft cockpit overload the pilot’s attention with too much information?) or the adaptation of interactive applications according to these states (e.g. to reduce the difficulty of a math exercise when the system realizes the student is experiencing a mental overload).

All these applications are very promising. On the other hand, even though proof-of-concept applications have been established and it’s been proven BCIs “work,” they don’t always work well and are often imperfect. They often wrongly recognize the mental command in the user’s brain signals. For example, when distinguishing between an imagined movement of the left hand and an imagined movement of the right hand, on average, current BCIs will manage it correctly about 80% of the time. Therefore, they’ll be wrong (recognize an imagined movement of the left hand when the user has actually imagined a movement of the right hand) roughly one in five times. In addition, there is a significant proportion of users (between 10 and 30% depending on the types of BCIs) for whom BCIs don’t work at all. The BCIs fail to correctly recognize the user’s mental commands. So currently, the major challenge for BCI research is to succeed in designing reliable and robust BCIs that work for everyone.

Another hot topic in the ICT sector is artificial intelligence. Are there any direct or indirect links between progress in the domains of BCI and AI?

Hakim Si-Mohammed: When we talk about artificial intelligence today, we’re almost automatically talking about Machine Learning and that’s where the link between the two disciplines lies. In other words, the purpose of the two disciplines is not the same, but they use very similar methods and approaches. So, just like current AI systems that use very large amounts of data to “learn” how to perform specific tasks or solve specific problems, BCIs use the same algorithms to be able to recognize precise patterns in EEG signals in order to deduce the user’s mental state, for example. In this sense, the progress made in the field of machine learning almost directly benefits the development of BCIs and, incidentally, most areas of IT. However, the amount of data available for training BCIs is much less than that available for AI systems like ChatGPT and so on. Therefore, the development of efficient methods and algorithms capable of generalizing from small amounts of data is key to eventually allowing the development of more robust BCIs. It’s also true to say some researchers are convinced the very development of artificial intelligence requires a better understanding of brain function in order to align the functioning of artificial neurons, used in neural networks, with that of natural neurons. The future will tell us which approach is the most viable.

Given the current state of technology, what interest could industrialists have in BCIs?

Fabien Lotte: I’m not an industrialist myself, so I can’t speak on their behalf, but I can give my academic opinion. I imagine there are two main approaches — industrial interest in immediate BCI applications using what works best with BCIs right now, and industrial interest with a longer-term vision in order to do R&D and have access to the technology when it’s mature, and to imagine/design what it could be in the future, for the applications of tomorrow.

In the first instance (immediate applications), assistive technology (text editors) and/or physical rehabilitation systems for stroke patients that rely on BCIs already exist and are being marketed. Examples include g.tec (Austria) or Bitbrain (Spain). There are also many marketed products using BCIs (or neurofeedback — its similar sister technology) for wellbeing applications to regulate sleep or attention disorders, for example (examples of the latter include the French companies Dreem, myBrain Tech or URGOTECH). But caution is advised, as these products combine very serious technologies and research with others (not mentioned here) that have no clinical evidence of the effects they advertise. There are also startups that are exploring the wider consumer use of BCIs in everyday life for video games or simple, hands-free control of a phone (e.g. French startups such as NextMind, Conscious Labs or Wisear).

In the second instance of a longer-term vision, there are several large groups investing in BCI R&D to develop new long-term uses and/or master the technology by offering proof of concept, but not a concrete product at this time. In France, this is the case for groups such as Orange or Capgemini and it’s the same for several Big Tech companies (Google, Facebook and Microsoft to name but a few) internationally.

Tell us about the BCI prototype designed in collaboration with Orange for controlling connected devices in the home (TVs, lamps, outlets).

Hakim Si-Mohammed: Through my thesis work conducted as a part of the HYBRID team led by Anatole Lécuyer at the Inria Center in Rennes, France, I had the opportunity to collaborate with Orange on the development of a BCI prototype for home automation application. The idea was that this prototype would allow the control of everyday objects such as a lamp or a television through brain activity. The result was a prototype capable of turning a connected lamp or outlet on or off, or controlling a more complex object such as a television—switching it on and off or changing channels—simply by focusing on a target. All these commands were selected by the user by focusing on the icon representing the command that flashed at a specified rate. The icons that were displayed in Augmented Reality through a HoloLens (an Augmented Reality headset by Microsoft) depended on the objects in the field of vision and the commands associated with these objects. But beyond this purely functional aspect, the prototype made it possible to demonstrate the viability of integrating a BCI into a real ecosystem—in this case Orange’s “connected home”—without interfering with other more traditional methods of interaction, since users could continue to use their voice or physical buttons in addition to the BCI to send their commands. This prototype was presented at SIDO Lyon, France and was the subject of a scientific publication at the EuroXR 2022 conference.

You are both part of the collective for transdisciplinary research on brain-computer interfaces (known as CORTICO in French), an association that unites French-speaking researchers in the field. Could you tell us a little more about the association, as well as the efforts it’s spearheading?

Fabien Lotte: CORTICO is a learned society whose objective is to promote and contribute to research on BCIs, in particular by facilitating exchanges within the French BCI research community and contributing to the discussions and dissemination of knowledge on the subject in France, but also internationally. More specifically, CORTICO organizes an annual scientific conference in France with scientific presentations, technical workshops, tutorials for young researchers and more. It unites academics and industrialists, but it’s open to anyone who’s interested! CORTICO also communicates with the media on the subject of BCIs, participates in discussions and ethical recommendations on their use in France, exchanges with international researchers on the subject and publishes a “Newsletter” for its members (with help from the authors of this article) with information on what’s new and what’s happening in the field. If you are interested in BCIs, you are very welcome, so come join us!