● The neurotechnology device makes use of invasive microelectrode arrays that are implanted in the motor cortex and a machine learning algorithm that translates neural activity.
● In tests it succeeded in decoding entire sentences composed from a 125,000-word vocabulary, but error rates remained high (26% to 54%).
Will scientists soon be able to give the power of speech to patients who have been deprived of verbal ability by disease or conditions that have never allowed them to speak? A specialist neurotechnologies research team at Stanford Medicine has conducted a study that made use of a brain computer interface (BCI) to detect and transcribe the “inner speech” of non-verbal or minimally verbal patients. “We found that inner speech appears to be a more weakly modulated version of attempted speech, although the two can be distinguished with the help of a neural ‘‘motor-intent,’’ explain the researchers. Recently, a team from Columbia University’s School of Engineering and Applied Science developed a new silicon-based BCI capable of creating a minimally invasive, high-bandwidth link with the brain to treat neurological disorders
Identifying phonemes in thought
The BCI tested by the researchers makes use of tiny microelectrode arrays surgically implanted on the surface of the brain, which directly record patterns of neural activity. These signals are then relayed by a cable to a computer algorithm, which translates them into actions such as the expression of words or computer cursor movements. In the study, language was recorded directly from neural activity of the motor cortex. With help from machine learning, the algorithm was able to recognise distinct neural patterns for individual phonemes. In this manner, the device was able to restore the ability to communicate without tiring out patients like systems that require them to attempt to produce speech such as those used for amyotrophic lateral sclerosis (ALS). “It is as though the brain prepared the movement without crossing the threshold required to activate it,” point out the researchers. By analysing these signals, scientists managed to decode entire sentences in real-time using a vocabulary of up to 125,000 words. Just by thinking about them, one of the patients was able to display on screen sentences such as “We don’t have a real strict budget.” When connected to a speech synthesiser, the system could say them aloud. However, more could be done to improve its accuracy: depending on the patient, the word error rate varied from 26% to 54%, which lags behind the scores achieved by other methods that require patients to attempt to produce speech.
By analysing these signals, scientists managed to decode entire sentences in real-time using a vocabulary of up to 125,000 words
Devices optimised by machine learning
For several years, vocal neuroprostheses, which decode brain signals associated with attempts to speak — movements of the lips and tongue that may not always be perceptible — have allowed people with ALS or spinal cord injuries to communicate. “Compared with attempted speech, the inner-speech BCI required less effort, offered improved comfort, and bypassed physiological constraints (e.g., breathing control) that slow attempted speech in people with paralysis,” point out the Stanford researchers. “Although they remain prohibitively expensive and involve a very long training period for patients, the research on these devices is very promising. The use of invasive electrodes remains a major obstacle to their acceptance. However, given the rapid development of BCIs in recent years and the increase in machine learning processing power, I wouldn’t be surprised if the technology evolves relatively quickly, perhaps to a point where we will have non-invasive electrodes,” remarks Orange Innovation project lead Foued Bouchnak.
The ethical challenge of protecting mental privacy
However, the system developed by the project poses a major ethical risk. It could spontaneously output thoughts that users would prefer to keep private: a possibility that was underlined by the fact that aspects of free-form inner speech could be decoded during sequence recall and counting tasks. To address this issue, the researchers developed two solutions. The first of these was to create a keyword, in this case “ChittyChittyBangBang” which was correctly detected with 98.75% accuracy, to lock or unlock the decoder. The second was to develop a decoder training strategy in which the system learned to ignore inner speech, which it labelled as private, and only output attempted speech. However, setting aside the question of the effectiveness of both of these strategies, the study has highlighted an urgent need for more regulation of neurotechnologies.
Foued Bouchnak
