Engineers translate brain signals directly into speech - Sensationalist Headlines

https://www.sciencedaily.com/releases/2019/01/190129081919.htm https://www.nature.com/articles/s41598-018-37359-z In a scientific first, Columbia neuroengineers have created a system that translates thought into intelligible, recognizable speech. By monitoring someone's brain activity, the technology can reconstruct the words a person hears with unprecedented clarity. This breakthrough, which harnesses the power of speech synthesizers and artificial intelligence, could lead to new ways for computers to communicate directly with the brain. It also lays the groundwork for helping people who cannot speak, such as those living with as amyotrophic lateral sclerosis (ALS) or recovering from stroke, regain their ability to communicate with the outside world.

Okay, I admit I don't know enough about what I'm reading so I'm gonna ask for help here on figuring this out from people way smarter than me: Is this actual breakthrough, or more sensationalized crap that we'll never hear about again because it's not nearly as accurate/feasible/whatever as they make it out to be?

It's simply mapping brain activity from the words a person hears. I expect they intend to develop it to recognise areas of the brain and work out what word the person wants to say but considering that every person stores information differently it seems like a long way off before anything comes of this.

Another small step towards telepathy I guess.

How will this be different to existing tech that people with ALS already use?

I think the idea is to develop it for people who can't speak by eventually communicating with a brain directly. They reference the concept of a neuroprosthetic so perhaps they're thinking along the lines of putting something in your head that would receive neural inputs for words and play them out. Though I think they may be trying to solve this the wrong way as they're studying the responses to audio stimuli, recording it and then using a computer to try and covert the neural signals back into the source audio. One thing they could do with this is perhaps make an implant that would effectively act like a cockpit flight recorder but for humans.

https://continuum.fandom.com/wiki/Cellular_Memory_Review

Presumably this is some form of machine learning, so I can see this tech improving rapidly pretty quickly.

I'm skeptical. I only had a skim through the paper, since I'm not very familiar with this field, but I do work with neural networks every day. First of all, this is heavily reliant on machine learning. Nothing really wrong with that (it's my job after all), but it does mean that they don't really understand how you go from brain measurements to speech or vice versa. It's a black box. One other thing to note is that when doing anything with machine learning, what data you have and how you use it is very important. Your method can succeed or fail based on how you use what data you have. They only train and test on 5 people, which is a very tiny sample size. Furthermore, they mention they do cross validation, which means you split the data into different parts: data you train your system with, data you tune your hyperparameters with, and finally the data you test on. It's very important you do this, otherwise your algorithm can cheat, by just remembering what it saw during training. However, they don't mention whether they split the 5 patients into those three categories (in fact they don't even mention what kind of cross validation method they use). This makes me think they use all patients during training. So there is no guarantee at all if this technique even works on other people.

So we'll have to wait for the replication before saying anything big.

good, yet another step closer to a brain-in-a-jar

https://www.youtube.com/watch?v=qdDo8WmS_es

Those samples are creepy as fuck!

I think without a doubt that it won't work on other patients, as the brains are different and it can be assumed the such complicated data as brain waves will be different too. But as a proof of concept, it's interesting and I think certainly useful. I can imagine each patient having their own little neural network that does the "transcoding" for them. As for the black box issue, they can still visualize the weight distributions at each layer, if it helps I have no idea about. But I personally don't see any significant breakthrough in this. We're more or less able to map any data to any other type of data using ML. All we need is to make it manageable for the networks to handle. A neural net is in the end nothing but a big function with millions of parameters that you optimize against some metric.

Understanding what's happening inside neural networks is still an active research topic. Just visualizing weights usually doesn't really help. But yeah, I agree. It's pretty interesting stuff, though not exactly groundbreaking.