Brainflight: controlling planes through neural activity
The day did not help, but nonetheless, with winds of up to 50km/h, the AR4 Light Ray took flight in the Lezirias Aerodrome and for a few moments, Nuno Loureiro, Champalimaud Foundation PhD student, commanded the plane with just his brain.
For the Champalimaud Neuroscience Programme’s Nuno Loureiro and Vitor Paixao, as well as the Tekever team accompanying this test session, seeing an unmanned plane being controlled by the mind was not new, but was an enormous challenge.
One thing is carrying out tests in a simulated environment where it is possible to control all the variables and parameters, with everything working well. But it is another thing entirely to do the test with a real plane, in the open air, with high expectations and on a particularly windy day. But nonetheless, it was mission accomplished.
“Is it difficult?” and “What do you think about when you want to give an order to the plane?” were two of the most frequent questions asked to Nuno Loureiro. The researcher, still with the Electroencephalogram sensors on his head, explains using an example: “It is like when we are learning to drive. Whenever we make a change we think about all the gestures that we have to make – stepping on the clutch, moving the gear stick to the right position. With practice and as we become more skilful, these movements become intuitive and we have moved from 3rd to 4th without even thinking about it, and we do it as we speak and look into the rearview mirror. "Apparently it is all a matter of learning and making an action an automated process by the brain. It’s hard to explain but it works ... and does the work.
But when Rui Costa, Foundation PI and head of the laboratory responsible for driving this project, is asked if anyone will one day be able to pilot a plane using only the mind, the answer is clear: “Each brain works, and carries out actions, in its own individual way. In theory, any one of us could, with training, learn how to control a plane using the mind, but it all depends on the individual learning capacity; and the truth is that…not all of us can be pianists.” So the aspirations of many of us will be unreached, but the potential of the application of the principle which was behind this “brain controlled flight” is huge. It is with this objective in mind that the Champalimaud Foundation team utilises this approach so that in the future we hope that we will be able to make full use of the brain’s potential, even when there are physical limitations.
But now we must move onto the scientific explanation of how everything works and the context in which this test flight through neural activity took place:
The brain has a notable capacity to learn complex rules and adapt itself to new situations. The European Brainflight project, comprising the Champaliamud Foundation (PT), a Tekever (PT), a Eagle Science (NL) and the Technische Universität München (DE), sought to take advantage of this learning process in order to develop a Brain Machine Interface (BMI), which allows the control of planes by neural activity.
The researchers from the Champalimaud Centre for the Unknown have, in recent years, been actively involved in the development of a BMI approach based on the natural process of learning. We can take tennis as an example: we begin by being unable to control the place where the tennis ball will fall. After some training we understand that there are a group of movements which are more successful than others, and we begin to hit our targets with the ball more frequently. In seeing the ball hit the target there is a positive reinforcement of the movement we have just done and so we select the movements we do, increasing our skill as tennis players. The process from which this occurs is called learning or operant conditioning, or the process by which a particular resource is selected when there is a positive reinforcement of the same. The learning process is thus a result of changes in electrical activity in the brain produced by neurons.
Rui Costa’s Neurobiology of Action Laboratory, used the basis of this learning process to develop a BMI system that allows the control of different devices. Nuno Loureiro, PhD student in the lab, explains: “In our group we are interested in measuring and understanding the changes in neural activity that occur when a person is being trained in a specific BMI task.” For this, the researchers developed a game that is carried out exclusively with the subject’s neural signals. “We record the signals measured by an Electroencephalogram (EEG) and transmit them in real time to a computer that translates them into stimuli for a computer game which the person is playing.”
In partnership with Tekever and the other Brainflight partners, the group of Champalimaud Foundation researchers successfully applied this process to the control of flight and drone simulators. Today’s flight session was one more step in this European project.
Following the successful simulator tests, the moment arrived to test this approach on real flights outside laboratory conditions. “The results which we have until now indicate that this paradigm can be used for neural control of a variety of devices, bringing possible benefits, for example, to people with motor problems.”, concluded Vitor Paixão, a postdoc from the laboratory. But there is much still to do, study and test. The work is currently in a phase of consolidating the results for scientific publication, bringing a moment in which it will be possible to find more information about the principal neurons involved in BMI and their possible applications, including medical uses.