Andy Clark on Embodied Cognition and Extended Mind

I recently watched a short talk by Andy Clark, a philosopher of mind who studies how humans perceive their bodies, control their bodies, and interact with the environment in a meaningful way. (Short summary of Clark’s ideas - these three seemingly different functions are basically just One function.) 

He has a new book called Surfing Uncertainty (haven’t read it yet) and some blog posts promoting the book (I’ve read some them). There is quite the buzz (and hype) on the internets these days about some of Clark’s main topics - robotics, predictive processing, Bayesian reasoning, and embodied cognition.

The video below discusses each topic in a way that is accessible, interesting and practical. Which is a nice contrast to other discussions of these topics I often see, which involve too much fancy language and talk of revolutionary new ideas. There are many variants of "embodied cognition” models for perception and action. Some pose as radical alternatives to conventional models and reject many of their core concepts (like computation, information processing, mental representations, and computer analogies.) Other versions are more moderate in their claims and, to my taste, more appealing.   

Clark seems to be more of a reformer than a revolutionary, so his talk has the flavor of “here is an interesting new way to look at motor control and perception”, instead of “everything we know is wrong!” Check it out for some interesting insights. (And in case you don't want to watch, I have a summary below.)   

The Extended Mind

Clark argues that a lot of our intelligence - the information processing that needs to occur for motor control, memory, cognition and perception - happens outside of the brain and even the nervous system.  Part of the reason is efficiency -  the brain doesn't need to work as hard if some information processing can occur in the body or the environment, or in interactions between the body, environment and brain.

Here’s a very simple, common sense example. If we want to remember a phone number, it is easier to write that number down and look at the paper, then to store the number in the brain and remember it. So we offload the task of memory to the environment, and then interact with the environment to retrieve the information. Clark uses the extreme example of a man who lost almost all of his short-term memory, Memento style, but used Evernote as a way to live a functional life. He argues that Evernote basically became part of his mind.

Another example would be using our fingers to count. By moving our fingers into certain configurations, they temporarily store information, so we don’t have to keep the count in our heads.

Or consider the hand gestures we make while talking. Clark believes their purpose is not just communicating our mind state to the listener through body language. (We make similar gestures even when no one else is listening.) Instead, hand motions might actually be part of the cognitive process of finding the right words: A certain thought causes a certain motion in the hands, which causes perception of the motion in the brain, which then stimulates other thoughts and so on. This dynamic feedback loop between thought, movement and perception is an essential part of the information processing which is needed for speech.  

Clark views the process of writing in a similar way - you have a thought, write it on paper, then read the paper, then have another thought, etc.  So writing doesn’t just happen in your head - it is a dynamic interaction between something that happens in your head, and then movements of your hand, a change in the environment, a perception about that change, and so forth. This definitely reflects my writing process. (For example, when I originally conceived this blog post in my head, it was quite a bit shorter. Apologies.)

These examples illustrate why people are interested in embodied cognition and the extended mind – we generally fail to appreciate how much information processing gets done in interactions between the brain, body and environment. From this point of view (which is very Feldenkrais-ish by the way), action, perception and cognition are all dynamically related and kind of inseparable.

Embodied Motor Control

Clark's ideas have many implications for movement as well, which is why people in robotics are interested. Just as thinking is more efficient if it offloads work to the body or environment, so is the process of creating the energy needed to move the body, and the intelligence needed to control it.

To illustrate this idea, Clark shows videos comparing the clunky and inefficient walking of a highly advanced robot to the smooth and almost human-like gait of a “passive walker” - a mechanical device with no motor or computer. Here is a video of a passive walker I have written about before.

Not quite John Travolta in Saturday Night Fever, but pretty impressive.  A great demonstration of how our movement patterns are governed not just by the nervous system, but by the structure of the body.  And of how creating efficient movement requires taking full advantage of the ability to do any work "for free." For example, when we swing an arm backwards while walking, the passive stretch of the anterior tissues will tend to swing the arm forward again, so that no energy is needed for the movement. And the shape and orientation of the arm and its soft tissues will cause it to swing it along a very definite and controlled pathway, so that no motor control is needed to trace that line. (Again, a very Feldenkrais-ish idea.)

Bluefin Tuna

Clark shares a cool example of some creative offloading done by the bluefin tuna. Apparently scientists were puzzled for quite a while about how the tuna could change directions so quickly with seemingly poor muscle development.   

It turned out that the tuna were powering movement through changing their environment - by creating a little swirl in the water with their tail, then swimming into that swirl in a way that would push them in the direction they wanted to go. They were basically creating their own waves to surf on.

Now humans of course can’t change their physical environments as easily as fish - but we can change our bodies, which changes the sensory information we receive, which in turn changes our ability to control movement. In this sense we are like the fish, using movements to create the sensory feedback necessary to create better movements. We are creating and surfing waves of information. 

Outfielders know where to run to field a ball not by calculating ball speed, and angle and then landing point, but by a far simpler process that takes advantage of the body. They center the ball in their field of vision, then try to keep it centered as the ball moves. If the ball moves left in the vision field, the body must move left to keep it centered. So positioning the body relative to the ball provides sensory feedback that is very useful in solving the motor task with a minimum of information processing. 

There are many other interesting examples here. How do you remember song lyrics or dance steps, or how to get to Grandma's house through the woods? You probably need to engage in the right series of movements to trigger your memory (or at least imagine them. ) 

Cognition is loopy

Clark challenges the idea that thinking or perception is the result of a simple process of sensory input (e.g. light on the retina), information processing (relay of the signals to the brain with filtering) and then output (subjective experience of vision and associated eye movements.)  

It’s not that this process doesn’t happen or that this process isn't important. It’s just that it is happening in a way that is very complex, dynamic, parallel, and looped. Information processing which involves a very linear process of input/processing/output is probably pretty idiotic and reflexive. A more intelligent and functional process requires many of these input-output events happening in hierarchies, in parallel, and in loops, where outputs immediately become inputs. Meaningful cognition lives in the dynamic interactions and complex patterns created by these loops.

Predictive coding

Clark also discussed the idea of predictive coding, which claims that the brain perceives and controls the body in large part by predicting the sensory feedback that will arise from an action and then correcting errors. 

In fact, Clark thinks that prediction is the essential task of the brain. Although many theorists characterize the brain as a "bag of tricks”  with many different ways of thinking and doing, Clark thinks that prediction and correction is a good way to unify all these seemingly separate functions. 

Predictive coding works like this. The brain uses stored knowledge about the world and the body to make predictions about the consequences of actions. Any mismatches between the prediction and actual feedback will result in corrections or learning. This is a pretty common sense process, but it can be applied in a very rigorous way using Bayesian probabilities, and the current scientific interest in this form of computation explains why we may be hearing these words a lot more in the future. 

There are several implications for this idea. First, our perceptions are very much driven by our expectations about what we will encounter. Also, the line between perception and cognition is blurred. What we think can change what we perceive and what we perceive can change what we think. I think the relevance to pain is clear.

Well this post has gone on far longer than I expected so time to wrap up! Leave a comment with your thoughts. Especially if I have anything wrong here - this is a complex topic!