Ant on the beach

Window of opportunity

2007-05-08T12:57:00.000-07:00

I'd like to try and formulate precisely the idea that is only a vague notion in swirling somewhere between my brain and the keyboard at this moment...

First where it came from:
1. In neural theory there is the idea that groups of neurons can communicate best if their firing rate is synchronized to the same frequency. 40 hz (40 spikes per second of the famous neuronal discharge that runs wavelike accross the membrane (axon) of the neuronal cell, from the core to the outer 'dendrites') seems to be a popular 'channel' in the human cortex. When the cells are synchronized, their 'windows for communication are open'. (source > FCDC)

2. According to our idea (Pim, Iris, Roel, Jelle) the brain and the physical environment interact and from this interaction dynamic cognitive structure is formed.

My idea is that the process in 2. is not unlike the process in 1. viz. that succesful cognitive structure can only be formed if the activity in the brain and the coupled activity in the environment (i.e. the physical changes in the environment) are 'synchronized'. Only then can we have a window of opportunity in which a stability (attractor) can be formed. You might also say that when environment and brain have the same rythm, the channel between them is open for communication.

EEC and PD

2007-05-08T12:39:00.000-07:00

I teach two courses on participatory design. This is the method of involving the end-users in that actual process of designing new products or services.

Theoretically I am interested in embodied, and even more in embedded cognition. Embedded cognition states that thinking is a process that emerges out of the interaction between brain, body and world. Parts of the world get recruited to perform important functions in a cognitive proces. They serve as external memory aids, external representations, visual clue's, they constrain possible behaviors (reducing choice options), present automatic orderings. The physical world is also the medium by which previous behavior of the agent itself, by the traces it leaves in the physical world, gets to influence subsequent patterns of behavior.

Yesterday I read a paper by Hollan, Hutchins and Kirsh. I already knew Hutchins and Kirsh as representatives of an embedded cognition perspective. As it turns out, Holland works in the same dept. in USCD (San Diego) and he is a representative of the participatory design movement (or to be more precise: ethnographic methods, which is in somewhere in the ballpark). The paper presents an argument of why, on the basis of an EEC perspective on cognition, one should do research and design using ethographic methods.

The argument runs as follows: Since parts of the local environment become part of the cognitive strategy that users use in dealing with a technology (think of an airline pilot in a cockpit), their expert knowledge is very 'personal' and the meaning of the interactions that the user engages in, and the function that certain parts of the artefact get to play in this interaction, is highly personal as well. This means that an objective, third-person investigation, especially in a sterile psychological lab, is not going to give you any inside into what is really going on in this user's real-life activities. EEC patterns of behavior grow in an historical process in which chance events and personal histories of users can have big influences on the resulting roles of the interface components in the activity of the user. So, we have to follow an empathic perspective on research and design, doing lot's of observation, interviews 'on site' and perhaps even 'participatory design': letting the user be part of the design team, as an expert of a knowledge domain that other people (not being the user) simply have no access to.

This is a nice article for me because it combines two of my interests that I hadn't really linked so explicitly. Apparently the various activities I engage in have some inner logic that I wasn't aware of yet. I wonder how Salsa dancing is going to fit in the picture...

Embedded neuromodulation

2007-05-03T12:15:00.000-07:00

This post gives a summary of an article I've just read. It is mainly a note for myself.
The article can be downloaded here

The article describes experiments with an autonomous Kephera robot. Sporns et al have modeled a neuromodulatory system, resembling the dopamine reward system in the brain. This dopamine reward system influences the plasticity of the robot. Whenever unexpected reward takes place, the dopamine system get's active, this leads to 'value-dependent learning', both the sensorimotor connections get directly affected (normal stimulus-response associations are formed) and the neuromodulatory system itself get's affected.

QUOTE from a related article
Value signals combine temporal specificity
(they are phasic and short-lasting) with spatial uniformity
(they affect widespread projection regions and act as a
single global signal). Value enters into traditional Hebbiantype
synaptic rules as a third factor, in addition to factors
representing pre- and postsynaptic activity. Because of their
phasic nature, value signals effectively gate plasticity, in
addition to influencing its magnitude and direction (see
below). Value affects plasticity more or less uniformly
throughout the widespread cortical and subcortical regions
to which value systems project.

The interesting thing is that they put this system (which I do not completely grasp at the moment but at any rate resembles something like a sensorimotor system that gets 'laden' with internal, bodily based 'value', depending on reward, which is like Damasio in a way, i.e.: embodiment) in a real environment with objects. The behavior of the robot in the world influenced the subsequent inputs of the robot, because at the beginning, reward giving objects (the red objects) were dispersed quite homogenous in the environment, but the behavior of the robot lead to the effect that clusters of red objects were formed. The result was that at first there was a quite predictable timing/rythm in which the robot would first detect, visually, a red object, then grab it (feeling it with a touch sensor, thereby receiving the reward, which was supposed to model 'tasting/eating' it). But later on, all red objects were clustered, so after an initial delay, suddenly the robot would get massive amounts of reward in short time intervals.

QUOTE from the article:
Our experiments document a progressive alteration of an
environmental variable (the spatial distribution of reward
throughout the environment) due to the behavioral activity
of the robot. This alteration, in turn, has consequences on
synaptic patterns encoding predictions about the
occurrence of future rewards.
It is especially noteworthy that the differences between
early and late phases in experiments with high object
densities are neither the result of purposeful
rearrangements of the environment by either robot or
experimenter, nor are they due to the adjustment of
“internal” variables over time such as learning rates, cell
response functions, or motor variables. Instead they are the
outcome of the coupling between brain, body and
environment. This coupling is strongly reciprocal.
Behavior affects the statistics of reward timings which
drive synaptic plasticity through activation of a
neuromodulatory system. In turn, synaptic changes alter
the coupling between visual and motor units which affects
behavior.
ENDQUOTE

Here they even suggest a possible role for embeddedness (i.e. reshaping your own environment) in the emergence of addiction:

QUOTE from the article
The experiments discussed in this paper may shed light
on the activity and functional role of neuromodulatory
systems (in particular, dopamine) in the course of
“natural”, self-guided behavior. The “attractive force”
exerted by clusters of rewarding objects, resulting in
restricted trajectories of robot movement and navigation as
well as repeated “rapid-fire” sequences of reward
encounters are especially intriguing. Disruptions of the
neurobiological bases of reward processing are thought to
form a major cause for lasting behavioral changes and,
eventually, chronic disease (addiction) in humans. Our
results suggest the hypothesis that a pattern of persistent
reward-seeking behavior may in part be generated as a
result of a progressive reshaping of the environment
coupled with long-lasting synaptic changes in specific
neural structures. Future experiments will investigate this
hypothesis in detail.
ENDQUOTE

For me this article shows that neuromodulation (embodiment) and embeddedness can be part of a larger perspective in which brain, body and world form a tightly coupled system, where the causal work depends on interaction between both world-events (behavior that reshapes the environment) and internal modulatory signals (reward leading to changes in synaptic connectivity - and hence, in the speed/ease of learning). In this article it is shown how this could work out in practice.

Time

2007-05-01T15:11:00.000-07:00

As of this week I am in a philosophy group at work. I get one day per week to write philosophical papers - to be published in professional journals.

The first paper will have to be a follow up on the paper that Iris, Pim, Roel and me wrote (and will be published in Theory and Psychology). This paper deals with the question of what the new 'role' of the brain is when one starts thinking about intelligent behavior as being emergent out of an interplay between brain, body and environment. In contrast, of course, stands the cognitivist idea of the brain as being a sort of computer that takes in sense data, deciphers from these sense data a message in the language of thought (i.e.: 'meaning'), and then, on the basis of internally stored knowledge of the world, produces a response in the form of a behavior/act. The new way of seeing the brain, we venture, is that whenever the brain comes into action, *there is already behavior going on*. Lot's of behavior emerges out of the interplay between our body and the world. On a low level, parts of our nervous system help in forming structural couplings between the organism and its environment, leading to adaptive behaviors. This is not based on representing the environment internally, but based on forming a stable 'relation' between brain and aspects of the environment. Once such a low-level sensorimotor relation has been formed, higher parts of the brain can (but don't always) put a *bias* onto this lower level system. This bias works as an internal control parameter in a dynamical system. Increasing the value of the bias gradually can lead to dramatic qualitative changes in behavior. But the bias itself did not cause the behavior, it is just a bias. You need a fully operational sensorimotor loop for the higher brain activation to be able to effect it's causal work. Just like you cannot steer by just having a steering wheel in your hands, you also need to have a car that the steering wheel is attached to. But perhaps this is a bad analogy.

The next step is to take this starting point and write a new paper which has to involve, in some way, the subject of TIME, since time is the main theme of our philosophy group. I proposed that 'timing' is essential for the formation of structural couplings between organism and environment. I will be working out this idea in the coming weeks here in this weblog.

Abstractions

2007-02-26T05:19:00.000-08:00

This blog (I mean: this post) is meant mainly for a discussion between Iris, Pim, Roel and me. But everybody is happily invited to join. This post is about the concept of 'abstraction' in relation to 'representational/non-representational theories of mind'. It is based on my reading of the following paper: Shimon Edelman (2003) But will it scale up? Not without representations.... [snap].

Ok here goes.

First let’s be clear on what is means to have (or to be) a representation.

Back to square one.
“A Rep is something that is used by a system as a stand-in for something else” Haugeland

My question would be this one: Does abstraction automatically mean ‘representations present’?

Let’s consider a few kinds of processes that could or could not be said to be representational.

Type 1 : Symbols of classes
Suppose I see a cow on a picture in a children’s book of animals. Next, I see a real cow in the field. I ‘know’ that both these ‘cows’ have ‘something’ in common. I say: These are both the same (these are both cows). And someone understanding English would agree, perhaps after adding ‘in some way they are the same, yes’ (implicating that in some other ways they are not!).

Now because of these things being the same I am allowed to call each of them by the same name: “cow”. The word cow is thus a representation of ‘that which makes both these objects ‘the same’. But it is not precisely defined what the word ‘cow’ actually refers to. So one can have ‘stand-ins’ of ‘somethings’ even when the something itself is ill-defined or even unknown (even unexistend, as with unicorns). When I equate the picture and the real cow what I am in fact performing is an abstraction. I can make an abstraction from the real cow, and from the picture, and in the abstraction these two items become one. Or at least they become items that belong to the same ‘abstract class’ of things.

But of course, the cow in the picture is not a real cow (ceci n’est pas une vache). The cow in the picture is a representation of a cow, some cow, a general cow, the idea of a cow, or what have you. The picture, like the word ‘cow’ is more a ‘symbol’ of the ‘concept cow’, whereas the real cow is an exemplar of a real cow (or: a token of the type).

Still, I could also have performed an abstraction on two real cows in saying ‘these are both the same’. I would still be performing an abstraction but without there being an external representation involved. Or is that impossible? The question is whether one needs internal representation in order to perform abstractions

Type 2: Representations of particulars
Suppose now I own a cow. One day I decide to make a painting of this cow on canvas. The next year, my cow dies. I have paintings of all the cows I once owned in my living room, since I am a real cow-lover. Someone asks me: which of the cows did you love most? I immediately point to the third painting from the right: that cow I loved most. Now here’s your classical representation. The painting serves as a stand-in for the real cow, in order to ... (whatever). In this case: I use it to communicate about my feelings and to differentiate a particular cow from other cows.

3 Signals in a communication channel
Suppose now I see a cow on a television screen. The broadcasting is live, so when the cow moves (out there) I see it making the same movement (here on the screen), realtime (i.e. semi-realtime, there is always a lag). The cow on the screen, that is: the pattern of electronic bursts on the tube, or the illuminated pixels on my TFT, is, of course, not the real cow. But the cow that I see (on the screen) is the real cow. How can this be? The real cow is, one could say, ‘delivered to me’ via an artificial channel. But it doesn’t matter how it is delivered, what matters is what is being delivered. In this case: the real cow. Even if the movie was not even live but a replay of happenings hours ago, it would still be the real cow, as it existed hours ago. There is only a technological distance between me and the cow but this is nothing else from seeing the cow ‘real-life’. For considere me standing in the field with a pair of binoculars. Would one say that what I see in the binoculars is not the real cow? In the latter case the technological distance would be short, perhaps only as thick as the bi-focal glasses on my nose or the binoculars in my hand. But whether it is glasses or TV-equipment, that does not make the fundamental difference between real or fake cows. Only the physics of the communication channel are a bit different.

Are the electronic bursts on the tube are a ‘representation’ of the cow? I believe not. I am not ‘using’ these electronic bursts at all. Or rather, I am ‘using’ the electronic bursts on the TV tube, but I am using them in precisely the same way that I am using the electronic bursts inside my brain, i.e. the patterns of activity in my retinae, in my optical tract, in my thalamus, in my visual cortex. So say that all of these patterns are representations of the cow is just a matter of definition. I could also say that they are not representations of the cow, merely ‘channels of communication’. All of these signals, inside or outside my brain, are part of the communication channel that connects the cow to ‘me’. If you would care to call these channels ‘representational’, fine. But: note that these kinds of representations are qualitatively different from type 1 and 2 since in this case the cow is ‘right there’ with us. The signals are not a ‘stand-in’, they are the connecting medium between something that is ‘still there’ and the cognitive agent.

[Right now I am not discussing what all this means for the definition of “I”/“me”/agent]

4 Biasing signals originating in other channels
Suppose now I drive my car in rural country. It is raining, can’t see a thing. Or, to be precise, I can see something. Big dark blots of somethings seem to be blocking the road. I cannot see ‘what they are’, though. People? Cars? Then I hear a big MOOOOOOH. Upon second inspection I now see tails wagging, heads turning: several cows are stumbling about in front of my car. Here, we have a multichannel situation. The first channel contains a lot of noise. I cannot see the cows, only ‘vague images of somethings’. The second channel, via the auditory nerve, provides me with a signal that biases my visual perception of the world. Suddenly I now do see the cows, because the auditory signal helps me. In terms of Haugeland, the sound of the cow is not (necessarily) a representation of a cow: the animals are right there, I can see them, but I couldn’t’ve recognized them without the biasing signal. So it is an addition of some sort, not a replacement. In dynamical terms one would say that a dynamical process that is seeking (but not finding) it’s attractor is pushed over the edge into an attractor valley by an additional forcing variable.

Afterthoughts:

Just the other day I was thinking: My biggest problem with cognitive science is the problem associated with abstraction. Consider the cognitive process that underlies our utterance: “the picture in the book and the animal in the field are the same thing; they are both cows”. Now people can do this. And in many circumstances it is a useful, functional, intelligent, smart, coherent, appropriate thing to do. But that does not mean that there exists a phenomenon called ‘dealing with cows’ and that the behavior associated with watching a picture in a children’s book is ‘in some way the same as’ the behavior associated with watching a cow in the field. Especially not if, for instance, there happens to be no fence between you and the real cow in the field. Then, suddenly, one realises, that real cows are in a lot of ways not the same as pictures of cows in children’s books. Suddenly, one realises that cows are very big animals. Suddenly it becomes extremely important to differentiate between a cow and a bull. In other words: the interaction between you and a real cow is qualitatively different from the interaction between you and a picture of a cow. So although the capacity of being able to abstract away from real cows and pictures onto the abstract class of ‘cow’ is an important cognitive achievement, it is not true that in order to understand how we deal with either real cows and or pictures of cows we need to take this abstraction-process as the basic cognitive operation that explains our behavior. We do not generally deal with cows on the basis of an abstract concept of cow to which both pictures and real cows belong. We can (e.g. in school, in an intelligence test, in reading this text), but just as often we don’t. The real cognitive structure that explains our behavior in the case of the children’s book is, I think, completely different from the real cognitive structure that explains my behavior when I’m out there in the field facing a real cow. (To elaborate on that just a little bit: when a child is reading a book with a picture of a cow, I guess the child is not doing anything that is even remotely connected to real cow. What the child is doing is connecting words, behaviors, and pictures in a learning process. The first connection between the real cow and the picture of the cow is made because other people are connecting the word ‘cow’ to both of these experiences. Thus, the connection is highly artificial, explicitly trained by an external tutor and linguistically mediated. In that sanse the picture is more a symbol than a realistic ‘image’ of the thing it refers to. This can hardly be called a ‘fundamental property’, it is more like an advanced specialisation of our cognitive system. The fact that in medieval times pictures of things were attributed all kinds of psychological or magical properties is another case in point: it is apparently only a recent discovery that a picture of something is ‘just a picture of something’).

EEC versus Cognitivism

2007-01-05T14:09:00.000-08:00

There has been a debate going on for some time now regarding two conceptual frameworks in cognitive science. The standard, mainstream framework is usually called cognitivism or "the information processing view" of cognition. Bluntly speaking it states that the human mind is a piece of software consisting of computational structure and stored knowledge (representations) that is running on (or as they say somewhat less bluntly: physically instantiated by) a piece of hardware aka the brain. The other conceptual framework is called situated/embodied cognition, which are actually two frameworks that sort of go hand in hand and the total of which I will hereforth call "embodied embedded cognition" or EEC. EEC states that the human mind is an emergent property of both the physical body and its characteristics, the physical local environment and its structure, and the processes of the brain. I will concentrate on embeddedness / situatedness in what follows.

Embeddedness/situatedness refers to the idea that 'the world itself' can do some (or all?) of the computations that we would usually hold to be a product of the processes of the brain. A famous example, cited by Lave, is about someone who, when asked to make a cake for 3 people when the original recipe was for 4 people and involved 500 grams of dough. What this person does is that he simply makes the dough for 4 people, squeezes the dough into a pancake, and then takes out 1/4'th of the circle. The remainder is just about 500 * 3/4, which is the amount that was needed. In this case, the brain of the person in question did not have to calculate the assignment 'in his head', instead, the physical properties of the world were used in concordance with our perceptual abilities (the visual ones) that happen to be such that allow us to quickly and quite directly perceive and cut out a quarter of a circle. (It is actually not unlike explicitly, physically, calculating the assignment (500 / 4) and then the result [* 3], which is what we learn if we have to multiply by ratio's by heart, since "division is multipyling by the reverse", as they say in Dutch schools.

Anyway.

Some people are die-hard cognitivists, other people are hard-core EECers. Practically, I would like to pursue EEC empirically, give it a chance, and see where it ends. If the framework is no good, it will eventually die out. If it is viable, it will live and we will learn from it. But this is not what this blog is about. This blog is about the fundamental theoretical issue of which framework is the preferred framework if we would have to choose *now*, based on what we know at this moment.

To me, the most interesting *theoretical* question of the debate between the opponents would be: which of the two frameworks constitutes the ultimate 'ground' on which the other framework is built? Because as anyone would acknowledge, the typical processes that are taken as fundamental in each of the frameworks *do happen*, at least descriptively, or in our conscious experience. So if one of the frameworks would turn out to be false, the processes that this framework took as fundamental would still need to be explained in the vocabulary of the *other* framework. To be precise: if cognitivism is false, EEC would need to be able to explain how people can represent facts, store knowledge, and 'reason', follow procedures (by heart!). For example, EEC would have to explain how it is possible that people *can* solve the assignment 500*3/4th by heart if they need to, without the help of pancakes. Conversely, if EEC is false, cognitivism needs to explain how this dependency of human cognizers on the external world can be sustained by a cognitivist system. Using 'knowledge in the world', as Donald Norman calls it, is a process that exists, but if cognitivism is the game, then it must explain how to do it using representations and computations.

Let us see what both camps would argue. Since cognitivism is mainstream, I probably need a bit more words to argue for EEC being the ultimate ground.

For a cognitivist, all this talk about situatedness is just a 'complex' version of cognitivism all along. The cognitivist would say: yes, of course the computational-representational system could *use* the physical world as a means for storing information, or as a tool for quickly computing stuff that internally would be more demanding. There are machines, road-signs, etc etc.. Nobody denies that. A cognitivist view of the mind does not mean that the cognitivist brain isn't aloud to use 'smart tricks' if it knows of them. But the 'currency' in which information is traded between brain and world consists of hard cognitivist coins nevertheless. Whenever you 'use' the world in order to let it calculate for you, you need to make contact with the world, and when you do, you do so in your guise of being a computational-representational system. There is no other way of interacting with the world cognitively than in terms of information-processing. Meaningful signs need to go in, and behavior needs to come out. In the Lave example, you need to be able to perceive, encode and store the information that came from the visual system that was observing the pancake. You need to understand that there is a smart way of calculating 500 * 3/4 without using your inner resources. You need the knowledge in order to know what to do in order to implement this smart plan of yours. You need to device a motor plan that actually let's you cut 1/4th of a pancake. You need, in sum, a complete 'psychological system' in order to do the 'embedded stuff' that these EEC-ers brag about.

For an EEC-er, things are completely the other way around. People like Merleau Ponty try to completely build up a new way of looking at the same old psychological phenomena, such that embodiment and embeddedness are now more fundamental, and 'prior' to processes like computation and storage of knowledge. This is somewhat counterintuitive because we have learned to talk about our experience in a cognitivist way, but phenomenologists say we should 'bracket out' this talk since it is artificial and return to the core experience itself. Anyhow, our intuition and conscious inner speech about what we are and how we do the things we do can of course not be trusted, even in a materialist empirical science. So, what EEC-ers say, e.g., is that in low level motor planning as well as in low level perception, EEC-like processes already exist. Perception and action are tightly coupled systems, such that, even the most low-level perception of objects is already 'aided' by certain specific actions undertaken by the agent, in the world. Perception is dependent on action! For example, the perception of objects is not just based on a passively received pattern of excitations on the retina, but it is also dependent on the pattern of *actions* taken by the agent as it saccades with its eyes in the scene of interest. That is: movement of the eyes *constructs* a series of inputs that contains the sort and type of information it contains precisely because this series was generated by movement of the eyes. In other words: part of what we 'see' is our own actions reflected in the effect these actions have on the world. And this is *low level*, in the order of milliseconds. All the rest of cognition needs to be built from these basic perceptual building blocks. E.g., take our perception of objects. Objects, as we perceive them, are not objective 'things' in the world, but already a blend of 'what is out there' and 'how we, with our physical capacities approach that what is out there'. Perception is already embedded and embodied. See also William Gibson's 'affordances' in this respect. Now if EEC is something that is already there in low level processes, then the question is: can we push this conceptual framework higher up (and why not?). Does it (Why wouldn't it?) scale up all the way to centre-court psychological processes? Even if everyday cognitive phenomena, like memory processes or planning and reasoning, need some 'extra' explanatory apparatus involving talk of computations and representations, it could still be argued that this 'extra' is just a 'complex version of EEC processes. EEC is the fundamental basis, and only somewhere upstream in the complexity of things are we able to perform 'computations' and 'store knowledge'.

As it stands, I think that cognitivism *appears* to have a better, detailed story about how EEC-like processes could be implemented by a classic information-processing system. I say appears, because it is my strong feeling that a lot of the *words* that cognitivism uses actually need a lot more explaining if one comes right down to asking what these words mean, precisely. Cognitivism has been very smart in assuming this Cartesian split between 'software' and 'hardware', which makes it such that practically anything can be 'assumed to exist' on the software level without the cognitivist having to explain exactly how it is 'implemented', as long as the process in question can "in principle" be implemented on an information-processing system. And what process could not? (Noncomputable processes? Nonrepresentational processes? But what are they?). So, cognitivists have the easy way out: as a framework it can account for any type of process that researchers come up with. But is also makes the cognitivist coin rather valueless. For if everything is possible in an information processing system, why are people not 'everything'? Why are we only the bunch of phenomena that we actually are?

EEC has a problem in that the question of how EEC-processes 'generate' full-blown cognition (planning a hike in the mountains, having a conversation with your partner about something that happened yesterday, remembering, by heart, or via reasoning/recalling, where you left your socks the day before, immediately recognizing a person, knowing his name, being able to recall your relation to this person, and so on) simply cannot even be *articulated* in an EEC way. The cognitivist response (oh well, it is just all computational procedures, even the EEC stuff is), is not possible. Suppose I sit in a train and think about what to have for dinner and what to remember to buy at the grocery's that evening. How would the train's interior, the lights, the other people present, the music on my I-pod, how would those physical actualities possible contribute to this cognitive process that is actually taking place right there and then? Yes, if I had a paper and pencil, I could come up with an EEC story. But I have no paper and pencil. I am thinking, creating and remembering my meal for the evening *in my head*. On my own. No environment present. I could saccade my eyes for all my worth but it wouldn't help me in remembering the recipe would it?

In conclusion, both frameworks have problems. Cognivism superficially is stronger, but I think that this is artificial due to the fact that in everyday life we talk in cognitivist vocabulary already so it easily seems as if cognitivism has 'explained' something where in fact it has explained nothing, it only resonated with our everyday way of naming and talking about things (which in itself is not an explanation of anything at least not scientifically). The problems that AI systems have in dealing with 'common sense' behavior are a case in point here. We are easily seduced by cognitivism. But EEC offers no real alternative either - yet. It is up to the EEC-ers (us, that is) to explain how cognition (real cognition) is possible in an EEC system without secretly, implicitly, assuming a cognitivist system that *does* all these fancy embedded embodied tricks.

Afterthought:
Merleau Ponty and his phenomenology could be a viable starting point, if not for the Very Complex and Inunderstandable Writings of these French guys. The problem with these kinds of writings is that once you understand it, you, for yourself, can use MP's ideas to 'explain' the EEC fundamentals allright, but then still the rest of the academic world does NOT understand MP and so you have put yourself on some island together with the other 2 people that understood what he is trying to say.

What's inside?

2006-12-14T13:30:00.000-08:00

One and something year old Jonas is playing games. Self-designed, highly entertaining games - at least for his two biggest fans. The semantics of the game are sometimes difficult to assess, but the behavior has a clear, observable, repeatable structure and it is clearly something new that was not witnessed before.

For instance, Jonas will walk to his mother, slap her knees with both hands, laugh with excitement, turn round, walk to his father who sits at the other side of the table, slap his knees with both hands, laugh with excitement, and repeat the procedure again and again, walking back and forth between his two parents. I had not seen this kind of structured, repetitive behavior before, it emerged somewhere in the last week or weeks. The behavior is marked by some arbitrarily sequence of acts that is being repeated over and over.

And now, the little man is asleep. As fatherly pride slowly fades, philosophical reflection appears on the scene.

If we try to describe what we, observers, experience while watching the child, we would quickly choose some appropriate words for what we experience to be 'a new behavioral capacity'. In the example above the words are 'playing games'. Cognitive science excels in the next step, which is to replace these words with other words that supposedly catch the 'essence' of the observed phenomenon. We can expect analyses such as this one: Playing games, is "essentially" the ability to express and follow a rule-structure.

At the same time, neuro-scientists are carefully investigating what has changed in the neural organisation of the brain, since it is clear, at least to these scientists, that new behavioral capacities must be caused by significant changes in brain organisation.

Suppose the brain-scientist indeed discovers a structural brain change that is reliably associated with the onset of 'game playing' behavior in small children. Would that be proof of the theory that the new neural organisation is responsible for "the ability to express and follow a rule structure"?

I'm afraid not.

What we have, as a fact, is that there is a brain change that correlates with a behavioral change. The behavior is complex, it is real, it is located in space and in time, it emerges within a physical and social environment. The 'game' is played using a physical body with physical properties. It is played in a context of social relations between father, mother and child. It emerged out of a situation in which the mother and father were sitting opposite each other at a table from which the child was just leaving after having had dinner. Within this *practical, real* situation, a brain change had its effect. Game playing was the result.

Abstracting away from the observed phenomenon to the underlying 'essence' is a dangerous activity. It is often grounded in values, beliefs and perceptions of the observer. It is also constrained by the language in which the abstraction is expressed.

But even if the abstraction is a valid one, there is no proof whatsoever that the brain change is in itself corresponding to (representing) this 'essence' that is described by the abstraction. It could be that the brain change, in itself, is something very different. Something that, in itself, has nothing to do with 'following a rule-structure'. The newly observed behavior of game playing, and its associated 'essence' - following rules - might be emerging only when the 'updated' brain is operating in an appropriate physical and an appropriate social environment, preceded by an appropriate history of actions.

Cognitive science has a strong history of what Churchland calls 'vertical analysis', in which behavior is broken down into several, vertical, 'columns', corresponding to the classic 'faculties' of mind, and its modern heirs, the computational-representational 'modules'. "Rule-following" is such a module. But in the process of breaking down the observed phenomenon into meaningful abstractions (essences), we can also choose to cut reality in horizontal slices. A horizontal slice corresponds to a full-blown, functional, complete sensorimotor loop in which parts of the brain, the body and the environment take part. Structurally new behaviors that clearly mark cognitive developmental phases, such as the emergence of 'game playing', might be explained by the development of a new second order influence within the brain upon such a horizontal, existing, and operational, sensorimotor cycle. The brain change thus comprises not of a new module that represents the new 'capacity', but it comprises of a new *bias* upon the existing agent-environment interaction.

As they say in pop-music: the band and the audience make the show.

Interactions (3)

2006-11-24T12:25:00.000-08:00

I would like to add a bit to my ideas about the concept of interaction. It is very abstract, these are just first thoughts that have to be worked out later.

We discussed interaction between things, between organism and things, and between organisms (specifically: between humans). There is another division that I like to introduce here, which divides the concept of interaction in yet two more forms. This division is not based on the kinds objects that interact, rather it is a typology of the kind of effect that the interactional process itself may have.

Consider two machines interacting. That is: each of the machines performs acts that have an effect in their respective environments. Both machines are part of each other environments. Moreover, there are reliable relations between the actions of the one machine and the effect such an action produces in the pattern of actions of the other machine. In common terms we would say that each machine 're-acts' to the other machine's actions. Another way of saying this is to say that the two machines 'inter-act'.

Although each machine influences the pattern of actions of the other machine, the rules that govern such interactions are fixed. That is, the interaction changes the behavior of both machines, but it does not change the pattern of interaction itself. I call these rules, or patterns if you whish, the 'structure' of the interaction.

Now the big divide I want to introduce is between systems that can, or cannot, change the structure of the interaction, by interacting.

The first category of interaction I call 'fixed interaction'. The latter case I call 'developmental interaction'.

Interacting machines generally are fixed systems. It is a technological-empirical question whether we will one day come to know of machines being able to develop, through their interactions, their own interactional structure. The current examples in Alife and AI do not convince me, yet. [cf. arguments of a.o. Tom Ziemke]

Organisms, however, interacting with their passive environments or with other organisms, constitute active interactive systems. (I just state this as a fact. It is of course very well possible to have a discussion about the validity of this claim). Such systems change their interactional structure, by interacting. This means that the rules that govern the interaction change. The psychological interpretation would be that such a system is able to learn from experience.

I want to end this discussion for the moment, but not before sharing with you a glimpse of where all this is leading: If we ask ourselves, what is an organism? What is the essence that makes something alive, and what makes a system an active, behaving system? It is my belief that such a system is a developmental interactive system and that most of what we call 'the organism', is in fact interactional structure that has developed, both on philogenetic and ontogenetic timescales, in so called 'layers' (I will explain this in a later blog). Because the newly developed interactional structure has a stability, we often forget that this structure is *interactional*, it is part of the interactional system, not merely 'part of the organism'. In fact, there is no 'organism' if we do not consider it in the context of its environment. But this is for tomorrow!

My funny Valentine

2006-11-03T13:05:00.000-08:00

Jass musicians (and likewise blues and pop musicians) never play what the score tells them to do. To be precise: they will deviate heavily from the prescribed rythm. Most notes will be played later or earlier. If one plays exactly what is written the music will sound dull, obligatory. A good example of this is when big orchestra's or choirs play well-known pop-songs (you know: London Philharmonic playing Nirvana's Smell's like teen spirit, with the heavy violin sections). What misses in these orchestra's is the 'groove' of the original song. It is because the orchestra is playing exactly what the score says.

Of course, the 'right way' of playing it, is what the performers do, not what is written down. So it is not so much that the musicians are making errors, rather than te fact that writing it down in a score is just a poor method of representing jazz.

The deviations from the score are not random, however. A researcher at the NICI once gave at talk based on having measured all deviations from all notes in the written score in several versions of Chet Baker's 'My Funny Valentine'. As it turned out, the deviations, as a whole, form a well-defined pattern. If a jazz-performer follows this pattern, the music will sound right, people will regard the music as 'a good piece of jazz'. In fact, the only version of Chet's Valentine that couldn't be mapped to this pattern was a bootleg version that Chet never wanted to bring out because he wasn't satisfied with it, for some reason.

It is interesting to see that the 'jazz flavor', which marks the difference between a dull reproduction of the written score and a lively, groovy jazz performance, is itself completely explained by a mathematical formula. In theory, we could make a computer play lively jazz if we put the pattern that the researcher discovered and the written score together. In that sense, if we take the score and the computational pattern of deviations, we have 'explained' My Funny Valentine. But does it also explain anything about how Chet actually plays My Funny Valentine? With this I mean: does it say anything about the possible *causal mechanism* that leads Chet to play MFV in this particular way? I feel not.

Of course, there is the possibility that Chet actually embodies a computational mechanism that takes the score, transforms all the notes by adding and substracting the required deviations here and there, and then outputs the result on his trumpet. To be sure, on some level of *description* Chet *does* embody such a mechanism since this *is* exactly what he outputs on his trumpet. But I propose to make a distinction between such descriptive characterisations of the mechanisms at work, and the *real* (whatever that may be) mechanisms that actually caused the notes to be played. In particular, I see no reason why networks of cells in the *brain* would actually have to form computational patterns like the one described above in order to let this same pattern emerge out of Chet's trumpet. It could very well be, for example, that certain typical parameters on the level of Chet's breathing, lip-tension, and so on, add to the emergence of this pattern. The pattern of deviations from the score (the jazziness of jazz) might emerge as a *collective property* of Chet-as-he-is-playing-the-piece. A pattern that evolves 'in the flow of things', not something that has been 'planned' by a control system. Perhaps induced by subcortical, emotional parameter-changes and body-posture, lip-tension, and so on. Such a pattern would not have to be explicitly represented as some kind of computational *program* inside the brain, even if it emerges every time Chet plays the song. (Nobody 'programs' traffic jams, still they emerge on the same hotspots almost everytime).

Most researchers are not interested or they heavily disagree with this alternative. The first group is generally not interested in 'causal mechanisms'. They simply seek to find 'patterns' in behavior. The patterns themselve are enough 'explanation'. The second group disagrees strongly with the difference between the 'descriptive computational patterns' as observed and the 'real computational patterns' that are proposed as an underlying cause. Which is sort of the same thing, really.

Your lips are laughable
Unphotographable
Yet your my favorite work of art..

If you can spray them, they're real

2006-10-24T07:19:00.000-07:00

Yessireeya. There it is once again: the age-old discussion between the realists and the instrumentalists.

It came up in a reply by Sander to my earlier posting on perception. Sander suggested that theories are only 'useful' and 'just mathematics'. Electrons are formulae, not real things. I replied quoting a philosopher who's name I'd forgotten, about spraying electrons, but it turned out that the example was about spraying photons (oh well) The correct story, with the name of the philosopher included, I refound on the web here.

'When Ian Hacking, for example, once asked a physics colleague what he was doing, the physicist replied that he was "spraying photons". Impressed, Hacking wrote: "From that day forth I've been a scientific realist. As far as I'm concerned, if you can spray them, then they are real."'

(I just love Google: I searched on "spraying" and "philosophy". Got a first hit!)

Turns out that this article that Google popped up for me is very interesting in itself. A researcher did an internet poll among physicists asking what kind of things they consider to be real, and which things they consider to be 'not real'. The list is long, including things like "concepts", "phlogiston", "electrons", "earth", "colours", basically everything you can name. He then writes a long review, discussing all issues involved, and along the way you basically learn about all the different philosophical positions one can possibly take.

Like, unreal dude!

Nijntje's restaurant

2006-10-13T03:52:00.000-07:00

Currently I've been listening a lot to a CD with tracks that tell the story of a particular female rabbit. If you happen to sync in my developmental zone, you probably know about this rabbit. Here name is Nijntje.

In one of the songs Nijntje is playing 'restaurant'. She puts on a paper hat, made of a news paper, and sings "Now I am the chef of Nijntje's restaurant".

From a certain age, children start to use physical objects as symbols, representing something else. The 'something else' can be a physical object, but it can also be some agreement, some fact, or a relation between objects. In the example above, there is a double-take as far as representational content is concerned: first, the newspaper is used as a stand-in for a real chef's hat. Second, putting on this chef's hat automatically signifies that Nijntje is now a chef. (In fact, there is a three-double take considering that Nijntje is implicitly representing a small child, where in fact she is a rabbit, or even mor factual: she doesn't exist at all). It is remarkable, if you come to think about it, how young children, in the age of 4+, are actually able to understand such subtle semantical relations.

There are several possible explanations to this representational capacity of young children (and adults, likewise). What I want to discuss here, however, is that regardless of the underlying mechanism, one sees that throughout development and over cultures human beings have a tremendous *need* to use physical objects as representational stand-ins, in going about their daily business. It is as if we couldn't achieve what we are achieving everyday, if we were not able to use physical stand-ins. Consider the effort Nijntje would have to display in convincing the other children that she is a chef in a restaurant, if she didn't have some physical object with which to symbolize this role in the game.

Adults still have the same need as children, although we have learned, in various ways, to strech our abilities of dealing without them. But at significant points we need physical stuff in order to help us pin down the ideas and concepts that flow in our mind. In my work as a teacher I couldn't explain anything without using practical examples, metaphors, pictures, schemata, and physical models. Our students learn that end-users of technology often have difficulty in using the technology precisely because the designer has not provided the user with obvious physical clues that reveal the underlying 'mental model' of the interaction. That is: if you don't have a clue about how to operate the machine, it is probably because there are no physical clues, that represent the functional possibilities and how to operate them. A good interface provides a natural mapping: the physical form of the interface 'maps' in a natural way to the functional effects each part of the interface will have. When designers want to explain to customers, or end-users, about the design they have in mind, and whether or not this satisfies the customers whishes and demands, it is also obligatory to make the design 'tangible', to physically represent your ideas, either in a scale-model or a good sketch.

All of this shows that human beings, even in adult life, always need a chef's hat in order to understand the restaurant-game...

Interactions (2)

2006-10-09T14:25:00.000-07:00

The mind-body problem is about how mind-stuff relates to body-stuff. Body-stuff is the stuff we all know about (tables, chairs, molecules, planets, billiart-balls), whereas mind-stuff describes the knowing itself: the thoughts, ideas, opinions, beliefs, desires, cravings and intentions that make up the mental realm.

The mind-body problem is essentially the question of how to make a thought out of a kilogram of brain matter. The question itself is already 'strange', imagine how the (ultimate and correct) answer would look like.

Now to turn to my previous discussion on interactions (here); the difference between interaction on the communicative level (between two active agents) and interaction on the physical level (between two physical systems), is not unlike the difference between mind and body.

When we say that two human beings 'interact', what we mean by the word interaction is a transmission of messages in, to take Fodor's term, the language of thought. Human beings do so by physically moving about their bodies (which is detected by the other's visual system) and emitting sound-waves (which is picked up by the nerve cells in our ears), but these physical changes are not crucial to the interaction that is taking place. In order to understand the interaction we have to look at all these events on a meaningful level, and the meaningful level, whenever human interaction is concerned, is on the "informational level". And with information it is meant: messages that are passed from one mental system to another.

When we say that two physical systems interact (for example, one nerve cell's nerve impuls ignites, via a chemical transmission system, another nerve cell that is connected to it), the meaningful level *is* the physical level. Physical levels (there are lot's of them, with their own translation problems between them) have a general description in terms of energy. I'm not that much of a phycisist, but as I understand it one of the laws of thermodynamics states that "things" (physical systems) over time generally get more and more disorderly. When a system has less order, it also contains less energy. The energy leaks out, and the system 'falls apart', so to speak. Any sand-castle will eventually become a flattend pile of sand again. We all turn to dust, someday. The disorderly-ness of a system is measured by its "entropy". Lot's of noise in a system means a high entropy. Lot's of rigidness/structure in a system means low entropy.

Now the funny thing is that although there is a huge theoretical gap between "people talking to one another" and "nerve cells talking to one another", there is a very straightforward way in which the concept of "information" is related to the concept of "entropy". Shannon equated information with uncertainty in this article, and since uncertainty can be - sort of - equated with entropy, information is entropy! Which seems paradoxical but that is because I'm being overly blunt here, see a discussion on this topic.

Now at first I thought this might be interesting because a theoretical closure between what they call thermodynamics and information theory via the concept of entropy might be, in effect, a solution to the mind body problem. Thermodynamics is about stuff, and information theory is about communication, about agents sending messages to one another (this is really the level at which Shannon speaks about it in the article and in his writings there is always a 'sender' and 'receiver' involved, how are not machines, but are considered to be sentient active agents)

However, I quickly found out that *within* entropy theory there is a lot of discussion about what the concept really means. For instance, it is only a matter of wordchoice that Shannon chose entropy to equal certain concept in his theory that he needed a name for. On the above website it is said:

"The story goes that Shannon didn't know what to call his measure so he asked von Neumann, who said `You should call it entropy ... [since] ... no one knows what entropy really is, so in a debate you will always have the advantage'" (see reference)

Still, all of this has to go into my thesis. If you want to know even more about it, read this book, by Ashby, one of the founders of cybernetics.

This last cybernetics link is also very interesting and funny with many anecdotical references. In it, it is described that Heinz von Foerster allegedly has said the following:

"FEEDBACK: An unpoetic inexpressive word that shrieks for replacement. Correct use of the word would refer to eating your own vomit. ".

Interactions

2006-10-07T03:38:00.000-07:00

Someday I want to write a PHD-thesis on the concept of interaction. Just for starters, let's make a quick inventory of types of interaction, based on the number of active agents involved.

1. No active agents involved
In physical systems with nog active agents, like human beings, other animals, or artificial intelligent systems involved, there is physical interaction based on the flux of energy between two physical sub-systems. For instance, when a billiartball hits another billiartball, energy is moved from one ball to the other, and the two balls thereby influence one another's behavior. Or when two chemical substances meet, there might be a chemical reaction, leading to a new stable state, in which some or all of the chemical substances have been combined, or falled apart, etc.. It is interaction of stuff with stuff. Stuff (or energy, which is the same) is being exchanged, moved about, all involved systems change state, i.e. they undergo some behavioral change, and new stabilities arise as a result of that.

2. Two active agents involved.
Interaction between two agents is different from interaction between two 'substances' since the interaction does not take place in the form of energy flow but in the form of *information* flow. Another word for this kind of interaction is "communication". It means that two active agents continously *interpret* the physical changes as they are received on their sensors, coming from the other agent, *as signals*, with an associated *meaning*. The level at which the interaction has meaning is on this informational level alone, the physical level is not interesting just so long as it exists, otherwise the information channel could not be physically realised, which would render communication impossible. But, where in case of the billiartballs the physical structure and energy processes in the system determined the nature of the interaction, in the case of two active agents interacting the nature of the interaction is determined not by the physical structure and energy flux that realises this interaction, but by the *meaning* of the communicative message that is send from one agent to the other.

Of course, one could envision a situation in which two agents bump into each other, as a purely physical accident. But in this case, I argue, the agent's should not be conceived of as active agents, but as passive physical systems only.

3. One active agent involved.
This is the most difficult case because it embodies a blend of the two definitions of interaction above. It is the case where a human agent 'interacts' with his (her) environment. Such kind of interaction is of central concern to cognitive science. The usual question there is: how does the active agent come to understand the (physical) environment or, how does the active agent know how to act appropriately in it? (which basically amounts to the same thing depending on your philosophy). Sensors on the active agent register input, the agent generates behavioral output on its 'actors' (body movement), which in turn leads to new sensory input, and so on. This perception-action cycle, which evolves over time, defines "the interaction". The interaction can be 'functional' with respect to the internal goals of the agent, or, likewise, it can be 'appropriate given the environmental situation. Biologically, one often speaks of 'adaptive' behavior, which relates to underlying evolutionary forces.

Now, what I want to discuss, in a later blog, is how one can mix physical interaction with communication, because I have a feeling that type 3 above contains some theoretical problems...

Movie-talk

2006-10-05T14:18:00.000-07:00

Today I've been watching two movies at the same time, zapping between the one and the other with my remote control. Both movies were interesting. One movie was about a famous club called studio 54. The owner was rich, famous, eccentric, addicted. He hand-picked his guests every night. Of all the hundreds of people shouting in the line, only few would get in. His 'door-policy', was the key to the succes of the club, however unfair such a policy might be, of course. Reflecting on it, I thought about how the talent for one specific thing might build you a complete empire (and bring you tons of money). Even being the famous owner, he still hand-picked the guests. This made the club special, attracting more special people (with money). He created a myth, but I did so by doing just one thing: going out there picking his guests. Of course he could not hand over this task to someone else, because the club was built on *his* talent and this talent was personal, which made the club personal, and the personality (unicity) of the club was the fundamental reasion why people wanted to be there in the first place. The great thing about going to this club is that once you were in, you knew that you had been 'handpicked' by this one guy.

The club, as it evolved, was driven entirly by a talent (based on intuition) and a forceful internal drive for success, and a continuous need for having more and more of it. Talent, intuition, need for succes; these are concepts completely alien to most of cognitive science. But as this movie shows: such human characteristics drive large parts of our society (the club being just an example, perhaps even a metaphor, of the human ways in general).

The other movie was about a guy who lied about almost everything, pretending to be a school-teacher, a crook, a policeman, and at some point even being appointed a doctor in a hospital, without having had the education (learning to talk and act like a doctor from television shows). Reflecting on this, I thought about how a pattern of behavior, on the outside, can be so completely fundamentally not be the same as "the real thing", which is somehow defined "on the inside". He was not a real doctor, but nobody noticed, since he said the right words. He even got somebody the right medicin or cure just by 'going with the behavioral flow' of things. Did he actually cause anything functional to happen in that hospital? Is it possible to get the effects using a system in which there is 'nobody home'? Ultimately, at least in this movie, his scheme exploded, something was bound to go wrong at some point, and it did. But if that wouldn've happened, would we have a right to say that this guy was not a doctor? Or should we accept the idea that a doctor is however does exactly as a doctor should do? What I ask here is of course exactly what Turing has asked of computers in his famous thesis on artificial intelligence: a classic.

The two movies, in all, couldn't have been more different. The one being about basic internal human capacities that are unexplained by objective science, the other being about the objective behavioral view of a human being, knowing that inside there is no body home. Objective science versus human intuition. Although I'm a real scientist, I liked the Studio 54 movie better. Why would that be?

I see!

2006-09-26T13:07:00.000-07:00

Cognitive psychology is often introduced by asking questions like: “How do we perceive the objects in our environment?” This question is then translated in slightly more technical terms as: “How does the brain create a meaningful picture on the basis of the light-rays that stimulate our retina?” The process that makes makes pictures from light-rays is called the process of visual perception. Instead of discussing the various theories that try to explain what the mechanism behind this process might be, I would like to question the question itself.

In my view, the question itself is seriously flawed, based on three errors of thought. I will call these errors the observer-error, the system error and the metaphorical error. (But perhaps better names can be found, or have been found, for each of the errors, elsewhere, since I have not made the effort of doing a literature search on this topic).

The observer error
The first thing to note about asking questions like the one above is that it involves asking after the explanation of something that might not even exist in the first place. We ask: How do we create meaningful pictures in our head? Well, perhaps we don’t create such pictures, so the entire question becomes empty. Let’s think about the question from this point of view and try to analyse how we come to take for granted that ‘making a picture in your head’ is a valid starting point for investigation. I seem to know that I create pictures in my head (Hey, I do see, right?). But how do I know that you create these meaningful pictures? For an intuitive observer, it might seem to be a clear fact that human beings create a picture of the world inside their heads. But there is no observable evidence for this on the outside. Note that all research on visual perception is *based* on first asking the above question. The fact that visual perception is some kind of information processing mechanism that should produce, as output, *a picture* (an image, a pattern, "the thing that we see", or what have you), is taken for granted. It is a starting point. This means that the further empirical evidence coming from research on visual perception cannot be counted as evidence for the existence of the phenomenon-to-be-explained. The evidence merely has something to say on the question of what the mechanism for visual perception is, *assuming* that it is something that will produce a "picture" in our heads.

Now why is it so difficult to believe that we are *not* creating pictures in our head? Well, this has to do with the fact that researchers are always human beings, and we, ourselves, are in the business of visual perception, and we are consciously aware of this ‘fact’. The personal reflection of how visual perception *is experienced* by us confounds with the scientific, observable definition of the phenomenon that needs to be explained. Now, there is nothing wrong with taking a conscious experience as a phenomenon to be explained, but we should be clear about the status of the phenomenon. So what we should ask instead is: How is the conscious experience of a meaningful, visual, picture of the world created? This is a different question. All options are open. We could, for instance, claim that the meaningful picture is just an illusion of our consciousness and that the sensory input on our retina does nothing to create such a picture in reality. We dream our way through life, one could say. We could also claim that the visual input is causing coherent pictures to arise in consciousness, but that this is not very *interesting*, because the function of visual input is mainly to directly constrain behavioral output, and that the coherent pictures and patterns, i.e., the "seeing" that we experience, is only an after-effect, a side-issue.

Whatever the answer, the observer-error states that we should not confound our personal experience of the phenomenon with the scientific (observable) definition of the phenomenon. In the scientific definition, all we have is either a physical process (light falling on the retina), or a conscious living being *reporting* that he "sees things". There is an intuitive, but dangerous habit of automatically assuming some information-processing device (a ‘machine’ that processes the light on our retina to produce the consciously experienced picture in our head) which is posed in between the physical and the conscious phenomenal levels. But this device is not a real phenomenon at all, it is a theory in itself, and it is a theory that could be false.

The system error
This has to with time, and with the assumption on the direction of the flow of events in the phenomenon to be explained. When we ask: how does the light on our retina get to be transformed into a meaningful picture of the world?, it is automatically assumed that *first* there is light on the retina, only after which this light is transformed, step by step, into a meaningful picture. The flow of events, through time, is linear (sequential, procedural), starting with the light out in the world (what is sometimes called one of the 'sensory qualities'), and ending with the conscious visual experience. The metaphorical comparison with a machine that receives input and produces output, going through a series of sequential steps is easily made. This picture of process flow is ubiquitous in college textbook explanations of the structure and processes in the brain. It is how I tell it to my students: Light hits a dog, reflects, hits our retina, excites nerve cells, which excite further nerve cells, which sets in motion a train of nerve impulses that travel from the eye into the optic chiasm, into the lower brain areas, into the cortex, starting in V1, splitting up into the dorsal (on the top of your head) "where system" (where is the dog) and into the ventral (on both sides of your head) "what system" (it's a dog!). Upstream, somehow all visual information gets to be "integrated", producing a coherent picture of a dog. Then you get to shout "hey, it's a dog over there", or something like that. But this picture is a charicature of what is really happening, and I’ll tell you why.

For starters, in the brain, activity runs in more than one direction, since every nerve cell is heavily connected with both feedforward as well as feedback connections. So, no matter which nerve-cell in the ‘stream’ we take as a starting point, activity is then spread both upwards and downwards, at the same time. It is one thing to acknowledge the fact that parallel processing takes place in a system (as everybody does), it is another thing to really think through the causal consequences. Consider that the phenomenon of me observing a dog takes some time in itself. I see something that will turn out to ‘be’ a dog, I continue to look at it, and in the continuous process of seeing this thing I see more and more of the dog. I might at first see that it is a brown living animal, even before I see it is a dog. But seeing that it is a brown living animal suggests that the complete visual stream is already activated (producing the brown living animal experience, right?). But that means that before we actually recognize the dog as a dog, massively parallel processing is going on. So even before we ‘recognize’ a dog being a dog, thousands and thousands of neural impulses are already shooting upwards and downwards all over the system. So what does it then mean to say that we “process the sensory input and thereby recognize the dog”? It means nothing, because the recognition of the dog might just as well be attributed to the feedback activity from the cortex back to the eye, instead of the activity that goes from the eye to the cortex. We might even consider the possibility that the observer just wanted to see the dog, which sort of places the causal origin of the perception of the dog completely inside the observers head. The whole process didn’t start with light falling on the dog at all, it started with my own mental activity!

I do not wish to claim that visual perception is a kind of dreaming that has nothing to do with what happens outside of us. But what I want to conclude here is that visual perception is not, at least not necessarily, a sequential process that starts out there and ends in here. In the brain, multiple parallel processes run upwards, downwards and sideways in a massively connected network. What gross simplification to tell our students that recognition of a dog is a five step procedure that hops from the eye to the temporal lobe and that’s all there is to it?

The metaphorical error
I will try to be quick about this one because my space is running out here. The metaphorical error is related to the errors above. It encompasses the above ones, I guess. It refers to the idea that in asking the question: “How does our brain create a meaningfull picture on the basis of visual input?”, we automatically, implicitly, apply a metaphor to human cognition that might be false. The metaphor is of course the metaphor of the information-processing machine. A real, physical machine that is, in the strong sense, comparable to other mechanical devices such as telephone-communication systems, bicycles, trains, and computers. (So I do not mean a machine in the general sense, in which all processes are by definition ‘machines’). If visual perception is a process that is executed by a machine, then this automatically necessary that this machine have some input, and what better input to take than the light that reflects on the retina? And if this machine is to have some output, than what better candidate might there be than our ‘visual thoughts’, the visual experiences we have when we look, see, observe the world out there? But there is no proof whatsoever that visual perception is such a machine, or explained by such a machine. In fact, it is a non-machine in the first place, since it has physical stuff as input and meaningfull information as output. Such a machine cannot exist. The metaphor is a smoke curtain that obscures the fact that between light on our retina an meaningfull pictures in our head lies a complete mind-body problem (how to get from stuff to ideas), a problem that has not been solved. You cannot just imagine a machine to solve that problem, the problem is fundamental. We probably need a completely different conception of cognition in general in order to solve it.

Ok

So, Jelle, if this is all wrong then, what is your own idea, what is visual perception if it is not the machine you just discarded?

Well, I don’t know, of course. But I do have some hints, some questions to ask. I will pose them here as my final thoughts:

• What comes first, seeing, or acting? When I move my eye, is that considered to be a response, or the active search for a new stimulus?
• What is the goal, the utility, of visual perception? Surely a device so complex as the visual system (the physical stuff that is, eye, brain, and so on) did not evolve in order to create pictures inside our head. Could it be that the adaptive value of visual perception has more to do with sustaining a satisfactory relation with our environment, instead of just seeing the environment?
• Is there really a difference between the inner eye (imagination/dreaming, and so on) and ‘real seeing’? Or is the perceived difference between the two phenomena itself an illusion we create ourselves?
• Who is doing the seeing? We automatically ask: How do people... (follows some cognitive function). But what is this ‘people’ we speak of? Is it my brain that sees? Is it the person Jelle? Is it the Jelle that other people speak about in their native language? Is it the Jelle you just imagined writing this short essay? As long as we haven’t solved the problem of what a human being is (and at which explanatorial level(s) he ‘exists’), how can we answer such a question?
• Should we explain behavior, or the mind? Is the mind really a phenomenon, or is it already in itself a theory (as Churchland would have it) that might be false?

Although I didn’t set out to end up where I do now, all these questions seem to run directly towards the good old mind-body problem again. Time to stop, I guess! If you want to read more about this famous problem in philosophy, you can do it here. See you next time!

Monsieur Tan

2006-09-20T00:35:00.000-07:00

So everybody please meet Monsieur Tan:

(The below text is from a comment I wrote to Sander in my other blog)

"...aphasia is a neuropsychological disorder where the patient specifically loses the ability to speak (there are various forms, of course). The most famous example is "Monsieur Tan", a patient of a doctor called Paul Broca, back in 18-something. This patient could not say anything but the word 'tan' (with full intonation suggesting sentences with different words and meanings, but the output was just that: tan tan tan tan). Broca hypothesized that the disorder was due to the fact that a specific area of the brain was damaged. When opening up mr tan after his death, Broca indeed found that a special area ("Broca's area") was damaged.

Well, this discovery just about started the whole of cognitive neuroscience and neuropsychology.

Broca's original article can be found here"

The systems reply

2006-09-19T12:37:00.000-07:00

Tomorrow a student of mine will present a talk on Searle's famous article Minds Brains and Programs. This student and I will probably be among the few in the classroom interested in those kinds of things. It is very theoretical, it is philosophy, it doesn't get you anywhere, it doesn't help you making money. It is just a discussion about the question of whether computers can really *understand* or whether computers are, at best, merely a *model* of the process of understanding. Fell asleep already did you? Well, be glad you're at home, and not sitting in this classroom, tomorrow, having to listen to it and even having to 'form an opinion' and 'discuss the topic with your neighbour'. But just in case you *are* interested in this question, here're some thoughts...

Rereading Searle I was reminded of the discussions that Iris and I have been having about the word "computation". This discussion I didn't start with Iris, I've had similar discussions with lot's of people. I clearly remember my Big Talk in Nijmegen, where I presented my results of my internship for my fellow students and teachers. I was claiming (hell, why not?) that cognition is not computational and that it is, instead, best explained by the workings of a nonlinear dynamical system. One of the teachers (Lambert Schomaker) put it to me empathically: But Jelle, *everything* is computational. How can you say that cognition is not computational? Even the rotation of the earth around the sun is computational! And furthermore, *Everything* is a dynamical system! How can you say that cognition is a dynamical system? Even the moon in its orbit around the earth is a dynamical system. So what are you claiming here!?

Let's go back a few steps in time, and start up this discussion by looking inside Pim's old copy of "The mind's I". Within this book is embedded the famous article by Searle, called Minds, Brains and Programs. If you already know this article you can skip to the next paragraph.. now. And within this article, Searle argues against the possibility of Strong Artificial Intelligence (AI). Strong AI claims that computers cannot only be used merely as 'models' of cognitive processes, instead, a computer program that performs some function that is comparable to human competence *is*, by matter of fact, a cognitive system. To put it simple: if it talks like a duck, walks like a duck, it's a duck. So if you build a computer that can process stories, and give responses the same way that I process stories and responses to it, then this computer can be said to really *understand* these stories, just like I *understand* them. To be sure: Searle is against this idea. He says (follows the Chinese Room Experiment): Suppose I sit in a closed room, with a large book of rules, and these rules tell me how to create an appropriate response to some linguistic input that is given to me via a small window, then I wouldn't necessarily by performing these rulebased mappings come to *understand* what I was doing. But the people outside the room would quickly come to believe that I really understood the input, since I was giving sensible reply's. (Searle takes the example of Chinese: suppose you have a book that gives you the procedure of writing an appropriate Chinese response to some Chinese input, you could by use of the book fool any native Chinese speaker without in reality actually understanding anything of Chinese).

One of the responses to his article, which he actually encorporated in the article, is called The Systems Reply. If you already know of this reply you can skip to the next paragraph... now. The systems reply says that "understanding is not being ascribed to the mere individual; rather it is being ascribed to this whole system of which he is a part". In other words, *you* didn't understand Chinese, but you, the rulebook and everything else you needed in order to do your input-output mappings, taken together, as a system, *did*.

Somewhere in discussing this reply, he says the following, which I would like to quote here in full:

"If strong AI is to be a branch of psychology, then it must be able to distinguish those systems that are genuinely mental from those that are not. It must be able to distinguish the principles on which the mind works from those on which nonmental systems work; otherwise it will offer us no explanations of what is specifically mental about the mental. And the mental-nonmental distinction cannot be just in the eye of the beholder but it must be intrinsic to the systems; otherwise it would be up to any beholder to treat people as nonmental and, for example, hurricanes as mental if he likes. But quite often in the AI literature the distinction is blurred in ways that would in the long run prove disastrous to the claim that AI is a cognitive inquiry. McCarthy, for example, writes. "Machines as simple as thermostats can be said to have beliefs, and having beliefs seems to be a characteristic of most machines capable of problem solving performance" (McCarthy 1979). Anyone who thinks strong AI has a chance as a theory of the mind ought to ponder the implications of that remark. We are asked to accept it as a discovery of strong AI that the hunk of metal on the wall that we use to regulate the temperature has beliefs in exactly the same sense that we, our spouses, and our children have beliefs, and furthermore that "most" of the other machines in the room—telephone, tape recorder, adding machine, electric fight switch—also have beliefs in this literal sense. It is not the aim of this article to argue against McCarthy's point, so I will simply assert the following without argument. The study of the mind starts with such facts as that humans have beliefs, while thermostats, telephones, and adding machines don't. If you get a theory that denies this point you have produced a counterexample to the theory and the theory is false. One gets the impression that people in AI who write this sort of thing think they can get away with it because they don't really take it seriously, and they don't think anyone else will either. I propose, for a moment at least, to take it seriously. Think hard for one minute about what would be necessary to establish that that hunk of metal on the wall over there had real beliefs, beliefs with direction of fit, propositional content, and conditions of satisfaction; beliefs that had the possibility of being strong beliefs or weak beliefs; nervous, anxious, or secure beliefs; dogmatic, rational, or superstitious beliefs; blind faiths or hesitant cogitations; any kind of beliefs. The thermostat is not a candidate. Neither is stomach, liver, adding machine, or telephone. However, since we are taking the idea seriously, notice that its truth would be fatal to strong AI's claim to be a science of the mind. For now the mind is everywhere. What we wanted to know is what distinguishes the mind from thermostats and livers. And if McCarthy were right, strong AI wouldn't have a hope of telling us that."

Now, although I disagree with Searle on many a thing (in fact, I still think The Systems Reply holds, and Searle is not succeeding in discarding it succesfully, and in the above quote he is being very rhetorical, as always), I think he has a valid point on this 'side issue'. The point, in my words, is that "computation" just has two meanings. One meaning of the word computation refers to the idea that all of reality can be described as a dynamic system in which variables are coupled in one way or another, and if some variable 'maps' onto another variable in some reliable way, we could say that a transformation, aka a computation has 'occurred'. But this kind of 'computation' has nothing to say about cognition or mental processes. It doesn't even by necessity say anything about the reality of these computational processes, because any physicist that is just a little bit of an instrumentalist in his hart will tell you that such dynamic systems and the computations that go on in these systems are *models* of reality, not the real stuff. Nobody would claim that the computer model of an atom, rotating on your PC, *is* actually an atom. The mathematical talk of systems and computations is a *language* in which we communicate scientific ideas. It is not the real thing. (But you could just as well hold that this language is actually referring to something very real that is in one way or another 'just like' that which the language describes). But the realism/instrumentalism distinction is really not at issue here anyway.

What's at issue here is that there is another meaning of the word computation that refers to cognitive processes exclusively. It states that cognitive processes are 'computational' and by that it is meant computational in a very special sense. This theory tells us that the state of the world (if there is such a thing as 'the state' of 'the world') is 'encoded' by the perceptual system as a perceptual 'representation', and that this representation is 'processed' as a set of symbols internal to the cognitive system. To be precise, it is the brain that physically 'instantiates' these symbols. The activity of the brain is representing them, and these symbols interact with one another via a set of 'procedures' (rules, computations), in such a way that the sensory representation of the world is reliably coupled to some 'intelligent' behavioral response.

Saying that all of nature is computational proves nothing about, what I call, (in reference to Searle's strong AI), Strong Computationalism, as described above. As Searle says above " the mental-nonmental distinction cannot be just in the eye of the beholder but it must be intrinsic to the systems; otherwise it would be up to any beholder to treat people as nonmental and, for example, hurricanes as mental if he likes. But quite often in the AI literature the distinction is blurred". In my view, Lambert Schomaker was talking about computation in the 'universal' sense and thereby blurring the distinction between mental and nonmental processes. What I was claiming then, as I still do now, is that mental processes are not 'computational' in the Strong sense.
...

Hey, did you all fall asleep there!?? Wasn't anybody listening to what I was saying??? Ok then, let's take a coffee break!

-=-=-=-

Searle's famous article Minds Brains and Programs

..

Test

2006-09-03T13:36:00.000-07:00

Test