http:^^www.cs.utexas.edu^users^chaput^inter.html

This data set contains WWW-pages collected from computer science departments of various universities

MIME-Version: 1.0
Server: CERN/3.0
Date: Monday, 06-Jan-97 21:15:55 GMT
Content-Type: text/html
Content-Length: 39917
Last-Modified: Wednesday, 14-Feb-96 19:20:39 GMT

<title>Symbol Emergence & Symbol Grounding</title>

<h1 align=center>Symbol Emergence & Symbol Grounding</h1>
<h2 align=center>Meaning and Communication in Man and Machine</h2>
<h3 align=center>December 16, 1995
<br><a href="mailto:[e-mail]">Harold Cliff Chaput</a></h3>

<h4>Introduction</h4>
<p>The artificial intelligence
debate has mainly centered on the representation problem.  On one side is
classical AI, maintaining that intelligence is a matter of symbol processing. 
The other side usually consists of connectionist, claiming that systems that
model the brain (i.e. neural systems) are more likely to approach the
functionality of the mind.</p>

<p>White the debate continues to sputter on, recent events have brought into
question whether symbol manipulation is necessary at all for some intelligent
behavior.  It is becoming clear that some sophisticated actions can be
accomplished with very little high-level computation, sometimes with no symbolic
processing at all.  Yet we also know that certain cognition tasks are performed
symbolically.  Symbol manipulation, and with it representative symbols, seem a
necessary component of intelligence, but not of all parts of intelligence.  It
also does not appear to be sufficient for creating intelligence.</p>

<p>What has been missing from this dialog is a discussion of how symbols and symbol
manipulation come about in the human mind.  How do symbols, or categories, emerge
from our cognitive apparatus?  How do we understand categories?  And what
relation does symbolic thought have with non-symbolic cognition?  Several efforts
in linguistics, cognitive psychology, philosophy and computer science have shed
some light of this area.  They reveal some aspects of the link between
connectionism and representationism.</p>

<p>These issues lie at the heart of communication: the ability to convey meaning
with symbols, the ability to evoke thought without symbols, and the ability to
translate between the two.  Communication is the golden fleece of AI, both
connectionist and representationist: nothing better exposes the complexities and
subtleties of intelligence, yet it remains completely out of reach.  Perhaps a
philosophical look at communication will point the way towards a possible
implementation.</p>

<p>In this paper, I plan to discuss these issues, which arose while I was
implementing a program for non-symbolic communication, RobotMap.  After a brief
overview of representational and connectionist AI, I'll talk about some activity
in robotics that has made people think differently about the problem.  Then,
drawing from different fields, I'll discuss the implications these ideas have on
cognition and communication, wrapping up with a look at RobotMap and what it
reveals about symbol grounding and symbol emergence.</p>

<h4>Natural Language Processing, Representation-Style</h4>
<p>The bulk of existing artificial
intelligence code is representational.  That is, artificial intelligence is
realized by representing a problem as a set of symbols which the computer can
then manipulate to find a correct and optimal solution.  Chess programs sort
through possible moves and look at symbols that represent future states of the
chess board; theorem provers take predicate logic statements (fortunately already
in symbolic form) and churn through them with manipulation rules to find new
theorems in the form of novel symbolic constructs.  The symbols that the computer
manipulates necessarily correspond to some thing in the problem domain. 
Therefore, answers which arise as a set of symbols can then be translated back
into the problem domain, e.g. a chess move or a new theorem.</p>

<p>As such, the essential design of a natural language processor on a computer is to
take a sentence, break it down into its symbols, and convert those symbols into
an internal representation.  Usually, this representation is some form of
predicate logic, which the computer uses to derive meaning from the sentence. 
For example, if we take the sentence:</p>

<blockquote>
	Jack kissed Jill.
</blockquote>

<P>We can take what we know about the syntactic structure of English and produce the
following syntactical analysis:</p>
<pre>	(S (NP "Jack") (VP (V "kissed") (NP "Jill.")))</pre>

<p>Looking up the word "kiss" (of which "kissed" is the past tense) in our lexicon,
we know that kiss takes an agent (the kisser) and a theme (the kissee).  Given
that information, we can build the following predicate logic form:</p>

<pre>	(PAST s1 KISS ACTION (AGENT s1 (NAME j1 PERSON "Jack"))
		(THEME s1 (NAME j2 PERSON "Jill")))</pre>

<p>This straightforward approach of converting sentences into predicate logic works
with a great deal of success, and gives the computer something to analyze and
manipulate.  But this approach quickly ran into some problems.  For example,
consider the following couplet:<p>

<blockquote>
Jane saw the bike through the store window.  She wanted it.
</blockquote>

<p>Suppose now that we want our parser to identify the antecedent of "She".  Well,
we know that the word "She" refers to an animate female object, and that there is
only one of those in the previous sentence ("Jane"), so we assign "Jane" as the
referent of "She".  For our parser to do this, we now must include extra
information in our lexicon, like gender, animacy, etc.  So far so good.  Now,
what about "it"?  Well, "it" is a genderless inanimate object, of which there are
three: "bike", "store", and "window".  Our parser is stuck, and on a problem that
people can answer to with little effort.  We might be tempted to create a hack
for this and say that the subject "it" refers to the subject of the previous
sentence, "bike." But this hack would fail quite quickly:</p>

<blockquote>
Jane saw the bike through the store window.  She pressed her nose up against it.
</blockquote>

<p>In order for our parser to interpret this sentence, we need to tell it about
pressing noses and wanting bikes and looking through store windows, etc.  This
can be done with scripts that describe these relationships [Schank and Abelson
1977], but now our implementation is quite daunting: In order to understand
natural language in an unconstrained setting, we would need to be armed with a
battery of scripts to parse some of the simplest texts.  The number of scripts
needed are tremendous, and the exceptions are quite numerous.  To illustrate,
consider how our parser would make sense out of some of the following examples:</p>

<ul>
<li>Teacher: Billy, can you name a color in the rainbow?<br>
Billy: Blue.<br>
Teacher: Very good.  Janet, can you name a color in the rainbow?<br>
Janet: Blue.<br>
Teacher: That's the color that Billy used. [Implication: choose another color.]</li>

<li>A: Teheran's in Turkey isn't it, teacher?<br>
B: And New York's in France, I suppose.</li>

<li>A: Is the Pope jewish?</li>

<li>A: Doesn't the teacher for this class suck?<br>
B: Hmm... How about them Cubs?</li>

<li>Johnny: Hey Sally let's play Sonic.<br>
Mother: How is your homework coming along, Johnny?</li>
</ul>

<p>This problem has been termed the "common-sense problem," because the solutions to
these problems of ambiguity lie in our knowledge of common sense.  Ironically, it
is these problems which are easy for people to understand that have stopped
representational AI in its tracks.  There is currently a representational
solution to this problem underway, the CYC project under Doug Lenat, that plans
to catalog all of common-sense as a set of logic statements.  One cannot help but
wonder if the solution to natural language understanding has to be this
convoluted and difficult.</p>

<h4>Undermining Symbols</h4>
<p>The philosophical argument against intelligence as symbol
processing has come in two major flavors.  One is that symbol crunching is not
sufficient for describing intelligence.  This position is demonstrated by
Searle's well-worn thought experiment of the Chinese Room [Searle 1980], which
essentially says that a system that translates Chinese to English would know
Chinese no more than a calculator knows math (i.e. not at all).  This claim is
highly controversial, and many rejoinders have been launched against it.  For our
purposes, the most useful is the response that knowing something is a matter of
degree.  A person who has memorized "The Wasteland" certainly knows the poem, but
in a different way and perhaps to a lesser degree than an author who has written
a biography on T. S. Elliot, or a student of 20th century American poetry.  In
this way, we might be more comfortable with the idea that a calculator might know
math... just not very well.</p>

<p>The other argument against representational AI is that some knowledge cannot be
captured by a set of rules.  In particular, Dreyfus and Dreyfus claim that
common-sense knowledge is particularly prone to this, being based on "whole
patterns" of experience [Dreyfus and Dreyfus 1988].  I think this is closer to
the matter, because it concedes that the logical approach might not work all the
time.  The Dreyfuses say that it's a mistake to think "that there must be a
theory for every domain."  He claims that proficiency is achieved by "similarity
recognition" in the domain.  We will see that this view is psychologically
supportable, and provides some insight to the problem.</P

<p>Researchers in computer science have been looking at non-symbolic computation
since 1943, when McCullock and Pitts first considered neurons as logic elements
[McCulloch & Pitts 1943].  In 1957, Rosenblatt developed the Perceptron model
[Rosenblatt 1958], which saw great success for the next ten years. 
Connectionism, a sister to representation, held a lot of promise and excelled at
problems that representation struggled with [Rosenblatt 1960a, 1960b, 1962;
Steinbuch 1961; Widrow 1962; Grossberg 1968] (and, predictably, vice versa).  But
this approach to the problem of artificial intelligence was brought to an abrupt
halt in 1969 with Minsky and Papert's "Perceptrons" [Minsky & Papert 1969]. 
Their paper forecast an inevitable ceiling in connectionism, and convinced a
generation of researchers to abandon the field.  Interest did not pick up again
until the 1980's and is just now regaining acceptance in the field of computer
science.</p>

<p>Lately, connectionist systems have been used to model neurology, learn complex
systems, and even manipulate symbols.  Chalmers trained a neural net that takes
active verbs as inputs as produces passive verbs as outputs [Chalmers 1990],
which many would classify as a symbolic task.</p>

<p>Consequently, connectionism has come under greater scrutiny as an approach to
artificial intelligence.  While it provides some architectural features that
representation AI is lacking