DISCONNECT?

      Kevin Kelly and Steve Talbott (kk@kk.org; stevet@oreilly.com)

This exchange is part of an ongoing dialogue about machines and organisms.
For the previous installment see NetFuture #136:

   http://www.netfuture.org/2002/Sep2602_136.html


                      *   *   *   *   *  *  *  *  *

STEVE TALBOTT:  In the last installment of our dialogue (NF #136) you asked, "What would you need as fully convincing evidence that machines and organisms are truly becoming one?"

You will recall that earlier (in NF #133) I pointed out what seems to me a
crucial distinction between mechanisms and organisms:  the functional idea
of the mechanism is imposed from without (by us) and involves an arrangement of basic parts that are not themselves penetrated and transformed by this idea.  In the organism, by contrast, the idea (or, if you prefer, the archetype, or being, or entelechy) works from within; it is not a matter of fixed parts being arranged, but of each individual part coming into existence (as this particular part with its own particular character) only as an expression of the idea of the whole.

I illustrated this organic wholeness by describing how we read the successive words of a text.  Almost with the first word we begin apprehending the governing idea of the larger passage, which comes into progressive focus as we proceed.  And this idea shines through and transforms every individual word.  Dictionary definitions alone would make a joke of any profound text; each word becomes what it is, with all its qualities and connotations, only by virtue of its participation in the meaning of the whole, only as it is infused by the whole.

Our atomistic habits of thought, of course, run counter to this description.  We can scarcely imagine a whole except as something "built up from" individual parts with their own self-contained character.  But the fact is that we could never write a meaningful text, and could never understand such a text, if the words were not caught up into a preceding whole that transformed them into expressions of itself.

When Craig Holdrege, in his study of the sloth (NF #97), said that every detail of the animal speaks "sloth", he was pointing to the same truth. The fine sculpting of every bone, the character of basic physiological processes, the smallest behavioral gesture -- all these are "shone through" by the coherent and distinctive qualities that we can recognize as belonging to the sloth.

So, Kevin, when you ask what would convince me that machines are becoming
organisms, certainly one prerequisite is that I would have to see that the
foregoing distinction is without basis in reality.  I would have to see
that its own idea is native to the mechanism and governs the mechanism in
the way the idea of the organism governs and shapes the organism from
within, bringing the parts into existence as expressions of itself -- or
else that organisms *fail* to show this sort of relation between part and
whole.

Now, I realize that in NF #133 your initial response to my distinction was
to deny it.  It seemed obvious to you that I was thinking of "old"
technology -- industrial-age machines -- and not things like cellular
automatons, neural nets, artificially intelligent robots, and all sorts of
other technologies that show complexly interacting elements.  I remain
hopeful, however, that your response was more a function of my brief and
inadequate effort to capture the distinction than a real disagreement.

With that hope in mind, let me explain why your counterexamples don't work
for me.  Think first of a computer.  The hardware can be implemented in
radically different materials with radically different designs.  (You're
doubtless aware of all the ways people have imagined constructing a
Universal Turing Machine.)  Then there is the programming, or software,
which defines the functional idea of the computer.  Does this program work
in the computer in the same way the idea of the organism works in the
organism?

Clearly not.  You could remove the software from one computer and install
what is essentially the same software in a wholly different computer.
Conversely, having removed the software from the first machine, you can
load a second program into it.  In the former case, you have the same
functional idea driving two computers that may be unrecognizably different
in materials and design.  In the latter case you have two completely
different functional ideas successively driving the same computer.  This
arbitrary relation between the programmatic idea and its hardware
embodiment is something you will never find in the psychosomatic unity of
an organism.  (Try putting the mind of a horse into the body of a pig!)

The relevance to my larger point is this.  If there is no horse/pig
problem with computers, it's because the software coordinates the pre-
existing elements of the hardware rather than enlivening them and bringing
them into being; and the different programs are therefore free to
coordinate the elements in different ways.  These elements are not
themselves transformed by the program from within, in the manner of words
in a text, or bones, muscle fibers and cells in a developing organism.
Nor is the program continually embodying itself in new, previously non-
existent forms of hardware as it "matures".  (If you think genetic
algorithms contradict this, then we need to talk about them.)

Does this capture the distinction I'm after a *little* better?

One other thing.  I get the feeling that you half expect me, upon
reviewing all the achievements in robotics and AI, to be stunned by the
sheer evidential weight in favor of the increasingly organic and life-like
character of mechanisms.  Rest assured:  I am impressed -- sometimes even
stunned -- by these achievements.  They reinforce my conviction that there
is no ultimate bound upon human creative potentials, and these certainly
include the possibility of housing our ever more sophisticated and subtle
ideas in mechanisms.  I see no end to this process, no limit to how life-
like our devices can become or how fully they will insert themselves into
the warp and woof of our lives.

This, in fact, is why I'm convinced that the decisive trial humanity must
now endure has to do with whether we can hold on to our own fullest
capacities so as to remain masters of our machines.  If we fail the test,
we will find that we can no longer differentiate ourselves from our
creations.  But this will not mean that machines have become organisms.
It will mean, rather, that we have continued to lose our ability to
distinguish the organism's act of creation from its products and therefore
have abdicated the very selfhood that is one with our creative powers.  We
will have succumbed to the downward pull of our machines, becoming like
them.

So what you and I are discussing is not at all a merely academic question!
I am grateful to you for your tenacity in demanding clarity from me in my
explanations.  I trust you will not relent.

                      *   *   *   *   *  *  *  *  *

KEVIN KELLY:  OK, so let's put your criteria to a test.  We'll take a few
organisms (a sparrow, a reindeer lichen, and a diatom) pull them apart,
and ask some experts if they can identify the organism -- if they can see
the whole organism -- in the parts.  And let's do the same with some
technology (a 747 plane, a book, and a watch).  We'll take them apart and
ask some experts if they can identify the technological species -- if they
can see the whole thing -- from the parts.  My guess is that the two teams
would have roughly the same degree of success, on average.

Would you agree that if they did have the same degree of success that this
would (as you seem to suggest) convince you that machines and organisms
are becoming one?

   > You could remove the software from one computer and install what
   > is essentially the same software in a wholly different computer.

Man, are you wrong about this.  Have you ever tried this?  Have you ever
spoken to *anyone* who has tried to port software for one computer onto a
wholly different computer?  They would universally tell you that it was
like "putting the mind of a horse into the body of a pig!"  There is
profound universality in computation (see my December *Wired* article) but
what this *does not mean* is that any particular implementation of it can
be moved to another matrix.  It simply never happens in practice.
Because: machines are just like organisms.

   > If there is no horse/pig problem with computers....

But there *is* a horse/pig problem, and this problems stems from the
commonality of machines and organisms as complex, dynamic systems in
reality.

   > One other thing .... I see no end to this process, no limit to how
   > life-like our devices can become or how fully they will insert
   > themselves into the warp and woof of our lives.

Now I am totally confused.  This is what I have been saying.

So let me see if I have this right.  You say that there is no limit to how
life-like our devices can become.  You admit that we'll add ever more
life-like functionality to our machines, that there is no limit to what
lessons we can extract from biology to import into machines, until
(without limit) we are able to grow and evolve them.  But while these
machine organisms will be used everywhere, and we'll depend on them the
way we depend upon organisms, and while these things look like organisms,
behave like organisms, and are used like organisms, in fact they aren't
organisms at all because they lack an unlocatable trait, a spark, a vital
something that we can't measure, can't pinpoint, and have trouble
perceiving over time because our third eye which can detect this spark of
real life is dimming.  So while we will be surrounded by vast quantities
and varieties of technology that will appear life-like to all who look and
in any way we measure, this lifeness will be an illusion because in fact
these things will lack an inner, unmeasurable quality that we -- ooops --
can no longer see.  That is why when a scientist says, I see no difference
between this man-made being and an organism, the proper response is:  that
is because you have lost Ulysses's vision.  The improper exfoliation of
life-likeness in machines has blinded your ancient sight. And if you can't
see the true inner life of life, than it must be because (aiyeee!) you
have turned into a machine.  True life recognizes true life; fake life
only recognizes fake life.  Blessed are those with true life.

Is this right?

                      *   *   *   *   *  *  *  *  *

ST:  Well, it must at least be right as a statement of how you have read
my words -- which has me very, very disappointed.  It is you, after all,
and not I who say machines "grow" and "evolve" when in fact everyone knows
we manufacture them.  And it is you who speak of an unlocatable vital
essence, when my entire effort has been to describe for you what numerous
people over the past few centuries (who have bothered to think about the
matter) have been able to recognize in organisms, wholes, parts, and
machines.

Please, please, Kevin, hold in your mind both aspects of my reiterated
claim:  (1) we can abstract a certain formal structure from our own
intelligent activity and impress something of this structure upon
mechanical devices; and (2) this impressing of ideas from without is
identifiably different from the living idea that organizes and constitutes
matter from within -- a difference recognizable in the relation between
whole and part.

Every thermostat, every printed page, every complex, electromechanical
loom or harvesting machine, every silicon chip testifies to our wonderful
ability to engrave something of the structure of our intelligence upon the
stuff of the world.  (Do you think all these are alive, more or less?  If
not, why?)  It would be insane for me to say there is some limit to this
process -- to say that at some particular point we will no longer be able
to take a next step.

But saying there is no limit to the structure we can imprint upon physical
materials is not the same as saying these materials must be alive.  I'm
frustrated that you keep trying to get me to infer life from complex
structure without giving me any reason for doing so apart from, "Gee, look
at this amazing spectrum of contraptions out there -- some of them sure
*seem* alive!"  Well, so do mechanical dolls and Aibos to some people.  Is
that supposed to be the convincing point?  Or could it be that we actually
need to think about it a little, even if this strikes you as miserably
"philosophical"?

As far as I can see, the idea of an unlocatable spark serves no role in
this conversation except to enable you to avoid discussing in its own
terms the actual distinction I've been making between organic wholeness
and mechanism.

As for asking a group of experts to pull a sparrow and airplane apart, the
issue was whether there's a different sort of relation between whole and
part in the two cases.  Are you really wanting to decide this by a
democratic vote of experts rather than through your own attempt to grasp
the substance of the matter?  And are you serious in suggesting such a
gruesome test for your experts?  Surely you realize that to pull the bird
apart is to destroy the very thing you're looking for!  "We murder to
dissect".

Your suggestion is the quintessential expression of the historical
development I mentioned earlier, whereby we have learned to ignore the
very aspects of the world that would have helped us to understand the
organism.  No wonder our culture must largely say to those who would point
to the organism, "I look, but I don't see".  The only looking we practice
is a murderous looking.  You can, if you wish, ridicule the attempt to
rise above such practice as a quest for "ancient sight", but the fact is
that whoever has not yet learned to transcend the limitations of his own
culture remains a prisoner of this culture -- a point I thought you agreed
with.

All this reminds me of the twentieth-century behaviorists, who dominated
academia with their denial of mind.  They kept proclaiming, "We don't see
it" while steadfastly refusing the only possible way of looking for it,
which was to attend to their own act of looking.  If the matter had been
decided by a vote of the experts in 1950, the cognitivist revolution
leading to the kind of computational stance you are now assuming would
never have happened.

Actually, Kevin, I suspect you could be one of the new revolutionaries we
need today, because I'm sure you yourself have an instinctive feel for the
truth of the matter.  Having witnessed the 747 being pulled apart, you
would not consider it outlandish if the plane were to be reassembled and
made to fly again.  It's just a matter of putting the parts back into the
right relationship with each other.  But if you watched the sparrow being
reassembled from its parts, you would not expect it to fly.  What's taken
flight is the inner being that enlivened it and made it an organic unity.

Remember the Star Wars robot, C3PO, lying dismembered on a table?  I'm
sure you complained of no deus ex machina when it was remanufactured; but
you ought to have complained if those were human parts on the table and
*they* were successfully "remanufactured".

There's a closely related point where I'm sure you also have sound
instincts.  An orthopedic surgeon manipulating your arm to discover a
"mechanical" defect regards the arm in a manner completely different from
when she is attending to its meaningful gestures.  Likewise, the doctor
examining your eyeball will step back in order to regard *you* when it is
time to report the results of her observation.  Your eye, face, and arm
are now taken as the unified outer expression of a whole -- an expression
of your inner being -- where before they were viewed (perhaps to the
detriment of your health) as the isolated parts of a mere mechanical
exterior.  The two ways of looking couldn't present a starker contrast.
You would in fact rebel if the doctor continued unrelentingly to objectify
you.  You tolerate it only as long as you think there's a legitimate
reason for the more external and mechanical approach, and you recognize a
difference between the two approaches.

I know; you don't need to say it:  "Some people now look into the eyes of
robots the way they look into the eyes of their friends".  Of course they
do.  Already in the 1970s there were those who projected a living
psychiatrist into Joseph Weizenbaum's ELIZA program.  This is what I meant
about losing our ability to distinguish between organisms and machines.
But in the face of disappearing capacities, are we obligated to go with
the flow?  Even if there were only one remaining person on earth who could
see colors, should he deny his color because of the prevailing blindness?
But I'm quite sure that at some level everyone (including you) still
recognizes the difference between a machine and an organism.

As for porting software between computers:  yes, I'm aware of the need for
machine-level code.  How else could the software "coordinate the elements
of the hardware" in the external way I described?  But this scarcely
alters my point:  you can take a massively complex program with its own
distinctive character (say, a connectionist AI program rather than an
expert system or "central command and control" program) and you can port
this program, with its distinctiveness largely intact, to utterly
different pieces of hardware.

Also, you ignored the other half of my example:  not only can you port the
same type of software to many different machines, but you can also drive
the same machine with many different software packages.  C3PO could have
been remanufactured with an entirely new "personality" -- or, for that
matter, with some of the character of a donkey.  So I say again:

   This arbitrary relation between the programmatic idea and its hardware
   embodiment is something you will never find in the psychosomatic unity
   of an organism.  (Try putting the mind of a horse into a pig's body!)

Finally, a look ahead.  We've been dealing very generally with the
relation between organisms and mechanisms.  We might obtain more traction
by specifically considering the human being.  Here is where the living
idea (or being or entelechy, if you prefer) of the organism lights up in a
bright, centered focus of self-consciousness.  In this self-consciousness
we certainly have no obscure trait requiring your "third eye" to perceive.
Rather, we have what is most immediate and undeniable, what is as close to
us as our most intimate selves -- the inescapable starting point for
anything we could possibly build or even hypothesize.

The problem of consciousness is a crucial stumbling block for the AI
project.  This is because intelligence as inner *activity* (as opposed to
the various outward results that always presuppose the activity) is
inseparable from consciousness, and we have no reason to think we can
endow any current or conceivable machine with consciousness.

                      *   *   *   *   *  *  *  *  *

KK:  In the end, Steve, we are just going to have to agree to disagree.  I
feel our conversation is circling back to itself, without covering any new
ground at this point.  Whatever evidence you supply that we can't ever
make living machines (or minds) I reject as shortsighted, and whatever
evidence I supply that this is possible you reject as irrelevant.

At this point, I think we should let the question lie.  It will be proven
one way or the other in time.  Unfortunately for me, I don't expect
artificial consciousness in my lifetime.

So for the moment (my lifetime) I will have to agree with you.  So I'll
state that we can tell the difference between machines and organisms now.
But what this means is that if *by some weird breakthrough*, nerds were
able to make in my lifetime a machine that 90% of humans thought was
conscious, or an artificial being that 90% of humans thought was alive,
then I will be pleasantly surprised, and you ... you would be what?  In
the 10% group who said it was all an illusion, or who said it didn't
really matter, or who suspected a hoax?  I'm not sure. I suspect you would
try to define the label away, since what we call it is a matter of words
and definitions anyway.  (The history of artificial life and mind is a
history of redefining life and mind.)

But I am not saying this to try to convince you, because I have just
agreed that I can't do that, and that for the sake of this argument I
agree with you within my lifetime.  I am only pointing out that you being
right doesn't change much, but if I am right, then it changes almost
everything.  Now, one could say the same thing about discovering an ET
intelligence:  however the fact that it would be momentous does not mean
that it is probable or likely.  But few would say encountering an alien
was impossible (on any timescale), which is what I think I hear you say
about AI and A-life.  (Part of what I am suggesting is that we will
encounter an alien being on this planet -- one that we make ourselves.)  I
mention this asymmetry only to indicate that when there is such a high-
impact it will pay to monitor it closely.

So I think I'd like to end my part in this conversation about the
relationship between machines and life with this suggestion.  I will
continue to rehearse in my mind the possibility that the demarcation
between the made and the born remains forever (not so hard for me because
I don't expect it to vanish completely in my lifetime); at the same time
you might try rehearsing what life (and your life and philosophy) would be
like if the border disappeared forever.

That's not a challenge, only a genuine suggestion for contemplation.  In
the meantime, perhaps another topic will come along that can engage us and
move our understanding forward.

                      *   *   *   *   *  *  *  *  *

ST:  So be it, Kevin, although this saddens me.

I will round out my own contribution to this discussion by answering your
question about what my response would be if ninety percent of my fellows
took a robot to be alive.  The obvious and inescapable answer: it would
depend on my understanding of robots and living things.  To the extent I
had some understanding, opinion polls would be irrelevant.

It's true that "anything might happen" is an appropriate expectation
whenever we lack all insight.  (A dragon might swallow the sun; a pot of
tepid water might spontaneously boil over.)  But the whole point of
science is to gain enough understanding of the essential principles of a
situation, however subtle they may be, so that we are no longer reduced to
saying "anything might happen".

In this regard, I've been puzzled by your preference for a kind of gut-
feeling populism, in which you are fortified by your subculture's common
hope that tomorrow anyone might walk through the door, including a living
robot.  Maybe the hope is justified, or maybe not, but the only way to get
a firmer grip on the situation is to deepen our understanding of living
beings and mechanisms.  To say "let's just keep building these things and
see what happens" does little good if we fail to understand what we have
built.  We merely "discover" what we expected to find all along.

There are, after all, ways to pursue the key issues.  The huge mechanist-
vitalist controversy focused on questions not unlike those you and I have
been discussing -- and, within mainstream science at least, the mechanists
came away confident that they had vanquished the vitalists for good.
(What's needed, I think, is to revisit that debate without the Cartesian
assumptions by which both sides were bound.)

All this may help you see why I'm uncomfortable with your repeated
suggestion that anyone who attempts to discuss the issues in substantive
terms must be engaging in mere empty play with definitions.  He may, of
course, but the charge needs to be demonstrated, not used as a catch-all
means of dismissal.

In any case, Kevin, I do want to say that I've benefited a great deal from
our vigorous interactions, and I thank you for your willingness to
participate.  It's been bracing -- and, for me, humbling at times.  I've
learned, among other things, how easily my most deeply felt words can
prove merely obscure to an extremely intelligent reader.  I've often had
the feeling, "Well, Steve, you sure blew that one.  Back to the drawing
board".

But, on a happier note, I'd like to issue you a standing invitation:  if
you wish to respond to anything I've just now said -- or anything I say in
the future -- the pages of NetFuture will be open to you.

==========================================================================