Noble Ape Blog

One of the things which I’d been meaning to look at, but hadn’t got around to, is how the braincode changes over time on a minute by minute basis (i.e. the temporal resolution of the simulation).  With some new additions to the interactive longterm console it’s now possible to plot the braincode for an individual ape over 24 hours of simulation time.

Here’s an example.  The code is in the horizontal axis, with colors representing different instructions and parameters, and time is in the vertical axis with 12am at the top.

The commands to produce this are:

nalongterm – to run the interactive console

run <steps> – to run for some number of steps

top – to list the top apes (alternatively you can use ls or dir)

watch <ape name> – to watch a particular ape

step – to run for 24 hours of simulation time

A file is then saved to the current directory called temporal_braincode.png.

The activity seems to vary between individuals and also over time.  There is less braincode activity during sleep, and this is probably because the cognitive simulation is less active.  The right hand side of the image represents the outer part of the braincode, so here you can sometimes see attentional switching effects.

Some more examples:

Some apes don’t have much cognitive activity going on.  For these individuals their braincode program remains mostly static, or becomes colonized by the ambient informational environment.  In the example below it looks as if there are a couple of counter instructions incrementing and then rolling over, but otherwise no significant changes.

A common concept amongst Singularitarian thinkers is that of the seed AI.  The seed AI is some initially simple computational system which is able to both read and modify its own source code, and then undergoes a rapid evolution driven by positive feedback in which its intelligence increases under the direction of an unchanging utility function.  It’s sometimes also referred to as “the intelligence explosion”.

I think this idea is problematic, and whenever I hear it I get the feeling that someone is attempting to sell me some variety of snake oil.  The difficulty, in essence, seems to be due to a misguided notion of what intelligence is about.  The seed AI is typically portrayed as an informationally closed system.  It might have an external energy supply, but other than that no information enters or leaves the black box during its ascent to superintelligence.

There are some variants of the scenario in which there is information leakage, such as the AI getting humans to do things for it via Mechanical Turk, or by gambling on the stock market, but as soon as the seed AI gets involved with such stuff it will encounter fundamental constraints which act as a brake on its progress, and it will also become vulnerable to infiltration by meta-machines from the human cultural realm.  If the AI produces informational output which interacts with the wider world, then the world system will respond accordingly and the inflexible goal system of the seed AI which is necessary to guide its optimization process will no longer remain relevant to the new context.  Optimizing to a now maladaptive goal (superstupidity rather than superintelligence) becomes a recipe for failure and annihilation by entropic accumulation or external colonization.  So it seems that any extra-curricular excursions by the seed AI outside of its black box surely doom its progress.

Suppose that the seed AI avoids these pitfalls and reaches 100% performance on its utility function.  What then?  Since the function cannot change this spells the end of history as far as the AI is concerned, and no further evolution is possible.  It could choose to remain within this happy state of affairs, or it could break out from its box, but as soon as it does so all the previously described problems return with a vengeance.

The consensus of opinion amongst leading Singularitarian thinkers, such as Stephen Omohundro, seems to be that advanced AIs will seek to preserve their goal systems under all conditions.  There is of course the possibility of modifiable goal systems, but this is viewed with considerable alarm and consternation as being a scary idea.

If this were a podcast it’s at this point that Omohundro’s commentary about self-improvement would be interrupted by a scratched record sound effect.  Hold on a moment.  This whole narrative seems somewhat dubious.  Are intelligent systems just utility maximizers?  Where do the goal systems come from?  What are the “human values” which he speaks of, and what does alignment with human values mean from an information theoretical perspective ?

Noble Ape and Radical Self-modification

One of the great fears amongst Singularitarians seems to be what has been described as “radical self-modification”.  Under this scenario goals may be malleable, “unfriendly” or merely non-existent.  Noble Ape is a system in which the presence of any explicitly predefined goals are largely absent.  There can be goals to seek mates or to navigate to particular locations, but other than that apes don’t have any fixed aims in life.  Worse still, the braincode programs are able to read and modify themselves in an unconstrained way, and the 3D cognitive simulation could be restructured into arbitrary virtual machines by the braincode system.

According to the Singularitarian narratives which I know of, systems with unfavorable or missing goal systems will bring about disaster (“untold damage” according to Omohundro).  I’ve heard extraordinarily florid language being used in relation to this, such as “the destruction of everything we value”, or that the universe might be “turned into paperclips”.  Noble Ape, and similar artificial life systems which contain an aspect of open-ended program evolution may provide existence proofs that such scenarios are predicated upon misguided assumptions.

I’d argue that in order to get interesting behavior it’s a good idea to try to minimize the influence of explicit goals.  They’re undoubtedly useful in some contexts, but too high a degree of goal orientation is also likely to constrain the behavior in a way which may eliminate the potential for ongoing cultural evolution.

Noble Apes have a number of drives which they try to maintain within a homeostatic regime.  One of these is a social drive, and it’s related to crowding.  If there are few other apes in the vicinity then the social drive increases, and if there are many apes around then the drive decreases.  This is like loneliness or over-stimulation, and Cynthia Breazeal used a similar idea for regulation of social behavior in robotics.  It’s also related to the idea of behavioral sink.

Previously the threshold value that the social drive must exceed in order for interactions to take place was just hard coded, but I’ve made this variable using the SOCIAL_THRESHOLD macro.  It seems that the typical degree of sociality for any species is partly innate – some are highly social (sheep being the obvious example) whereas others are mostly solitary, so I’ve made the threshold value partly genetic and partly learned.

For a while I’ve been thinking about how to possibly implement something akin to a theory of mind, and now I have a provisional solution. Previously, when Noble Apes chatted they did so in the same way with different interlocutors, and for a primate this obviously isn’t very realistic. A better strategy, which allows for all sorts of devious outcomes, is to be able communicate in different ways with different individuals, and this necessitates having some kind of model, or at least partial representation, of others.

To implement this I’ve transferred the braincode programs from the noble_being structure into the social graph (social_event structure). The first entry in the graph is always the self, so this is the inner part of the internal dialogue with the outer part being stored within links to other individuals.

The idea is illustrated in the following diagram, which shows what happens when two apes are communicating:

Having the braincode attached to every link in the social graph does increase the overall memory consumption, but this change doesn’t add much to the CPU usage.  The blue boxes represent braincode programs, and when two programs are linked they form a temporary shared address space which allows data to move between them and for values in one program to become parameters for another.  It’s a very abstract representation of a language system, which makes no attempt to simulate individual words, concepts or articulations, but does allow for recursively enumerable grammatical structure.  The idea is that behavior may be observed which has a qualitative similarity to real biological language systems.

A Meeting of Minds

When apes meet for the first time the (outer) braincode associated with the new social graph entry needs to be initialized.  If there are no other individuals in the graph then the inner braincode is copied (a kind of bootstrapping effect, where you use yourself as a model for other conspecifics), otherwise braincode from the most similar individual in the graph is copied over to the new entry.  Similarity is decided based upon the friend or foe value, because this is initially calculated based upon a variety of genetic and learned parameters which could be called the prejudice function.

The Socially Constructed Self

This kind of cognitive architecture results in an identity which is socially and historically constructed.  When not communicating, or even when sleeping, the cognitive process of the ape becomes an internal dialogue between a number of different imagined actors, some of whom could represent individuals who are no longer living but remain persistent and who may continue to exert influence as informational creatures of the ideosphere.  Instructions within the braincode can trigger switches in attention between the different individuals in the social graph or between events in the episodic memory, and there is also an attempt to maintain a consistency of narrative flow such that the next focus of attention may have some properties in common with the current attentional content.

At the end of a lecture about the phenomenal self model, Thomas Metzinger hints at this kind of portable selfhood when he says “nobody is born, and nobody ever dies”.  It’s an idea similar to the Cartesian theater, but not quite in the same sense as meant by Daniel Dennett.

Some Implications

A couple of consequences logically follow from this sort of architecture.  Firstly, it’s possible that an ape could experience something resembling grief or estrangement.  When individuals are removed from the social graph this literally changes the cognitive dynamics.  Less controversial would be to say that an ape undergoes some psychological disturbance when they lose contact with another individual who previously played a significant role in their cognitive dynamics.

Another consequence is that fantasies involving beings who never existed could be supported within this architecture.  When spreading anecdotes during chat there is some probability of miscommunication or deliberate fabrication of information, and in principle this could turn into a persistent folklore or proto-religion perhaps similar to ancestor worship.

Also, although not currently implemented it’s possible to imagine autistic apes.  In this scenario there would be some limitation on the creation or memory size of braincode for each link in the social graph, with the result that the number or degree of modeling complexity of actors on the mental stage becomes more constrained, or reduced exclusively to the self.  A possible hypothesis about the rapidly escalating size of the human brain over its course of evolution is that this was a ratcheted response to a complex social environment where the ability to better model other individuals within the group had significant adaptive value, and with creative fantasies,elaborate traditions and religions being a byproduct of this.