Net Think

The Boys and the Bees: Humans' Role in Recursively Self-Improving Machine Intelligence

Farmers today, with all the agri-tech at their disposal, still use bees as a key part of the fertilization cycle for many crops. Why? The bees do their job well, and there just hasn’t been a big win from inventing a way to do it better.

Today, the “reproductive cycle” for software systems always includes human developers, who use software (tools) to develop other software, including tools, and every other kind of software.  (These are the ‘boys’, with apologies to less alliterative female engineers). Today, the human role is definitely closer to that of the farmer. Singularity theorists have posited a future where machine intelligence becomes recursively self-improving – essentially putting the human developer out of a job. It seems clear that the scope, speed and complexity of the software will continue to grow exponentially – both tools and products, including AI software. As such, on any brute force measure such as processing cycles, over the next few decades, the size of the human developer's role would be significantly reduced. In such a scenario, you could compare the humans to the bees – their role, while perhaps relatively smaller, is still necessary nonetheless. Even if that’s not a good analogy for the human role (and I don’t think it is), the point is that it’s still a big step to go from a role that’s “relatively much reduced” to “eliminated all together”.

Consider this cycle for building successive generations specifically of AI software development tools. Clearly, those improving tools (and collaboration) will enable our human developer to deliver ever bigger leaps forward in AI performance and/or ever shorter release cycles. Suppose that there is another AI whose development cycle relies instead on an early version of a pure-software AI developer, with no human involved at all. As an early version, by definition, this would be minimally capable of doing the things to generate a new AI version – but still less productive and capable than the human developer. What this means is that the AI with the human developer will be on a more rapidly-improving exponential curve than the one that’s pure software.

Is there then a path whereby the AI developer becomes more productive than the human developer? There may specific, narrow scenarios that play to software strengths – for example, making relatively structured, deterministic changes where rapid reaction is needed. However, even there, for bigger, less structured and time-sensitive version changes, the human developer (and, come to that, the human-developed AI) would still have the edge. This kind of behavior seems best characterized as simply another incremental evolution where the software development tools become yet more flexible and productive – still not a complete replacement of the human developer.  In other words, there's simply no advantage - even from the AI's "perspective" - in dispensing with the human altogether.

And in fact, this is what we should expect from the lessons of Adam Smith, and indeed, real ecological and economic systems. Increasing complexity tends to drive increased specialization, and division of labor. “Relative competitive advantage” theory says that the notion of, say, “China being better at everything” is misplaced – and so also with AIs. Each sub-system, species or nation tends to increase its focus on its area of relative advantage. So, the human developer's role – whether as farmer or bee – seems likely to remain.

The Metaverse: how it matters... and enables superintelligent

The Metaverse is a term used to refer to 3D virtual worlds. The Accelerating Studies Foundation has a project called the Metaverse Roadmap Project with the goal of analyzing the state of different such worlds today, and their directions and influences for the future. The Metaverse notion seems to me hugely important. However, I have a sense the some big – perhaps the largest – influences on the future Metaverse are missing from our current concept of it. The Metaverse Roadmap has a Wiki site inviting contributions. I recently posted to that Wiki here… but since it’s a Wiki, have copied that post below.

There is a very large sector of current "virtual world" activity which has been largely ignored by MVR because it is not, today, represented in any 3D, or even graphical form. There is, however, I think, a strong case to be made that this activity will come to dominate the Metaverse of 2016.

That sector is the set of all Internet-based representations of real people, businesses and objects - including software and services - for the purpose of accomplishing real communications and transactions between one another. This corresponds, in a sense, to the "Mirror Worlds" notion in MVR. It is, however, much broader and more valuable, because it encompasses the full set of Net-intermediated interactions between them, rather than limiting it to those that already, today, have some 3D visualization component. In particular, it includes all Internet-based e-commerce, B2B and B2C, all P2P communications (voice, chat and email), and all social networking.

This might seem an unhelpful broadening of the MVR concept. However, these technologies have the potential to converge with the Metaverse as currently defined to the extent that a visualization "layer" or "toolset" gives them a Metaverse presence - whether or not that's intended by the owner of those technologies, or the other participants in those interactions. For example, an individual Metaverse user might, for their own reasons, prefer to remain in a Metaverse environment for all their interactions, including those with non-Metaverse users via those "legacy" technologies. In a business setting, Metaverse visualizations might enhance the usability of systems involving complex sets of partners, relationships, products and services. Indeed, 3D visualization tools already exist for certain logistics and supply chain applications. People or businesses that consciously created their Metaverse instantiation or avatar would of course have more control of its behaviors. It's important to point out, though, that in many cases that would have nothing to do with 3D or graphics - the focus first would in most cases likely be on those attributes that mattered in the context of their important relationships, such as the richness of types of interaction that were possible with their online presence.

While this convergence may still be some way off, its arrival, if, as and when it happens, could rapidly dominate the pre-existing Metaverse for two reasons. One, the number of such individually identifiable and addressable entities or objects in the real economy and society is vastly greater than that in "pure 3D" virtual worlds. And two, the intensity of engagement and variety of relationships between those entities and objects will be vastly greater, because it encompasses an increasingly large proportion of what most real people and businesses spend their days and lives doing. The Metaverse economy and society thus becomes not an interesting adjunct to that in the real world, but rather subsumes it, at least for those who choose to experience it that way.

From a systems perspective, there are a few major pieces of infrastructure that remain unbuilt, for the most part. The Metaverse visualization layer is one obvious one.  More interesting though, perhaps, are (i) Human-computer interfaces (e.g. conversational interfaces) that couple software more tightly to their human users; (ii) creating a software-addressable "Internet proxy" for every software instance (and thereby also for their human users), and (iii) integration systems and standards that enable broad and easy interoperability between those Internet proxies. More technically, this broad and easy interoperability requires (a) broad discoverability of different entities (via registries), (b) definitions of the set of possible interactions between them (standards and APIs, official and de facto), and (c) integration and translation software and service systems.

The notion of interoperability could be seen as redefining what language is a Metaverse context. In some cases, that might approximate or support human language (perhaps enhanced with session management or tagging). In other cases, interactions might mostly be occurring between software entities, with only a relatively small element of human-oriented content. Indeed, it might be predicted that the volume of Metaverse conversations is likely to follow that of Internet traffic broadly, where voice (human-oriented) traffic has declined to less than 1%, with data being most of the rest. The bulk of that even, is machine driven, as opposed to human-consumed "data" such as music or video.The Metaverse, as here, more broadly defined, has the potential to be a very-fast accelerating driver of change. Arguably, it will be the fastest - faster than the nano or any other technology sphere - given the combinatorial explosion of potential interactions, and the speed with which such connections could be established.

It could also be argued that this Metaverse is what the Strong AI of the future will really look like.  It has the essential structures of intelligence that we observe in the brain - vastly diverse and flexible connections between different subsystems (neurons and higher level subsystems).  It has the characteristics of massively technology-enhanced scale and speed of processing that we expect from Strong AI.  Strong AI purists might object to the significant role in this Metaverse system of entities that 'are' (or at least represent) human beings.  However, I'd suggest that for the purposes of functional effectiveness, that doesn't matter.  This is a system of loosely coupled subsystems - it shouldn't matter what's inside those systems - i.e. whether they are in fact human or software.  From the Metaverse (or Strong AI) perspective, everything is an addressable object, each with its own set of supported interactions, and service levels or response times associated with those interactions.  In the diverse ecology and economy of this future Metaverse, humans and software entities all find their niche and role.

How Strong AI and Global Brains may evolve

This post follows from an interesting discussion I’ve been having with Eliezer Yudkowsky. He also pointed me to a couple of interesting threads on Strong / Friendly AI, and Global Brains on Steve Jurvetson’s blog, here, and here. I’d been thinking about this for a while, so here’s a rather long post with those thoughts. Essentially, I’m suggesting that the evolution of strong AI needs to be seen, literally, as evolution – with consideration of adaptive behaviors in a co-evolving environment. What’s more, this is potentially a big evolutionary change – but not the first.  Other, previous major evolutionary transitions cast significant light, I think, on how this will happen.

Strong AI, believers would probably agree, represents a significant potential step change in the capability and complexity of the biosphere and its artifacts.  But that description could apply to several previous changes. These include the appearance of life, RNA to DNA, pro- to eukaryotic cells, single to multi-cellular life... and more debatably, significant human changes such as language, agriculture, the limited liability firm. The book “The Major Transitions in Evolution” explores this in detail. These changes all share several significant characteristics. An important observation on evolution in general – none of these changes was designed with the end in mind. Each gradual step conferred incremental advantages. But the result was that the primary basis of selection shifted to more complex entities at a higher level, i.e. aggregates of the entities that existed before. Those new aggregated entities were fitter in the new landscape, and came to predominate (though this didn’t usually mean the complete disappearance of the lower level forms).  The increased complexity of the new entities came from one of several different mechanisms – symbiosis being perhaps the most important in most of these transitions. Initially what we see is increased reproductive success of the symbiotic entities. Over time, two other important changes gradually occur – an increased division of labor, and changes in heredity, i.e. how the system is described for replication.

In summary, what you see is an increasingly tight symbiotic coupling of existing entities which, gradually, but at an accelerating pace, becomes the basis or platform for an explosion of new, more complex forms.

Re Strong AI, there’s been much discussion about what “recursively self-improving machine intelligence” will look like. Well, look around – it already exists. Not of course, in the form that Strong AI purists are thinking of. But the lesson of evolutionary history is that the messy, evolving, symbiotic cycles of self-improvement early on are the place to look for the origins of the tightly-encoded future entities. So what is this recursive machine intelligence cycle I’m talking about? Essentially, it’s the computer software and hardware business. Here are some of the steps in that cycle: 

1. Sophisticated design and development tools running on powerful computers enable (employ!?) humans to produce a new generation of processors and software.
2. Those processors and software replicate by competing for people’s and companies’ attention and dollars. The successful ones are more widely manufactured / installed.
3. Increased “fitness” of these “phenotypes”, i.e. software products and computers with improved feature/function performance and “user-friendliness”, drives reproductive success. Increased demand at this level also drives demand for better, more powerful design tools.
4. New design and development tools are developed (using and extending the existing software design tools). Typically, each successive generation allows greater abstraction, modularity, speed and flexibility.

But there are humans involved, I hear you protest. The machines aren’t self-improving. Well, look at it from the “point-of-view” of the software. As a replicator, it doesn’t “need” to develop certain capabilities when a supply of sufficiently capable and willing humans is readily available. The evolutionary history of increased division of labor would indicate that there’s not a reproductive advantage for the software in developing those design capabilities itself. 

So what does this imply for “real” strong AI? First, I think it implies that the space to be looking in is “prosthetic intelligence” rather than “artificial intelligence” – what you might call “machine symbionts” or “software prostheses”. These are in some senses just fancy terms for consumer and enterprise technology products. However, I think this mindset implies certain things about the nature of those technologies.  One is that they will be intensely, and increasingly individual – and tailored to their particular user (person, and company if applicable). But they’ll also be tightly tied to other, distributed parts of this symbiotic system – notably other people and companies, via their respective software prostheses (see this previous post). We’ll still have commercial software – though perhaps more as components designed for configuration and customization. Think of the human genome. Over half is shared with most other living species, 95%+ with all primates, 99% all humans, and the small remainder defines group, family and individual characteristics.

A further key implication is that the interfaces with such software prostheses will be fundamental – two, in particular, I think. One is the interface with humans; one is the interface with other software (whether networked, or other locally-assembled components). The evolution of these interfaces is what will enable tighter coupling, increased division of labor, and probably changes in heredity. This will lead to the emergence of strong AI that looks rather closer to what’s usually meant by that term. 

First, the interface with humans. Although the human brain is a wonderful and flexible thing, it’s wired in certain ways. Most of the interface change is likely to come from machines communicating more and more the way humans like. Developing a natural language voice – the way we communicate with other humans – is a key step along the way. As technologies for directly interfacing with the brain improve, it will become “the voice(s) in your head” (like schizophrenia, but in a good way). Ultimately, it will be little different from the way “other parts” of the brain talk to one another. (I’m deliberately implying here that distinguishing the machine piece from the rest of the brain will become ever less meaningful). Subjectively, I suspect this will feel like a new level of human consciousness, as much beyond humans today as we are beyond apes. Intriguingly, this may parallel the way human consciousness originally arose. For more on that, see Julian Jaynes’ mind-blowingly counter-intuitive book, (“The Origin of Consciousness in the Breakdown of the Bicameral Mind”).

On a side note, re “friendliness” of AI, it should be obvious that this type of prosthetic intelligence is by definition friendly. “Symbiosis is the ultimate friendliness”, as Carl Carpenter put it in a comment on Steve Jurvetson’s blog. “User-friendliness” is already a key design requirement for technology to be successful. More profoundly, I’d suggest that our intuitive understanding of “friendliness” is much more useful than any algorithmic or architectural constraint. We consider a person “friendly” – or a technology product “user-friendly” – based on repeated, positive individual interactions. Increasingly, with technology too, that means two-way interaction. “Friendliness” means, inter alia, getting to know me. Someone (or something) being friendly does depend partly on design (genes)… but also socialization. So in general, we can expect AIs (other people’s software prostheses) to be no less – or more – friendly than other humans. For each of us, our own software prosthesis or symbiont, will of course be very friendly to us indeed.

Second, the interface with other software prostheses. It’s already clear to everyone that we live in a very networked world. Our software prostheses (and those of our businesses) will interact on our behalf with other people and companies’ prostheses to do all the things we want them to. They will act as managers, secretaries, research assistants, and teachers, to pick a few roles. Increasingly, they will evolve their own languages for this communication (with some human help). Again, the clues can be found in what’s real today. EDI and other transactional communication automates routine interactions between many, mostly larger companies. It’s still pretty much at the “grunts and squeals” stage in terms of richness of meaning. As this example shows though, machine-to-machine (or prosthesis-to-prosthesis) communication will be about a very different set of subjects from those in human-to-human communication. Data already account for the vast majority of Internet traffic. I’m not sure how much of this is machine-to-machine versus, say, music downloads. But eventually, I suspect, this machine-to-machine chatter will dominate. Over time, it will also explode in richness, encompassing and extending the richness of human language.

I further suspect that the notion of the “global brain” continues to emerge through this process. Arguably, it already exists – the sum total of humans communicating, recently much accelerated via the Internet. But the interactions between software prostheses – at machine speed, and planetary scale – promise to accelerate this significantly further. (Steve Jurvetson and others talked in a comment to this post about this notion of acceleration through agent-based systems).  Jeff Hawkins’ ‘memory-prediction framework’, as described in his book “On Intelligence” shows, I think, how this might happen. It makes a good start at describing the specific types of messages that neuronal subsystems exchange in recognizing and recalling patterns at all levels, and (intelligently) making predictions. I think it’s reasonable to suppose that such types of messages would become part of the message set, or language, that’s developed for software prostheses to communicate. He seems to be thinking, again, in terms of pure software implementations. But a system where the individual software components are prosthetic symbionts able to leverage the intelligence of “their” human or company should perform significantly better. Again, here, from the “point-of-view” of the distributed system, each subsystem is a black box – it doesn’t “care” how they’re built, still less if a human is part of the functioning of that subsystem. (Chinese room, anyone?).  The "group minds" this constructs probably exist at multiple scales.  Companies probably already, supply chains may be a new type of group, probably others I've not thought of.  Also, to the extent that software prostheses start making software component selection decisions on your behalf, that can become an accelerator on the clock-speed at which selection pressures can operate.

Does it mean anything here to talk about “prosthesis-to-prosthesis” communication, as opposed to just talking in more traditional terms of, say, B2B or app-to-app (A2A) integration? I think it does. First, at any point in time, most of the value in technology is tied up or embodied in some form of legacy software. Anyone in IT knows this – or come to that, any consumer who’s grappled with migrating applications and data to a new PC. That’s not easily discarded or replaced. But for the most part, they’re not designed for this kind of seamless, networked integration. They have the interfaces they have, and that’s just life (at least until the developer, and user, have compelling reasons to change). Today, you can have integration that’s deep, but point-to-point, and very costly. Or you can have connections that are broad, relatively inexpensive, but without machine-to-machine integration at all. There’s not much in between. 

This is in some ways similar to the challenges the early Internet (well, ARPANet) designers faced. A – arguably the – key architectural innovation, was realizing that none of the mainframe owners wanted the hassle of installing and running extra stuff on their system. The key was to develop a separate, uniform networking box – a platform that could talk a single standards-based language to all the other boxes. The messiness of interfacing natively to the various legacy systems was kept contained to a local “adapter”. (See “When Wizards Stay Up Late” for more of this history). That platform was focused on a relatively narrow problem, just the most basic plumbing layer. Of course, its evolution into the Internet and the Web, is now all history. But all its splendid complexity, in particular via globally-used used Internet applications, formed as additional layers, most of them also relatively simple, on top of the uniform platform created by previous layers.

The point here is that for “intelligent” software prostheses in the future to behave as described here, there are certain architectural implications. In particular, there are architectural implications for creating the unified interface for interacting with humans, on the one hand, and with other software, on the other. For the global, seamless, any-to-any communication between software applications, it implies the need for a unified, global platform, separate from but increasingly integrated with those applications.

The second, machine-to-machine problem is essentially what my company, Traxian, is solving - though my colleagues would barely recognize it from this description. What we’re building today is hardly “intelligent” at all, certainly not in the AI sense. It’s focused on solving a very narrow, immediate set of problems for real, mostly smaller businesses in connecting them to their trading partners. Our customers would find the concepts here even more foreign. But again, in all the historical examples of these evolutionary changes, that’s the way they happened.

What machine intelligence - and collaborative intelligence - will look like

I'm prompted to write this, my first post, after all the debate that's sprung up recently on this topic. Ray Kurzweil and others talk about the 'Singularity', when machine intelligence far surpasses that of humans.  Others focus on 'Intelligence Augmentation' - when billions of humans are hooked up in ways cleverly augmented by technology. I was really disappointed to have only just heard about the 'Accelerating Change' conference on this last month.

A lot of this misses some key questions, I think, on both sides of the debate. First, where will all the future superintelligent computing horsepower come from? What will it initially be built to do, and what does that imply about its behavior?  On the other side, how does 'Intelligence Augmentation' change human-to-human interactions, in particular as more and more machine intelligence is involved, and as humans get more used to using it?

At risk of seeming too grounded in the now, I think the answers overlap. People will spend money to buy technology that meets their goals, personal or business.  Applications in other words. Increasingly they use those apps to access services... and to communicate with other people (who have their own apps).  Increasingly though - and this is the interesting part - more and more communication will be happening between the apps, without the humans even being aware of it.  This is something I think the 'blognoscenti' tend to miss.

Think of diplomatic sherpas preparing the way for a summit meeting of leaders.  Or Sigourney Weaver in Alien, strapped into the robot suit, fighting the Alien queen. In both cases, you've got power and effort that's effecting the will of the person driving it, but magnifying many times over the capabilities and resources that they personally could bring to bear. Over time, these things become second nature to those who use them often, the way a skillful musician's instrument can come to seem an extension of themselves.

I'd suggest that this massively-augmented and connected human brain is more the model for the future. The wetware doesn't need to evolve at all I think, genetically, for us to adapt to these vastly enhanced capabilities, and behave as if we've always had them.  (Which will be the case for our children/grandchildren).

More subtle, perhaps, are the ways in which this world lets us treat thinking capacity anywhere outside our heads as an extension of our own brains - whether it ultimately comes from other human beings, the computers in the middle, or some fuzzy combination of the two.

There's a notion you hear that people in the "post-Singularity world" will be left comprehending no more of what's going on than could a goldfish of this discussion.  But seeing each humans as a "will", massively enhanced in this way, leads me to see that notion as flawed.

In thinking about our inability to comprehend an intelligence vastly greater than our own, a rather different kind of analogy comes to mind.  That is, to think of ourselves individually as ants in a colony, or neurons in a brain.  The whole here, of course, being the collective intelligence of human civilization.  It's hard to even think about parameters that would define such an emergent intelligence. The word 'civilization' itself could be taken to be such a reference... but that's been around a long time.  How to think about the changes that computers and the Internet represent, at that scale?  Hard of course, especially since we're just ants/neurons.

ac2005  social software singularity  web2.0