The Boys and the Bees: Humans' Role in Recursively Self-Improving Machine Intelligence

Farmers today, with all the agri-tech at their disposal, still use bees as a key part of the fertilization cycle for many crops. Why? The bees do their job well, and there just hasn’t been a big win from inventing a way to do it better.

Today, the “reproductive cycle” for software systems always includes human developers, who use software (tools) to develop other software, including tools, and every other kind of software.  (These are the ‘boys’, with apologies to less alliterative female engineers). Today, the human role is definitely closer to that of the farmer. Singularity theorists have posited a future where machine intelligence becomes recursively self-improving – essentially putting the human developer out of a job. It seems clear that the scope, speed and complexity of the software will continue to grow exponentially – both tools and products, including AI software. As such, on any brute force measure such as processing cycles, over the next few decades, the size of the human developer's role would be significantly reduced. In such a scenario, you could compare the humans to the bees – their role, while perhaps relatively smaller, is still necessary nonetheless. Even if that’s not a good analogy for the human role (and I don’t think it is), the point is that it’s still a big step to go from a role that’s “relatively much reduced” to “eliminated all together”.

Consider this cycle for building successive generations specifically of AI software development tools. Clearly, those improving tools (and collaboration) will enable our human developer to deliver ever bigger leaps forward in AI performance and/or ever shorter release cycles. Suppose that there is another AI whose development cycle relies instead on an early version of a pure-software AI developer, with no human involved at all. As an early version, by definition, this would be minimally capable of doing the things to generate a new AI version – but still less productive and capable than the human developer. What this means is that the AI with the human developer will be on a more rapidly-improving exponential curve than the one that’s pure software.

Is there then a path whereby the AI developer becomes more productive than the human developer? There may specific, narrow scenarios that play to software strengths – for example, making relatively structured, deterministic changes where rapid reaction is needed. However, even there, for bigger, less structured and time-sensitive version changes, the human developer (and, come to that, the human-developed AI) would still have the edge. This kind of behavior seems best characterized as simply another incremental evolution where the software development tools become yet more flexible and productive – still not a complete replacement of the human developer.  In other words, there's simply no advantage - even from the AI's "perspective" - in dispensing with the human altogether.

And in fact, this is what we should expect from the lessons of Adam Smith, and indeed, real ecological and economic systems. Increasing complexity tends to drive increased specialization, and division of labor. “Relative competitive advantage” theory says that the notion of, say, “China being better at everything” is misplaced – and so also with AIs. Each sub-system, species or nation tends to increase its focus on its area of relative advantage. So, the human developer's role – whether as farmer or bee – seems likely to remain.

Comments

[Carrieann]

Great work.