Polymath David Wolpert sits down for an interview with Abigail Beall of NewScientist (15 April 2023, paywall), and I thought this was interesting, if only for the sloppy thinking:
What are distributed systems and why are they interesting?
Think of the “flash crash” of 2010, an event in which stock markets fell by trillions of dollars in minutes before recovering most of their value in about half an hour. It was caused by a lot of bots that do automated trades. On their own, these bots are based on simple if/then programs, but they somehow interacted collectively to suddenly cause the entire market to nosedive. The market slowly crawled its way back to where it was, so this wasn’t like the much more protracted economic downturn that began in 2007. But to this day, nobody can understand what went on. No new regulations have been put in place to try to prevent a repeat, because nobody knows exactly why it happened. It is described in the scientific literature as having been like some kind of alien ecosystem that we don’t understand.
Now, imagine something like that, but with artificial intelligence systems like AlphaGo that are practising and learning across the whole web. What sort of vastly more complicated versions of the flash crash ensue when the bots are replaced by these kinds of machines? It’s not going to be some human-like intelligence any more, it’s going to be different. It’s hard not to believe that, in some ways, it will be vastly more powerful.
Between CRISPR [our currently best gene-editing technology] and distributed, interacting AIs, I can’t imagine that, by the year 2100, we will still be “the most intelligent creatures on Earth”. Our progeny will be here.
He forgets that the accumulation of wealth, prosperity, or a stable situation, is a driving force for much, even most, of humanity, and crashes of the sort he recalls represent potential and actual losses. After all, a drop in price posted to an exchange is caused by an actual buy & sell; it’s not a hypothetical or potential[1]. Has there been another flash crash of that magnitude since?
Not that I’ve heard of.
Look, an intelligent creature will, in most cases, correct errors and minimize losses in order to stabilize a situation. All those algorithms that, running in concert, caused a crash?
I’ll guarantee they’ve been modified since.
Look, I don’t know how their behavior has been modified. Frankly, most of these are short-term trader algorithms, and, as a long-term investor, I don’t see needing algorithms to implement the long-term strategy. My interest would be only academic, and I haven’t tried to find what are probably proprietary algorithms.
But the lack of disaster since does suggest modification, not to mention the “short-circuits” installed by the exchanges. If Wolpert were brave, he’d suggest the entire species of algorithm went through an improvement phase, much like trilobite predecessors acquiring eyes. However, did they interact to do so, or was each one acting in isolation?
But his remark about … distributed, interacting AIs … reminded me of an observation I had a few months after smartphones began infiltrating the populace: the best distributed, interacting entities are … humans.
We’ve always interacted with each other; smartphones have enabled geography-free interactions with an efficiency on the order of face-to-face interactions, and all that may imply. So AIs can interact. Do we think we’re going to be overwhelmed?
I doubt it. I’m reminded of David Gerrold’s SF novel, Voyage of the Star Wolf, which posits creatures created by humans, that have decided that their superior physical and mental capabilities make them superior to humans. Too late, they discover humans, with their million years of predatory training, are far more deadly than, to use Wolpert’s terminology, progeny, that are only one or two hundred years (? it’s been a far while since I read this book) in existence, and have … savage temperaments.
Now, if you want to be unsettled, the following article in the print edition, The Hidden Extinction (by staff writer Graham Lawton), which covers how the number of microbial species seems to be falling precipitously, is a far better candidate. It suggests our ignorance combined with our avariciousness may lead to disaster. But that’s a topic for another post.
1And that the crash was erased is reflective of the fact that good investing is based on fundamentals, not technicals, using these words from their investing context. The former evaluates the industry, potentials, and competency of a business, while the latter is little more than an ad hoc psychological profile of investor behavior, unconnected to the fundamentals of a business, and while occasionally yielding positive results, is quite vulnerable to failure when the fundamentals change.
