I understand that some physicists are trying to understand reality not as the traditional particles and fields, but as information, and even had some success where more traditional approaches have not yet found solutions. I will not pretend to understand the approach.
But perhaps they’re not quite on the right track, I suggest perhaps a trifle facetiously. Whatever, I found this post on Slate Star Codex by Scott Alexander vastly amusing, as he asks Gwern Branwen to take a text prediction program named GPT-2 into other realms, such as writing poetry, writing music, and now …
Last month, I asked him if he thought GPT-2 could play chess. I wondered if he could train it on a corpus of chess games written in standard notation (where, for example, e2e4 means “move the pawn at square e2 to square e4”). There are literally millions of games written up like this. GPT-2 would learn to predict the next string of text, which would correspond to the next move in the chess game. Then you would prompt it with a chessboard up to a certain point, and it would predict how the chess masters who had produced its training data would continue the game – ie make its next move using the same heuristics they would.
Gwern handed the idea to his collaborator Shawn Presser, who had a working GPT-2 chess engine running within a week …
It’s always fascinating to use the wrong tool to solve a problem and have it work. It suggests a misapprehension of reality, and misapprehensions are where new knowledge hides, along with the occasional Nobel prize. Naturally, this isn’t perfect:
GPT2 Chess update: I wrote some code to calculate the probability of all valid chess moves. It can reach endgame now. https://t.co/QQzhZJmgQ9
It starts to blunder every game at around move 13. We suspect it’s losing track of board state. (It’s trained solely on PGN notation.)
— Shawn Presser (@theshawwn) January 4, 2020
Here’s Alexander’s last word:
What does this imply? I’m not sure (and maybe it will imply more if someone manages to make it actually good). It was already weird to see something with no auditory qualia learn passable poetic meter. It’s even weirder to see something with no concept of space learn to play chess. Is any of this meaningful? How impressed should we be that the same AI can write poems, compose music, and play chess, without having been designed for any of those tasks? I still don’t know.
I didn’t really mean anything with the idea that pattern recognition and generation is at the heart of reality, it was a bit of a humor hook – and, yet, if we presume we’re some sort of artificial creature in an artificial universe, it’s not an impossible thought – we do something that may involve free will, and a pattern matching algorithm monitoring us does … something.
I like my surrealism on the side with a swizzle stick and Ed “Too Tall” Jones as my conversational partner.
