In the late 1970s I lived and worked briefly in the Netherlands. Often, on Sundays, I would travel to Amsterdam, go to the morning concert in the Spiegelzaal of the Concertgebouw, and afterwards walk over to the Rijksmuseum, Holland’s national gallery, and spend a couple of hours there. The museum is a wonderful storehouse of Dutch art and there was always much to explore. But on nearly every visit I found myself being drawn back to one of Rembrandt’s most famous pictures – The Night Watch – which I guess is to the Rijksmuseum what the Mona Lisa is to the Louvre.
Its official title is Militia Company of District II under the Command of Captain Frans Banninck Cocq. It came to be called The Night Watch because by the end of the 18th century it had darkened considerably through the accumulation of layers of dirt and varnish, leading to the belief that the painted scene had occurred after dusk.
It’s a huge painting (379 x 453cm) and it has an overwhelming presence. One is stunned by its sweep and scale. What I hadn’t known, all those years ago though, was that I was only contemplating a part of the original painting. It was completed by Rembrandt in 1642, having been commissioned by Captain Bannick Cocq and 17 members of the kloveniers (civic militia guards) for the banqueting hall of their newly built meeting hall in Amsterdam. But 75 years later the decision was made to move it to the Arquebusiers Guild Hall in Amsterdam’s city hall. City officials wanted the painting to hang between two doors but discovered that it was too big to fit, and so these municipal vandals decided to cut the masterpiece down to size. A 60cm-wide panel was removed from the lefthand side, 22cm from the top, 12cm from the bottom and 7cm from the righthand side. And then, to add insult to injury, the trimmings were lost!
In 2019 the Rijksmuseum embarked on Operation Night Watch – a multi-year, multimillion-euro project to restore the missing edges. Two factors encouraged the curators to believe that this might be feasible. One was the fact that a 17th-century Dutch artist, Gerrit Lundens, had painted a copy of the original – which now hangs in the National Gallery in London. So there was a visual reference copy, even if Lundens was an inferior artist to Rembrandt. The other was the idea that they might be able to train a machine-learning system to analyse (and perhaps imitate) Rembrandt’s style.
The story of how they achieved their goal is fascinating and nicely told in a series of short videos on the project website. Basically, the original painting was photographed in extraordinary detail involving nine different methods – a process that yielded over 51 terabytes of data. Then three neural networks were trained on: faces and bodies; colour; and painting style. In each case, the Lundens painting was treated as the question and the Rembrandt version as the answer. When the process had reached a degree of convergence that was judged adequate (presumably by human experts), the machine renditions of the missing sections were printed on appropriate canvas and finished and varnished so that visitors to the museum will be able to see what the original – reassembled – painting might have looked like.
Operation Night Watch has been an enthralling – and instructive – project on a number of levels. It provides, for example, a compelling demonstration of what a combination of imagination, resources, technical virtuosity and mastery of a powerful technology can achieve. In that sense it explains why so many in the tech industry and academia are enamoured of its power and potential, and why they and ruling elites seem unable to comprehend why there should also be such anxiety about its potential for harm.
In part, that incomprehension is based on what Margaret Heffernan called “wilful blindness” in her marvellous book of the same title. In it, she argues that the biggest threats and dangers we face are the ones we don’t see – not because they’re secret or invisible, but because we’re wilfully blind to them.
In some cases one might excuse this kind of ignorance because the future is genuinely opaque to all of us. But that doesn’t apply to machine learning, the technology for which “AI” is currently such a polite euphemism. One doesn’t have to be a genius to appreciate that this is a technology that has awesome possibilities for human augmentation, but also a powerful tool for oppression, discrimination and worse. And this downside cannot be dismissed as just the product of gloomy or uninformed speculation – because there exists a powerful state that already has an impressive working model of the technology in operation on a continental scale. It’s called China, and anyone who is an unthinking booster for widespread adoption of machine learning simply hasn’t been paying attention to what’s going on there.
So perhaps the significance of Operation Night Watch is that it reminds us of how good machine learning could be for human flourishing – and how important that it should be deployed for that purpose rather than merely for corporate profit.
What I’ve been reading
Covid conundrum
Where Did the Coronavirus Come From? A thoughtful New York Times essay by Zeynep Tufekci, one of the most reliable commentators on the virus.
Truth to power
The Constitution of Knowledge. Or how do we get to the truth about anything? Read Jonathan Rauch’s post on the Persuasion blog.
Close encounters
Preliminary Assessment: Unidentified Aerial Phenomena. (UAP = UFOs) A paper from the US director of national intelligence, no less. As far as I know, he doesn’t have a tinfoil hat. Yet.
