In the age of the algorithm, the human gatekeeper is back
Greg Linden may not be a household name, but he changed the way we interact with culture and transformed retail forever. An engineer at Amazon in the late 1990s, Linden worked on a curious problem: how to recommend books without human intervention. Until then Amazon relied on editors who wrote hundreds of reviews every year. It was a costly and time-consuming process.
Automating recommendations proved trickier than anyone expected. Linden cracked it. He hit on “personalisation”, which paradoxically meant looking not at an individual’s purchasing history, but only at correlations among products. Regardless of what you had bought in the past, Amazon realised that if product A was often bought alongside product B, it meant almost anyone buying product A would also want product B. Amazon tested the results to see which method sold more books. No surprises: the editors were soon looking for new jobs. Humans out; machines in. Some estimates suggest a third of Amazon sales arise from these recommendations. Ever since, the rise of algorithms has been relentless. Now books, articles, music, films, not to mention holidays and clothes, are all suggested by machines.
Last year 1m new books were published in English. Since at least the ancient Greeks, people have believed there is too much to read; now they may be right. That, of course, doesn’t even count all the self-published works, the reams of news or the Borgesian vastness of the internet. By any measure, we have an astounding surplus of reading matter.
The more we have, the more we rely on algorithms and automated recommendation systems. Hence the unstoppable march of algorithmic recommendations, machine learning, artificial intelligence and big data into the cultural sphere.
Yet this isn’t the end of the story. Search, for example, tells us what we want to know, but can’t help if we don’t already know what we want. Far from disappearing, human curation and sensibilities have a new value in the age of algorithms. Yes, the more we have the more we need automation. But we also increasingly want informed and idiosyncratic selections. Humans are back.
This is why, despite having the world’s most powerful book recommendation engine, Amazon bought Goodreads, a website based around personal book reviews. It is why sites such as Canopy.co thrive atop Amazon. Canopy knows many of Amazon’s best items are hidden in the mediocre morass. Canopy’s founders, all designers, trawl through thousands of entries a day to highlight exceptional products.
It’s why publishers keep producing new imprints, to allow for more diverse and personal lists, and why bookshops are once again flourishing, even though we can find any book we want online. We go to browse their tables. In Japan they talk about tsundoku, or the uneasy feeling of having too many books to read. They also have its solution: a bookshop in Tokyo’s Ginza that sells only one book at a time.
This rejuvenated interest in curation isn’t just happening in publishing. On Spotify you can listen to 30m songs, 20% of which have never been streamed once. To help manage this huge catalogue, Spotify spent a reported $100m (£77m) acquiring a company called the Echo Nest, which pioneered a technique known as audio-fingerprinting, which automatically categorises songs. At the same time, however, Spotify has massively expanded its range of playlist makers, musical experts who are rapidly becoming the new DJs.