Joel and the Giant Algorithm

WHILE THOSE JOURNALISTS who cover the book business (or pretend to; see this pointed David Gaughran piece) lavish most of their attention on the Amazon-Hachette “war,” a million life-and-death literary skirmishes grind on in the shadows.

I know because I’m one of the skirmishers. There are thousands of us. Hundreds of thousands, actually—a number too big to dwell upon, lest I curl into a fetal position and give up the fight in despair.

We don’t battle one another, however. Not directly, anyway. We assault the esoteric algorithms that decide whether we will get noticed on the websites that sell books.

Of course, most of our effort goes into attacking the Amazon algorithm. First and foremost, this is because Amazon is the website that sells the most books. But it is also because Amazon has the most freewheeling system.

If Barnes & Noble is like an old club in a northeastern city where no one makes it through the entrance without proper credentials (and good luck getting those; you gotta know someone), Amazon is a saloon in the wild west where anyone can bust through the swinging doors demanding to be heard over the din.

For the purpose of argument, let’s think of the gilded club entrance and the swinging saloon doors as “gates.” Much has been made about the role of the gatekeeper in modern publishing. Those who have been rejected by Big Pub editors (which is to say, nearly all of those who write books) are understandably upset. I say “understandably” not because the Big Pub editors are wrong to reject them, but because that’s human nature. Rejection sucks. If you’re like most humans, you deal with it in three stages: first you get depressed, then you get mad, and finally you get even. The author’s version of getting even in this day and age is sometimes to spout off on blogs about how terrible big publishers are, but more often to self-publish and prove those Big Pub gatekeepers wrong.

The problem with this method of revenge is that, more often than not, we prove them right instead. It’s hard to sell books—always has been—even with a big corporation behind you. In the days before ebooks, recycling operations ground many tons of the hard hard evidence into pulp on a daily basis. For all of our technological advances, they probably still do. To publish anything is almost always to be proved wrong.

Publishing is an act of hope in a world of soul-crushing cynicism. Something like whistling past the graveyard, except those tombstones you’re passing are probably facing spine-out.

Today it’s likely more difficult to succeed at this endeavor than ever. People have so many entertainment options, so little free time. Plus, there’s the problem of that damn saloon door that never locks.

And yet, the great democratizer called Amazon also has its gatekeeper: the aforementioned mighty algorithm. While it’s true that any author who plays by Amazon’s loose rules can publish a book in no time, getting that work noticed by anyone beyond your immediate family is another matter entirely. Unless a customer goes looking specifically for something, the algorithm gets to decide what books she sees. Or, more likely, doesn’t see. And you have a much better chance of landing in the latter category than the former.

Amazon’s most powerful insight with regard to publishing has nothing to do with price points or even the promulgation of ebooks. It is having the business wisdom to set up a system where fallible (and expensive) human gatekeepers barely play a role. No longer are we authors prey to the whims of New York editors who maybe got stuck in traffic on the way to work and took out their frustrations that morning by rejecting everything that crossed their desk—including a world-changing saga by the greatest genius alive, who tapped out the manuscript one finger at a time over twelve years at the folding card table in his unheated shack.

Now the gatekeeper is a few thousand lines of code.

From an author’s perspective, however, the problem with code (as anyone who ever got an error message from Microsoft knows) is that you can’t cajole it or argue with it or bribe it with lunch. The programmer who wrote the code already has all the Coke and Fritos he can consume, and he’s under strict instructions to ignore emails. You play by his rules or you don’t play at all.

Thus does every author who wants to sell books eventually clash with the modern gatekeeper, the mighty Amazon algorithm. We assail it with metadata and beg people to post reviews that will add up, in the algorithm’s “mind” to—well, something. We buy BookBub marketing to drive our rankings up in hopes the book will stick near the top for a while. We (some of us) sign up exclusively with Amazon’s KDP program so the book gets lent out to stingy but voracious readers, improving our rankings enough so that higher-paying readers will buy it. We give the first book in a series away and/or manipulate our prices up and down to find the sweet spot of yield. We put out boxed sets with other writers hoping to leverage our respective fan bases.

Those who don’t cotton to this game invariably sink to oblivion. I have watched my own books go from #2 on the entire free list (#1 in many categories) to rankings so low that I can’t share them with you here, as the mere thought of it would necessitate a trip down the hall to throw up.

To make matters worse, lately it seems that the algorithm has a shorter attention span than the most jaded acquiring editor. In the time it took me to write this, one of my books went from a decent ranking in the top ten thousand (top fifty on several paid bestseller lists) to the basement, making me queasy again. Then it floated back up. Maybe it’s a computer cache thing, but I fear it was a hiccup in the algorithm. You can try here yourself, although you won’t see what I saw. It’s been days now, an eternity in the life of an algorithm.

And—who knows? Perhaps just this morning Amazon updated the algorithm to consider a factor no one has thought of yet. If so, you can bet they won’t tell us. That’s understandable because every move they make has a counter move. There are authors who have become rich by gaming the system, while much better writers, out of pride or ignorance, never get an inch of traction.

Your agent won’t return calls? Neither will this algorithm. What worked to move it yesterday won’t budge it today. That which affected it this morning may have no influence this afternoon.

Yet, what can we authors do? We ignore it at our peril. Say we took a break and just let our books circle the drain for a while. Before long, we’re back to girding our loins (or holding our noses) in preparation for attacking the algorithm once again. And if we don’t? The algorithm cares even less about our success than that overworked editor in New York. Which is saying a lot. This thing has no shame. No compunction. No regret.

Let’s not kid ourselves. As my wife never tires of pointing out (usually while I’m crying into my soup), save for a few lucky (and very famous) ducks, the plight of the author has always been thus. Once our challenge was fighting through the hardcover editors and the mass market jobbers and the chain store merchandisers and the book club judges and the book review editors. Now it’s a bunch of ones and zeroes—the coldly calculating algorithm.

The gatekeeper is dead. Long live the gatekeeper!

I don’t know whether, as Thoreau famously said, “The mass of men lead lives of quiet desperation.” But the mass of authors most certainly do.

There are too many books and too few readers. That, friends, is the crux of the issue. Thus we authors skirmish or we die. The Amazon-Hachette war, for most of us, is a laughably irrelevant sideshow. No matter who wins or loses, we have our own code to crack.