NovelRank Accuracy: Fact or Myth?

NovelRank has been around for a while. It’s a site that gathers sales ranks from Amazon so you can have a ranking history of your books sold on Amazon at your reach. It also shows you an estimate of what your book sales might be based on those ranks. But how accurate are they? Looking them up online brings up lots of comments from 2011 – 2013 saying NovelRank is accurate for low sales or underestimates high sales which all sounds like a good thing, right?

The first part is easy to confirm. NovelRank uses the Amazon’s API to pull the rankings directly from Amazon’s publicly available data. What you see on NovelRank Sales Rank graphs is exactly what you will find when you look up your book listing on Amazon under Amazon Best Sellers Rank, the overall ranking number that tells your where your book sits between all Paperbacks (or all Kindle books) currently on sale. You can see for yourself simply by looking up your book on Amazon and then comparing it to the information on NovelRank. It accurately displays the same ranking each time.

The second part isn’t so easy. Sales may come out closely estimated for some and not for others. I had the opportunity to research and compare with some Amazon reports and go back to see what NovelRank estimated for sales for the same time period. The results were inconsistent. If you take the time to research online you will find people sitting on both sides claiming accuracy or huge discrepancies. Some claim it underestimates, some overestimates. I found it does both, and the consensus seems to be that it tends to go in the overestimation direction.

These charts were picked at random. They don’t correlate.

So where is the problem? It comes down to Amazon algorithms. Amazon uses algorithms, complex multi-factored math equations, to analyze data to predict sales and trends. They change with regular frequency to determine sales rankings. Amazon doesn’t share what exactly goes into them which makes it so that NovelRank’s estimates of sales are being made on guesses of how privately managed algorithms may be functioning. It’s not firm ground to stand on.

We do know that more recently Amazon has made a shift to using a mix of data from sales (frequency, volume) and customer engagement (data gathered from things they own, like the Amazon website, Kindle accounts, KU subscriptions, Prime user data, Audible accounts, Goodreads, etc.).

Here’s some insight into Amazon’s more recent ranking strategies:

• “What’s really being read: Amazon Charts Top 20 Most Read is the first list to rank books by the average number of daily Kindle readers and daily Audible listeners each week – giving customers the opportunity to see what’s actively being read or listened to every week.
• What’s really being bought or borrowed: Amazon Charts Top 20 Most Sold ranks books according to the number of copies sold and pre-ordered through Amazon.com, Audible.com and Amazon Books stores and books borrowed from Amazon’s subscription programs such as Kindle Unlimited, Audible.com, and Prime Reading.” https://www.businesswire.com/news/home/20170518005322/en/

Amazon algorithms that determine ranking are like a chef’s secret recipe. You can identify some of the ingredients, and guess at others, but you can’t know exactly what all of them are, in what quantities, or what precise method and temperature was used to cook them. Amazon’s secret recipe of algorithms is constantly changing and no one outside of Amazon can make factual statements of all the ingredients in it or what they mean.

Having worked for a top ranking worldwide fashion retailer, I know the secret to retail success is to constantly change with the pulse of consumer trends even as it changes from one day to the next (explanation here for the curious: https://digit.hbs.org/submission/zara-leverages-data-analytics-to-understand-consumer-tastes/). Amazon has certainly picked up on that tidbit and run with it checking that pulse hourly and spitting it out as ever-changing rankings.

Those rankings are there for the customer’s sake presenting to them what the current hourly trend says consumers want and delivering them on a platter of recommendations, not for anything else. As such, using them to focus on guessing number of sales is like using a screwdriver to hammer a nail. You might get the nail in or you might not, but in the end you are using the wrong tool. Your likelihood of success is not as consistent as with a hammer.

The current consensus among publishers and authors of various genres is that NovelRank continues to be close in it’s estimates for small sales, but the higher the volume of sales, the higher the discrepancy in novelrank’s guesses generally resulting in overestimation. My research fell into place with this consensus. Yet, NovelRank presents itself as accurate or harmlessly underestimating figures.

From the NovelRank website:

“SALES ESTIMATES

When Amazon sales rank improves, it usually means at least 1 sale. We estimate sales based on those changes, but we aren’t perfect; for bestsellers, we underestimate.”
https://www.novelrank.com/features

Of course they have to present themselves in the best light to get more users beyond marketers. But NovelRank is at it’s heart just a marketing tool. They state this as a primary use of the data in their FAQ’s also on their website.

“NovelRank uses historical data to determine what constitutes 1, 2, or more sales based upon the change in SalesRank. It is a tool for showing real-time effects of marketing, type of sales (continuous, burst, etc), and not as an absolute sales number resource.”

and

“Book sales estimates are still estimates, and for books selling a low volume ( less than 100 copies a month for instance ) the estimates are most likely accurate within 1%. In the end, it is all based on sales rank changes rather than sales numbers, and NovelRank should not be used to dispute hard sales figures from publishers or Amazon.” https://www.novelrank.com/faq

Notice how they focus on the low sales being “most likely” accurate. They do not make any accuracy statements on any volume greater than 100 copies a month. The silence speaks for itself.

So, per NovelRank’s own words, their data should serve as a marketing tool, not as accurate sales data. I can vouch for it being a great resource to see the effects of marketing strategies we put into place. If it works, there’s an uptick in the sales rank trend. If it doesn’t, there’s no change in the current trend. It also saves me from monitoring hourly and having to create my own ranking charts.

This use of Amazon ranks fits with Amazon’s own use of them to better place product in front of the right customer at the right time and be more effective at sales based on very current consumer trends.

In conclusion, by it’s own admission NovelRank collects and displays a history of publicly available Amazon sales ranks for marketing purposes. It only guesses at private and unavailable sales numbers and will not vouch for accuracy on higher volume sales.

Z