ALL BLOG POSTS AND COMMENTS COPYRIGHT (C) 2003-2016 VOX DAY. ALL RIGHTS RESERVED. REPRODUCTION WITHOUT WRITTEN PERMISSION IS EXPRESSLY PROHIBITED.

Saturday, April 11, 2015

Mailvox: Objectivity

Northern Hamlet objects to my appeal to average Amazon ratings as evidence that the 2015 shortlist is objectively superior to recent previous Hugo shortlists:
By this criteria for distinctive works: Hemingway's The Sun Also Rises at 3.8 < Vox's A Throne of Bones 4.2 Also, you're also nearly tied there with Twilight at 4.1 for distinctive storyness.

Online ratings are no more an accurate measure of distinctive works than sales are. It's an extension of the same argument... consider: we could predict 1 million Big Mac sales might result in a large number of people saying they sure do like Big Macs. There's brand loyalty there among other things. While for Lima Beans, people might not report loving them as much. None of this has anything to do with healthiness in the same way that sales and ratings have nothing to do with distinctiveness.

Think of the NYC art world. When they award Jeff Koons or Damien Hirst with some award for their accomplishments in art, do you imagine that the average person would even understand anything about the pieces? You place an unneeded emphasis on reception (sales or ratings, take your pick here). Though art and literature's quality can be determined there if we like, it's hardly the only way (nor the common way these niche communities have developed in the past)

Now, you can go different ways with this... Shakespeare was great because of how many people have learned to appreciate him or Robbe-Grillet is great and we do need judges (gatekeepers if you will) to help refine our understanding of the art and literature experience.
Northern Hamlet's response is neither unfair nor unexpected. It does, however, manage to completely miss the point. His error is obvious: he substitutes "distinctive works" for "objective superiority" without realizing that the former is a subset of the latter. He furthere demonstrates that he still doesn't grasp the purpose for citing the metric when SirHamster points out his mistake:
SirHamster: He provided an objective measure for Hugo recognition, not for story distinctiveness. Whether or not Amazon average ratings provide a measure of story distinctiveness, they provide an objective measure of user-perceived quality, which may have some relation to distinctiveness.

Northern Hamlet: Yes, and superior in ratings alone, not in reception. Because, well, we need it to mean anything the SJWs didn't mean.
No, we don't need it to mean anything at all beyond the fact that it is an objective measure of quality. We have been repeatedly informed, by people who admit that they have not even read the works concerned, that those works are inferior to other, previous works that those same people may or may not have read.

Now, we could appeal to the same subjective standard to what they are appealing, which is to say our own opinions. We can even argue that our opinions are more informed and reliable than theirs; there are more people on this blog who have read John Scalzi's and Charles Stross's and George Martin's work than there are people at Whatever and Not A Blog who have read the work of John C. Wright, Tom Kratman, and Vox Day. It should be obvious that those of us who have read multiple works by each of all six authors can much more fairly compare them than those who have not.

But we don't need to rely upon subjective metrics. We can cite objective metrics, and, lo and behold, whether we turn to Amazon or the more left-leaning Goodreads, we observe the same thing at work: the 2015 shortlist is more highly regarded than the previous shortlists. Marc DuQuesne did the math. Can you tell which list is objectively and quantitatively superior?

A: 4.60 Amazon, 4.16 Goodreads
B: 4.64 Amazon, 4.16 Goodreads
C: 4.46 Amazon, 4.11 Goodreads
D: 3.90 Amazon, 3.91 Goodreads

Let's look at my list of Top 10 SF and Fantasy books of all time. For science fiction, my top ten averages 4.32 on Amazon. For Fantasy, it averages 4.53, giving a net average of 4.43. This is considerably higher than the pre-Puppy 1986-2013 Hugo shortlist average of 4.00. Of course, my Top 10 list is wholly subjective, but review the list before you dismiss it; my more esoteric selections such as China Mieville's Embassytown and Tanith Lee's The Book of the Damned tend to bring the average down. So, I would certainly invite similar comparisons to other all-time top 10 lists.

This metric even picks up the perceived decline in the quality of Hugo nominees about which so many people have complained over the years:

1986 to 1995: 4.13
1996 to 2005: 3.93
2006 to 2013: 3.94

Now, unless Northern Hamlet wishes to entirely discount a metric which clearly shows the objective superiority of The Lord of the Rings (4.7) to The Sword of Shannara (3.7), Starship Troopers (4.4) to Redshirts (3.8), The Golden Age (4.1) to Rainbow's End (3.6), and For Whom the Bell Tolls (4.5) to A Throne of Bones (4.2) in favor of opinions that are rooted in nothing objective and are entirely subjective, I suggest that despite the occasional flaws, average review ratings are a perfectly reasonable measure that any sensible SF/F reader can use as a basic quality heuristic given a sufficient number of reviews.

Labels: ,

94 Comments:

OpenID crazdmadman April 11, 2015 12:04 PM  

You might want to read this, Mary Kowal is buying votes:

http://maryrobinettekowal.com/journal/talk-with-me-about-being-a-fan-of-science-fiction-and-fantasy/

"And to my readers — If you can afford it, I encourage you to buy a membership to WorldCon and become part of fandom. If you cannot afford it… I will buy a supporting membership to WorldCon for ten people who cannot afford it. All I ask is that you join the conversation."

Anonymous Trimegistus April 11, 2015 12:16 PM  

Sounds like a golden opportunity for more Puppy-loving fans to get a free membership!

Blogger Markku April 11, 2015 12:30 PM  

I would rather say that if the Worldcon were informed of this, and they were to ignore it, then it would be a great talking point later.

Blogger Anthony April 11, 2015 12:44 PM  

NH is right that the Amazon rating (and the good reads rating) is not an objective measure of quality. However, it does have the advantage of being an *independent* measure of quality. Since its an aggregate rating, it's not as subject to whims or groupthink.

Also, "distinctive" is not a subset of "objectively superior" unless you want to argue that "Dinosaur" is not "distinctive", when it's distinctively bad.

Blogger Anthony April 11, 2015 12:46 PM  

This comment has been removed by the author.

Anonymous DissidentRight April 11, 2015 12:59 PM  

NH is right that the Amazon rating (and the good reads rating) is not an objective measure of quality.

Is the Amazon rating a measure of quality? Yes, of course.
Is the Amazon rating objective? Yes, it's a number and numbers are objective.

Blogger guest April 11, 2015 1:04 PM  

I majored in a STEM degree, but I still was required to take x number of credit hours in liberal arts studies. So it was in college that I was first introduced to the idea that good art and literature could only be appreciated with those who possessed an elite education about these works.

Pride and Prejudice, written in the 1800s still enchants women of all ages. I've reread the book hundreds of times, and I introduced it to my teeny bopper niece who was just as smitten. Neither of us, have any elite education in literature. We love it because we understand so many of the experiences in that book, and over two hundred years later we can still relate to Elizabeth and Jane. What makes a good book, isn't having to be educated to enjoy it. And although there are books that are dated to a particular period, if a book isn't even enjoyed when it is first published, as a work that speaks to our imaginations and hearts, it is unlikely that a later date is going to make the book more palatable.

Blogger YIH April 11, 2015 1:09 PM  

Online ratings are no more an accurate measure of distinctive works
And yet, with movies one of the first things you see is ''it got an XX% on Rotten Tomatoes''

Blogger bob k. mando April 11, 2015 1:23 PM  

the simplest rebuttal is this:

failure at perfection is in no way an indictment of the metric.

all metrics have flaws. at least we have suggested a metric.

the only metric that the SJWs have is that SP / RP nominated the works, therefore must be bad.

Anonymous Scintan April 11, 2015 1:28 PM  

NH is right that the Amazon rating (and the good reads rating) is not an objective measure of quality.

Anthony, I'll tell you the same thing I told Northern Hamlet, who ignored the obvious as well:

You're failing, for whatever reason, to grasp the difference between "objective" and "flawless".

Blogger doofus April 11, 2015 1:41 PM  

My only comment is that Lord of Light is on the wrong list. It is not fantasy, but science fiction.

Anonymous Roundtine April 11, 2015 1:43 PM  

So it was in college that I was first introduced to the idea that good art and literature could only be appreciated with those who possessed an elite education about these works.

Great works can be enjoyed by everyone, but to truly understand them at a deeper level often requires an education. The bulk of older Western literature cannot be fully understood without having read the Bible. There are historical and literary allusions that a casual reader will miss. You can find a classical music piece beautiful, but there is so much more going on that a beautiful piece of music.

The modern/post-modernists inverted this all on its head and said that you cannot understand it at all unless you are "educated." They make filth, garbage and intentionally esoteric works, as if the common man's rejection of such work, or lack of understanding, is a sign of its greatness. Their aim is to obscure; they traffic in lies instead of truth. It is no coincidence that most of these people have a disdain for faith and revelation.

Blogger bob k. mando April 11, 2015 1:46 PM  

i will note this:
i think it was a tactical error for Vox to suggest a metric at all.

it's the rabbits making the accusation of 'low quality'.

rather than allowing them frame control and attempting to provide a metric for them ( which is only going to result in rabbits such as Northern Hamlet accusing it of 'weaknesses' ), throw it back in their faces.

Vox: "PUPPIES titles are low quality? on WHAT BASIS?"

sjWhore: "on my opinion."

Vox: "well, you're the dumbasses who are destroying SF sales and respectability by publishing and giving awards to trash. so, fuck you."

force THEM to defend THEIR asinine statements and metrics. they will continue to do plenty of stupid shit like the EW article or GRRM revealing his marxist ideological roots.

i certainly wouldn't provide a metric from this side until AFTER the sjWhores first show one.

Blogger bruce April 11, 2015 1:50 PM  

People are voting for four different Hugos- the Social Justice Hugo, the Action Hugo, the Hugo Hugo, and the Crimethink Hugo. People with no appetite for any of the other three have no chance to develop good taste in them.

Social Justice- Maybe Ann Leckie is writing the great Social Justice SF of our time. I have no idea. I doubt VD can tell. No appetite, no chance to develop taste. Making Light readers consider it great Hard SF and great military SF and great space opera. They don't know any better.

Action Hugo- Monster Hunter Nemesis was the best action SF I read last year. Off the ballot. Riding the Red Horse was the best mil-SF collection last year, maybe ever- William S Lind doing nonfiction essays? From the publisher of Martin van Creveld? John Campbell had General Abrams for a subscriber, but he never bagged Abrams as a contributor. So, VD for best editor and publisher. Eric Raymond for most promising writer, a John Campbell Award Winner whom John Campbell would have loved to publish. I think he'd make the third one in fifty years.

Hugo Hugo- Stories in the Hugo Gernsbeck, John Campbell, Poul Anderson, Heinlein, Niven, Zelazny, genre. Niven's The Goliath Stone and Shipstar were the best books for Hugo Hugos last year. Barbara Hambly's Crimson Angel for a potential alternate world where human vivisection was the basis for nineteenth century medicine.
'This is a bloody shirt, suitable for waving,' said Teresa NH about the failure to nominate Patterson's Robert Heinlein, the Man Who Learned Better'. Bloody right.

Crimethink Hugo- VD, then John C Wright, Kratman.

Numerical metrics could work inside each of the four.

I gather the only money at stake in the Hugos comes from libraries that bulk-buy the Hugo winner full price out of habit. No Award could break that habit.

Anonymous Jack Amok April 11, 2015 2:10 PM  

Well this is a nice little bit of two-faced two-stepping. Step one, complain that Sad Puppies aren't objectively better. Step two, reject objective measures as not subjective enough. I'm a bit dissapointed in NH for participating in the choreography.

Blogger Anthony April 11, 2015 2:16 PM  

Scintan - now that I'm not on my phone, I can develop my argument.

The Amazon rating is the average of a whole bunch of people's *subjective* opinions. To some extent, literary quality is inherently subjective, once you get past issues like correct grammar. It is possible to decide that there are things a story should do, and to judge its quality based on its success in doing those things.

Lots of early sci-fi has characters who aren't very interesting in and of themselves, just that they're doing interesting things. It's definitely appropriate to say that newer sci-fi is generally better than early sci-fi on that basis, but whether the characters in any particular book are more interesting or realistic than in some other book is, ultimately a matter of individual taste. Some people prefer that all their characters be fully fleshed out, real people with real traits, while others prefer minor characters (or even major ones) who are caricatures. Is Ebenezer Scrooge less good as a character because he's a caricature? That's a matter for individual judgements. Is the poverty of the characters of early sci-fi a serious defect? Possibly, but how serious? The stories are generally pretty enjoyable because they excel in other areas. If you find well-written characters to be very important, you will rate those books lower than someone who finds that good action or interesting science (or speculation about science) is more important than characters.

Taking an average across a whole lot of subjective opinions is still a subjective opinion, though it's a better measure than any one individual's subjective opinion. The process of aggregation washes out a lot of crankishness, but if lots of people have criteria which aren't really related to quality, or overweight one aspect of quality due to the latest fad, the results may still miss.

However, ratings sites like Amazon do have the property of being *independent* of the Sad Puppies, independent of Tor.com, independent of the publishers, etc. Amazon in particular is also fairly neutral, as there isn't any real discerible bias in people who buy or rate books compared to the population at large, because Amazon's near-monopoly of online book buying means that it's a pretty close approximation of the population at large. Amazon ratings allow Vox to "appeal to authority" using an authority which both the Puppies and the Torlings recognize as neutral, and as relatively reliable.

Anonymous Jack Amok April 11, 2015 2:22 PM  

"PUPPIES titles are low quality? on WHAT BASIS?"

sjWhore: "on my opinion."

...force THEM to defend THEIR asinine statements and metrics...i certainly wouldn't provide a metric from this side until AFTER the sjWhores first show one.


I like Bob's thinking here, but I'm not going to accuse Vox of a tactical mistake (though I assume Ender's dwarves are still slaughtering Vox's elves...). The danger of forcing the SJWs to name their metric first is they may not. Actually, they likely won't, because they wont find an objective metric where they win. They'll just repeat the charge that Sad Puppies is a slate of hackworthy crap by unworthy authors gaming the system out of envy.

By responding with Amazon metrics, Vox does force the debate onto the subject of quality and away from accusations of petty jealousy. Frame control often boils down to letting the other side stupidly introduce a frame bad for their own arguments and then locking them into it.

Anonymous grey enlightenment April 11, 2015 2:38 PM  

How do some authors and books have so many votes? Did 700,000 people really vote on Robert A. Heinlein books? seems like too many. Another annoyance is the name. It should be 'good books' irritating

Blogger bob k. mando April 11, 2015 2:39 PM  

Jack Amok April 11, 2015 2:22 PM
Actually, they likely won't



which is the point.

either they will refuse ( puppies win )

OR

they will proffer something fucking retarded like needing to be some flavor of Cultural Marxist as 'objective' ( puppies win ).

all solutions cohere to the Xanatos gambit AND there is less work involved for the Puppies.

i'm not denying that Vox may think he's going somewhere by suggesting Amazon rankings BUT by doing this he now forces diversion of his own time and resources into defending the point he put forward.

when you think of it in Game terms, the Alinsky rules are actually very deep:
the Other ( and the Marxist are always defining the Right as Other ) must always justify itself to the Establishment.

well, goddamnit, WE ARE the Establishment.

let these sorry, mewling, incompetent fucks justify themselves TO US.

WE ARE the Establishment, it's damn well time we started acting like it.

clearly, in this specific case it's too late. the point has been advanced, now it needs to be ( and can be with logic and facts ) defended.

BUT THIS IS A RHETORICAL BATTLEFIELD.

1 - Maintain Frame
2 - Demand They Justify *To US*
3 - Destroy All Dialectical Arguments They Attempt
4 - Rhetorically Humiliate Them At Every Opportunity, whether in the middle of a dialectic exchange or not

Blogger Thordaddy April 11, 2015 2:43 PM  

Objective = objective Supremacy = Perfection...

All the radicals deny the "objective" both as empirical reality witnessed as the life of Jesus Christ AND EVEN MORE degenerately as a CONCEPT which self-evidently motivates man at the deepest intellectual level.

Anonymous karsten April 11, 2015 2:48 PM  

"The modern/post-modernists inverted this all on its head and said that you cannot understand it at all unless you are "educated." They make filth, garbage and intentionally esoteric works, as if the common man's rejection of such work, or lack of understanding, is a sign of its greatness. Their aim is to obscure. They traffic in lies instead of truth."

And for "educated" read "indoctrinated." Very nicely said, Roundtime, and so true. It's no wonder that Leftist English profs (which these days means practically all English profs) habitually and unironically refer to first-year students as "little fascists," simply because these students have not yet been fully brainwashed into Cultural Marxist and still show signs of independent thought. But by the end of a four-year degree, almost anyone who has remained in a humanities department has been body-snatched into a True Believer.

Blogger DaveofSpades April 11, 2015 2:57 PM  

I still don't know how anyone can rationally argue that Hugos have NOT been crazily biased (intentionally or unintentionally) when confronted with the fact that Jim Freakin Butcher, effective creator of an ENTIRE SUN GENRE, couldn't get nominated without Sad Puppies but Dinosaur Revenge Porn could.

Anonymous Roundtine April 11, 2015 3:07 PM  

By responding with Amazon metrics, Vox does force the debate onto the subject of quality and away from accusations of petty jealousy.

The point of the Hugos is to award the best science fiction! Getting SJWs to discuss the quality of the books is like getting Rage Against the Machine to talk about the music.

Anonymous Aeoli Pera April 11, 2015 3:09 PM  

@bob k. mando,

You are misunderstanding the rhetorical battlefield. The main thing is to convince people who just showed up five minutes ago, which means consistently making a strong, reasonable prima facie case to those people.

Under that constraint, claiming WE ARE THE ESTABLISHMENT and trying to "maintain frame" on that is absurd.

Anonymous Salt April 11, 2015 3:11 PM  

i think it was a tactical error for Vox to suggest a metric at all.

it's the rabbits making the accusation of 'low quality'.


On the other hand, Amazon ranking are an objective standard. Like pointing to a dictionary. It defines. The fence sitters will note that. SJWs will whine as usual.

Blogger Feather Blade April 11, 2015 3:13 PM  

OT: I'm looking for some kind of computer or video game that could help a person develop situational awareness and appropriate target identification. Does anyone know of some game that might be suitable? The player would be an older, non-gamer with a Mac.

Anonymous Jack Amok April 11, 2015 3:17 PM  

1 - Maintain Frame
2 - Demand They Justify *To US*


Aeoli Pera is right, the audience isn't the SJWs, it's the outside spectators drawn to the sudden noise coming from the Hugos. To those folks, the Torlings do look like the establishment, since they've won the recent awards (not to mention GRRM is probably the biggest name outside of SF/F fandom involved in the scuffle). We don't want the SJWs to justify anything to us. What we want is the outsiders to start thinking the SJWs need to justify things to them.

Anonymous DissidentRight April 11, 2015 3:19 PM  

The tactical error was for the SJWs to claim that the SP/RP books aren't objectively superior.

Anonymous Scintan April 11, 2015 3:24 PM  

On the other hand, Amazon ranking are an objective standard. Like pointing to a dictionary. It defines.

Please explain that to people like Northern Hamlet and Anthony. They don't seem to have a grasp on it and, at the moment, I'm too busy to be beating my head against a wall.

Anonymous FP April 11, 2015 3:32 PM  

http://kukuruyo.com/2015/04/11/vivian-and-the-sad-puppie/

Don't throw me in that briar patch!

So worth the $40. A week of joyous laughter so far.

Blogger bob k. mando April 11, 2015 3:45 PM  

Alpha Is Assumed, Sigma Doesn't Care.

Anonymous NorthernHamlet April 11, 2015 3:51 PM  

Vox,

Thank you for responding to my criticism. If I may clear the air for some of the other commentators first: I am in no way against Sad Puppies. In fact, I'm both impressed and intrigued with what as Vox has done in the campaign, and I've had an interest in his fiction for a long time. David's W's comment was stupid, but I'd like to offer whatever I can to strengthen Vox's strategy. Not that he likely needs it, but we'll find out.

His error is obvious: he substitutes "distinctive works" for "objective superiority" without realizing that the former is a subset of the latter.

Incorrect. I knew this was the case. But you miss the point: You are objectively showing their superiority as distinctive stories. You accepted the premise in the original comment that objective superiority would mean distinctive story-ness per David W. What you think is my error is actually yours.

He further demonstrates that he still doesn't grasp the purpose for citing the metric when SirHamster points out his mistake:

In fairness, I was drunk. I didn't much think this statement through and was merely rambling. I'd ask that it be struck from the record. My usual rule is not to comment drunk, but I broke it.


Now, unless Northern Hamlet wishes to entirely discount a metric which clearly shows the objective superiority of [...].

Cherry-picking. Merely because you ignore the black swans doesn't mean that "all swans are white" is true. You admitted your system wasn't perfect already. If it can't filter truth, or here ratings as an objective measure of distinct story-ness, it can't measure the same for Hugo nomination.

Blogger ajw308 April 11, 2015 3:52 PM  

The people who submit rankings toAmazon are probably serial rankers. What this does to the data is being in an element of fairness that helps normaliza the data. Joe might score book X higher than Bob, but when they read Y, a better book, they'll both score it higher. Their Individual scores may not match, but the aggregate ranking will show Y ranked higher than X.

I'd think the same people who ranked the Sad Puppy works have also read other recent books and ranked them appropriately.

Blogger Anthony April 11, 2015 3:58 PM  

Scintan - there's a big difference between "objective" and "useful". The Amazon rankings are useful, because they're independent, because they are a bigger popularity contest than the Hugo voter pool, and because they can be used across multiple years of books. The SJWs can't believably accuse the Puppies of gaming the Amazon ratings. So when they say "these books are crap", we can say "that's not what Amazon customers think", and they can't answer that without making ludicrous claims.

None of that adds up to "objective". Objective means measured against a reproducible standard. Book sales are objective - a book was sold, or not; at its original price, or at a discount. It's hard to measure book sales accurately (and part of that is by design), but it's not a matter of opinion whether a book was sold. But since there's no way to objectively measure literary quality, aggregating people's opinions from a very large population is the next best thing.

Blogger Doctor Awesome April 11, 2015 4:03 PM  

The ratings argument is a good one. When I first read about Sad Puppies I looked up last year's best novel winner on Goodreads - it didn't even break 4 stars, with a 3.98 rating. This went a long way in convincing me that the awards were not being given out on the basis of quality.

Sure there are flaws in rating systems, but I believe the "wisdom of crowds" tends to be fairly accurate in these cases.

Anonymous clk April 11, 2015 4:20 PM  

"Now, unless Northern Hamlet wishes to entirely discount a metric which clearly shows the objective superiority of The Lord of the Rings (4.7) ....... A Throne of Bones (4.2) "

Tell us honestly ... LTR 4.7. and ToB 4.2 .... does that in any way seem objective comparion between the two works .(a .5 difference) .. not even if it were a power log scale.... I have read many of your books and I am sure not even you put them in the same class as Tolkien ... if the amazon ratings are objective, the linearity is so poor that they are useless....NH is right.

Anonymous clk April 11, 2015 4:22 PM  

It doesnt mean you are wrong about the bias in the hugos...

Blogger S1AL April 11, 2015 4:28 PM  

Clk - it's a bell curve, generally, which is even more extreme than a power or log scale. 4.7 is an incredible rating. 4.2 is merely the high end of good.

OpenID cailcorishev April 11, 2015 4:39 PM  

when confronted with the fact that Jim Freakin Butcher, effective creator of an ENTIRE SUN GENRE, couldn't get nominated without Sad Puppies

No kidding. If I'd never visited this blog, never heard of Vox or John Wright or the terms "Pink SF" or "SJW," and had no idea who had won the Hugos for the last twenty years, that fact alone -- the Dresden books haven't gotten a single nomination -- would be enough to tell me things were screwy.

I wouldn't know right away whether they'd become politicized, or just so elitist that they refuse to give awards to books that are are popular because they're loads of fun to read. Turns out it's both!

Anonymous Salt April 11, 2015 4:40 PM  

Please explain that to people like Northern Hamlet and Anthony.

They mistake when small number subjective becomes large number objective; that when averaged, holds an objective position of that which is subjective. One does not have to agree with the objective position, but one is certainly up against a consensus used in a viable manner.

Blogger Thordaddy April 11, 2015 4:44 PM  

Objective means measured against a reproducible standard. -- Anthony

A "reproducible standard" = redundant phenomenon which then nullifies the notion of a singular standard not reproducible and standing as that which defines objectivity.

The Superior Book is the Perfect Book. The Perfect Book is that book which tells the right story at all places. Perfection is the standard that crystallizes all notions of objectivity.

Anonymous NorthernHamlet April 11, 2015 4:48 PM  

S1AL,

Clk - it's a bell curve, generally, which is even more extreme than a power or log scale. 4.7 is an incredible rating. 4.2 is merely the high end of good.

Here's why that can't be the case: online reviews skew toward people who really love something or really hate it. There are other problems with these reviews as well, but it's why you'll find ratings pooled to either side, but not more in the middle.

So, there is no bell curve for the ratings.

Anonymous Salt April 11, 2015 4:52 PM  

It's so easy. An Amazon rating of 4.7 is an objective, known measurement, of the average of subjective entries. When the number averaged is large enough, one has a viable consensus of opinions. Over time that number may change, but when large enough even a few 1 stars will not move the censensus. Once large enough, you therefore know what it is.

Anonymous Aeoli Pera April 11, 2015 5:01 PM  

Reviews do not need to fall on a bell curve for 4.7 to be an extraordinary score. It merely needs to be unusual for a book to receive such consistently high scores, so that it falls in a high centile. 4.7 is probably somewhere in the 99.9th centile, whereas 4.2 is more like 90th centile.

Anonymous Aeoli Pera April 11, 2015 5:03 PM  

(Given a large number of ratings, that is. There is probably a large number of books rated 4.7 and above with only a handful of ratings.)

Anonymous NorthernHamlet April 11, 2015 5:14 PM  

I'll just leave this here:

A Statistical Analysis of 1.2 Million Amazon Reviews
http://minimaxir.com/2014/06/reviewing-reviews/

Again, the perfect rating of 5 is most popular for products.

queue up someone saying BUT THOSE ARE FOR ELECTRONICS NOT SCI FI BOOKS

Anonymous Aeoli Pera April 11, 2015 5:21 PM  

I'm getting less sure these days how clear I need to be, so I'll note that I'm responding primarily to this sentiment by clk:

"Tell us honestly ... LTR 4.7. and ToB 4.2 .... does that in any way seem objective comparion between the two works .(a .5 difference) .. not even if it were a power log scale...."

That has nothing to do with objectivity per se, but rather the difference between ordinal and interval scales of measurement: http://en.wikipedia.org/wiki/Level_of_measurement

Amazon ratings are an ordinal scale:

"The ordinal type allows for rank order (1st, 2nd, 3rd, etc.) by which data can be sorted, but still does not allow for relative degree of difference between them. Examples include, on one hand, dichotomous data with dichotomous (or dichotomized) values such as 'sick' vs. 'healthy' when measuring health, 'guilty' vs. 'innocent' when making judgments in courts, 'wrong/false' vs. 'right/true' when measuring truth value, and, on the other hand, non-dichotomous data consisting of a spectrum of values, such as 'completely agree', 'mostly agree', 'mostly disagree', 'completely disagree' when measuring opinion."

Incidentally...

"The median, i.e. middle-ranked, item is allowed as the measure of central tendency; however, the mean (or average) as the measure of central tendency is not allowed. The mode is allowed."

...but I have a feeling the rabbits don't carry the right caliber for this ammunition.

Anonymous Salt April 11, 2015 5:25 PM  

Again, the perfect rating of 5 is most popular for products.

I rated a product five stars. It was a bluetooth unit for the car. Damn, but did it work great... till it didn't work at all. A few months after purchase left another review; one star.

Anonymous VD April 11, 2015 5:25 PM  

i think it was a tactical error for Vox to suggest a metric at all.

The day I need your help with either strategy or tactics, they can pull the plug. Your reasoning is based on false assumptions. We will not win if they don't answer. They won't answer and no one will care. You don't even understand what the battleground is.

But the fact that we have an actual metric pointing to our superiority and they have nothing pointing to our inferiority is something that we can meaningfully point out to every neutral party.

Anonymous Aeoli Pera April 11, 2015 5:28 PM  

Thanks for the link NorthernHamlet. Fun stuff.

Anonymous VD April 11, 2015 5:30 PM  

Tell us honestly ... LTR 4.7. and ToB 4.2 .... does that in any way seem objective comparion between the two works .(a .5 difference) .. not even if it were a power log scale.... I have read many of your books and I am sure not even you put them in the same class as Tolkien.

Yes, certainly. Half a star is a huge difference. Consider that we've all noticed the decline in Hugo nominees over the years and that difference is less than half that, at 0.2. And as wonderful as Tolkien is, he's not as strong on Style as he is on Concepts, Characters, and Story. I'd give LOTR a 10 myself, but I'd understand those who went as low as an 8.

Let's face it, slogging through Mordor is a bit rough.

You are objectively showing their superiority as distinctive stories. You accepted the premise in the original comment that objective superiority would mean distinctive story-ness per David W. What you think is my error is actually yours.

Seriously? You're going with that? "I was drunk" was more convincing.

Anonymous Roundtine April 11, 2015 5:31 PM  

There's a reliable bias in ratings. For general audience fare, I won't pay full price for a movie below 70% on Rotten Tomatoes. 70% to 80%, I have to be interested.

Anonymous Scintan April 11, 2015 5:39 PM  

Scintan - there's a big difference between "objective" and "useful".

Yes

Objective means measured against a reproducible standard.

No or, rather, that is not the only meaning of objective. That seems to be where both Northern Hamlet and yourself are getting hung up.

Perhaps a glance at some synonyms (impartial, fair, impersonal, disinterested) is in order.

Anonymous NorthernHamlet April 11, 2015 5:42 PM  

VD

Seriously? You're going with that? "I was drunk" was more convincing.

Damn it.

Here's how I see it... you either accepted that premise or you didn't ... if you didn't, you didn't address the point, it's something like:

David W: "Your restaurant might sell a lot but it sucks because it's not haute cuisine, it's just mac and cheese. Mac and cheese isn't distinct by our cuisine judges and gatekeepers."
VD: "We didn't just sell a lot of mac and cheese, EVERYONE ON THE BLOCK LOVES OUR MAC AND CHEESE"
David W: "Yeah, great. It's still not distinct."

I'd say fine, but as I've indicated above, ratings aren't effective, because they're heavily skewed.

Furthermore, David W doesn't have nothing to show you're inferior...he has gatekeepers... which you're technically a gatekeeper too. and a fairly successful one at that.

Anonymous BigGaySteve April 11, 2015 5:43 PM  

While the metric might not be perfectly objective at least its better than global warming data. Remember when you had to listen to only jews for movie reviews, that's why Woody Allen was a movie star.

"… I will buy a supporting membership to WorldCon for ten people who cannot afford it. All I ask is that you join the conversation."
Lets sign up and get those 10 free memberships. That way her readers that cant afford 4 packs of cigs cant dilute the vote.

"OT: I'm looking for some kind of computer or video game that could help a person develop situational awareness and appropriate target identification. Does anyone know of some game that might be suitable?"

Just show him you tube videos of Bait Car

Anonymous Jack Amok April 11, 2015 5:54 PM  

Again, the perfect rating of 5 is most popular for products.

Right, which is exactly why a 0.5 difference is very significant. When things cluster like that, small differences are magnified. Saying they aren't is like saying a 6 foot 1 guy and a 6 foot 8 guy are practically the same height, since there's only 7 inches difference. 6'1 is on the tall side of average. 6'8 is someone even I would think of as tall.

Anonymous Jack Amok April 11, 2015 6:11 PM  

NH, you're still missing the point. Dave W complained the puppies had to be judged on "something other than sales". Vox provided something other than sales. If you don't like Vox's metric, you have to propose an alternative.

But, you can't. Because there isn't one that's anything beyond the subjective "stuff us lefties like" and. well, then Vox simply has to point back to the Amazon ratings and say "guess most SF/F readers don't share your tastes." Which would be kinda devastating to the award, eh?

Because the thing you're forgetting is Vox didn't just say "EVERYONE ON THE BLOCK LOVES OUR MAC N CHEESE." He also said they like it a lot better than the play doh sandwiches the SJWs have been pinning blue ribbons on.

And honestly, go back and re-read David W's post. They’re intended to recognize works that are distinctive, not derivative...

Do you really think the folks on the side of Redshirts ought to badmouth Puppies on the basis of being derivative? Is that an avenue you really want to go down?

Anonymous NorthernHamlet April 11, 2015 6:28 PM  

Jack, did you even look at the link for Amazon ratings?

And I certainly can purpose an alternative.

Anonymous Aeoli Pera April 11, 2015 6:45 PM  

@Featherblade,

Any rail shooter will probably work, if I understand your meaning.

Blogger bob k. mando April 11, 2015 6:59 PM  

NorthernHamlet April 11, 2015 6:28 PM
And I certainly can purpose an alternative.



neither your drunkenness nor your ability to propose an alternative was at issue.

the VALIDITY of your alternate metric would be.

which, given that you haven't sacked up and provided one, i'm going to have to presume is pretty piss poor sorry. since, you know, somebody already has suggested one.




VD April 11, 2015 5:25 PM
The day I need your help with either strategy or tactics, they can pull the plug. ... We will not win if they don't answer. They won't answer and no one will care.



uh huh. that's why the Hippies 'lost' every time they tried to occupy the college Dean's office.

because they were concerned with having to justify their stupidity to TPTB.

oh, wait. that's not what happened at all.

Anonymous taqiyyologist April 11, 2015 7:01 PM  

Roundtine:

The point of the Hugos is to award the best science fiction! Getting SJWs to discuss the quality of the books is like getting Rage Against the Machine to talk about the music.

Taq:

+infinity.

Blogger Noah B April 11, 2015 7:08 PM  

"And I certainly can purpose an alternative."

Why didn't you just do that in the first place?

Blogger rcocean April 11, 2015 7:29 PM  

Amazon is an objective standard. "Maintaining the Frame" is a useless strategy when dealing with SJW's.

Anonymous NorthernHamlet April 11, 2015 7:33 PM  

Alright guys, this is getting tedious. The comments are all right there.

Blogger Noah B April 11, 2015 7:50 PM  

It is tedious. So how about a simple answer to a simple question: what alternative metric do you propose that you believe is better than Amazon ratings?

Anonymous Jack Amok April 11, 2015 8:11 PM  

And I certainly can purpose an alternative.

And yet you don't.

I'm not holding my breath waiting either, because if you don't already know, I suspect you'll realize soon enough that any alternative you propose that might give Ancillary Justice (4.2 stars) or, Redshirts (3.8) or Among Others (3.7) the nod over Monster Hunters (4.7) or City Beyond Time (4.6), or The Chaplain's War (4.5) will hand Vox the opening to point out your alternative is out of step with what people actually want to read. You'll be left arguing that meeting someone's subjective criteria for being "distinctive" is more important than meeting reader's criteria for being "good." Best to throw in the towel now and get it over with so we can help you up out of the hole before you dig it too deep for us to reach.

Anonymous taqiyyologist April 11, 2015 8:11 PM  

How many Grammy Believers have heard the whole catalogues, never mind even heard OF of AWOLNATION, Jack Johnson, Ray LaMontagne, Emile Sande, Gnarles Barkley, or Matt Kearney? Six artists who deserve dozens of awards.

"Nobody." is the answer.

Sad/Rabid. That's the only way. Kanye, get your feet off the Situation Room desk, you buffoon. The adults are back.

Anonymous taqiyyologist April 11, 2015 8:19 PM  

I know, Cee-Lo is probably not on the same page as Alfonzo Rachel, but still. Point.

Many of those might be liberals, progressives, or rabbits. I don't know. I doubt it, because they haven't been recognized as excellent by the Establishment award-givers. I like their lyrics. I would both nominate and vote for all their Lifetime Achievement Awards if that actually meant something other than being celebrated by SJWs for too freakin' many decades.

Grammys. Oscars. That's the next battle. This is like prep.

Anonymous taqiyyologist April 11, 2015 8:35 PM  

*(And the music on all is impeccable for the stage they're at. *Gotye*. Good grief, one E.P. and an album. Both masterpieces. Both unknown, for some "stitched-up" reasons.)

Anonymous NorthernHamlet April 11, 2015 8:36 PM  

Jack Amok,

And yet you don't.

I already stated mine in the previous thread and I believe hinted in this one as well.

your alternative is out of step with what people actually want to read.

This is very narrow thinking for how rich the reading process is. People use many methods to select works to enjoy. Many read Harry Potter. Personally, I don't. A lot of people don't care about awards or ratings either. I imagine I'm not the only boy who's father was the gatekeeper of sci-fi once upon a time and who's memory lingers with each selection.

Look, I'm not trying to take ratings away from you in some SJW-gotcha. I don't even think Vox's strategy is a bad one, but I disagree theoretically. If you like using reviews to choose books or make cases, go for it. I don't think it's a strong case myself.

But what do i know, Harry Potter is apparently objectively better than most of the literature filling my house.

Anonymous taqiyyologist April 11, 2015 8:38 PM  

I wish Negativland would get retroactive Grammys. For every album.

I'm wired like that.

Anonymous taqiyyologist April 11, 2015 8:41 PM  

Negativland, were they not San Fran SJWs, would have a field day with both the Hugo Apocalypse, and Gamergate.

Too bad, so sad. They just want to throw bricks through bank windows, while raging against the machine.

Blogger James Dixon April 11, 2015 8:41 PM  

> But what do i know, Harry Potter is apparently objectively better than most of the literature filling my house.

If you've only been buying the Hugo award winners for the past 20 years, it is.

Anonymous jSinSaTx April 11, 2015 8:42 PM  

@ Bob

Whenever I believe that VD has mad an ill advised statement or approached an issue wrong, particularly in the realm of conflict with 'establishment' media, I keep it to myself. Biggest reason for that is that VD is the one who engaged in the fight, not me. He is the one who has put the effort into writing the books, making the games, launching the blogs, developing the slates, doing the interviews, starting a publishing company and taking all of the nasty heat that came with that. He is the person whose real name is dragged across the internet with vile associations. I have no real skin in the game outside a 40.00 contribution to a organization and a reasonably anonymous internet identity. Why would he give a crap what I think? From his point of view I have not exposed myself or participated in any meaningful way in the manner he has. People who 'do' are more inclined to care about the opinions of others who 'do'. Even if he doesn't agree with them, I would posit that LC or BT making a comment about tactics and strategy would carry more weight. Both of them have been on the receiving end of the same heat and have put in the work to make a change.

Bottom line: our opinions are more respected in environments where we and our achievements are known. Doesn't make VD always correct in how he approaches issues, but this is not exactly a perfect science and he more than most has thrown himself into the fray. It is my read on why he has reacted increasingly irritated with criticism. Most of those criticizing from his side of the field aren't even on the bench let alone in the game.

Anonymous taqiyyologist April 11, 2015 8:43 PM  

Negativland was Aural SABO.

Anonymous taqiyyologist April 11, 2015 8:49 PM  

jSinSaTx

Indeed. I seldom comment here, and read here vociferously. Drink is usually in play when I do comment.

For me, this is usually a place of learning, not teaching. To understate things.

Blogger Noah B April 11, 2015 8:54 PM  

"I already stated mine in the previous thread and I believe hinted in this one as well."

Since several of us apparently missed that, how about a link to this comment where you suggested an alternative metric?

Anonymous taqiyyologist April 11, 2015 9:00 PM  

Did Moers' "The 13 1/2 lives of Captain Bluebear" get nominated for a Hugo?

Tad Williams' "Otherland" series?

No? I cannot imagine why. Other than Stitching Up.

Blogger Vox April 11, 2015 9:57 PM  

People who 'do' are more inclined to care about the opinions of others who 'do'. Even if he doesn't agree with them, I would posit that LC or BT making a comment about tactics and strategy would carry more weight. Both of them have been on the receiving end of the same heat and have put in the work to make a change.

This is true. But moreover, Bob's position is simply wrong. He doesn't understand the other side well enough to know that his "either or" is nothing of the kind.

And it's not the sort of thing I'm inclined to waste time explaining. My position is pretty simple. If you think you can do better, then go out and do it. We can certainly use the help. In the meantime, you are not my coach, my adviser, or my teacher.

Anonymous Scintan April 11, 2015 10:09 PM  

People who 'do' are more inclined to care about the opinions of others who 'do'.

That's a dangerous habit to develop, particularly as it begins to get more and more exclusive, and it's one which can easily be fatal to one's goals down the line.

The comparative accuracy and value of previous opinions is much more important as a predictor, over the long haul. But that's a comfort area that each individual needs to carve out for himself.

Blogger Noah B April 11, 2015 10:33 PM  

While there would have been some satisfaction in telling David W or whatever his name is that he is a hypocrite who never demanded justification for the slate of Hugo nominees in prior years, and that he could therefore go fuck himself with some large, sharp-corned object -- and this is probably what I would have done myself -- this response would not have been received well or have convinced neutral onlookers of the seriousness of the Puppies.

The troops are already rallied, and our morale does not need to be boosted.

Anonymous Jack Amok April 11, 2015 10:59 PM  

I already stated mine in the previous thread and I believe hinted in this one as well.

I looked through what I assume to be the previous thread (Objectively superior) before my previous post specifically to see if you already had. I didn't notice anything under your name that struck me as an alternative measure, but perhaps I simply misunderstood. Could you restate it for me please?

This is very narrow thinking for how rich the reading process is...If you like using reviews to choose books or make cases, go for it. I don't think it's a strong case myself...Harry Potter is apparently objectively better than most of the literature filling my house.

You're still not accepting the situation you find yourself in. I'm not saying reviews are the only criteria by which to judge a book. What I am saying is that whatever criteria you might care to select will create one of two problems for anyone claiming the Puppies slate is inferior compared to recent Hugo winners.

One, the criteria will demonstrate the Puppies slate is not in fact inferior to its Tor-ish predecessors.

Two, the criteria will consistently praise works poorly received by the reading audience. Not just one or two (e.g. Harry Potter), but nearly all of them. It's fine for you to point out that Harry Potter is well-loved by the Mac N' Cheese reading public, but if 9 out of 10 books you claim is worthy of priase is considered inferior by general SF/F fandom, then you have to accept that your criteria simply does not represent fandom but instead represents the subjective viewpoint of a particular clique. At which point we can debate the tastes of that clique and it's preference for such works of towering merit as If You Were A Dinosaur, My Love.

Blogger Noah B April 11, 2015 11:00 PM  

I looked too, Jack.

Anonymous Jack Amok April 12, 2015 1:45 AM  

I looked too, Jack.

He did mention hinting at something, so I'm happy to assume his hints were just a little to obscure for us to recognize. NH is a decent guy, I'm hoping he recognizes the box he put himself in.

Alas though, I will miss the next few days of excitement. I'm off to visit a large, wealthy mouse in southern California. Figured I should get my kids there before it becomes completely untenable. Say what you will about the company that bears his name today (and I can say a few nasty words after being screwed second hand by them), but Walt Disney was a creative force. Seeing what he made is inspiring.

Speaking of kids, has anyone heard an update from Beau? I hope he's on soon to give us a good report on his son.

Blogger Noah B April 12, 2015 2:00 AM  

"Figured I should get my kids there before it becomes completely untenable."

I think you're a good 30 years too late, but I do hope you all make it back safely and manage to enjoy it.

Anonymous NorthernHamlet April 12, 2015 8:00 AM  

Jack Amok,

Could you restate it for me please?

Of course. By on-going conversations within self-selected groups. You and Vox are using this same type of measure yourselves. So are the SJWs. You need something rigid and definitive. They need something nebulous and shifting. I say, choose your conversations more carefully.

You're still not accepting the situation you find yourself in.

I'm fully aware.

What I am saying is that whatever criteria you might care to select will create one of two problems for anyone claiming the Puppies slate is inferior compared to recent Hugo winners.

And here is the problem. You and Vox are so wrapped up, not in showing a way to measure quality, but a way to measure the quality of Hugo Winners to ensure the Puppies can't be claimed as inferior.

It's a fine strategy to shut the critics up... as long as they don't realize what I've pointed out.

Anonymous NorthernHamlet April 12, 2015 8:07 AM  

Vox'll have my ass, but I should add: When I heard him talking about the difference in scores, even before I had thought about it much, I thought "how silly." I couldn't have been the only regular ol' sci-fi/general reader who thought that.

I find it a far more powerful strategy to let your critics here cry insanely, instead of shutting them up. As a neutral, I'm far more convinced when Vox says "I take my writing seriously" than I am when I hear "Amazon likes Vox over Scalzi by .2 points"

Anonymous Aeoli Pera April 12, 2015 10:32 AM  

>I find it a far more powerful strategy to let your critics here cry insanely, instead of shutting them up.

It's not though. Like bob k. mando, you're failing to comprehend the situation. And your technical quibbles so far have been incorrect.

Blogger James Dixon April 12, 2015 1:24 PM  

> You need something rigid and definitive.

There is no such thing, which would make that difficult.

Blogger Feather Blade April 12, 2015 4:21 PM  

@ BigGaySteve, Aeoli Pera:

Thanks for the suggestions. I'll look into those.

Blogger Noah B April 12, 2015 4:54 PM  

"Of course. By on-going conversations within self-selected groups. You and Vox are using this same type of measure yourselves. So are the SJWs. You need something rigid and definitive. They need something nebulous and shifting. I say, choose your conversations more carefully."

Gobbledygook.

Blogger Vox April 12, 2015 8:01 PM  

I find it a far more powerful strategy to let your critics here cry insanely, instead of shutting them up. As a neutral, I'm far more convinced when Vox says "I take my writing seriously" than I am when I hear "Amazon likes Vox over Scalzi by .2 points"

We cover all bases and do both.

Anonymous NorthernHamlet April 15, 2015 5:49 PM  

We cover all bases and do both.

Fair point. Tsk tsk on my binary thinking.

Blogger Marc DuQuesne April 16, 2015 12:45 PM  

Finally finished my data entry, Based on Goodreads, the sp/rp slates are the higher rated than every years hugo noms except 2000, which was a very good year.

As well as the rating data, I also looked at the number of people rating a particular work as a proxy for how well read the work was. The information was interesting in some respects, but I haven't come up with a way to relay the information helpfully yet, I did come up with a percentage of how well read the winner was compared to the rest of the nominees that year for a simple popularity metric. It's heavily influenced by a popular nominee not winning, or in last years case by being unable to find a decent metric for the whole wheel of time series compared to individual novels, so I just dropped it from the calcs.

Year Winner AVG Winner % Winner Title
1953 4.00 The Demolished Man
1955 3.02 They'd Rather Be Right (also known as The Forever Machine)
1956 3.85 Double Star
1958 3.28 The Big Time
1959 3.66 3.66 14.73 A Case of Conscience
1960 3.97 3.80 60.43 Starship Troopers
1961 3.96 3.82 86.23 A Canticle for Leibowitz
1962 3.87 3.74 98.82 Stranger in a Strange Land
1963 3.82 3.92 81.05 The Man in the High Castle
1964 4.01 3.93 2.12 Here Gather the Stars (also known as Way Station)
1965 3.49 3.73 53.15 The Wanderer
1966 3.94 3.85 1.32 ...And Call Me Conrad (also known as This Immortal)
1967 4.16 3.95 20.49 The Moon Is a Harsh Mistress[Note 2]
1968 4.10 3.63 79.98 Lord of Light
1969 3.96 3.89 53.35 Stand on Zanzibar
1970 4.04 3.86 6.50 The Left Hand of Darkness
1971 3.94 3.71 89.79 Ringworld
1972 3.93 3.96 24.89 To Your Scattered Bodies Go
1973 4.05 3.79 80.21 The Gods Themselves
1974 4.02 3.90 69.60 Rendezvous with Rama
1975 4.15 3.83 43.60 The Dispossessed
1976 4.13 3.79 90.48 The Forever War
1977 3.88 3.75 7.12 Where Late the Sweet Birds Sang
1978 4.06 3.87 41.39 Gateway
1979 3.84 3.99 14.19 Dreamsnake
1980 3.93 3.87 57.50 The Fountains of Paradise
1981 3.95 3.92 17.94 The Snow Queen
1982 3.88 3.89 38.06 Downbelow Station
1983 4.01 3.96 47.08 Foundation's Edge
1984 4.02 3.96 31.52 Startide Rising
1985 3.85 3.89 87.54 Neuromancer
1986 4.28 3.94 94.66 Ender's Game
1987 4.01 3.82 81.08 Speaker for the Dead
1988 4.04 3.94 29.58 The Uplift War
1989 3.97 3.82 11.57 Cyteen
1990 4.17 3.95 78.87 Hyperion
1991 4.25 3.91 17.76 The Vor Game
1992 4.30 3.95 11.24 Barrayar
1993 4.04 3.97 27.81 Doomsday Book
1994 3.89 3.83 42.81 Green Mars
1995 4.31 3.95 69.87 Mirror Dance
1996 4.18 3.81 78.45 The Diamond Age
1997 3.89 3.94 53.13 Blue Mars
1998 3.72 3.81 30.92 Forever Peace
1999 4.14 3.80 53.55 To Say Nothing of the Dog
2000 4.30 4.22 1.15 A Deepness in the Sky
2001 4.48 4.11 78.17 Harry Potter and the Goblet of Fire
2002 4.10 3.84 84.24 American Gods
2003 3.75 3.78 20.44 Hominids
2004 4.11 3.87 29.55 Paladin of Souls
2005 3.75 3.85 79.13 Jonathan Strange & Mr Norrell
2006 3.99 3.96 4.65 Spin
2007 3.74 3.86 15.57 Rainbows End
2008 3.68 3.76 55.93 The Yiddish Policemen's Union
2009 4.09 3.89 73.57 The Graveyard Book
2010 3.90 3.69 27.03 The City & the City
2011 3.81 3.86 16.90 Blackout/All Clear
2012 3.67 4.00 3.54 Among Others
2013 3.81 3.82 56.95 Redshirts
2014 3.98 3.95 57.07 Ancillary Justice
2015 4.13

Post a Comment

Rules of the blog
Please do not comment as "Anonymous". Comments by "Anonymous" will be spammed.

<< Home

Newer Posts Older Posts