| 4 people marked this as a favorite. |
I’d personally like less granularity, to be frank. The numerical rating makes little sense to me given everyone is using a different standard in assigning stars anyhow.
I’d prefer just a three option: liked it-ambivalent-disliked it.
| 4 people marked this as a favorite. |
Just look at some of the reviews certain adventures have received. "If I could have given this adventure zero stars, I would have".
You know that if the option to give no stars was present, the comment would have said "if I had the option to give negative stars, I would have", right? It's a common sentiment to express "I hate this so much I wish I could give it a rating lower than is actually possible." The opposite of "this one goes to 11... it's one louder than 10".
I mean, really, having no stars is merely mediocre, right? When your kid comes home from school without a star on their forehead, you don't punish them for not trying hard enough, right?
Maybe what we should do for clarity is add a scale below neutrality. A product can have up to five... communicable diseases, to represent how bad it is. Or up to five... tax audits. Or if we've watched Deadpool recently, we could go for up to five nutshots.
I think what I'm saying here is... meh, leave it alone.
| 1 person marked this as a favorite. |
Alas, if the system was changed now it would make it difficult to correlate data from older and newer reviews.
Alas, if the system was changed now it would make it difficult to correlate data from older and newer reviews.
So go to the following ...
Really disliked
disliked
ambivalent
liked
Really liked
Hrmmm - that's 5 ratings.... almost like you could represent it by.... 5 stars?
|
Disclaimer: I'm only looking at that from the perspective of a potential buyer of a book, so if there is some tanglible benefit the rating system grants to Paizo (I kinda doubt it^^), I'm totally ignoring that.
To be honest, I think that ratings are basically meaningless anyways. Even (or better: especially) with the excellent Endzeitgeist reviews, the rating is the least valuable part of the whole thing.
What matters is the actual review. And there, what bothers me with a lot of reviews is that they don't tell me much about the product in question itself. I'm basically not interested in if you like a product or not (apart from some individuals where I think I have an idea about what they like and what not), and while I know that not everyone has it in them to write elaborate Endzeitgeist reviews (I've written some of those myself over the years so I know how much time and effort you have to put in them), but most are so short and vague in their criticism (be it positive or negative) that as a reader, I still have no idea about if that product might be for me.
Oh yeah, and any personal attack against Paizo or the author of a product will make me dismiss a review immediately as not worth my time.
Back to ratings: I'm not sure if Paizo has an explicit definition of their 5-star rating system, but even if they have a good definition, most reviewers tend to ignore that definition anyways. At another board, some of my reviews caused the impression that the product in question was utter crap, even when the 3 stars it got from me were defined as "has some weaknesses, but is still useful and worth buying". Problem being that everyone was throwing around 4- and 5-star ratings even for average products (because they liked them personally), so in the end no none took the ratings as per their definition anyways. And if something had less than 4 stars, it couldn't be good (because if it was average, I would have given it at least 4 stars, right?)
I see some of that here and I'm part of that myself. I've only written a few reviews at paizo.com so far, and with one exception, they all got five star ratings. Part of that is naturally my experience from that other board, part of that is that I really liked those products, but no matter what, if everything gets five stars, there's nothing for the reader to take awy from. And worse, that 4-star product might, in that context, get a worse reputation than it deserves, because it is a really good one as far as content is concerned.
But at least I try and give some context about what I liked and why I liked it, and more importantly, about the actual content. Because after you've read my review, no matter what you think about my opinion, at least you should have an idea if that product's content might be something you're interested in.
|
|
As long as there is a minimum, someone will give the minimum and someone else will want to give less than the minimum.
If I got to make a change to the ratings system, I would have each reviewer's average review rating and total number of reviews appear next to their rating for that item. If Player32 gives a 1-star rating but his average is 1.15, I'm not going to pay much attention. If his average is 3.94, then that 1-star review has weight.
Especially for Pathfinder Society Scenarios, there are a lot of people who only review scenarios they had a terrible experience with. And a lot of the time their reviews really consist of complaints about the GM not being prepared or not knowing the rules. ("This was supposed to be a social scenario, but the GM didn't do any role-playing, just had us roll skill checks and was done in an hour." "The GM didn't understand swarm rules and TPKed us.")
| 2 people marked this as a favorite. |
We have no plans to revisit the 5-star system we have in place for product reviews. This system is fairly standard in eCommerce and it's widely accepted that a 1-star rating is viewed as the bottom of the spectrum.