General Discussion : I hate the star rating system

I think there was a discussion on this before... my take is that the star system is a bit silly. Assigning a quantitative value to something so subjective just doesn't make sense, especially when the people giving the rating aren't necessarily qualified to do so (as opposed to a publication's book or film critic giving out star ratings).  I'd prefer honest feedback about what works and what doesn't, without my work being rated against anyone else's, which is essentially what a star rating system does.

Instead of a star rating system, I'd prefer an option for a reviewer to add a "recommended" tag to any story he or she reviews. That way, people browsing can have the option of looking for something considered really good, or something that needs a bit more feedback.

Personally, I'll know my work is good if it gets accepted by a legit publication. Otherwise, it needs polish.

@ Bret, that's a great point... prose doesn't seem to count...

@ Sparrow... I just think that a star rating tries to apply a concrete, objective value to someone's own personal opinion.

I'm not really keen on movie reviewers giving star ratings, either, so it's not just about qualifications... that was just an example.

Maybe a better way of approaching it:  what function does a star rating serve, as far as helping the person receiving the review? If some part of a story needs work, the review can state that in detail.

I don't mind them, although I think maybe one or two more rating groups wouldn't hurt. Maybe a drop down that is unrelated to the overall rating.

My big complaint is reviews don't seem to know how to use the star rating. In the LBL they write, "Oh, I loved the concept! It was ordinal, vibrant, imaginative, best idea I've seen in years!" and then give it a 1 or a 2 star rating in concept. Or, "Decent writing, horrid concept," and give it a 3. I know it's subjective, but 1 should be really bad, 2 should be bad, 3 should be mediocre, 4 should be good, 5 should be great. I don't feel like I'm out on a limb here.

I basically don't even look at the stars. They're pretty much worthless because they don't really mean anything. You can love a story and still think it's absolutely plauged with problems, so stars don't really help. Dialogue in pitch might be great, but the writing might be off, or the voice, or whatever. Stars can't tell you that.

I think Dwayne's suggestion of a couple more rating groups is solid. What's there seems a touch skeletal, and as of now, you can't even get an accurate score for a story with no dialogue, so it might be nice if categories could be removed as well.

I use it like Avery, an at a glance to get me ready for what's coming. I also know where to really pay attention in the review/LBL if I see a low star rating on something--especially if I see it over and over again.

And, although the workshop has a lot of members, I do get reviews from the same people a lot, and if I remember that they rated my dialogue as a three the last two times and a four this time, I know that if I have been taking their commentary to heart, I'm making progress.

Jumping off what Renee said, I think it can be really helpful if you know the reviewer. Everyone has a different scale for the stars. Dwayne's scale is skewed from mine -- I usually give the concept at least three if it was fully explored, even if it was a terrible idea. Four if it's authentic. Five if it's complete, well-executed, and authentic -- which, to me, means that it could be a vampire story written in a new way that makes it seem original.

I try so hard to be unbiased in the workshop that when I rate concept, the factors are usually whether it's a fully fleshed out idea and whether it's authentic. I've given some truly terrible stories -- a subjective feeling on my part -- fours and fives because they explored the idea as far as it could be done and were truly original.

Then there are some stories that had ideas that I loved, absolutely adored, but gave a one or two because I've seen it so many goddamn times -- from that writer or in general. If this is the seventh time you've written about person one having a secret and person two finding it out, then reacting somehow, I'll give it a one even if it's the coolest idea ever. And if you write a full story but leave an entire plotline unfinished, I'll give it a bad rating.

I try really, really hard to be unbiased. I think people (myself included, when I was new) are really afraid to submit stories that aren't horror, noir, transgressive, subversive, or similar to those genres. I forced myself to use my fifteen free points for a terrible journal extract that was lifeless and about feelings and emotion so that I'd get over it, but a lot of people don't. I don't really want to read your romance stories, your Edward Cullen fanfiction, but if I can help you improve it in a way that stays true to what you've written, I'll do it. And I'll try my damndest to not let my personal preferences get in the way.

That's the problem with the stars -- I can't tell every reviewee how I use the star system so they understand. But once I've reviewed you a couple of times, it starts to make sense, and the stars actually mean something.

Should I automatically get a lower star rating because it's my third noir story?

God, no. I wasn't saying that. Authenticity is doing something that may have been done before in a fresh new way. Look at the novel R. Moon is shopping at the moment -- it may not be the most unique idea, but it's thrillingly authentic. I've only reviewed one of your stories -- doing the second part today, by the way -- but if you had another story that was noir in the workshop, that wouldn't damage my concept rating. I'd lower my rating if it had the same plot as this one, though.

Themes are the same way. If you can tell a story with the same theme as your last one in a new way, go for it. If I've seen it before but I'm still enthralled, you've probably done it in a way that's fresh enough that I'll give you a good rating on concept.

But see? Everyone has different criteria for their star ratings and I can't tell every person I review my process. I try to say "I rated your concept this low because I thought that you didn't explore the idea," but until they've been reviewed by me a few times, they'll need to read the review to figure out what the stars actually mean.

I mean, in almost any case, a four-star review is great for me. I think of it like an A-.

It's called The Naomi Pick-up. The chapters I've reviewed are all still in the workshop, plus a revised one that I'm reviewing tomorrow. It's so fucking good.

I give that post 5 stars across the board.

Stacy, I don't think those are the kinds of stars we're talking about. 

Fuck, I miss scratch and sniff stickers.  

I'd like it if that was made offical, posted all over the writer's workshop. Or any set standard.

@kirk: from the cheap seats. My trepidation about the star rating was centered on the fact that it wasn't a standard it seemed so subjective and so individual. But, with a legend like the one you provide then the stars mean more and I can feel more confident about assigning them.

1 star - needs a lot of work, consider doing major revisions
2 stars - needs a lot of work but has fundamental elements worth chasing
3 stars - off to a great start and has potential but needs work
4 stars - very good, has a few minor things that need to be addressed
5 stars - great and polished, I see no reason you can't send this out to publishers

You can explain the stars and how they came about in the body of the review. The only other thing to do is to make sure the star categories are the ones that will do the most for the writer's development.

Concept                                                                                                                                    Structure                                                                                                                                        Characters                                                                                                                                          Dialogue

And that they each have a definition that can be followed confidently.

Of course if there is a faq I have totally missed, a pointer will be all that is necessary.

How about

1: Reads like a first draft.  

2: Reads like a second draft that hasn't been workshopped.

3: Reads like a second draft that has been workshopped.

4: Reads like a third draft that have been workshopped.

5: Reads like publishable material.  

Dear Stacy

We did indeed see your system above, but unfortunately there are serious issues that at this time prevent us from moving forth.

In this time of cut backs who are you suggesting pays for the fridge, the Mom to put them up, the surprise bag, the surprises to go into the bad, and various transpiration/storage costs? I think I speak for the group when I say there isn't money handy for this. Also, we aren't certain that it is ethical to have someone be a mother for hire. We would need need more information on the proposed position of 'Mom to put them up'.

Your rate of stars to prize exchange means a writer gets to pick out of the bag once every 45 stories (assuming they received a average rating 2.5 stars which would result from a equal distribution of stars in the 1 through 5 brackets). Since most of these costs are not effected by how often someone picks a prize it doesn't work towards cost savings and seems less then ideally motivating.

Due to the above listed reasons we are not able to treat you idea as serious, regardless of it being 'brilliant and you get prizes'. Please note I do not concede those points as being true since I haven't taken the time to examine them, simply not relevant to the point I am making.

If you can raise funding, perhaps through http://www.kickstarter.com please revise your proposal and let us know.

sincerely

Dwayne

I like the star rating system. It gives you an overall of how well recieved your story was. You can or can't take it seriously, it's up to the person. I think the crtiques (lbl or comment) are the core of it.