5 Star Ratings Are Worthless

I’ve been looking up on Amazon and other shopping sites and aggregators for various products over the last few weeks, and have found something very interesting in my behaviour.

I don’t really read the 5 Star reviews

Why not? Because my behaviour points to a different way of approaching reviews. When I find a product online, I am very unlikely to purchase it without a clear and compelling reason. One of the best reasons is if one of my trusted friends tells me to.  All well and good at this point.

If there are no trusted friends to check out, then I will go to my next best source of information, and that is a highly trusted reviewer. Think of a magazine/blog that you like and enjoy reading and if that person recommends something, it is likely you will look into it.

However, in the absence of a trusted review from a friend or a trusted reviewer, what I look for is the minority public opinion.

Why the “Minority Opinion” counts

First off, there has to be some opinion to see.  If there are no user reviews of a product then how am I going to know? The marketing gumph is set out to make you want to purchase it, and often you won’t be looking unless it has the features you want e.g. “I need a phone” means you’re not going to be looking at washing machines.

When there are reviews, then they will often fall into one of a few camps:

• sycophant – “I love this product” with a 5-star rating
• never satisfied – “This product is rubbish” with a 0-star rating
• partially considered – either one of the above 2, but with a 0.5-star or 4.5-star rating
• considered opinion – “This is what I think” – between 1 and 4 stars

What I tend to take notice of first is the overall number and trend of the ratings. This will give me a clue as to whether the product is good or bad.  If lots of people have reviewed it, then if it is positive or negative, it gives gives you a good idea of how a user has used it.

The next thing I will do is look at the opposite reviews to the trend. I will ignore both the sycophant and the never satisfied (that’s the 0 star and 5 star ratings), and then, if the trend is positive, I will read the considered but negative views, and if the trend is negative, then I will read the considered and positive views.

This gives me a quick overview of either what is good about a product I might not want, or what is bad about a product that I might want.

Minority Opinion informs quickly

I still don’t own an iPhone (or an iPod Touch). When chatting with people about what phone I should get next, I often get told about the iPhone (and then handed one to play with).  The most important thing that they then say is something like:

• it’s a great device, but the phone’s not very good
• I love the apps and the free data, but wish that I could get it on a different provider

These reviews are positive, but considered. This is much more useful to me than either a sycophantic yes or a never satisfied no.

Are 5 Star Ratings Worthless?

No, not completely. But the way that I read information online and the speed with which I have to do it means that I will take less notice of it than a 3 or 4 star considered opinion.

Just getting positive reviews doesn’t help me one bit.

Mashed 08 – Travel Streamr and FireEagle

I’ve update Travel Streamr so that it can now use FireEagle and it works great!  It does not store your FireEagle credentials at all, so you’d have to continually authorise.  This is intentional as I want to store as little as possible (and it’s not like it’s onerous to login to FireEagle).

Doing bits and pieces on it ad hoc. It’s just a bit of a proof of concept project, but it works quite nicely as a mobile application currently.  I wanted to generate a web application in rails that was mobile, used FireEagle and did something useful, so I came up with this at Mashed 08.

Still have to implement user uploading of text, images, audio and video (all geocoded) and using Amazon S3 so it can start to be a useful tourist/local application, but it’s getting there.

And then, when that’s all done, I’ll be adding in lots of other stuff, including geocoded advertising options, so that if you’re in a city and you’re looking for stuff, ads for restaurants and hotels may come up.  Cheap ad options are very very simple.

The API is still “non-commercial” though so until that changes, I can’t make money on this.

Note: Already have the code for S3 because I built another proof of concept called deadparrotimages a while back, which is a temporary image store.  Images are deleted after an hour or a day.  I thought it would be useful to send round images that you don’t want saved on the net forever, something like when selling an item and you don’t want everyone to know about it.

Communities part deux – Reputation Management

One of the things I dislike about online “communities” is that there is often a nastiness that comes along with the interaction. The fact that you don’t have to meet a person to be in a dialogue with them means that you can pretty much say what you like. The gloves are off.

One of the things I really like about online “communities” is that there is often an openness and honesty that comes along with the interaction. You find out a bit more about the real person and some friendships can be created and massively enhanced because of the interaction.

It’s an inherent contradiction that the openness of the internet allows both openness and nastiness.  The worst part of this is that people online will often make assumptions about a person’s ideas or beliefs, and act on that understanding. The reaction of others can often taint a person online, especially in the age of “search” where nothing online gets hidden.

You can’t change other people, you can only change how you react.  If you don’t like the idea of the community being that open, then don’t become a part of it. Dialoguing is sometimes useful, but there can often become a point where it just brings both down in the estimation of everyone who views the interaction.

Reputation Management is a term that has started to appear because of this openness.  I first heard it about 2 years ago in relation to online, where a person in one of my online business networks started offering a service to get a person’s Google search results changed. It meant that things that were undesirable (including negative comments) were pushed down the list and replaced with a more positive view of that person, but the negative things could not be eradicated. It’s spin for the internet generation. Wikipedia founder, Jimmy Wales, is an interesting case of poor reputation management in the fracas with his girlfriend.  Internet spin gone wrong!

I like the idea of reputation management, but I also find it slightly disingenuous. Part of what attracts people is the “warts and all” interactivity of the net. The fact that people/companies with spare cash can in some way portray a positive view of themselves is good sense in a business world, but a person with no skeletons is much more scary than a person with a few. In my view, it’s a balancing act.

What I much prefer is to be able to find other people’s views of a person (or company). If their view of that person is positive then I am going to be more favourable to that view than of the person’s “spin”. Kind of like an Amazon rating for people.

The problem with an amazon rating is that there are many types of people on the internet. Books are different, because the content of the book means that only certain types of people are going to read it. Books on C# will be read by techies. Mills and Boon by women of a certain age.  So the ratings are going to be from people with similar interests.  People are a different matter. They are a part of different groups and have different aims.  So how do you judge their rating?

You have to find people who know that person for the same reason as you may want to. In business that’s quite simple – to use their services or find out about them as a prospective client. That’s why businesses try so hard to get testimonials from their clients. However, they control those testimonials and what you see.  You never, ever see the bad bits.

When you are in an online community with someone, you have the opportunity to find out another person’s reputation from their interactions with others. Most communities have a friends list, and you can find those connections to you.  It’s the idea behind LinkedIn, because you are constantly working through your contacts to find other contacts, a partial trust has appeared, but there’s no rating. I like this approach to a point, but it attracts the wrong type of people in my view. People with a big contact base (sales people generally) mean you end up with a bunch of people all trying to get the biggest number of contacts.

In the end, the best way to manage a business reputation is for a company is for it to be the best it can be and to be clear about what it does. The community around it will then speak out loud about it, and the nay-sayers will be available but will be just noise.  That kind of reputation, where the negative is available, but outweighed by the positive says more about that company than any kind of management of what is visible.

Lots said… need to work!  I’ll probably talk about the idea of followership in my next blog… this one’s got too long!