In learning gamification, I am always in favour of creating content leaderboards. When course participants review the content and give it a star-rating with 5 stars being top and 1 star the low end, it gives the learning and development team and content owners feedback on the content provided. Content that receives 5 stars consistently by the majority of people is obviously the kind of content you want to create more of.
For the unmarked, yet taken courses and the low marked courses, you want to explore what the problem could be. It could be not relevant for the problem the individual was trying to solve. It could be just bad content. Ideally, I would look for participants to give a bit more feedback on those kinds of courses, to find out what could be improved to make it better.
As I was working through a content leaderboard for a client, I was also discussing the measurement system with my partner, who works with sports statistics. I was explaining that I gave a 5 star rating 50 points, 4 star 40, 3 star 30, etc. and then divided it by the number of actual votes received, to create an average score. Because some courses received 100s of ratings and others only a handful, I was considering a weighting in favour of the heavily rated ones. My partner immediately responded that it then would create an unfair advantage towards higher number of votes always ending up higher in the rankings, which made me conclude that the average was probably the fairest way of showing the ratings.
We talked some more about it and then looked at the App Store on our iPad, sharing the star rating for an app and in brackets the number of votes. That completes total transparency of any given rating score. It is ultimately what I am aiming to implement for my client, both a leaderboard with the top 25 rated courses and for each course a star rating with the number of votes received in brackets beside it.
Sometimes the answers are right in front of us, but talking it through with another person gives new perspectives. For me the client ultimately gains and we can present a clear way forward to the developer team as to what the exact plugin needs to be able to do.
