You are here

Rating System Editorials - No System Change



The voting system change being proposed by M.O.L.M. (Movement Of the Loud Minority) has rightly received a great deal of attention. I will start my article with why I am qualified to speak on this topic:


I have invested a year and a half of my life in to this community, and have produced four 12-20 hour modules that together make up the only high level Hall of Fame module series. I could easily of started a second career with the time and energy put into NWN and would be a much richer and thinner person today if I had. For me this silly little game has provided some profound opportunities to connect with people that I never would have met and ways to learn more about myself and to practice what’s really important to me in my life. I have met and worked alongside some absolutely fantastic builders, and we all sat down together (four times) and made something fun and meaningful (for us) out of nothing. I want to call the voter’s attention to the fact that for me, for those I work with and for most of the people arguing against me this debate isn’t about just a game. This is about how our ideas are going to be measured. This decision matters, that’s why so much passion is being expressed about something that looks superfluous on the surface.

Statistics Matter

Ask anyone who is grounded in statistics (I’ve completed a graduate level course in statistics & written a thesis) and they will tell you that the way that you measure a number carries just as much weight in determining the outcome of your decision as the numerical value of the number itself.

Anyone who tells you that one method is more fair than another is glossing over the fact that any time you adopt a system you create winners and losers. The various MOLM faction members are all people who prioritize one group (their winners) over another (their losers) and believe for various reasons that their winners deserve to get more credit than they do at the expense of their losers.


There is a very vocal minority of the community members that want the way that the votes are counted changed to a way that favors them. This is normal in any competitive arena. Regardless of which way this poll is decided, there will be winners who are favored by the outcome and losers who are penalized by it.

I don’t claim to be an expert in statistics, but judging from what I’ve read on the forums, most of the MOLM people advocating median, mean, Bayesian, trimmed mean or whatever else don’t really understand what they are talking about either. They just know that they want higher scores for the type of games that they like to play and (to them) the only way to be more fair is to tilt the scale towards their style of (fun) game rather than the types of (not fun) games that others like to play. There is always going to be a group like MOLM, no matter what voting system we use or what change we make. It would be a big mistake in my opinion to establish the precedent that we are going to change the system every time a small group can make enough noise.

The whole idea of one statistical system being ‘more fair’ than another is on it’s face invalid and akin to political spin. There is no such thing as ‘fairness’ in the measurement of numbers. Any time you decide what is ‘fair’ you are really deciding what is important and should be emphasized and what should not. No matter what decision you make, you are going to penalize one side at the expense of another. That’s just how it is. Fairness and numbers are oil and water.

The Fatal Flaw in MOLM’s Argument

If we recognize that no one system is inherently more fair than other, then the only real argument for change left standing is that there are too many modules bunched at the top and bottom of the list. I agree that there are too many ‘10’ rated mods, but I also think that in an imperfect world the existing rating system is the most fair. Under the current system once a good module gets ten votes it shoots to the top of the ratings where it is given a lot of exposure and a very public opportunity to stay on the top rated list. Most of the mods (as they should) fall quickly from the top of the list and eventually settle in around where they belong. My last two modules both made it to #1, but neither stayed there for very long. This is how it should be. Right now everyone has an equal shot under the same rules that have always been there. We have no victims in the community right now and no one who was robbed or victimized or who can complain about the rules. All of the rules were in place before we started and no one had anything taken away from them. However, if MOLM is successful in usurping the system we have now and putting in what one of the MOLM factions want, there will be a new vocal minority of injured builders who are frustrated and angry that the system was changed, in addition to those current MOLM faction members who wanted a different system in the first place. I urge you to ask yourself, what will be best for the community?

There used to be an argument that the system needed to be changed to meet the needs of the vault webpage, but Maximus has acknowledged that in practice the proposed changes did little to improve the webpage’s performance.

It’s More Fair Not to Change

If fairness is what we are seeking to achieve, then the worst thing that we can do is change the rules, especially under these circumstances. Everyone who made every single mod on the vault did so under the same rules and with the same expectation of what those rules would be on the vault. They were given notice in that there was an existing voting system that they could see and anticipate BEFORE they started their building. By the same token all of the mods currently on the vault were and continue to be judged by the exact same standard, and for right now, no one has any room to complain. We all started under the same system and we are all judged by the same system. Nobody has grounds to complain or feel like they got cheated because the rules were all in place before we got started. Treating everyone equally under the same system from start to finish is the essence of fairness.

But, if we act to change that system now we will be taking all of that security and credibility away. To change the system now we will be endorsing one type or group of modules over another, and we will be creating a group of injured parties who will rightly feel that they were cheated.

I urge you to exercise your voice and stop this movement of the loud minority by not voting on the poll. The various MOLM factions are required to get at least 1000 votes for any change to take place, and I do not believe that they have enough votes to make it happen, especially if the voters take the time to read and consider all the arguments. The last vote on this topic fell over 300 votes short of the 1000 that would be necessary to put a MOLM faction’s system into place, and that says a lot in of itself. MOLM is just a loud minority.

Clearly the best thing for the community is to stick with what we have. The fact that there are too many ‘10’ rated modules or troll voters isn’t a good enough reason (for me) to redesign the whole system and to in the process create a new group of people who (unlike MOLM) have good reason to feel cheated. You don’t bend an entire system around to fix the problems being experienced by a few of the members of that system, that’s bad business.

(Some people have suggested that the rating system was changed slightly several months ago. I did not see any difference in any of my mod’s ratings, and did not see any announcement on the vault so I am unsure if this is true and even if it is, how much the change affected the ratings. I figure if it wasn’t a big enough change for me to notice or for Maximus to announce, then it must not have been a very big change in the method of measurement if it even happened at all.)

The Psychology of Voting

To address the concerns of anyone who may not agree that the method of measurement inherently favors one side over the other, I offer the following illustration: Different personality types of people like different modules. There are many different groups of gamers that we group by preference. There are Role Players, Power Gamers, Pkers and dozens of other factions all grouped around the preferences they share. These groups naturally gravitate towards certain types of modules and away from other types. These groups also share voting patterns. People who tend to emphasize details prefer complex social settings and light -to-no-combat, while the all-or-nothing crowd likes heavy combat and flashy graphics with minimal reading. Detail-oriented people vote with lots of middle ground votes while the all or nothings tend to love or hate a module. Each type of module and module fan carries with it it’s own positive and negative aspects. The detail people are more likely to come back to vote and the all or nothing ones are more likely to vote really high or low.

If we accept this train of thought, it quickly becomes apparent why it’s so important how you measure votes. One method might favor the extremists, while the other might favor the middle grounders.

If we acknowledge that any time you adopt a system you are choosing sides and making a choice, and that all the systems are equally unfair and biased in one way or another the question of how to be most fair presents itself. In my opinion the answer to that question is that you keep the existing system. The existing system is just as flawed as all the ones that the MOLM people want, but it is more fair than any MOLM system could be because it has always been there.


Migrate Wizard: 
First Release: 
  • up
  • down