Blind Voting in Stock Watch Threads

[Allan Houston]

This is a proposal I brought up with the Stock Watch Committee and we thought to bring it to the rest of you to vote on.

With our current voting procedure, where all votes are open, previous votes influence later votes.  If everyone so far voted player A to be ranked between, say, 60 and 62, but you really think Player A is a 50, you can find yourself in a conundrum.  Voting so far off from others, you may worry that they will improperly judge you as stupid, or it could put you into unwanted arguments where you have to strongly justify your vote, while the consensus-meeting votes go unjustified.  So, you end up either abstaining or bumping your vote closer to the established mean.  I know, as someone who has bucked a trend a few times now, I have been tempted to do both, at least until liquid courage kicked in.  And that's bad.  Taking the truncated mean (or median or whatever accumulation function we use) takes care of bad outliers when they exist, and if people aren't voting truly, influenced by "consensus", then the result isn't truly the league's aggregate rating of the player.

Hence, I propose that we institute blind voting.  That is, each vote, including that of the OP, can't be seen by the other voters.  How will this work?

A new Stock Watch Post will have the player's name, his current rating, and an argument why that rating is too low or too high.  However, you don't give a new proposed score, or even a range.  Soon after the thread is created (not sure how quick it will always be but almost certainly within 24 hours), a responsible party (I think me?) will post a link to a Google Form.  The form will have two questions, one asking what score you think the player deserves, and a second question asking you to identify yourself by your GM name, so as to prevent vote stuffing.  Results will then be sent to a Google sheet that at least Billy King and I will have access to, and after the appropriate amount of time we will announce the resulting truncated mean score.

It's encouraged that discussion still takes place in the Stock Watch thread itself, discussing the merits of the proposed player.  But all voting takes place in the Google Form.   

[Blake Bowman]

Is there a set lowest rating?

[billy]

40 is the floor. 99 is the ceiling.

[Allan Houston]

40 is the floor? There are multiple rostered players with ratings in the 30s. Markel Brown is a 34.

[billy]

Jul 20, 2015 0:53:31 GMT -5 Allan Houston said:

40 is the floor? There are multiple rostered players with ratings in the 30s. Markel Brown is a 34.

yes they should be a 40, you cant make a decent build represenative of an actual nba player with a rating less than that. even the lower 40s is basically impossible. Those players in the 30's won't be there when we get around to having stock watch threads :)

[Allan Houston]

Jul 20, 2015 0:55:26 GMT -5 billy said:

Jul 20, 2015 0:53:31 GMT -5 Allan Houston said:

40 is the floor? There are multiple rostered players with ratings in the 30s. Markel Brown is a 34.

yes they should be a 40, you cant make a decent build represenative of an actual nba player with a rating less than that. even the lower 40s is basically impossible. Those players in the 30's won't be there when we get around to having stock watch threads :)

Good to know, as that changes my rating algorithm. I had replacement level determined "empirically", by looking at established ratings, but maybe I should hard set it at like 45? It was 43.

[billy]

Jul 20, 2015 1:04:58 GMT -5 Allan Houston said:

Jul 20, 2015 0:55:26 GMT -5 billy said:

yes they should be a 40, you cant make a decent build represenative of an actual nba player with a rating less than that. even the lower 40s is basically impossible. Those players in the 30's won't be there when we get around to having stock watch threads :)

Good to know, as that changes my rating algorithm. I had replacement level determined "empirically", by looking at established ratings, but maybe I should hard set it at like 45? It was 43.

A guy rated a 50 should be the 12th man on a roster at best, maybe 10th on the Phillies :P

[Allan Houston]

Yea, the Phillies would be a pretty bad basketball team. They're not even good at baseball

[billy]

Jul 20, 2015 1:10:03 GMT -5 Allan Houston said:

Yea, the Phillies would be a pretty bad basketball team. They're not even good at baseball

I wonder if Maikel Franco can dunk it?

[Steve Jobs]

I saw something along these lines looking through the PH stock watch threads, and so I was wondering: Is there anything in place in D720 to keep opposing GMs from tampering with other teams by proposing negative stock watch threads for players they don't own? Even if the player deserves it, I feel like it's within a GMs right to maintain that player's value at least through the end of the season. I feel like an issue like this could be fixed by proposing a rule where players are only eligible for negative stock watch threads in the offseason. Or alternatively, by allowing a negative stock watch thread but instead of processing the change right away like with a rating increase, the player retains their current rating until the offseason, at which time the newly proposed rating decrease comes into effect.

I mean... I know it's not perfect for any of those ideas, but I think it's important to have rules like that in place that keep stock watch as a way to promote faithful representation of talent in the league as opposed to a weapon to make other teams worse. I believe Blind Voting will help give slightly more accurate ratings, but I don't think it solves the potential issue of tampering.

[billy]

Jul 20, 2015 1:12:57 GMT -5 Steve Jobs said:

I saw something along these lines looking through the PH stock watch threads, and so I was wondering: Is there anything in place in D720 to keep opposing GMs from tampering with other teams by proposing negative stock watch threads for players they don't own? Even if the player deserves it, I feel like it's within a GMs right to maintain that player's value at least through the end of the season. I feel like and issue like this could be fixed by proposing a rule where players are only eligible for negative stock watch threads in the offseason. Or alternatively, by allowing a negative stock watch thread but instead of processing the change right away like with a rating increase, the player retains their current rating until the offseason, at which time the newly proposed rating decrease comes into effect.

I mean... I know it's not perfect for any of those ideas, but I think it's important to have rules like that in place that keep stock watch as a way to promote faithful representation of talent in the league as opposed to a weapon to make other teams worse. I believe Blind Voting will help give slightly more accurate ratings, but I don't think it solves the potential issue of tampering.

Nope. If a player is rated incorrectly they should be fixed.

Always think in terms of how this player will be performing IRL in the near future. It's the most fun way to run a league. Try to keep the meta as IRL oriented as possible. So you don't have to worry "oh I know this player will be good but how will he be rated in d720?" We want him to be rated as close to as how they perform IRL as possible.

[Steve Jobs]

Jul 20, 2015 1:14:11 GMT -5 billy said:

Jul 20, 2015 1:12:57 GMT -5 Steve Jobs said:

I saw something along these lines looking through the PH stock watch threads, and so I was wondering: Is there anything in place in D720 to keep opposing GMs from tampering with other teams by proposing negative stock watch threads for players they don't own? Even if the player deserves it, I feel like it's within a GMs right to maintain that player's value at least through the end of the season. I feel like and issue like this could be fixed by proposing a rule where players are only eligible for negative stock watch threads in the offseason. Or alternatively, by allowing a negative stock watch thread but instead of processing the change right away like with a rating increase, the player retains their current rating until the offseason, at which time the newly proposed rating decrease comes into effect.

I mean... I know it's not perfect for any of those ideas, but I think it's important to have rules like that in place that keep stock watch as a way to promote faithful representation of talent in the league as opposed to a weapon to make other teams worse. I believe Blind Voting will help give slightly more accurate ratings, but I don't think it solves the potential issue of tampering.

Nope. If a player is rated incorrectly they should be fixed.

Except it can be almost purely subjective, so there's no way to keep this 100% legitimate.  If there was a definitive way to rank and rate NBA players, a statistician would have discovered it already and there wouldn't be so much push back against advanced analytics.  I know we're applying the opinions of more than one person, but 30ish people isn't a large enough sample size to avoid bias.

[Steve Jobs]

Jul 20, 2015 1:14:11 GMT -5 billy said:

Jul 20, 2015 1:12:57 GMT -5 Steve Jobs said:

I saw something along these lines looking through the PH stock watch threads, and so I was wondering: Is there anything in place in D720 to keep opposing GMs from tampering with other teams by proposing negative stock watch threads for players they don't own? Even if the player deserves it, I feel like it's within a GMs right to maintain that player's value at least through the end of the season. I feel like and issue like this could be fixed by proposing a rule where players are only eligible for negative stock watch threads in the offseason. Or alternatively, by allowing a negative stock watch thread but instead of processing the change right away like with a rating increase, the player retains their current rating until the offseason, at which time the newly proposed rating decrease comes into effect.

I mean... I know it's not perfect for any of those ideas, but I think it's important to have rules like that in place that keep stock watch as a way to promote faithful representation of talent in the league as opposed to a weapon to make other teams worse. I believe Blind Voting will help give slightly more accurate ratings, but I don't think it solves the potential issue of tampering.

Nope. If a player is rated incorrectly they should be fixed.

Always think in terms of how this player will be performing IRL in the near future. It's the most fun way to run a league. Try to keep the meta as IRL oriented as possible. So you don't have to worry "oh I know this player will be good but how will he be rated in d720?" We want him to be rated as close to as how they perform IRL as possible.

I agree with this last part, and like I said, I want stock watch to be a method of promoting faithful representation of player's talents, but there are a ton of ways that it can be misused based on the current rules and all I'm trying to say is there should be some checks and balances in place to keep outliers opinions from having a chance at affecting a player's rating.  If we were getting a vote from all 30 GMs every time I'd feel differently, but within the 10-15 votes that usually come in, it's 100% possible the minority dissenting opinion on a player is the opinion represented most strongly in the thread.

[billy]

Jul 20, 2015 1:17:33 GMT -5 Steve Jobs said:

Jul 20, 2015 1:14:11 GMT -5 billy said:

Nope. If a player is rated incorrectly they should be fixed.

Except it can be almost purely subjective, so there's no way to keep this 100% legitimate.  If there was a definitive way to rank and rate NBA players, a statistician would have discovered it already and there wouldn't be so much push back against advanced analytics.  I know we're applying the opinions of more than one person, but 30ish people isn't a large enough sample size to avoid bias.

I still don't see why we would stop negative threads... You are not sabotaged by the thread but by your players performance in real life... In which case the rating SHOULD change. Always take a few seasons into account though, but recent trend should matter as well.

[Allan Houston]

I actually think 30 properly reported subjective votes properly aggregated could do a solid job of approximating an IRL value. Wisdom of the crowd and all.

The issue is if people strategically vote, voting to manipulate the game rather than truly reflect their opinion in the real player. In the extreme case, I could vote all my players to be 100 and all opponents' players to be be 40, thus skewing the average (ignoring that with free agency that's a bad strategy). But those extreme strategic votes are taken out with the truncated mean. If multiple people do this, more measures would have to be taken. We could consider taking the median to eliminate all outliers (though maybe legit outlier votes should be given their due). For now, those of us with access to the voting results spreadsheet can see the votes, and any obvious strategic votes can be curated out.

[Steve Jobs]

Jul 20, 2015 1:22:14 GMT -5 billy said:

Jul 20, 2015 1:17:33 GMT -5 Steve Jobs said:

Except it can be almost purely subjective, so there's no way to keep this 100% legitimate.  If there was a definitive way to rank and rate NBA players, a statistician would have discovered it already and there wouldn't be so much push back against advanced analytics.  I know we're applying the opinions of more than one person, but 30ish people isn't a large enough sample size to avoid bias.

I still don't see why we would stop negative threads... You are not sabotaged by the thread but by your players performance in real life... In which case the rating SHOULD change. Always take a few seasons into account though, but recent trend should matter as well.

I honestly don't have any problem with negative threads in general.  But everything comes down to 15 numbers, here, where in real life there are millions of different factors.  Talented players go on full season slumps relatively often, but it doesn't mean they've lost the ability to do the things they could do. If Lance Stephenson was re-rated realistically now based on his 14-15 performance, someone who spent a solid draft pick or traded appropriate value for him could be seriously burned, but the chances of him returning to form with the Clippers are exceptionally high. When only one thread can be made for a player per season, it's possible to make a reactionary change that can't be rectified once the player returns to form.  

I mean, more than anything, I'm not trying to say the system should be nuked for decreasing a rating, I'm just saying there are a lot more consequences and ways to collude with a negative thread than a positive one, and for that reason I think there should be more protection in place for what can be done to decrease a player vs. what can be done to increase a player.  

Maybe a simple answer here is capping the amount of decrease allowed to a player's rating based on the length of time for which their value remains decreased... a few bad games = can't decrease, half-season slump = 1-5 point decrease, full-season slump = 5-10 point decrease, etc.  

[Steve Jobs]

After reflecting on the issue further, I think I have a better idea:

My previous suggestions have all been aimed at restricting the content of a negative stock watch thread, but in reality my issue has nothing to do with the content. If a player deserves a decrease in rating, they should get it, and I agree with that premise. However, the thing I don't like is the delivery: in a system where only one thread per player can be made in a year, someone else can decide, on a whim, to make a thread about my player whom they don't like and end up resulting in his rating being decreased. I don't care at all, however, if somewhere between 1-30 people come to a consensus that my player is rated too high. I care that someone else gets to make that decision for me unchecked...

So my proposal here: Negative Stock Watch threads must be proposed to the ratings committee (the people in the league who are most familiar with the ratings of certain players and when those ratings might be skewed), whereupon the ratings committee will vote amongst themselves (yes or no) whether a player needs a decrease in rating or not. If the majority of the Ratings Committee says no, then it can be reasonably determined that the player's rating is sufficient (or not sufficiently worth downgrading). Alternatively, if the majority of the Ratings Committee believes a player needs a decrease then it can be reasonably determined that a player's rating is incorrect, thus the thread will be posted by the GM who proposed the decrease, and the league votes on the new rating based on the evidence assembled.

[James K. Polk]

Why don't we just add a poll to the thread?

[billy]

Jul 20, 2015 1:30:02 GMT -5 Allan Houston said:

I actually think 30 properly reported subjective votes properly aggregated could do a solid job of approximating an IRL value. Wisdom of the crowd and all.

The issue is if people strategically vote, voting to manipulate the game rather than truly reflect their opinion in the real player. In the extreme case, I could vote all my players to be 100 and all opponents' players to be be 40, thus skewing the average (ignoring that with free agency that's a bad strategy). But those extreme strategic votes are taken out with the truncated mean. If multiple people do this, more measures would have to be taken. We could consider taking the median to eliminate all outliers (though maybe legit outlier votes should be given their due). For now, those of us with access to the voting results spreadsheet can see the votes, and any obvious strategic votes can be curated out.

Do you think perhaps blind voting would cause people not to experience shame, and therefore to vote in their interests? Someone voting obviously biased would get called out normally...

[billy]

Jul 20, 2015 1:41:47 GMT -5 Steve Jobs said:

Jul 20, 2015 1:22:14 GMT -5 billy said:

I still don't see why we would stop negative threads... You are not sabotaged by the thread but by your players performance in real life... In which case the rating SHOULD change. Always take a few seasons into account though, but recent trend should matter as well.

I honestly don't have any problem with negative threads in general.  But everything comes down to 15 numbers, here, where in real life there are millions of different factors.  Talented players go on full season slumps relatively often, but it doesn't mean they've lost the ability to do the things they could do. If Lance Stephenson was re-rated realistically now based on his 14-15 performance, someone who spent a solid draft pick or traded appropriate value for him could be seriously burned, but the chances of him returning to form with the Clippers are exceptionally high. When only one thread can be made for a player per season, it's possible to make a reactionary change that can't be rectified once the player returns to form.  

I mean, more than anything, I'm not trying to say the system should be nuked for decreasing a rating, I'm just saying there are a lot more consequences and ways to collude with a negative thread than a positive one, and for that reason I think there should be more protection in place for what can be done to decrease a player vs. what can be done to increase a player.  

Maybe a simple answer here is capping the amount of decrease allowed to a player's rating based on the length of time for which their value remains decreased... a few bad games = can't decrease, half-season slump = 1-5 point decrease, full-season slump = 5-10 point decrease, etc.  

This is why we can change their ratings each season... why should Lance Stephenson perform well in the 2014 sim for D720 but perform like trash in real life... if he starts performing better for the clippers, then he can get a rating boost again. Lance is actually one of the first negative threads I was going to post up after this first initial stock watch batch.

I don't think there should be cap for the ratings... everyone should vote how they feel. Some people would still give whiteside a 50, saying he needs to "prove it" over the course of a season, some people thinks he has earned his rating, some people thinks he is better than his current ratings. Thats why we vote. As much as we want to think our opinion on a player is right, in the end it is just an opinion.

[Allan Houston]

Jul 20, 2015 9:05:40 GMT -5 billy said:

Jul 20, 2015 1:30:02 GMT -5 Allan Houston said:

I actually think 30 properly reported subjective votes properly aggregated could do a solid job of approximating an IRL value. Wisdom of the crowd and all.

The issue is if people strategically vote, voting to manipulate the game rather than truly reflect their opinion in the real player. In the extreme case, I could vote all my players to be 100 and all opponents' players to be be 40, thus skewing the average (ignoring that with free agency that's a bad strategy). But those extreme strategic votes are taken out with the truncated mean. If multiple people do this, more measures would have to be taken. We could consider taking the median to eliminate all outliers (though maybe legit outlier votes should be given their due). For now, those of us with access to the voting results spreadsheet can see the votes, and any obvious strategic votes can be curated out.

Do you think perhaps blind voting would cause people not to experience shame, and therefore to vote in their interests? Someone voting obviously biased would get called out normally...

There is certainly less social pressure to vote a certain way, as that's the entire point of blind voting.  But since some people do see the votes, that shame isn't totally gone.  Obvious strategic voters can be called out in PMs.

[billy]

VOTING CLOSED.

Blind voting will get a trial run during the offseason and into next season.