Page Index Toggle Pages: 1 ... 3 4 [5] 6 7 ... 12
Topic Tools
Very Hot Topic (More than 25 Replies) Yelena Dembo on Chess.com (Read 99760 times)
Smyslov_Fan
God Member
Correspondence fan
*****
Offline


Progress depends on the
unreasonable man. ~GBS

Posts: 6902
Joined: 06/15/05
Re: Yelena Dembo on Chess.com
Reply #117 - 09/23/10 at 15:17:39
Post Tools
drogo wrote on 09/23/10 at 14:50:41:


...
@Volcanor: I think you may be right (and I guess that RHP calculated such error probabilities). That being said, I don't have that many examples of analysis results to guess a distribution. And we just cannot assume the normal distribution because, as I said, no human passed the 65/75/85 threshold.


Finally, somebody from chess.com acknowledges that statistical testing may be useful, and they haven't done it yet!

Also, this comment that "no human passed the 65/75/85 threshold" is not true. You have had numerous examples of humans passing that threshhold, just not for more than 20 games. When you consider that fatigue is a major cause of human error and that CC chess removes that factor, isn't that threshhold a bit tenuous given all the factors that others here have pointed out?

Also, consider that in the recent WC match between Anand and Topalov, top players are surprised by Anand's errors. They expected a much lower rate of mistakes!
  
Back to top
 
IP Logged
 
drogo
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 40
Joined: 09/21/10
Re: Yelena Dembo on Chess.com
Reply #116 - 09/23/10 at 14:50:41
Post Tools
@TN: you seem unable to understand that all serious CC players from 2000 on use computers to assist their play.

@Volcanor: I think you may be right (and I guess that RHP calculated such error probabilities). That being said, I don't have that many examples of analysis results to guess a distribution. And we just cannot assume the normal distribution because, as I said, no human passed the 65/75/85 threshold.
  
Back to top
 
IP Logged
 
Volcanor
Junior Member
**
Offline



Posts: 61
Location: Switzerland
Joined: 03/16/09
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #115 - 09/23/10 at 14:30:31
Post Tools
Zygalski wrote on 09/23/10 at 13:34:43:
Volcanor,

As I keep saying, I do not know how chess.com arrive at their banning decisions.

The process I have been involved in is that I and several others who all use Batch Analyzer tool in conjunction with various engines on various systems analyse the games.  Maybe 2 analysts per suspect or 4 as in the Dembo case.
This evidence is submitted to an IM on the chess.com staff, a ticket is raised & that is the end of our involvement.

Many players with similar match up rates to Dembo are (at the moment) still active on site, whilst some with lower rates have already been banned.
This suggests that chess.com's methods are significantly different to ours, though about 60% of the players we've submitted on have since been banned.

I hope that makes sense?

Thanks for the reply, it makes sense. I don't mean to be rude, but if I'm correct, you're not of IM or GM stength at correspondance chess: in which case, I don't see how it is possible for you to judge such a case. Of course, you're not responsible (in fact, yes, you could be better at correspondance chess  Wink), but the choice of expert from chess.com is open for discussion...

@ Drogo: I find it strange to state that "It is not possible to calculate the probability of error because there is simply no data". I'm no expert in statistics but if data from correspondance chess from the pre-computer area are used to calculate an avarage for t1, t2, t3 (of 60%, 75% and 85%, respectively according to Zygalski's post), they could be used to calculate a standard deviation and a confidence interval. Thanks to these values, you can say that player YD is a cheater according to his/her last 20 games with a risk of X% of being wrong. Of course, whether the set of data is relevant or not (both the original data and the games from YD) is another issue, as raised by TN and others. But at least, the t3/t4 test would have some statistical signification: if you're unable to associate a risk or error with a decision, you're not doing statistics!
  
Back to top
 
IP Logged
 
TN
God Member
*****
Offline



Posts: 3420
Joined: 11/07/08
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #114 - 09/23/10 at 14:06:24
Post Tools
drogo wrote on 09/23/10 at 13:21:22:
TN wrote on 09/23/10 at 07:28:52:
Here's my view on the entire incident:

a) Yelena did not cheat. 

b) There is no serious evidence whatsoever to prove that she cheated. 

c) Comparing pre-1980 correspondence games or OTB games to current correspondence chess is ludicrous. 

d) Chess.com is the one who has lost in this incident. Kicking out a strong titled player based on an ill-founded suspicion does not encourage titled players to join or stay on chess.com.


a) How can you know that?
b) Nothing can be proven: either that he cheated or that she didn't cheat. I guess Yelena doesn't have multiple webcams in her house to broadcast all her actions, 24 hours per day.
c) The comparison was between supposedly current UNASSISTED CC games and pre-1980 CC games or modern OTB games.
d) Chess.com has a bunch of titled players. The highest rated GM is Julio Becerra, rated at least 300 points below Yelena on chess.com. Other GM's play there and their ratings are even smaller.


a) I clearly stated that it was my view, not a fact. 

b) Like Zygalski, you miss the point. If Chess.com decided to compare Dembo's games with correspondence games from 2000 on rather than the 1960s and 1970s Correspondence games and found Dembo's percentages to be higher than the samples by a set amount, there would be serious evidence suggesting that Dembo cheated in her correspondence games on chess.com. Nowadays humans are more creative and are more likely to find computer-like moves independently (in correspondence - take note) than players from the pre-computer era. 

c) And this comparison does little to suggest that Dembo cheated. 

d) Non-sequitur. Titled players will stay on chess.com, but this incident makes the chess.com environment less appealing, at least for me. FIDE rated games and correspondence games are completely different, so your comparison is not relevant. 

  

All our dreams come true if we have the courage to pursue them.
Back to top
 
IP Logged
 
TalJechin
God Member
*****
Offline


There is no secret ingredient.

Posts: 2892
Location: Malmö
Joined: 08/12/04
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #113 - 09/23/10 at 13:38:30
Post Tools
Zygalski wrote on 09/23/10 at 10:46:47:
TalJechin wrote on 09/23/10 at 10:33:57:
You mean there's something wrong with throwing people in the well and see if they float?

Well if it's good enough for a Fide 2300 who has been a games mod for many years, then t3 or t4 seems at least a reasonable place to start.
Many thanks for your positive contribution though. Smiley



I've read somewhere that when chess started getting serious in the mid/late 1800s, players would shake hands before the game as confirmation that they would accept no outside assistance. 

Today, it seems hard to avoid accusations of cheating even in an otb world championship match. 

Having a special "police force" with secret formulas to determine and publicly shame statistically found suspects sounds like overkill for a game that's supposed to be played for fun. 

My best suggestion would be to go back to the "handshake", i.e. let them before the start of a game click to approve an agreement of not to cheat and not to accuse, on their soul or honour - and leave it at that. As I understand it there are no starting fees or money prizes in these chessdotcom tms - so why make such a big deal of it?


Otherwise, there are two choices: either allow everything, which removes both cheating and accusations of cheating or use a statistical system that may catch the most stupid cheaters and potentially some innocents - but it will never stop accusations of cheating, which is just as cancerous a problem as actual cheating...

T3 or T4 - there are a limited number of moves to choose from at every turn, and the stronger the player the fewer moves get consideration. With small differences in engine evals it's hard to say more than: X chooses moves similar to rybka at 18ply, which doesn't mean that X is using rybka. Engines are programmed to play more humanly, so maybe rybka is playing like X?

Anyway, people could always use an engine for a quick blunder check - or would you stipulate that everyone must make very suboptimal moves regularly?  Grin

Players intent on cheating (or afraid of being suspected) could perhaps check rybka's top 3-4 and make sure to choose a fifth one, though there's no guarantee that the fifth move won't be number 1-3 on the next ply depth.


To sum up, chess.com can ban their players all they like but I think we all agree they should stop publishing names of individuals as "known cheaters" on the internet.

Well, this is my final entry on the subject. Good luck.
  
Back to top
 
IP Logged
 
drogo
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 40
Joined: 09/21/10
Re: Yelena Dembo on Chess.com
Reply #112 - 09/23/10 at 13:29:07
Post Tools
Volcanor wrote on 09/23/10 at 13:21:05:
Like explained by Markovich, there is a statisical risk to conclude that a non-cheater is a cheater. If I undestood correclty, Markovich asked the level of risk (5%, 1%, 0.1%, 0.01% ?) you consider, and he didn't get a reply.


It is not possible to calculate the probability of error because there is simply no data. All OTB world champions are below this threshold. All current 2600+ GM's are below this threshold. All CC top players are below. Also, while randomly picking a player and analyzing his games, the results are way below the threshold. 

To me it looks that this method is a good indicative of playing strength. For instance, it was surprising to see that Capablanca or Alehin have a higher matchup rate that current super-GMs.

It is impossible to analyze ALL the games played until now, so there is the possibility to have false positives. Likewise, since it is impossible for us to visit all the corners of the Earth, there is the possibility that flying pigs might exist somewhere.
  
Back to top
 
IP Logged
 
drogo
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 40
Joined: 09/21/10
Re: Yelena Dembo on Chess.com
Reply #111 - 09/23/10 at 13:21:22
Post Tools
TN wrote on 09/23/10 at 07:28:52:
Here's my view on the entire incident:

a) Yelena did not cheat. 

b) There is no serious evidence whatsoever to prove that she cheated. 

c) Comparing pre-1980 correspondence games or OTB games to current correspondence chess is ludicrous. 

d) Chess.com is the one who has lost in this incident. Kicking out a strong titled player based on an ill-founded suspicion does not encourage titled players to join or stay on chess.com.


a) How can you know that?
b) Nothing can be proven: either that he cheated or that she didn't cheat. I guess Yelena doesn't have multiple webcams in her house to broadcast all her actions, 24 hours per day.
c) The comparison was between supposedly current UNASSISTED CC games and pre-1980 CC games or modern OTB games.
d) Chess.com has a bunch of titled players. The highest rated GM is Julio Becerra, rated at least 300 points below Yelena on chess.com. Other GM's play there and their ratings are even smaller.
  
Back to top
 
IP Logged
 
Volcanor
Junior Member
**
Offline



Posts: 61
Location: Switzerland
Joined: 03/16/09
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #110 - 09/23/10 at 13:21:05
Post Tools
Zygalski wrote on 09/23/10 at 09:55:41:
I was one of the 4 analysts who fairly recently looked at 20 of WGM Yelena Dembo's chess.com games.
These games were objectively chosen (insomuch as is possible) in that they were the then most recently completed games vs 2200+ rated chess.com opponents.  All the games had 35 or more moves.
I found only 18 games that fulfilled this criteria, so I then selected the 2 most recently completed games which were vs near-2200 rateds which also had 35+ moves.

I would like to point-out that there seems to be a commonly held misconception that chess.com closed Yelena's account purely on the back of top 3 or top 4 engine match up results.  This simply is not the case - the t3/t4 analysis was used by a group of players simply to suggest members who were possible engine-users.  

As a result of engine match up analysis of the 20 games by 4 separate analysts using 4 different systems & 3 different engines, full ply-by-ply analysis for the games was forwarded to site staff for them to peruse.

Zygalski, thanks for partially explaining how chess.com tries to identify cheaters. But I'm confused with the t3/t4 test. Like explained by Markovich, there is a statisical risk to conclude that a non-cheater is a cheater. If I undestood correclty, Markovich asked the level of risk (5%, 1%, 0.1%, 0.01% ?) you consider, and he didn't get a reply.

I'd be glad to have a reply to Markovich's question, but I have another question. By reading your statement, it seems that you (or chess.com) use the t3/t4 test to detect potential cheater. And then, you send the 20 games to 4 reviewers and await for their conclusion.

Did I undestood correctly? And if so, are these 4 reviewers strong correspondance chess players (IM or GM)? And do the 4 reviewers need to agree that the "suspect" really cheated?

Zygalski wrote on 09/23/10 at 09:55:41:
Maybe a more positive approach would be for some of you guys to outline in some detail an alternative methodology for detecting possible engine users other than the one I've partly described in my last few posts...

If the answer to all my above questions is yes, I think your methodology is pretty good. If not, you have an alternative methodology (a time-consuming one, I agree!). Of course, I don't claim that such a methodology is flawless, but at least it makes more sense than to rely blindly on the "magical" t3/t4 test. Ultimately, and whatever the methodology used, I'm against creating a "known cheater" list, but that's not my question.
  
Back to top
 
IP Logged
 
TalJechin
God Member
*****
Offline


There is no secret ingredient.

Posts: 2892
Location: Malmö
Joined: 08/12/04
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #109 - 09/23/10 at 10:33:57
Post Tools
You mean there's something wrong with throwing people in the well and see if they float?
  
Back to top
 
IP Logged
 
TN
God Member
*****
Offline



Posts: 3420
Joined: 11/07/08
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #108 - 09/23/10 at 08:25:27
Post Tools
ChevyBanginStyle was right.  Roll Eyes

  

All our dreams come true if we have the courage to pursue them.
Back to top
 
IP Logged
 
TN
God Member
*****
Offline



Posts: 3420
Joined: 11/07/08
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #107 - 09/23/10 at 07:37:23
Post Tools
Zygalski wrote on 09/23/10 at 07:32:55:
TN wrote on 09/23/10 at 07:28:52:
Here's my view on the entire incident:

a) Yelena did not cheat. 

b) There is no serious evidence whatsoever to prove that she cheated. 

c) Comparing pre-1980 correspondence games or OTB games to current correspondence chess is ludicrous. 

d) Chess.com is the one who has lost in this incident. Kicking out a strong titled player based on an ill-founded suspicion does not encourage titled players to join or stay on chess.com.

How would you attempt to look for possible engine use in online cc?


Comparing Dembo's play to correspondence games from the 2000s instead of the 1960s or 1970s would be an improvement. 

As chess players have become steadily stronger over the years, it shouldn't come as a surprise that the top correspondence players on chess.com are playing better chess than the top correspondence players of the 1960s. The influence of the engine has meant that humans today have much more effective 'computer eyes' than the old masters.
  

All our dreams come true if we have the courage to pursue them.
Back to top
 
IP Logged
 
TN
God Member
*****
Offline



Posts: 3420
Joined: 11/07/08
Gender: Male
Re: Yelena Dembo on Chess.com
Reply #106 - 09/23/10 at 07:28:52
Post Tools
Here's my view on the entire incident:

a) Yelena did not cheat. 

b) There is no serious evidence whatsoever to prove that she cheated. 

c) Comparing pre-1980 correspondence games or OTB games to current correspondence chess is ludicrous. 

d) Chess.com is the one who has lost in this incident. Kicking out a strong titled player based on an ill-founded suspicion does not encourage titled players to join or stay on chess.com.
  

All our dreams come true if we have the courage to pursue them.
Back to top
 
IP Logged
 
ChevyBanginStyle
Full Member
***
Offline


2 \infty & *CRUNCH*

Posts: 238
Joined: 01/03/10
Re: Yelena Dembo on Chess.com
Reply #105 - 09/23/10 at 06:13:58
Post Tools
lol where's a non-cheating GM around to prove your moves are human when you need him? I'm not going to bother to attempt a serious response to something so inane. The chess world does not revolve around chess.com.
  
Back to top
 
IP Logged
 
Smyslov_Fan
God Member
Correspondence fan
*****
Offline


Progress depends on the
unreasonable man. ~GBS

Posts: 6902
Joined: 06/15/05
Re: Yelena Dembo on Chess.com
Reply #104 - 09/23/10 at 04:48:10
Post Tools
Zygalski seems to be deaf to the question that Markovich has repeatedly asked:  What is the statistical model, where are the tests, what are the false-positives, and how is this replicable?   

Not, how is the computer analysis replicable, but how is the determination that player X is a computer and player Y is not a computer replicable?   

You have stated that the top3/top4 method wasn't the deciding factor in chess.com's decision.

A) Why not?

and 

B) What was?  (See Markovich and many others for the follow-up questions regarding the statistical basis for the decision.)
  
Back to top
 
IP Logged
 
ChevyBanginStyle
Full Member
***
Offline


2 \infty & *CRUNCH*

Posts: 238
Joined: 01/03/10
Re: Yelena Dembo on Chess.com
Reply #103 - 09/23/10 at 04:37:34
Post Tools
It's like a broken record with these guys. They repeat the same things even after glaring flaws in their logic have been pointed out (more than once). Good luck with your mission (which appears to have the inevitable result of discrediting yourself).

The irony of all this (at least from my perspective) is that I consider this use of computer engines to be perhaps one of the most abusive. Objections to abuse of statistics to make defaming claims are not trivial. You have been warned.
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1 ... 3 4 [5] 6 7 ... 12
Topic Tools
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo