Page Index Toggle Pages: 1 2 3 [4] 5 6 ... 12
Topic Tools
Very Hot Topic (More than 25 Replies) Yelena Dembo on Chess.com (Read 99709 times)
Schaakhamster
God Member
*****
Offline


I Love ChessPublishing!

Posts: 650
Joined: 05/13/08
Re: Yelena Dembo on Chess.com
Reply #132 - 09/23/10 at 18:46:40
Post Tools
Zygalski wrote on 09/23/10 at 18:38:54:
Well at least no-one can say I haven't done any analysis!

No-one here can honestly say they can't see any discrepancy between the 2 or 3-year event CC World Championships & an IM playing online chess moving daily with between 10-20 games in progress?

Honestly...
Look at the stats.
Come on!  Roll Eyes


They are meaningless because the way you have constructed them are unclear. Furthermore statistics have proven methods to test a hypothesis. 

I think it is possible to construct a viable method but it would require a lot more statistical background and a good program that would use a standardized method to generate the required data. I think this would be a nice project for a doctorate. It would require quite a bit of work though.     
  
Back to top
 
IP Logged
 
Schaakhamster
God Member
*****
Offline


I Love ChessPublishing!

Posts: 650
Joined: 05/13/08
Re: Yelena Dembo on Chess.com
Reply #131 - 09/23/10 at 18:35:52
Post Tools
Flooding the thread proofs that you are very industrious not that you are right.
  
Back to top
 
IP Logged
 
Schaakhamster
God Member
*****
Offline


I Love ChessPublishing!

Posts: 650
Joined: 05/13/08
Re: Yelena Dembo on Chess.com
Reply #130 - 09/23/10 at 18:33:18
Post Tools
Zygalski wrote on 09/23/10 at 18:14:06:
You want me to publish all my benchmarks with ply-by-ply analysis here?
What are this site's servers like?  Smiley


No he want to see a sound statistical approach which is quite commonly used.

Otherwise you are just a drone with a computer using a black box device which just spits out a number you don't really understand based on other numbers you feed into it. How many numbers you feed into doesn't mater unless the black box device makes statistical sense. 

As my statistics prof used to sat: "there are lies, damned lies and statistics"  Wink   

  
Back to top
 
IP Logged
 
Markovich
God Member
*****
Offline



Posts: 6099
Location: Columbus, Ohio
Joined: 09/17/04
Re: Yelena Dembo on Chess.com
Reply #129 - 09/23/10 at 18:01:33
Post Tools
Zygalski wrote on 09/23/10 at 17:50:44:
I spent part of the last 3 years creating benchmarks.
How long did you spend?
Ideally yes, you'd of course have as many pertinent benchmarks as possible.
My benchmarks matched those of RHP games mods & a colleague at chess.com has also created his own & again, this remarkable consistency in unassisted game match up rates occurs.


Your benchmarks are of unknown provenance.  The method of their estimation is unknown.  If they are estimates, they are subject to uncertainty, and they fall within given confidence intervals.  Further a set of matchup stats produced from a 20-game sample is itself subject to random error.  Put these two sources of uncertainty together in an analytical framework, and you get a degree of confidence with which you can reject the null hypothesis, that someone is playing unassisted.  I don't need to have worked on this problem to know that this is true.  

You do not appear to have understood this or analyzed it.  Go back, do the analysis, best with the help of someone with a stat background, and then I will say you have what seems to be a competent analysis.  Until then, I'll say the opposite.  

I do not, by the way, reject your claim that machines play differently than humans, or that it might be possible to tell, with some degree of uncertainty, whether an engine or a human is playing.  I would not have objected at all if you used Tarot cards, tea leaves or "Method X" to discontinue people's accounts on chess.com.  What I object to is calling people cheaters based on uncertain, apparently dubious methods.
  

The Great Oz has spoken!
Back to top
 
IP Logged
 
Markovich
God Member
*****
Offline



Posts: 6099
Location: Columbus, Ohio
Joined: 09/17/04
Re: Yelena Dembo on Chess.com
Reply #128 - 09/23/10 at 17:48:28
Post Tools
Zygalski wrote on 09/23/10 at 17:25:59:
Markovich wrote on 09/23/10 at 16:55:40:
Zygalski wrote on 09/23/10 at 15:47:52:
Smyslov:
Yes you're quite right, in a single or a few games the match up rates can be very high indeed. Unasissted players simply cannot seem to sustain this engine-like play in many games over time though.
I've yet to see benchmarks from any player which exceed the 60/75/85% for more than 20 games with 500+ non-database moves.  Another member of chess.com has also generated many top-quality player benchmarks & had virtually identical findings to mine.
If I were to analyse an unasissted player & find much higher match rates than the thresholds in 20 objectively chosen games with 500+ non-database moves, I would wonder if yes, some of the lower matching players who've been banned may be innocent!

60/75/85% for top 1/2/3 (all 3 figures=/+) and I would submit & maybe expect a ban, maybe not.
65/80/90% (all 3 figures=/+) and I would expect that player to be banned.


Start with a sample set that includes all available corr games...

What- from all years?
Even ones where computers/engines have no doubt been used.  What will that prove exactly?

I'm not spending the next 10 years generating useless benchmarks for someone who has no understanding of what they're talking about & is simply an internet troll.
Sorry! Wink


No do you see, I said "pre-computer?"  

Markovich wrote on 09/23/10 at 16:55:40:


Start with a sample set that includes all available corr games and only games played pre-computer by players of some minimum strength.  Do not include any OTB games.  Calculate matchup percentages for both players.  Be careful to apply an objective criterion of what constitutes a move elligible for consideration, and apply the same criterion to each move.  Be careful to use the same engine and engine settings across all games. 



As to whether I know what I am talking about, I have repeatedly said that not enough of chess.com's method's is public to enable anyone to know if they know what they are doing.  So to that extent, no, I don't know about these methods.

But I modestly do assert that I understand something about statistics; I buttered my bread pretty well with it for 30 years, anyway.  I have not seen from you or from chess.com's other defenders here anything resembling a appreciation for the statistical aspects of this problem, which are fundamental.  Instead I have just seen people pointing at numbers and saying "See?"

I don't think it would take 10 years to develop an estimate of the distribution of computer mathup percentage, following the methods I outlined.  I think it would take about two months.  But someone would have to write a Python script so as to interract with the engine and the game set, as well possibly with a db so as to decide which moves are worthy of being considered.  Or you could just take all moves between the 20th and the appearance of less than seven pieces, as a pretty good shortcut, I would think.

In any case, that a proper analysis would consume time is hardly an argument against it.  A proper analysis would take too much time, so let us accuse people of cheating based on an improper analysis? 

@drogo:  I don't think it is especially idiotic for some well-meaning people to have tried to draw conclusions without a proper statistical analysis.  I do think it's idiotic to persist in this error, having been informed of it.  I don't live in a world surrounded by idiots, just in a world where those defending chess.com here appear to be idiots.
  

The Great Oz has spoken!
Back to top
 
IP Logged
 
drogo
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 40
Joined: 09/21/10
Re: Yelena Dembo on Chess.com
Reply #127 - 09/23/10 at 17:39:57
Post Tools
Markovich wrote on 09/23/10 at 17:03:04:


You have said that Yelena Dembo is cheating, drogo, you just haven't been honest enough to say it in four short words. 

I suppose I would seem to be insane to you, since I seem to be talking very far over your head.


1. I only said that Yelena Dembo appears to play CC games more computer-like than any known human. Of course, that doesn't mean she cheated.

2. Have fun in your fantasy world, where you are the only smart person surrounded by idiots!
  
Back to top
 
IP Logged
 
Markovich
God Member
*****
Offline



Posts: 6099
Location: Columbus, Ohio
Joined: 09/17/04
Re: Yelena Dembo on Chess.com
Reply #126 - 09/23/10 at 17:03:04
Post Tools
drogo wrote on 09/23/10 at 16:38:46:
Markovich, there is no point for me to reply. If you really cannot understand that I NEVER SAID YELENA DEMBO IS CHEATING, there is nothing to discuss with you. You cannot understand plain English and you are talking about statistics? It's like a 4-year kid would talk about quantum mechanics.

You are totally insane!


You have said that Yelena Dembo is cheating, drogo, you just haven't been honest enough to say it in four short words.  I've been talking about statistics because the problem of trying to identify those who rely upon computers is statistical, which is something that you and the arbiters at chess.com don't appear to understand.

I suppose I would seem to be insane to you, since I seem to be talking very far over your head.
  

The Great Oz has spoken!
Back to top
 
IP Logged
 
Markovich
God Member
*****
Offline



Posts: 6099
Location: Columbus, Ohio
Joined: 09/17/04
Re: Yelena Dembo on Chess.com
Reply #125 - 09/23/10 at 16:55:40
Post Tools
Zygalski wrote on 09/23/10 at 15:47:52:
Smyslov:
Yes you're quite right, in a single or a few games the match up rates can be very high indeed. Unasissted players simply cannot seem to sustain this engine-like play in many games over time though.
I've yet to see benchmarks from any player which exceed the 60/75/85% for more than 20 games with 500+ non-database moves.  Another member of chess.com has also generated many top-quality player benchmarks & had virtually identical findings to mine.
If I were to analyse an unasissted player & find much higher match rates than the thresholds in 20 objectively chosen games with 500+ non-database moves, I would wonder if yes, some of the lower matching players who've been banned may be innocent!

60/75/85% for top 1/2/3 (all 3 figures=/+) and I would submit & maybe expect a ban, maybe not.
65/80/90% (all 3 figures=/+) and I would expect that player to be banned.


Start with a sample set that includes all available corr games and only games played pre-computer by players of some minimum strength.  Do not include any OTB games.  Calculate matchup percentages for both players.  Be careful to apply an objective criterion of what constitutes a move elligible for consideration, and apply the same criterion to each move.  Be careful to use the same engine and engine settings across all games.  

From the games the test set, select a large number of sets of 20 games (say 2,000), not necessarily played by the same player, and in each randomly select Black's or White's performance.  Sample randomly and with replacement.  Then publish the percentiles of % matchups from these 20-game sets.

Actually I would probably conduct three such analyses, separately by win, loss and draw, since I would expect a different standand of play across that dimension.  If I did not obtain significantly different results across that dimension then I would just pool across it for a final estimate.

That, I would consider to be a proper method of estimating the distribution of % matchup across 20 games.  Publish the estimated percentiles along with the criteria for elligibility of a move and the engine and settings, and we shall see what we shall see.  

You think that's too much work to do before you publicly call someone a cheater?  Rather not share your methods that much?  Then stop calling people cheaters.
  

The Great Oz has spoken!
Back to top
 
IP Logged
 
Smyslov_Fan
God Member
Correspondence fan
*****
Offline


Progress depends on the
unreasonable man. ~GBS

Posts: 6902
Joined: 06/15/05
Re: Yelena Dembo on Chess.com
Reply #124 - 09/23/10 at 16:53:08
Post Tools
I hope that drogo and zygalski check out some of the wonderful analysis available while they visit the chess pub. 

I do fear they will say that this analysis is not allowed at chess.com because much of it has been verified by computers! 

(Remember of course that many of the denizens of the pub are actually published chess authors!)
  
Back to top
 
IP Logged
 
drogo
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 40
Joined: 09/21/10
Re: Yelena Dembo on Chess.com
Reply #123 - 09/23/10 at 16:38:46
Post Tools
Markovich, there is no point for me to reply. If you really cannot understand that I NEVER SAID YELENA DEMBO IS CHEATING, there is nothing to discuss with you. You cannot understand plain English and you are talking about statistics? It's like a 4-year kid would talk about quantum mechanics.

You are totally insane!
  
Back to top
 
IP Logged
 
Markovich
God Member
*****
Offline



Posts: 6099
Location: Columbus, Ohio
Joined: 09/17/04
Re: Yelena Dembo on Chess.com
Reply #122 - 09/23/10 at 16:36:17
Post Tools
Zygalski wrote on 09/23/10 at 15:53:54:
Smyslov_Fan wrote on 09/23/10 at 15:44:06:
Yelena Dembo's first round game in the Olympiad supposedly had something like a 91% correlation to your top three method.

One of the team thought to analyse Dembo's OTB games;
"I picked the last 20 games >= 35 moves from chessgames.com. All against high-rated opposition (lowest ELO in the set is 2222, highest is 2655)"
thinking that perhaps Dembo's natural style of unassisted play is rather more engine-like than should be expected.  The results of the analysis were:
Dembo OTB games:
Stockfish 1.8, 512MB hash, min/max ply=12/30, 40s/ply, 2GHz Core Duo:
{ Yelena Dembo (Games: 20) }
{ Top 1 Match: 422/929 ( 45.4% )
{ Top 2 Match: 615/929 ( 66.2% )
{ Top 3 Match: 717/929 ( 77.2% )
{ Top 4 Match: 777/929 ( 83.6% )


So what?  She plays differently in CC than in OTB.  Who doesn't?

You don't have a test, you know?  You don't have a statistic or a confidence interval.  Bothering to treat the problem in a proper way would, of course, force you to admit that some probability exists that a "known" cheater is not actually a cheater.  But it would be much more convincing that what you and the other defenders of chess.com have presented here, because it would demonstrate something replicable and demonstrate an objective standard.

Have you considered this question:  what if someone looked at exactly the same numbers you've produced and said, "Nope, clearly she's not cheating."  You have no answer, because you have a set of numbers of unknown provenance, you have a conclusion, and you have no statistical analysis that mediates between the two.  So it really comes down to "trust us, we know."  Which isn't good enough in accusations of cheating.
  

The Great Oz has spoken!
Back to top
 
IP Logged
 
Markovich
God Member
*****
Offline



Posts: 6099
Location: Columbus, Ohio
Joined: 09/17/04
Re: Yelena Dembo on Chess.com
Reply #121 - 09/23/10 at 16:27:36
Post Tools
drogo wrote on 09/23/10 at 03:01:18:
Wow Markovich, this is indeed smart! So, if I want to cheat, all I have to do is make one single move that is not the top choice of the engine. 

...

At move 35 a human can see that Black has a won game and should simplify as much as possible (thus exchanging queens). I don't think that, in my original analysis, I included the statistics for moves after 34.


Right, and of course, Dembo is a witch.  I haven't seen her cast a spell, but why would a witch be stupid enough to cast a spell right before the eyes of an expert - witch judge like me.

drogo wrote on 09/23/10 at 03:01:18:

The method is certainly replicable: just get yourself an engine and start analyzing games! Shall I make a drawing for you to understand or you admit that you are a fool? To use your very friendly language.


Yes, and my witch-detection method is also replicable.  All you have to do is get yourself a pair of blue spectacles and a switch of willow cut from the banks of the Olentangy River.  Precisely how to use them?  My secret!  It would never do if the witches found out!!

Do you know what "replicable" means?  It means someone else can apply your method to the same or similar data and reach the same conclusion.  Your method is a secret, you know?  Your "thresholds" have no provenance, but are merely baldly asserted here.  They have no derivation; they depend on no clear set of data; the estimation method is unknown; their dependence on radomness in the sample is unknown and is not even acknowledged; their dependence on the engine used and its setup is unknown and not acknowledged.

And upon this basis, you come here and say this IM is a cheater.  And no, on that basis, I really do not use friendly language with someone who really deserves to be horse whipped.
  

The Great Oz has spoken!
Back to top
 
IP Logged
 
drogo
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 40
Joined: 09/21/10
Re: Yelena Dembo on Chess.com
Reply #120 - 09/23/10 at 15:56:15
Post Tools
Smyslov_Fan wrote on 09/23/10 at 15:44:06:
Yelena Dembo's first round game in the Olympiad supposedly had something like a 91% correlation to your top three method.


Is this the game?

[Event "Chess Olympiad (Women)"]
[Site "Khanty-Mansiysk RUS"]
[Date "2010.09.21"]
[EventDate "2010.09.21"]
[Round "1"]
[Result "0-1"]
[White "Choe Tammy Segarra"]
[Black "Yelena Dembo"]
[ECO "E94"]
[WhiteElo "2014"]
[BlackElo "2452"]
[PlyCount "76"]

1. d4 Nf6 2. c4 g6 3. Nc3 Bg7 4. e4 d6 5. Nf3 O-O 6. Be2 Nbd7 7. O-O e5 8. d5 Nh5 9. Ne1 Nf4 10. Be3 Nxe2+ 11. Qxe2 f5 12. f3 f4 13. Bf2 g5 14. Nd3 Nf6 15. c5 g4 16. Bh4 Qe8 17. Bxf6 gxf3 18. Qxf3 Rxf6 19. cxd6 cxd6 20. Rac1 Rg6 21. Nf2 Bf6 22. Ne2 h5 23. h3 Rg7 24. Rc3 Bd8 25. Rfc1 Bd7 26. Kh2 h4 27. R1c2 Ba5 28. b4 Bxb4 29. Rc7 Bc5 30. Ng4 Bxg4 31. Rxg7+ Kxg7 32. Qxg4+ Qg6 33. Qxh4 Kf7 34. Nxf4 exf4 35. Qxf4+ Kg7 36. Rc1 Rf8 37. Qh4 Rf7 38. e5 Bf2 0-1

I guess it would be interesting to analyze. Maybe Yelena Dembo naturally plays like an engine.
  
Back to top
 
IP Logged
 
Smyslov_Fan
God Member
Correspondence fan
*****
Offline


Progress depends on the
unreasonable man. ~GBS

Posts: 6902
Joined: 06/15/05
Re: Yelena Dembo on Chess.com
Reply #119 - 09/23/10 at 15:44:06
Post Tools
Yelena Dembo's first round game in the Olympiad supposedly had something like a 91% correlation to your top three method.
  
Back to top
 
IP Logged
 
drogo
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 40
Joined: 09/21/10
Re: Yelena Dembo on Chess.com
Reply #118 - 09/23/10 at 15:32:04
Post Tools
Uh, I'm not from chess.com.

Humans who passed the 65/75/85 threshold? Games please!
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1 2 3 [4] 5 6 ... 12
Topic Tools
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo