Latest Updates:
Page Index Toggle Pages: [1] 2 3 
Topic Tools
Very Hot Topic (More than 25 Replies) Guerrillas Gambit Style (Read 2476 times)
TopNotch
God Member
*****
Offline


I only look 1 move ahead,
but its always the best

Posts: 2098
Joined: 01/04/03
Gender: Male
Re: Guerrillas Gambit Style
Reply #40 - 04/04/21 at 00:52:11
Post Tools
Bibs wrote on 04/02/21 at 08:27:03:
Nickajack wrote on 04/02/21 at 07:04:25:
(Since we're already off-topic), I can't help but wonder: what does Radjabov's (self-) exclusion from the Candidates have to do with this (cheating related) matter?


I'm assuming that it is in the sense of a perceived FIDE faux pas.


Thanks for the assist Bibs, I thought the inference would have been obvious, but sometimes I forget that not all forum users are native english speakers.

Not much else to add to the debate for now really, except to say that if being more transparent with quality of evidence means cheaters get a bit more insight on how to better conceal their deceit, then I still consider that to be the lesser of two evils.
  

The man who tries to do something and fails is infinitely better than he who tries to do nothing and succeeds - Lloyd Jones Smiley
Back to top
 
IP Logged
 
Bibs
God Member
*****
Offline



Posts: 2230
Joined: 10/24/06
Re: Guerrillas Gambit Style
Reply #39 - 04/02/21 at 08:27:03
Post Tools
Nickajack wrote on 04/02/21 at 07:04:25:
(Since we're already off-topic), I can't help but wonder: what does Radjabov's (self-) exclusion from the Candidates have to do with this (cheating related) matter?


I'm assuming that it is in the sense of a perceived FIDE faux pas.
  
Back to top
 
IP Logged
 
Nickajack
Junior Member
**
Offline


Yabba Dabba Doo!

Posts: 57
Location: USA
Joined: 02/21/17
Gender: Male
Re: Guerrillas Gambit Style
Reply #38 - 04/02/21 at 07:04:25
Post Tools
(Since we're already off-topic), I can't help but wonder: what does Radjabov's (self-) exclusion from the Candidates have to do with this (cheating related) matter?
  

Dubious, therefore playable Undecided
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1170
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Guerrillas Gambit Style
Reply #37 - 04/01/21 at 14:13:26
Post Tools
Dink Heckler wrote on 04/01/21 at 08:06:02:
The problem with full transparency is that if you clearly lay out the criteria that will lead to an exclusion, it becomes trivially simple for cheaters to cheat in a way that doesn't meet the threshold for exclusion.

Actually, a cheat has enough information *now* to defeat the algorithms for all time. I could trivially write a program that would do that, but I won't say how so as not to help any cheats who don't already know. Of course it would come at a cost to the cheater, in that they would only win some of the time, instead of all the time.

One thing that is militating against the cheaters is there's no real money in it. There's bragging rights and putting accomplishments on a CV and so forth, but reputation is not transferable. So you have a bunch of individuals using their own more or less unsophisticated cheating approaches. They can't share information and "best" practices widely, because there's no monetary incentive for their confidant to keep the secret.

I love how nobody cares how far off-topic this thread has become.
  
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1170
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Guerrillas Gambit Style
Reply #36 - 04/01/21 at 13:40:07
Post Tools
Bibs wrote on 04/01/21 at 01:18:00:
I eventually think I understand his meaning - that it is suspicious in terms of the computer use allegation. Maybe. That it does not fit, as the move played is not optimal.  I guess the software is crying out for Rxa6, but she punted Qg4 instead. I would have played that too.

Isn't it an example of human bias to focus on individual moves like this? Edited:
And I did the same thing in an earlier post.
Chess24's algorithm, or Regan's, or pgn-spy (if indeed these are different), will look at all the moves and weight them equally. From a statistical point of view Rxa6 vs Qg4 is just one data point out of the 174 available, indistinguishable from the others. Of course there can be some sophistication beyond just a binary =(top engine move) true/false, but it couldn't really get to the level of =(type of move an engine would prefer but a human wouldn't). The only way the engine stylistic preference would come into play is second-order, through the evaluation.

hicetnunc wrote on 04/01/21 at 10:02:44:
I've laid out the pgn-spy scores with the few samples I have for rapid play.

Great. I will look at these after work. Google docs is blocked here on the work network, as it should be.

Was thinking about my understanding or lack thereof. A strong player is expected to agree more with the engine. In the article Bibs linked, it's stated (or implied) that Regan corrects the output for this. Does pgn-spy do the same thing? Or does pgn-spy produce a raw engine difference and leave that up to us to interpret based on previous differences measured? Edited:
I realize the google doc probably lays this out, but at some point it should be summarized.


TopNotch wrote on 04/01/21 at 00:48:05:
This has the potential to turn into as much of a fiasco as Radjabov's exclusion from the Candidates was.

Bibs wrote on 04/01/21 at 12:44:31:
If she’s legit (but how to ultimately determine? How much ‘evidence’?) ), a big apology and some nice invites should be forthcoming. Oh dearie me on this kerfuffle.

Taking these two quotes together...

From my point of view it's a boon to get such a clearly unclear case early in the process. One of the biases people have is consistency, so if the fair play process has been perceived to be "working well" for a while, then an individual injustice becomes more tolerable -- in the biased human mind.

It almost doesn't matter if Osmak cheated or not. Of course it matters to her. But to me it matters more that people have doubts that the fair play process can have any hope of being error free. As Dink Henckler said, it's *always* a choice between type 1 or type 2 errors.
  
Back to top
 
IP Logged
 
Bibs
God Member
*****
Offline



Posts: 2230
Joined: 10/24/06
Re: Guerrillas Gambit Style
Reply #35 - 04/01/21 at 12:44:31
Post Tools
That’s a lot of interesting data. Very interesting.
Might be worth noting there that Magsoodloo also got busted.

It’s noted elsewhere (Doggers, chess.com) that she has very limited vision in one eye (16%) and has a habit of seemingly looking away, in order to see the screen properly.

I wonder whether there may be a perfect storm here of unfortunate situations leading to this decision.

I’d hate to be on a panel judging such things - it’s unenviable.

If she’s legit (but how to ultimately determine? How much ‘evidence’?) ), a big apology and some nice invites should be forthcoming. Oh dearie me on this kerfuffle.
  
Back to top
 
IP Logged
 
hicetnunc
Full Member
***
Offline


"Do something scary every
day"

Posts: 221
Location: Paris, France
Joined: 01/04/03
Gender: Male
Re: Guerrillas Gambit Style
Reply #34 - 04/01/21 at 10:02:44
Post Tools
an ordinary chessplayer wrote on 03/31/21 at 22:07:21:
hicetnunc wrote on 03/31/21 at 20:21:55:
You can have a very strong TPR and have engine correlation scores average for your rating or only slightly above what's expected.

Doesn't this imply that some or all of your opponents played badly? In other words they have poor correlation scores for their rating.

If that's not the case then there is something I am not understanding about how the scores are calculated.


I've laid out the pgn-spy scores with the few samples I have for rapid play. I can't draw any conclusions from this (baselines are too slim), but it gives an idea why a detection system based on statistics may have been triggered here.

Line 18 is an example of a (non-suspect) young French player crushing a field of lower-rated opponents (2200 vs. 1700)

https://docs.google.com/spreadsheets/d/1JJvdAg9eQsVbUMWpk5bVWOgJ8q-y6lPEYRViSC3n...
  

48 yo, 1920 elo
Back to top
 
IP Logged
 
Dink Heckler
God Member
*****
Online


Love-Forty

Posts: 797
Joined: 02/01/07
Gender: Male
Re: Guerrillas Gambit Style
Reply #33 - 04/01/21 at 08:06:02
Post Tools
We all desire transparency; the lack thereof is contrary to our sense of natural justice.

The problem with full transparency is that if you clearly lay out the criteria that will lead to an exclusion, it becomes trivially simple for cheaters to cheat in a way that doesn't meet the threshold for exclusion. Again, this puts us in the invidious position that the demands of justice and competitive integrity are at loggerheads.
  

'Am I any good at tactics?'
'Computer says No!'
Back to top
 
IP Logged
 
Bibs
God Member
*****
Offline



Posts: 2230
Joined: 10/24/06
Re: Guerrillas Gambit Style
Reply #32 - 04/01/21 at 01:18:00
Post Tools
Quick note. I know exactly the point you note, and I couldn't get that either for a bit. 'Huh?' I thought.

Hikaru was unfortunately very unclear in his meaning here. That's the problem in thinking on one's feet sometimes - a lack of clarity.

I eventually think I understand his meaning - that it is suspicious in terms of the computer use allegation. Maybe. That it does not fit, as the move played is not optimal.  I guess the software is crying out for Rxa6, but she punted Qg4 instead. I would have played that too.

Yes, the relatively poor play of the losing opponents (no disrespect intended to anyone - we are looking at the winner here) means that the wins generate an unduly high TPR, but without requiring spectacular 2700/2800-like play.

  
Back to top
 
IP Logged
 
TopNotch
God Member
*****
Offline


I only look 1 move ahead,
but its always the best

Posts: 2098
Joined: 01/04/03
Gender: Male
Re: Guerrillas Gambit Style
Reply #31 - 04/01/21 at 00:48:05
Post Tools
an ordinary chessplayer wrote on 03/31/21 at 22:20:21:
Dink Heckler wrote on 03/31/21 at 14:30:57:
These discussions always go around in circles, because the twin goals of due process and sporting integrity are at odds with each other, and can not be readily reconciled.

The only discussion we can sensibly have is whether we tolerate more type one errors or type two errors. There are no ideal answers; only messy compromises with reality (which can be anathema to the chess mind). If we insist on no bans /annulment of results without definitive proof, we accept that online chess as a competitive enterprise is not viable. If we want competitive online chess to be viable, we accept some false positives as the cost of doing business.


hicetnunc wrote on 03/31/21 at 16:26:43:
I think the situation isn't this drastic. I believe it's possible to have reasonable anti-cheating measures by mixing statistics and other systems, provided you give players the benefit of the doubt - ie. you accept some false negatives to avoid false positives.

By tinkering with the pgn-spy tool for many years, I've observed that (except maybe at the very top), human play is still very different from engine play. In rapid, I don't see why it wouldn't be possible to develop a reasonably robust system. There are many variables you can look at (engine correlation scores, blunder rates, complexity of position (Regan), time spent per move). I suspect not enough time/resources have been devoted to the topic yet, but frankly, it shouldn't be more difficult to design an adequate detection system than to develop 3600-playing engines.


Well I thought Dink Heckler's post was spot on. My own view is strongly leaning towards the position: "If we insist on no bans /annulment of results without definitive proof, we accept that online chess as a competitive enterprise is not viable."

I recognize there is a large plurality that has a different view, but I resolve the conflict simply by shunning online competitions. The most I am willing to do is chew on these issues in discussions here, but it's easy to be dispassionate if it will never affect me directly.


I don't think definitive proof should necessarily be the Benchmark, but the evidence should be compelling, transparent and detailed enough to stand scrutiny, and certainly an accused player must have some right of appeal to plead his or her case. 

In my estimation Osmak's opponents played too poorly for a statistical analysis to be convincing, with the possible exception of the King's Indian game I reckon I could have found all the moves she did under the same conditions. There is one point I sort of disagreed with Hikaru on though, and that is in one game where he thought not recapturing a pawn and instead going for a mate in one threat to be a bit suspicious, to me however this seemed like typical human play for Rapid & Blitz.

This case just smells of over reach and while I acknowledge that electronic cheating is a vexing problem that threatens the veracity of competitive chess, we still must take care not to throw out the baby with the bath water.

This has the potential to turn into as much of a fiasco as Radjabov's exclusion from the Candidates was.


« Last Edit: 04/01/21 at 18:06:47 by TopNotch »  

The man who tries to do something and fails is infinitely better than he who tries to do nothing and succeeds - Lloyd Jones Smiley
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1170
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Guerrillas Gambit Style
Reply #30 - 03/31/21 at 22:20:21
Post Tools
Dink Heckler wrote on 03/31/21 at 14:30:57:
These discussions always go around in circles, because the twin goals of due process and sporting integrity are at odds with each other, and can not be readily reconciled.

The only discussion we can sensibly have is whether we tolerate more type one errors or type two errors. There are no ideal answers; only messy compromises with reality (which can be anathema to the chess mind). If we insist on no bans /annulment of results without definitive proof, we accept that online chess as a competitive enterprise is not viable. If we want competitive online chess to be viable, we accept some false positives as the cost of doing business.


hicetnunc wrote on 03/31/21 at 16:26:43:
I think the situation isn't this drastic. I believe it's possible to have reasonable anti-cheating measures by mixing statistics and other systems, provided you give players the benefit of the doubt - ie. you accept some false negatives to avoid false positives.

By tinkering with the pgn-spy tool for many years, I've observed that (except maybe at the very top), human play is still very different from engine play. In rapid, I don't see why it wouldn't be possible to develop a reasonably robust system. There are many variables you can look at (engine correlation scores, blunder rates, complexity of position (Regan), time spent per move). I suspect not enough time/resources have been devoted to the topic yet, but frankly, it shouldn't be more difficult to design an adequate detection system than to develop 3600-playing engines.


Well I thought Dink Heckler's post was spot on. My own view is strongly leaning towards the position: "If we insist on no bans /annulment of results without definitive proof, we accept that online chess as a competitive enterprise is not viable."

I recognize there is a large plurality that has a different view, but I resolve the conflict simply by shunning online competitions. The most I am willing to do is chew on these issues in discussions here, but it's easy to be dispassionate if it will never affect me directly.
  
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1170
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Guerrillas Gambit Style
Reply #29 - 03/31/21 at 22:07:21
Post Tools
hicetnunc wrote on 03/31/21 at 20:21:55:
You can have a very strong TPR and have engine correlation scores average for your rating or only slightly above what's expected.

Doesn't this imply that some or all of your opponents played badly? In other words they have poor correlation scores for their rating.

If that's not the case then there is something I am not understanding about how the scores are calculated.
  
Back to top
 
IP Logged
 
hicetnunc
Full Member
***
Offline


"Do something scary every
day"

Posts: 221
Location: Paris, France
Joined: 01/04/03
Gender: Male
Re: Guerrillas Gambit Style
Reply #28 - 03/31/21 at 20:21:55
Post Tools
an ordinary chessplayer wrote on 03/29/21 at 22:46:33:
(...)

To be clear, when I played through Osmak's games I could plainly see she played way above her 2400 rating. hicetnunc's analysis confirms it. But to me it's not a red flag. How many players were in the event? Not just counting her 6-player group, but counting all players. In a group that large, the distribution of net-Elo performances will easily be from -400 to +400. Of course most will perform close to their published Elo, but even without any real improvement *somebody* is bound to have an outlandish performance; it just happened to be her. That's one explanation. The fair play people gave a different explanation.


Just to clarify, I meant that the scores output of PGN-spy is similar to what you would expect from a 2750 in a long game, which is different from having a +300/+400 perf. (although in this case both happened). You can have a very strong TPR and have engine correlation scores average for your rating or only slightly above what's expected. That's why there's cause for concern here.

Having strong TPR happens, but having both a strong TPR and very high engine correlation scores at the same time must be less frequent.
  

48 yo, 1920 elo
Back to top
 
IP Logged
 
hicetnunc
Full Member
***
Offline


"Do something scary every
day"

Posts: 221
Location: Paris, France
Joined: 01/04/03
Gender: Male
Re: Guerrillas Gambit Style
Reply #27 - 03/31/21 at 16:26:43
Post Tools
Dink Heckler wrote on 03/31/21 at 14:30:57:
These discussions always go around in circles, because the twin goals of due process and sporting integrity are at odds with each other, and can not be readily reconciled.

The only discussion we can sensibly have is whether we tolerate more type one errors or type two errors. There are no ideal answers; only messy compromises with reality (which can be anathema to the chess mind). If we insist on no bans /annulment of results without definitive proof, we accept that online chess as a competitive enterprise is not viable. If we want competitive online chess to be viable, we accept some false positives as the cost of doing business.


I think the situation isn't this drastic. I believe it's possible to have reasonable anti-cheating measures by mixing statistics and other systems, provided you give players the benefit of the doubt - ie. you accept some false negatives to avoid false positives.

By tinkering with the pgn-spy tool for many years, I've observed that (except maybe at the very top), human play is still very different from engine play. In rapid, I don't see why it wouldn't be possible to develop a reasonably robust system. There are many variables you can look at (engine correlation scores, blunder rates, complexity of position (Regan), time spent per move). I suspect not enough time/resources have been devoted to the topic yet, but frankly, it shouldn't be more difficult to design an adequate detection system than to develop 3600-playing engines.
  

48 yo, 1920 elo
Back to top
 
IP Logged
 
Dink Heckler
God Member
*****
Online


Love-Forty

Posts: 797
Joined: 02/01/07
Gender: Male
Re: Guerrillas Gambit Style
Reply #26 - 03/31/21 at 14:30:57
Post Tools
These discussions always go around in circles, because the twin goals of due process and sporting integrity are at odds with each other, and can not be readily reconciled.

The only discussion we can sensibly have is whether we tolerate more type one errors or type two errors. There are no ideal answers; only messy compromises with reality (which can be anathema to the chess mind). If we insist on no bans /annulment of results without definitive proof, we accept that online chess as a competitive enterprise is not viable. If we want competitive online chess to be viable, we accept some false positives as the cost of doing business.
  

'Am I any good at tactics?'
'Computer says No!'
Back to top
 
IP Logged
 
Page Index Toggle Pages: [1] 2 3 
Topic Tools
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo