Latest Updates:
Page Index Toggle Pages: 1 [2] 3 
Topic Tools
Very Hot Topic (More than 25 Replies) Fide approves online chess regulations (Read 9126 times)
ReneDescartes
God Member
*****
Offline


Qu'est-ce donc que je
suis? Une chose qui pense.

Posts: 1236
Joined: 05/17/10
Gender: Male
Re: Fide approves online chess regulations
Reply #22 - 01/12/21 at 13:53:35
Post Tools
My 1,000 games example was meant to address this suggestion:

an ordinary chessplayer wrote on 01/08/21 at 02:43:21:
This is entirely aside from whether it's even possible to detect engine use.
https://www.researchgate.net/publication/267275282_On_the_Limits_of_Engine_Analy...

and this:

an ordinary chessplayer wrote on 08/27/20 at 04:09:59:
(...) People get mesmerized by the large number of moves fed into the algorithm, and start to believe that the laws of large numbers apply. It's true in the computation of the algorithm sense, but in the legal sense it boils down to a single fact. And it's an indirect fact.

It's not a murder case. Chess,  like language, is abstract. Much closer to the case of cheating in chess is that of  plagiarizing passages of a book in a paper. The resemblance between the paper and the suspected source is also "a single indirect fact." No one saw the student using the source, the ink from the book is not on the student's hands, and the student didn't tell anyone he was cheating.  We just have a single fact--that parts of the paper closely resemble another piece of writing. A lot of matches, in other words. Furthermore, they're not always exact matches, a lot of the paper does not appear plagiarized, and the student is intelligent enough to have written the disputed phrases considered one at a time. Yet we're comfortable enough using this ex post facto evidence, sometimes without material corroboration. All those matches a single indirect fact? Ok, but it's one hell of a fact.

Regarding the number of moves needed to make a judgment, the man in the street may misunderstand, but he will not be making policy or decisions for major platforms or FIDE. Statistical evidence is fine; it just has to be interpreted thoughtfully by scientifically competent people, like other technical evidence.

Regarding the scientific study cited, it's a lost cause. It's not a confirmation of general relativity: no one, but no one, publishes a paper as an example of a routine statistical effect--and even if they did, they would certainly mention it in the analysis! The authors just embarrassed themselves, plain and simple. The best one can say is that their idiocy illustrates how someone else could go wrong, too, drawing conclusions from too-small samples. But we don't have to choose between 40 moves 40,000--we can quantify the effect of the sample size and weigh this evidence alongside other evidence.

« Last Edit: 01/12/21 at 20:58:41 by ReneDescartes »  
Back to top
 
IP Logged
 
trw
YaBB Moderator
*****
Offline


I Love ChessPublishing!

Posts: 1413
Joined: 05/06/08
Gender: Male
Re: Fide approves online chess regulations
Reply #21 - 01/11/21 at 06:46:00
Post Tools
Nickajack wrote on 01/10/21 at 04:25:31:
an ordinary chessplayer wrote on 01/10/21 at 04:16:57:
Nickajack wrote on 01/10/21 at 00:29:16:
... smart statistics could reveal an excessive variance between the quality of moves, IF the cheater isn't that strong.

Excessive variance between the quality of the moves is the hallmark of the weaker player.



Yes, but I'm writing about mixing GM-level moves with lots of weak(er) moves, which is not the hallmark of weaker players.



It absolutely is my hallmark... I had a GM coach for years say, "How do find these moves even a 2800 would struggle to find then the next move find play something even a 1000 player wouldn't consider?"
  
Back to top
 
IP Logged
 
MaxJudd
Full Member
***
Offline


I Love ChessPublishing!

Posts: 105
Joined: 12/09/09
Re: Fide approves online chess regulations
Reply #20 - 01/10/21 at 20:11:22
Post Tools
an ordinary chessplayer wrote on 01/09/21 at 22:53:58:
@MaxJudd - I agree with every single point you made, except this one: "for blitz it doesn't matter if I lose to a computer cheater". I had someone using his phone against me in OTB blitz, I don't know how he thought he was getting away with it, but it most definitely did matter to me. But if we modify to say "for blitz it matters less" then I will agree. E.g., he got off lightly because it was unrated blitz. By the way, his phone was pretty good at blitz! It's not like the old days of tabletop computer chess.


That is a completely fair point.  What I meant is that for me (recognizing others will not necessarily share this view) I feel much less invested in the outcome of online blitz games vs OTB  games or longer online games.  That makes it much easier to not get upset when something bad happens like a mouse slip or blatant cheating.  Likewise, if I was wrongly banned by an online platform based on their cheating algorithm, I probably wouldn't get too upset either.
  
Back to top
 
IP Logged
 
Nickajack
Junior Member
**
Offline


Yabba Dabba Doo!

Posts: 57
Location: USA
Joined: 02/21/17
Gender: Male
Re: Fide approves online chess regulations
Reply #19 - 01/10/21 at 04:25:31
Post Tools
an ordinary chessplayer wrote on 01/10/21 at 04:16:57:
Nickajack wrote on 01/10/21 at 00:29:16:
... smart statistics could reveal an excessive variance between the quality of moves, IF the cheater isn't that strong.

Excessive variance between the quality of the moves is the hallmark of the weaker player.



Yes, but I'm writing about mixing GM-level moves with lots of weak(er) moves, which is not the hallmark of weaker players.
  

Dubious, therefore playable Undecided
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1667
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Fide approves online chess regulations
Reply #18 - 01/10/21 at 04:16:57
Post Tools
Nickajack wrote on 01/10/21 at 00:29:16:
... smart statistics could reveal an excessive variance between the quality of moves, IF the cheater isn't that strong.

Excessive variance between the quality of the moves is the hallmark of the weaker player.
  
Back to top
 
IP Logged
 
Nickajack
Junior Member
**
Offline


Yabba Dabba Doo!

Posts: 57
Location: USA
Joined: 02/21/17
Gender: Male
Re: Fide approves online chess regulations
Reply #17 - 01/10/21 at 00:29:16
Post Tools
an ordinary chessplayer wrote on 01/09/21 at 22:27:47:
Far from being naive, the paper is a routine demonstration (a lot of science is only that), and just provides some hard numbers that may be persuasive for non-statisticians. If it gets some of them to stop squawking "cheat" whenever someone plays a good game against them, it would be useful.

"No statistical power" was my own language, not that of the study authors. I agree with both of you, but by "no statistical power" I meant in comparison to human intuition about what is possible. I submit that most people (but no statisticians) intuit the moves of a single game are enough to tell if a player has been cheating. In fact isn't this where all (or almost all) the online accusations come from? A single game, perhaps two games at most. Whereas we probably couldn't draw any statistical conclusions from a nine-round tournament, yet any human could be convinced beyond a reasonable doubt by such a sample.

By the way, 40,000 moves is approximately 1,000 games. If a player used an engine for that many games, their rating would by then be the same as the engine's. So if we are trying to detect cheating by someone whose current rating is not at engine-level, it seems we need to be looking at fewer than 40,000 moves. How many moves of Petrosian did chess.com examine before defaulting him and his team? He didn't play 40,000 moves in the event, more in the 400-800 range.


Another problem is posed by the selective cheaters, those who don't cheat all the time, but even in those cases smart statistics could reveal an excessive variance between the quality of moves, IF the cheater isn't that strong. A titled player who cheats selectively would be the hardest to catch.

The intrusive search perhaps should only be required for those players which the statistical methods identify as highly suspect. This alone could deter most wannabe cheaters.

It would be wrong and counterproductive to treat everyone like a criminal, especially while such draconian measures can allow for more sophisticated methods of cheating to occur anyway.
  

Dubious, therefore playable Undecided
Back to top
 
IP Logged
 
cathexis
God Member
*****
Offline


No matter where you go,
there you are.

Posts: 599
Joined: 03/03/20
Re: Fide approves online chess regulations
Reply #16 - 01/10/21 at 00:20:40
Post Tools
I don't understand about the venues of play - Are we talking rented halls, etc.? As now happens in some "Pro-Gamer" competitions? Or just me in my basement with Zoom setup to show my basement area so the Arbiter can "look around"?
'Cause then -

This will be a real boon for deaf players; Now they can just have their BFF signing the engine results off-camera and there's nothing for the Arbiter to see or hear.  Roll Eyes
  
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1667
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Fide approves online chess regulations
Reply #15 - 01/09/21 at 22:53:58
Post Tools
@MaxJudd - I agree with every single point you made, except this one: "for blitz it doesn't matter if I lose to a computer cheater". I had someone using his phone against me in OTB blitz, I don't know how he thought he was getting away with it, but it most definitely did matter to me. But if we modify to say "for blitz it matters less" then I will agree. E.g., he got off lightly because it was unrated blitz. By the way, his phone was pretty good at blitz! It's not like the old days of tabletop computer chess.
  
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1667
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Fide approves online chess regulations
Reply #14 - 01/09/21 at 22:27:47
Post Tools
Far from being naive, the paper is a routine demonstration (a lot of science is only that), and just provides some hard numbers that may be persuasive for non-statisticians. If it gets some of them to stop squawking "cheat" whenever someone plays a good game against them, it would be useful.

"No statistical power" was my own language, not that of the study authors. I agree with both of you, but by "no statistical power" I meant in comparison to human intuition about what is possible. I submit that most people (but no statisticians) intuit the moves of a single game are enough to tell if a player has been cheating. In fact isn't this where all (or almost all) the online accusations come from? A single game, perhaps two games at most. Whereas we probably couldn't draw any statistical conclusions from a nine-round tournament, yet any human could be convinced beyond a reasonable doubt by such a sample.

By the way, 40,000 moves is approximately 1,000 games. If a player used an engine for that many games, their rating would by then be the same as the engine's. So if we are trying to detect cheating by someone whose current rating is not at engine-level, it seems we need to be looking at fewer than 40,000 moves. How many moves of Petrosian did chess.com examine before defaulting him and his team? He didn't play 40,000 moves in the event, more in the 400-800 range.
  
Back to top
 
IP Logged
 
MaxJudd
Full Member
***
Offline


I Love ChessPublishing!

Posts: 105
Joined: 12/09/09
Re: Fide approves online chess regulations
Reply #13 - 01/09/21 at 22:14:24
Post Tools
Since some cheating seems inevitable with online play, you need to simply live with the risk of both false accusations and some cheaters slipping through. 

Limiting online people to your own circle or chess club is not a guarantee of clean play but certainly makes cheating less likely.  On the other hand, not everyone has access to enough local players to take my approach (e.g., a junior player in a rural area) and one of the things that is fun about online play is the chance to play people around the world.  Personally, I'd rather play people I know online in serious or semi-serious games but for blitz it doesn't matter if I lose to a computer cheater.

Another possibility that has been mentioned and implemented professionally is to supplement computer-aided detection with multiple webcams focused on each player.  Certainly that makes cheating detection easier but it isn't failsafe and in any event is much too intrusive and cumbersome for 99% of amateur play in my opinion.

It is tough to fault FIDE or one of the national associations in at least trying to come up with workable rules.
  
Back to top
 
IP Logged
 
ReneDescartes
God Member
*****
Offline


Qu'est-ce donc que je
suis? Une chose qui pense.

Posts: 1236
Joined: 05/17/10
Gender: Male
Re: Fide approves online chess regulations
Reply #12 - 01/09/21 at 19:22:47
Post Tools
I looked through the article quickly. It seems mathematically naive to me. The authors look at the distribution of computer matches to individual games, and it's no surprise that this shows a high degree of variability with many false positives. But that is due to an extremely fundamental statistical effect. If you take many very small samples  from a normally-distributed population (here a "population" of computer-matches for moves, where the small samples are individual games) and then look at the mean values of those samples, you will find they vary all over the place, so that one sample gives little idea of the true mean of the population. But if you look at very large samples from that same population , you will find the means are much, much steadier. That is just fundamental statistics, whether you're talking about arm-wrestling skill, one-second temperatures, computer move matches, or anything normally distributed whatsoever. The exact quantification of this effect is one thing you would learn in a good undergraduate statistics course. Admittedly, it's not that intuitive if you haven't studied statistics in earnest. But I would have expected the authors of such an article to know all this. Their article is embarassing, since it is little more than a demonstration of this elementary effect.

Engine matching may not be perfect, but if it is looking at a decent number of moves, not just individual games, it detects quite easily the gross difference between a Fischer and an ordinary GM, between an ordinary GM and a master, etc.,. Regan's work on intrinsic performance rating is one illustration of that. Guid and Bratko also have a study showing that, again, over a large number of moves, differences in the engine depth used don't really matter that much (I suppose because humans make relatively gross blunders).

If a player of whom no one has heard produces far better computer matches on the average than does Capablanca, far better than does Smyslov, far better than does Fischer-- over the course of 40,000 moves--it's beyond a reasonable doubt that he is using an engine.

Edit: I see hictenunc beat me to it.
« Last Edit: 01/09/21 at 21:06:13 by ReneDescartes »  
Back to top
 
IP Logged
 
hicetnunc
Full Member
***
Offline


"Do something scary every
day"

Posts: 233
Location: Paris, France
Joined: 01/04/03
Gender: Male
Re: Fide approves online chess regulations
Reply #11 - 01/09/21 at 18:44:06
Post Tools
an ordinary chessplayer wrote on 01/09/21 at 04:20:46:
The problem with detection methods based on actual moves is they don't have any statistical power. Check the link I gave in reply #4. In fact this is why accusations of cheating are so glibly made. Humans are very good at seeing designs in data which are easily explained by random variation.

I have certainly won a few OTB games through perfect opening play where my opponent stumbled into something I had analyzed carefully. And I have lost a few games the same way. The randomness is that almost any other opening choice would have elicited a different measure of perfection.


If I've understood the research correctly, the authors say they've found some games where engine correlation scores (match of engine best move and avg. centipawn loss) were extremely high, and they conclude that you can't accuse someone of cheating based on seeing a game like those. Which I agree with, and I think is very reasonable : if we play "guess the rating" from a single game, it's highly likely we'll get it wrong.

However, I wouldn't conclude that this kind of indicator has no statistical power ; I'm even firmly convinced that if you compile data from many games by the same player, you'll find very good correlation between such indicators and a player strength. So although drawing conclusions from a single (or even a couple games) may be misleading, comparing someone's play with an engine benchmark over a large sample of games could give excellent results. Now it becomes the statisician's job to find the adequate indicators and the relevant sample size  to not get it wrong. I see no reason why it couldn't be done Cool
  

48 yo, 1920 elo
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1667
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Fide approves online chess regulations
Reply #10 - 01/09/21 at 04:20:46
Post Tools
The problem with detection methods based on actual moves is they don't have any statistical power. Check the link I gave in reply #4. In fact this is why accusations of cheating are so glibly made. Humans are very good at seeing designs in data which are easily explained by random variation.

I have certainly won a few OTB games through perfect opening play where my opponent stumbled into something I had analyzed carefully. And I have lost a few games the same way. The randomness is that almost any other opening choice would have elicited a different measure of perfection.
  
Back to top
 
IP Logged
 
Nickajack
Junior Member
**
Offline


Yabba Dabba Doo!

Posts: 57
Location: USA
Joined: 02/21/17
Gender: Male
Re: Fide approves online chess regulations
Reply #9 - 01/09/21 at 03:15:09
Post Tools
It sounds like intrusion for the sake of intrusion, since it doesn't really prevent various cheating schemes, as you pointed out. It's therefore illogical, even becoming a deterrent to every decent person from taking part in [rated tournament] online chess.

At this rate, I'd rather take my chances against potential cheaters, but surely there must be better cheating detection techniques, based on patterns.
  

Dubious, therefore playable Undecided
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 1667
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Fide approves online chess regulations
Reply #8 - 01/08/21 at 18:17:35
Post Tools
That announcement specifically mentions a waiver for over-the-board tournaments in 2021 and 2022. Not sure about the status of online tournaments.
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1 [2] 3 
Topic Tools
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo