Nickajack wrote on 04/02/21 at 07:04:25:


I'm assuming that it is in the sense of a perceived FIDE faux pas.

Dink Heckler wrote on 04/01/21 at 08:06:02:

Actually, a cheat has enough information *now* to defeat the algorithms for all time. I could trivially write a program that would do that, but I won't say how so as not to help any cheats who don't already know. Of course it would come at a cost to the cheater, in that they would only win some of the time, instead of all the time.

One thing that is militating against the cheaters is there's no real money in it. There's bragging rights and putting accomplishments on a CV and so forth, but reputation is not transferable. So you have a bunch of individuals using their own more or less unsophisticated cheating approaches. They can't share information and "best" practices widely, because there's no monetary incentive for their confidant to keep the secret.

I love how nobody cares how far off-topic this thread has become.

That’s a lot of interesting data. Very interesting. 
Might be worth noting there that Magsoodloo also got busted. 

It’s noted elsewhere (Doggers, chess.com) that she has very limited vision in one eye (16%) and has a habit of seemingly looking away, in order to see the screen properly. 

I wonder whether there may be a perfect storm here of unfortunate situations leading to this decision. 

I’d hate to be on a panel judging such things - it’s unenviable. 

If she’s legit (but how to ultimately determine? How much ‘evidence’?) ), a big apology and some nice invites should be forthcoming. Oh dearie me on this kerfuffle.

We all desire transparency; the lack thereof is contrary to our sense of natural justice.

The problem with full transparency is that if you clearly lay out the criteria that will lead to an exclusion, it becomes trivially simple for cheaters to cheat in a way that doesn't meet the threshold for exclusion. Again, this puts us in the invidious position that the demands of justice and competitive integrity are at loggerheads.

an ordinary chessplayer wrote on 03/31/21 at 22:20:21:


I don't think definitive proof should necessarily be the Benchmark, but the evidence should be compelling, transparent and detailed enough to stand scrutiny, and certainly an accused player must have some right of appeal to plead his or her case.  

In my estimation Osmak's opponents played too poorly for a statistical analysis to be convincing, with the possible exception of the King's Indian game I reckon I could have found all the moves she did under the same conditions. There is one point I sort of disagreed with Hikaru on though, and that is in one game where he thought not recapturing a pawn and instead going for a mate in one threat to be a bit suspicious, to me however this seemed like typical human play for Rapid & Blitz.

This case just smells of over reach and while I acknowledge that electronic cheating is a vexing problem that threatens the veracity of competitive chess, we still must take care not to throw out the baby with the bath water.

This has the potential to turn into as much of a fiasco as Radjabov's exclusion from the Candidates was.

 

hicetnunc wrote on 03/31/21 at 20:21:55:

Doesn't this imply that some or all of your opponents played badly? In other words they have poor correlation scores for their rating. 

If that's not the case then there is something I am not understanding about how the scores are calculated.

Dink Heckler wrote on 03/31/21 at 14:30:57:


I think the situation isn't this drastic. I believe it's possible to have reasonable anti-cheating measures by mixing statistics and other systems, provided you give players the benefit of the doubt - ie. you accept some false negatives to avoid false positives.

By tinkering with the pgn-spy tool for many years, I've observed that (except maybe at the very top), human play is still very different from engine play. In rapid, I don't see why it wouldn't be possible to develop a reasonably robust system. There are many variables you can look at (engine correlation scores, blunder rates, complexity of position (Regan), time spent per move). I suspect not enough time/resources have been devoted to the topic yet, but frankly, it shouldn't be more difficult to design an adequate detection system than to develop 3600-playing engines.

TopNotch wrote on 03/30/21 at 22:38:35:


Thanks for the link. Hikaru is really good on YouTube I think. Incredibly strong, obviously, he is personable and cheerful, and he explains well for punters. I don't watch the Twitchy stuff and all sorts of challenges against pseudo-slebs and all that, just the blitz chess against other strong GMs, and it is instructive and fun.

> The games and the moves
I looked through the games there as Hikaru played through them. Looks like a reasonable player beating others who all misplay their openings. Not impressed by her opponents particularly. I would have fancied winning all those too - as the opponents kinda rolled over and died. Repeated opponents going for early baths will result in a z-score overreach over just several games, of course.

Question 1 - Is this just a decent FM/IM strength player putting some not-so-good play to sleep? 
Question 2 - What other evidence is there?

> Sutovsky FB
Sutovsky's FB page is open to all (and which is curiously still seemingly both his personal and official page). Discussions there. Notable are comments by Macieja. 

One would very much hope that when such accusations (and resulting punishments) occur that the evidence - matches, move times, physical observations etc - is just so strong as to be incontrovertible. 

Yes, I agree, it is intriguing.

Hikaru has now also weighed in on his Youtube channel and shares my opinion that nothing in the five games themselves seems to suggest engine cheating by Osmak, Iulija, however he feels there is more to the story than meets the eye. I have to admit, this case really has me intrigued.

Here is the Hikaru link: https://www.youtube.com/watch?v=m0Jt4WB3KQ0

Thoughts?

Analyzing one event at a time will probably never give enough moves to provide a statistically reliable conclusion. Either events will need to be aggregated, or other types of evidence added, or both. It sounds like that's what they've done here, but it's not an easy job.

Let's not jump to conclusions though. It's not at all unusual for a tournament winner to outperform their published Elo by 300 or even 400. It's not even a red flag, especially in shorter events. Some play better than their Elo, others play worse. Just take a look at the winner's TPR in a big open like Gibraltar. Only if someone does it in event after event is it a red flag. But then what would one say about Fischer's streak in 1970-1971? Impossible, right?

Even multiple events is still not conclusive, because you have to allow for people studying like crazy and just improving. I remember being a sub-1600 player for about a year. Some 43 years ago when USCF ratings came out quarterly and were already out-of-date when published, my number was 1599 and I knew this was it. I entered every U1600 event I could find and even traveled out-of-state to find some more. I won every game except one! I was winning that one too but became over-confident. My next published rating was 1760, and three months after that I was a little over 1800. No cheating whatsoever, I just needed to ripen as a player. But statistically impossible I guess.

One other effect is the amplification provided by the playing-the-rating blunder. For example, what if some improving junior (me 40+ years ago) has a published rating of 1599 but a true strength of about 1800? In that scenario, a 1700 opponent may take some risks against a 1600 that they wouldn't try against an 1800. And that's how an (unpublished) 1800 player ends up with a TPR closer to 2000. Once the published rating catches up, opponents start playing more sanely, and the consecutive outlandish performances stop.

To be clear, when I played through Osmak's games I could plainly see she played way above her 2400 rating. hicetnunc's analysis confirms it. But to me it's not a red flag. How many players were in the event? Not just counting her 6-player group, but counting all players. In a group that large, the distribution of net-Elo performances will easily be from -400 to +400. Of course most will perform close to their published Elo, but even without any real improvement *somebody* is bound to have an outlandish performance; it just happened to be her. That's one explanation. The fair play people gave a different explanation.

I have been reviewing the Fair Trading Committee statements and edicts more closely and it made me shudder to think that a governing body has the audacity to assume they can get away with such autocracy. Some of their policies are complete lunacy, and makes me wonder who is running the asylum. 

In this case the cure being worse than the disease is an understatement and opens the door to wholesale corruption. Imagine banning Magnus or Caruana for fairplay violations and then stipulating the verdict or sentence is final and cannot be challenged or appealed in anyway while also claiming that a ban for fair play violation does not harm the player's reputation. Whoever was involved in drafting such a ridiculous  document should be impeached and removed forthwith.

Judge for yourselves:

"FIDE World University online chess championships 2021 Fair Play FIDE World University Individual Online Blitz Championship (14 March) 

The Fair Play Panel (FPP) of the FIDE World University Chess Championships, after examining the games followed by several meetings, disqualified 5 (five) players from the World University Individual Online Blitz Championship for breach of Fair Play. 
Due to the large number of games (4763 in total), the procedure took 70 hours to perform the fair play check. 
The investigation included: 
- statistical evidence 
- Host internet platform (HIP) evidence
- physical evidence 
- expert opinion 
The statistics included several parameters, which combined with the other criteria lead to the decision for disqualification. 

As part of our ongoing efforts to build a community of players that play fairly at all times, a Fair Play review is conducted during every event. We supervise players during their games and undertake a comprehensive analysis of all games played in order to protect players with exceptional performances from accusations of unfair play. 
The decision of FPP to disqualify a player for a suspected fair play violation is final and is not subject to any appeal, review or other challenge. The disqualified players have lost their right to participate in the next events of the 2021 Online University Championships.

It's an open-source tool to collect some statistics from pgn files. I've sent you a pm.

https://github.com/MGleason1/PGN-Spy

Quick report (for player Osmak): pgn-spy scores are indeed very high for someone at this level (they are similar to what you would expect from a 2750+ player in 2hrs games), but the sample is too small to be fully conclusive imo.

Definitely ground for suspicions though, so I can understand the ban if they use better tools (hopefully !) or do have additional pieces of evidence.

disclaimer : I'm using my own methods and baselines

hicetnunc wrote on 03/29/21 at 10:01:07:


Relevant pgn file attached, please let me know what you find.

hicetnunc wrote on 03/29/21 at 10:01:07:


I can produce a pgn from chess24. Does pgn-spy use move times? If yes, what time format is required? Chess24 gives per-move time consumed like so: 
1 d4 0.29 Nf6 0.31 3s
2 c4 0.26 1s g6 0.53 1s

I can produce times either incremental/total, consumed/remaining. Format m:ss would be like so: 
1. d4 {0:00} Nf6 {0:03}
2. c4 {0:01} g6 {0:01}

One of the games the arbiter stopped the clocks, I don't know if I could produce valid times remaining for that game.

Yes, I share some of your feelings there TopNotch. The lack of a right of appeal leaves me feeling rather queasy. It does appear to run counter to proper and accepted ways of doing things. 

Obviously the provider, whichever it may be for any of these events, does not wish to reveal too much of the means of identifying cheating, as it then becomes an arms race between cheaters and the websites. That can’t happen, no. And I guess the content of the published statement would have been agreed in advance of such an event. 

Presumably when providers announce big ‘contravening the rules’ things like this, they believe they have the player(s) pretty much 100% bang to rights. Due to a mix of all types of evidence (as stated). 

I’ll share a cheating story from OTB while I am here. (Have I said before? It’s possible.) 2004 Calvia Olympiad. Versus Hong Kong on board one. Neither player much good really (around 2200), must be said, but all countries can go and fight! The guy touched a piece, then tried to move another. Really blatant. I’ve never been so utterly shocked at the board - that he could even think of trying such, and even doing it. And at this flagship event too. 
My reaction was a huge ‘WTF?’ Complete shock (both were fairly short of time, near time control btw). I looked at him, looked around for a witness, in desperate hope. Incredibly, happily, an arbiter was right behind him and saw this unfold clearly. He made the HK player move the touched piece - a knight on the edge iirc. Phew. I won. Happy ending, but I’ll have words if I see the bloke again. Cheating ******d. Plus, huge respect to that arbiter who was alert and there. Wish I knew who. 

Well, OTB can’t be far off again for all of us, wherever we are. Six months, a year...

Cheers
B

What does PGNspy say? Anyone use that? 

It’s not just the moves of course. It’s clock times too. 
And anything odd if players are on camera (this was a Petrosian issue iirc). When published ratings are known, it’s also about SD and Z-scores too, as we know.

Note - I have no particular comment or opinion on this case. I’ve not seen the games yet, and know no one involved.

So, thoughts on this one?

https://chess24.com/en/read/news/cheating-scandal-hits-fide-world-online-univers...

an ordinary chessplayer wrote on 03/28/21 at 05:04:16:


If the arbiter is from the same country and that country has a vested interest in gaining more titled players, then it is a serious conflict of interest and just the appearance of a conflict of interest is a problem. Moreover most of the cheating does not happen directly in front of the Arbiter, usually outside assistance is involved or some sort of Restroom shinanigens so a DGT board and the presence of an arbiter is irrelevant to this argument. 

What is more pertinent is that the Opponent in a Hybrid match is in a remote location, and it is the opponent who has the most to lose when facing a cheater. In most cases it is the opposing player or players protecting their own self interests that eventually lead to these cheaters being caught, Arbiters for the most part have no skin in the game and many really couldn't care less.

TopNotch wrote on 03/27/21 at 23:56:07:


Not sure about your objection here. Are you confusing Hybrid chess with Centaur chess? Hybrid chess refers to a player using a DGT board in the presence of an arbiter, with the opponent doing the same thing in a remote location. I would rate the risk of cheating to be somewhat higher than a typical OTB tournament, but astronomically lower than the online tournaments currently being held. Basically the only way to cheat is to have a corrupt arbiter, but even that might not be sufficient to get away with it.