My 1,000 games example was meant to address this suggestion:
an ordinary chessplayer wrote on 01/08/21 at 02:43:21:
and this:
an ordinary chessplayer wrote on 08/27/20 at 04:09:59:
(...) People get mesmerized by the large number of moves fed into the algorithm, and start to believe that the laws of large numbers apply. It's true in the computation of the algorithm sense, but in the legal sense it boils down to a single fact. And it's an indirect fact.
It's not a murder case. Chess, like language, is abstract. Much closer to the case of cheating in chess is that of plagiarizing passages of a book in a paper. The resemblance between the paper and the suspected source is also "a single indirect fact." No one saw the student using the source, the ink from the book is not on the student's hands, and the student didn't tell anyone he was cheating. We just have a single fact--that parts of the paper closely resemble another piece of writing. A lot of matches, in other words. Furthermore, they're not always exact matches, a lot of the paper does not appear plagiarized, and the student is intelligent enough to have written the disputed phrases considered one at a time. Yet we're comfortable enough using this
ex post facto evidence, sometimes without material corroboration. All those matches a single indirect fact? Ok, but it's one hell of a fact.
Regarding the number of moves needed to make a judgment, the man in the street may misunderstand, but he will not be making policy or decisions for major platforms or FIDE. Statistical evidence is fine; it just has to be interpreted thoughtfully by scientifically competent people, like other technical evidence.
Regarding the scientific study cited, it's a lost cause. It's not a confirmation of general relativity: no one, but no one, publishes a paper as an example of a routine
statistical effect--and even if they did, they would certainly mention it in the analysis! The authors just embarrassed themselves, plain and simple. The best one can say is that their idiocy illustrates how someone else could go wrong, too, drawing conclusions from too-small samples. But
we don't have to choose between 40 moves 40,000--
we can quantify the effect of the sample size and weigh this evidence alongside other evidence.