Latest Updates:
Page Index Toggle Pages: [1] 2 
Topic Tools
Hot Topic (More than 10 Replies) Stockfish 11 (Read 992 times)
cma6
Full Member
***
Offline


I love ChessPublishing.com!

Posts: 167
Location: US
Joined: 07/17/04
Re: Stockfish 11
Reply #15 - 02/13/20 at 02:54:21
Post Tools
In SF11, when doing long analysis for correspondence chess, what would be the best settings for Contempt and for Analysis Contempt?
BTW, I don't see any parameter called Dynamic Contempt?
Thanks in advance to the experts?
  
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 659
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Stockfish 11
Reply #14 - 02/06/20 at 15:49:56
Post Tools
That contempt test was all SF11 vs SF11 games. More interesting would be a tournament where some of the opponents were contemptible.
  
Back to top
 
IP Logged
 
IM_Serious
YaBB Newbies
*
Offline


Bibliophile

Posts: 39
Joined: 06/04/16
Re: Stockfish 11
Reply #13 - 02/06/20 at 11:20:39
Post Tools

FYI: Stockfish 11 tournament using 7 different contempt settings

Code
Select All
     Program                 Elo    +   -  Games   Score   Av.Op.  Draws
   1 Stockfish 11 C=0      : 3558   4   4  9000    50.7 %   3553   79.3 %
   2 Stockfish 11 C=+15    : 3558   4   4  9000    50.6 %   3554   77.8 %
   3 Stockfish 11 C=-24    : 3555   4   4  9000    50.1 %   3554   82.7 %
   4 Stockfish 11 C=-15    : 3554   4   4  9000    50.0 %   3554   81.7 %
   5 Stockfish 11 C=+24    : 3554   4   4  9000    50.0 %   3554   76.2 %
   6 Stockfish 11 C=+40    : 3551   4   4  9000    49.5 %   3555   73.8 %
   7 Stockfish 11 C=-40    : 3549   4   4  9000    49.2 %   3555   84.5 % 

Link: https://www.sp-cc.de/experiments.htm

  
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 659
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Stockfish 11
Reply #12 - 02/05/20 at 13:15:30
Post Tools
"As I tried to explain" ... Now you are starting to "communicate" like the devs I was complaining about. If I disagree with you, that doesn't necessarily mean I don't understand the issues. I'm sure I don't understand everything. I am equally sure you don't understand everything.

Example: "The engine is demonstrably stronger (and thus it is objectively more correct)"... I do not accept this equivalence you have stated (strength = correctness), neither in the case of intra-engine comparison, nor in the case of inter-engine comparison. I already gave a hypothetical of a bug which raises Elo. Don't say it can't happen. All complex software has bugs. I am confident Stockfish 11 has bugs. I'm sure I could find one if I examined the code closely.

I haven't modeled the evaluation function with/without dynamic contempt enabled. Others have stated they want to turn it off. Have you measured the effects? Or are you just pontificating at this point?
  
Back to top
 
IP Logged
 
JFugre
Junior Member
**
Offline


I Love ChessPublishing!

Posts: 77
Joined: 01/22/19
Re: Stockfish 11
Reply #11 - 02/05/20 at 07:16:28
Post Tools
an ordinary chessplayer wrote on 02/04/20 at 22:34:45:
I don't necessarily agree that for analysis I would prefer a +1.5 middlegame over a +1.5 endgame.


There's nothing to agree or disagree with here. The engine is demonstrably stronger (and thus it is objectively more correct) if it prefers the middlegame one. If this wasn't the case dynamic contempt would be removed as a simplification.

As I tried to explain, what this means is that the positions should be scored midgame +1.7 vs endgame +1.5 or +1.5 vs +1.3, but knowing this doesn't tell you how exactly to modify the evaluation to get there.

Quote:
Dann Corbit's patch shows it's trivial to implement, why would there be any resistance to doing it?


Because it makes the engine worse without any upsides. It's literally a "prefer trading into drawn endgames when won and prefer losing middlegames instead of trading down into drawable endgames" switch. If I state it for what it is, instead of calling it the extremely misleading "dynamic contempt", then hopefully it's obvious why the devs won't expose it.
  
Back to top
 
IP Logged
 
proustiskeen
God Member
*****
Offline


Hello from Omaha!

Posts: 652
Joined: 08/11/08
Re: Stockfish 11
Reply #10 - 02/04/20 at 22:44:33
Post Tools
If anyone wants a version of SF11 with contempt=0 hard-coded into the .exe, here it is.

https://firstlookchess.com/blog/2020/01/18/stockfish-11-c0/
  
Back to top
WWW  
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 659
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Stockfish 11
Reply #9 - 02/04/20 at 22:34:45
Post Tools
I agree with everything you wrote, RE: the naming problem, the fact that it's from the side-with-advantage perspective so not so much a white-black issue, the dependence on previous score.

I don't necessarily agree that for analysis I would prefer a +1.5 middlegame over a +1.5 endgame. (Actually I'm not sure I would agree for practical play either. In fact my simple brain likes to retain pawns, but not always, and trade pieces, but not always.) So I definitely do NOT agree with the decision not to expose this as a UCI option. Dann Corbit's patch shows it's trivial to implement, why would there be any resistance to doing it?

JFugre wrote on 02/04/20 at 21:20:29:
PS. I didn't find this explained well elsewhere, so the above is fully reverse engineered from the source.

The part that was making me bitter is the *attitude* of the devs that they don't have to explain anything. (Here I should say that while I keep saying "devs", I don't know these people personally and some of the poor communicators I read online might not actually be involved in the code.)

As far as losing Elo is concerned, I have the feeling that if some dumb-ass bug were discovered in the code (e.g. a sign error, or incorrect arguments in a library call), and fixing the bug caused a loss of Elo, then the devs might just leave the bug in. Okay, maybe that's harsh, but the point is that Elo is not everything.
  
Back to top
 
IP Logged
 
JFugre
Junior Member
**
Offline


I Love ChessPublishing!

Posts: 77
Joined: 01/22/19
Re: Stockfish 11
Reply #8 - 02/04/20 at 21:20:29
Post Tools
Quote:
And the devs continually ask "What is the problem?" No matter what answer is given, they accuse the answerer of not understanding what dynamic contempt does, how it works, what it's for, how it's calculated, around and around.


As far as I understand dynamic contempt doesn't have a white vs black inequality nor score flipping effect. The only reason to disable it would be that it has a dependence on previous searches.

While having that dependence is not optimal, the problem with disabling is that it adds some Elo, because the engine isn't optimally trading/avoiding to trade in some situations without this code, and they haven't found a way to express it that is independent of a carried over score that is as strong. So you do weaken it a bit, even in analysis mode, by disabling it.

I think the name is the problem here, because it's not really contempt in any standard sense. What they want to express is: if you are winning (it activates at approximately +1), given positions with an equal advantage for one side (according to the normal eval), prefer the middlegame one over the endgame one. They do this by adding a bonus and halving it in the endgame. (The reverse if you are losing)

The problem is that the static eval can't tell who is winning - at least not reliably! It's much more reliable to use the last full search. So that is what it does. And that adds the dependence.

Now, if you followed along, you understand that:

a) Dynamic contempt don't-trade-off-all-pieces-bonus activates if the score goes above/below ~ +1.
b) It slightly increases how the engine evaluates a middlegame advantage vs the same advantage in the endgame.

And so, yes, indeed "what is the problem" with leaving it enabled? Any minor inconsistencies with scores between searches don't sound like they'd weigh up against an improved assessment of when to trade down or not.

In theory, it sounds like you would get the same effect by increasing all middle game evaluation values a bit and reducing all endgame evaluation values a bit. But until someone figures out exactly what "a bit" needs to be, you'll have the previous score dependence or you'll lose Elo.

So, to summarize, contrasting "contempt" vs "dynamic-contempt-really-dont-be-too-hasty-to-trade-down-while-winning":

1) contempt = always think you are better = inconsistent when switching sides = makes analysis worse

2) no-hasty-trades-bonus (aka dynamic contempt) = consistent between sides but minuscule differences from search to search = makes analysis better

Disabling (1) makes sense for analysis and Stockfish supports it out of the box. Disabling (2) never really seems to make sense and that's why the developers resist an option for it. If you are using a version of Stockfish without dynamic contempt because you think contempt sounds wrong, then I hope the above explanation convinces you you were mistaken (or at least, that it's misnamed!).

PS. I didn't find this explained well elsewhere, so the above is fully reverse engineered from the source.
  
Back to top
 
IP Logged
 
Pawnpusher
Full Member
***
Offline


I Love ChessPublishing!

Posts: 140
Joined: 01/04/18
Gender: Male
Re: Stockfish 11
Reply #7 - 02/04/20 at 11:33:23
Post Tools
Some engines that are derivative of Stockfish have a command line, or button (as I like to say) to check dynamic contempt. There are many children of Stockfish out there.
  
Back to top
 
IP Logged
 
tp2205
Full Member
***
Offline


I Love ChessPublishing!

Posts: 176
Joined: 09/11/11
Re: Stockfish 11
Reply #6 - 02/04/20 at 00:30:31
Post Tools
dynamic contempt has to set to 0 in the source code as far as I know. It is set here:

Stockfish/src/search.cpp:441:              int dct = ct + (102 - ct / 2) * previousScore / (abs(previousScore) + 157);
  
Back to top
 
IP Logged
 
an ordinary chessplayer
God Member
*****
Offline


I used to be not bad.

Posts: 659
Location: Columbus, OH (USA)
Joined: 01/02/15
Re: Stockfish 11
Reply #5 - 02/03/20 at 22:27:13
Post Tools
I knew about Stockfish's dynamic contempt "at a distance", as it were. Now I have looked at it a little, and the discussion I can see online is making me a little bitter.
  1. variable ct, UCI option "Contempt"
  2. variable ct, UCI option "Analysis Contempt" && (UCI option "UCI_AnalyseMode" || command go infinite)
  3. variable dct, UCI option ??? does not exist

And the devs continually ask "What is the problem?" No matter what answer is given, they accuse the answerer of not understanding what dynamic contempt does, how it works, what it's for, how it's calculated, around and around.

Some code by Dann Corbit, for Stockfish 10:
http://talkchess.com/forum3/viewtopic.php?f=2&t=68965&start=40#p781530
  
Back to top
 
IP Logged
 
brabo
God Member
*****
Offline


Welcome chessfriend

Posts: 925
Joined: 02/02/07
Re: Stockfish 11
Reply #4 - 02/03/20 at 20:03:41
Post Tools
proustiskeen wrote on 02/03/20 at 15:29:20:
How did you switch off dynamic contempt? I thought that was impossible without some fairly significant changes to code.

Sorry somehow I misread analysis contempt as dynamic contempt. I doubt both are the same so you can disregard that statement.
  
Back to top
WWW  
IP Logged
 
proustiskeen
God Member
*****
Offline


Hello from Omaha!

Posts: 652
Joined: 08/11/08
Re: Stockfish 11
Reply #3 - 02/03/20 at 15:29:20
Post Tools
How did you switch off dynamic contempt? I thought that was impossible without some fairly significant changes to code.
  
Back to top
WWW  
IP Logged
 
brabo
God Member
*****
Offline


Welcome chessfriend

Posts: 925
Joined: 02/02/07
Re: Stockfish 11
Reply #2 - 02/03/20 at 07:06:38
Post Tools
Stockfish 11 scored 4 points more against Leela in a recently finished 100 rapidmatch on my PC compared with Stockfish 10. Same batch of openingpositions was used.
Lc0 (T40) - Stockfish 10 : 52,5 - 47,5
Lc0 (T40) - Stockfish 11 : 48,5 - 51,5
The Lc0 (T60) networks are starting to get mature. I don't have time now to test it but for sure it could be already interesting.

Don't forget to switch off the contempt when you install Stockfish 11 if you want to use it for analysis otherwise you will get some very big fluctuations of the evaluation for each move. I not only brought the contempt from 24 to 0 but also switched off the dynamic contempt. It was this contempt-free version which I matched against Leela.
  
Back to top
WWW  
IP Logged
 
Leon_Trotsky
Senior Member
****
Offline


Кто был никем — тот станет
всем!

Posts: 499
Location: Barcelona, CAT
Joined: 08/11/17
Gender: Male
Re: Stockfish 11
Reply #1 - 01/21/20 at 08:38:30
Post Tools
I find that I have to let it settle for a little bit longer than for example Komodo in order for Stockish 11's evaluation to stabilise. Although not as long as waiting for Leela to evaluate.
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: [1] 2 
Topic Tools
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo