Latest Updates:
Page Index Toggle Pages: 1 ... 5 6 [7] 8 9 ... 13
Topic Tools
Very Hot Topic (More than 25 Replies) The best analysis program? (Read 151923 times)
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: The best analysis program?
Reply #96 - 06/16/14 at 19:45:27
Post Tools
Well, well, well - too many questions, indeed!  Cheesy
Michael, most of the questions can be answered by Uncle Google. For example, look at the processor you have: Intel(R) Core(TM) i3 CPU 540 @ 3.07GHz
http://ark.intel.com/products/46473/Intel-Core-i3-540-Processor-4M-Cache-3_06-GH...
Pointng your mouse on "# of Cores" or "# of Threads" and so on...there and clicking on them you can see the definitions, such as: "A Thread, or thread of execution, is a software term for the basic ordered sequence of instructions that can be passed through or processed by a single CPU core."
Althouh the language is somewhat technical, you have to browse through most of these categories in order to be able to explain for yourself all unclear points you have up to now.
I see you have black screens when you click on Strelka 6 x32 and Protector x64 only.
I never used Protector, so I can tell you nothing about it.. As for Strelka 6, I'm sure this engine doesn't have parameters (it's a raw version of Houdini, after all), so it cannot return you even a single piece of information. (Suppose, the case with Protector is the same.)
Anyway, it seems your processor is a "modern" one and supports all SSE.., AVX and so on versions of the engines. From the abovementioned site when clicking on "Instruction Set Extensions" for your processor (SSE4.2): "Instruction Set Extensions are additional instructions which can increase performance when the same operations are performed on multiple data objects. These can include SSE (Streaming SIMD Extensions) and AVX (Advanced Vector Extensions)."
Some engines (when you click on their exe files in your Windows Explorer) read your PC configuration and return pieces of information, which can be very different at first sight. Well, don't worry - they cannot read different things on your PC! You know you have one processor with two cores (which have two threads each - summing up to 4 thread in common). As for how each engine will return information to you (2 processores, 4 threads or two cores...) is another story. When you know what you have you'll understand what they mean..
I definitely don't use Arena although I have it on my PC. I recommend you to buy (at a discount) an old version of Fritz (say 11, 12 or 13 version - the last version is 14 and has some new features which are not important for you in connection with installing engines and so on) and to forget all your troubles with Arena.
As for Arena 3.5 - it's a free GUI which can do many things but is somehow poor in details when you have to do something different than just "kick and rush". My major tip here can be: First see how the engine-exe returns information about your PC when clicking on it! Then go to the major Settings menu (right-clicking cannot be a serious attempt to fix something Smiley ) and try to "untick" all the common settings. Install engines one by one (after knowing how the given engine reads the configuration of your PC) and configure every engine for itself cosidering its specific features (some of them "say" cores, some "say" threads...but you'll know what they really mean), configure the number of cores and threads they should use as you want.
Let me say one important thing here: Your processor has 2 cores, which have 4 threads in common. Most of the chess engines are designed to use real cores in the manner [one core (two threads) - one engine]. Don't try to fix and change parameters in order to use 4 engines on all 4 threads simoultaneously. It will definitely freeze your computer...and the engines won't run. Every engine, when started, is trying to use one core (and not one thread out of this core). Threads can help for other tasks, such as....when you run a chess engine on two cores simultaneously and trying at the same time to open a program or browse the Internet, one of these 4 threads will fulfil the task preventing the freezing of your PC (and then will return to fulfil the tasks of the chess engine). So, the maximum of your processor is to run an engine on 2 cores (say, for faster analysis), or to run two engines simultaneously on one core each. Every try to run two engines that are set to use two cores each will bring the performance of your PC to a freezing. And that's because they both try to use two cores (2 x 2 = 4 cores), but you have only two, so they'll try to use two threads each instead of two cores each (because there are no others available) and therefore all your 4 threads will be taken by the two engines (and not a single thread will be left untaken) - therefore even a click with your mouse will struggle for the resources of your PC...and the engines are usually very hungry for resources, especially Houdini....  Wink
  
Back to top
 
IP Logged
 
Michael Ayton
God Member
*****
Offline


‘You’re never alone with
a doppelgänger.’

Posts: 1810
Location: durham
Joined: 04/19/03
Gender: Male
Re: The best analysis program?
Reply #95 - 06/16/14 at 15:54:13
Post Tools
Hi Vass and many thanks, incl. for the test suites!

I am learning, but I have lots of questions! – so please only respond to any of this if you have time …

I’ve listed my ‘black screens’ below. I am perplexed as to why some engines say ‘4 CPUs found’, when (so I believe) I have 1 CPU, 2 cores, and 4 threads. Also, as you can see, only two engines give me a totally blank screen, but these engines seem to run fine – in what way does my CPU not support them? (And what is POPCNT – is it also an instruction?)

My next issue is changing engine parameters! In Arena, I can see only two ways of doing this. With the engine open in the GUI you can right-click on the engine analysis, but although this brings up a screen showing a lot of parameters, cores (‘threads’) is not among them (except with Stockfish DD)! The other way is via the pulldown menu Engines—Manage—Uci, and this (which I presume applies to all engines*) shows ‘Common max CPU cores setting’ at the bottom. At present this shows ‘1’ and is ticked (so I guess I have only ever been using 1 core per engine!). So if I change it to ‘2’, does that mean that while it is changed, all my engines that can use 2 cores will do so, while those that can't will continue to use 1 without problems? The screen also says ‘For values not set as “common” here, the engine-specific values are in effect (Ctrl +1/Ctrl +2/Ctrl +3)’, so presumably if I untick it (whatever number it says), most of my engines will use both of my cores? So to sum up: is this pulldown menu a perfectly satisfactory/the best/the only way of changing the ‘core’ parameters of UCI engines installed in Arena, or is there a better way?

It seems to be similar in WinBoard. ‘Engine—Engine #1 Settings’ brings up an engine-specific list of parameters which does not include cores (threads), but ‘Options—Common Engine Settings’ brings up a ‘common’ menu which does. Except that it says ‘Max nr of CPUs’ (set to ‘1’) – so does ‘CPUs’ here mean ‘threads’, parallel perhaps with what those ‘black screens’ mean when they say ‘CPUs’?

Finally: when you speak of running two engines, each having a 1-core parameter, ‘simultaneously’, do you mean I can do this without problems just by launching the GUI (Arena or WinBoard) twice? Also, presumably, if I am conducting a match between two engines each using 1 core, I should put Ponder On for maximum strength? And similarly, if I am analysing using ‘Demo’ mode (where the engine actually plays itself), presumably I can do this in two ways: 2 core + Ponder Off or 1 core + Ponder On – but if I am right, which would be stronger/best?

* Strangely(?), Arena 3.5 has (I notice) introduced 'Common max. CPU cores setting for UCI engines', whereas in Arena 3.0 it was possible to alter the thread settings of far more engines individually by right-clicking on the analysis pane!

*****

Stockfish_14053109_x64_modern --- 5 64 SSE4.2
Stockfish_14053109_x64 --- 5 64
Stockfish dd_x64_modern --- DD 64 SSE4.2
Houdini_15a_x64 --- Info string POPCNT enabled; info string 128 MB hash
Gull 3 x64 --- Gull 3 x64
Strelka6 --- [Completely blank]
Komodo-5-64bit --- Using hardware POPCNT
Critter_1.6a_64bit --- Hardware POPCNT enabled
PanChess 00.537.x64 --- HardPOPCNT : Yes
Fire_3.0_SSE42 --- Fire 3.0 x64
Bouquet 1.8 x64 --- 4 CPUs found, using 1 CPU; POPCNT
Ivanhoe999946h Mode_Analysis --- 4 CPUs detected; Prefetch + PopCnt + multicore + hyper hash enabled
Ivanhoe999946h Mode_Game_Play ---4 CPUs detected; Prefetch + PopCnt + multicore enabled
Robbolito 0.21Q x64 --- 4 CPUs found, using 4 threads
Black Mamba_MP_x64 --- 4 CPUs found, using 1 thread; POPCNT supported
DeepSaros_416_popcnt_w64 --- Compiled with POPCNT, compiled with Prefetch
DeepSaros_416_w64 --- Compiled with Prefetch      
Sting SF 3 VE_pop64 --- SF 3 VE_pop64; CPU has hardware POPCNT
Sting SF 3 VEx64 --- SF 3 VEx64
Protector_Win64 --- [Completely blank]
Hannibal1.4bx64 --- Hannibal1.4bx64
« Last Edit: 06/16/14 at 17:16:34 by Michael Ayton »  
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: The best analysis program?
Reply #94 - 06/14/14 at 18:19:20
Post Tools
Well, I'm not an expert, but I'll try to give you some tips.
As long as I know all Intel(R) Core(TM) i3 CPU-s have 2 cores, which means you can use both for analysis simultaneously or separately. You can play matches between engines with "ponder-on"giving one core to each engine...and so on.
When you install an engine on your PC, you'll have to go to the parameters of the engine and give it the number of cores you wish to use. Most of them automatically read your PC and take all the cores of your PC, but not all of them. The number of cores is under the parameter "threads". Don't try to give all your 4 threads on an engine, because your computer can become irresponsible to commands and even if it doesn't happen, it is counter-productive for the analysis. So, under parameters' "threads" you always have to understand real CPU-cores.
For example, you can install an engine twice - the first time on 2 cores, while changing the threads' parameter to 2 and use this engine for analysis on two cores....and the second time to install it on one core (changing the name of the engine like this: Stockfish 5_1core) to use it for games between engines or for analysis on one core. And if you have two engines installed on one core you can easily run them both simultaneously to analyse a position and see the differences they show.
As for how to determine which version of the engine you can install on your PC, say Stockfish, you can easily see if your CPU supports the instructions that are put into the engine from the CPU-Z screen. And even if you are not sure about it, you can double-click on any engine exe in your Windows Explorer - if you see a black screen only, with no information, then your CPU doesn't support this version of the engine, while if you see the same black screen, but with the information for your PC about cores, threads and so on, then your CPU supports this version of the engine.
Of course, if you can use the modern versions of an engine it will be better, because you will gain speed - the specific SSE.., AVX and other CPU instructions help the modern engines to perform faster.
And yes, running analysis on Multi PV is slightly weakening the performance of the engine, because the engines are created for being "warriors" - to fight and win against alikes.. No one created an engine to be good for analysis.
Here I uploaded some tests in pgn, cbv and epd format that some friends of mine gave to me:
http://www19.zippyshare.com/v/89246204/file.html - tactical positions tests
http://www19.zippyshare.com/v/97036595/file.html - endgame tests
Feel free to use them! My friends tell me that a standart test for an engine is to give it a set of 100 tactical positions, giving the engine 2 minutes on each position on one CPU core. Therefore the performance of every engine is counted in per cents automatically - say, Stockfish's performance is 35 out of 100 (35%), Houdini's one 38 out of 100 (38%) and so on..
I myself, never ran a test on my own..  Roll Eyes
  
Back to top
 
IP Logged
 
Michael Ayton
God Member
*****
Offline


‘You’re never alone with
a doppelgänger.’

Posts: 1810
Location: durham
Joined: 04/19/03
Gender: Male
Re: The best analysis program?
Reply #93 - 06/14/14 at 12:10:00
Post Tools
Hi Vass and thank you for a very helpful reply, as ever!

I ran CPU-Z and it tells me 'Cores 2, Threads 4', and under 'Instructions' it gives MMX; SSE, plus SSE2, 3, 4.1, 4.2; SSSE3; EM64T; VT-x. What is it here that tells me (or doesn't) that I can use Stockfish 'Modern'? (I never knew I had a dual-core machine!)

Re engine tests, it sounds as if my small tournaments are virtually useless, even if they are good fun! (In short blitz matches between SF5 and SF5 Modern I got the weird results of 3-1(!), 1.5-2.5, 2-2, 2-2!) Yes, if you could post a test suite or two that would be great -- I imagine other people might be interested too.

I have never played with parameters and almost do not dare! But tell me if there is anything simple (with any engine) I should try changing/tweaking. When I look at my own games I put on Multi PV (to see if that brilliant sacrifice had one refutation, two refutations or no refutation! Cheesy), but otherwise I believe this weakens an engine slightly -- is that true?

  
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: The best analysis program?
Reply #92 - 06/13/14 at 13:57:12
Post Tools
Hi, Michael!
As Jury Osipov himself said, Strelka 6 x32 is a decompiled one-core x32 only engine by reversed engineering from Houdini 4 with some minor changes that he mentioned, too. Here comes the question: why use an x32 version of Houdini 4 on one core, when you can use Houdini 4 on four cores, for example?
The GUI you use doesn't count in testing/analysing if you properly put the parameters of your engines.
Hardware is an important factor, yes, because the best engines are usually made for high-end hardware.
Well, your Intel(R) Core(TM) i3 CPU 540 @ 3.07GHz is not a high-end processor, but is still "modern" and you don't have to have big deviations in your engine tests.
The "professional testers" say that if you want to make a valuable test you have to make a big match between two engines in order to see which one is better. They say, that even 5000 games on a fixed time control are not enough and give some high-sophisticated mathematical values which show that even a match from 10000 games can be only 95% reliable.
Anyway, there are several test matches between Strelka 6 x32 and Stockfish 5 that show that Strelka 6 is designed for blitz time control and can hold against Stockfish for 200-300 games, but when you prolong the test you'll see that Stockfish 5 will win convincingly after all. As for the long time controls, Strelka 6 will not perform well against Stockfish 5 if you let the last to use more than one processor's core.
There are several suites of tactical (and other) positions which are used by the testers to test the engines. And I've never been interested in, but if you want, I'll try to supply you with some of them.
I think you can use Stockfish 5 modern version - you can see what kind of instructions (SSE.. AVX and so on..) supports your Intel(R) Core(TM) i3 CPU 540 @ 3.07GHz by downloading a nice tiny program CPU-Z, completely free from the Internet (I think it's a little exe-program that doesn't install on your computer). Of course, if your processor supports these instructions, then the "modern" engines can perform far better on your PC than the "old" ones.
That's it!  Wink
  
Back to top
 
IP Logged
 
Michael Ayton
God Member
*****
Offline


‘You’re never alone with
a doppelgänger.’

Posts: 1810
Location: durham
Joined: 04/19/03
Gender: Male
Re: The best analysis program?
Reply #91 - 06/13/14 at 12:13:37
Post Tools
Interesting thread. I was meaning to write a post here a few weeks ago when … Strelka 6 came out! And then, blow me down, before I’d got round to it Stockfish 5 appeared!

I'm a bit confused, because I understood that an engine’s relative strength might be influenced by (1) the GUI you use it in and (2) the specific characteristics of your PC, but even so, the results of at least one of the engine matches I’ve conducted on my PC seem a bit radical! (I use Arena 3.5 because I have only an older version of ChessBase.) I got these results:

Strelka 6, 4; Houdini 1.5a, nil!! (5 min. games)

Stockfish 5, 2; Strelka 6, 2 (5 min. games)

Stockfish 5, 2.5; Strelka 6, 1.5 (10 min. games; also, same result at 15-min. time limit)

Strelka 6, 4.5
Stockfish 5, 4
Gull 3.0, 3.5
Houdini 1.5a, nil!! (12 x 5 min. games)

The first result seems appreciable since Strelka 6 is after all essentially a 32-bit version of Houdini 4, and maybe one would not expect Stockfish to beat Strelka by a wider margin than it did, but what of the final engine match? What accounts for Houdini flopping as it did? (For info, I’ve included my PC spec and ArenaMark, etc. below.)

On the subject of different engines’ differential analysis and evaluation of the same position, maybe someone expert should devise a test suite of positions geared to allowing ‘style’ (and not just ‘strength’) conclusions to be drawn from this. Anyone know of such a suite? Anyway, in the opening stages of a game at least, rightly or wrongly I tend to trust engines’ actual move suggestions rather more than I do their numerical evaluations, which as we all know often get revised after some subsequent moves are inputted …

PS. Is it best to use Stockfish 5 x64, or Stockfish 5 x64 modern? What is the difference? And what about engines like Sting or DeepSaros that also have a 'pop(cnt)' version, or Fire, that has an 'AVX' and a 'SSE' version, both with or without 'FRC'? -- how is it possible to know which version is best?

*********

Hardware: Intel(R) Core(TM) i3 CPU 540 @ 3.07GHz with 2.7 GB Memory.
Operating system: Windows 7 Home Premium Home Edition (Build 7600) 64 bit.
CPU speed: 3149.69 MHz
ArenaMark: 26.59
CPUEfficiency: 1.27
  
Back to top
 
IP Logged
 
katar
Senior Member
****
Offline


look another year went
by

Posts: 423
Location: LA
Joined: 09/21/05
Gender: Male
Re: The best analysis program?
Reply #90 - 06/11/14 at 16:24:31
Post Tools
Smyslov_Fan wrote on 06/11/14 at 15:47:43:
I'm gonna take a guess about post #83. Regardless of the numeric evaluation, I'm gonna guess that the different engines all agree on the top three candidate moves from that position. Is that true?

I agree with that.
IMO a centipawn is a different "currency" depending on the engine. 
Stockfish may show -1 ; Critter may show -1.5.  Despite the different numbers, I don't think we can say that Critter necessarily gives a greater advantage to Black in that case.  Certain engines may not even be based on the 1.00 = 1 pawn standard.  And that is not even considering the variable relative strength of pawns (e.g., a protected passed pawn on e6 could be worth a rook).

We need a conversion table in order to compare a debt of one Euro (currency) to a debt of two US dollars.  Same with evaluations of two chess engines IMO.

Different engines may also have different thresholds for what is considered a "clear advantage" versus a "decisive advantage."

Of course the real answer to the OP query is, "unless you are a  professional, it doesn't matter, just use one of the top engines," but that sounds cynical.
  

2078 uscf
Back to top
WWW  
IP Logged
 
Smyslov_Fan
YaBB Moderator
Correspondence fan
*****
Offline


Progress depends on the
unreasonable man. ~GBS

Posts: 6902
Joined: 06/15/05
Re: The best analysis program?
Reply #89 - 06/11/14 at 15:47:43
Post Tools
I'm gonna take a guess about post #83. Regardless of the numeric evaluation, I'm gonna guess that the different engines all agree on the top three candidate moves from that position. Is that true?

The advice that the seasoned correspondence players give makes great sense. Since you seem to have access to multiple engines, you don't have to choose just one. Use them all and use your judgement to decide which lines and which evaluations you prefer.
  
Back to top
 
IP Logged
 
Alias
God Member
*****
Offline



Posts: 1503
Location: East of the river Svartån
Joined: 11/19/04
Re: The best analysis program?
Reply #88 - 06/11/14 at 05:35:25
Post Tools
As a chess book reader, I sometimes feel that the authors could have suggested moves/plans in positions with unbalanced material. I recall the queen sac line in the Safest sicilian. Playing such positions can be tricky since one does not practise them often.

I've also played CC. (I have 2370 on ICCF I think.) It's of course true that even after much analysis, the lines can be difficult to evaluate, but trying them is of course better than just stopping at the said position. If all positions could be easily evaluated after some computer analysis, we wouldn't have CC.
  

Don't check me with no lightweight stuff.
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: The best analysis program?
Reply #87 - 06/10/14 at 22:14:55
Post Tools
Well, as a correspondence chess player I think I know a thing or two about engines.
And I can speak for hours, but I don't think it's necessary.
Just some hints:
There is no engine which can give you "most accurate evaluations" in infinite analysis mode. You can analyse a position for days reaching 44 plys depth and even more...and still don't have a clue about the real evaluation of the position you analyse. It's all because the so-called "horizon" of the engines. The lines they give may be 'true' up to the 8th-10th ply or so (out of these 44 plys), so leaving them counting for hours is counter-productive. (As for the best engine in the background while browsing through your repertoire, just use Houdini! It's more reliable than Stockfish and doesn't scale too fast while heavy pruning as Stockfish does.. While Komodo is too slow to be reliable for being in the background.)
So, if a long (infinite) analysis doesn't help - then what?
I would suggest two methods: the first one callled Aquarium IDeA and the second one - starting matches (at least 300 games on 5min time control) between engines from the given position.
Aah, and the third one is analysing with a GM behind your left shoulder..  Wink
  
Back to top
 
IP Logged
 
tony37
God Member
*****
Offline


I Love ChessPublishing!

Posts: 602
Joined: 10/16/10
Re: The best analysis program?
Reply #86 - 06/10/14 at 21:35:29
Post Tools
TheDragon wrote on 06/10/14 at 15:39:51:
So much for Larry Kaufman and Komdo's "the other engines overvalue the queen...".

From the brief analysis I have conducted it seems to me more that it is Komodo which undervalues the queen.

Can anyone confirm?

I think it really depends on the position, sometimes all engines overvalue the side with the queen, sometimes they all undervalue the side with the queen
  
Back to top
 
IP Logged
 
TheDragon
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 17
Joined: 04/27/14
Re: The best analysis program?
Reply #85 - 06/10/14 at 15:39:51
Post Tools
So much for Larry Kaufman and Komdo's "the other engines overvalue the queen...".

From the brief analysis I have conducted it seems to me more that it is Komodo which undervalues the queen.

Can anyone confirm?
  
Back to top
 
IP Logged
 
Alias
God Member
*****
Offline



Posts: 1503
Location: East of the river Svartån
Joined: 11/19/04
Re: The best analysis program?
Reply #84 - 06/10/14 at 14:34:51
Post Tools
Check the ideas/moves of the different programs. It should become clear after a while which program has evaluated the position correctly.

Never stop at a position! Always check the lines that the programs suggest are the best.
  

Don't check me with no lightweight stuff.
Back to top
 
IP Logged
 
TheDragon
YaBB Newbies
*
Offline


I Love ChessPublishing!

Posts: 17
Joined: 04/27/14
Re: The best analysis program?
Reply #83 - 06/10/14 at 13:56:33
Post Tools
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
*

Hi. Now this is the kind of position which really confuses me. I came across it during my work on the 3.f3 Grunfeld with 3...d5.

Komodo says: Clear Advantage to White.
Houdini says: Very slight Advantage to Black/Equal.
Stockfish says: Very slight Advantage to White/Equal
Larry Kaufman says: White is much better (agrees with Komodo).
Svetushkin says: White has nothing here.
I say: The position looks better and easier to play for white.

The question is: which to believe? Smiley

Regards,
TheDragon
  
Back to top
 
IP Logged
 
Smyslov_Fan
YaBB Moderator
Correspondence fan
*****
Offline


Progress depends on the
unreasonable man. ~GBS

Posts: 6902
Joined: 06/15/05
Re: The best analysis program?
Reply #82 - 06/10/14 at 13:24:30
Post Tools
I think only correspondence chess players would care about the differences because they are so minute.

Both engines are excellent and tend to agree with each other in most positions. The answer you are seeking really depends on the positions. Take a look at the games the two engines played and see if you can come up with patterns that demonstrate strengths or weaknesses in the two programs.

One trait of Stockfish is that it tends to prefer having the Q over various piece combinations more than other engines. In some cases that is a strength, in others that's a weakness.

Analyse the games played between Stockfish and Komodo and see if you can come to conclusions about the types of positions each engine won.

  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1 ... 5 6 [7] 8 9 ... 13
Topic Tools
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo