Latest Updates:
Page Index Toggle Pages: 1 [2] 3 4 
Topic Tools
Very Hot Topic (More than 25 Replies) IDEA (Read 33771 times)
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: IDEA
Reply #41 - 11/05/13 at 18:24:07
Post Tools
Hi, trandism!
CPU time is divided between the projects automatically by the software. For example, 4-core computer - 4 projects simultaneously -> one core per project. Two projects - > two cores each....and so on.
As for the glitches, the only cure I know is to backup your DATA folder (by hand - copy & paste method) from time to time - let's say one per day...replacing the old backup. Thus, you won't have a big loss - only the daily work.  Wink
One of the best things in the IDeA analysis is that you can use it successfully even in the opening phase, right after, say, 10th move or so. It is because IDeA is digging deeper and deeper and thus it connects the opening moves with the early middlegame phase. Usually, not typical of other analysis methods.  Wink
You can even start the IDeA right from the start, to see if 1.e4 or 1.d4 is best.  Grin But, I'm not sure how much time is needed for that kind of analysis.  Roll Eyes
  
Back to top
 
IP Logged
 
trandism
Full Member
***
Offline


I Love ChessPublishing!

Posts: 181
Location: Greece
Joined: 04/12/08
Gender: Male
Re: IDEA
Reply #40 - 11/05/13 at 14:08:24
Post Tools
Vass, do you happen to know how the CPU time is divided amongs projects by default?
  
Back to top
 
IP Logged
 
trandism
Full Member
***
Offline


I Love ChessPublishing!

Posts: 181
Location: Greece
Joined: 04/12/08
Gender: Male
Re: IDEA
Reply #39 - 11/05/13 at 13:55:07
Post Tools
Well, I'm a noob to correspondence chess myself, but being a software engineer I was able to start using IDEA and other tools fairly quickly. IDEA took me a couple of days to figure out and I'm enthusiastic about the analysis it provides though for the time being only because it seems beneficial to my chess in general just by browsing the tree. As for results regarding correspondence, I'm not in a position to decide yet but I believe Vass who seems to have worked his brains a lot with this stuff.

An I'd like to ask him if ever IDEA seems to have stuck in a project during the generating tasks procedure. I have a specific position that does this and I'm curious. The position is normal - no repetition (graph cycling) stuff - right out of the opening. I wouldn't like to give it here since it concerns an on-going game.

Anyway, glitches aside, the tool is very useful even if someone does not want to play correspondence. The insights it gives into a middlegame position is beyond anything you'd ever get from infinite analysis. I am very excited about certain features of it like the tree shaping you can do and the thematic moves feature.

  
Back to top
 
IP Logged
 
Göran
Senior Member
****
Offline


ChessPublishing is great!

Posts: 451
Location: Sweden
Joined: 02/13/08
Gender: Male
Re: IDEA
Reply #38 - 10/31/13 at 22:10:42
Post Tools
Thanks Tullius.
  

What kind of proof is that?
Back to top
 
IP Logged
 
Tullius
Junior Member
**
Offline


I Love ChessPublishing!

Posts: 71
Joined: 05/03/11
Re: IDEA
Reply #37 - 10/31/13 at 19:15:26
Post Tools
I would wait for Aquarium 2013 which should come in the next weeks and i think they will include Houdini 4.
  
Back to top
 
IP Logged
 
Göran
Senior Member
****
Offline


ChessPublishing is great!

Posts: 451
Location: Sweden
Joined: 02/13/08
Gender: Male
Re: IDEA
Reply #36 - 10/31/13 at 18:47:13
Post Tools
Does anyone know if it is worth while upgrading Aquarium 2011 to Aquarium 2012?
  

What kind of proof is that?
Back to top
 
IP Logged
 
Dink Heckler
God Member
*****
Offline


Love-Forty

Posts: 784
Joined: 02/01/07
Gender: Male
Re: IDEA
Reply #35 - 10/24/13 at 11:17:35
Post Tools
Thanks, Vass; I think I understand it now in general terms, although of course, there is no substitute for playing around with it to get a real feel for its capabilities.
  

'Am I any good at tactics?'
'Computer says No!'
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: IDEA
Reply #34 - 10/24/13 at 07:05:36
Post Tools
It's easy, pal - we sacrifice time.
But even this is not sure. I think we can achieve the same results faster with the IDeA than with another tool.
And here comes your question - what if we use the same time for the infinite analysis?
You can try and compare.. I've tried it so many times and the results are what made me believe.
And it's all because of the pruning and the hash, as well as the right use of the computer cores. The infinite analysis stores a hash file where the engine writes the lines and evaluations, So, let's imagine it checks the thematic move Bc2xh7+ at the start of the calculation, prolongs it a little bit and giving it an evaluation of -1.40 writes the resulting lines in the hash. Then it turns out to another move, say Nf3-g5, and it gives it an evaluation +1.05, then - to another one (say Qd1-d3) and gives it +1.00. Then to Re1-e3 - +0.83 and so on.. How do you think, how much time is needed to find that only Bc2xh7+ is the right move? It will prolong the other lines for so much time, comparing the resulting evaluations to the evaluation of Bc2xh7+, thus reading the hash time and time again..
Now let's see the IDeA - it will generate all those moves, it will prolong them and depending on the settings you use it will find the right Bc2xh7+ line sooner or later. And if it's very complicated to find this winning line, the user can go along the promising lines and enter some 'human' moves by hand (say, the quiet Ra1-f1 at the end of it 12 or 15 moves later), thus giving the engine a push..  Wink
Besides, there are options in this IDeA tool - for example, the 'aggressive' one. It instructs the engine to prolong the lines even if they don't look promising.
As for the quiet positions where there are many moves one can choose from (and all the evaluations are in the range of +0.20 - +0.30), the IDeA gives all the trees and you can choose 'your' move, not the move that the engine gives in the infinite analysis.
As for the right use of cores, it is known that 4 cores, with a task to calculate 4 moves/lines for example, do the job one by one and thus eating about 20 per cents of the time which is needed for the same 4 cores to calculate the same amount of moves/lines. The computer specialists explain all this with parallel threads and so on..
  
Back to top
 
IP Logged
 
Dink Heckler
God Member
*****
Offline


Love-Forty

Posts: 784
Joined: 02/01/07
Gender: Male
Re: IDEA
Reply #33 - 10/23/13 at 22:19:34
Post Tools
Thanks, Vass; I appreciate you taking the time to expalin this.

I guess the part I don't understand is this: for any given amount of time and computing power and a given evaluation function etc, you can make x computations. So, if you want to do A, you can't do B. If IDEA is going deeper by not pruning the tree and treating each new node as a new start position (if I understand correctly?), what is being sacrificed relative to regular analysis? If the answer instead is that it takes a lot longer, why can we not achieve the same results by normal means, perhaps tweaking the parameters slightly?

I suppose what I am saying is that since I can't fathom what are the real conceptual differences, it just seems to me to be a 'brutier' brute-force algo. In which case, if you ran a regular analysis for the same time, why would it not achieve the same results? Where would it be wasting  time / resources relative to IDEA?

I'm sure this questioning comes across as hopelessly misinformed, but I'm just trying to understand as a casual outsider (in the same way as I would 'understand' string theory, perhaps, but nevertheless...).
  

'Am I any good at tactics?'
'Computer says No!'
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: IDEA
Reply #32 - 10/23/13 at 21:09:16
Post Tools
Dink Heckler wrote on 10/23/13 at 20:04:34:
Guys, I've tried, I really have, but I don't understand what the issue is here. How is this different from normal computer analysis? Could someone please ELI5 (explain like I'm 5)...

Well, anyone? Someone who studied pedagogics?  Grin
Apart from the sense of humour..
How do you think? Shooting from a long distance with a super-rifle like Houdini, Stockfish or Komodo, or just go near the target and stab it with a knife? The last is the philosophy of the IDeA, so definitely it is a different approach.
Loading Houdini, for example, to calculate milions of lines in the so-called "infinite analysis" in order to find the best one is not as productive as one might guess not knowing how chess engines work. We shall never forget that every engine is cutting off millions of lines on every ply in depth while trying to find the best ones. Sometimes the engine cuts off the best one at once, right at the start of calculating, because it's not promising. Almost all correspondence chess players know that if they don't enter a line by hand and insist on it, they probably would never find the winning line (in a tactical, or complicated position) if there is one, of course. These are usually the moves (in the line) that the engine eliminated in its search while storing their evaluation in the hash file. And as we all know, the hash, if small-sized, is to change in the process of the calculating by entering evaluations of the new calculated moves while deleting the old ones. As for the big-sized hash, it can sometimes delay the performance of the engine, because the last re-reads it to compare the evaluations of the stored lines while calculating.
So, these procedures decrease the performance of the engine significantly.
The other bad issue is that running 4, 6 or even 8 cores to calculate different lines that often transpose in order to find the best one is counter-productive. The computer specialists know this issue and recommend to use one core for calculating a different task in order to use all your cores effectively.
Knowing all this leads us to some new methods of working while analysing. And here comes the IDeA tool, which is constructed to work in a different manner. It uses all your cores but giving them different tasks for analysing. It gives us the opportunity to fine-tune our engines in order to extract the best they can. We can generate every possible move, step on it and generate moves further and further.. There are no "horizons" here. In the infinite analysis a line which is given by an engine and consists of 35 moves in depth is usually good up to the 8 or maximum 10 move (the so-called "horizon" effect). If you go along this line (with your engine) it will start to fluctuate with other promising moves as soon as you start moving. How can you be sure that on the 4th move there is not one which is significantly better and can change the evaluation of the whole line?
Now turning the IDeA on and looking at the evaluations of all the possible moves, you know that these evaluations are not 'horizon'-limited. These are the evaluations of the positions that are at the end of the calculated lines (because of the so-called 'backsolving' - returning the values back to the very beginning of the line). Knowing all this you can choose your line and make the move you think is best.
Am I missing to explain something? I don't know.. It sounds clear for me. But if it's not for you, then feel free to ask!  Wink
And one more time, this is a tool for "InterActive Deep Analysis" and as such, it works only if you know how to use it. And it's not for those who never played chess OTB at decent level.
  
Back to top
 
IP Logged
 
Dink Heckler
God Member
*****
Offline


Love-Forty

Posts: 784
Joined: 02/01/07
Gender: Male
Re: IDEA
Reply #31 - 10/23/13 at 20:04:34
Post Tools
Guys, I've tried, I really have, but I don't understand what the issue is here. How is this different from normal computer analysis? Could someone please ELI5 (explain like I'm 5)...
  

'Am I any good at tactics?'
'Computer says No!'
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: IDEA
Reply #30 - 10/20/13 at 14:47:30
Post Tools
Well, let's set things straight!
1. Make sure that every engine you installed works on 1 thread, low priority, 128 hash size minimum and TBs assigned to each engine - names given, for example,  Houdini 3_1, Houdini 3_2 and so on. Low priority is needed so you can use all your 6 cores of your computer without having problems to work whatever you wish while analysing with the Aquarium's IDeA.
2. Make sure you have a root assigned on the desired move correctly. With 6 cores you can assign at least 5 more roots wherever you like on some promising lines. For example, if you want to analyse the starting chess position when you're the first player, the basic root is at the very start. And if you are planning to play 1.e4, then you can place the second root after 1...e5, the third one after 1....c5, the fourth - after 1.e4 e5 2.Nf3 Nc6 3.Bb5 a6 4.Ba4 Nf6 5.0-0 Be7 6.Re1 b5 7.Bb3, the fifth after 1.e4 c5 2.Nf3 d6 3.d4 cxd4 4.Nxd4 Nf6 5.Nc3 a6 ....and so on..
You can change these roots whenever you like while analysing - the software starts to use them on the next alternatives / promising lines stage. If you have 6 roots, every engine will take one root and will start generating moves from this root further.
3. First make infinite analysis with different engines (great depth is not needed) - no matter which software you like to use, write the results in a pgn-file, paste it in the so-called "Sandbox" in the Aquarium. Make the IDeA-project from there, making sure your mouse cursor is on the right move that will be the first root of the project. Return to the Sandbox, copy all the moves after the root move and paste it into the IDeA project that you've just created. Start the project and press the button "All positions", thus giving all the moves to the IDeA as tasks to analyse. It will save you time, because the IDeA will start on the right paths at once. And of course, it will make alternatives for every move of your lines.
4. Making a project is better to start as 'default', leaving it at least for a night. 30secs AND 21 plies (maximum 40 seconds) does the job usually. Endgames may need 60 secs OR 30 plys.. Tactical positions may need the 'aggressive' option instead of the "default" one (and don't forget to make your Houdini 3 engines "tactical"). I use 35 to 40 per cents "tree width" most of the times.
5. Next stage is to prolong the analysis changing the "default" option to "longest", while limiting white and black alternatives to 10, 8 or 6 depending on the position you analyse. It can be done "on the fly" also, as almost everything in this software.  Wink
6. You can use every option you see in the GUI as you like - my numbers here are as an example. Some users have completely different styles to extract results from this powerful tool. For example, 15 secs AND 15 plys, "wider" tree, then "default", then "longest" for most of the positions.
7. Do not forget that this tool is 'Interactive' - you have to follow lines and stepping on a move you can press on "current position" or "alternative" buttons, as well as input interesting moves and lines manually moving the pieces directly on the chess board and pressing "current position" button. This way you can direct your analysis on the needed path. It's absolutely necessary, otherwise you'll have to wait for till the Kingdom Come. Of course, there is a way to let the software do the job itself - "wider" option, 75 per cents tree shape and so on... But then the time will be not on your side..  Wink
  
Back to top
 
IP Logged
 
TonyRo
God Member
*****
Offline


I'm gonna crack your skull!

Posts: 1765
Location: Cleveland, OH
Joined: 11/26/07
Gender: Male
Re: IDEA
Reply #29 - 10/20/13 at 04:28:33
Post Tools
Thanks for all of the advice - I'll look deeper into the reasons why for some reason it didn't work that one time. Since then I have run other IDeA analyses successfully.

What settings would you recommend for longer analyses, let's say a few days, for a CC game? My computer has an i7 6-core CPU in it. Just testing out IDeA, I started @ 100s OR 21 ply and it worked okay (I was also parsing through the tree and assigning 'bad moves' to anything that wasn't promising or looked bad - I'm not sure if there's a more efficient way to prune the tree), but given the IDeA methodology that extends the variations indefinitely, it makes a lot of sense to choose less ply for speeds sake. It also basically never hit the 100s - at 21 ply it worked it's way through relatively quickly.
  
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: IDEA
Reply #28 - 10/19/13 at 09:34:51
Post Tools
@ TonyRo
If you are serious about your intention to learn using this tool properly, I can help you.
First advices:
If you are looking for solving a problem or to find a tactic in a position you feel there is something, you'll have to use the "aggressive" option (not the default). Thus, the software looks for every possible move in the position and prolongs the lines even if they are not so promising. It is connected with other settings you'll have to change as well.
Analysing a normal position is easier if you first use the default option (under 'tree shape') for some time and then change this option to "longest" to prolong the promising lines you already have.
Don't try to shoot at distant targets setting the options in the way that the engines calculate more than 30 seconds per move (maximum 60) or so and depth more than 25 plys deep. Such way your engines cut off the alternatives and aim at distant targets. It is useful to shoot at close targets (see my post about the targets above). Some IDeA users set their engines to calculate 15 (or even 10 or 5) seconds per move maximum. It is enough time for the engine to generate a move, step on it and give its evaluation. As you already know, the IDeA will prolong the line and correct this evaluation whenever it steps on the next generated move(s).  Wink
  
Back to top
 
IP Logged
 
Vass
God Member
*****
Offline


I Love ChessPublishing!

Posts: 1105
Joined: 06/22/11
Re: IDEA
Reply #27 - 10/19/13 at 09:06:03
Post Tools
TonyRo wrote on 10/18/13 at 14:56:00:
I sucessfully ran my first IDeA analysis overnight - nothing major. I tested it on a position I had analyzed a large amount by myself, and it came to mostly the same conclusions.

I did try to run another one after it by entering a position in the Sandbox and sending it over to an IDeA project I had already created, but nothing happened - it generated no tasks to start with, and so nothing happened. Has anyone had this problem before?


Many times.. It's not a problem of the GUI, It's something you didn't set properly. For example - the root (you have to place the cursor on the root move before you sending the position to the IDeA, for example). Or telling the IDeA what to do - 'Send to IDeA queue and ...."setting.  Roll Eyes
Anyway, there are many, many settings you have to set properly. The default option is not good for a decent analysis. It's good only for a start. Once you start with it, you'll have to change some options to give a good shape of the tree you want. So to speak, to tell IDeA what to analyse.. Otherwise, it turns out one just wastes processor's time.  Smiley

Edit: The only way to learn using this tool it to test it in the positions you know very, very well. For example in a correspondence chess game that you've played before and you think you don't have secrets in.
Aah, and one more thing: if you don't achieve some good results with it - it's not because the tool is not good. It's because...it is you who is not good with it.  Wink
  
Back to top
 
IP Logged
 
Page Index Toggle Pages: 1 [2] 3 4 
Topic Tools
Bookmarks: del.icio.us Digg Facebook Google Google+ Linked in reddit StumbleUpon Twitter Yahoo