Openings FAQ

From TCEC wiki
Revision as of 17:44, 26 December 2020 by Kan (talk | contribs) (TCEC Openings FAQ)
Jump to: navigation, search

TCEC Openings FAQ

Q
Why do TCEC chess games not start from the opening position?
A: The answer is rooted on our site’s general objectives, past experience trying different play formats, and our judgment on what makes an entertaining competition and therefore a successful website.
Our primary goal is to provide our web audience with fair and entertaining chess competitions.
We believe that a champion chess program ought to be able to perform well in a wide variety of opening systems, not merely those that it favors. This approach is different from human tournaments by design.
We see the traditional opening position as merely one position out of many thousands that might present a chess program with a challenge. What distinguishes the opening position in computer chess is that it is relatively balanced, meaning it has a high probability of resulting in a draw when played by top competitors at long time controls on very strong hardware.
Many chess programs repeatedly play a given position the same way. When two such programs meet, they tend to produce repetitive contests. A 100-game test we ran between leading engines a few years ago resulted in 88 of the games concentrating into just five ECOs. Such outcomes frustrate the majority’s desire for entertaining chess.
The fact that different competitors must cope with an unpredictable and unequal set of openings does add an element of chance to the competition. Mitigating this chance element,
  1. competitors play the selected openings from both sides of the board in consecutive games,
  2. the length of our season tournaments assures a wide variety of non-repetitive openings, and
  3. as a whole, the openings tend to balance out in terms of drawishness and bias.
This final point underscores the principal risk to our approach, namely, that competitors can be saddled with openings that are too draw-prone or too one-sided, resulting in game-pairs that fail to differentiate the competitors on the cross-table. Our openings team is skilled at selecting opening positions that are neither hopelessly draw-prone nor so one-sided that the weaker side has no chance of holding the draw.
Q
Who selects the openings?
A: Nelson Hernandez (“CatoTheYounger_TCEC”) has led the openings team since mid-way through Season 5 (2013). Over time, the team has grown and evolved in order to encompass different approaches, tools and philosophies. Veteran computer chess book-maker Jeroen Noomen (“Jeroen_TCEC”) has headed Superfinal opening selections since Season 9. More recently, the addition of Eduardo Saucedo (“Cookie_Monster”) and Bastiaan J. Braams (“kbg519v1a”) gave us a quartet. Each of these gentlemen bring chess knowledge, statistical savvy, practical experience as well as their unique personalities to their opening selections.
Q
How are the openings structured?
A: Each seasonal tournament, from the Qualifying League through the Superfinal, has a different opening ‘book’ that issues starting positions:
Tournament Author Length Selection Opening Bias
Qualifying League none 0 moves none none
League 4 Nelson 2 moves random none
League 3 Nelson 4 moves random none
League 2 Eduardo varied consecutive high
League 1 Bastiaan varied consecutive high
Premier Division Nelson 8 moves consecutive high
Superfinal Jeroen varied consecutive high
Selection indicates how positions are selected. Random selections are not repeated once used. Consecutive selections are chosen from a .pgn file in the order submitted to the tournament director. Bookmakers submit their positions weeks or months in advance and have no insight into which engines will be participating or how they will be seeded.
Opening Bias indicates the degree to which the average opening is skewed to favor one color or the other. In past seasons bias was graduated as the level of competition increased, but with the increases in program strength that we have seen in recent years—and the upgrade of TCEC hardware—we now find that considerable bias is needed starting with League 2. We reassess after each season.
Q
To what degree do you bias openings?
A: This varies from bookmaker to bookmaker. But it is a bit difficult to quantify because different programs use different evaluation scales. What is +1.10 to Stockfish is comparable to +0.60 for Leela or Komodo. Nelson, for example, ranks candidate book-exit positions on a percentile basis and narrows his selections to those between the 84th and 93rd percentile, which comes to an average of around +0.90 on Stockfish’s scale. Above that, he deems the position too biased. Below, not enough. (The other bookmakers do not use MS Excel to select openings.)
Q
Why such an aversion to draws? If chess is naturally drawish, why try to distort that reality?
A: The ultimate objective of a tournament is to crown a deserving champion. If you play a match of 100 Superfinal games and 95 of them are draws you will not only drive away much of your audience, you wind up crowning a champion that may have only scored three wins against his opponent’s two. In that event, statistics tell us that the winner may have qualified for the Superfinal based on skill but may have won it entirely on luck. A greater number of decisive game-pairs (i.e. 1.5-0.5, 2-0) reduce the probability of luck being the critical factor as there is a greater chance of one engine pulling away for a clear win.
While we do not object to close contests—on the contrary—we would prefer as many decisive (i.e. not tied) game-pairs as possible, as this way the strongest competitor will have the greatest chance to demonstrate all-around superiority. Plus, it is more entertaining!
Q
How long can you maintain your targeted 60-80% draw-rate when engines are continually improving?
A: Draw abatement will definitely be more challenging as the competitors increase in strength in the years ahead. A mental comparison between today’s leading chess programs and those of ten years ago, combined with extrapolation into the future, points to a time where something will have to give. On the other hand, who really knows what the future will bring? Recent years have been more revolutionary than evolutionary.
Q
What impact will the advent of neural nets and their very different evaluative frame of reference have on your opening set selections?
A: This is impossible to answer as each executable/neural net pairing is unique and presents a different behavioral profile with respect to strength, resistance to draws, evaluation volatility and reliability, etc. The whole field of AI is undergoing rapid development with innovations happening continually; the competitive landscape changes from season to season in surprising ways.