Difference between revisions of "Openings FAQ"

Revision as of 18:41, 26 December 2020

TCEC Openings FAQ

Q

Why do TCEC chess games not start from the opening position?

A: The answer is rooted on our site’s general objectives, past experience trying different play formats, and our judgment on what makes an entertaining competition and therefore a successful website.

Our primary goal is to provide our web audience with fair and entertaining chess competitions.

We believe that a champion chess program ought to be able to perform well in a wide variety of opening systems, not merely those that it favors. This approach is different from human tournaments by design.

We see the traditional opening position as merely one position out of many thousands that might present a chess program with a challenge. What distinguishes the opening position in computer chess is that it is relatively balanced, meaning it has a high probability of resulting in a draw when played by top competitors at long time controls on very strong hardware.

Many chess programs repeatedly play a given position the same way. When two such programs meet, they tend to produce repetitive contests. A 100-game test we ran between leading engines a few years ago resulted in 88 of the games concentrating into just five ECOs. Such outcomes frustrate the majority’s desire for entertaining chess.

The fact that different competitors must cope with an unpredictable and unequal set of openings does add an element of chance to the competition. Mitigating this chance element,

competitors play the selected openings from both sides of the board in consecutive games,
the length of our season tournaments assures a wide variety of non-repetitive openings, and
as a whole, the openings tend to balance out in terms of drawishness and bias.

This final point underscores the principal risk to our approach, namely, that competitors can be saddled with openings that are too draw-prone or too one-sided, resulting in game-pairs that fail to differentiate the competitors on the cross-table. Our openings team is skilled at selecting opening positions that are neither hopelessly draw-prone nor so one-sided that the weaker side has no chance of holding the draw.

Q: Who selects the openings?; A: Nelson Hernandez (“CatoTheYounger_TCEC”) has led the openings team since mid-way through Season 5 (2013). Over time, the team has grown and evolved in order to encompass different approaches, tools and philosophies. Veteran computer chess book-maker Jeroen Noomen (“Jeroen_TCEC”) has headed Superfinal opening selections since Season 9. More recently, the addition of Eduardo Saucedo (“Cookie_Monster”) and Bastiaan J. Braams (“kbg519v1a”) gave us a quartet. Each of these gentlemen bring chess knowledge, statistical savvy, practical experience as well as their unique personalities to their opening selections.

Q: How are the openings structured?; A: Each seasonal tournament, from the Qualifying League through the Superfinal, has a different opening ‘book’ that issues starting positions:

Tournament	Author	Length	Opening Bias
Qualifying League	none	0 moves	none	none
League 4	Nelson	2 moves	random	none
League 3	Nelson	4 moves	random	none
League 2	Eduardo	varied	consecutive	high
League 1	Bastiaan	varied	56	consecutive	high
Premier Division	Nelson	8 moves	1000+	consecutive	high
Superfinal	Jeroen	varied	50	consecutive	high

Selection indicates how positions are selected. Random selections are not repeated once used. Consecutive selections are chosen from a .pgn file in the order submitted to the tournament director. Bookmakers submit their positions weeks or months in advance and have no insight into which engines will be participating or how they will be seeded.

Opening Bias indicates the degree to which the average opening is skewed to favor one color or the other. In past seasons bias was graduated as the level of competition increased, but with the increases in program strength that we have seen in recent years—and the upgrade of TCEC hardware—we now find that considerable bias is needed starting with League 2. We reassess after each season.

Q: To what degree do you bias openings?; A: This varies from bookmaker to bookmaker. But it is a bit difficult to quantify because different programs use different evaluation scales. What is +1.10 to Stockfish is comparable to +0.60 for Leela or Komodo. Nelson, for example, ranks candidate book-exit positions on a percentile basis and narrows his selections to those between the 84th and 93rd percentile, which comes to an average of around +0.90 on Stockfish’s scale. Above that, he deems the position too biased. Below, not enough. (The other bookmakers do not use MS Excel to select openings.)

Q: Why such an aversion to draws? If chess is naturally drawish, why try to distort that reality?; A: The ultimate objective of a tournament is to crown a deserving champion. If you play a match of 100 Superfinal games and 95 of them are draws you will not only drive away much of your audience, you wind up crowning a champion that may have only scored three wins against his opponent’s two. In that event, statistics tell us that the winner may have qualified for the Superfinal based on skill but may have won it entirely on luck. A greater number of decisive game-pairs (i.e. 1.5-0.5, 2-0) reduce the probability of luck being the critical factor as there is a greater chance of one engine pulling away for a clear win.; While we do not object to close contests—on the contrary—we would prefer as many decisive (i.e. not tied) game-pairs as possible, as this way the strongest competitor will have the greatest chance to demonstrate all-around superiority. Plus, it is more entertaining!

Q: How long can you maintain your targeted 60-80% draw-rate when engines are continually improving?; A: Draw abatement will definitely be more challenging as the competitors increase in strength in the years ahead. A mental comparison between today’s leading chess programs and those of ten years ago, combined with extrapolation into the future, points to a time where something will have to give. On the other hand, who really knows what the future will bring? Recent years have been more revolutionary than evolutionary.

Q: What impact will the advent of neural nets and their very different evaluative frame of reference have on your opening set selections?; A: This is impossible to answer as each executable/neural net pairing is unique and presents a different behavioral profile with respect to strength, resistance to draws, evaluation volatility and reliability, etc. The whole field of AI is undergoing rapid development with innovations happening continually; the competitive landscape changes from season to season in surprising ways.

@@ Line 1: / Line 1: @@
 ==TCEC Openings FAQ==
-*'''Q:  Why do TCEC chess games not start from the opening position?'''
+;Q:  '''Why do TCEC chess games not start from the opening position?'''
+:A:  The answer is rooted on our site’s general objectives, past experience trying different play formats, and our judgment on what makes an entertaining competition and therefore a successful website.
+::Our primary goal is to provide our web audience with fair and entertaining chess competitions.
+::We believe that a champion chess program ought to be able to perform well in a wide variety of opening systems, not merely those that it favors.  This approach is different from human tournaments by design.
+::We see the traditional opening position as merely one position out of many thousands that might present a chess program with a challenge.  What distinguishes the opening position in computer chess is that it is relatively balanced, meaning it has a high probability of resulting in a draw when played by top competitors at long time controls on very strong hardware.
+::Many chess programs repeatedly play a given position the same way.  When two such programs meet, they tend to produce repetitive contests.  A 100-game test we ran between leading engines a few years ago resulted in 88 of the games concentrating into just five ECOs.  Such outcomes frustrate the majority’s desire for entertaining chess.
+::The fact that different competitors must cope with an unpredictable and unequal set of openings does add an element of chance to the competition. Mitigating this chance element,
+::#competitors play the selected openings from both sides of the board in consecutive games,
+::#the length of our season tournaments assures a wide variety of non-repetitive openings, and
+::#as a whole, the openings tend to balance out in terms of drawishness and bias.
+::This final point underscores the principal risk to our approach, namely, that competitors can be saddled with openings that are too draw-prone or too one-sided, resulting in game-pairs that fail to differentiate the competitors on the cross-table.  Our openings team is skilled at selecting opening positions that are neither hopelessly draw-prone nor so one-sided that the weaker side has no chance of holding the draw.
-*A:  The answer is rooted on our site’s general objectives, past experience trying different play formats, and our judgment on what makes an entertaining competition and therefore a successful website.
+;Q:  '''Who selects the openings?'''
+:A:  Nelson Hernandez (“CatoTheYounger_TCEC”) has led the openings team since mid-way through Season 5 (2013).  Over time, the team has grown and evolved in order to encompass different approaches, tools and philosophies.  Veteran computer chess book-maker Jeroen Noomen (“Jeroen_TCEC”) has headed Superfinal opening selections since Season 9.  More recently, the addition of Eduardo Saucedo (“Cookie_Monster”) and Bastiaan J. Braams (“kbg519v1a”) gave us a quartet.  Each of these gentlemen bring chess knowledge, statistical savvy, practical experience as well as their unique personalities to their opening selections.
-:·         Our primary goal is to provide our web audience with fair and entertaining chess competitions.
-:·         We believe that a champion chess program ought to be able to perform well in a wide variety of opening systems, not merely those that it favors.  This approach is different from human tournaments by design.
-:·         We see the traditional opening position as merely one position out of many thousands that might present a chess program with a challenge.  What distinguishes the opening position in computer chess is that it is relatively balanced, meaning it has a high probability of resulting in a draw when played by top competitors at long time controls on very strong hardware.
-:·         Many chess programs repeatedly play a given position the same way.  When two such programs meet, they invariably produce repetitive contests.  A 100-game test we ran between leading engines some time ago resulted in 88% of the games concentrating into just five ECOs.  Such outcomes frustrate the majority’s desire for entertaining chess.
-:·         The fact that different competitors must cope with an unpredictable and unequal set of openings does add an element of chance to the competition.  Mitigating this chance element,
-::1) competitors play the selected openings from both sides of the board in consecutive games,
-::2) the length of our season tournaments assures a wide variety of openings, and
-::3) the openings tend, over time, to balance out in terms of drawishness and bias.
-:This latter point underscores the principal risk to our approach, namely, that competitors can be saddled with openings that are too draw-prone or too one-sided, resulting in game-pairs that fail to differentiate the competitors on the cross-table.  Our openings team is skilled at selecting opening positions that are neither hopelessly draw-prone nor so one-sided that the weaker side has no chance of holding the draw.
+;Q:  '''How are the openings structured?'''
+:A:  Each seasonal tournament, from the Qualifying League through the Superfinal, has a different opening ‘book’ that issues starting positions:
-*'''Q:  Who selects the openings?'''
-*A:   TCEC has a team working on the openings books, supervised by Nelson Hernandez.
-:For S20 the Qualifying League (QL) the games shall be 'bookless', i.e. games starting from the opening position.
-:Leagues 4 and 3 will use short books, 2 and 4 moves, respectively, created by Nelson Hernandez (aka CatoTheYounger_TCEC).  These shall be common chess openings, played randomly.
-:League 2 openings will be selected for a second season by Eduardo Sauceda (aka Cookie Monster).  These shall be of varied length, played in order, and will feature substantially biased openings.
-:League 1 openings will be selected by newcomer to the book team Bastiaan J. Braams (aka bjbraams).  These also shall be of varied length, played in order, and will feature substantially biased openings.
-:Premier Division openings will be selected by Nelson Hernandez, now serving TCEC for his 16th season.  This new book shall be of fixed length (16 plies), played randomly, and will feature the most biased openings Nelson has ever dared to include in a book.  The primary emphasis will be bias, confirmed across multiple engines, and not ECO-diversity as has been the case in past seasons.
-:The S20 Superfinal will once again be selected by esteemed computer chess veteran Jeroen Noomen (aka Jeroen_TCEC), now performing TCEC SuFi service for his 12th season.  Jeroen provides separate commentary on his picks each season.
-*'''Q:  How are the openings structured?'''
-*A:  Each seasonal tournament, from the Qualifying League through the Superfinal, has a different opening ‘book’ that issues starting positions:
 {| class="wikitable" style="margin-left: 20px;"
 !Tournament
-!Book
 !Author
-!Book Length
+!Length
-!# Positions
-!Selection
 !Opening Bias
-!Perpetual
 |-
 |Qualifying League
-|Bookless
 |none
 |0 moves
-|1
-|sequential
 |none
-|yes
+|none
 |-
 |League 4
-|Book A
 |Nelson
 |2 moves
-|1000
 |random
 |none
-|yes
 |-
 |League 3
-|Book B
 |Nelson
 |4 moves
-|1000
 |random
 |none
-|yes
 |-
 |League 2
-|Book C
 |Eduardo
 |varied
-|45
+|consecutive
-|sequential
+|high
-|moderate/high
-|no
 |-
 |League 1
-|Book D
 |Bastiaan
 |varied
 |56
-|sequential
+|consecutive
 |high
-|no
 |-
 |Premier Division
-|Book E
 |Nelson
 |8 moves
 |1000+
-|random
+|consecutive
 |high
-|yes
 |-
 |Superfinal
-|SuFiBook
 |Jeroen
 |varied
 |50
-|sequential
+|consecutive
-|high/very high
+|high
-|no
-|-
-|Cup Final/Bronze
-|Book F
-|Jeroen
-|varied
-|500+
-|random
-|high/very high
-|no
 |-
 |}
-:<nowiki>#</nowiki> Positions signifies the number of unique opening positions contained within each book.
+:Selection indicates how positions are selected.  Random selections are not repeated once used.  Consecutive selections are chosen from a .pgn file in the order submitted to the tournament director.  Bookmakers submit their positions weeks or months in advance and have no insight into which engines will be participating or how they will be seeded.
-:Selection indicates method by which positions contained within the books are selected.
+:Opening Bias indicates the degree to which the average opening is skewed to favor one color or the other.  In past seasons bias was graduated as the level of competition increased, but with the increases in program strength that we have seen in recent years—and the upgrade of TCEC hardware—we now find that considerable bias is needed starting with League 2.  We reassess after each season.
-:Opening Bias indicates the degree to which the average opening is skewed to favor one color or the other.
-:Perpetual indicates which books are for one-time use and which are intended for use in multiple seasons.
-*'''Q:  What do you mean when you say you “bias” openings, and why do you do it?'''
-*A:  Bias is the degree to which a particular opening, or a set of openings, seem to favor one side or the other according to empirical data (historical outcomes) and evaluative data (deep analysis from top engines).
+;Q:  '''To what degree do you bias openings?'''
+:A:  This varies from bookmaker to bookmaker.  But it is a bit difficult to quantify because different programs use different evaluation scales.  What is +1.10 to Stockfish is comparable to +0.60 for Leela or Komodo.  Nelson, for example, ranks candidate book-exit positions on a percentile basis and narrows his selections to those between the 84th and 93rd percentile, which comes to an average of around +0.90 on Stockfish’s scale.  Above that, he deems the position too biased.  Below, not enough.  (The other bookmakers do not use MS Excel to select openings.)
-:The opening position typically offers white a slightly favorable evaluation of +0.15 to +0.20 on traditional chess programs where the “comtempt” setting is neutralized.  Low or zero bias in an opening book makes little difference in a tournament where time controls are shorter and contestants have a wider range of Elos.  In that case, there will be ample decisive games.
-:However, if very strong, closely matched programs are playing at long time controls with no bias that invites a draw-rate approaching 90-95%.  To combat this, we introduce bias in League 2 and ramp it up from there as we progress through the season.  Our four different book authors each have different approaches to striking a balance between too much and not enough.
-*'''Q:  To what degree do you bias openings?'''
-*A:  This is hard to quantify precisely because different engines scale their evaluations differently, plus different authors take different approaches, but at very high bias levels one color will typically have a pawn advantage at book-exit, more or less.  In League 2 it isn't quite so high.  In the Premier Division, a score of +/-0.50 to +/-0.80 is more common.  Sometimes he will deviate from these guidelines to the upside or downside to satisfy opening variety goals.
+;Q:  '''Why such an aversion to draws?  If chess is naturally drawish, why try to distort that reality?'''
+:A:  The ultimate objective of a tournament is to crown a deserving champion.  If you play a match of 100 Superfinal games and 95 of them are draws you will not only drive away much of your audience, you wind up crowning a champion that may have only scored three wins against his opponent’s two.  In that event, statistics tell us that the winner may have qualified for the Superfinal based on skill but may have won it entirely on luck.  A greater number of decisive game-pairs (i.e. 1.5-0.5, 2-0) reduce the probability of luck being the critical factor as there is a greater chance of one engine pulling away for a clear win.
-:In each season’s Superfinal Jeroen has a free hand to do whatever he thinks will result in an interesting and varied contest.  Sometimes his book-exit evaluations will exceed +/-1.00; sometimes he will offer speculative gambits.  His goal is to keep the draw-rate in the 65-80% range each season without a surfeit of one-sided openings.
+:While we do not object to close contests—on the contrary—we would prefer as many decisive (i.e. not tied) game-pairs as possible, as this way the strongest competitor will have the greatest chance to demonstrate all-around superiority.  Plus, it is more entertaining!
-*'''Q:  Why such an aversion to draws?  If chess is naturally drawish, why try to distort that reality?'''
-*A:  The ultimate objective of a tournament is to crown a deserving champion.  If you play 100 Superfinal games and 95 of them are draws you will not only drive away much of your audience, you wind up crowning a champion that may have only scored three wins against his opponent’s two.  In that event, statistics tell us that the winner may have qualified for the Superfinal based on skill but may have won it entirely on luck.  A greater number of decisive game-pairs (i.e. 1.5-0.5, 2-0) reduce the probability of luck being the critical factor as there is a greater chance of one engine pulling away for a clear win.
+;Q:  '''How long can you maintain your targeted 60-80% draw-rate when engines are continually improving?'''
+:A:  Draw abatement will definitely be more challenging as the competitors increase in strength in the years ahead.  A mental comparison between today’s leading chess programs and those of ten years ago, combined with extrapolation into the future, points to a time where something will have to give.  On the other hand, who really knows what the future will bring?  Recent years have been more revolutionary than evolutionary.
-:While we do not object to close contests in the Superfinal — on the contrary — we would prefer as many decisive (i.e. not tied) game-pairs as possible, as this way the strongest competitor will have the greatest chance to demonstrate all-around superiority.  Plus, it is more entertaining!
-*'''Q:  How long can you maintain your targeted 65-80% draw-rate in the Superfinal when engines are continually improving?'''
-*A:  Draw abatement will definitely be more challenging as the competitors increase in strength in the years ahead.  A mental comparison between today’s leading chess programs and those of ten years ago, combined with extrapolation into the future, points to a time where something will have to give.
+;Q:  '''What impact will the advent of neural nets and their very different evaluative frame of reference have on your opening set selections?'''
+:A:  This is impossible to answer as each executable/neural net pairing is unique and presents a different behavioral profile with respect to strength, resistance to draws, evaluation volatility and reliability, etc.  The whole field of AI is undergoing rapid development with innovations happening continually; the competitive landscape changes from season to season in surprising ways.
-*'''Q:  What impact will the advent of neural nets and their very different evaluative frame of reference have on your opening set selections?'''
-*A:  This is impossible to answer as each executable/neural net pairing is unique and presents a different behavioral profile with respect to strength, resistance to draws, evaluation volatility and reliability, etc.  The whole field of AI is undergoing rapid development with innovations happening continually; the competitive landscape changes from season to season in surprising ways.
-:That said, TCEC’s history is one of gradual evolution in response to emerging trends and practical experience; we expect that to continue.
 [[Category:FAQ]]

Difference between revisions of "Openings FAQ"

Revision as of 18:41, 26 December 2020

TCEC Openings FAQ

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools