Site views: 2335423


Last update: 4 October 2023.




Benchmarks:




Ongoing series of private book tournaments:




Participants:

- Grendel CTG
- Goi CTG
- DON CTG
- Solista Attack v6.2 CTG
- Fauzi 3.3 CTG
- Hiarcs15jBook

Computer specs
Time used: 5 minutes per game + 2 seconds increment
Number of threads for each engine: 4
Hashtable size: 1024 MBs (average 40% usage rule)

Situation so far (3th October, 1st tournament finished):
Books Tournament October 2023


First tournament finished. Now tuning and learning time.

-

I've planned to buy all the commercial books around and setup a serie of tournaments against my books; I've chosen to not include 1337chess PRO for the moment because of the price too high and for the selling policies (79$ for 5 versions instead of a lesser amount for a single version). Plus the author never replied to my request e-mail.

Any book is at its last version (or nearly... Maybe DON was updated a few more times since I bought it, 12 September).

The games will remain private but you can follow the results time after time.

The playing Grendel is a free book of mine that you can find here (last version below).

My books will be tuned and/or "experienced" time after time until they are definitely the best amongst all of them. So please be patient and expect a lot of changings in the results. These tournaments will last for weeks.


-------o-------



Engine tournament of August 2023




Participants:

- Blue Marlin 15.7
- BrainLearn 24
- Cfish 060821
- CorChess 4 210723
- Crystal 5 KWK
- Stockfish Polyglot 220623
- Swordfish 15.5

Time per game: 5 minutes + 2 seconds increment
Threads used: 1 for each engine
Hashtable used: 512 MB for each engine
Ponder: off
Concurrency: yes, 4 games in the same time
Tournament type: round-robin
Number of games: 420
All engines with default settings

Computer specs


Results:

Engine tournament, Blitz 5.0min+2.0se 2023

1 CorChess 4 230731 65.0/120
2 Swordfish 15.5 62.0/120
3 Brainlearn 24 61.5/120
4 Stockfish Polyglot 220623 61.0/120
5 Cfish 060821 58.0/120 3497.00
6 Blue Marlin 15.7 58.0/120 3490.00
7 Crystal 5 KWK 54.5/120


Download the games in Chessbase and pgn format

Comment:

I was thinking "Wow! This CorChess is very, very strong!", but then I looked at the games: the most of the lost ones are in the B12 opening, not a good one (1.e4 c6), at least for what regards modern computer chess... So I'm guessing that that version of Crystal has an old and not good nnue network...


-------o-------



Engine tournament of May 2023



As usual, any few months I setup an engine tournament.

I pick up the best engines (thanks Sedat for the list) and make them clash.

This time I wanted to try something more Sedat Canbaz-oriented, alhtough I admit to not have still fully grasp the method: 1 thread for any engine, many games, time 1+2, concurrential interfaces. In other words, more than one engine is playing at the same time. In this way we should have a result sooner.

What I still not have understood is how to cross data to make any engine compared to each other...

There were two simultaneous tournaments, one with all the games draw, and the other with the latest Stockfish beta winning and Crystal 5 KWK doing the punchball, another thing I didn't understood. This engine has a sharp analysis and it solves complicated puzzles like no one else before, how comes that it loses so much versus the others?

Mister Zerbinati was so gentle to give me a copy of his SugaR XPro that will enter in the tournament.

Great performances by the last Stockfish beta and the last BrainLearn, good performances by Stockfish Polyfglot 15.1.


First attempt (with Fritz):
Participants:

- Stockfish,dev-20230507-65e215
- Brainlearn 24
- Stockfish Polyglot 15.1,64-bit
- Crystal 5 KWK BMI2
- Polyfish 230510
- CorChess 4,dev-20230508-3473a0
- Blue Marlin 15.7 64-bit BMI2
- SugaR XPrO 140523

Results:

Engines tournament tranche A, Blitz 1.0 2023 (partial)

1 Stockfish,dev-20230507-65e215 21.0 - 21.021.5 - 20.522.5 - 18.5** 65.0/125
2 Brainlearn 24 21.0 - 21.021.5 - 19.521.5 - 20.5 ** 64.0/125
3 Stockfish Polyglot 15.1,64-bit BMI2 20.5 - 21.519.5 - 21.522.5 - 19.5 ** 62.5/125
4 Crystal 5 KWK BMI2 18.5 - 22.520.5 - 21.519.5 - 22.5 ** 58.5/125

Engines tournament tranche B, Blitz 1.0 2023

1 Polyfish 230510 37.5 - 37.537.5 - 37.5** 75.0/150 5625.00
2 37.5 - 37.537.5 - 37.5 ** 75.0/150 5625.00
3 Blue Marlin 15.7 64-bit BMI2 37.5 - 37.537.5 - 37.5 ** 75.0/150 5625.00

Games (hotlink).


Second attempt (with Arena):
Participants:

- Stockfish,dev-20230507-65e215
- Brainlearn 24
- Stockfish Polyglot 15.1,64-bit
- Crystal 5 KWK BMI2
- Polyfish 230510
- CorChess 4,dev-20230508-3473a0
- Blue Marlin 15.7 64-bit BMI2
- SugaR XPrO 140523

Final result:

Program Elo + - Games Score Av.Op. Draws
1 Stockfish,dev-20230507-65e215380612914751.0 %379996.6 %
2 Polyfish 230510380613514751.0 %379998.0 %
3 SugaR XPrO 140523380411814750.7 %379997.3 %
4 Brainlearn 24380412414750.7 %379998.6 %
5 Stockfish Polyglot 15.1,64-bit380212314750.3 %380099.3 %
6 Blue Marlin 15.7 64-bit BMI23802111014750.3 %380096.6 %
7 CorChess 4,dev-20230508-3473a037987914749.7 %380098.0 %
8 Crystal 5 KWK BMI23777141914746.3 %380391.2 %

Total: 588 games.

Download the games in Chessbase and pgn format



-------o-------



Lc0 v0.29.0 tests



Recently I have discovered that a new version Lc0 was published. I decided to give it a try.



1. Specifications:




System:



Participants:




Lc0 v0.29.0 Graphic Card vs Lc0 v0.29.0 CPU.

Download the games in Chessbase and pgn format



2. Specifications:




System:



Participants:




Lc0 v0.29.0 Graphic Card vs Stockfish Polyglot 15.1.

Download the games in Chessbase and pgn format



Lc0 keeps to be a no match for Stockfish, with an enourmous difference of 280 elo points. I really doubt, with such gap, that even having the new GeForce 4090 would make Lc0 win. Yet again this engine was confirmed to be not worthy to be hosted on this site. Maybe in a thematic tournament the difference could be lower.




-------o-------



Engine tests with books or without books?



I apologize for my bad memory, recently I've discovered a couple of tests I did in Avril that I planned to publish here but I never did. The matter was to see whether the books were relevant in modifying the results of the engine strenght tests or not. It came out they can heavily do it. Just this single line opening book changed everything: Philidor Defence single line: 1.e4 e5 2.Nf3 d6 3.d4 exd4 4.Nxd4 c5 5.Nxe2 g6 6.Nbc3 a6 7.g3 Nc6 8.Bg2 Be6 9.0-0 Bg7 10.Nf4.



Specifications:




System:



Participants:




NO BOOK:


Stockfish 15 vs Dragon by Komodo 2.6.1 NO BOOK.

Download the games in Chessbase and pgn format


WITH BOOK:


Stockfish 15 vs Dragon by Komodo 2.6.1 WITH BOOK.

Download the games in Chessbase and pgn format



-------o-------



Lc0 v.0.30.0 vs Stockfish 270922 vs Dragon by Komodo 3




Specifications:




System:



Participants:




Result:


Lc0 v.0.30.0 vs Stockfish 270922 vs Dragon by Komodo 3.

Download the games in Chessbase and pgn format



-------o-------



Chess engines tournament June 2022 - part 2




Specifications:




System:



Participants:




This tournament lasted more than I expected, mostly because of my continuous pauses and focusing on my private life - sorry for that.


StockfishMZ is the official winner of this tournament, although it has the same score than ProteusSF. Now we know what will be the next engines to be hosted on this site.




Chess engines tournament June 2022 part 2 result.

Download the games in Chessbase and pgn format



-------o-------



Chess engines tournament June 2022 - part 1




Specifications:




System:



Participants:



Note: I would have liked to have the latest Lc0 in the tournament...
I tried the CPU version, but I stopped as the CPU fan hurled...
The past month I kept a tournament with Lc0 and it damaged my graphic card, I have a 2014 PC and not enough money to replace the components, so I have to be extra-careful. In view of this, I had sadly to exclude Lc0 from the tournament. If this chess engine used only two cores instead of sticking on four maybe it would have been possible.




Chess engines tournament June 2022 part 1 result.

Download the games in Chessbase and pgn format



StockfishMZ has won this tournament. Most of the wins were made with Queen's Opening and against Dragon 3 by Komodo. Stockfish Polyglot lost one game to StockfishMZ but then won one against ShashChess.




-------o-------



Testing the engines strength with books is legit?



Looks like that testing engines with books to find what is the strongest is not a really good idea.

I was introduced to unbalanced books by the creator of chess.com, Eric, through an epistolar exchange... I was intrigued by testing these unbalanced books, so recently I created a single line book [1. e4 e5 2. Nf3 d6 3. d4 exd4 4. Nxd4 c5 5. Ne2 g6 6. Nbc3 a6 7. g3 Nc6 8. Bg2 Be6 9. O-O Bg7 - a Philidor variant (C47)] to see wether they would be trustable or not to do this task.

It came out they could even invert the final result... In order to be more scientific, I am repeating this test by using two standard engines: Stockfish 15 and Dragon by Komodo 2.6.1, using the same settings and the same single lined opening book.



SETTINGS:

Time used: 1+1
Threads used: 3
Hashtable used: 256 MB

CPU: Intel I7-4771

Tablebases used: 3-4-5 pieces Syzygy

Interface used: Deep Fritz 14
Operating system: Windows 10.


Polyfish 220412 vs Honey 14.1.01 NO BOOK:

Polyfish 220412 vs Honey 14.1.01 NO BOOK


Download the games in Chessbase and pgn format


Polyfish 220412 vs Honey 14.1.01 WITH BOOK:

Polyfish 220412 vs Honey 14.1.01 WITH BOOK


Download the games in Chessbase and pgn format



-------o-------



Lc0 vs swordfish vs stockfish



lc0 vs swordfish vs stockfish


Download the games in Chessbase and pgn format

I didn't think Lc0 made such a great progress! In facts with my almost cheap video card (GTX 780 Ti OC) it managed to be at the same level of Stockfish NNUE and Swordfish NNUE (except one game), making errors only in the ending (most probably because of the lack of time). THIS Lc0 (v0.30.0) is able to take advantage even of the average-low graphic cards. I used the CUDNN flavour. It's been a while in which I'm trying it in analysis also and it's doing great results (thanks janus!). I finally plan to host Lc0 because it's not anymore an unstable, low quality chess engine as I saw a couple of years ago...

Unluckily I had to stop the match because my screen gone black due to some graphic card issue (I guess the card is too old and Lc0 uses it too much). Who knows... maybe Lc0 has its own "NNUE" potentiation now (oh well, Lc0 had always a weighted neural network, but I meant something similar to what Stockfish uses now). I wouldn't be surprised if Lc0 v0.30.0 beats Stockfish without the NNUE net..




All the tests of the past years will be merged into a single downloadable archive.