OpenBench Testing Framework

Finished
fix-accidentally-lagging-completed-depth
N=5000
Elo: -0.00 +- 0.00 [-5.00, 0.00]
Games: 3074 W: 1002 L: 1002 D: 1070
Ptnml(0-2): 0, 0, 1537, 0, 0
Fix accidentally lagging completed depth After a refactoring, SearchStats::from_ctx was called before updating completed_depth. This was just a reporting issue until trying to add thread voting, since the implementation assumed all threads finish at least depth 1.
multithreaded-tt-clearing
N=5000
Elo: -0.00 +- 0.00 [-5.00, 0.00]
Games: 14336 W: 4707 L: 4707 D: 4922
Ptnml(0-2): 0, 0, 7168, 0, 0
multithreaded tt clearing
thread-voting
4.0+0.04
Elo: -3.21 +- 2.90 [-5.00, 0.00]
Games: 14412 W: 3248 L: 3381 D: 7783
Ptnml(0-2): 40, 1780, 3705, 1635, 46
restore thread_vote_offset
thread-voting
4.0+0.04
Elo: -32.41 +- 39.60 [0.00, 5.00]
Games: 86 W: 16 L: 24 D: 46
Ptnml(0-2): 0, 16, 19, 8, 0
restore thread_vote_offset
thread-meritocracy
4.0+0.04
Elo: -8.49 +- 6.81 [0.00, 5.00]
Games: 2578 W: 534 L: 597 D: 1447
Ptnml(0-2): 10, 325, 681, 264, 9
try ethereal meritocracy
reverse-thread-meritocracy
4.0+0.04
Elo: -45.80 +- 14.22 [0.00, 5.00]
Games: 618 W: 103 L: 184 D: 331
Ptnml(0-2): 3, 121, 139, 46, 0
reverse thread meritocracy
thread-voting
4.0+0.04
Elo: 0.23 +- 8.58 [0.00, 5.00]
Games: 1482 W: 334 L: 333 D: 815
Ptnml(0-2): 0, 171, 400, 168, 2
use hash map for scores
thread-meritocracy
4.0+0.04
Elo: -2.71 +- 6.27 [0.00, 5.00]
Games: 2816 W: 630 L: 652 D: 1534
Ptnml(0-2): 8, 320, 769, 308, 3
bugged meritocracy attempt
thread-voting
4.0+0.04
Elo: -12.87 +- 30.27 [0.00, 5.00]
Games: 108 W: 25 L: 29 D: 54
Ptnml(0-2): 0, 13, 32, 9, 0
add trace
catch-mate-maths *
8.0+0.08
Elo: -3.80 +- 6.79 [-5.00, 0.00]
Games: 2380 W: 513 L: 539 D: 1328
Ptnml(0-2): 8, 265, 667, 245, 5
catch mate maths
catch-mate-maths *
8.0+0.08
Elo: -0.00 +- 26.40 [-5.00, 0.00]
Games: 144 W: 35 L: 35 D: 74
Ptnml(0-2): 1, 12, 45, 14, 0
catch mate maths
master
40.0+0.40
Elo: 62.86 +- 10.65 (95%) [N=1024]
Games: 1028 W: 325 L: 141 D: 562
Ptnml(0-2): 0, 53, 231, 223, 7
catch-mate-maths *
8.0+0.08
Elo: -4.82 +- 8.23 [0.00, 5.00]
Games: 1370 W: 322 L: 341 D: 707
Ptnml(0-2): 6, 125, 439, 112, 3
catch mate maths
let-search-run-in-depth-0
40.0+0.40
Elo: 0.00 +- 0.00 [0.00, 5.00]
Games: 0 W: 0 L: 0 D: 0
Ptnml(0-2): 0, 0, 0, 0, 0
let search run in depth 0
let-search-run-in-depth-0
8.0+0.08
Elo: -0.17 +- 3.20 [0.00, 5.00]
Games: 12306 W: 2857 L: 2863 D: 6586
Ptnml(0-2): 49, 1481, 3094, 1485, 44
let search run in depth 0
catch-mate-maths
8.0+0.08
Elo: 0.00 +- 0.00 [0.00, 5.00]
Games: 0 W: 0 L: 0 D: 0
Ptnml(0-2): 0, 0, 0, 0, 0
catch mate maths
thread-voting
4.0+0.04
Elo: -6.51 +- 5.69 [0.00, 5.00]
Games: 3468 W: 745 L: 810 D: 1913
Ptnml(0-2): 7, 427, 928, 368, 4
retore previous decisive logic
thread-voting
4.0+0.04
Elo: -16.32 +- 13.80 [0.00, 5.00]
Games: 618 W: 119 L: 148 D: 351
Ptnml(0-2): 1, 89, 158, 60, 1
don't use thread voting for decisive results perhaps this helps, are we promoting false mates
replace-commandchannel-with-mpsc-channels
8.0+0.08
Elo: -2.35 +- 2.17 [-5.00, 0.00]
Games: 24076 W: 5351 L: 5514 D: 13211
Ptnml(0-2): 34, 2876, 6384, 2707, 37
Replace CommandChannel with stdlib channels Although CommandChannel is nice, we can just use the stdlib for this as it prevents us from having to pull in atomic_wait.
simpl-check-extensions
8.0+0.08
Elo: -2.06 +- 1.82 [-5.00, 0.00]
Games: 37554 W: 8641 L: 8864 D: 20049
Ptnml(0-2): 123, 4637, 9498, 4378, 141
simpl check extensions
thread-voting
4.0+0.04
Elo: -9.50 +- 6.70 [0.00, 5.00]
Games: 2524 W: 531 L: 600 D: 1393
Ptnml(0-2): 3, 330, 666, 259, 4
thread voting tweaks
thread-voting
8.0+0.08
Elo: -2.87 +- 5.10 [0.00, 5.00]
Games: 4118 W: 887 L: 921 D: 2310
Ptnml(0-2): 2, 485, 1118, 453, 1
more thread voting changes
thread-voting
8.0+0.08
Elo: -9.12 +- 6.59 [0.00, 5.00]
Games: 2630 W: 558 L: 627 D: 1445
Ptnml(0-2): 5, 339, 696, 270, 5
more thread voting refactoring
replace-commandchannel-with-mpsc-channels *
4.0+0.04
Elo: -5.39 +- 4.21 [-5.00, 0.00]
Games: 6836 W: 1498 L: 1604 D: 3734
Ptnml(0-2): 14, 887, 1727, 771, 19
thread-voting
8.0+0.08
Elo: -9.93 +- 6.90 [0.00, 5.00]
Games: 2450 W: 520 L: 590 D: 1340
Ptnml(0-2): 3, 332, 625, 262, 3
more thread voting refactoring