OpenBench Testing Framework

Finished
catch-mate-maths *
8.0+0.08
Elo: -0.00 +- 26.40 [-5.00, 0.00]
Games: 144 W: 35 L: 35 D: 74
Ptnml(0-2): 1, 12, 45, 14, 0
catch mate maths
master
40.0+0.40
Elo: 62.86 +- 10.65 (95%) [N=1024]
Games: 1028 W: 325 L: 141 D: 562
Ptnml(0-2): 0, 53, 231, 223, 7
catch-mate-maths *
8.0+0.08
Elo: -4.82 +- 8.23 [0.00, 5.00]
Games: 1370 W: 322 L: 341 D: 707
Ptnml(0-2): 6, 125, 439, 112, 3
catch mate maths
let-search-run-in-depth-0
40.0+0.40
Elo: 0.00 +- 0.00 [0.00, 5.00]
Games: 0 W: 0 L: 0 D: 0
Ptnml(0-2): 0, 0, 0, 0, 0
let search run in depth 0
let-search-run-in-depth-0
8.0+0.08
Elo: -0.17 +- 3.20 [0.00, 5.00]
Games: 12306 W: 2857 L: 2863 D: 6586
Ptnml(0-2): 49, 1481, 3094, 1485, 44
let search run in depth 0
catch-mate-maths
8.0+0.08
Elo: 0.00 +- 0.00 [0.00, 5.00]
Games: 0 W: 0 L: 0 D: 0
Ptnml(0-2): 0, 0, 0, 0, 0
catch mate maths
thread-voting
4.0+0.04
Elo: -6.51 +- 5.69 [0.00, 5.00]
Games: 3468 W: 745 L: 810 D: 1913
Ptnml(0-2): 7, 427, 928, 368, 4
retore previous decisive logic
thread-voting
4.0+0.04
Elo: -16.32 +- 13.80 [0.00, 5.00]
Games: 618 W: 119 L: 148 D: 351
Ptnml(0-2): 1, 89, 158, 60, 1
don't use thread voting for decisive results perhaps this helps, are we promoting false mates
replace-commandchannel-with-mpsc-channels
8.0+0.08
Elo: -2.35 +- 2.17 [-5.00, 0.00]
Games: 24076 W: 5351 L: 5514 D: 13211
Ptnml(0-2): 34, 2876, 6384, 2707, 37
Replace CommandChannel with stdlib channels Although CommandChannel is nice, we can just use the stdlib for this as it prevents us from having to pull in atomic_wait.
simpl-check-extensions
8.0+0.08
Elo: -2.06 +- 1.82 [-5.00, 0.00]
Games: 37554 W: 8641 L: 8864 D: 20049
Ptnml(0-2): 123, 4637, 9498, 4378, 141
simpl check extensions
thread-voting
4.0+0.04
Elo: -9.50 +- 6.70 [0.00, 5.00]
Games: 2524 W: 531 L: 600 D: 1393
Ptnml(0-2): 3, 330, 666, 259, 4
thread voting tweaks
thread-voting
8.0+0.08
Elo: -2.87 +- 5.10 [0.00, 5.00]
Games: 4118 W: 887 L: 921 D: 2310
Ptnml(0-2): 2, 485, 1118, 453, 1
more thread voting changes
thread-voting
8.0+0.08
Elo: -9.12 +- 6.59 [0.00, 5.00]
Games: 2630 W: 558 L: 627 D: 1445
Ptnml(0-2): 5, 339, 696, 270, 5
more thread voting refactoring
replace-commandchannel-with-mpsc-channels *
4.0+0.04
Elo: -5.39 +- 4.21 [-5.00, 0.00]
Games: 6836 W: 1498 L: 1604 D: 3734
Ptnml(0-2): 14, 887, 1727, 771, 19
thread-voting
8.0+0.08
Elo: -9.93 +- 6.90 [0.00, 5.00]
Games: 2450 W: 520 L: 590 D: 1340
Ptnml(0-2): 3, 332, 625, 262, 3
more thread voting refactoring
adjust-rfp-eval-with-tt-score
8.0+0.08
Elo: -2.93 +- 6.66 [0.00, 5.00]
Games: 2850 W: 646 L: 670 D: 1534
Ptnml(0-2): 11, 361, 700, 347, 6
Adjust RFP eval with tt score
thread-voting-prep-2
4.0+0.04
Elo: 2.11 +- 3.36 [-5.00, 0.00]
Games: 10536 W: 2430 L: 2366 D: 5740
Ptnml(0-2): 25, 1226, 2698, 1298, 21
simplify nwaiter
thread-voting-prep
4.0+0.04
Elo: 3.82 +- 4.11 [-5.00, 0.00]
Games: 6908 W: 1610 L: 1534 D: 3764
Ptnml(0-2): 18, 760, 1819, 842, 15
only notify when we reach threads 0 or 1 remaining
reduce-less-for-ttpv
8.0+0.08
Elo: 3.12 +- 2.48 [0.00, 5.00]
Games: 19716 W: 4647 L: 4470 D: 10599
Ptnml(0-2): 62, 2255, 5043, 2440, 58
reduce less if tt_pv
hl-1536
40.0+0.40
Elo: -9.30 +- 6.53 [0.00, 5.00]
Games: 2504 W: 517 L: 584 D: 1403
Ptnml(0-2): 2, 313, 690, 244, 3
Increase HL to 1536
thread-voting
8.0+0.08
Elo: -2.10 +- 3.76 [0.00, 5.00]
Games: 8116 W: 1801 L: 1850 D: 4465
Ptnml(0-2): 11, 987, 2107, 946, 7
capture more err logs
thread-voting
8.0+0.08
Elo: -2.08 +- 3.74 [0.00, 5.00]
Games: 8166 W: 1856 L: 1905 D: 4405
Ptnml(0-2): 16, 967, 2164, 922, 14
remove atomic_wait usage in nwaiter
thread-voting * *
8.0+0.08
Elo: 1.68 +- 5.95 [0.00, 5.00]
Games: 3310 W: 760 L: 744 D: 1806
Ptnml(0-2): 4, 393, 846, 407, 5
add thread voting
thread-voting
8.0+0.08
Elo: -4.66 +- 19.57 [0.00, 5.00]
Games: 298 W: 63 L: 67 D: 168
Ptnml(0-2): 0, 38, 77, 34, 0
thread voting
thread-voting
8.0+0.08
Elo: 4.83 +- 26.41 [0.00, 5.00]
Games: 144 W: 35 L: 33 D: 76
Ptnml(0-2): 0, 14, 42, 16, 0
thread voting