‎Should I train NNUE with or without PruneAtShallowDepth?

[ChessOverflow]

I want to train a Neural Network (NN).

What is the difference between a network trained with ShallowDepth and a network trained without ShallowDepth?

Which mode is stronger? Which mode Stockfish 16 has been trained in?

[joergoster]

To my knowledge, our trainer has no such option. See https://github.com/glinscott/nnue-pytorch
Also take a look at the wiki pages. https://github.com/glinscott/nnue-pytorch/wiki

If you're using the old nodchip trainer https://github.com/nodchip/Stockfish you have to try out what works best.

[ChessOverflow]

Thank you for your excellent guidance and I'm using old nodchip trainer.

I want to know if these networks are trained with or without PruneAtShallowDepth?

• nn-82215d0fd0df.nnue (Stockfish 12)

• nn-62ef826d1a6d.nnue (Strockfish 13)

[joergoster]

This can probably only answer the author of those nets. Afaik, this setting only affects the generation of training and validation data. When in doubt, I would switch PruneAtShallowDepth off ...

[ChessOverflow]

Does switching the PruneAtShallowDepth off mean to disable the "Pruning at shallow depth" in search.cpp before starting train?

      // Step 14. Pruning at shallow depth (~120 Elo). Depth conditions are important for mate finding.
      if (  !rootNode
          && pos.non_pawn_material(us)
          && bestValue > VALUE_TB_LOSS_IN_MAX_PLY)
      {
          // Skip quiet moves if movecount exceeds our FutilityMoveCount threshold (~8 Elo)
          moveCountPruning = moveCount >= futility_move_count(improving, depth);

          // Reduced depth of the next LMR search
          int lmrDepth = newDepth - r;

          if (   capture
              || givesCheck)
          {
              // Futility pruning for captures (~2 Elo)
              if (   !givesCheck
                  && lmrDepth < 7
                  && !ss->inCheck
                  && ss->staticEval + 197 + 248 * lmrDepth + PieceValue[EG][pos.piece_on(to_sq(move))]
                   + captureHistory[movedPiece][to_sq(move)][type_of(pos.piece_on(to_sq(move)))] / 7 < alpha)
                  continue;

              Bitboard occupied;
              // SEE based pruning (~11 Elo)
              if (!pos.see_ge(move, occupied, Value(-205) * depth))
              {
                 if (depth < 2 - capture)
                    continue;
                 // Don't prune the move if opponent Queen/Rook is under discovered attack after the exchanges
                 // Don't prune the move if opponent King is under discovered attack after or during the exchanges
                 Bitboard leftEnemies = (pos.pieces(~us, KING, QUEEN, ROOK)) & occupied;
                 Bitboard attacks = 0;
                 occupied |= to_sq(move);
                 while (leftEnemies && !attacks)
                 {
                      Square sq = pop_lsb(leftEnemies);
                      attacks |= pos.attackers_to(sq, occupied) & pos.pieces(us) & occupied;
                      // Don't consider pieces that were already threatened/hanging before SEE exchanges
                      if (attacks && (sq != pos.square<KING>(~us) && (pos.attackers_to(sq, pos.pieces()) & pos.pieces(us))))
                         attacks = 0;
                 }
                 if (!attacks)
                    continue;
              }
          }
          else
          {
              int history =   (*contHist[0])[movedPiece][to_sq(move)]
                            + (*contHist[1])[movedPiece][to_sq(move)]
                            + (*contHist[3])[movedPiece][to_sq(move)];

              // Continuation history based pruning (~2 Elo)
              if (   lmrDepth < 6
                  && history < -3832 * depth)
                  continue;

              history += 2 * thisThread->mainHistory[us][from_to(move)];

              lmrDepth += history / 7011;
              lmrDepth = std::max(lmrDepth, -2);

              // Futility pruning: parent node (~13 Elo)
              if (   !ss->inCheck
                  && lmrDepth < 12
                  && ss->staticEval + 112 + 138 * lmrDepth <= alpha)
                  continue;

              lmrDepth = std::max(lmrDepth, 0);

              // Prune moves with negative SEE (~4 Elo)
              if (!pos.see_ge(move, Value(-27 * lmrDepth * lmrDepth - 16 * lmrDepth)))
                  continue;
          }
      }

[joergoster]

No, it's an UCI option. Please look at the nodchip repository.
It's described under Generating Training Data and Training a Completely New Network.

[joergoster]

It will slightly increase the quality of fixed depth searches.