LibtorchNeuralNetControl in place of LibtorchDRLControl object

[Luca-R27]

Hello MOOSE users,

I was trying to reproduce the example in
https://mooseframework.inl.gov/modules/stochastic_tools/examples/libtorch_drl_control.html

The training procedure goes fine, I obtain the same plot for the reward as in the documentation page

After the training step, there is only a very brief sentence for the final step, namely reading existing NN parameters:
"Following the training procedure, we can replace the LibtorchDRLControl object with LibtorchNeuralNetControl to evaluate the final version of the neural network without the additional randomization."

Could you please elaborate on this slightly? At first I supposed that this meant using the flag in the Controls block to switch from the first to the second (changing to inactive = 'src_control')

[Controls]
  inactive = 'src_control_final'
  [src_control]
    type = LibtorchDRLControl
    parameters = "BCs/top_flux/value"
    responses = 'center_temp_tend env_temp'

    # keep consistent with LibtorchDRLControlTrainer
    input_timesteps = 2
    response_scaling_factors = '0.03 0.03'
    response_shift_factors = '290 290'
    action_standard_deviations = '0.02'
    action_scaling_factors = 200

    execute_on = 'TIMESTEP_BEGIN'
  []
  [src_control_final]
    type = LibtorchNeuralNetControl

    filename = 'mynet_control.net'
    num_neurons_per_layer = '16 6'
    activation_function = 'relu'

    parameters = "BCs/top_flux/value"
    responses = 'center_temp_tend env_temp'

    # keep consistent with LibtorchDRLControlTrainer
    input_timesteps = 2
    response_scaling_factors = '0.03 0.03'
    response_shift_factors = '290 290'
    action_standard_deviations = '0.02'
    action_scaling_factors = 200

    execute_on = 'TIMESTEP_BEGIN'
  []
[]

The result I get is not what is in the final plot of the documentation. A warning sign might also be that the mynet_control.net in the input file is not present in the folder, I had to change to control.net which is indeed produced during the training step.

This is what I get using LibtorchNeuralNetControl, as opposed to a target temperature that is much more tightly close to the target in the instructions (I also added the result of an intermediate (350) and the final (440) simulation of the training to convince myself that the training was going well).

How do I follow the instruction "replace the LibtorchDRLControl object with LibtorchNeuralNetControl "?

Thank you,
Luca

[GiudGiud]

@grmnptr

[Luca-R27]

Following up on this—any updates or guidance would be appreciated! Thank you.

[GiudGiud]

@somu15 or @grmnptr
We're in milestone season for another couple weeks. Next month will be easier if there is something to look at