In your script, the parameter 'momentum' in Batchnorm2d has been set to 0.0003(default=0.1). Do you use it during training, or just for inference?