Torchvision Resnet 50 accuracy

[Esaada]

Hey, Pytorch’s (torchvision) Resnet 50 accuracy is declared to be 76.15.
But when I’m using the training script from PyTorch’s repo, which is mentioned in the official torchvision website(https://pytorch.org/docs/stable/torchvision/models.html#classification):
[https://github.com/pytorch/examples/blob/master/imagenet/main.py]
and the Resnet50 from torchvision:
[https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py]
When training it, after one epoch I’m getting an accuracy of 76.6, how can it be? isn’t the models fully trained?

Thanks!

[fmassa]

Hi,

Can you share the code you are using to train the model? I suspect that you are passing the --pretrained flag to the training code, which starts the model from pre-trained weights

[Esaada]

I tried multiple ways, but the straight way is exactly the same as the script I mentioned:
https://github.com/pytorch/examples/blob/master/imagenet/main.py

You asled about the pretrained part:
`
if args.pretrained:

    print("=> using pre-trained model '{}'".format(args.arch))
    model = models.__dict__[args.arch](pretrained=True)

else:

    print("=> creating model '{}'".format(args.arch))
    model = models.__dict__[args.arch]()

`
All I'm doing is to continue training using --lr=0.0001.

[fmassa]

There can be slight variations of the model accuracy during training, and this could be due to that.
Plus, if I remember correctly, when the resnet models were trained (not by me), there was a post-process step that was done to recompute the batch norm statistics. This means that doing some extra fine-tuning steps could potentially change the results a bit.

Given the above, I'm closing this issue, but let me know if you have further questions

[hukkai]

@fmassa Hi, I also found the problem that the reported error rates for ResNet50 and ResNet101 in the official torch vision website(https://pytorch.org/docs/stable/torchvision/models.html#classification) is higher than our re-trained results. For example, group normalization reported a 22% top 1 error rate of ResNet101 on ImageNet, which is 0.63% lower (I did the experiments and 22% can be achieved ). I think the results of group normalization are also trained from PyTorch? So would you re-train the network to change the results?

Maybe the reported results from torch vision website did not use zero-gamma trick (use zero to initialize the last BN weights in each residual blocks). But the difference is not so big.

[fmassa]

@KaiHoo the resnet-pretrained models didn't use the zero-initialization trick, and we don't provide pretrained models with group-normalization in torchvision.

[hukkai]

@fmassa Sorry I did not make myself clear. The reported ResNet101 error rate with Batch Normalization in the GN paper is 22.0% (https://arxiv.org/pdf/1803.08494.pdf page 6 Deeper models. They use a batch size of 32 per GPU and 8 GPUs. All other training settings are the same as shown in the torchvision training scripts.)

As a reference, Mixup: Beyond empirical risk minimization also shows that ResNet101 with Batch Normalization and without extra regularization can achieve a 22.0% error rate.

I understand the resnet-pretrained models didn't use the zero-initialization trick. But the difference should not be so big (0.64%). I believe it is beyond statistical significance.

[fmassa]

Performance of resnet has gradually improved over time. The first officially-reported pre-trained weights for ImageNet for ResNet101 had a top1 error of 23.6, see https://github.com/KaimingHe/deep-residual-networks

So the zero-initialization trick (which was used in the GN paper) can give (on ResNet50) 0.5 top1 accuracy, see #498 (comment)

[kaiyuyue]

Hi @fmassa,

I want to ask which training type should I use for training ResNet on ImageNet to reproduce the torchvision.models performances - DP or DDP in examples/imagenet/main.py?

I mean that I can't reproduce the torchvision performance using DDP with default settings, like for ResNet-50 I only got 75.420% (vs. torchvision reported 76.130%) top-1 accuracy even after running several rounds.

• Python: 3.7
• PyTorch: 1.8.1
• CUDA: 10.2

Thanks.