Some weights of XLMRobertaForMaskedLM were not initialized from the model checkpoint at xlm-roberta-base and are newly initialized

[Syavaprd]

Where can I find weight that won't give the following error?
The code:
model = XLMRobertaForMaskedLM.from_pretrained('xlm-roberta-base')

[LysandreJik]

Could you put the full error in the description of the issue rather than in the title? We don't know which weights are not initialized.

[Syavaprd]

Could you put the full error in the description of the issue rather than in the title? We don't know which weights are not initialized.
Code:
model = XLMRobertaForMaskedLM.from_pretrained('xlm-roberta-base')
Warning:
Some weights of XLMRobertaForMaskedLM were not initialized from the model checkpoint at xlm-roberta-base and are newly initialized: ['lm_head.decoder.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

[LysandreJik]

You don't need to worry about that message, it lets you know that the bias in the LM head is not initialized - it will be initialized to all zeros.

I'm removing this warning in #9615.