Wers computed diff from kaldi

[zh794390558]

https://github.com/PaddlePaddle/DeepSpeech/blob/edaed68f33b7fff2455c26b880340ba680621f5c/test.py#L110

Why the wers compute method is different from kaldi's method? Which one is correct?

[kuke]

Can you list the detailed difference, or have you ever tested the WER computation under the same inputs? They should give identical results in principle.

[zh794390558]

@kube kaldi use thie code below:

This repo compute wer per sentence, sum all the wers and divided by the total sentence num, which is diff from kaldi.

    BaseFloat percent_wer = 100.0 * static_cast<BaseFloat>(word_errs)
        / static_cast<BaseFloat>(num_words);

when test the wer on test dataset, the diff compute method has different results, that can be about 7% different when test on my own data.

[pkuyym]

@zh794390558 Thanks, our implementation may have some mistakes and will fix it ASAP.

[kuke]

@zh794390558 Thanks for your reminder. This computation diff has been fixed by #51, and the benchmark results also has been updated in #55.

[zh794390558]

@kuke The result get much better, do you known the reason? Why remove space from string to compute WER?

[pkuyym]

It seems that weight of long sentence and short sentence should be same or not is an open question.
Why remove space from string to compute WER?, can't get your point. Generally, consecutive
spaces will be considered as one.

[zh794390558]

@pkuyym

Why remove space from string to compute WER?, can't get your point. Generally, consecutive
spaces will be considered as one.

Sorry, I forget it is CTC model.