Full-width punctuation before the end of emphasis breaks the emphasis

[ptmkenny]

When I use emphasis (bold or italics) with Japanese full-width punctuation, if the punctuation comes immediately before the end of the emphasis, the parser ignores it.

CommonMark 1.4.3

What should happen: the bold/italics should be processed just like it is for English punctuation.

Example characters that cause this issue:

。！？、

Input:

**テスト。**テスト

**テスト**テスト

。**。テスト**テスト

。**テスト。**テスト

**テスト**。テスト

**テスト**テスト。

**テスト、**テスト

**テスト**テスト

、**、テスト**テスト

、**テスト、**テスト

**テスト**、テスト

**テスト**テスト、

**テスト。**テスト

**テスト**テスト

！**！テスト**テスト

！**テスト！**テスト

**テスト**！テスト

**テスト**テスト！

**テスト？**テスト

**テスト**テスト

？**？テスト**テスト

？**テスト？**テスト

**テスト**？テスト

**テスト**テスト？

Here's a screenshot of the output:

[colinodell]

Do you get the same results with other CommonMark parsers like https://spec.commonmark.org/dingus/?

[ptmkenny]

Hmm, yes that URL returns the same incorrect output.

This Markdown is originally from Pandoc which handles it just fine.

[colinodell]

The CommonMark spec does have some strict, complex rules around what should be considered valid emphasis, especially when dealing with punctuation: https://spec.commonmark.org/0.29/#emphasis-and-strong-emphasis Because other CommonMark parsers are producing the same result, I would guess the issue is one of two things - either:

• This behavior is valid and expected per the spec; or,
• The spec does not take Japanese full-width punctuation into an account

Either way, because this parser behaves the same as other CommonMark parsers, I don't think this is an issue with this library. I'd highly recommend taking this issue/question upstream:

• https://github.com/commonmark/commonmark-spec/
• https://talk.commonmark.org/

They should be able to identify which of those two possibilities it is. And if something does need to change in the spec or reference implementation, we'll be sure to make corresponding changes here too.

[ptmkenny]

Thanks, I opened a new issue upstream!

[colinodell]

Because our implementation currently matches the spec, even though it doesn't handle your case as you'd expect, I'm going to close this issue. Please know that I will be tracking your upstream issue closely and will implement any resulting changes back here :)