Skip to content

fix(core): withhold partial fence and math markers at the end of a streamed prefix - #1132

Open
Mohith26 wants to merge 2 commits into
software-mansion:mainfrom
Mohith26:fix/streaming-partial-fence-markers
Open

Mohith26 wants to merge 2 commits into
software-mansion:mainfrom
Mohith26:fix/streaming-partial-fence-markers

Conversation

@Mohith26

@Mohith26 Mohith26 commented Nov 4, 2026

Copy link
Copy Markdown

What/Why?

Fixes #1130. While a fence is streaming in, its first one or two backticks (or tildes) showed as prose and then vanished when the third arrived, which adds and removes a line in the native renderers. The same thing happens at the other end: inside an open fence, the first characters of the closing fence showed as a line of code until the fence closed. A lone $ before $$ had the same flash.

scanBlocks now notes when the unterminated last line is only the start of one of these delimiters:

  • one or two backticks or tildes after up to three spaces, including inside blockquotes and list items;
  • inside an open fence, a run of the fence's own character shorter than the fence (so a 4-backtick fence also holds back a trailing three);
  • a line that is only $, read the same way cutPendingMath reads $$ lines.

In that case the filter returns the result for the text before those characters, so nothing changes on screen until the next character decides. Anything else (`code`, ~~strike~~, $5, four-space indents, mixed fence characters, an ended line) is released as before, and once the third character arrives the existing waiting-opener rule takes over. Quote markers and list bullets on the same line stay visible, so a list item that starts with inline code does not lose its bullet for a tick.

Testing

  • scripts/test-core-parser.sh passes (filter unit tests, fuzz, cost-per-tick bench, golden AST, coexistence check).
  • New unit tests: the invariant filter(head + partial) == filter(head) for each partial marker after nine different heads in both modes, plus explicit cases for the hand-off at three characters, releases, closing fences in quotes/lists, longer fences, tilde fences and math.
  • The fuzz test gets a second md4c-based check: no streamed prefix may show a text fragment made only of fence characters or a lone $ that the finished document does not contain. I added a tilde fence and a $$ block to its corpus. Before this change that check fails on 42 prefix/mode combinations across the corpus and fixtures; after it, 0.
  • clang-format clean on the touched files.

I could not run the iOS/Android example apps or Maestro here, so the native checkboxes are unticked; the change is confined to the shared C++ filter.

PR Checklist

  • Code compiles and runs on iOS
  • Code compiles and runs on Android
  • Updated documentation/README if applicable
  • Ran example app to verify changes
  • E2E tests are passing
  • Required E2E tests have been added (if applicable)

…reamed prefix

The first one or two backticks or tildes of a fence opener, the start of an open fence's closing fence, and a lone $ before $$ showed as text or as a line of code and then vanished once the delimiter completed. The streaming filter now withholds them until the next character decides, so the result is the one the stream had before they arrived. Fixes software-mansion#1130.
@jnowakow

jnowakow commented Nov 5, 2026

Copy link
Copy Markdown
Collaborator

Hello @Mohith26! Thanks for your effort in developing the library! I reviewed the code and it looks solid but I think it doesn't cover all cases. Could you checkout and possibly add unit tests for those examples:

1. A fence that starts a list item or quote still collapses

The new hold keeps the - , but at the third backtick the existing waiting-opener rule cuts the whole line. The bullet disappears and comes back once the opening line ends.

Stream so far Filter output Parsed as
- `` - list with an empty item
- ``` empty nothing
- ```js\n - ```js\n list item with a code block

The same happens with 1. , > , > - and nested items such as - a\n - ```.

Suggestion: while the opener waits, cut at the fence marker rather than at the line start, so - ``` returns - .

2. Closing fences in lists and quotes still add a line and then remove it

The backticks are withheld, but the line's prefix is kept, and md4c reads that prefix as a blank code line. The new a quoted closing fence withheld and a listed closing fence withheld tests expect this output.

Stream so far Code block contents
> ```js\n> x\n x\n
>js\n> x\n> `` ``` x\n\n
> ```js\n> x\n> ``` x\n

A list closer does the same through its indent, for example -js\n x\n `` ```.

Suggestion: inside an open fence, drop a last line that holds only whitespace or container markers.

… in open fences

While a fence waits for its opening line to end (or stays hidden), cut it at
its marker instead of at the line start, so a list item or quote that starts
with a fence keeps its bullet or quote bar (a streamed "- ```" now gives "- ").

Inside an open fence, withhold a last line that holds only whitespace or
container markers. md4c read it as a blank line of code that then
disappeared when the closing fence arrived in a list item or quote.

Adds unit tests for list, ordered, quoted, quoted-list and nested items, the
closing-fence cases in quotes and lists, and a fuzz check that no streamed
code block shows more lines than the finished one.
@Mohith26
Mohith26 force-pushed the fix/streaming-partial-fence-markers branch from 38726d8 to 84ad877 Compare November 7, 2026 03:45
@Mohith26

Mohith26 commented Nov 7, 2026

Copy link
Copy Markdown
Author

Thanks @jnowakow, both were real. Fixed in 84ad877 with tests for each case.

1. Fence at the start of a list item or quote. While the opener waits, the filter now cuts at the fence marker instead of the line start, so - ``, - ``` and - ```js all give - (an empty list item) until the opening line ends, and -js\n shows the item with its code block as before. Same for `1. `, `> `, `> - `, `> > ` and nested items like `` - a\n - . I applied the same cut in `codeBlockMode: 'hidden'`, since otherwise the bullet would show at two backticks and then disappear for the whole hidden block. That changes two existing expectations: > ```js `` now gives > instead of empty, and the hidden quoted fence case gives `A\n\n> `.

2. Closing fences in lists and quotes. Inside an open fence, a last line that holds only whitespace or container markers (up to the fence's quote depth) is now withheld from its start. Your examples now give code contents x\n at every step: > ```js\n> x\n and > ```js\n> x\n>, > , > ``. The list version ( - js\n x\n `` ) works the same way. A partial nested marker (> > ```js\n> > x\n>) also keeps the fence open now instead of closing it for a tick. A > line inside a top-level fence is still shown, since there it is code. I updated the a quoted/listed closing fence withheld tests to expect this.

I also added a list-item fence to the fuzz corpus and a second md4c check there: no streamed prefix may show a code block with more lines than the finished block has. On the previous commit it fails on the corpus; now it passes.

scripts/test-core-parser.sh passes locally (filter tests, fuzz, bench, golden AST unchanged, coexistence), and clang-format is clean. I still can't run the iOS/Android apps here; the change stays in the shared C++ filter.

@jnowakow jnowakow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for the changes! It's significant improvement but I found one edge case in the current implementation. Intro:\n- ```js would be turned into Intro:\n- and rendered as heading:

Image

Could you address it?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Streaming filter lets partial opening fence markers flash as prose

2 participants