webui: Fix selecting generated output issues during active streaming

[allozaur]

Close #17132

• Implements incremental rendering: Splits markdown content into stable blocks and a single unstable block (initial draft created by @ServeurpersoCom):
  • Stable blocks (all but the last) are cached and only rendered once
  • Only the last block (unstable) is re-rendered during streaming
  • Prevents DOM reconstruction of already-rendered content, enabling smooth text selection
  • Using HAST node positions for stable block identification
• Reduces unnecessary re-processing of unchanged markdown blocks
• Improves code organization in MarkdownContent component

demo.mp4

[allozaur]

@ggerganov @ngxson @ServeurpersoCom

Please do some testing on your end as well and let me know if we aren't missing anything in this PR to address the issue.

[ServeurpersoCom]

Fantastic! I absolutely must stress test it @ MoE A3B on 5090 because my draft kept crashing after a while!

Edit : We have the same edge case bug as in the POC/Draft (video watched together). We need to try to narrow down by generating the content that causes it.

[ServeurpersoCom]

Running GPT-OSS-20B at 330 tok/s inference was faster than rendering, making it easier to trigger the race condition between stable/unstable block updates, solution: await tick() to force DOM sync
I can no longer reproduce the bug with this patch
I haven't noticed any performance drop, although it could be improved by limiting rendering to one per requestAnimationFrame or a submultiple of them later.

[ggerganov]

Generally this also works on my end, but I do see occasionally bigger selections than expected. Maybe related to the race that @ServeurpersoCom found:

webui-selection-0.mp4

[ggerganov]

Actually, I did some more testing and the problem that I observed occurs only when I am trying to select text inside a code block that is currently being generated. After it gets closed, then selecting for that block works ok.

I think this is acceptable.

[allozaur]

Actually, I did some more testing and the problem that I observed occurs only when I am trying to select text inside a code block that is currently being generated. After it gets closed, then selecting for that block works ok.

I think this is acceptable.

@ggerganov @ServeurpersoCom I've added some changes after @ngxson's review. Please re-test this on your ends.

[ServeurpersoCom]

No regression on my end: >10 long rich markdown generations with success, whereas the corruption was systematically occurring after 2 or 3 generations
Testing : GPT-OSS-20B
Prompt : Writes a long and rich markdown

• Smartphone test OK

[ggerganov]

Found a bug introduced here - when you "Copy code", it forgets the whitespaces:

[ngxson]

Tested on my side, it works except for inside a generating code block as Georgi spotted earlier.

We can improve this in the future by somehow prevent setting innerHTML via {@html block.html}. Instead, I think it's best to have a system where it can take a HastRoot and use depth-first search to get the diff between 2 virtual DOM, and only update changed nodes on HTML. Probably there are some libraries already do all of these heavy-lifting works for us, but we can have a look later.

For now, I think this PR is good to merge.

[ServeurpersoCom]

Same on Windows, copy paste -> no \n or no \r\n

[allozaur]

Found a bug introduced here - when you "Copy code", it forgets the whitespaces:

@ggerganov @ServeurpersoCom this should be fixed with 30c2c18

[thomasjfox]

Thanks so much for this one! 🥳

It fixes a usability annoyance for good. The users will love it.

[allozaur]

Thanks so much for this one! 🥳

It fixes a usability annoyance for good. The users will love it.

Great to hear! It's still not in a perfect state and we are planning a better strategy for rendering the generated content, but it solves the most pressing issue.