Skip to content

feat: telegram use parse mode ModeMarkdownV2 instead of ModeHTML#1018

Merged
alexhoshina merged 16 commits intosipeed:mainfrom
Alexandersfg4:feat/telegram-use-md2
Mar 18, 2026
Merged

feat: telegram use parse mode ModeMarkdownV2 instead of ModeHTML#1018
alexhoshina merged 16 commits intosipeed:mainfrom
Alexandersfg4:feat/telegram-use-md2

Conversation

@Alexandersfg4
Copy link
Contributor

@Alexandersfg4 Alexandersfg4 commented Mar 3, 2026

📝 Description

🗣️ Type of Change

  • ✨ New feature (non-breaking change which adds functionality)
  • ⚡ Code refactoring (no functional changes, no api changes)

🤖 AI Code Generation

  • 🛠️ Mostly AI-generated (AI draft, Human verified/modified)

🔗 Related Issue

Currently, Picoclaw sends messages from the agent to Telegram in the following way:
Input in MD format -> Parsed into HTML -> Sent as a message.
If the message fails to send, a fallback message is sent with HTML format and an empty parse mode.
In current implementation I often observe something like this (heading tags, HTML tags
and etc
.):
image
image

My proposal:
Input in MD format -> Parsed into Telegram MD2 format -> Sent as a message. (less bugs could be handled)
If the message fails to send, a fallback message is sent with MD format and an empty parse mode.
Heading is not supporting so we transform heading -> bolding.
Unit tests created for the feature.

📚 Technical Context (Skip for Docs)

  • Reference URL:
  • Reasoning:

🧪 Test Environment

  • Hardware: Apple M2 Pro
  • OS: 26.3 (25D125)
  • Model/Provider: groq/meta-llama/llama-4-scout-17b-16e-instruct"
  • Channels: Telegram

📸 Evidence (Optional)

Click to view Logs/Screenshots image

☑️ Checklist

  • My code/docs follow the style of this project.
  • I have performed a self-review of my own changes.
  • I have updated the documentation accordingly.

@sipeed-bot sipeed-bot bot added type: enhancement New feature or request domain: channel labels Mar 3, 2026
@Alexandersfg4
Copy link
Contributor Author

@alexhoshina Hi, could you look at my PR?

@alexhoshina
Copy link
Collaborator

Thanks for the reminder and for your patience.
This PR proposes switching Telegram’s parse mode to MarkdownV2.
At the same time, there’s another open PR (#935) that updates the Telegram sending logic under HTML mode—particularly around message formatting and output behavior.
These two PRs represent different directions: one assumes we’ll keep using HTML, the other assumes we’ll move to MarkdownV2.
Since they’re mutually exclusive in intent, we need to decide which parse mode we want to standardize on as a telegram.

Should our long-term Telegram output use HTML or MarkdownV2?

Thanks again for your understanding and collaboration!

@alexhoshina
Copy link
Collaborator

Thanks for the reminder and for your patience. This PR proposes switching Telegram’s parse mode to MarkdownV2. At the same time, there’s another open PR (#935) that updates the Telegram sending logic under HTML mode—particularly around message formatting and output behavior. These two PRs represent different directions: one assumes we’ll keep using HTML, the other assumes we’ll move to MarkdownV2. Since they’re mutually exclusive in intent, we need to decide which parse mode we want to standardize on as a telegram.

Should our long-term Telegram output use HTML or MarkdownV2?

Thanks again for your understanding and collaboration!

@Alexandersfg4 @putueddy

@alexhoshina alexhoshina self-assigned this Mar 5, 2026
@putueddy
Copy link
Contributor

putueddy commented Mar 5, 2026

Hey @alexhoshina, thanks for flagging this — it's a good discussion to have before we go in two different directions!

After looking at both PRs, I'd lean toward sticking with HTML. Here's my thinking:

LLM output is messy by nature
The agent loves throwing around *, _, ., ! and all sorts of characters that happen to be reserved in MarkdownV2 (there are 19+ of them!). One unescaped dot or mismatched asterisk and Telegram rejects the whole message. With HTML, we only need to worry about <, >, and & — which almost never show up in normal responses.

The community generally says the same thing
Most Telegram bot framework maintainers (Telegraf, GramIO, etc.) actually recommend HTML over MarkdownV2 for dynamic content. Some go as far as calling MarkdownV2 "dangerous" when you don't fully control the input — and with LLM output, we definitely don't.

A few concerns with the MarkdownV2 parser here
I noticed some things that could bite us — there's a potential out-of-bounds panic in the expandable blockquote handler when the input is short, the link detection relies on a single-char lookbehind that can misfire (e.g. array0), and backslashes aren't being escaped. The test suite covers the happy path nicely but doesn't quite get to these edge cases yet.

The formatting issues are fixable under HTML
The screenshots in this PR showing leaked HTML tags are real and annoying, but I think that's more about bugs in the current converter than a fundamental problem with HTML mode. My PR (#935) tackles the chunking and fallback logic, which should clean those up.

All that said — I really appreciate the work in this PR, @Alexandersfg4! The heading-to-bold conversion is a nice touch and something we could totally bring over to the HTML path too.

Curious what you both think!

@CLAassistant
Copy link

CLAassistant commented Mar 5, 2026

CLA assistant check
All committers have signed the CLA.

@Alexandersfg4
Copy link
Contributor Author

Hey @alexhoshina, thanks for flagging this — it's a good discussion to have before we go in two different directions!

After looking at both PRs, I'd lean toward sticking with HTML. Here's my thinking:

LLM output is messy by nature The agent loves throwing around *, _, ., ! and all sorts of characters that happen to be reserved in MarkdownV2 (there are 19+ of them!). One unescaped dot or mismatched asterisk and Telegram rejects the whole message. With HTML, we only need to worry about <, >, and & — which almost never show up in normal responses.

The community generally says the same thing Most Telegram bot framework maintainers (Telegraf, GramIO, etc.) actually recommend HTML over MarkdownV2 for dynamic content. Some go as far as calling MarkdownV2 "dangerous" when you don't fully control the input — and with LLM output, we definitely don't.

A few concerns with the MarkdownV2 parser here I noticed some things that could bite us — there's a potential out-of-bounds panic in the expandable blockquote handler when the input is short, the link detection relies on a single-char lookbehind that can misfire (e.g. array0), and backslashes aren't being escaped. The test suite covers the happy path nicely but doesn't quite get to these edge cases yet.

The formatting issues are fixable under HTML The screenshots in this PR showing leaked HTML tags are real and annoying, but I think that's more about bugs in the current converter than a fundamental problem with HTML mode. My PR (#935) tackles the chunking and fallback logic, which should clean those up.

All that said — I really appreciate the work in this PR, @Alexandersfg4! The heading-to-bold conversion is a nice touch and something we could totally bring over to the HTML path too.

Curious what you both think!

Hi, thank you for the feedback! I appreciate the thorough review.
The arguments for using HTML are very convincing, especially regarding how messy LLM outputs can be. I agree that HTML is a safer default.
That said, I’d be happy to refine my PR to address your concerns:

  • Add a configuration flag so users can select MarkdownV2 if they prefer (keeping HTML as the default).
{
  "channels": {
    "telegram": {
      "enabled": true,
      "token": "YOUR_BOT_TOKEN",
      "allow_from": ["YOUR_USER_ID"],
      "use_markdown_v2": true
    }
  }
}
  • Improve the MarkdownV2 parser by adding more test cases and fixing the edge cases you pointed out (like the potential panic and backslash escaping).

Let me know if that sounds like a good middle ground!

@alexhoshina
Copy link
Collaborator

Hey @alexhoshina, thanks for flagging this — it's a good discussion to have before we go in two different directions!
After looking at both PRs, I'd lean toward sticking with HTML. Here's my thinking:
LLM output is messy by nature The agent loves throwing around *, _, ., ! and all sorts of characters that happen to be reserved in MarkdownV2 (there are 19+ of them!). One unescaped dot or mismatched asterisk and Telegram rejects the whole message. With HTML, we only need to worry about <, >, and & — which almost never show up in normal responses.
The community generally says the same thing Most Telegram bot framework maintainers (Telegraf, GramIO, etc.) actually recommend HTML over MarkdownV2 for dynamic content. Some go as far as calling MarkdownV2 "dangerous" when you don't fully control the input — and with LLM output, we definitely don't.
A few concerns with the MarkdownV2 parser here I noticed some things that could bite us — there's a potential out-of-bounds panic in the expandable blockquote handler when the input is short, the link detection relies on a single-char lookbehind that can misfire (e.g. array0), and backslashes aren't being escaped. The test suite covers the happy path nicely but doesn't quite get to these edge cases yet.
The formatting issues are fixable under HTML The screenshots in this PR showing leaked HTML tags are real and annoying, but I think that's more about bugs in the current converter than a fundamental problem with HTML mode. My PR (#935) tackles the chunking and fallback logic, which should clean those up.
All that said — I really appreciate the work in this PR, @Alexandersfg4! The heading-to-bold conversion is a nice touch and something we could totally bring over to the HTML path too.
Curious what you both think!

Hi, thank you for the feedback! I appreciate the thorough review. The arguments for using HTML are very convincing, especially regarding how messy LLM outputs can be. I agree that HTML is a safer default. That said, I’d be happy to refine my PR to address your concerns:

  • Add a configuration flag so users can select MarkdownV2 if they prefer (keeping HTML as the default).
{
  "channels": {
    "telegram": {
      "enabled": true,
      "token": "YOUR_BOT_TOKEN",
      "allow_from": ["YOUR_USER_ID"],
      "use_markdown_v2": true
    }
  }
}
  • Improve the MarkdownV2 parser by adding more test cases and fixing the edge cases you pointed out (like the potential panic and backslash escaping).

Let me know if that sounds like a good middle ground!

I think this is feasible

@Alexandersfg4
Copy link
Contributor Author

use_markdown_v2

Hi, I pushed changes with flag use_markdown_v2

@alexhoshina
Copy link
Collaborator

Hi, we merged #935 yesterday, so you might need to resolve the conflicts

@Alexandersfg4
Copy link
Contributor Author

Alexandersfg4 commented Mar 8, 2026

Hi, we merged #935 yesterday, so you might need to resolve the conflicts

@alexhoshina Hi, I resolved the conflicts

Also tested the new feature (when use_markdown_v2=false/true):

Screenshot 2026-03-08 at 11 54 20 Screenshot 2026-03-08 at 11 51 56

@alexhoshina
Copy link
Collaborator

Sorry! I was a bit late in reviewing. Could you please resolve the conflicts again? Thank you very much.

@Alexandersfg4
Copy link
Contributor Author

Alexandersfg4 commented Mar 10, 2026

Sorry! I was a bit late in reviewing. Could you please resolve the conflicts again? Thank you very much.

Hi, I resolved again the conflicts.

@alexhoshina kind tag you, thank you very much!

@alexhoshina
Copy link
Collaborator

make lint plz

@Alexandersfg4
Copy link
Contributor Author

Alexandersfg4 commented Mar 11, 2026

make lint plz

I fixed the linter and fixed MC :)

Kind ping @alexhoshina

}

if err := c.sendHTMLChunk(ctx, chatID, threadID, htmlContent, chunk, replyToID); err != nil {
if err := c.sendChunk(ctx, chatID, threadID, content, chunk, replyToID, useMarkdownV2); err != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter order was passed incorrectly

  • The chunk was passed into the replyToID parameter
  • The actual replyToID was passed into the mdFallback parameter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I was a sick

FIxed the bug

Comment on lines +300 to +302
parsedContent := parseContent(content, useMarkdownV2)
editMsg := tu.EditMessageText(tu.ID(cid), mid, parsedContent).
WithParseMode(telego.ModeMarkdownV2)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current edit request enforces the use of MarkdownV2 parsing mode. When v2 is not enabled, message edits will either fail to parse or fall back

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@Alexandersfg4
Copy link
Contributor Author

@alexhoshina Hi, sorry for the delay. I was out sick, but I've fixed the issues now. Could you please take another look at my PR?

@alexhoshina
Copy link
Collaborator

Thank you for your contribution!

@alexhoshina alexhoshina merged commit 12f4029 into sipeed:main Mar 18, 2026
3 checks passed
j0904 pushed a commit to j0904/picoclaw that referenced this pull request Mar 22, 2026
…eed#1018)

* feat: telegram use parse mode ModeMarkdownV2 instead of ModeHTML

* handle expandable block quotation starts, add test for all md2 formats

* fix: linter issue

* feat: added flag use_markdown_v2, corrected config, updated
documentation

* move parseChatID to parser_markdown_to_html

* fix: tests and linter issues

* fix: case with ~

* test: fixed Test_markdownToTelegramMarkdownV2

* fix: regex block-quote line  >

* fix: linter issues

* fix: send chunk param mismatched, in edit msg use HTML parse mode too

* fix: remove from .gitignore redundant comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants