Additional metadata with JSON/Telegram import

[Terrance]

I'm looking to import my XMPP chat history into Signal from Conversations, which provides an encrypted database backup and standalone media files. I can already unpack the backups into a SQLite database, and the JSON/Telegram importer here seems like the easiest route forward.

I've started writing a script to build a result.json file and the media folder structure, but Conversations has additional metadata for reactions and delivery status which would be nice to maintain. These aren't in Telegram's backups or schema -- would it be possible to extend the supported schema (either in the main JSON file itself or as a standalone extra file) and add fields for these?

[bepaald]

Hi!

Interesting. I have no problem extending the schema to add reactions and delivery receipts, they would simply not be present in real Telegram exports, but that should not cause any problems and just keep working as it does now.

So I suggest within a message object, to add an (optional) array of reaction-obejcts (bold part new):

    {
     "id": 2,
     "type": "message",
     "date": "2023-10-19T22:54:23",
     "date_unixtime": "1697738864",
     "from": "Devphone",
     "from_id": "user123",
     "photo": "chats/chat_2/photos/photo_1@19-10-2023_22-55-44.jpg",
     "width": 720,
     "height": 1280,
     "text": "Message-body",
     "text_entities": [
      {
       "type": "plain",
       "text": "Message-body"
      }
     ],
     "reactions": [
      {
       "emoji":"😀",
       "author":"user123",
       "timestamp":1697739999
      },
      {
        [another reaction...]
      },...
     ]
    }

Would that work? For delivery status, probably the same thing, except instead of "emoji":"", it would have "status":"read/delivered"?

Of course, I still need to implement it afterwards, but I think it shouldn't be too difficult (assuming the importjson function is still working correctly, I don't think it's used often).

Thanks!

EDIT Probably better to rename the new fields to something unlikely, in case Telegram ever decides to export reactions officially. Like custom_reactions, or something similar.

[Terrance]

I've got a seemingly-working script that brings all my messages with the currently-supported fields (not yet tried actually replacing Signal's data with the backup, but the exports from signalbackup-tools look sane for both new and existing contact imports), so I think the JSON import is working as advertised. 🙂

Suggested format seems fine to me. Conversations has a few message states, including some 'undelivered' ones -- not sure if Signal has an equivalent that can be set; if not then I'll just drop those from my import.

[bepaald]

I've quickly added some initial code to import reactions from a custom_reactions field in the json. Notes:

1. The hard part of the json-import is matching the json-contacts to the signal-contacts. For this, all contacts in the json file are gathered and a mapping is attempted. When mapping for some contacts fails, you are warned and asked to supply a manual mapping for them. Currently, none of this is done for contacts listed as reaction-authors! If a contact appears as a reaction author and it is not already mapped, the reaction will simply not be added.
2. While implementing and (very brief) testing, I was reminded reactions in Signal do not have a single timestamp: they have a date_sent and a date_received. Currently I'm parsing timestamp, as suggested in my previous message, and inserting it for both received and sent dates. We could change this if you have separate timestamps available in your Conversations-data.
3. Signal uses timestamps in milliseconds (mostly), while the telegram-json uses seconds. I wasn't sure what you had available, but the current code will handle both, making a guess about the input based on the number of digits in the timestamp.

So, this is not complete yet, but I thought I'd push it so you could start testing when you have time.

---

Conversations has a few message states, including some 'undelivered' ones -- not sure if Signal has an equivalent that can be set; if not then I'll just drop those from my import.

I believe the delivery_receipts are simply 'none', 'delivered' or 'read', with each successive one overwriting the previous. 'None' implies undelivered, so for those, just make no entry in the array.

There are of course also fail states, but they are more a local state of the message and not a property in relation to some recipient. If that makes sense. Any fail-state comes down to a failure to send the message to Signal's servers, but this is not a receipt between you and some recipient. So given an outgoing group message to contacts A, B, and C. Either the message-type is SENT_FAILED, or the message type is SENT, and there are 'delivery_receipts' for contacts A, B, and C which could differ in 'delivered', 'read', or nothing at all (meaning not delivered).

I think I can quite easily set a message type to SENT_FAILED for some delivery reports if you want, but for these cases, the state of all recipients of that message must be failed. Given an array of delivery reports:

"custom_delivery_reports": [
      {
       "state":"delivered/read/...",
       "recipient":"user123",
       "timestamp":1697739999
      },
      {
        [another delivery...]
      },...
     ]

What values of state do you have available from your Conversations-data?

EDIT Just checking over the database: for group messages, Signal does have explicit 'undelivered' delivery receipts. Additionally, there are receipts indicating 'viewed' and 'skipped'. I'm not sure what they mean, I've never seen them in a database (only in the source code). 'Viewed' does seem to have an intuitive meaning, but I don't know how it differs from 'read' then...

[Terrance]

Thanks for this! I'll give it another go later today.

If a contact appears as a reaction author and it is not already mapped, the reaction will simply not be added.

That shouldn't be a problem for me at least as I'm only working with 1-to-1 chats, and in fact all the reactions are from me. I'm currently doing the mapping manually: chat and user IDs are just being written as 0, 1, ... and I'm passing --mapjsoncontacts to assign each recipient to a suitable contact.

Currently I'm parsing timestamp, as suggested in my previous message, and inserting it for both received and sent dates.

That's fine, Conversations doesn't track the read date as far as I can see.

Signal uses timestamps in milliseconds (mostly), while the telegram-json uses seconds.

I initially tried sending milliseconds as seconds with a decimal part but ended up with half the messages being set to epoch zero / 1970-01-01, so they're currently being rounded to seconds which worked fine. I'll try it again sending plain milliseconds.

What values of state do you have available from your Conversations-data?

These are the statuses I'm working with:

STATUS_RECEIVED = 0;  // All incoming messages
STATUS_UNSEND = 1;  // (None exist) Preparing to send?
STATUS_SEND = 2;  // (Only old messages) Sent before delivered/read tracking?
STATUS_SEND_FAILED = 3;  // Given up trying to send
STATUS_WAITING = 5;  // (None exist) Waiting for connection
STATUS_OFFERED = 6;  // (None exist) File transfer pending accept from recipient
STATUS_SEND_RECEIVED = 7;  // Delivered to recipient
STATUS_SEND_DISPLAYED = 8;  // Delivered to and read by recipient

Incoming messages are all status 0. I'm only importing private chats, no groups, so I think it works if I drop the handful of 3s, then set 8 to "read", 7 to "delivered", and 0 and 2 to none (unless it makes more sense to set incoming messages as "read"?).

[Terrance]

Importing reactions seems to work great. 👍

Signal uses timestamps in milliseconds (mostly), while the telegram-json uses seconds.

I initially tried sending milliseconds as seconds with a decimal part but ended up with half the messages being set to epoch zero / 1970-01-01, so they're currently being rounded to seconds which worked fine. I'll try it again sending plain milliseconds.

Sending message timestamps as milliseconds gives me 5-digit years (e.g. 1752162784778 = 57493-10-29), i.e. they're being treated as seconds, for the message dates (it's handled correctly for reactions). I've just put it back to seconds again -- tbh I'm not overly worried about millisecond accuracy for messages with network latency and the like affecting it anyway.

---

One other feature I've just spotted in my message archive is embedded location (i.e. a map pin) in messages -- these are represented by Conversations as a geo URI, but Telegram exports already include a location field (Message.location_information) and type which I can convert it to. I can see Signal also supports sharing location, so any chance of adding a mapping for this too?

[bepaald]

Sending message timestamps as milliseconds gives me 5-digit years (e.g. 1752162784778 = 57493-10-29), i.e. they're being treated as seconds, for the message dates (it's handled correctly for reactions). I've just put it back to seconds again -- tbh I'm not overly worried about millisecond accuracy for messages with network latency and the like affecting it anyway.

O, yeah, sorry for the confusion, the milliseconds-support was only for these newly added custom_ fields, not for the existing Telegram data. But I just quickly changed it to work for the whole json file (hopefully).

These are the statuses I'm working with:

[...]

Incoming messages are all status 0. I'm only importing private chats, no groups, so I think it works if I drop the handful of 3s, then set 8 to "read", 7 to "delivered", and 0 and 2 to none (unless it makes more sense to set incoming messages as "read"?).

Ok, the accepted statuses in the json are: failed, sending, delivered, and read. I think what you suggested will work, except you do not have to drop the 3's (though you could if you wanted to).

The current state of the delivery report import is the same as with the reactions: largely untested and without checks for the recipients. Also, deviating from what I wrote earlier, the format is:

"custom_delivery_reports": [
 {
  "status":"delivered/read/sending/failed",
  "recipient":"user123",
  "timestamp":1697739999
 },
 {
 },...
]

Note the status instead of state.

---

One other feature I've just spotted in my message archive is embedded location (i.e. a map pin) in messages -- these are represented by Conversations as a geo URI, but Telegram exports already include a location field (Message.location_information) and type which I can convert it to. I can see Signal also supports sharing location, so any chance of adding a mapping for this too?

While Signal does have an interface for sharing location, there is no actual location-message-type that is any different from any other message. The interface simply allows access to the phone's location services and lets you pick a place on Google maps. The message is then just a normal message with a Google maps-link in the body (an address is also prepended if available, and a screenshot of the map as an attachment). There is nothing that differentiates such a 'location message' from any other message with a maps-link manually typed out (or pasted) into the body and an attachment added.

So, I could convert the longitude and latitude values into a Google maps-link. But that's all I can do. To add the address and screenshot would be way too much work: it would require the tool to have internet access (which is unwanted anyway) and I assume accessing the Google maps API is also technically not allowed without some sort of account.

Would you like the tool to convert location messages into normal messages with body = https://maps.google.com/maps?q=<lat>,<lng>?

Thanks!

[bepaald]

Location information is now imported as well. A Google maps-link is generated and appended to the message body. The location information is expected in the format Telegram exports it:

     {
      "id": 53,
      "type": "message",
      "date": "2025-07-14T10:21:53",
      "date_unixtime": "1752481313",
      "from": "Devphone",
      "from_id": "user123",
      "location_information": {
       "latitude": 30.037388,
       "longitude": 31.247435
      },
      "text": "",
      "text_entities": []
     },

Note, while Telegram does not add anything else in a location message, I think adding a screenshot and a message body (an address for example) should work correctly if you have either available in your source data.

---

Also, now that I've exported some fresh Telegram data, it's good we decided on the custom_-prefix, as Telegram now apparently also exports reactions. I'll support those at some point in the future as well (in addition to the custom_reactions that are already supported now).

[Terrance]

I think these are all working perfectly -- I have GMaps links for locations and delivery statuses both showing nicely in the HTML export. 🙂 Just getting the last contact onboarded with Signal and then I'll have a go at importing these properly.

Thanks again for the quick turnaround!

[bepaald]

Glad to hear it. I did slightly rewrite the reaction and delivery report handling functions (the reaction function should now also handle the official Telegram reaction json object), so a quick re-check might be in order. Though I think the functions result in the same output.

then I'll have a go at importing these properly.

This is the part that's making me nervous... Let me know how it goes! (no hurry of course, take your time).

Thanks!

[Terrance]

Backups now imported, and looks like everything's present and correct!

Only hit one snag which was probably my bad: I imported both sides of a conversation between myself and another party who just joined Signal (using two separate Conversations backups), and while my side correctly merged the imported messages into our new Signal thread, their imported messages ended up in a separate thread (i.e. they had two chats for me). I think this happened because I'd mapped them to my phone number thread, but the new Signal thread was tied to my username instead (I have privacy on and apparently Signal then treats those as separate). Thankfully there's a --mergerecipients option here which tidied this up nicely, and we now have the same coherent thread on both sides. 😀

[bepaald]

That's great to hear! Do make sure to scroll around a little in through the newly imported messages, sometimes bugs only show themselves when the messages are coming into view in Signal.

In previous commits I had already added some of the missing checks for the reaction and delivery report functions (to make sure the recipients were mapped) and to support the Telegram-style reaction json also. So, if all is working for you I think this issue is done?

Thanks!

[Terrance]

I've looked for a few of the interesting ones I'm aware of (media, locations, delivery failures) but I don't think there are any problems to report, so nothing more to do here.

---

For anyone following along at home, here's the script I wrote to convert the Conversations data to Telegram's JSON format. Media directories may be at /Conversations/Media or /Pictures/Conversations on Android's internal storage -- copy them all to another folder you can reference. The steps should therefore look something like:

# Conversations (encrypted) -> Conversations (SQLite)
.../conversations-backup-decrypt.py -i .../user@example.com.2025-07-18-00-00.ceb -o ./user-conv.db
# Conversations (SQLite) -> Telegram (JSON)
.../con2tg4sig.py ./user-conv.db .../user-media/ ./user-json/ <chat UUIDs>

# Find thread IDs for next command
signalbackup-tools .../signal-2025-07-18-00-00-00.backup --listthreads
# Telegram (JSON) -> Signal
signalbackup-tools .../signal-2025-07-18-00-00-00.backup --importjson user-json/result.json \
  --mapjsoncontacts -1=<your thread ID>,0=<first chat's thread ID>,1=<second's ID>,... -o ./user-signal.backup

Then copy the new backup file to your phone, reset Signal, and import it when promped to restore.