Generate native JS functions from string declarations to achieve automatic tree-shaking

[MurzNN]

There is another great project that provides translations - Paraglide JS, and the killer feature of it is tree-shakable translations!

And, I believe, we can implement the same approach in Lingui too, as we have all the background for this already now!

So, instead of using a function like this:

import { i18n } from "@lingui/core";
i18n._(
  /** i18n */ {
    id: "nwR43V",
    message: "Attachment {name} saved",
    values: { name },
  }
);

we can generate a native JS function like this:

export const nwR43V = (locale: string, values?: Record<string, string>) => {
  if(locale === 'en') {
    return "Hello"
  }
  if(locale === 'pl') {
    return "Cześć"
  }
}

So the macro will transform a string from this:

t`Hello`;

to just this:

nwR43V(i18n.locale)

and this will be bundled into the target JS code inline, without needing to include the full array of translation messages at all!

Thinking forward, we can even prepare functions for concrete locales like this:

export const nwR43V_en = (values?: Record<string, string>) => {
    return "Hello"
}
export const nwR43V_pl = (values?: Record<string, string>) => {
    return "Cześć"
}

and call directly the function in the right locale, if the locale is static on the transpiling stage.

[timofei-iatsenko]

To have a tree shaking working for the language - you need to bundle your app for each locale. If by tree-shaking, you and Paraglide author means code splitting, when different messages could go to a diffrent chunks of a bundle - yes, that actually possible, but not sure current bundlers could split the common files. Anyway - i like the idea, this is doable, however this would be a huge change in the architecture and shift in the paradigm of the lingui. Not sure how integrate this without a massive disruption for a users.

[MurzNN]

They have a completely different approach. They are "message first", where is lingui is a "code first"

This is exactly why I am a fan of Linugui! Not I18Next, Paraglide JS, and others... Declaring the strings directly in the code without inventing unique ids for each string manually is amazing!

So, I don't ask to break this paradigm and get back to the manual ids at all!

I just want to find ways to resolve two conceptual issues of the current Lingui implementation, that are pretty major for my projects:

1. Lingui doesn't work in pure JS without macro support.

2. Lingui doesn't support tree-shaking when rendering the page.

And I believe, these limitation are crucial for many others projects too, but many of them just decline Lingui and use other solutions, because of these issues, without flooding the issue tracker like me 😉

[MurzNN]

So, let me elaborate on the 2nd, "tree shaking":

Let's get a React project having in total 1000 translation strings and 5 languages.

And get a single page, which just renders one translatable string "Hello" in a single language, eg, pl.

• With Paraglide JS, we load all languages at once, but a single function with one string "m.hello()". So, 5 strings in total is loaded to the frontend browser, instead of one string in Polish.

• With Lingui - yes, we load only a single language pl, not all languages at once, that's great!
  But, to translate only one string "Hello", we load all 1000 strings for all pages into the frontend browser!

So, in comparison, 1000 strings in Lingui, instead of 5 strings in Paraglide JS.

Which page will be loaded faster? ;)

P.S. I don't want to say that Lingui approach is bad, and Paraglide JS is good! Lingui it is good for many cases, and provides much better developer experience!

But for this specific case, Paraglide does the job much better. So, I just want to discuss how we can make Lingui even better, with true tree-shaking!

[andrii-bodnar]

Thanks for the detailed proposal, @MurzNN - the concern is legitimate, but I don't think this is the right trade-off for us right now.

You're right that loading a full locale catalog on a page that uses only a handful of messages is wasteful, especially for multi-page apps. That's a real pain point.

But compiling every message into a standalone JS function is a fundamental change to how Lingui transforms and loads messages - macros, compiler, runtime, and the whole catalog model. That's a lot of surface area and maintenance cost, and as @timofei-iatsenko noted, the tree-shaking payoff is limited: locale is almost always a runtime value, so bundlers can't statically eliminate unused branches. Big architectural shift, benefit that mostly doesn't materialize - not justified for now.

I think the real fit here is the experimental dependency-tree extractor (https://lingui.dev/guides/message-extraction#dependency-tree-crawling). Instead of one monolithic catalog, it crawls imports from your entry points and scopes each catalog to what that page actually uses - so pages don't load messages they don't need. That targets your concern directly, without changing how messages are transformed. The natural follow-up would be emitting a separate chunk per entry point to load on demand - an incremental step on top of something that already exists, not a rewrite.

Would love to hear whether that approach covers your use case, or where it falls short.

[timofei-iatsenko]

And get a single page, which just renders one translatable string "Hello" in a single language, eg, pl.

With Paraglide JS, we load all languages at once, but a single function with one string "m.hello()". So, 5 strings in total is loaded to the frontend browser, instead of one string in Polish.

With Lingui - yes, we load only a single language pl, not all languages at once, that's great!
But, to translate only one string "Hello", we load all 1000 strings for all pages into the frontend browser!

So, in comparison, 1000 strings in Lingui, instead of 5 strings in Paraglide JS.

This is quite synthetic example. Very far from a real life. Also tree-shking, and code splitting in a real life scenarios works much different that people expect. So in the real life you would probably have your page with 1 message, and also bunch of common chunks with many messages not used on this page (again, because tree shaking is very picky, it's designed to be very conservative to not broke your app).

So both of the approaches should be tested on the real app and measured which works better.

The approach from ParaglideJs is blowing application size when number of supported langauges grows - which user never would need.

The approach from LinguiJS - catalogs grows - when the application grows. Which user might need, because sooner or later might use this functionality.

If you want compare 1 to 1, if you would use a "experimental extractor", for the same app/page paraglide would have 5 strings, lingui would have 1.

Also Lingui supports OTA delivery, so you can edit translations on the fly without app rebuild/republish. With paragliadejs approach it's not possible at all.

Lingui doesn't work in pure JS without macro support.

There is a plan for this. And if you want you can actively participate, i can guide you thru the process and code.

[MurzNN]

This is quite synthetic example. Very far from a real life. Also tree-shking, and code splitting in a real life scenarios works much different that people expect. So in the real life you would probably have your page with 1 message, and also bunch of common chunks with many messages not used on this page (again, because tree shaking is very picky, it's designed to be very conservative to not broke your app).

With server-side rendered pages in frameworks like Next.js, when the user opens a single concrete page in the browser, usually 95% of the messages are translated on the server side, and only 5% of them should come to the client side. And when the user clicks on a link to open another page, again the server prepares the response and translates the most of messages for the new page. So we don't need to download to the browser all the messages for all further pages too, just a tiny part of concrete messages that are needed for the frontend-side translation.

So, this example is actually not very far from a real life in a modern SSR world 😉

About downsides of ParaglideJS - I completely agree with your points, in that specific areas Lingui does the job much better!
The only part, missing in Lingui, is a per-page tree shaking, where ParaglideJS has a better, but still not ideal solution.

So, I am just trying to discus how we, taking this ParaglideJS approach into consideration, make Lingui better with per-page tree shaking!

the experimental dependency-tree extractor (https://lingui.dev/guides/message-extraction#dependency-tree-crawling).

That is awesome, missed this feature for some reason... Will try to apply it in my projects and share the feedback!

Lingui doesn't work in pure JS without macro support.

There is a plan for this. And if you want you can actively participate, i can guide you thru the process and code.

Awesome, I really want to participate and test this feature! Could you please describe the steps to go into this? Maybe even better to make this as a separate open discussion here in GitHub, to involve there other people interested in testing and discussing this feature.

P.S. We have a Discord server for discussions, but chat-based discussions are not very convenient for long-standing feature discussions and involving new people, so GitHub Discussions work much better for such cases.