v0.2.1

Major changes

• [Python] Supported wheels and builds for Python 3.13 and 3.14(rc1) #1134, #1127, #1121, #1111, #1110, #1104, #1103, #1099, #1091
• [Python] Added an experimental support for free-threading. #1134, #1127, #1110 https://github.com/google/sentencepiece/tree/master/python#free-threading-support
• [Python] Updated the supported Python version to 3.9 or later.

New features

• [ALL]: Added new build mode to prevent the precompiled normalization rules being embedded in *.so and *.a. (-DSPM_DISABLE_EMBEDDED_DATA=ON). This reduces the runtime size by approximately 1-2 MB. This mode is enabled to build python wheels. The rules are loaded as the data package.

Bug fixes & minor changes

• [ALL]: Security fix to address a heap overflow issue that could occur when using a model containing an invalid precompiled normalization model.
• [Python]: Deprecates the wheel package for Linux i686.
• [Python]: Supported wheel for Windows Arm64. #1114
• [Python]: Fixed the crash issue on batch decoding #1051
• [ALL]: Updated the Unicode normalization rule with the latest ICU/Unicode rules.
• [ALL]: Unused code and build mode cleanup.