Skip to content

UnicodeEncodeError when running tortoise makemigrations due to ASCII encoding in migration writer #2096

@chuwei-dev

Description

@chuwei-dev

Describe the bug
When running tortoise makemigrations, a UnicodeEncodeError is raised if any model metadata, field description, choices, or comments contain non-ASCII characters (e.g., Chinese characters).

The issue appears to be caused by the migration writer explicitly using encoding="ascii" when writing migration files.

This prevents projects using non-English metadata from generating migrations successfully.

Environment

  • tortoise-orm: 1.0.0
  • Python: 3.12.3
  • OS: Windows 10
  • Database: SQLite (also reproducible with others)

To Reproduce

  1. Create a model containing non-ASCII characters:
from tortoise import fields, models

class User(models.Model):
    name = fields.CharField(max_length=50, description="用户名")

    class Meta:
        table = "user"
  1. Run:
tortoise makemigrations
  1. Observe the error:
UnicodeEncodeError: 'ascii' codec can't encode characters in position ...

Stack trace shows the failure occurs at:

tortoise/migrations/writer.py  line 266
path.write_text(self.as_string(), encoding="ascii")

Expected behavior
Migration files should be written using UTF-8 encoding (or default system encoding), allowing non-ASCII characters in:

  • field descriptions
  • choices labels
  • Meta attributes
  • comments
  • docstrings

Additional context
Migration generation fails if any non-ASCII characters are present, because migration files are written using ASCII encoding.

Analysis
In tortoise/migrations/writer.py, migration files are written with:

path.write_text(self.as_string(), encoding="ascii")

ASCII encoding cannot represent Unicode characters outside the 0–127 range.

Since Python 3 defaults to UTF-8 in most modern environments, explicitly forcing ASCII seems unnecessary and restrictive.

Suggested Fix
Replace:

path.write_text(self.as_string(), encoding="ascii")

With either:

path.write_text(self.as_string(), encoding="utf-8")

or simply:

path.write_text(self.as_string())

Using UTF-8 would align with modern Python defaults and support internationalized projects.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions