Fix segmentation fault on Python 3.13 #960

shadchin · 2025-11-27T11:55:09Z

Fix for #910 and #868

The problem is only with Python 3.13, in Python 3.14 PyObject_GC_New does this initialization itself, see python/cpython#123192

provinzkraut

Seems like a reasonable fix to me.

I'm curious though how you tracked it down to this?

Also, can you re-enable the regular tests for 3.13 again here?

msgspec/.github/workflows/ci.yml

Lines 63 to 64 in 1bc64dd

    
           # TODO: Enable this once fixed https://github.com/jcrist/msgspec/issues/910 
        
           # - "3.13"

shadchin · 2025-11-27T13:01:35Z

I'm curious though how you tracked it down to this?

It was difficult :) The short version:

From gdb bt found following calls:

4  make_dict_from_instance_attributes
#5  _PyObject_MaterializeManagedDict_LockHeld
#6  _PyObject_MaterializeManagedDict
#7  ensure_managed_dict

In next step, I found that values was not initialized:

(gdb) frame 4
#4  0x00000000023d7c59 in make_dict_from_instance_attributes (interp=0x2fcc2e8 <_PyRuntime+104400>, keys=0x7ffff4e24f20, values=0x7ffff54dc070) at /home/shadchin/arc/arcadia/contrib/tools/python3/Objects/dictobject.c:6736
6736                track += _PyObject_GC_MAY_BE_TRACKED(val);
(gdb) info locals
val = <unknown at remote 0xcdcdcdcdcdcdcdcd>
i = 0
used = 1
track = 0
size = 30
res = 0x7ffff1b535b0
(gdb) print values
$1 = (PyDictValues *) 0x7ffff54dc070
(gdb) print keys
$2 = (PyDictKeysObject *) 0x7ffff4e24f20
(gdb) x/10gx values
0x7ffff54dc070: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc080: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc090: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc0a0: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd
0x7ffff54dc0b0: 0xcdcdcdcdcdcdcdcd      0xcdcdcdcdcdcdcdcd

Then I had to study what managed dict/PyDictValues and Struct_alloc. I found that it's true for 3.13 that we don't initialize PyDictValues, looked at the CPython code, how it does it, found the _PyObject_InitInlineValues call.

ofek · 2025-11-27T15:42:38Z

Do we know why 3.14 is fine?

provinzkraut · 2025-11-27T15:55:49Z

@shadchin seems like _PyObject_InitInlineValues isn't available?

https://github.com/jcrist/msgspec/actions/runs/19736064121/job/56566395047?pr=960#step:8:228

shadchin · 2025-11-27T17:31:47Z

Do we know why 3.14 is fine?

Python 3.14+ does this initialization itself - https://github.com/python/cpython/blob/main/Python/gc.c#L2377-L2379

provinzkraut · 2025-11-27T18:01:20Z

src/msgspec/_core.c

    memset((char *)obj + sizeof(PyObject), '\0', type->tp_basicsize - sizeof(PyObject));
+#if PY313_PLUS && !PY314_PLUS
+    if (type->tp_flags & Py_TPFLAGS_INLINE_VALUES) {
+        _PyObject_InitInlineValues(obj, type);


Maybe it's worth asking upstream about how to deal with this without relying on private APIs?

Not like we don't rely on private APIs elsewhere, so I wouldn't consider it a blocker, but if there's a way, that might be nice?

shadchin · 2025-11-27T18:51:26Z

@shadchin seems like _PyObject_InitInlineValues isn't available?

https://github.com/jcrist/msgspec/actions/runs/19736064121/job/56566395047?pr=960#step:8:228

We have our own build system, fix is working for us, but I forgot to check on a clean environment, sorry. I tried to make a fix but it didn't help. There's a conflict with write_u64.

I'll go upstream with it.

shadchin · 2025-12-13T16:45:17Z

I took a different approach and it helped)

Maybe something else will prompt in upstream.

ZeroIntensity

Coming from upstream

ZeroIntensity · 2025-12-16T13:51:57Z

src/msgspec/_core.c

+#endif
+
    if (is_gc) {
        obj = PyObject_GC_New(PyObject, type);


Manually calling PyObject_GC_New/PyObject_New doesn't really work for subtypes. It might work in practice sometimes, but it's not something that's supposed to be supported.

I believe the root of this bug comes from the fact that on 3.13, inline values are initialized in the tp_alloc slot of type objects, which is skipped here. In 3.14, we moved that logic to PyObject_GC_New because we decided to support more types.

So, to fix this, just call type->tp_alloc instead; that will invoke the supertype (PyType_GenericAlloc) and initialize the inline values.

Thanks, I switched to type->tp_alloc

ZeroIntensity · 2025-12-16T13:53:16Z

src/msgspec/_core.c

    bool is_gc = MS_TYPE_IS_GC(type);

+#if PY313_PLUS && !PY314_PLUS
+    type->tp_flags &= ~Py_TPFLAGS_INLINE_VALUES;


I'm almost certain you'll trigger assertion failures (or worse, get spurious crashes) with this. Supertypes don't expect their flags to be different in subtypes.

ofek · 2025-12-17T07:10:52Z

Thanks! How would that change affect performance?

shadchin · 2025-12-19T07:38:54Z

Thanks! How would that change affect performance?

https://github.com/jcrist/msgspec/blob/main/benchmarks/bench_structs.py

main:

+----------------------+-------------+-------------+---------------+------------+
|                      | import (μs) | create (μs) | equality (μs) | order (μs) |
+======================+=============+=============+===============+============+
| **msgspec**          | 12.82       | 0.10        | 0.02          | 0.03       |
+----------------------+-------------+-------------+---------------+------------+
| **standard classes** | 9.68        | 0.43        | 0.07          | 0.21       |
+----------------------+-------------+-------------+---------------+------------+
| **attrs**            | 466.53      | 0.43        | 0.07          | 2.10       |
+----------------------+-------------+-------------+---------------+------------+
| **dataclasses**      | 606.02      | 0.44        | 0.19          | 0.21       |
+----------------------+-------------+-------------+---------------+------------+
| **pydantic**         | 320.21      | 1.84        | 1.01          | N/A        |
+----------------------+-------------+-------------+---------------+------------+

this pr:

+----------------------+-------------+-------------+---------------+------------+
|                      | import (μs) | create (μs) | equality (μs) | order (μs) |
+======================+=============+=============+===============+============+
| **msgspec**          | 13.75       | 0.11        | 0.02          | 0.03       |
+----------------------+-------------+-------------+---------------+------------+
| **standard classes** | 10.17       | 0.46        | 0.08          | 0.22       |
+----------------------+-------------+-------------+---------------+------------+
| **attrs**            | 497.77      | 0.48        | 0.07          | 2.24       |
+----------------------+-------------+-------------+---------------+------------+
| **dataclasses**      | 651.33      | 0.48        | 0.20          | 0.21       |
+----------------------+-------------+-------------+---------------+------------+
| **pydantic**         | 340.21      | 1.86        | 1.08          | N/A        |
+----------------------+-------------+-------------+---------------+------------+

https://github.com/jcrist/msgspec/blob/main/benchmarks/bench_gc.py

main:

+-----------------------------------+--------------+-------------------+
|                                   | GC time (ms) | Memory Used (MiB) |
+===================================+==============+===================+
| **standard class**                | 133.60       | 204.03            |
+-----------------------------------+--------------+-------------------+
| **standard class with __slots__** | 96.68        | 120.11            |
+-----------------------------------+--------------+-------------------+
| **msgspec struct**                | 24.18        | 120.11            |
+-----------------------------------+--------------+-------------------+
| **msgspec struct with gc=False**  | 2.38         | 104.85            |
+-----------------------------------+--------------+-------------------+

this pr:

+-----------------------------------+--------------+-------------------+
|                                   | GC time (ms) | Memory Used (MiB) |
+===================================+==============+===================+
| **standard class**                | 132.79       | 204.03            |
+-----------------------------------+--------------+-------------------+
| **standard class with __slots__** | 97.56        | 120.11            |
+-----------------------------------+--------------+-------------------+
| **msgspec struct**                | 23.94        | 120.11            |
+-----------------------------------+--------------+-------------------+
| **msgspec struct with gc=False**  | 2.39         | 104.85            |
+-----------------------------------+--------------+-------------------+

python -VV
Python 3.12.11 (main, Jun  4 2025, 08:56:18) [GCC 9.4.0]

Fix segmentation fault on Python 3.13

452fddb

shadchin mentioned this pull request Nov 27, 2025

Segmentation fault on Python 3.13 when running under ASan #910

Open

provinzkraut approved these changes Nov 27, 2025

View reviewed changes

Re-enable the regular tests for 3.13

d14b921

Add include

639f7ee

provinzkraut reviewed Nov 27, 2025

View reviewed changes

shadchin mentioned this pull request Dec 13, 2025

Crash msgspec with Python 3.13.* python/cpython#142671

Open

Disable Py_TPFLAGS_INLINE_VALUES

6ffe4e3

ZeroIntensity reviewed Dec 16, 2025

View reviewed changes

Switch to type->tp_alloc

24da222

	# TODO: Enable this once fixed https://github.com/jcrist/msgspec/issues/910
	# - "3.13"

Fix segmentation fault on Python 3.13 #960

Are you sure you want to change the base?

Fix segmentation fault on Python 3.13 #960

Uh oh!

Conversation

shadchin commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

provinzkraut left a comment

Choose a reason for hiding this comment

Uh oh!

shadchin commented Nov 27, 2025

Uh oh!

ofek commented Nov 27, 2025

Uh oh!

provinzkraut commented Nov 27, 2025

Uh oh!

shadchin commented Nov 27, 2025

Uh oh!

provinzkraut Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

shadchin commented Nov 27, 2025

Uh oh!

shadchin commented Dec 13, 2025

Uh oh!

ZeroIntensity left a comment

Choose a reason for hiding this comment

Uh oh!

ZeroIntensity Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

shadchin Dec 19, 2025

Choose a reason for hiding this comment

Uh oh!

ZeroIntensity Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

ofek commented Dec 17, 2025

Uh oh!

shadchin commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

shadchin commented Nov 27, 2025 •

edited

Loading

shadchin commented Dec 19, 2025 •

edited

Loading