Skip to content

Make py_zstd_compress_mt2 thread-safe - #321

Open
KowalskiThomas wants to merge 5 commits into
sergey-dryabzhinsky:masterfrom
KowalskiThomas:kowalski/fix-thread-safety
Open

Make py_zstd_compress_mt2 thread-safe#321
KowalskiThomas wants to merge 5 commits into
sergey-dryabzhinsky:masterfrom
KowalskiThomas:kowalski/fix-thread-safety

Conversation

@KowalskiThomas

@KowalskiThomas KowalskiThomas commented Jun 19, 2026

Copy link
Copy Markdown

What is this PR?

This PR is a fix for a crash we witnessed in a service of ours. py_zstd_compress_mt2 used a single global compression context (at the module level), m_cctx, which is thus shared across all calls:

python-zstd/src/python-zstd.h

Line 130 in fec54dd

static ZSTD_CCtx* m_cctx;

python-zstd/src/python-zstd.c

Lines 279 to 281 in fec54dd

return NULL;
}
ZSTD_CCtx_setParameter(cctx, ZSTD_c_compressionLevel, level);

Since ZSTD_CCtx is not thread-safe, and because compression releases the GIL, two Python threads calling these functions at the same time use the same context concurrently. This can corrupt the state and lead to out-of-bounds access inside zstd.

The crash was detected in this form:

ZSTD_compressBlock_doubleFast_noDict_generic   zstd/lib/compress/zstd_double_fast.c:185
ZSTD_compressBlock_doubleFast_noDict_4         zstd/lib/compress/zstd_double_fast.c:557
ZSTD_compressBlock_doubleFast                  zstd/lib/compress/zstd_double_fast.c:577
ZSTD_buildSeqStore                             zstd/lib/compress/zstd_compress.c:3420
ZSTD_compressBlock_internal                    zstd/lib/compress/zstd_compress.c:4399
ZSTD_compress_frameChunk                       zstd/lib/compress/zstd_compress.c:4646
ZSTD_compressContinue_internal                 zstd/lib/compress/zstd_compress.c:4834
ZSTD_compressEnd_public                        zstd/lib/compress/zstd_compress.c:5412
ZSTD_compressStream_generic                    zstd/lib/compress/zstd_compress.c:6151
ZSTD_compressStream2                           zstd/lib/compress/zstd_compress.c:6540
ZSTD_compressStream2_simpleArgs                zstd/lib/compress/zstd_compress.c:6561
ZSTD_compress2                                 zstd/lib/compress/zstd_compress.c:6581
py_zstd_compress_mt2                           src/python-zstd.c:276
_PyObject_MakeTpCall
_PyEval_EvalFrameDefault
_PyObject_MakeTpCall
_PyEval_EvalFrameDefault

Looking further into it, we found another race as well. When called with different level values, reset_cContext frees and recreates m_cctx. If one thread does this while another is still inside the GIL-released ZSTD_compress2, the latter compresses into a freed context.

Proposed fix

The fix proposed in this PR is to make py_zstd_compress_mt2 create and free its own ZSTD_CCtx on each call, instead of sharing the module-global one. This is the same pattern already used by py_zstd_compress_mt:

python-zstd/src/python-zstd.c

Lines 141 to 149 in fec54dd

cctx = ZSTD_createCCtx();
ZSTD_CCtx_setParameter(cctx, ZSTD_c_compressionLevel, level);
ZSTD_CCtx_setParameter(cctx, ZSTD_c_nbWorkers, threads);
Py_BEGIN_ALLOW_THREADS
cSize = ZSTD_compress2(cctx, dest, (size_t)dest_size, source, (size_t)source_size);
Py_END_ALLOW_THREADS
ZSTD_freeCCtx(cctx);

@KowalskiThomas
KowalskiThomas marked this pull request as ready for review June 19, 2026 12:56
@KowalskiThomas
KowalskiThomas force-pushed the kowalski/fix-thread-safety branch from 645526c to 693cd72 Compare June 19, 2026 12:57
@sergey-dryabzhinsky

sergey-dryabzhinsky commented Jun 20, 2026

Copy link
Copy Markdown
Owner

I intentionally created context only one time, hoped it speed things up.
Is there easy way to create test for it?

@KowalskiThomas

KowalskiThomas commented Jun 20, 2026

Copy link
Copy Markdown
Author

I intentionally created context only one time, hoped it speed things up.

That's what I assumed 😅

I think if you want to keep caching contexts, I could change the code to have a kind of pool for contexts and re-use them across calls while guaranteeing a single context is never used for more than one call at a time.

Is there easy way to create test for it?

If you want a reproducer, I don't know if it's possible to create one easily, not one that will consistently fail anyway (the crash is rare). Maybe under free-threaded Python, it would show up more. I can try to give it a look.

@sergey-dryabzhinsky

sergey-dryabzhinsky commented Jun 20, 2026

Copy link
Copy Markdown
Owner

a kind of pool for contexts
Then I'll look for it.

I can try to give it a look.
Shure, I'll wait.

@KowalskiThomas

KowalskiThomas commented Jun 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

I have a reproducer now, which consistently triggers crashes even with the GIL (and triggers TSan, on a free-threaded TSan build). I think this could be added as a regression test after the fix is added.

import threading

import zstd

# Large payload so each call takes long enough for threads to overlap.
DATA: bytes = b"hello world this is a test " * 200_000  # 5 MB

THREADS: int = 64
ITERS: int = 500
LEVELS: list[int] = [1, 3, 5, 9, 3, 1]  # mix of levels triggers reset_cContext

errors: list[Exception] = []
lock = threading.Lock()

def compress_loop() -> None:
    for i in range(ITERS):
        level = LEVELS[i % len(LEVELS)]
        try:
            zstd.compress2(DATA, level)
        except Exception as exc:
            with lock:
                errors.append(exc)


threads = [threading.Thread(target=compress_loop) for _ in range(THREADS)]

for t in threads:
    t.start()
for t in threads:
    t.join()

Output

% python3 repro.py
<frozen importlib._bootstrap>:549: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'zstd', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.
==================
WARNING: ThreadSanitizer: data race (pid=11580)
  Read of size 8 at 0x0001139026b0 by thread T2:
    #0 ZSTD_freeCCtx <null> (zstd.cpython-316t-darwin.so:arm64+0x10610)
    #1 py_zstd_compress_mt2 <null> (zstd.cpython-316t-darwin.so:arm64+0x1014)
    #2 cfunction_call methodobject.c:575 (python.exe:arm64+0x1001387b0)
    #3 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #4 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #5 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #6 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #7 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #8 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #9 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #10 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #11 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #12 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #13 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #14 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #15 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #16 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
    #17 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #18 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #19 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #20 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #21 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #22 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #23 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #24 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #25 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #26 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #27 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #28 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #29 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #30 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #31 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Previous write of size 8 at 0x0001139026b0 by thread T1:
    #0 ZSTD_CCtx_init_compressStream2 <null> (zstd.cpython-316t-darwin.so:arm64+0x1c1d4)
    #1 ZSTD_compressStream2 <null> (zstd.cpython-316t-darwin.so:arm64+0x1bc24)
    #2 ZSTD_compress2 <null> (zstd.cpython-316t-darwin.so:arm64+0x14824)
    #3 py_zstd_compress_mt2 <null> (zstd.cpython-316t-darwin.so:arm64+0x1090)
    #4 cfunction_call methodobject.c:575 (python.exe:arm64+0x1001387b0)
    #5 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #6 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #7 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #8 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #9 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #10 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #11 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #12 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #13 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #14 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #15 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #16 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #17 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #18 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
    #19 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #20 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #21 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #22 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #23 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #24 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #25 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #26 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #27 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #28 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #29 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #30 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #31 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #32 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #33 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Location is heap block of size 5280 at 0x000113901800 allocated by thread T1:
    #0 calloc <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x64ea0)
    #1 ZSTD_createCCtx <null> (zstd.cpython-316t-darwin.so:arm64+0x1018c)
    #2 py_zstd_compress_mt2 <null> (zstd.cpython-316t-darwin.so:arm64+0x1018)
    #3 cfunction_call methodobject.c:575 (python.exe:arm64+0x1001387b0)
    #4 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #5 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #6 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #7 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #8 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #9 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #10 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #11 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #12 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #13 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #14 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #15 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #16 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #17 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
    #18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #19 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #20 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #21 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #22 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #23 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #24 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #25 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #26 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #27 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #28 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #29 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #30 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #31 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #32 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)

  Thread T2 (tid=21203051, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCall_StackRefSteal ceval.c:724 (python.exe:arm64+0x100290228)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3528 (python.exe:arm64+0x100299d1c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

  Thread T1 (tid=21203050, running) created by main thread at:
    #0 pthread_create <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x100389300)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x100389138)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x10044dd10)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10044d7dc)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x10044c844)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100138744)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #8 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #9 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #10 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #11 _PyEval_EvalFrameDefault generated_cases.c.h:3325 (python.exe:arm64+0x10029964c)
    #12 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fa34)
    #13 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fa34)
    #14 PyEval_EvalCode ceval.c:677 (python.exe:arm64+0x10028fa34)
    #15 run_eval_code_obj pythonrun.c:1369 (python.exe:arm64+0x100366a70)
    #16 run_mod pythonrun.c:1472 (python.exe:arm64+0x1003667cc)
    #17 pyrun_file pythonrun.c:1296 (python.exe:arm64+0x100361d38)
    #18 _PyRun_SimpleFileObject pythonrun.c:518 (python.exe:arm64+0x100361d38)
    #19 _PyRun_AnyFileObject pythonrun.c:81 (python.exe:arm64+0x1003614c8)
    #20 pymain_run_file_obj main.c:411 (python.exe:arm64+0x1003a3328)
    #21 pymain_run_file main.c:430 (python.exe:arm64+0x1003a3328)
    #22 pymain_run_python main.c:715 (python.exe:arm64+0x1003a2658)
    #23 Py_RunMain main.c:796 (python.exe:arm64+0x1003a2658)
    #24 pymain_main main.c:826 (python.exe:arm64+0x1003a2bd0)
    #25 Py_BytesMain main.c:850 (python.exe:arm64+0x1003a2cd0)
    #26 main python.c:15 (python.exe:arm64+0x100000a78)

SUMMARY: ThreadSanitizer: data race (zstd.cpython-316t-darwin.so:arm64+0x10610) in ZSTD_freeCCtx+0x7c
==================

(more reports)

==================
==11580==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000118 (pc 0x0001129893f4 bp 0x000381005e30 sp 0x000381005d90 T21203175)
==11580==The signal is caused by a READ memory access.
==11580==Hint: address points to the zero page.
    #0 ZSTDMT_compressStream_generic <null> (zstd.cpython-316t-darwin.so:arm64+0x9d3f4)
    #1 ZSTD_compressStream2 <null> (zstd.cpython-316t-darwin.so:arm64+0x1b974)
    #2 ZSTD_compress2 <null> (zstd.cpython-316t-darwin.so:arm64+0x14824)
    #3 py_zstd_compress_mt2 <null> (zstd.cpython-316t-darwin.so:arm64+0x1090)
    #4 cfunction_call methodobject.c:575 (python.exe:arm64+0x1001387b0)
    #5 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x10009030c)
    #6 _PyObject_VectorcallTstate pycore_call.h:142 (python.exe:arm64+0x100090f14)
    #7 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090f14)
    #8 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #9 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #10 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #11 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #12 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #13 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #14 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #15 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #16 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1002dce38)
    #17 context_run context.c:728 (python.exe:arm64+0x1002dce38)
    #18 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:421 (python.exe:arm64+0x1000a8644)
    #19 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100090e80)
    #20 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100090e80)
    #21 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:766 (python.exe:arm64+0x100290968)
    #22 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x100295ff8)
    #23 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x10028fe5c)
    #24 _PyEval_Vector ceval.c:2134 (python.exe:arm64+0x10028fe5c)
    #25 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000914bc)
    #26 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100092c3c)
    #27 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x100092c3c)
    #28 method_vectorcall classobject.c:55 (python.exe:arm64+0x100095fdc)
    #29 _PyVectorcall_Call call.c:273 (python.exe:arm64+0x10009112c)
    #30 _PyObject_Call call.c:348 (python.exe:arm64+0x10009112c)
    #31 PyObject_Call call.c:373 (python.exe:arm64+0x1000911a4)
    #32 thread_run _threadmodule.c:388 (python.exe:arm64+0x10044df04)
    #33 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038a12c)
    #34 __tsan_thread_start_func <null> (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31cf4)
    #35 _pthread_start <null> (libsystem_pthread.dylib:arm64e+0x6c54)
    #36 thread_start <null> (libsystem_pthread.dylib:arm64e+0x1c18)

==11580==Register values:
 x[0] = 0x0000000115b5c000   x[1] = 0x0000100000000230   x[2] = 0x000000004d8e3f0f   x[3] = 0x0000000000000002  
 x[4] = 0x0000000000000001   x[5] = 0x0000000000000001   x[6] = 0x0000000010000000   x[7] = 0x0000000000000000  
 x[8] = 0x000000000d8e3f00   x[9] = 0x000000000d8e3f0f  x[10] = 0x0000000116ca8e28  x[11] = 0x0000000116ca8e20  
x[12] = 0x0000100000000230  x[13] = 0x0000000000000000  x[14] = 0x00000000000000ff  x[15] = 0x000000000000003f  
x[16] = 0x000000018bec6c8c  x[17] = 0x0000000000000008  x[18] = 0x0000000000000000  x[19] = 0x0000000113901800  
x[20] = 0x0000000000000000  x[21] = 0x0000000381005ea0  x[22] = 0x0000000000000002  x[23] = 0x0000000000000001  
x[24] = 0x0000000000000c0c  x[25] = 0x00000000000000e4  x[26] = 0x0000000381005ea0  x[27] = 0x00000000000000b4  
x[28] = 0x0000000381005eb8     fp = 0x0000000381005e30     lr = 0x00000001129893f4     sp = 0x0000000381005d90  
ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV (zstd.cpython-316t-darwin.so:arm64+0x9d3f4) in ZSTDMT_compressStream_generic+0x64
==11580==ABORTING
zsh: abort      python3 repro.py

@sergey-dryabzhinsky

sergey-dryabzhinsky commented Jun 28, 2026

Copy link
Copy Markdown
Owner

Ok, maybe there is no simple way. And it's safer to do a new context every time.
How work on context thread pool going?

@sergey-dryabzhinsky

sergey-dryabzhinsky commented Jun 28, 2026

Copy link
Copy Markdown
Owner

Marked 15.7.2 as last safe release.

@sergey-dryabzhinsky sergey-dryabzhinsky mentioned this pull request Jun 29, 2026
Open
@KowalskiThomas

KowalskiThomas commented Jun 29, 2026

Copy link
Copy Markdown
Author

How work on context thread pool going?

Ah, I thought you were looking into it! I'll have a look this week :)

@sergey-dryabzhinsky

sergey-dryabzhinsky commented Jun 30, 2026

Copy link
Copy Markdown
Owner

Ok, I'll wait

@KowalskiThomas

KowalskiThomas commented Jul 11, 2026

Copy link
Copy Markdown
Author

👋 Hi!

I just pushed a commit that adds a pool as I proposed. Initially I wanted to do in on the Python side, but then I realised I could do it directly in the C extension, which took me a bit more time than I it would have had in Python.

Anyway, I've run a test with multiple threads as well as the reproducer and nothing crashes anymore.

@sergey-dryabzhinsky

sergey-dryabzhinsky commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Do not really understand how you change pool size.
Maybe you should move new pool code into new .h & .c files.

@KowalskiThomas

KowalskiThomas commented Aug 23, 2026

Copy link
Copy Markdown
Author

I just pushed new changes!

Maybe you should move new pool code into new .h & .c files.

I did this -- it's now in src/cctx_pool.c and src/cctx_pool.h.

Do not really understand how you change pool size.

I added a comment about that. In short: the size of the pool only really matters when we push contexts back into it (as long as all a context is in use (so outside the pool), it doesn't need/use any pool capacity. When a context is released, we check the current size of the pool; if the pool size is enough to push the context back into it we simply place it back; if not we grow the pool (by a factor of 2 like std::vector does, so that amortised complexity for releasing is O(1)).

The pool never shrinks because it would add more complexity than needed. It doesn't mean its growth is unbounded though, since the size of the pool is always at most 2 * max_number_of_contexts_used_at_once, so in the worst case that would be bounded by the maximum number of threads using a context at once (times 2 if we're unlucky).

Does that clarify?

I also added a regression test with the reproducer I initially posted in the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@KowalskiThomas @sergey-dryabzhinsky