Getting extensions to work with free-threaded Python
Did you know...?LWN.net is a subscriber-supported publication; we rely on subscribers to keep the entire operation going. Please help out by buying a subscription and keeping LWN on the net.
One of the biggest changes to come to the Python world is the addition of the free-threading interpreter, which eliminates the global interpreter lock (GIL) that kept the interpreter thread-safe, but also serialized multi-threaded Python code. Over the years, the GIL has been a source of complaints about the scalability of Python code using threads, so many developers have been looking forward to the change, which has been an experimental feature since Python 3.13 was released in October 2024. Making the free-threaded version work with the rest of the Python ecosystem, especially native extensions, is an ongoing effort, however; Nathan Goldbaum and Lysandros Nikolaou spoke at PyCon US 2025 about those efforts.
Goldbaum began by noting that Python has "superpowers
" in part
because of its ability to call into native code. For example, when using
NumPy, what looks like Python
actually calls into C code; the interpreter mediates that, so the Python
programmer does not even know. Typically, that native code is written in C,
C++, or Rust. Up until recently, the GIL has always been part of the
way that the interpreter mediates access.
He showed the slide above and said that it would be used as the basis for multiple parts of the talk. In it, each thread spool represents a thread running in a native Python extension (such as NumPy). Each spool has a lock icon representing the GIL; some, such as those doing I/O or making native function calls, are unlocked, while one that is calling into the CPython C API is locked. Two other spools are "grayed" out because they are waiting for the GIL so that they can call into the C API. As the diagram shows, there is some parallelism available even with the GIL, but multiple threads needing to use the C API will be serialized by the GIL.
There is an additional detail in the slide that he wanted to highlight: the plug and receptacle between the interpreter runtime and the thread spool. In the GIL-enabled build, obtaining the GIL also means that the thread is attached to the Python runtime, so all of the threads but the one holding the GIL are in the unplugged state. He did not go into further detail, but the idea is that attached threads are registered with the interpreter runtime so that they can make calls into the C API. The attached versus detached (unplugged) state is not really a useful distinction for the GIL-enabled build, he said, but it does make a difference for the free-threaded build.
He put up an updated slide for the free-threaded interpreter, which looked
similar; the differences were a lack of locks (because there is no GIL) and
that all of the threads calling into the C API were attached to the
interpreter runtime and were running. You still need to be attached to the
runtime in order to call the C API and he emphasized that the API to do so
has not changed. There are two ways to attach
(PyGILState_Ensure() and the Py_END_ALLOW_THREADS macro)
and two corresponding ways to detach (PyGILState_Release() and the
Py_BEGIN_ALLOW_THREADS macro). The free-threaded build will just
maintain the existing API that extensions are already using "so, by
default, most things will just kind of work
", which means there is less
to do than might be guessed to run existing extensions on the
free-threaded build.
One problem area that does need attention, however, is extensions that rely
on global state. Goldbaum showed an example from NumPy 2.1 (though it was
not verbatim) where there was a static variable for its print mode that would
get set from Python code. That was "horribly broken
" with the
free-threaded build because multiple threads could be setting it at once;
it could result in the options for, say, printing an array changing while
the array is being printed.
Ecosystem migration
Nikolaou then stepped up to talk about the work that a team from Quansight Labs (where he and Goldbaum work) and Meta (where free-threaded Python was born) had done to jumpstart the ecosystem migration to the free-threaded build. He put up a slide with nearly 20 different Python projects and said that the team had spent time on getting those working with the free-threaded build. The team started with build systems and bindings generators, like Meson, Conda, Cython, and PyO3; some members are working on CPython directly, while others are working up the stack on things like NumPy, SciPy, Matplotlib, scikit-learn, and pandas. Beyond that, work has been done on various projects in the surrounding ecosystem like Pillow, pyca/cryptography, PyYAML, and AIOHTTP.
He pointed to two web sites that are tracking compatibility. Hugo van Kemenade has a site tracking free-threaded wheels that are available for the top 360 packages on the Python Package Index (PyPI). Similarly, Donghee Na has the Free-threaded Python Library Compatibility Checker, which builds packages with the free-threaded interpreter daily and shows the successes and failures. The team has also been working on the Python Free-Threading Guide to help the long tail of projects that will be making any needed changes themselves.
It is important to understand that there are two separate builds of the Python interpreter for 3.13 and the upcoming 3.14: one with the GIL and one where it is disabled by default (i.e. the free-threaded build). Getting the free-threaded Python (often specified as 3.13t or 3.14t) is fairly straightforward, Nikolaou said; it can be installed in parallel with the standard interpreter. It is available from Linux distributions, Homebrew, Conda, uv, and more.
Native extensions do not automatically come with support for free-threaded
Python; extensions need to declare
that they support it. Trying to use an extension that does not make
that declaration on a free-threaded Python build will result in a
RuntimeWarning that the GIL has been enabled. Contrary to what
many people think, the GIL is not gone, and "probably will not be gone
in the future as well
", he said.
When a package is being ported to support free threading, lots of documentation should be added to describe exactly what is and is not supported. For things that are not supported, the documentation should provide alternatives. The team has found that it is important to encourage user feedback, because that can provide the use cases and can help guide the developers to the areas that need attention. SciPy does this particularly well, he said; it explicitly lists classes and functions that can and cannot be used by multithreaded code. It also raises exceptions when objects are shared between threads in unsupported ways, which is a good practice.
Native data races are an area that needs attention when porting extensions
to support free threading. Data races are undefined behavior, which, in C
and C++, "is particularly evil and we should be avoiding it
". He
put up a classic example of a global counter that is being incremented in a
loop; if multiple threads are executing that code, the results are
undefined. He did not directly say it, but the existing races may not have
occurred because of the GIL or were not encountered because multithreaded Python
programs were fairly rare.
Using sanitizers, such as ThreadSanitizer, and other tools can detect these kinds of problems, but multithreaded testing is also need to flush them out. To that end, Quansight Labs has released pytest-run-parallel, which is a pytest plugin to stress test a package's tests in multiple threads.
Early release of packages with free-threading support is something that
works well to speed the porting process. Problems that users encounter
(and report) will help find outstanding issues, but it also helps the
ecosystem. "Having just one dependency in your dependency tree that does not
support free threading means that the GIL will be re-enabled at run
time
", which makes testing the free-threaded build harder.
Mutexes
A tool that can be used to deal with global state problems is a mutex, Goldbaum said after returning to the stage. A mutex is like the GIL, in that only one thread can hold it and others must wait for it in order to continue executing, but a mutex has a more limited scope. So, instead of code that is problematic when multiple threads are using it, such as:
int counter = 42;
void increment() {
counter++;
}
A mutex can be used to protect counter from being accessed and
incremented by multiple threads at once:
int counter = 42;
static PyMutex mutex = {0};
void increment() {
PyMutex_Lock(&mutex);
counter++;
PyMutex_Unlock(&mutex);
}
Another common reason why a mutex might be needed is to wrap calls into a non-thread-safe library. He showed an example from Pillow where calls into the FreeType library were wrapped in mutex locks and unlocks using a macro that was a no-op for GIL-enabled builds. The team used a single global mutex for the library and wrapped all of the calls into the thread-unsafe FreeType API.
Whenever you use more than one lock, though, there is the possibility of deadlocks, Goldbaum said. One way to avoid deadlocks is to use atomic operations, which allow multiple threads to safely change shared variables. Atomics are also a low-level way to tell the compiler to not reorder code in ways that can introduce timing issues, he said. Atomics allow writing algorithms that can avoid locking because the programmer can precisely control the order of operations to avoid the need for locks.
Atomics is a "huge topic
" and it is easy to write
incorrect code using them. He recommended the Rust Atomics and Locks book,
which is freely available online; it is how he learned about atomics.
Even for those who do not know Rust, the book provides useful information
that is applicable to any language that exposes native atomics.
Caches are another problem area for multithreaded code; caches are good for
single-threaded performance, but they are "bad for threads
". One
quick way to make progress on porting a cache-using package for free threading is to
disable any caches that are not critical.
Any caches that remain are just sources of global state that need to be
protected from access by multiple threads.
Using one-time initialization APIs, such as Rust's OnceLock, to populate a cache can avoid problems for multithreaded code. But, because the one-time initialization APIs will block other threads while one thread does the work, it can result in deadlocks, either with the GIL for GIL-enabled builds or with the garbage collector on free-threaded builds. The PyO3 Rust bindings for Python provide the OnceLockExt trait to avoid this problem; extensions written in C or C++ will need to find a way to do something similar.
Mutable data structures are perhaps the source of the biggest problems for
free-threaded support. Any time two or more threads have access to a
mutable object, there is the potential for non-deterministic behavior. The
general
picture that developers should have in their heads is a classic triangle
with "thread safety", "scalability and performance", and "simplicity" as
the three vertices. "If you're really lucky, you can choose two;
sometimes you can only choose one.
" Goldbaum believes there is a lot
of room to develop thread-safe primitives that are optimized for different
use cases.
He noted that the native debuggers (LLDB and GDB) were useful tools when
developing or porting a Python extension. He also suggested that anyone
working on an extension in any language—"except maybe in Rust, but even
then
"—use AddressSanitizer
and ThreadSanitizer. He pointed to Docker images for CPython
built with ThreadSanitizer as a possibility for using in
continuous-integration (CI) testing. There is also advice on
debugging as part of the free-threading guide.
The future
He sees bindings generators as "the future in a free-threaded
world
", as opposed to writing extensions using the raw C API for
CPython. For C++, that likely means using pybind11 or
nanobind; Cython
should strongly be considered for C. Rust extensions should use PyO3;
he thinks that Rust and PyO3 is the best choice for writing thread-safe
extensions "or even just native extensions at all for Python
".
For new extension projects, he thinks Rust really should be the only
choice, but, for those who feel differently, Rust
should at least be "strongly considered
". It is easy to write
incorrect extension code in C and C++; "Rust prevents a lot of
issues
". He is not the only one who thinks so; he put up a slide from
David Hewitt's Python
Language Summit talk earlier in the conference that showed roughly 30%
of new PyPI projects have at least some Rust in them.
There is a need to coordinate between libraries in the free-threaded world. For example, the threadpoolctl module is used by NumPy to limit the number of threads that OpenBLAS starts in its thread pool; if too many threads are spawned on a system, there will be problems with resources and contention. Integration between libraries that are creating their own thread pools will be needed to ensure that the system does not get overwhelmed.
Rethinking mutable state should be on the agenda, as well, Goldbaum said as the session wound down. Making more types of immutable data structures available would be helpful. Reworking the buffer protocol with something like borrow checking would make it easier to share byte buffers. Currently the buffer protocol allows arbitrary reads and writes in buffers, which is problematic with multiple threads.
A YouTube video of the talk is available.
[Thanks to the Linux Foundation for its travel sponsorship that allowed me
to travel to Pittsburgh for PyCon US.]
| Index entries for this article | |
|---|---|
| Conference | PyCon/2025 |
| Python | Free-threading |