Autobahn on Free-Threaded Python (PEP 703)

Preface

Running Autobahn on free-threaded Python is a deep rabbit hole, and there is no single “easy” answer we can give. Autobahn|Python supports a wide range of combinations:

  • Runtimes: CPython with CPyExt, and PyPy with cffi

  • Operating systems: Linux (epoll), BSD (kqueue), and Windows (IOCP)

  • Event loops: Python stdlib asyncio and Twisted reactors

Multiplying these dimensions quickly produces a large compatibility matrix, and with it many opportunities for subtle issues.

Autobahn|Python itself is mostly pure Python. Optionally, there is a small amount of native code for acceleration (primarily for CPython): autobahn/nvx. This code is written with CFFI, avoids writable global state, and is therefore considered low risk in the context of free-threaded Python.

Most of the potential problems do not originate in Autobahn itself, but in the layers beneath it (Python runtime, OS event APIs, and event loop implementations). These only become relevant when a user chooses to run Autobahn on a free-threaded Python build.

It is also worth stressing that Crossbar.io does not rely on free-threaded Python. Crossbar.io is designed for process-based scalability with efficient IPC, which avoids threading pitfalls while still taking full advantage of many-core (SMP) systems. We have tested Crossbar.io at scale on machines with 64+ CPU cores under real-world load, and the process-based architecture remains robust and performant.

Introduction

This page gives an overview of running Autobahn on Python builds where the Global Interpreter Lock (GIL) is disabled (so-called free-threaded builds). This feature originates from PEP 703 and is available experimentally in CPython 3.13+.

Autobahn’s networking relies on Python’s runtime, the event loop (asyncio or Twisted), and the host operating system’s networking APIs. With the GIL disabled, new pitfalls appear because concurrency semantics change.

Status Summary

  • CPython (default, GIL enabled): Stable and supported.

  • CPython (``–disable-gil`` / ``-X gil=0``): Experimental. Known thread-safety issues in asyncio internals (see asyncio issue tracker).

  • PyPy: Uses cffi rather than the CPython C-API. Pure-Python code (like most of Autobahn) runs fine, but C extension dependencies may break.

Operating System Event Backends

Autobahn is transport-agnostic but uses asyncio or Twisted, which in turn depend on the OS event notification API:

  • Linux: epoll (readiness-based)

  • BSD / macOS: kqueue (filter-based)

  • Windows: IOCP (completion-based), or selector fallbacks

These kernel interfaces are thread-safe. The complexity lies in how Python libraries wrap them:

  • epoll: Level- or edge-triggered; widely understood semantics.

  • kqueue: Requires explicit re-arming; subtle differences vs. epoll.

  • IOCP: Completion-based (not readiness-based); asyncio provides a dedicated ProactorEventLoop for this.

Pitfalls in Free-Threaded Mode

  1. asyncio thread-safety * Not all internal data structures are safe without the GIL. * Concurrent socket register/unregister can corrupt internal state.

  2. Twisted reactor thread affinity * Twisted expects all reactor interaction from a single thread. * Use callFromThread / callInThread when crossing threads.

  3. C extension assumptions * Many extensions assume the GIL exists. * Some re-enable a GIL internally, negating free-threading benefits. * On PyPy, C extensions must be ported to cffi.

  4. Cross-platform divergence * Different semantics between epoll, kqueue, and IOCP. * Test across all intended platforms.

Practical Guidance

  • Start small: run an Autobahn echo server under python -X gil=0.

  • Stress test: with many concurrent connections and rapid open/close.

  • Audit dependencies: especially TLS libraries, HTTP parsers, DB drivers.

  • Stick to documented APIs: e.g. in Twisted, do not touch reactor internals outside its thread.

  • Be prepared to pin versions: many libraries may only work with the traditional GIL until they are ported.

Testing Matrix

Recommended environments to validate:

  • Runtimes: CPython (GIL on), CPython (GIL off), PyPy

  • OS backends: Linux (epoll), BSD/macOS (kqueue), Windows (IOCP)

  • Frameworks: asyncio (default/proactor), Twisted

Future Work

  • CPython developers are actively fixing stdlib thread-safety issues.

  • Third-party libraries are starting to publish guidance for free-threaded support.

  • Autobahn will track these developments and update documentation.

References

Note

Free-threaded Python builds are experimental and not yet recommended for production use. Autobahn maintainers are tracking compatibility and will update this page as the ecosystem matures.