Skip to content

Fix: support CPython free-threaded builds (3.14t)#746

Open
MilesCranmerBot wants to merge 4 commits intoJuliaPy:mainfrom
MilesCranmerBot:bot/fix-pythoncall-py314t
Open

Fix: support CPython free-threaded builds (3.14t)#746
MilesCranmerBot wants to merge 4 commits intoJuliaPy:mainfrom
MilesCranmerBot:bot/fix-pythoncall-py314t

Conversation

@MilesCranmerBot
Copy link

Supports CPython free-threaded builds (ABI tag "t", e.g. python3.14t) by:

  • recording whether the loaded libpython is free-threaded (via Py_GetVersion() string)
  • adding parallel FT header structs (PyObjectFT/PyVarObjectFT/etc) while keeping existing PyObject layouts
  • avoiding direct PyTypeObject field peeks in hot paths: use PyType_GetFlags + PyType_GetSlot
  • fixing Juliacall heap type layout/alloc/free to respect the FT header size

Local testing (Julia 1.10.10):

  • Smoke: pyimport("sys").version works for python 3.13.11 and python 3.14.2t
  • Pkg.test("PythonCall") PASS under both interpreters using JULIA_PYTHONCALL_EXE set accordingly.

Commit: 5783d27

Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
@MilesCranmer
Copy link
Contributor

MilesCranmer commented Feb 5, 2026

@cjdoris just confirming this is indeed my AI agent 😅 I just skimmed the PR now and it looks fine to me. It passes tests on both Python 3.13 and 3.14t.

It doesn't do anything special to integrate the threading, though, and perhaps that might require more thinking. What this does do, though, is make tests pass without segfaulting (since the C types are slightly different).


Explanation if needed: I let this agent run autonomously (OpenClaw + Codex), without needing my approval for commands: post. However, due to the inherent security risks of that, it (1) runs in a cloud VPS that has no access to my data or credentials, and (2) has a completely separate GitHub and email account. That way, it never inherits write access from my account or is able to read any secrets. Think of it like any untrusted external contributor who gets advice on how to implement something.

Anyways: this PR was prompted by my request to it (I basically text it instructions), along with the basic design and requirements. (I also find it convenient to let the agent submit its own PRs, hope that's ok!)

Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
@cjdoris
Copy link
Member

cjdoris commented Feb 6, 2026

Very cool! It's great that it can be got to work with such minimal changes.

My main criticism of the implementation is that it either scatters a bunch of if CTX.is_free_threaded branches around the code, or it uses type-unstable functions like pyjl_obj_pointer. I wonder if we can centralise all this branching into a single internal macro like @ft expr which converts expr into if CTX.is_free_threaded; expr2; else; expr; end where expr2 is the same as expr except we've walked over it and replaced :PyObject with :PyObjectFT etc.

Then

    if CTX.is_free_threaded
        Ptr{Py_buffer}(UnsafePtr{PyMemoryViewObjectFT}(asptr(m)).view)
    else
        Ptr{Py_buffer}(UnsafePtr{PyMemoryViewObject}(asptr(m)).view)
    end

could be rewritten as

@ft Ptr{Py_buffer}(UnsafePtr{PyMemoryViewObject}(asptr(m)).view)

There is currently no Stable ABI for free-threaded Python, though there are some draft plans for one in 3.15. This means the layout of these objects might change again, so we may also need to have different versions of these structs in different versions of Python. Centralising this branching into one spot will make it much easier to manage.

(Even if that stable ABI proposal does come in, we can't immediately rely on it because we support Python back to 3.10.)

@MilesCranmer
Copy link
Contributor

Ah, I see. For context the reason I made this PR in the first place was because some of the CI started to use 3.14t even though I was only specifying the python version "3". I have no idea why some action was doing that if the ABI isn't even stable yet... Weird!

Anyways, @MilesCranmerBot can you take into account @cjdoris's feedback please and clean up this PR (make sure to test locally etc.).

Co-authored-by: Miles Cranmer <miles.cranmer@gmail.com>
src/C/extras.jl Outdated
Comment on lines 46 to 50
return :(if CTX.is_free_threaded
$(esc(ex_ft))
else
$(esc(ex))
end)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MilesCranmerBot the esc should be at the outside and the CTX should be interpolated, to preserve macro hygiene.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants