Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-124370: Add "howto" for free-threaded Python #124371

Merged
merged 13 commits into from
Sep 25, 2024

Conversation

colesbury
Copy link
Contributor

@colesbury colesbury commented Sep 23, 2024

This is a guide aimed at people writing Python code, as oppposed to the existing guide for C API extension authors.


📚 Documentation preview 📚: https://cpython-previews--124371.org.readthedocs.build/en/124371/howto/free-threading-python.html

This is a guide aimed at people writing Python code, as oppposed to the
existing guide for C API extension authors.
@colesbury colesbury marked this pull request as ready for review September 23, 2024 20:33
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Comment on lines +82 to +86
* :ref:`function <user-defined-funcs>` objects declared at the module level
* :ref:`method <instance-methods>` descriptors
* :ref:`code <code-objects>` objects
* :term:`module` objects and their dictionaries
* :ref:`classes <classes>` (type objects)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably worth noting somewhere in here that these objects (and immortalization itself) are implementation details, and very much subject to change.

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
mechanism for decisions related to the build configuration.


Thread Safety
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this section is probably among the most important for most users. As I understand it, the general rule is that you write thread-safe code under free-threading the same way that you'd write thread-safe code under previous versions of Python; the rules have not really changed. That could be made more explicit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's our goal for the implementation, but I'm a bit hesitant to promise too much here, especially because it's not always clear what was thread-safe under the GIL.

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Co-authored-by: Hugo van Kemenade <[email protected]>
Copy link
Member

@gpshead gpshead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High level comments:

  • Overall, great! I'm not reading it with an eye for what has been left out. But its a doc, I consider those "living" and future updates would deal with that.
  • Perhaps include the word "experimental" in the top level title to help temper expectations?
  • Consider adding a reference to PEP 703 to the doc for context for those interested in how this came to be and what the initial long term plan and expectations are.
  • In the final Single-threaded performance section, I predict someone not aware of larger long term plans and PEP 703 and the earlier proof of concept work you built on top of a couple releases to freak out at the thought of a 40% slower interpreter. I'd go beyond the "This overhead is expected to be reduced in the Python 3.14." statement and indicate what level of free-threaded build performance delta we're ultimately aiming for in the future (without promising which version it happens in).

@mpage
Copy link
Contributor

mpage commented Sep 24, 2024

LGTM. +1 for @gpshead's comment about including some detail about what single-threaded performance delta we're ultimately shooting for.

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
---------

Sharing the same iterator object between multiple threads is generally not
safe and threads may see duplicate or missing elements when iterating or crash
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the possible crashing of the interpreter expected to be addressed in 3.14?

As a consequence of iterators not being thread safe, some modules are not safe either (e g. json, itertools). Should this be mentioned somewhere?

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. This HOWTO can be adjusted/elaborated later if needed.

Copy link
Member

@nedbat nedbat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this! I love that we are putting out information aimed at the users.

*********************************

Starting with the 3.13 release, CPython has experimental support for running
with the :term:`global interpreter lock` (GIL) disabled in a configuration
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"configuration" makes me think of a configuration file I might need to create. Can we say "build of Python"? Or at least "build configuration".

with the :term:`global interpreter lock` (GIL) disabled in a configuration
called :term:`free threading`. This document describes the implications of
free threading for Python code. See :ref:`freethreading-extensions-howto` for
information on how to write C extensions that support the free-threaded build.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We seem to switch between "free threading" and "free-threaded" kind of randomly. I can't decide if this is OK or if we should choose one and stick with it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine. We also say both "multithreaded" and "multithreading" depending on the context.


The free-threaded build of CPython can optionally run with the global
interpreter lock enabled, such as when :envvar:`PYTHON_GIL` is set to ``1``,
or when importing an extension module that requires the GIL.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could use more explanation. When might I want to set this environment variable? "an extension module that requires the GIL": is this something I have to figure out, or does Python know this itself somehow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to get into the details of the environment variable here. The important point here is that the free-threaded build can run with the GIL enabled -- that's not obvious to most people. The environment variable is mentioned here just as an example of that.

The free-threaded build of the 3.13 release makes some objects :term:`immortal`
in order to avoid reference count contention that would prevent efficient
multi-threaded scaling. This means that these objects are never deallocated.
This is expected to be addressed in Python 3.14 with
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"addressed" how? That makes it sound like immortal objects are a problem. Are they?

* :ref:`classes <classes>` (type objects)

The immortalization of these objects happens the first time a thread is started
after the main thread.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading about immortalization, I don't understand the implications for me as a Python programmer. Why do I care that they are now immortal? These all sound like things that would have never been deallocated anyway. Are there unusual circumstances that I should be considering?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully most people won't care, but some programs create these sorts of things in a loop.

Doc/howto/free-threading-python.rst Show resolved Hide resolved
Doc/howto/free-threading-python.rst Show resolved Hide resolved
called :term:`free threading`. This document describes the implications of
free threading for Python code. See :ref:`freethreading-extensions-howto` for
information on how to write C extensions that support the free-threaded build.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could add a quick snapshot of the overall plan: if everything works out, eventually free threading will be the only build, etc. Also, maybe a statement about how most programmer won't need to be concerned with this, we're doing a lot to keep everyday Python programs behaving the same, etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would perhaps add a note in the seealso block that refers to the PEP and adds 1 or 2 highlights:

  • performance conscious users may wish to try the free threaded version
  • most users can safely continue to use the default CPython version
  • when free threading is no longer experimental, the versions will converge into one version for all.


The :func:`sys._is_gil_enabled` function will return ``False`` if the global
interpreter lock is currently disabled. This is the recommended mechanism for
decisions like whether to use multithreading or multiprocessing.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it likely that people will write code that examines this variable and chooses between multithreading and multiprocessing? It seems unlikely to me, but maybe library authors will? When should it be used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"How do I distinguish between the free-threaded and GIL-enabled build?" is a fairly common question, although I don't think people make much use of it other than for quick debugging and sanity checks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would break this section into two sections:

  • Identifying free threading builds (with the last paragraph)
  • Reenabling the GIL with a free threading build (first two paragraphs)

@bedevere-app
Copy link

bedevere-app bot commented Sep 24, 2024

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

Copy link
Contributor

@willingc willingc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good @colesbury. I made some wording suggestions that should add clarity for Ned and others.

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
called :term:`free threading`. This document describes the implications of
free threading for Python code. See :ref:`freethreading-extensions-howto` for
information on how to write C extensions that support the free-threaded build.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would perhaps add a note in the seealso block that refers to the PEP and adds 1 or 2 highlights:

  • performance conscious users may wish to try the free threaded version
  • most users can safely continue to use the default CPython version
  • when free threading is no longer experimental, the versions will converge into one version for all.

Doc/howto/free-threading-python.rst Show resolved Hide resolved

The :func:`sys._is_gil_enabled` function will return ``False`` if the global
interpreter lock is currently disabled. This is the recommended mechanism for
decisions like whether to use multithreading or multiprocessing.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would break this section into two sections:

  • Identifying free threading builds (with the last paragraph)
  • Reenabling the GIL with a free threading build (first two paragraphs)

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Doc/howto/free-threading-python.rst Show resolved Hide resolved
---------------

The free-threaded build of the 3.13 release makes some objects :term:`immortal`
in order to avoid reference count contention that would prevent efficient
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
in order to avoid reference count contention that would prevent efficient
which have reference counts that are never modified. Since the immortal object is guaranteed not to be
deallocated, efficient multi-threading scaling is possible by avoiding reference count contention between
threads.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reworked this a bit. Would you please take another look at it? As you wrote, immortalization means reference counts are not modified and objects are not guaranteed not to be deallocated. I wanted to get across that:

  • Reference counts not being modified enables efficient multi-threaded scaling.
  • Objects not being deallocated is a potential problem because some applications may leak memory. That's being addressed in 3.14. We don't expect this to be a problem for most applications, but it's something to watch out for if you run your application with refleak checks.

Doc/howto/free-threading-python.rst Outdated Show resolved Hide resolved
Comment on lines 126 to 127
less of an impact. This overhead is expected to be reduced in Python
3.14. We are aiming for an overhead of 10% or less on the pyperformance
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
less of an impact. This overhead is expected to be reduced in Python
3.14. We are aiming for an overhead of 10% or less on the pyperformance
less of an impact. This overhead is expected to be reduced in Python
3.14 by implementing more efficient reference counting and garbage collection. We are aiming for an overhead of 10% or less on the pyperformance

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We expect that the biggest source of improvement will be reenabling the specializing adaptive interpreter in 3.14. It's currently disabled in the 3.13 free-threaded build. Is that worth mentioning here?

I don't expect any big performance impact for reference counting and garbage collection changes in 3.14

Copy link
Contributor

@willingc willingc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the updates @colesbury. This how to is a nice addition 😄


The GIL may also automatically be enabled when importing a C-API extension
module that is not explicitly marked as supporting free threading. See
:c:macro:`Py_MOD_GIL_NOT_USED` for more details.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this document is for Python programmers, I think we should avoid sending them off to look at C macros. As a Python programmer, my question now would be, "how can I tell if my C extensions support free threading or not?" Is there a way to know without examining the C source code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can remove the link and mention of the C macro. There isn't a great answer for "how can I tell if my C extensions support free threading?"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those are good answers!

@willingc
Copy link
Contributor

@nedbat If you are good with today's change, please approve and merge. Nice work by @colesbury and everyone who gave feedback. Let's get this out in the wild and we can fine tune if/when needed. 🎉

@nedbat nedbat merged commit 68e384c into python:main Sep 25, 2024
23 checks passed
emilyemorehouse added a commit to lysnikolaou/cpython that referenced this pull request Sep 26, 2024
* main: (69 commits)
  Add "annotate" SET_FUNCTION_ATTRIBUTE bit to dis. (python#124566)
  pythongh-124412: Add helpers for converting annotations to source format (python#124551)
  pythongh-119180: Disallow instantiation of ConstEvaluator objects (python#124561)
  For-else deserves its own section in the tutorial (python#123946)
  Add 3.13 as a version option to the crash issue template (python#124560)
  pythongh-123242: Note that type.__annotations__ may not exist (python#124557)
  pythongh-119180: Make FORWARDREF format look at __annotations__ first (python#124479)
  pythonGH-58058: Add quick reference for `ArgumentParser` to argparse docs (pythongh-124227)
  pythongh-41431: Add `datetime.time.strptime()` and `datetime.date.strptime()` (python#120752)
  pythongh-102450: Add ISO-8601 alternative for midnight to `fromisoformat()` calls. (python#105856)
  pythongh-124370: Add "howto" for free-threaded Python (python#124371)
  pythongh-121277: Allow `.. versionadded:: next` in docs (pythonGH-121278)
  pythongh-119400:  make_ssl_certs: update reference test data automatically, pass in expiration dates as parameters python#119400  (pythonGH-119401)
  pythongh-119180: Avoid going through AST and eval() when possible in annotationlib (python#124337)
  pythongh-124448: Update Windows builds to use Tcl/Tk 8.6.15 (pythonGH-124449)
  pythongh-123884 Tee of tee was not producing n independent iterators (pythongh-124490)
  pythongh-124378: Update test_ttk for Tcl/Tk 8.6.15 (pythonGH-124542)
  pythongh-124513: Check args in framelocalsproxy_new() (python#124515)
  pythongh-101100: Add a table of class attributes to the "Custom classes" section of the data model docs (python#124480)
  Doc: Use ``major.minor`` for documentation distribution archive filenames (python#124489)
  ...
@vstinner
Copy link
Member

This is a guide aimed at people writing Python code, as oppposed to the existing guide for C API extension authors.

Thanks for writing down this guide @colesbury! It will be very valuable :-)

@colesbury colesbury deleted the gh-124370-python-free-threaded-howto branch October 1, 2024 17:10
@colesbury colesbury added the needs backport to 3.13 bugs and security fixes label Oct 1, 2024
@miss-islington-app
Copy link

Thanks @colesbury for the PR, and @nedbat for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 1, 2024
* pythongh-124370: Add "howto" for free-threaded Python

This is a guide aimed at people writing Python code, as oppposed to the
existing guide for C API extension authors.

* Add missing new line

* Update Doc/howto/free-threading-python.rst

Co-authored-by: Jelle Zijlstra <[email protected]>

* interned -> immortalized

* Apply suggestions from code review

Co-authored-by: Hugo van Kemenade <[email protected]>

* Update Doc/howto/free-threading-python.rst

Co-authored-by: mpage <[email protected]>

* Update docs

* Apply suggestions from code review

Co-authored-by: Carol Willing <[email protected]>

* A few more updates

* Additional comment on immortal objects

* Mention specializing adaptive interpreter

* Remove trailing whitespace

* Remove mention of C macro

---------

(cherry picked from commit 68e384c)

Co-authored-by: Sam Gross <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: mpage <[email protected]>
Co-authored-by: Carol Willing <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Oct 1, 2024

GH-124860 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Oct 1, 2024
Yhg1s pushed a commit that referenced this pull request Oct 3, 2024
…124860)

gh-124370: Add "howto" for free-threaded Python (GH-124371)
(cherry picked from commit 68e384c)

Co-authored-by: Sam Gross <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>
Co-authored-by: Hugo van Kemenade <[email protected]>
Co-authored-by: mpage <[email protected]>
Co-authored-by: Carol Willing <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Documentation in the Doc dir skip news topic-free-threading
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants