Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The PyOS_* hooks interact poorly with subinterpreters #104668

Closed
brandtbucher opened this issue May 19, 2023 · 1 comment
Closed

The PyOS_* hooks interact poorly with subinterpreters #104668

brandtbucher opened this issue May 19, 2023 · 1 comment
Assignees
@brandtbucher
Labels
3.12 bugs and security fixes 3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-C-API topic-subinterpreters topic-tkinter type-bug An unexpected behavior, bug, or error

Comments

@brandtbucher
Copy link
Member

brandtbucher commented May 19, 2023 •
edited by bedevere-bot
Loading

PyOS_InputHook and PyOS_ReadlineFunctionPointer are globally-registered input hooks that have no way of recovering module-level state for the extensions that registered them. In practice, extensions like readline and tkinter get around this by using global state, which obviously isn't subinterpreter-friendly.

What's more, extensions without advertised subinterpreter support (like readline and tkinter) who register these hooks might find themselves called from within a subinterpreter (where their extension hasn't even been loaded). That's definitely a situation we want to avoid.

It seems like the best solution for 3.12 is to only call these hooks from the main interpreter, which makes the bad situation quite a bit better. If libraries really need better APIs that work per-interpreter and have ways to access module state, we can certainly add them later (but for now maybe we'll just cross our fingers and hope that nobody actually cares).

@ericsnowcurrently, how does this sound to you?

Linked PRs
@brandtbucher brandtbucher added type-bug An unexpected behavior, bug, or error interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-tkinter topic-subinterpreters topic-C-API 3.12 bugs and security fixes labels May 19, 2023
@brandtbucher brandtbucher self-assigned this May 19, 2023
@bedevere-bot bedevere-bot mentioned this issue May 19, 2023
Merged
@brandtbucher brandtbucher mentioned this issue May 19, 2023
Open
25 tasks
@arhadthedev arhadthedev added the 3.13 bugs and security fixes label May 22, 2023
brandtbucher added a commit that referenced this issue May 22, 2023
@brandtbucher
GH-104668: Don't call PyOS_* hooks in subinterpreters (GH-104674)
357bed0
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 22, 2023
@brandtbucher @miss-islington
pythonGH-104668: Don't call PyOS_* hooks in subinterpreters (pythonGH…
ef66708 
…-104674)

(cherry picked from commit 357bed0)

Co-authored-by: Brandt Bucher <[email protected]>
@bedevere-bot bedevere-bot mentioned this issue May 22, 2023
Merged
brandtbucher added a commit that referenced this issue May 23, 2023
@miss-islington @brandtbucher
[3.12] GH-104668: Don't call PyOS_* hooks in subinterpreters (GH-104760)
905d419 
GH-104668: Don't call PyOS_* hooks in subinterpreters (GH-104674)
(cherry picked from commit 357bed0)

Co-authored-by: Brandt Bucher <[email protected]>
@ericsnowcurrently
Copy link
Member

Thanks for getting this sorted out, @brandtbucher.

@erlend-aasland erlend-aasland mentioned this issue Jun 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brandtbucher brandtbucher

Labels
3.12 bugs and security fixes 3.13 bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-C-API topic-subinterpreters topic-tkinter type-bug An unexpected behavior, bug, or error
Projects
Subinterpreters
Status: Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ericsnowcurrently @arhadthedev @brandtbucher