[C-API Docs] Clarify About When PyMem_SetAllocator() Can Be Called

[ericsnowcurrently]

There are two uses for PyMem_SetAllocator():

1. set the allocator for the runtime to use
2. set an allocator hook that wraps the current allocator

The first must be done (only by embedders) "just after the pre-initialization, and before the Python initialization", while the second may be done at any point (e.g. by extension modules, like tracemalloc). See #96975 (comment).

The docs for PyMem_SetAllocator() should make the distinction more clear, as well as identifying the restriction on (1).

• PR: gh-96997: Clarify the contract of PyMem_SetAllocator() #98977

• PR: [3.11] gh-96997: Clarify the contract of PyMem_SetAllocator() (GH-98977) #99027

• PR: [3.10] gh-96997: Clarify the contract of PyMem_SetAllocator() (GH-98977) #99028

[ericsnowcurrently]

CC @vstinner

[encukou]

Victor mentioned that some limitations for PyMem_SetAllocator are documented elsewhere:

PyMem_SetAllocator() can be called after Py_PreInitialize() and before Py_InitializeFromConfig() to install a custom memory allocator. It can be called before Py_PreInitialize() if PyPreConfig.allocator is set to PYMEM_ALLOCATOR_NOT_SET.

[pablogsal]

I am preparing a PR to document this in the API docs, which I think is the correct place. This should also be a warning box, I think

[vstinner]

Pablo's documentation LGTM.

[vstinner]

Allocating memory is easy. The problem is more to know how to release allocated memory :-) Python/initconfig.c temporarily override the memory allocators to get a known memory allocator at exit. Example:

    /* Py_SetStandardStreamEncoding() can be called before Py_Initialize(),
       but Py_Initialize() can change the allocator. Use a known allocator
       to be able to release the memory later. */
    PyMemAllocatorEx old_alloc;
    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

    // ... code using PyMem_RawMalloc() and PyMem_RawFree() ...

    PyMem_SetAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

An alternative would be to add functions with a "context" parameter which can be used to get memory allocator functions. I considered that for Python/fileutils.c functions like Py_DecodeLocale() and _Py_EncodeLocaleRaw() which use PyMem_RawMalloc(). But I wasn't convinced that it's worth it, since _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc) is only used in a few places.

Code search, where I annotated public functions:

$ git grep '_PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW'
Objects/obmalloc.c:        (void)_PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, NULL);

# PyImport_ExtendInittab()
Python/import.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/import.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

# Py_SetStandardStreamEncoding()
Python/initconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/initconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/initconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/initconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

# Py_SetPath(), Py_SetPythonHome() and other deprecated functions
Python/pathconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pathconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pathconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pathconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pathconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pathconfig.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

# _PyRuntimeState_Init(), _PyRuntimeState_ReInitThreads()
Python/pystate.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pystate.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pystate.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/pystate.c:        _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);

# PySys_AddWarnOption(), PySys_AddXOption()
Python/sysmodule.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);
Python/sysmodule.c:    _PyMem_SetDefaultAllocator(PYMEM_DOMAIN_RAW, &old_alloc);