Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-94597: Deprecate child watcher getters and setters #98215

Merged
merged 8 commits into from
Oct 15, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions Doc/library/asyncio-policy.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,12 +88,16 @@ The abstract event loop policy base class is defined as follows:

This function is Unix specific.

.. deprecated:: 3.12

.. method:: set_child_watcher(watcher)

Set the current child process watcher to *watcher*.

This function is Unix specific.

.. deprecated:: 3.12


.. _asyncio-policy-builtin:

Expand Down Expand Up @@ -158,12 +162,16 @@ implementation used by the asyncio event loop:

Return the current child watcher for the current policy.

.. deprecated:: 3.12

.. function:: set_child_watcher(watcher)

Set the current child watcher to *watcher* for the current
policy. *watcher* must implement methods defined in the
:class:`AbstractChildWatcher` base class.

.. deprecated:: 3.12

.. note::
Third-party event loops implementations might not support
custom child watchers. For such event loops, using
Expand Down Expand Up @@ -245,6 +253,8 @@ implementation used by the asyncio event loop:

.. versionadded:: 3.8

.. deprecated:: 3.12

.. class:: SafeChildWatcher

This implementation uses active event loop from the main thread to handle
Expand All @@ -257,6 +267,8 @@ implementation used by the asyncio event loop:
This solution is as safe as :class:`MultiLoopChildWatcher` and has the same *O(N)*
complexity but requires a running event loop in the main thread to work.

.. deprecated:: 3.12

.. class:: FastChildWatcher

This implementation reaps every terminated processes by calling
Expand All @@ -269,6 +281,8 @@ implementation used by the asyncio event loop:
This solution requires a running event loop in the main thread to work, as
:class:`SafeChildWatcher`.

.. deprecated:: 3.12

.. class:: PidfdChildWatcher

This implementation polls process file descriptors (pidfds) to await child
Expand Down
6 changes: 6 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,12 @@ asyncio
if supported and :class:`~asyncio.ThreadedChildWatcher` otherwise).
(Contributed by Kumar Aditya in :gh:`94597`.)

* :func:`asyncio.set_child_watcher`, :func:`asyncio.get_child_watcher`,
:meth:`asyncio.AbstractEventLoopPolicy.set_child_watcher` and
:meth:`asyncio.AbstractEventLoopPolicy.get_child_watcher` are deprecated
and will be removed in Python 3.14.
(Contributed by Kumar Aditya in :gh:`94597`.)


pathlib
-------
Expand Down
56 changes: 32 additions & 24 deletions Lib/asyncio/unix_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -195,30 +195,32 @@ def _make_write_pipe_transport(self, pipe, protocol, waiter=None,
async def _make_subprocess_transport(self, protocol, args, shell,
stdin, stdout, stderr, bufsize,
extra=None, **kwargs):
with events.get_child_watcher() as watcher:
if not watcher.is_active():
# Check early.
# Raising exception before process creation
# prevents subprocess execution if the watcher
# is not ready to handle it.
raise RuntimeError("asyncio.get_child_watcher() is not activated, "
"subprocess support is not installed.")
waiter = self.create_future()
transp = _UnixSubprocessTransport(self, protocol, args, shell,
stdin, stdout, stderr, bufsize,
waiter=waiter, extra=extra,
**kwargs)

watcher.add_child_handler(transp.get_pid(),
self._child_watcher_callback, transp)
try:
await waiter
except (SystemExit, KeyboardInterrupt):
raise
except BaseException:
transp.close()
await transp._wait()
raise
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
kumaraditya303 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't safe to disable warnings here, this is a process global and you're disabiling it for the whole wait

with events.get_child_watcher() as watcher:
if not watcher.is_active():
# Check early.
# Raising exception before process creation
# prevents subprocess execution if the watcher
# is not ready to handle it.
raise RuntimeError("asyncio.get_child_watcher() is not activated, "
"subprocess support is not installed.")
waiter = self.create_future()
transp = _UnixSubprocessTransport(self, protocol, args, shell,
stdin, stdout, stderr, bufsize,
waiter=waiter, extra=extra,
**kwargs)

watcher.add_child_handler(transp.get_pid(),
self._child_watcher_callback, transp)
try:
await waiter
except (SystemExit, KeyboardInterrupt):
raise
except BaseException:
transp.close()
await transp._wait()
raise

return transp

Expand Down Expand Up @@ -1469,6 +1471,9 @@ def get_child_watcher(self):
if self._watcher is None:
self._init_watcher()

warnings._deprecated("get_child_watcher",
"{name!r} is deprecated as of Python 3.12 and will be "
"removed in Python {remove}.", remove=(3, 14))
return self._watcher

def set_child_watcher(self, watcher):
Expand All @@ -1480,6 +1485,9 @@ def set_child_watcher(self, watcher):
self._watcher.close()

self._watcher = watcher
warnings._deprecated("set_child_watcher",
"{name!r} is deprecated as of Python 3.12 and will be "
"removed in Python {remove}.", remove=(3, 14))


SelectorEventLoop = _UnixSelectorEventLoop
Expand Down
16 changes: 10 additions & 6 deletions Lib/test/test_asyncio/test_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -2058,11 +2058,13 @@ def setUp(self):
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
watcher = asyncio.SafeChildWatcher()
watcher.attach_loop(self.loop)
asyncio.set_child_watcher(watcher)
watcher.attach_loop(self.loop)
asyncio.set_child_watcher(watcher)

def tearDown(self):
asyncio.set_child_watcher(None)
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
asyncio.set_child_watcher(None)
super().tearDown()


Expand Down Expand Up @@ -2657,13 +2659,15 @@ def setUp(self):
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
watcher = asyncio.SafeChildWatcher()
watcher.attach_loop(self.loop)
asyncio.set_child_watcher(watcher)
watcher.attach_loop(self.loop)
asyncio.set_child_watcher(watcher)

def tearDown(self):
try:
if sys.platform != 'win32':
asyncio.set_child_watcher(None)
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
asyncio.set_child_watcher(None)

super().tearDown()
finally:
Expand Down
8 changes: 6 additions & 2 deletions Lib/test/test_asyncio/test_streams.py
Original file line number Diff line number Diff line change
Expand Up @@ -797,15 +797,19 @@ def test_read_all_from_pipe_reader(self):
watcher = asyncio.SafeChildWatcher()
watcher.attach_loop(self.loop)
try:
asyncio.set_child_watcher(watcher)
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
asyncio.set_child_watcher(watcher)
create = asyncio.create_subprocess_exec(
*args,
pass_fds={wfd},
)
proc = self.loop.run_until_complete(create)
self.loop.run_until_complete(proc.wait())
finally:
asyncio.set_child_watcher(None)
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
asyncio.set_child_watcher(None)

os.close(wfd)
data = self.loop.run_until_complete(reader.read(-1))
Expand Down
29 changes: 20 additions & 9 deletions Lib/test/test_asyncio/test_subprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -582,7 +582,9 @@ async def kill_running():
# manually to avoid a warning when the watcher is detached.
if (sys.platform != 'win32' and
isinstance(self, SubprocessFastWatcherTests)):
asyncio.get_child_watcher()._callbacks.clear()
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
asyncio.get_child_watcher()._callbacks.clear()

async def _test_popen_error(self, stdin):
if sys.platform == 'win32':
Expand Down Expand Up @@ -696,13 +698,17 @@ def setUp(self):

watcher = self._get_watcher()
watcher.attach_loop(self.loop)
policy.set_child_watcher(watcher)
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
policy.set_child_watcher(watcher)

def tearDown(self):
super().tearDown()
policy = asyncio.get_event_loop_policy()
watcher = policy.get_child_watcher()
policy.set_child_watcher(None)
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
watcher = policy.get_child_watcher()
policy.set_child_watcher(None)
watcher.attach_loop(None)
watcher.close()

Expand Down Expand Up @@ -752,7 +758,9 @@ def test_create_subprocess_fails_with_inactive_watcher(self):
)

async def execute():
asyncio.set_child_watcher(watcher)
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
asyncio.set_child_watcher(watcher)

with self.assertRaises(RuntimeError):
await subprocess.create_subprocess_exec(
Expand All @@ -761,7 +769,9 @@ async def execute():
watcher.add_child_handler.assert_not_called()

with asyncio.Runner(loop_factory=asyncio.new_event_loop) as runner:
self.assertIsNone(runner.run(execute()))
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
self.assertIsNone(runner.run(execute()))
self.assertListEqual(watcher.mock_calls, [
mock.call.__enter__(),
mock.call.__enter__().is_active(),
Expand All @@ -788,15 +798,16 @@ async def main():
with self.assertRaises(RuntimeError):
asyncio.get_event_loop_policy().get_event_loop()
return await asyncio.to_thread(asyncio.run, in_thread())

asyncio.set_child_watcher(asyncio.PidfdChildWatcher())
with self.assertWarns(DeprecationWarning):
asyncio.set_child_watcher(asyncio.PidfdChildWatcher())
try:
with asyncio.Runner(loop_factory=asyncio.new_event_loop) as runner:
returncode, stdout = runner.run(main())
self.assertEqual(returncode, 0)
self.assertEqual(stdout, b'some data')
finally:
asyncio.set_child_watcher(None)
with self.assertWarns(DeprecationWarning):
asyncio.set_child_watcher(None)
else:
# Windows
class SubprocessProactorTests(SubprocessMixin, test_utils.TestCase):
Expand Down
21 changes: 12 additions & 9 deletions Lib/test/test_asyncio/test_unix_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -1709,33 +1709,36 @@ def test_get_default_child_watcher(self):
self.assertIsNone(policy._watcher)
unix_events.can_use_pidfd = mock.Mock()
unix_events.can_use_pidfd.return_value = False
watcher = policy.get_child_watcher()
with self.assertWarns(DeprecationWarning):
watcher = policy.get_child_watcher()
self.assertIsInstance(watcher, asyncio.ThreadedChildWatcher)

self.assertIs(policy._watcher, watcher)

self.assertIs(watcher, policy.get_child_watcher())
with self.assertWarns(DeprecationWarning):
self.assertIs(watcher, policy.get_child_watcher())

policy = self.create_policy()
self.assertIsNone(policy._watcher)
unix_events.can_use_pidfd = mock.Mock()
unix_events.can_use_pidfd.return_value = True
watcher = policy.get_child_watcher()
with self.assertWarns(DeprecationWarning):
watcher = policy.get_child_watcher()
self.assertIsInstance(watcher, asyncio.PidfdChildWatcher)

self.assertIs(policy._watcher, watcher)

self.assertIs(watcher, policy.get_child_watcher())
with self.assertWarns(DeprecationWarning):
self.assertIs(watcher, policy.get_child_watcher())

def test_get_child_watcher_after_set(self):
policy = self.create_policy()
with warnings.catch_warnings():
warnings.simplefilter("ignore", DeprecationWarning)
watcher = asyncio.FastChildWatcher()
policy.set_child_watcher(watcher)

policy.set_child_watcher(watcher)
self.assertIs(policy._watcher, watcher)
self.assertIs(watcher, policy.get_child_watcher())
with self.assertWarns(DeprecationWarning):
self.assertIs(watcher, policy.get_child_watcher())

def test_get_child_watcher_thread(self):

Expand Down Expand Up @@ -1769,7 +1772,7 @@ def test_child_watcher_replace_mainloop_existing(self):
with warnings.catch_warnings():
warnings.simplefilter("ignore", DeprecationWarning)
watcher = asyncio.SafeChildWatcher()
policy.set_child_watcher(watcher)
policy.set_child_watcher(watcher)
watcher.attach_loop(loop)

self.assertIs(watcher._loop, loop)
Expand Down
6 changes: 4 additions & 2 deletions Lib/test/test_asyncio/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
import threading
import unittest
import weakref

import warnings
from unittest import mock

from http.server import HTTPServer
Expand Down Expand Up @@ -544,7 +544,9 @@ def close_loop(loop):
policy = support.maybe_get_event_loop_policy()
if policy is not None:
try:
watcher = policy.get_child_watcher()
with warnings.catch_warnings():
warnings.simplefilter('ignore', DeprecationWarning)
watcher = policy.get_child_watcher()
except NotImplementedError:
# watcher is not implemented by EventLoopPolicy, e.g. Windows
pass
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Deprecated :meth:`asyncio.AbstractEventLoopPolicy.get_child_watcher` and :meth:`asyncio.AbstractEventLoopPolicy.set_child_watcher` methods to be removed in Python 3.14. Patch by Kumar Aditya.