gh-109974: Fix more threading lock_tests race conditions #110089

vstinner · 2023-09-29T09:13:01Z

Add context manager on Bunch class.
Bunch now catchs exceptions on executed functions and re-raise them at exit() as an ExceptionGroup.
Rewrite BarrierProxy.test_default_timeout(). Use a single thread. Only check that barrier.wait() blocks for at least default timeout seconds.
test_with(): inline _with() function.

Issue: test_threading: test_set_and_clear() failed on Windows x64 #109974

* Add context manager on Bunch class. * Bunch now catchs exceptions on executed functions and re-raise them at __exit__() as an ExceptionGroup. * Rewrite BarrierProxy.test_default_timeout(). Use a single thread. Only check that barrier.wait() blocks for at least default timeout seconds. * test_with(): inline _with() function.

vstinner · 2023-09-29T11:24:13Z

Windows x64 failed with:


0:22:17 load avg: 0.08 [467/467/1] test.test_concurrent_futures.test_shutdown process crashed (Exit code 1)
Timeout (0:10:00)!
Thread 0x00001b44 (most recent call first):
  File "D:\a\cpython\cpython\Lib\subprocess.py", line 1597 in _readerthread
  File "D:\a\cpython\cpython\Lib\threading.py", line 1003 in run
  File "D:\a\cpython\cpython\Lib\threading.py", line 1066 in _bootstrap_inner
  File "D:\a\cpython\cpython\Lib\threading.py", line 1023 in _bootstrap

Thread 0x00000e08 (most recent call first):
  File "D:\a\cpython\cpython\Lib\subprocess.py", line 1597 in _readerthread
  File "D:\a\cpython\cpython\Lib\threading.py", line 1003 in run
  File "D:\a\cpython\cpython\Lib\threading.py", line 1066 in _bootstrap_inner
  File "D:\a\cpython\cpython\Lib\threading.py", line 1023 in _bootstrap

Thread 0x00001a48 (most recent call first):
  File "D:\a\cpython\cpython\Lib\threading.py", line 1160 in _wait_for_tstate_lock
  File "D:\a\cpython\cpython\Lib\threading.py", line 1140 in join
  File "D:\a\cpython\cpython\Lib\subprocess.py", line 1626 in _communicate
  File "D:\a\cpython\cpython\Lib\subprocess.py", line 1209 in communicate
  File "D:\a\cpython\cpython\Lib\test\support\script_helper.py", line 139 in run_python_until_end
  File "D:\a\cpython\cpython\Lib\test\support\script_helper.py", line 149 in _assert_python
  File "D:\a\cpython\cpython\Lib\test\support\script_helper.py", line 166 in assert_python_ok
  File "D:\a\cpython\cpython\Lib\test\test_concurrent_futures\test_shutdown.py", line 32 in test_interpreter_shutdown

vstinner · 2023-09-29T11:45:04Z

Windows x64 failed with: FAIL: test_wait_for_timeout_less_then_0_or_0_coroutine_do_not_started (test.test_asyncio.test_waitfor.AsyncioWaitForTest.test_wait_for_timeout_less_then_0_or_0_coroutine_do_not_started).

Good news: PR #110092 is going to fix it.

…n#110089) * Add context manager on Bunch class. * Bunch now catchs exceptions on executed functions and re-raise them at __exit__() as an ExceptionGroup. * Rewrite BarrierProxy.test_default_timeout(). Use a single thread. Only check that barrier.wait() blocks for at least default timeout seconds. * test_with(): inline _with() function. (cherry picked from commit 743e357)

…110346) * gh-109974: Fix threading lock_tests race conditions (#110057) Fix race conditions in test_threading lock tests. Wait until a condition is met rather than using time.sleep() with a hardcoded number of seconds. * Replace sleeping loops with support.sleeping_retry() which raises an exception on timeout. * Add wait_threads_blocked(nthread) which computes a sleep depending on the number of threads. Remove _wait() function. * test_set_and_clear(): use a way longer Event.wait() timeout. * BarrierTests.test_repr(): wait until the 2 threads are waiting for the barrier. Use a way longer timeout for Barrier.wait() timeout. * test_thread_leak() no longer needs to count len(threading.enumerate()): Bunch uses threading_helper.wait_threads_exit() internally which does it in wait_for_finished(). * Add BaseLockTests.wait_phase() which implements a timeout. test_reacquire() and test_recursion_count() use wait_phase(). (cherry picked from commit 4e356ad) * gh-109974: Fix more threading lock_tests race conditions (#110089) * Add context manager on Bunch class. * Bunch now catchs exceptions on executed functions and re-raise them at __exit__() as an ExceptionGroup. * Rewrite BarrierProxy.test_default_timeout(). Use a single thread. Only check that barrier.wait() blocks for at least default timeout seconds. * test_with(): inline _with() function. (cherry picked from commit 743e357)

…110355) [3.12] gh-109974: Fix threading lock_tests race conditions (#110057) (#110346) * gh-109974: Fix threading lock_tests race conditions (#110057) Fix race conditions in test_threading lock tests. Wait until a condition is met rather than using time.sleep() with a hardcoded number of seconds. * Replace sleeping loops with support.sleeping_retry() which raises an exception on timeout. * Add wait_threads_blocked(nthread) which computes a sleep depending on the number of threads. Remove _wait() function. * test_set_and_clear(): use a way longer Event.wait() timeout. * BarrierTests.test_repr(): wait until the 2 threads are waiting for the barrier. Use a way longer timeout for Barrier.wait() timeout. * test_thread_leak() no longer needs to count len(threading.enumerate()): Bunch uses threading_helper.wait_threads_exit() internally which does it in wait_for_finished(). * Add BaseLockTests.wait_phase() which implements a timeout. test_reacquire() and test_recursion_count() use wait_phase(). (cherry picked from commit 4e356ad) * gh-109974: Fix more threading lock_tests race conditions (#110089) * Add context manager on Bunch class. * Bunch now catchs exceptions on executed functions and re-raise them at __exit__() as an ExceptionGroup. * Rewrite BarrierProxy.test_default_timeout(). Use a single thread. Only check that barrier.wait() blocks for at least default timeout seconds. * test_with(): inline _with() function. (cherry picked from commit 743e357) (cherry picked from commit 1d032ea)

…n#110089) * Add context manager on Bunch class. * Bunch now catchs exceptions on executed functions and re-raise them at __exit__() as an ExceptionGroup. * Rewrite BarrierProxy.test_default_timeout(). Use a single thread. Only check that barrier.wait() blocks for at least default timeout seconds. * test_with(): inline _with() function.

bedevere-app bot added the awaiting review label Sep 29, 2023

bedevere-app bot mentioned this pull request Sep 29, 2023

test_threading: test_set_and_clear() failed on Windows x64 #109974

Closed

vstinner force-pushed the lock_tests2 branch from 68d5deb to 7173696 Compare September 29, 2023 09:52

vstinner requested review from brettcannon, ericsnowcurrently, ncoghlan and warsaw as code owners September 29, 2023 09:52

vstinner added the skip news label Sep 29, 2023

vstinner enabled auto-merge (squash) September 29, 2023 10:09

Merge branch 'main' into lock_tests2

3b0ab76

vstinner merged commit 743e357 into python:main Sep 29, 2023
21 checks passed

vstinner deleted the lock_tests2 branch September 29, 2023 12:21

bedevere-app bot removed the awaiting review label Sep 29, 2023

vstinner mentioned this pull request Oct 4, 2023

[3.11][3.12] gh-109974: Fix threading lock_tests race conditions (#110057) … #110355

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-109974: Fix more threading lock_tests race conditions #110089

gh-109974: Fix more threading lock_tests race conditions #110089

vstinner commented Sep 29, 2023 •

edited by bedevere-app bot

Loading

vstinner commented Sep 29, 2023

vstinner commented Sep 29, 2023

gh-109974: Fix more threading lock_tests race conditions #110089

gh-109974: Fix more threading lock_tests race conditions #110089

Conversation

vstinner commented Sep 29, 2023 • edited by bedevere-app bot Loading

vstinner commented Sep 29, 2023

vstinner commented Sep 29, 2023

vstinner commented Sep 29, 2023 •

edited by bedevere-app bot

Loading