test_poplib hangs with the changes done in PR

[jayyyin]

BPO	33099
Nosy	@smontanaro, @tiran, @methane, @jayyyin, @iritkatriel
PRs	bpo-33099: Fix test_poplib hangs after error #6428

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-03-18.22:18:03.823>
labels = ['3.8', '3.7', 'tests']
title = 'test_poplib hangs with the changes done in PR'
updated_at = <Date 2021-12-13.17:00:34.891>
user = 'https://github.com/jayyyin'

bugs.python.org fields:

activity = <Date 2021-12-13.17:00:34.891>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2018-03-18.22:18:03.823>
creator = 'jayyin11043'
dependencies = []
files = []
hgrepos = []
issue_num = 33099
keywords = ['patch']
message_count = 8.0
messages = ['314064', '315118', '315120', '315121', '315141', '315142', '315147', '408465']
nosy_count = 5.0
nosy_names = ['skip.montanaro', 'christian.heimes', 'methane', 'jayyin11043', 'iritkatriel']
pr_nums = ['6428']
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue33099'
versions = ['Python 3.7', 'Python 3.8']

[jayyyin]

my test hangs locally on my computer with the changes I've done in bpo-32642 but doesn't hang on TravisCI, anyone able to help with checking what's wrong here (sounds like another edge case with my env but I could be wrong)

the trace for the command https://pastebin.com/q4FKnPZH

[methane]

There are some resource leaks:

• When error occurred in setUp() function, server thread is not stopped. It leaked threads and sockets for the server.
• When error occurred in server thread's run() method, asyncore.close_all() is not called.

These leaks makes dangling threads. It caused hang in test_poplib shutdown process.

Additionally, the error is not printed in these cases. It makes harder to fix real problem.

[tiran]

This is likely a duplicate of BPO bpo-32706 and bpo-32753

[methane]

@christian

Precisely, my PR doesn't fix bpo-32706 and bpo-32753.
It fixes only hang caused by them and print stack trace for them.

[smontanaro]

Just in case this would be useful to others, git bisect led me to this commit as the source of the problem:

61d478c is the first bad commit
commit 61d478c
Author: Christian Heimes <[email protected]>
Date: Sat Jan 27 15:51:38 2018 +0100

bpo-31399: Let OpenSSL verify hostname and IP address (bpo-3462)

bpo-31399: Let OpenSSL verify hostname and IP

The ssl module now uses OpenSSL's X509_VERIFY_PARAM_set1_host() and
X509_VERIFY_PARAM_set1_ip() API to verify hostname and IP addresses.

* Remove match_hostname calls
* Check for libssl with set1_host, libssl must provide X509_VERIFY_PARAM_set1_host()
* Add documentation for OpenSSL 1.0.2 requirement
* Don't support OpenSSL special mode with a leading dot, e.g. ".example.org" matches "www.example.org". It's not standard conform.
* Add hostname_checks_common_name

Signed-off-by: Christian Heimes <[email protected]>

:040000 040000 09f4c8a18941f926c1f79e2f07dfd7731edf74c2 798ae3fab880e3365c490d56935901a4c18fd3bd M Doc
:040000 040000 f35a1b4e2144050dfc4c7dbb02b7d92e3de1c2d0 7af4e4a8e82eafc6e7f2f4392c951213777f12c4 M Lib
:040000 040000 e287c27f4db61beae905c21b96666194bb930c13 59e6fc63d296f7fe3187ff0987a04c9999e2d1d0 M Misc
:040000 040000 95250f4c1009b5f10ec392c186908698d78f9cd3 160ebeb2c63d1ca305c7e37a0b555900b5084d20 M Modules
:040000 040000 8adc7f664a8eaa667f4540f537304d7f9de672d4 41ea621f5433077f6dd776f4c2c51f7a8e4cab0f M PC
:100644 100644 a6f4488cc99941abaa8b6fb7c53079b7cc292f9f ba0a7624cfcd0129e73bc218510f398001514df8 M setup.py

[tiran]

Thanks for you confirmation. I already knew which commit lead to the issue. :)

The problem didn't "pop" up in my local tests or on Travis CI. I can only reproduce it locally, when I run the test in a tight loop and put extra load on the other CPU cores.

[smontanaro]

@christian.heimes I figured you probably didn't need it, but I did that work before I discovered this ticket. And as a nice bonus, I got to learn git bisect. :-)

[iritkatriel]

bpo-32706 is now fixed, does that resolve this issue as well?