TSan reported race condition in _PyPegen_is_memoized in test_importlib

[colesbury]

Bug report

I think this is a race on the global _PyRuntime.parser.memo_statistics. That's only used in the debug build for stats, so the race isn't particularly dangerous, but we should fix it.

WARNING: ThreadSanitizer: data race (pid=8921)
  Write of size 8 at 0x55b27b2a92f0 by thread T84:
    #0 _PyPegen_is_memoized /home/runner/work/cpython/cpython/Parser/pegen.c:351:39 (python+0x166292) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #1 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:[12](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:13)893:9 (python+0x1842c2) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #2 bitwise_or_raw /home/runner/work/cpython/cpython/Parser/parser.c:12951:18 (python+0x184511) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #3 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12906:22 (python+0x184511)
    #4 comparison_rule /home/runner/work/cpython/cpython/Parser/parser.c:12146:18 (python+0x19bc7f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #5 inversion_rule /home/runner/work/cpython/cpython/Parser/parser.c:12097:31 (python+0x19b869) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #6 conjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11974:18 (python+0x19a9b8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #7 disjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11886:18 (python+0x1938f8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #8 expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11174:18 (python+0x182e81) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #9 named_expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11832:31 (python+0x181b40) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #10 if_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:5927:18 (python+0x1769b7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #11 compound_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:2114:28 (python+0x1769b7)
    #12 statement_rule /home/runner/work/cpython/cpython/Parser/parser.c:1419:18 (python+0x17582f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #[13](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:14) _loop1_3_rule /home/runner/work/cpython/cpython/Parser/parser.c:25738:30 (python+0x17582f)
    #14 statements_rule /home/runner/work/cpython/cpython/Parser/parser.c:1376:18 (python+0x17582f)
    #15 file_rule /home/runner/work/cpython/cpython/Parser/parser.c:1178:18 (python+0x170f5a) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #16 _PyPegen_parse /home/runner/work/cpython/cpython/Parser/parser.c:42410:18 (python+0x170f5a)
    #17 _PyPegen_run_parser /home/runner/work/cpython/cpython/Parser/pegen.c:869:17 (python+0x167e01) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #18 _PyPegen_run_parser_from_string /home/runner/work/cpython/cpython/Parser/pegen.c:992:[14](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:15) (python+0x1684e7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #19 _PyParser_ASTFromString /home/runner/work/cpython/cpython/Parser/peg_api.c:13:21 (python+0x2175dd) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #20 Py_CompileStringObject /home/runner/work/cpython/cpython/Python/pythonrun.c:1435:11 (python+0x5fa5ab) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
...

Previous write of size 8 at 0x55b27b2a92f0 by thread T83:
    #0 _PyPegen_is_memoized /home/runner/work/cpython/cpython/Parser/pegen.c:351:39 (python+0x166292) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #1 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12893:9 (python+0x1842c2) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #2 bitwise_or_raw /home/runner/work/cpython/cpython/Parser/parser.c:12951:18 (python+0x184511) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #3 bitwise_or_rule /home/runner/work/cpython/cpython/Parser/parser.c:12906:22 (python+0x184511)
    #4 comparison_rule /home/runner/work/cpython/cpython/Parser/parser.c:12146:18 (python+0x19bc7f) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #5 inversion_rule /home/runner/work/cpython/cpython/Parser/parser.c:12097:31 (python+0x19b869) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #6 conjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11974:18 (python+0x19a9b8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #7 disjunction_rule /home/runner/work/cpython/cpython/Parser/parser.c:11886:18 (python+0x1938f8) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #8 expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11174:18 (python+0x182e81) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #9 star_expression_rule /home/runner/work/cpython/cpython/Parser/parser.c:11551:31 (python+0x1a0839) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #10 star_expressions_rule /home/runner/work/cpython/cpython/Parser/parser.c:11391:18 (python+0x19f602) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #11 simple_stmt_rule /home/runner/work/cpython/cpython/Parser/parser.c:17[63](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:64):18 (python+0x20241e) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #12 simple_stmts_rule /home/runner/work/cpython/cpython/Parser/parser.c:1618:18 (python+0x177d4a) (BuildId: 5a0193[64](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:65)4aa87641efb42b5a646a83190f2fd33b)
    #13 statement_rule /home/runner/work/cpython/cpython/Parser/parser.c:1443:34 (python+0x175911) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #14 _loop1_3_rule /home/runner/work/cpython/cpython/Parser/parser.c:25738:30 (python+0x175911)
    #15 statements_rule /home/runner/work/cpython/cpython/Parser/parser.c:1376:18 (python+0x175911)
    #16 file_rule /home/runner/work/cpython/cpython/Parser/parser.c:1178:18 (python+0x170f5a) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #17 _PyPegen_parse /home/runner/work/cpython/cpython/Parser/parser.c:42410:18 (python+0x170f5a)
    #18 _PyPegen_run_parser /home/runner/work/cpython/cpython/Parser/pegen.c:869:17 (python+0x1[67](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:68)e01) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #19 _PyPegen_run_parser_from_string /home/runner/work/cpython/cpython/Parser/pegen.c:992:14 (python+0x1[68](https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577#step:13:69)4e7) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #20 _PyParser_ASTFromString /home/runner/work/cpython/cpython/Parser/peg_api.c:13:21 (python+0x2175dd) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
    #21 Py_CompileStringObject /home/runner/work/cpython/cpython/Python/pythonrun.c:1435:11 (python+0x5fa5ab) (BuildId: 5a0193644aa87641efb42b5a646a83190f2fd33b)
...

Job: https://github.com/python/cpython/actions/runs/10204655178/job/28233674832?pr=122577
Full log archive: https://gist.github.com/colesbury/bfd65049b17505fcae64f5afd3ed388e

Linked PRs

• gh-122581: Avoid data races when collecting parser statistics #122694
• [3.13] gh-122581: Avoid data races when collecting parser statistics (GH-122694) #122733

[colesbury]

cc @lysnikolaou, since you were looking at parser thread-safety recently