Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_threading_local_clear_race segmentation fault in free-threaded CI #118490

Open
colesbury opened this issue May 1, 2024 · 1 comment
Open

test_threading_local_clear_race segmentation fault in free-threaded CI #118490

colesbury opened this issue May 1, 2024 · 1 comment
Labels
tests Tests in the Lib/test dir topic-free-threading type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@colesbury
Copy link
Contributor

colesbury commented May 1, 2024 •
edited by github-actions bot
Loading

Crash report

A segmentation fault in the macOS M1 arm64 refleaks buildbot:

Seen in https://buildbot.python.org/all/#/builders/1367/builds/146/steps/5/logs/stdio

test_threading_local_clear_race (test.test_threading_local.PyThreadingLocalTest.test_threading_local_clear_race) ... Fatal Python error: Segmentation fault
Current thread 0x00000001d8941300 (most recent call first):
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_threading_local.py", line 206 in test_threading_local_clear_race
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/case.py", line 606 in _callTestMethod
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/case.py", line 651 in run
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/case.py", line 707 in __call__
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/suite.py", line 122 in run
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/suite.py", line 84 in __call__
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/suite.py", line 122 in run
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/suite.py", line 84 in __call__
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/unittest/runner.py", line 240 in run
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 57 in _run_suite
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 37 in run_unittest
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 132 in test_func
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/refleak.py", line 106 in runtest_refleak
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 84 in regrtest_runner
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 135 in _load_run_test
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 178 in _runtest_env_changed_exc
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 278 in _runtest
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/single.py", line 309 in run_single_test
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/worker.py", line 77 in worker_process
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/worker.py", line 112 in main
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/libregrtest/worker.py", line 116 in <module>
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/runpy.py", line 88 in _run_code
  File "/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/runpy.py", line 198 in _run_module_as_main
@colesbury colesbury added tests Tests in the Lib/test dir type-crash A hard crash of the interpreter, possibly with a core dump topic-free-threading labels May 1, 2024
@colesbury
Copy link
Contributor Author 

WARNING: ThreadSanitizer: heap-use-after-free (pid=1832131)
  Read of size 8 at 0x72a400041018 by main thread:
    #0 PyThreadState_Next /raid/sgross/cpython/Python/pystate.c:2488:20 (python+0x58fd3a) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #1 local_clear /raid/sgross/cpython/./Modules/_threadmodule.c:1579:22 (python+0x67b585) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #2 local_dealloc /raid/sgross/cpython/./Modules/_threadmodule.c:1597:5 (python+0x67abbe) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #3 _Py_Dealloc /raid/sgross/cpython/Objects/object.c:2855:5 (python+0x30ff14) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #4 _Py_MergeZeroLocalRefcount /raid/sgross/cpython/Objects/object.c (python+0x310542) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #5 Py_DECREF /raid/sgross/cpython/./Include/object.h:890:13 (python+0x49d3b8) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #6 Py_XDECREF /raid/sgross/cpython/./Include/object.h:1040:9 (python+0x49d3b8)
    #7 _PyEval_EvalFrameDefault /raid/sgross/cpython/Python/generated_cases.c.h:5627:13 (python+0x49d3b8)
    #8 _PyEval_EvalFrame /raid/sgross/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x47a833) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #9 _PyEval_Vector /raid/sgross/cpython/Python/ceval.c:1818:12 (python+0x47a833)
    #10 _PyFunction_Vectorcall /raid/sgross/cpython/Objects/call.c (python+0x25552c) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #11 _PyObject_VectorcallTstate /raid/sgross/cpython/./Include/internal/pycore_call.h:168:11 (python+0x25b01b) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #12 method_vectorcall /raid/sgross/cpython/Objects/classobject.c:92:18 (python+0x259722) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #13 _PyVectorcall_Call /raid/sgross/cpython/Objects/call.c:273:16 (python+0x254f15) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #14 _PyObject_Call /raid/sgross/cpython/Objects/call.c:348:16 (python+0x25512b) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #15 PyObject_Call /raid/sgross/cpython/Objects/call.c:373:12 (python+0x255357) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #16 _PyEval_EvalFrameDefault /raid/sgross/cpython/Python/generated_cases.c.h:1353:26 (python+0x483601) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #17 _PyEval_EvalFrame /raid/sgross/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x47a833) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #18 _PyEval_Vector /raid/sgross/cpython/Python/ceval.c:1818:12 (python+0x47a833)
    #19 _PyFunction_Vectorcall /raid/sgross/cpython/Objects/call.c (python+0x25552c) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #20 _PyObject_VectorcallDictTstate /raid/sgross/cpython/Objects/call.c:135:15 (python+0x25392d) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #21 _PyObject_Call_Prepend /raid/sgross/cpython/Objects/call.c:504:24 (python+0x25594d) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #22 slot_tp_call /raid/sgross/cpython/Objects/typeobject.c:9400:15 (python+0x3822bc) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
...

  Previous write of size 8 at 0x72a400041018 by thread T371:
    #0 free <null> (python+0xdc10f) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #1 _PyMem_RawFree /raid/sgross/cpython/Objects/obmalloc.c:90:5 (python+0x337339) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #2 _PyMem_DebugRawFree /raid/sgross/cpython/Objects/obmalloc.c:2754:5 (python+0x33c14b) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #3 PyMem_RawFree /raid/sgross/cpython/Objects/obmalloc.c:963:5 (python+0x3398a8) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #4 free_threadstate /raid/sgross/cpython/Python/pystate.c:1436:9 (python+0x590fe4) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
    #5 _PyThreadState_DeleteCurrent /raid/sgross/cpython/Python/pystate.c:1838:5 (python+0x590fe4)
    #6 PyGILState_Release /raid/sgross/cpython/Python/pystate.c:2812:9 (python+0x590fe4)
    #7 temporary_c_thread /raid/sgross/cpython/./Modules/_testcapimodule.c:1623:5 (_testcapi.cpython-313td-x86_64-linux-gnu.so+0x1d684) (BuildId: 7b30b566ddc9cb08f2fcc308c2a802d985ffb2ef)
    #8 pythread_wrapper /raid/sgross/cpython/Python/thread_pthread.h:243:5 (python+0x5b54bb) (BuildId: 99e53ab3ef9a039265821c7e1b283c799a17a778)
@corona10 corona10 mentioned this issue Jun 29, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tests Tests in the Lib/test dir topic-free-threading type-crash A hard crash of the interpreter, possibly with a core dump
1 participant
@colesbury