gh-127933: Add option to run regression tests in parallel#128003
gh-127933: Add option to run regression tests in parallel#128003colesbury merged 11 commits intopython:mainfrom
Conversation
This adds a new command line argument, `--parallel-threads` to the regression test runner to allow it to run individual tests in multiple threads in parallel in order to find multithreading bugs.
Co-authored-by: Tomas R. <tomas.roun8@gmail.com>
mpage
left a comment
mpage
left a comment
There was a problem hiding this comment.
This looks good to me. I'm a little unsure about marking tests as thread-unsafe at the source level. I worry a bit that the use of thread_unsafe / __unittest_thread_unsafe__ will create too much noise, but I think we can see how it goes.
| barrier = threading.Barrier(self.num_threads) | ||
| threads = [] | ||
| for case, r in zip(cases, results): | ||
| thread = threading.Thread(target=self.run_worker, |
There was a problem hiding this comment.
It might be nice to give these threads an informative name, something containing the testcase and the Nth thread it is.
|
|
||
| # Note: We can't call result.addError, result.addFailure, etc. because | ||
| # we no longer have the original exception, just the string format. | ||
| for r in results: |
There was a problem hiding this comment.
This looks like something that might be worth adding to unittest.result.TestResult, something to aggregate multiple results. (Not for this PR, but it might be a nice small feature for someone to work on.)
| result.addSuccess(self) | ||
|
|
||
| # Note: We can't call result.addError, result.addFailure, etc. because | ||
| # we no longer have the original exception, just the string format. |
There was a problem hiding this comment.
FWIW, another reason not to call those is that they raise right away if failfast is set.
| @@ -0,0 +1,3 @@ | |||
| Add an option ``--parallel-threads=N`` to the regression test runner that | |||
| runs individual tests in multiple threads in parallel in order to find | |||
| concurrency bugs. | |||
There was a problem hiding this comment.
Should this (or other documentation) mention that most of the testsuite isn't appropriately reviewed/marked yet?
There was a problem hiding this comment.
I've updated the blurb. It might be worth adding something to the devguide when this gets further along.
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: T. Wouters <thomas@python.org>
This reverts commit 64bf295.
…ongh-128003) This adds a new command line argument, `--parallel-threads` to the regression test runner to allow it to run individual tests in multiple threads in parallel in order to find multithreading bugs. Some tests pass when run with `--parallel-threads`, but there's still more work before the entire suite passes.
…ongh-128003) This adds a new command line argument, `--parallel-threads` to the regression test runner to allow it to run individual tests in multiple threads in parallel in order to find multithreading bugs. Some tests pass when run with `--parallel-threads`, but there's still more work before the entire suite passes.
6 participants
This adds a new command line argument,
--parallel-threadsto the regression test runner to allow it to run individual tests in multiple threads in parallel in order to find multithreading bugs.Some tests pass when run with
--parallel-threads, but there's still a bunch more work before the entire suite passes:@thread_unsafesupport.gc_collect()are sensitive to threads becausegc.collect()returns immediately without running the GC if it's already running in a different thread