gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception by ueshin · Pull Request #136487 · python/cpython

ueshin · 2025-07-09T21:39:37Z

Fixes unittest to return exit code 5 when setUpClass raises an exception.

import unittest

class BugTest(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        raise ValueError("Simulated setup failure")

    def test_example(self):
        self.assertTrue(True)

before

$ ./python.exe -m unittest bug_test.py
E
======================================================================
ERROR: setUpClass (bug_test.BugTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/.../cpython/bug_test.py", line 6, in setUpClass
    raise ValueError("Simulated setup failure")
ValueError: Simulated setup failure

----------------------------------------------------------------------
Ran 0 tests in 0.000s

FAILED (errors=1)
$ echo $?
5

after

$ ./python.exe -m unittest bug_test.py
E
======================================================================
ERROR: setUpClass (bug_test.BugTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/.../cpython/bug_test.py", line 6, in setUpClass
    raise ValueError("Simulated setup failure")
ValueError: Simulated setup failure

----------------------------------------------------------------------
Ran 0 tests in 0.000s

FAILED (errors=1)
$ echo $?
1

Issue: unittest may incorrectly return exit code 5 when setUpClass raises an exception #136442

python-cla-bot · 2025-07-09T21:39:42Z

All commit authors signed the Contributor License Agreement.

bedevere-app · 2025-07-09T21:39:45Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

gaogaotiantian

Overall I think it's good, just a few minor suggestions.

gaogaotiantian · 2025-11-14T23:16:54Z

Lib/test/test_unittest/test_program.py

+        self.assertEqual(cm.exception.code, 1)
+        out = stream.getvalue()
+        self.assertIn('\nERROR: setUpClass (test.test_unittest.test_program.'
+                      'Test_TestProgram.SetUpClassFailure)\n', out)


I think the error message you are trying to catch here is a bit too specific. Part of the strings is related to how the testing infra is designed, we should not test those. I think the actual test case related info is SetUpClassFailure, setUpClass and probably ERROR, let's just write two assertIn for those info.

gaogaotiantian · 2025-11-14T23:19:24Z

Lib/test/test_unittest/test_setups.py

        self.assertEqual(result.testsRun, 2)
        self.assertEqual(len(result.errors), 0)

+    def test_setup_class_raising_error(self):


Is this test case related to this issue? Seems like you are trying to test that an exception in setUpClass will stop the test? If it's not directly related, I think it's better to have the commit to include only related tests, so it's easier for us to follow the commit history.

Misc/NEWS.d/next/Tests/2025-07-09-21-45-51.gh-issue-136442.jlbklP.rst

bedevere-app · 2025-11-14T23:22:23Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>

ueshin · 2025-11-14T23:55:06Z

I have made the requested changes; please review again.

bedevere-app · 2025-11-14T23:55:11Z

Thanks for making the requested changes!

@gaogaotiantian: please review the changes made to this pull request.

gaogaotiantian · 2025-11-15T00:18:25Z

Lib/test/test_unittest/test_program.py

+                testLoader=self.TestLoader(self.SetUpClassFailure))
+        self.assertEqual(cm.exception.code, 1)
+        out = stream.getvalue()
+        self.assertIn("ERROR", out)


Let's check "ERROR: setUpClass" and "SetUpClassFailure" here. Otherwise I think it's good!

Updated. Thanks!

miss-islington-app · 2025-11-15T00:59:55Z

Thanks @ueshin for the PR, and @gaogaotiantian for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

…aises an exception (pythonGH-136487) (cherry picked from commit 53d65c8) Co-authored-by: Takuya UESHIN <ueshin@happy-camper.st>

bedevere-app · 2025-11-15T01:00:01Z

GH-141576 is a backport of this pull request to the 3.14 branch.

…aises an exception (pythonGH-136487) (cherry picked from commit 53d65c8) Co-authored-by: Takuya UESHIN <ueshin@happy-camper.st>

bedevere-app · 2025-11-15T01:00:07Z

GH-141577 is a backport of this pull request to the 3.13 branch.

…raises an exception (GH-136487) (#141577) gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception (GH-136487) (cherry picked from commit 53d65c8) Co-authored-by: Takuya UESHIN <ueshin@happy-camper.st>

…raises an exception (GH-136487) (#141576) gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception (GH-136487) (cherry picked from commit 53d65c8) Co-authored-by: Takuya UESHIN <ueshin@happy-camper.st>

…aises an exception (python#136487)

Fix unittest to return exit code 5 when setUpClass raises an exception.

854c0f0

bedevere-app bot added the awaiting review label Jul 9, 2025

bedevere-app bot mentioned this pull request Jul 9, 2025

unittest may incorrectly return exit code 5 when setUpClass raises an exception #136442

Closed

blurb-it bot and others added 3 commits July 9, 2025 21:45

📜🤖 Added by blurb_it.

950f7a0

Merge branch 'main' into pythongh-136442

57c610f

Fix.

7fb4075

gaogaotiantian requested changes Nov 14, 2025

View reviewed changes

bedevere-app bot added awaiting changes and removed awaiting review labels Nov 14, 2025

ueshin and others added 3 commits November 14, 2025 15:45

Apply suggestion from @gaogaotiantian

290e999

Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>

Merge remote-tracking branch 'upstream/main' into pythongh-136442

802ebb2

Fix.

e3ca9ab

bedevere-app bot added awaiting change review and removed awaiting changes labels Nov 14, 2025

bedevere-app bot requested a review from gaogaotiantian November 14, 2025 23:55

gaogaotiantian reviewed Nov 15, 2025

View reviewed changes

gaogaotiantian added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Nov 15, 2025

Fix.

e0ced4e

gaogaotiantian approved these changes Nov 15, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting change review labels Nov 15, 2025

gaogaotiantian merged commit 53d65c8 into python:main Nov 15, 2025
49 of 50 checks passed

bedevere-app bot removed the awaiting merge label Nov 15, 2025

bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Nov 15, 2025

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Nov 15, 2025

StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025

pythongh-136442: Fix unittest to return exit code 5 when setUpClass r…

8e1f040

…aises an exception (python#136487)

Uh oh!

Conversation

ueshin commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

python-cla-bot bot commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app bot commented Jul 9, 2025

Uh oh!

gaogaotiantian left a comment

Choose a reason for hiding this comment

Uh oh!

gaogaotiantian Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

gaogaotiantian Nov 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bedevere-app bot commented Nov 14, 2025

Uh oh!

ueshin commented Nov 14, 2025

Uh oh!

bedevere-app bot commented Nov 14, 2025

Uh oh!

gaogaotiantian Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

ueshin Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Nov 15, 2025

Uh oh!

bedevere-app bot commented Nov 15, 2025

Uh oh!

bedevere-app bot commented Nov 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ueshin commented Jul 9, 2025 •

edited

Loading

python-cla-bot bot commented Jul 9, 2025 •

edited

Loading