X Tutup
The Wayback Machine - https://web.archive.org/web/20240106163433/https://github.com/python/cpython/issues/106584
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Return the correct exit code if all tests have been skipped in unittest #106584

Closed
EliseevEgor opened this issue Jul 10, 2023 · 1 comment
Closed

Return the correct exit code if all tests have been skipped in unittest #106584

EliseevEgor opened this issue Jul 10, 2023 · 1 comment
Labels
3.12 bugs and security fixes 3.13 new features, bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@EliseevEgor
Copy link
Contributor

EliseevEgor commented Jul 10, 2023 •
edited by zware

Bug report

In Python 3.12, a new exit code (5, instead of 0) was added if all tests were skipped in unittest. But in some cases it still returns 0. See code examples.

Code example 1:

import unittest


class TestSimple(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        raise unittest.SkipTest("Skip whole Case")

    def test_true(self):
        self.assertTrue(True)

    def test_false(self):
        self.assertTrue(False, msg="Is not True")

Output:

Ran 0 tests in 0.000s

NO TESTS RAN (skipped=1)

Ok, we get exit code 5 here.

Code example 2:

import unittest


class TestCase(unittest.TestCase):
    @unittest.skip("something")
    def test(self):
        self.assertTrue(False)

Output:

Ran 1 test in 0.000s

OK (skipped=1)

Here we can see output 0. Ran one test and skipped. But I think it is incorrect, because all tests were skipped.

Your environment

  • CPython versions tested on: 3.12
  • Operating system and architecture: I think it does not matter.

Linked PRs
@EliseevEgor EliseevEgor added the type-bug An unexpected behavior, bug, or error label Jul 10, 2023
@bedevere-bot bedevere-bot mentioned this issue Jul 10, 2023
Merged
EliseevEgor added a commit to EliseevEgor/cpython that referenced this issue Jul 10, 2023
@EliseevEgor
pythonGH-106584: Fix exit code for unittest in Python 3.12
6e1be71 
Increase the number of running tests only if it hasn't been skipped.
@AlexWaygood AlexWaygood added stdlib Python modules in the Lib dir 3.12 bugs and security fixes 3.13 new features, bugs and security fixes labels Jul 10, 2023
EliseevEgor added a commit to EliseevEgor/cpython that referenced this issue Jul 11, 2023
@EliseevEgor @sunmy2019
pythonGH-106584: Apply comment suggestion
43248a9 
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
EliseevEgor added a commit to EliseevEgor/cpython that referenced this issue Jul 11, 2023
@EliseevEgor @sobolevn
pythonGH-106584: Remove the version info in the NEWS
901cf01 
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
EliseevEgor added a commit to EliseevEgor/cpython that referenced this issue Jul 11, 2023
@EliseevEgor
pythonGH-106584: Remove empty line in the NEWS
1ab5cfa
EliseevEgor added a commit to EliseevEgor/cpython that referenced this issue Jul 11, 2023
@EliseevEgor
pythonGH-106584: Check buffers before seek
6313c98
ambv pushed a commit that referenced this issue Sep 22, 2023
@EliseevEgor @sunmy2019 @sobolevn
gh-106584: Fix exit code for unittest in Python 3.12 (#106588)
8fc0713 
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Sep 22, 2023
@EliseevEgor @sunmy2019
@sobolevn @miss-islington
pythongh-106584: Fix exit code for unittest in Python 3.12 (pythonGH-…
95fcc0d 
…106588)

(cherry picked from commit 8fc0713)

Co-authored-by: EliseevEgor <egor.eliseev@jetbrains.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@bedevere-app bedevere-app bot mentioned this issue Sep 22, 2023
Merged
csm10495 pushed a commit to csm10495/cpython that referenced this issue Sep 28, 2023
@EliseevEgor @sunmy2019
@sobolevn @csm10495
pythongh-106584: Fix exit code for unittest in Python 3.12 (python#10…
53fda70 
…6588)

Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
Yhg1s pushed a commit that referenced this issue Oct 2, 2023
@miss-islington @EliseevEgor
@sunmy2019 @sobolevn
[3.12] gh-106584: Fix exit code for unittest in Python 3.12 (GH-106588)…
551aa6a 
… (#109725)

gh-106584: Fix exit code for unittest in Python 3.12 (GH-106588)
(cherry picked from commit 8fc0713)

Co-authored-by: EliseevEgor <egor.eliseev@jetbrains.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
@terryjreedy terryjreedy mentioned this issue Oct 15, 2023
Closed
@hugovk
Copy link
Member

hugovk commented Nov 10, 2023

Thank you for the fix!

@hugovk hugovk closed this as completed Nov 10, 2023
@mgedmin mgedmin mentioned this issue Dec 6, 2023
Closed
@terryjreedy terryjreedy mentioned this issue Jan 2, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.12 bugs and security fixes 3.13 new features, bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Unittest issues
Status: Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hugovk @EliseevEgor @AlexWaygood
X Tutup