X Tutup
The Wayback Machine - https://web.archive.org/web/20250526021111/https://github.com/python/cpython/issues/68841
Skip to content

Mock.assert_has_calls([]) is surprising for users #68841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rbtcollins opened this issue Jul 17, 2015 · 13 comments
Open

Mock.assert_has_calls([]) is surprising for users #68841

rbtcollins opened this issue Jul 17, 2015 · 13 comments
Labels
3.8 (EOL) end of life 3.9 only security fixes 3.10 only security fixes docs Documentation in the Doc dir

Comments

@rbtcollins
Copy link
Member

BPO 24653
Nosy @rbtcollins, @merwok, @voidspace, @methane, @berkerpeksag, @serhiy-storchaka, @kushaldas, @ztane, @tirkarthi, @iritkatriel
PRs
  • bpo-24653: Fix the documentation that passing an empty list to assert_has_calls([]) does not raise an exception #9882
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2015-07-17.09:40:04.074>
    labels = ['3.8', '3.9', '3.10', 'docs']
    title = 'Mock.assert_has_calls([]) is surprising for users'
    updated_at = <Date 2020-10-09.16:50:09.847>
    user = 'https://github.com/rbtcollins'

    bugs.python.org fields:

    activity = <Date 2020-10-09.16:50:09.847>
    actor = 'iritkatriel'
    assignee = 'docs@python'
    closed = False
    closed_date = None
    closer = None
    components = ['Documentation']
    creation = <Date 2015-07-17.09:40:04.074>
    creator = 'rbcollins'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 24653
    keywords = ['patch']
    message_count = 12.0
    messages = ['246849', '246850', '246988', '247044', '330000', '342887', '343080', '343083', '343163', '343164', '343189', '378334']
    nosy_count = 11.0
    nosy_names = ['rbcollins', 'eric.araujo', 'michael.foord', 'methane', 'docs@python', 'berker.peksag', 'serhiy.storchaka', 'kushal.das', 'ztane', 'xtreak', 'iritkatriel']
    pr_nums = ['9882']
    priority = 'normal'
    resolution = None
    stage = 'resolved'
    status = 'open'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue24653'
    versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

    @rbtcollins
    Copy link
    Member Author

    From testing-cabal/mock#243

    from unittest import mock
    mmock = mock.MagicMock()
    mmock.foobar("baz")
    mmock.assert_has_calls([])  # No exception raised. Why?mmock.assert_has_calls(['x'])  # Exception raised as expected.

    Traceback (most recent call last):
      File "tt.py", line 7, in <module>
        mmock.assert_has_calls(['x'])  # Exception raised as expected.
      File "/home/robertc/work/cpython/Lib/unittest/mock.py", line 824, in assert_has_calls
        ) from cause
    AssertionError: Calls not found.
    Expected: ['x']
    Actual: [call.foobar('baz')]

    @rbtcollins
    Copy link
    Member Author

    This might go back further, haven't checked 3.3, but IIRC we're only doing fixes on 3.4 up anyhow.

    @voidspace
    Copy link
    Contributor

    assert_has_calls checks that the calls you specified were made. So if you specify an empty list it *should* pass (or be disallowed as it has no meaning).

    If you want to check that these calls and *only* those calls were made you should use something like:

        self.assertEqual(mock.mock_calls, [])

    http://www.voidspace.org.uk/python/mock/mock.html#mock.Mock.assert_has_calls

    Maybe a documentation improvement instead.

    @rbtcollins
    Copy link
    Member Author

    Ok, so as a doc bug this should still be tracked here - I'm going to reopen it to reflect that, hope thats ok.

    @rbtcollins rbtcollins added the docs Documentation in the Doc dir label Jul 21, 2015
    @rbtcollins rbtcollins reopened this Jul 21, 2015
    @rbtcollins rbtcollins changed the title Mock.assert_has_calls([]) incorrectly passes Mock.assert_has_calls([]) is surprising for users Jul 21, 2015
    @serhiy-storchaka
    Copy link
    Member

    Why this case should be explicitly mentioned? Is it an exception from some rules? Is the empty list special? Does existing documentation say that assert_has_calls([]) should raise an exception?

    @methane
    Copy link
    Member

    methane commented May 20, 2019

    I concur with Serhiy. Current document and example describe clearly
    that this API is for checking inclusion, not equality.

    Current example:

    """
    >>> mock = Mock(return_value=None)
    >>> mock(1)
    >>> mock(2)
    >>> mock(3)
    >>> mock(4)
    >>> calls = [call(2), call(3)]
    >>> mock.assert_has_calls(calls)
    >>> calls = [call(4), call(2), call(3)]
    >>> mock.assert_has_calls(calls, any_order=True)
    """

    Empty list is allowed by same rule. There are no exception.

    And when people understand this API checks inclusion, calling with
    empty list doesn't make sense at all. So I don't think it is worth enough.

    @methane methane closed this as completed May 20, 2019
    @merwok
    Copy link
    Member

    merwok commented May 21, 2019

    And when people understand this API checks inclusion, calling with
    empty list doesn't make sense at all.

    I can see a thought process being «all my other tests use thing.assert_has_calls([call0, call1]), here let’s check there are no calls with thing2.assert_has_calls([])».

    Are the correct methods advertised in the right place? (I don’t use mock so I forgot if there is a method to assert not called or if it’s assertEqual(mock calls count, 0) or some False property)

    @tirkarthi
    Copy link
    Member

    Are the correct methods advertised in the right place? (I don’t use mock so I forgot if there is a method to assert not called or if it’s assertEqual(mock calls count, 0) or some False property)

    There is assert_not_called https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_not_called

    >> from unittest.mock import Mock
    >> m = Mock()
    >> m.assert_not_called()

    @rbtcollins
    Copy link
    Member Author

    I'm reopening this because I don't agree.

    I opened the bug because we have evidence that users find the current documentation confusing. Saying that its not confusing to us doesn't fix the confusion.

    Why should we mention the special case of an empty set? Because its the only special case.

    A simple single sentence: "Note: to assert that no calls were made see assert_not_called" would probably both cover the special case and direct users to the right place for that use case.

    @rbtcollins rbtcollins reopened this May 22, 2019
    @methane
    Copy link
    Member

    methane commented May 22, 2019

    I opened the bug because we have evidence that users find the current documentation confusing. Saying that its not confusing to us doesn't fix the confusion.

    Is there evidence people get confused by the document?

    I suppose people get confused because they guessed behavior from existing assert_has_calls usages, without reading docs.
    If they guess without reading doc, adding anything in doc doesn't help them.

    The document describe well it tests inclusion, not equality. There are no confusion about it.

    @serhiy-storchaka
    Copy link
    Member

    alist.extend([]) is also a special case, but it is not explicitly documented, because it is not exceptional.

    @iritkatriel
    Copy link
    Member

    I agree that the confusion was probably due to assumptions rather than a misreading of the doc, and I would not object to closing this issue.

    However, Robert did object so I would like to suggest an adjustment that may help:

    Currently the documentation of assert_not_called is just after that of assert_has_calls. If we reverse their order, then there can be less room for confusion about how to check that nothing was called - you would read that first.

    https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_has_calls

    @iritkatriel iritkatriel added 3.8 (EOL) end of life 3.9 only security fixes 3.10 only security fixes labels Oct 9, 2020
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @Ark-kun
    Copy link

    Ark-kun commented Oct 17, 2023

    I do not find the behavior of mock.assert_has_calls([]) confusing.

    However I have an opposite problem with assert_has_calls which feels more confusing.

    assert_has_calls fails even when all required calls are there and in the same order, but when there is an extra call in between the calls.

    >>> mock = Mock(return_value=None)
    >>> mock(1)
    >>> mock(2)
    >>> mock(3)
    >>> mock(4)
    >>> calls = [call(1), call(3)]
    >>> mock.assert_has_calls(calls)
    
    E               AssertionError: Calls not found.
    E               Expected: [call(1), call(3)]
    E               Actual: [call(1), call(2), call(3), call(4)]
    

    Imagine you are changing a library which adds call(2) and suddenly the tests start failing and telling you that call(1) and call(3) calls are not found. This is very confusing.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.8 (EOL) end of life 3.9 only security fixes 3.10 only security fixes docs Documentation in the Doc dir
    Projects
    Status: No status
    Development

    No branches or pull requests

    8 participants
    X Tutup