IntFlag iteration behaviour is inconsistent between versions #93210

dignissimus · 2022-05-25T08:46:02Z

On 3.10, iterating an IntFlag enum will include multi-bit aliases but on 3.11 and above, multi-bit aliases aren't included. Since IntFlag enums shouldn't list multi-bit aliases 3.10 should be patched so its behaviour matches that of 3.11 and 3.12.

sam@samtop ~/Documents/git/cpython (git)-[v3.11.0a5~234] % ./python                                             
Python 3.11.0a4+ (tags/v3.11.0a4-30-g83d544b929:83d544b929, May 21 2022, 09:41:53) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 3
... 
>>> list(E)
[]
>>>

Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 3
... 
>>> list(E)
[<E.x: 3>]
>>>

Tested on Linux with 3.10 and compared with behaviour in 3.11 and 3.12

Copied from #93035

83d544b introduced the error when creating the enum but the underlying issue is that multi-bit flags aren't included when iterating the enum. eea8148 Introduced support for multi-bit members but they weren't listed with the enum. On 3.10 this was fixed in 9bf7c2d and remained fixed in 2a9ab75. But it was never fixed in 3.11 and above.

sam@samtop ~/Documents/git/cpython (git)-[v3.11.0a5~234] % ./python                                             
Python 3.11.0a4+ (tags/v3.11.0a4-30-g83d544b929:83d544b929, May 21 2022, 09:41:53) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 3
... 
>>> list(E)
[]
>>>

Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 3
... 
>>> list(E)
[<E.x: 3>]
>>>

The problem might be around here

cpython/Lib/enum.py

Lines 713 to 727 in b96e20c

    
           elif Flag is not None and issubclass(enum_class, Flag): 
        
               # ensure _all_bits_ is correct and there are no missing flags 
        
               single_bit_total = 0 
        
               multi_bit_total = 0 
        
               for flag in enum_class._member_map_.values(): 
        
                   flag_value = flag._value_ 
        
                   if _is_single_bit(flag_value): 
        
                       single_bit_total |= flag_value 
        
                   else: 
        
                       # multi-bit flags are considered aliases 
        
                       multi_bit_total |= flag_value 
        
               enum_class._flag_mask_ = single_bit_total 
        
               # 
        
               # set correct __iter__ 
        
               member_list = [m._value_ for m in enum_class]

I will try looking more into it later and I think an extra test should be added for this.

Originally posted by @dignissimus in #93035 (comment)
Bug report

Actually, @ethanfurman, which is (or should be?) the correct behaviour when listing IntFlag enums with multi-bit members? This is what happens on Python 3.10.4

sam@samtop ~ % python          
Python 3.10.4 (main, Mar 23 2022, 23:05:40) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 1
...     y = 3
... 
>>> list(E)
[<E.x: 1>, <E.y: 3>]
>>>

And on trunk

sam@samtop ~/Documents/git/cpython (git)-[main] % ./python
Python 3.12.0a0 (heads/main:fa2b8b75eb, May 21 2022, 10:35:37) [GCC 12.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from enum import IntFlag
>>> class E(IntFlag):
...     x = 1
...     y = 3
... 
>>> list(E)
[<E.x: 1>]

Originally posted by @dignissimus in #93035 (comment)

@ethanfurman How should the following work? (should it work?)

def test_bizarre(self):   
    class Bizarre(Flag):
        b = 3
        c = 4
        d = 6
    self.assertEqual(repr(Bizarre(7)), '<Bizarre.d|c|b: 7>')

My current patch produces the following

>>> repr(Bizarre(7))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 396, in __call__
    return cls.__new__(cls, value)
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 729, in __new__
    raise exc
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 711, in __new__
    result = cls._missing_(value)
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 857, in _missing_
    possible_member = cls._create_pseudo_member_(value)
  File "/home/sam/Documents/git/cpython/Lib/enum.py", line 872, in _create_pseudo_member_
    raise ValueError("%r is not a valid %s" % (value, cls.__qualname__))
ValueError: 7 is not a valid Bizarre

Behaviour with single-bit members is correct

>>> class Bizarre(Flag):
...     x = 1
...     y = 2
...     z = 4
... 
>>> Bizarre(7)
<Bizarre.z|y|x: 7>
>>>

Originally posted by @dignissimus in #93035 (comment)

The text was updated successfully, but these errors were encountered:

dignissimus added the type-bug label May 25, 2022

AlexWaygood added the stdlib label May 25, 2022

AlexWaygood assigned ethanfurman May 25, 2022

dignissimus mentioned this issue May 25, 2022

[3.10] Bring IntFlag enumeration behaviour in-line with that of 3.11+ (GH-93210) #93211

Open

Apr	MAY	Jun
	25
2021	2022	2023

python / cpython Public

IntFlag iteration behaviour is inconsistent between versions #93210

IntFlag iteration behaviour is inconsistent between versions #93210

dignissimus commented May 25, 2022

python / cpython Public

IntFlag iteration behaviour is inconsistent between versions #93210

IntFlag iteration behaviour is inconsistent between versions #93210

Comments

dignissimus commented May 25, 2022