`TypeError: 'NoneType' object is not iterable` in `_top_level_inferred` #344

harupy · 2021-08-26T03:59:19Z

Reposting #340 (comment) as an issue.

I'm observing importlib_metadata.packages_distributions() throw the following error:

  File "/opt/mlflow/mlflow/utils/requirements_utils.py", line 129, in <module>
    _MODULE_TO_PACKAGES = importlib_metadata.packages_distributions()
  File "/miniconda/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 1016, in packages_distributions
    for pkg in _top_level_declared(dist) or _top_level_inferred(dist):
  File "/miniconda/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 1026, in _top_level_inferred
    return {

https://github.com/mlflow/mlflow/runs/3428616955#step:5:7078

It looks like the line 1028 in _top_level_inferred tries to iterate on dist.files that can be None and then throws the error above:

importlib_metadata/importlib_metadata/__init__.py

Lines 1025 to 1030 in 15df009

    
           def _top_level_inferred(dist): 
        
               return { 
        
                   f.parts[0] if len(f.parts) > 1 else f.with_suffix('').name 
        
                   for f in dist.files 
        
                   if f.suffix == ".py" 
        
               }

https://docs.python.org/3/library/importlib.metadata.html#distribution-files says:

In the case where the metadata file listing files (RECORD or SOURCES.txt) is missing, files() will return None. The caller may wish to wrap calls to files() in always_iterable or otherwise guard against this condition if the target distribution is not known to have the metadata present.

The text was updated successfully, but these errors were encountered:

andersk · 2021-08-26T05:23:17Z

I happened to fix this in #342 with:

     @property
-    def files(self):
+    def files(self) -> List[PackagePath]:
…
-        return file_lines and list(starmap(make_file, csv.reader(file_lines)))
+        return list(starmap(make_file, csv.reader(file_lines))) if file_lines else []

Although if the None return is documented, maybe a different fix is needed. (Edit: done.)

Fixes python#344. Signed-off-by: Anders Kaseorg <andersk@mit.edu>

jaraco · 2021-08-26T13:13:30Z

I've yanked the 4.7.0 release from PyPI while I address the issue.

v4.8.1 #348: Restored support for EntryPoint access by item, deprecating support in the process. Users are advised to use direct member access instead of item-based access: - ep[0] -> ep.name - ep[1] -> ep.value - ep[2] -> ep.group - ep[:] -> ep.name, ep.value, ep.group v4.8.0 #337: Rewrote EntryPoint as a simple class, still immutable and still with the attributes, but without any expectation for namedtuple functionality such as _asdict. v4.7.1 #344: Fixed regression in packages_distributions when neither top-level.txt nor a files manifest is present. v4.7.0 #330: In packages_distributions, now infer top-level names from .files() when a top-level.txt (Setuptools-specific metadata) is not present. References: python/importlib_metadata#348 python/importlib_metadata#337 python/importlib_metadata#344 python/importlib_metadata#330 Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

v4.8.1 #348: Restored support for EntryPoint access by item, deprecating support in the process. Users are advised to use direct member access instead of item-based access: - ep[0] -> ep.name - ep[1] -> ep.value - ep[2] -> ep.group - ep[:] -> ep.name, ep.value, ep.group v4.8.0 #337: Rewrote EntryPoint as a simple class, still immutable and still with the attributes, but without any expectation for namedtuple functionality such as _asdict. v4.7.1 #344: Fixed regression in packages_distributions when neither top-level.txt nor a files manifest is present. v4.7.0 #330: In packages_distributions, now infer top-level names from .files() when a top-level.txt (Setuptools-specific metadata) is not present. References: python/importlib_metadata#348 python/importlib_metadata#337 python/importlib_metadata#344 python/importlib_metadata#330 (From OE-Core rev: 01eb9d4384ae78b02780cea3b8690d99484b2602) Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

v4.8.1 #348: Restored support for EntryPoint access by item, deprecating support in the process. Users are advised to use direct member access instead of item-based access: - ep[0] -> ep.name - ep[1] -> ep.value - ep[2] -> ep.group - ep[:] -> ep.name, ep.value, ep.group v4.8.0 #337: Rewrote EntryPoint as a simple class, still immutable and still with the attributes, but without any expectation for namedtuple functionality such as _asdict. v4.7.1 #344: Fixed regression in packages_distributions when neither top-level.txt nor a files manifest is present. v4.7.0 #330: In packages_distributions, now infer top-level names from .files() when a top-level.txt (Setuptools-specific metadata) is not present. References: python/importlib_metadata#348 python/importlib_metadata#337 python/importlib_metadata#344 python/importlib_metadata#330 (From OE-Core rev: 4272ca45d137b91ec368c94b3e0dbd7d56c616dd) Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

v4.8.1 #348: Restored support for EntryPoint access by item, deprecating support in the process. Users are advised to use direct member access instead of item-based access: - ep[0] -> ep.name - ep[1] -> ep.value - ep[2] -> ep.group - ep[:] -> ep.name, ep.value, ep.group v4.8.0 #337: Rewrote EntryPoint as a simple class, still immutable and still with the attributes, but without any expectation for namedtuple functionality such as _asdict. v4.7.1 #344: Fixed regression in packages_distributions when neither top-level.txt nor a files manifest is present. v4.7.0 #330: In packages_distributions, now infer top-level names from .files() when a top-level.txt (Setuptools-specific metadata) is not present. References: python/importlib_metadata#348 python/importlib_metadata#337 python/importlib_metadata#344 python/importlib_metadata#330 Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

v4.8.1 #348: Restored support for EntryPoint access by item, deprecating support in the process. Users are advised to use direct member access instead of item-based access: - ep[0] -> ep.name - ep[1] -> ep.value - ep[2] -> ep.group - ep[:] -> ep.name, ep.value, ep.group v4.8.0 #337: Rewrote EntryPoint as a simple class, still immutable and still with the attributes, but without any expectation for namedtuple functionality such as _asdict. v4.7.1 #344: Fixed regression in packages_distributions when neither top-level.txt nor a files manifest is present. v4.7.0 #330: In packages_distributions, now infer top-level names from .files() when a top-level.txt (Setuptools-specific metadata) is not present. References: python/importlib_metadata#348 python/importlib_metadata#337 python/importlib_metadata#344 python/importlib_metadata#330 Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

v4.8.1 #348: Restored support for EntryPoint access by item, deprecating support in the process. Users are advised to use direct member access instead of item-based access: - ep[0] -> ep.name - ep[1] -> ep.value - ep[2] -> ep.group - ep[:] -> ep.name, ep.value, ep.group v4.8.0 #337: Rewrote EntryPoint as a simple class, still immutable and still with the attributes, but without any expectation for namedtuple functionality such as _asdict. v4.7.1 #344: Fixed regression in packages_distributions when neither top-level.txt nor a files manifest is present. v4.7.0 #330: In packages_distributions, now infer top-level names from .files() when a top-level.txt (Setuptools-specific metadata) is not present. References: python/importlib_metadata#348 python/importlib_metadata#337 python/importlib_metadata#344 python/importlib_metadata#330 (From OE-Core rev: 21d72ace8f9486bd1b478e28d53da64087d790fa) Signed-off-by: Tim Orling <timothy.t.orling@intel.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

andersk mentioned this issue Aug 26, 2021

Complete type annotations #342

Open

andersk added a commit to andersk/importlib_metadata that referenced this issue Aug 26, 2021

_top_level_inferred: Handle dist.files being None

81cc147

Fixes python#344. Signed-off-by: Anders Kaseorg <andersk@mit.edu>

jaraco added a commit that referenced this issue Aug 26, 2021

Add test capturing failure. Ref #344.

f00d5c3

jaraco mentioned this issue Aug 26, 2021

Wrap .files call in always_iterable per recommendation #345

Merged

jaraco closed this in #345 Aug 26, 2021

Apr	MAY	Jun
	08
2021	2022	2023

python / importlib_metadata Public

`TypeError: 'NoneType' object is not iterable` in `_top_level_inferred` #344

`TypeError: 'NoneType' object is not iterable` in `_top_level_inferred` #344

harupy commented Aug 26, 2021 •

edited

andersk commented Aug 26, 2021 •

edited

jaraco commented Aug 26, 2021

python / importlib_metadata Public

TypeError: 'NoneType' object is not iterable in _top_level_inferred #344

TypeError: 'NoneType' object is not iterable in _top_level_inferred #344

Comments

harupy commented Aug 26, 2021 • edited

andersk commented Aug 26, 2021 • edited

jaraco commented Aug 26, 2021

`TypeError: 'NoneType' object is not iterable` in `_top_level_inferred` #344

`TypeError: 'NoneType' object is not iterable` in `_top_level_inferred` #344

harupy commented Aug 26, 2021 •

edited

andersk commented Aug 26, 2021 •

edited