You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With python 3.12, using shutil.unpack_archive without a filter argument will raise a warning:
/usr/lib/python3.12/tarfile.py:2220: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
I interpret this that it is recommended to always be explicit about the filter argument. However, it is not supported for all formats, and trying to unpack a zip file with filter='data' will raise an error.
One of the advantages of the unpack_archive function is that it supports a variety of common formats. But with this warning, it is no longer possible to have code that unpacks both tar and zip files, and not have warnings. It makes code more complicated, as one has to have different code paths for different file formats. This is unfortunate, and I believe it beats the purpose of shutil.unpack_archive.
I believe it would be desirable to have consistent behaviour across formats, and probably at least the filter='data' (which in most cases is the safe default) should be supported by the zip format as well. My initial thought was that zip may not support many of the problematic features to begin with, but it actually does support unix permissions. Yet it appears the python zip unpacker already strips certain flags. (It would not let me unpack an executable file with an x flag.)
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
The text was updated successfully, but these errors were encountered:
I agree, this is unfortunate. Ideally, zip would take at least filter='data'. But, I wouldn't be comfortable doing that without checking that the zip defauilts actually are secure.
The ideal end state is that filter='data' is the default for unpack_archive, since if you don't know the format you probably don't want the tar-specific features. But that's a backwards-incompatible change, which... requires a warning.
For consistency, zipfile and shutil.unpack_archive() could gain support for a filter argument. However, this would require research that this PEP’s author can’t promise for Python 3.12.
Filters for zipfile would probably not help security. Zip is used primarily for cross-platform data bundles, and correspondingly, ZipFile.extract’s defaults are already similar to what a 'data' filter would do. A 'fully_trusted' filter, which would newly allow absolute paths and .. path components, might not be useful for much except a unified unpack_archive API.
Filters should be useful for use cases other than security, but those would usually need custom filter functions, and those would need API that works with both TarInfo and ZipInfo. That is definitely out of scope of this PEP.
If only this PEP is implemented and nothing changes for zipfile, the effect for callers of unpack_archive is that the default for tar files is changing from 'fully_trusted' to the more appropriate 'data'. In the interim period, Python 3.12-3.13 will emit DeprecationWarning. That’s annoying, but there are several ways to handle it: e.g. add a filter argument conditionally, set TarFile.extraction_filter globally, or ignore/suppress the warning until Python 3.14.
Also, since many calls to unpack_archive are likely to be unsafe, there’s hope that the DeprecationWarning will often turn out to be a helpful hint to review affected code.
Feature or enhancement
Proposal:
With python 3.12, using shutil.unpack_archive without a filter argument will raise a warning:
I interpret this that it is recommended to always be explicit about the filter argument. However, it is not supported for all formats, and trying to unpack a zip file with filter='data' will raise an error.
One of the advantages of the unpack_archive function is that it supports a variety of common formats. But with this warning, it is no longer possible to have code that unpacks both tar and zip files, and not have warnings. It makes code more complicated, as one has to have different code paths for different file formats. This is unfortunate, and I believe it beats the purpose of shutil.unpack_archive.
I believe it would be desirable to have consistent behaviour across formats, and probably at least the filter='data' (which in most cases is the safe default) should be supported by the zip format as well. My initial thought was that zip may not support many of the problematic features to begin with, but it actually does support unix permissions. Yet it appears the python zip unpacker already strips certain flags. (It would not let me unpack an executable file with an x flag.)
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
The text was updated successfully, but these errors were encountered: