Links:

• Deprecate os.popen() function (2020)
• test_os fails on Windows if current directory contains spaces (2017) - os.spawn*() issue
• marking os.system() as deprecated in the docs (2010)
• os.popen documentation is probably wrong (2009)

Reminder: https://docs.python.org/dev/glossary.html#term-soft-deprecated

• a soft deprecation does not imply scheduling the removal of the deprecated API.
• a soft deprecation does not issue a warning.

As a soft-deprecation with docs only, no warning raised, and no planned removal... I like this for all of the listed APIs. Including the widely used os.system(). Our goal is to always point users towards the best way to do things.

Our goal is to always point users towards the best way to do things.

Could we do that as soon as possible?

For a pattern issue, the deprecation note is only at the section's end, right?
However, the spawn* functions, for instance, have a long section (almost two pages), so I think this notice must be earlier.

We have a note suggesting subprocess use, but it doesn't mention the deprecation state.

So, what do you guys think about:

1. Move this paragraph from here and merge it with the deprecation note at the bottom. It's more detailed than the current note.
2. In this place, say something like These functions are soft deprecated. See note at the bottom. in a highlighted way, if possible.

Implemented as the change d44c550.

Could we do that as soon as possible?

I don't think that we have to be more aggressive about deprecating these functions. A soft deprecation mention at the end of the doc should be enough for now.

On Windows, bear in mind that sharing file descriptors with a child process is non-obvious if you're not using os.spawn*(). You have to get the OS handle value via mscvrt.get_osfhandle(). Make it inheritable, at least temporarily. Spawn the child process via subprocess.Popen with close_fds=False (or use the STARTUPINFO handle list). Communicate the handle value to the child (as opposed to the fd value). Then implement the child process to wrap the handle back as a new fd via C _open_osfhandle() (e.g. msvcrt.open_osfhandle()). With os.spawn*(), if the child process uses CRT file descriptors, then one just has to make the file descriptor inheritable and communicate the fd value to the child (e.g. with a command-line argument or environment variable).

On Windows, bear in mind that sharing file descriptors with a child process is non-obvious if you're not using os.spawn*().

On Windows, I prefer to pass handles to child processes :-) https://docs.python.org/dev/library/subprocess.html#subprocess.STARTUPINFO.lpAttributeList

On Windows, I prefer to pass handles to child processes :-) https://docs.python.org/dev/library/subprocess.html#subprocess.STARTUPINFO.lpAttributeList

If one's current workflow is based around file descriptors, then switching from spawn*() to subprocess.Popen entails adapting the code in both the Python script as well as every affected child script or application to use the CRT's _get_osfhandle() and _open_osfhandle() functions, such as via msvcrt.get_osfhandle() and msvcrt.open_osfhandle(). If you don't control the source code for the spawned processes, the latter is obviously impossible.

Even from Python script to a child Python script, we have nothing implemented to make this workflow obvious and simple.

• In the parent script, an fd_list list has to be constructed manually by duplicating each file descriptor via os.dup(). Then build the handle_list by calling msvcrt.get_osfhandle() on each file descriptor in fd_list, making it inheritable via os.set_handle_inheritable(), and appending it to handle_list. Next create a startup record via subprocess.STARTUPINFO(lpAttributeList={"handle_list": handle_list}) and spawn the child process with this startupinfo parameter, as well as with the handle values, and whether each should be opened in append mode, passed to the child in some way (e.g. command line, environment variable, pipe, shared memory, etc). Finally, close all of the duped file descriptors in fd_list.
• In the child Python script, at startup get the communicated list of handles as handle_list in order to build fd_list. Make each handle non-inheritable via os.set_handle_inheritable() and, depending on whether it was opened in append mode, open a file descriptor via msvcrt.open_osfhandle(h, os.O_NOINHERIT) or msvcrt.open_osfhandle(h, os.O_APPEND | os.O_NOINHERIT), and append the opened fd to fd_list.