Currently, on a Windows machine, the clipboard contents is lost when the root is closed. This, therefore requires you to keep the IDLE instance open until after the copy has been complete (particularly annoying when copying between different IDLE instances). The solution is to pipe the tkinter clipboard contents to "clip.exe" (built-in to Windows) which will preserve it after root closure.

Please write out a manual test example (steps 1, 2, ..., N) that fails now and passes with the patch.

Test example:

1. Open IDLE (shell or editor)
2. Copy some text in the IDLE window
3. Close the IDLE window (instance should end)
4. Paste into application of choice
   Without the patch, the clipboard is cleared when the instance ends so nothing is pasted. With the patch, the content remains in the clipboard as expected.

Note: This is not a problem if clipboard history is turned on or the content is pasted before closure, however the order of events above is fairly common for me.

Encoding:

In the latest commit to the pull, I have changed the encoding from UTF-8 to UTF-16 and my reasoning is as follows:

1. Most importantly, using UTF-8, characters with a unicode value >=2^8 are incorrectly copied as multiple characters. UTF-8 does support these characters but uses a different number of bytes per character (which is presumably what is causing these issues - for example, "AĀ" is encoded as "\x41\xc4\x80", which pastes as "A─Ç") UTF-16, however, correctly works for all characters supported by Tcl (see next point).

2. "Strings in Tcl are encoded using 16-bit Unicode characters" (https://www.tcl.tk/man/tcl8.2.3/TclCmd/encoding.htm). Therefore, the encoding we choose should have at least 16 bits allocated per character (meaning either UTF-16 or UTF-32).

3. Windows' "Unicode-enabled functions [...] use UTF-16 (wide character) encoding, which is [...] used for native Unicode encoding on Windows operating systems" (https://docs.microsoft.com/en-us/windows/win32/intl/unicode). For me, this was what decided the new encoding (between the two given in the previous point).

Now that I think about it, I have run into enough problems with ^V not pasting something copied with ^C that I always leave source windows open until successful. I had not noticed that there is only a problem between windows (but this may be true) or after closing IDLE or that it only occurs when copying *from* IDLE. In fact, the last might not be true if other apps have the same bug. (I will start paying more attention after this is fixed.)

eryksun, Piping to clip.exe is not working well. On the patch, I asked if you know what Windows system call it uses, but I cannot request your review there.

I can reproduce this behavior on macOS as well, so this doesn't seem to be Windows-specific.

This does not happen with other apps in general, so it is not normal behavior for apps.

Testing with a minimal tkinter app (see code below) gives similar behavior, so this appears to be an issue with tkinter and/or tcl/tk.

import tkinter
text = tkinter.Text()
text.pack()
tkinter.mainloop()

I closed the PR but IMO this issue should remain open.

I am changing the title, though, since this is not actually Windows-specific.

Serhiy: what do you know of tcl finalization?

Tal: why does your patch only expose the option that you said crashes ubuntu.

Spacekiwigames: if you have a cpython fork and local repository, you could try the patch on your linux mint.

Tal: cancel comment above. I was confused because FinalizeThread is exposed as just finalize.

On Windows, PR fixes issue whether Shell or editor is closed last.

Can we backport? The change to _tkinter, not directly exposed in tkinter, could be considered an enhancement, but is necessary to fix what I consider a real bug in tkinter, and hence in IDLE.

Pablo, would you allow the _tkinter change in .0b2?

If we cannot backport the _tkinter change, can the tcl function be accessed through root.tk.call('????')? I am guessing that 'exit' exits the process from tcl, which is probably not what we want.
https://www.tcl.tk/man/tcl8.6/TclLib/Exit.htm
suggests that when tcl is loaded as an extention, Tcl_Finalize should be called instead of Tcl_Exit to *not* exit the process.

As Tal notes on the PR, the _tkinter patch is the minimum needed to fix the problem for IDLE, not the undetermined patch to tkinter that should be done to properly expose the fix to all tkinter users.

If the tcl function were exposed with a presumably temporary private name, '_finalize_tcl', rather than a public name, 'finalize_tcl', then I presume we should be able to backport without question.

Can we backport? The change to _tkinter, not directly exposed in tkinter, could be considered an enhancement, but is necessary to fix what I consider a real bug in tkinter, and hence in IDLE.

I'm +1 for backporting the latest PR. It only adds the clearly internal and undocumented _tkinter.finalize_tcl(), which is only used by IDLE.

Pablo, would you allow the _tkinter change in .0b2?

I checked the change and I think is scoped enough and doesn't change the public API (and it solves a bug). Is fine with me, but please make sure of the following:

• At least a core dev reviews and approves the change.
• You run with the buildbots before merging.

Thanks for checking!

What if register Tcl_Finalize as an atexit callback?

What if register Tcl_Finalize as an atexit callback?

Precisely that was tried a long time ago (2002?) and decided against, as can be seen by the comment left behind at the end of _tkinter.c:

#if 0
    /* This was not a good idea; through <Destroy> bindings,
       Tcl_Finalize() may invoke Python code but at that point the
       interpreter and thread state have already been destroyed! */
    Py_AtExit(Tcl_Finalize);
#endif

That comment may be outdated. The atexit callbacks are called very early at the interpreter shutdown stage. The interpreter is still entirely intact at this point.

Wow, I'm very surprised! Simply uncommenting that works perfectly on Ubuntu 20.04 and Windows 10.

I'll make a new PR...

It was commented out in

commit 43ff868
Author: Guido van Rossum <guido@python.org>
Date: Tue Jul 14 18:02:13 1998 +0000

Temporarily get rid of the registration of Tcl_Finalize() as a
low-level Python exit handler.  This can attempt to call Python code
at a point that the interpreter and thread state have already been
destroyed, causing a Bus Error.  Given the intended use of
Py_AtExit(), I'm not convinced that it's a good idea to call it
earlier during Python's finalization sequence...  (Although this is
the only use for it in the entire distribution.)

Maybe Guido remember more details about reproducing a Bus Error. Was it an issue from the other side, when the Tcl/Tk interpreter was used after calling Tcl_Finalize()?

Simply uncommenting that works perfectly on Ubuntu 20.04 and Windows 10.

I can't remember the details, but I don't think this worked if the Tcl interpreter had been garbage collected before the Python interpreter shutdown. Therefore, Tcl_Finalize (or similar) would preferably be called at Tkapp dealloc, but we cannot do that because the user may create more interpreters later. I will try and give a reproducible example when I have time.

Tcl_Finalize can be called more than once, says the tcl doc. It also says it, or T_FThread should be called when unloading. If the tcl implementation on some system does not match the doc, then there is a bug that should be reported.

If the registration is done by tkinter, rather than IDLE, then all tkinter apps benefit.

Tcl_Finalize can be called more than once, says the tcl doc.

I take that to mean that when calling Tcl_Finalize more than once, the second and following calls will do nothing rather than crash the process, corrupt memory etc.

The Tcl docs are quite clear that Tcl is no longer usable after calling Tcl_Finalize. Therefore, E. Paine's latest comment appears to be correct in the sense that we can't call Tcl_Finalize when cleaning up a tkinter.Tk instance. The same goes for Tcl_FinalizeThread.

After thinking about this a bit more, it seems to me that using an atexit handler is definitely worth pursuing, but not for 3.10.0.

I'm in favor of merging the existing PR as an immediate fix for IDLE, and following up with a general solution for tkinter using atexit, which should also include some tests so that we can see that it works on our CI and buildbots.

I don't think this worked if the Tcl interpreter had been garbage collected

Turns out I was wrong, it appears it *does* work. The only thing we need to change is ensuring the GIL is held when calling finalise (because, as Guido's comment says, this process can call Python code).

I don't recall anything about that, sorry.