Python 3.13.0b3 REPL empties ~/.python_history

[hroncok]

Bug report

Bug description:

As discussed in https://discuss.python.org/t/python-3-13-0-beta-3-now-available/56847/7 and #120766 (comment)

When I updated from Python 3.13.0b2 to b3 and started the REPL for the first time, pressing arrow up showed the history.

Upon exiting and starting again, there is no more history when I press arrow up. I noticed it is also missing from my other Python REPLs. ~/.python_history is empty.

To reproduce, I did:

1. run python3.13
2. press arrow up, hit enter
3. Ctrl+D
4. run python3.13 again -- no history

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

• gh-121245: Amend d611c4c8e9 (correct import) #121255
• gh-121245: a regression test for site.register_readline() #121259
• [3.13] gh-121245: Amend d611c4c8e9 (correct import) (GH-121255) #121261
• [3.13] gh-121245: a regression test for site.register_readline() (GH-121259) #121322

[hroncok]

This is always reproducible:

$ python3.12
Python 3.12.3 (main, Apr 17 2024, 00:00:00) [GCC 13.2.1 20240316 (Red Hat 13.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 'Save this'
'Save this'
>>> ^D

$ cat ~/.python_history 
'Save this'

$ python3.13
Python 3.13.0b3 (main, Jun 28 2024, 00:00:00) [GCC 13.3.1 20240522 (Red Hat 13.3.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> [arrow up]'Save this'
'Save this'
>>> ^D

$ cat ~/.python_history 
(empty)

$ python3.12
Python 3.12.3 (main, Apr 17 2024, 00:00:00) [GCC 13.2.1 20240316 (Red Hat 13.2.1-7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> [arrow up]
(empty)

[hroncok]

@domdfcoding wrote:

... The first time Python 3.13 runs the history is there, but after exiting and running again the history has gone. Running an older Python populates the history with new entries but they're lost the next time Python 3.13 exits.

Git bisect tracked the issue to 38cfa92. The actual issue is in site.py, where it tries to import CAN_USE_PYREPL from __main__, which it's no longer located in as of that commit.

cpython/Lib/site.py

Lines 527 to 532 in c7991cc

	def write_history():
	try:
	# _pyrepl.__main__ is executed as the __main__ module
	from __main__ import CAN_USE_PYREPL
	except ImportError:
	CAN_USE_PYREPL = False

Changing this to from _pyrepl.main import CAN_USE_PYREPL or adding from .main import CAN_USE_PYREPL to the top of _pyrepl.__main__.py seems to fix it. (I couldn't make it work reliably at first but I think that was me forgetting that site.py is frozen)

[hroncok]

Moreover, I believe that even if the code in site.py "cannot use pyrepl", it should never ever write empty history.

[skirpichev]

PR is ready: #121255

[hroncok]

Suggestion: Let's merge #121255 as is but do not close this issue yet. I think there are multiple improvements possible:

• add a regression test
• prevent empty history write
• make write_history() more symmetrical wrt reading the history file (see gh-121245: Amend d611c4c8e9 (correct import) #121255 (comment))

[skirpichev]

Yep, I'll work on this in second pr.

[skirpichev]

Second PR is ready for review: #121259

[skirpichev]

See #121351. @hroncok, I think that means that simple patch from the first PR doesn't fix problem on all systems.

Probably we should use my patch 8195067 from #121259. To make it work we can set CAN_USE_REPL before loading the site module, see #121259 (comment)