Error while running python3-fail2ban with "PY_SSIZE_T_CLEAN macro must be defined for '#' formats"

Question

I'm trying to run the python3-fail2ban component built by yocto, but I'm encountering an error related to the PY_SSIZE_T_CLEAN macro. The error message is PY_SSIZE_T_CLEAN macro must be defined for '#' formats.

I've looked into the issue and found that in the Python 3.10 C-API documentation https://docs.python.org/3.10/c-api/arg.html#arg-parsing, it's mentioned that,

for all # variants of formats (like s#, y#, etc.), the PY_SSIZE_T_CLEAN macro must be defined before including Python.h. Additionally, it's stated that in Python 3.9 and older, the type of the length argument is Py_ssize_t if the PY_SSIZE_T_CLEAN macro is defined, or int otherwise.

• I tried to resolve this issue by defining the PY_SSIZE_T_CLEAN macro before including Python.h. But the issue was not resolved, patch snippet here,

  +#ifndef PY_SSIZE_T_CLEAN
  +#define PY_SSIZE_T_CLEAN 1
  +#endif
  #include "Python.h"

• Also included CFLAGS += "PY_SSIZE_T_CLEAN " in the python3-fail2ban component. No impact on observation

• Later, tried to add CFLAGS += "PY_SSIZE_T_CLEAN " in the python3 component directly, but it creates lots of build issues in yocto python3, seems like not recommended

I'm using Yocto with a specific version of python3-fail2ban (0.11.2) and python3(10.12), and I've verified that the necessary PY_SSIZE_T_CLEAN macro is defined before including Python.h.

Despite this, the error persists. Has anyone encountered a similar issue when running python3-fail2ban? Are there any additional steps or configurations that need to be done to properly include the PY_SSIZE_T_CLEAN macro? I'd appreciate any insights or suggestions to resolve this issue.

Thanks in advance!

Answer 1

Just for the record - as already answered at https://github.com/fail2ban/fail2ban/issues/3553#issuecomment-1673492045

Because of usage of systemd backend (which requires journal.Reader from python-systemd module) I believe it is https://github.com/systemd/python-systemd/issues/107 (seems to be already fixed and a fixed version published on pypi, but probably did not reach your distro/python installation). To check whether it is really abovementioned issue, try this... here is (successful) output with 3.11 on Debian 12:

$ fail2ban-python --version
Python 3.11.2
$ fail2ban-python -c 'from systemd import journal; print("v:", journal.__version__); j = journal.Reader(); j.add_match(_SYSTEMD_UNIT="sshd"); print("OK")'
v: 235
OK

If you see the error, try to upgrade systemd-python package, e. g. using pip (pip3), like:

pip3 install --upgrade systemd-python
#or
fail2ban-python -m pip install --upgrade systemd-python

or contact your Python installation or OS distribution maintainers.