improve TypeError messages for missing arguments (meta issue)

[ezio-melotti]

BPO	16543
Nosy	@terryjreedy, @mdickinson, @ezio-melotti, @merwok, @cjerdonek, @serhiy-storchaka
Files	unpacktuple.diff: Patch against 3.2.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Tasks

• TypeError message incorrect for max() with one keyword arg #60719
• subprocess.Popen() TypeError message incorrect without args argument #60724
• Add suggestion about keyword arguments to this error message: "builtins.TypeError: my_func() takes 1 positional argument but 2 were given" #66842
• C argument errors and Python arguments error are different #59406
• solidify argument/parameter terminology #60194

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-11-23.18:03:13.251>
labels = ['interpreter-core', 'type-feature', '3.9', '3.10']
title = 'improve TypeError messages for missing arguments (meta issue)'
updated_at = <Date 2020-11-15.18:56:08.110>
user = 'https://github.com/ezio-melotti'

bugs.python.org fields:

activity = <Date 2020-11-15.18:56:08.110>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2012-11-23.18:03:13.251>
creator = 'ezio.melotti'
dependencies = []
files = ['28090']
hgrepos = []
issue_num = 16543
keywords = ['patch']
message_count = 22.0
messages = ['176202', '176205', '176207', '176208', '176209', '176210', '176212', '176213', '176218', '176223', '176226', '176229', '176230', '176232', '176234', '176236', '176237', '176241', '176243', '176314', '176372', '176397']
nosy_count = 6.0
nosy_names = ['terry.reedy', 'mark.dickinson', 'ezio.melotti', 'eric.araujo', 'chris.jerdonek', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue16543'
versions = ['Python 3.9', 'Python 3.10']

[ezio-melotti]

This came up in #60719.
While using PyArg_UnpackTuple to parse the positional arguments in a function that receives both positional and keyword arguments, the error message returned when the number of arguments is incorrect is misleading, e.g.:

>>> max(foo=1)
TypeError: max expected 1 arguments, got 0

This can be fixed by adding "positional" before "arguments" in the error message. The attached patch fixes this and the pluralization of "argument(s)".

[serhiy-storchaka]

+                (min == 1 ? "" : "s"), l);

In second part of patch should be "max". Reformat the code so that min and (min == 1 ? "" : "s") will be in one line and it will be more clear.

[mdickinson]

Now that I look at uses of PyArg_UnpackTuple, I'm wondering whether this needs to be fixed at all. Apart from max and min, do you know of any other cases where this gives a misleading error message?

Almost all the uses I can find are for simple functions/methods where all arguments are positional-only. (Ex: range, pow, slice, dict.pop).

max and min *do* clearly need an error message fix.

(Apologies: I know it was me who suggested that PyArg_UnpackTuple needed fixing in the first place. But now I'm not sure that's true.)

[mdickinson]

Of course, the 'arguments' -> 'argument' fix would still be nice to have.

[ezio-melotti]

do you know of any other cases where this gives a misleading error message?

Actually my concern was about cases where the new error might be wrong/misleading, but I haven't checked yet. If there are no such cases I think we should fix it; the error message will be better in the few cases we have in the stdlib and for all the external libs that are using the function.

Another thing that I haven't checked is if there are similar functions (e.g. to parse kwargs) that needs the same treatment (bpo-16520 might be one such example).

[mdickinson]

One more thing: I'm not sure this is the right terminology. Here we're using 'positional' to refer to parameters that PEP-362 calls "POSITIONAL_ONLY", whereas the regular TypeErrors (e.g. produced for user-defined functions) use 'positional' to mean what PEP-362 calls "POSITIONAL_OR_KEYWORD". This seems like it'll only increase the current confusion between different parameter types.

[mdickinson]

If there are no such cases I think we should fix it.

Okay, but what exactly is this fixing? I can't find any examples apart from max and min where the new error messages seem any better than the old. Your first message talks about "a function that receives both positional and keyword arguments", by which I guess you mean 'both required and optional arguments'; in any case, I can't find any such function (apart from max and min) that uses PyArg_ParseTuple. So it seems to me that there's not actually any problem to be solved.

(And I think max and min probably need to be reworked to not use PyArg_ParseTuple at all.)

[ezio-melotti]

Okay, but what exactly is this fixing?

If the function parses positional args, it should say so in the error message. If the function is never used (except for min/max) or if the error is never reported, then we have a different problem.

Even if this code is not used I don't think we can remove it, since this it's a public function, and even if we are not using it someone might be using it and benefit from the improved error message.

[cjerdonek]

Here we're using 'positional' to refer to parameters that PEP-362 calls "POSITIONAL_ONLY", whereas the regular TypeErrors (e.g. produced for user-defined functions) use 'positional' to mean what PEP-362 calls "POSITIONAL_OR_KEYWORD".

Here the error message is about "positional arguments," which are different from parameters. (As you know, bpo-15990 is to document this distinction in the glossary.)

Incidentally, it looks like the documentation for several of the functions in this part of the C API should probably be updated in this regard, for example:

http://docs.python.org/dev/c-api/arg.html#PyArg_UnpackTuple
http://docs.python.org/dev/c-api/arg.html#PyArg_ParseTuple

The PyArg_UnpackTuple documentation says, for example:

"A simpler form of parameter retrieval which does not use a format string to specify the types of the arguments. Functions which use this method to retrieve their parameters..."

The two occurrences of "parameter" here should be changed to "argument."

[cjerdonek]

This can be fixed by adding "positional" before "arguments" in the error message.

Do we know for sure that the values in the args argument in a call to PyArg_UnpackTuple always correspond to the positional arguments of a call to a Python function? Can PyArg_UnpackTuple be used in other ways?

[mdickinson]

Here the error message is about "positional arguments," which are
different from parameters.

Okay. So is it also planned to change the existing error messages for user-defined functions? E.g.:

>>> def f(a, b=1):
...     pass
... 
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 1 required positional argument: 'a'

If yes, then I think all these changes need to be coordinated, and that probably involves submitting a proposal to the mailing list. If no, then I think the change proposed in this issue is not an improvement, since it makes the error messages overall inconsistent.

[ezio-melotti]

Maybe we should follow a more systematic approach and collect the different errors raised in the different situations, and then come up with error messages that are accurate and follow the terminology that we are adopting in bpo-15990.

[cjerdonek]

TypeError: f() missing 1 required positional argument: 'a'

Yes, I think this should also be changed because passing "a" as a keyword argument is okay:

>> f(a=1)

I would suggest something like--

TypeError: f() missing argument for parameter 'a'

[ezio-melotti]

I would be fine with just dropping "positional" there, and say "required argument". With no further specification it means the "default" type of argument, i.e. positional or keyword. See also the end of #60194 (comment) (msg170876).

[cjerdonek]

TypeError: f() missing 1 required positional argument: 'a'

By the way, changing this message is the subject of bpo-16520 (which should probably be retitled).

[ezio-melotti]

bpo-16520 could be assimilated by this issue, if we decide to widen its scope as I suggested in #60747 (comment) (msg176229).

[mdickinson]

+1 to msg176229

[cjerdonek]

I would keep any "global" discussion (for decision and coordination purposes) as part of a single meta-issue, but retain individual instances that need to be corrected as separate issues. I don't think we should try to fix them all as part of one patch or issue. I'm okay with the discussion happening here, or with opening a fresh issue that points to the various issues (including this one).

[ezio-melotti]

Using this as meta-issue is fine -- there are already enough issues :)
The issues can probably be fixed in a single patch too, since it should just be a matter of changing the text of a few error messages.

[cjerdonek]

TypeError: f() missing 1 required positional argument: 'a'

The error message for round(), for example, has a different form:

>>> round()
TypeError: Required argument 'number' (pos 1) not found

It would be good if these had the same form.

[ezio-melotti]

See also bpo-15201.

[merwok]

I think it’s Terry who has an enlightening distinction between arguments definition (positional-or-keyword, positional-only, keyword-only, starargs, starkwargs) and function call (positional, keyword, *-unpacked, **-unpacked). It may have been Nick or David though.

[iritkatriel]

See also

• Add suggestion about keyword arguments to this error message: "builtins.TypeError: my_func() takes 1 positional argument but 2 were given" #66842
• TypeError message incorrect for max() with one keyword arg #60719