Why do keywords have to be reserved words?

Question

In many languages, the authors are hesitant to add more keywords, because doing so would break any existing code that happens to use those keywords as identifiers. However, they could do what C is doing for bool true and false by building them in but not reserving them as keywords. So if someone has an existing variable named any of those the code will still compile.

Are there any disadvantages to this practice that avoids breaking legacy codebases?

Answer 1

Why do keywords have to be reserved words?

They don't. The designers of a language must decide, very early in the process, whether keywords will also be reserved words.

Fortran for instance has gotten along quite well for two thirds of a century without requiring that its keywords also be reserved words:

Keywords are specifically NOT reserved words as per the Fortran specifications (F03 section 2.52, F90 section 2.52, and F77 section 2.2).
— Keywords in Fortran

It's only when a language's designers decide to reserve specific keywords (or don't even think about it), that potential problems arise.

(One simple way of getting the best of both worlds is to provide a simple rule that defines all reserved words, past, present, and future. E.g. all identifiers that begin with "_" are reserved, and no other identifiers will ever be reserved.)

Answer 2

Better compiler error handling

When producing parsing errors, a compiler (or interpreter, if you're into that) which disallows keywords being used in other contexts can much more easily isolate the issue, and continue looking for other errors to report. E.g., this C/Rust/JS-hybrid pseudocode would make it very difficult for the compiler to be sure how to parse everything after the first line:

type function == Callable; // Oops! Supposed to be one '='

function function() { // function keyword is 'function'
    function function = function() || function() { function }; // recurse, or default to an anonymous function, and assign with C-style syntax to a local variable shadowing the top-level 'function' function

    function // implicit return
}

While something as ridiculous as this would never occur in practical code, it'd be much easier to recover from that small parsing issue at the start if, e.g., function was always a keyword. It's also harder to provide useful error messages when there are many different things that could have been meant by one string (e.g., both a keyword and a type, or both a keyword and an identifier, or all three as in the code above).

Answer 3

Ambiguity

Especially in beginner languages, ambiguity can be problematic. Statements like:

false = true
if false == false

can be confusing for new users of a language, as it can be hard to tell whether the false on the left is a boolean or a variable name. Misinterpretations of the statement can lead to inaccuracy (if the false on the left is interpreted as a variable name, the if statement will produce false, but if it is interpreted as the boolean, it will produce true.)

Another case of ambiguity:

import pandas as pd
pd = pd.DataFrame(stuff)
x = pd.read_csv(more stuff)

In this case, it may be hard for a beginner to determine whether read_csv() is a function of the pd DataFrame or the pd pandas library. (I tested this, by the way, and it crashed the program.)

Answer 4

Ambiguity is the primary reason, as others have explained. I would like to also add that in a language without reserved keywords, it's very easy to shadow basic constructs, like if. This code works in Bash (but not when in POSIX mode):

$ alias if=echo
$ if true
true
$ if true; then echo yes; fi
bash: syntax error near unexpected token `then'
$ unalias if
$ if true; then echo yes; fi
yes

---

Instead of the fully reserved keywords, one way to introduce keywords without breaking old code is Contextual Keywords.

Basically you introduce keywords that can only be used in places where normal identifiers cannot. For example, C# did that with newer keywords, so did Kotlin, but JavaScript did the opposite and allowed all keywords in certain places like dot-notation and object literals (eg, both x = { if: 1 } and x.if became legal).

You can also provide a "quoting mechanism" to allow reserved words (or, depending on your use case, even illegal characters, including punctuation, spaces, and even newlines) to be used as identifiers: C#, VB.NET, F#, and Rust, and Kotlin, and SQL has "delimited identifiers", and others. This general trick is called Stropping.

Answer 5

The existing answers are informative, but I feel that they might lack an overview.

Why do keywords have to be reserved words?

The right answer is that "it depends".

Summarizing the other answers, it might be because:

1. of the way the language was designed...

   a. ...making keywords non-reserved a big syntactic problem

   // New syntax in PHP to create a closure
   $x = fn() => 'hi';
   
   // Theoretically, let's create a function called 'fn'..
   function fn(){
       return 'x';
   }
   
   // ..which causes a syntax ambiguity:
   $y = [ fn() => 'bye' ];
   // Will $y be an array with one value (a closure like $x),
   // or will it be ['x' => 'bye'] ?
   

   b. ...or not a problem at all (as per the Fortran example mentioned earlier)

2. it might be also be an intentional design decision, to avoid ambiguities:

   let definitelyTrue = true;
   let true = false;
   
   if (definitelyTrue === true) {
      // this won't work as expected
   }
   
   // imagine the havoc of doing the same with a global variable:
   window.true = false;
   

3. it may also drastically simplify and speed up parsing or execution. Consider the various function-like constructs in PHP; if PHP allowed redeclaration as functions, it would need to search in the symbol table on each use. Instead, it knows there can't be such a function, so it's extremely fast.

   exit(1);              // reserved language construct; very fast
   array_merge($a, $b);  // standard PHP function; quite slower (even if built-in)
   

   (this is also why it's recommended to use operators like $x[] = $y instead of functions like array_push())

4. in some cases, it could just be a decision of the language vendor, possibly without any specific reasons or even just to follow the crowd.

5. it could also relate to standards compliance, e.g. if the language want to comply with some specific language standard or dialect

Answer 6

There's also the option, of course, of treating newly-added keywords as not reserved words while any pre-existing keywords remain reserved, like Python did for a few versions with Async and Await. While they removed that feature in later versions after a deprecation period, it could just as easily have remained forever.

Answer 7

When writing a compiler front-end, if you reserve some words as keywords it simplifies writing the BNF (or whatever notation you use) for the language.

The point of syntax notation is to turn a stream of tokens into an abstract syntax tree which actually represents what the coder intended for the program to do. It's simpler if you know in advance which ones are variables and which ones are language keywords.

Then again, you can do a first-pass over the source code to figure out which identifiers are which, then do a second pass to create the actual AST.

This sounds like a high-cost operation, but some languages don't allow declarations outside of classes, which can be buffered, allowing the dual pass to be made on each class rather than the entire codebase, so doing two passes isn't really so awful.

So it's a tradeoff. You put less effort into your front-end by reserving keywords, but you can get away with not doing so with a little additional work.

If you take the additional effort, you can add keywords without breaking older code. Whether that's worth the trouble is a judgment call.