Need add a space before character in changing category of character?

Question

At the bottom of page 307 of The TeXbook, it says

TEX always reads the token following a constant before evaluating that constant.

So,

{\catcode‘\>=2 >

is different from

{\catcode\`>=2>

It is true, as {\catcode`\>=2>hello\par outputs

But if there is

\catcode`\<=1

before that, the result is changed. Such as

{\catcode`\>=2>hello\par
\catcode`\<=1
{\catcode`\>=2>hello\par
\bye

outputs

Why does

\catcode`\<=1

affects

{\catcode`\>=2>hello\par

Answer 1

Let's see: if you try

\tt
{\catcode‘\>=2 >
\bye

you get No pages of output.

On the other hand, with

\tt
{\catcode`\>=2>
\bye

you get output, precisely “>” and

(\end occurred inside a group at level 1)

### simple group (level 1) entered at line 3 ({)
### bottom level

The explanation is easy: in the second case, the character > is tokenized before the category code assignment is finalized, hence it has category code 12. The use of \tt is to avoid getting “¿” in output.

Actually \catcode`\<=1 affects nothing and you get the same output (and the same warning) without it.

I'll just use h and remove \par that serve no purpose and just complicates tha analysis:

{\catcode`\>=2>h{\catcode`\>=2>h

You have the following token list:

{₁ |catcode| `₁₂ |>| =₁₂ 2₁₂ >₁₂ h₁₁ {₁ |catcode| `₁₂ |>| =₁₂ 2₁₂ >₂ h₁₁

Can you see the difference? The first > has catcode 12 for the reasons explained above, but the second one has category code 2, because the assignment has been performed. So the second assignment is actually performed in a group and has essentially no effect. But you also see that adding \catcode`\<=1 has no role whatsoever.

The output is “>hh” and the warning about the unclosed group is the same, because the initial {₁ isn't balanced, while the second one is, by >₂.

Side note

The codes

\catcode`\>=2
\catcode`>=2

are completely equivalent when > has its standard category code 12. Escaping the character is a safety measure that's actually only needed when the character has category code 0, 5, 9, 14 or 15. For instance, you cannot assign active category code to ^^@ (byte 0) with

\catcode`^^@=13

because normally ^^@ has category code 9 (ignored). To the contrary

\catcode`\^^@=13

would do. In the \obeylines macro you'll find

\catcode`\^^M\active

for the same reason.

Answer 2

{\catcode`\>=2>hello\par
\catcode`\<=1
{\catcode`\>=2>hello\par
\bye

\catcode<=1` is not relevant here. MWE:

{\catcode`\>=2>hello\par
{\catcode`\>=2>hello\par
\bye

When TeX reads the first line, > has its standard meaning, which is to produce a punctuation symbol. So because TeX reads the > before evaluating the 2 and changing the meaning of >, the punctuation is produced in the output stream.

TeX then evaluates the 2 and makes the category code change. That is, it changes the meaning of >. Then it continues with the line.

When TeX reads the next line, > no longer has that meaning. It now has category code 2, so it is an end-of-group marker, like }. So when TeX reads it before evaluating the 2 this time, it reads it as the end of the group rather than an instruction to typeset something.

TeX then evaluates the 2 and makes the category change (which doesn't actually change anything). Then it continues with the line.

The second line is thus equivalent to

{\catcode`\>=2}hello\par

> produces ¿ by design, just as Q produces Q. That's just how you input that punctuation mark. In LaTeX, you get the same symbol with the default font encoding, which is no surprise as it is just the default TeX encoding (OT1). If you're using a later encoding, such as T1, you can produce the same symbol with \textquestionmarkdown.

That is, the reason > produces that particular symbol is just the same reason \textquestionmarkdown produces it: that's how TeX has been told to expand those tokens (> in the one case and a macro in the other).

To see this, try

>hello\par
{\catcode`\>=2>hello\par
{\catcode`\>=2>hello\par
\bye

It might be easier to rewrite Knuth's example to use something more familiar. Consider

{\catcode`\Q=2Qhello\par
{\catcode`\Q=2Qhello\par
\bye