When not to use \ensuremath for math macro?

Question

I have gotten into the habit of using \ensuremath around math components in every macro that I define, so that I can use the same macro inside and outside of math mode. But egreg's comments in \DeclareMathOperator won't take arguments has me thinking that perhaps I should not be doing this all the time:

It's not a good idea, in my opinion, to add \ensuremath; why should it be? Disclaimer: this is part of my \ensuremath-only-when-really-needed campaign. :) - egreg

A related comment in Force line break inside a \lim argument in align environment:

Why \ensuremath? Wouldn't $\scriptstyle#1$ be easier to read? - egreg

In this case it was required to use math mode as we were within a \makebox (i.e., text mode). So, out of habit I used \ensuremath but egreg was right that in this case using $...$ would have worked just as well and was certainly easier to read. But, even in this case I still prefer to use \ensuremath so that it is more obvious that this can be used in math and text mode without having to think about \makebox being in text mode.

So, is there really any harm in always using \ensuremath, at least for any of my personal macros? I realize that there is a slight extra overhead in processing, but is there something else to consider?

If there are other considerations when they macros are coming packages that are intended for general purpose use, I would be interested in those as well. The most obvious one is that we may want to make sure that the user of the macro knows that this macro requires math mode, so require them to only use the macro in math mode.

Since this references egreg's campaign, I felt I should include a MWE for new users as per my MWE-as-often-as-possible campaign:

\documentclass{article}

\newcommand{\FunctionF}{\ensuremath{x^2}}%

\begin{document}
Text mode: \FunctionF

Math mode: $G(x) = x^3 - \FunctionF$
\end{document}

Answer 1

Two objections come to mind:

• If you don't know you will be entering math mode, then you may write something "semantically" correct that breaks.

For example:

\documentclass{article}
\newcommand\mathmacro[1][A]{\ensuremath{{#1}_1}}
\begin{document}
 \mathmacro[$x^2$] % The dollar signs *leave* math mode
\end{document}

What's going on here is that \mathmacro[$x^2$] expands to

\ensuremath{{$x^2$}_1}

which expands to (effectively)

\ifmmode
 {$x^2$}_1
\else
 ${$x^2$}_1$
\fi

and you can see that if you write it outside of math mode, the second branch is taken, so the first dollar sign brings you into math mode and the one at the front of $x^2$ takes you out of it, with the reverse operation happening afterwards. This gives an error.

Of course, you aren't supposed to do that, since \mathmacro is actually a "math macro usable in text mode", so you should think of the thing between the brackets of its argument as being in math mode. Alas, this confuses both the author and the text editor's syntax highlighting, since it is a nonstandard assumption.

Edit: I would define this macro as:

\newcommand\mathmacro[1][A]{{#1}_1}

and use it as:

$x^2$ sub one: $\mathmacro[x^2]$.

This way, the parts that are math are clearly math.

• In the unlikely event that you or some package sets \everymath, you will be very surprised when your apparently text-mode macros start to look different.

On the subject of semantics, though, the issue is clear: \ensuremath breaks the separation between math and text, which are two very different things. TeX even has the distinction built in: different fonts, different spacing rules, different parsing rules. You can probably construct a lot more counterexamples by exploiting these.

What I mean by this is that in the following situation:

\documentclass{article}
\newcommand\mathmacro[1][A]{\ensuremath{{#1}_1}}
\begin{document}
 A sub one: \mathmacro

 \bfseries A sub one: \mathmacro
\end{document}

you may be surprised that the bold text does not extend to the contents of the apparently text-mode \mathmacro.

What I'm saying is not so much that \ensuremath actually breaks anything as that it violates your expectations to the point that it makes things harder rather than easier.

Answer 2

There's nothing bad about \ensuremath per se. However, if we examine this question of yours I see no advantage whatsoever in being able to say \FunctionF in both text mode and math mode, instead of always sticking to \(\FunctionF\) (or $\FunctionF$ as others might prefer). There are many other posts here on TeX.SX that do the same. Take this one: what's the advantage of being able to write the ring~\Z{n} instead of the ring~$\Z{n}$ when, maybe, one has also to write the ring $\Z{m}\times\Z{n}$, which is highly probable if one wants that command?

Another post has the following code:

\newcommand{\vrel}[3]{
\vcenter{\halign{\hfill##\hfill\cr
\ensuremath{#1}\cr
\rotatebox[origin=c]{270}{\ensuremath{#2}}\cr
\ensuremath{#3}\cr
}}}

This is bad programming, in my opinion, as what's inside \ensuremath is to be typeset in math mode; the "correct" code should be

\newcommand{\vrel}[3]{
\vcenter{\mathsurround=0pt \halign{\hfill##\hfill\cr
  $#1$}\cr
  \rotatebox[origin=c]{270}{$#2$}\cr
  $#3$\cr
}}}

Other cases in which \ensuremath makes things more complicated than necessary are this one and this one.

Answer 3

This comment of user egreg to the question "Why are so many symbols restricted to math mode?" is very valuable:

If it allowed this, people would write \alpha+\beta getting the space wrong. Math should always treated as math, even single symbols; consider it as markup, if you want.

Put simply, use of \ensuremath can easily teach users of the macro in question to be lazy about explicitly switching to math mode (which is often "not needed" for all symbols to appear on the page if one has no super- or subscripts), with the consequence that spacing for an expression can be wrong if the expression should follow mathmode spacing but is typeset in horizontal textmode. This defeats one purpose of (La)TeX, namely correct typesetting of mathematical expressions.

If users can type "\myalpha+\mybeta" (or "\myalpha + \mybeta", and shouldn't it be "\myalpha{} + \mybeta{}" then? etc.) where they should be typing "\(\myalpha+\mybeta\)", that might lead to bad habits, with the spacing not being as it should.

Answer 4

I think the answer is provided in fixltx2e, although for most cases I don't see it as a problem, from the documentation:

4.4.1   Notes on the implementation strategy

Pr/3400 made \@fnsymbol decide between text-mode and math-mode, which requires a certain level of robustness somewhere as the decision between text and math must be made at typesetting time and not when inside \protected@edef or similar commands. One way of dealing with this is to make sure the value seen by \@fnsymbol is a fully expanded number, which could be handled by code such as

\def\fnsymbol#1{\expandafter\@fnsymbol
  \expandafter{\the\csname c@#1\endcsname}}

This would be a good solution if everybody used the high level commands only by writing code like \fnsymbol{footnote}. Unfortunately many classes (including the standard classes) and packages use the internal forms directly as in \@fnsymbol\c@footnote so the easy solution of changing \fnsymbol would break code that had worked for the past 20 years.

Therefore the implementation here makes \@fnsymbol itself a non-robust command again and instead uses a new robust command \TextOrMath, which will take care of typesetting either the math or the text symbol. In order to do so, we face an age old problem and unsolvable problem in TeX: A reliable test for math mode that doesn’t destroy kerning. Fortunately this problem can be solved when using eTeX so if you use this as engine for your LaTeX format, as recommended by the LaTeX3 Project, you will get a fully functioning \TextOrMath command with no side effects. If you use regular TeX as engine for your LaTeX format then we have to choose between the lesser of two evils: 1) breaking ligatures and preventing kerning or 2) face the risk of choosing text-mode at the beginning of an alignment cell, which was suppodes to be math-mode. We have decided upon 1) as is costumary for regular robust commands in LaTeX.