mathpalette-version for fontstyle instead of fontsize

Question

The link the-mysteries-of-mathpalette gives a nice overview of the mathpalette command. My question is pseudo-related. Is there a version of mathpalette where instead of "preserving the font size" it "preserves the font style"?

For example_practicality, the underline command \underline has the property that any math written within it preserves the fontstyle (e.g. \mathit, \mathtt, \mathrm) but the underbracket command \underbracket[rule thickness][bracket height]{arg} does not have this feature. I wish to make my own custom version of underbracket which mimics the underline command in this regard. If the mathpalette-version exists (I suspect not after a quick internet search), then I can see a way to get the desired code. Any suggestion is welcome.

Answer 1

The current math family is in the TeX primitive \fam register, for which LaTeX introduces the \mathgroup wrapper. The kind of problem you describe arises every time a macro sets its math content in a box for alignment or measuring or whatever. This does not happen with \overline because it is (in math mode) a primitive, but \underbracket uses a \ialign.

For a macro with a single argument you could define a wrapper similar¹⁾ to \mathpalette, but for \underbracket this is trickier because it has two optional arguments. I just looked into the code for the default values of the optional arguments and built a wrapper around \underbracket. The basic idea is to save the current value of \mathgroup and use it later in the argument.

\documentclass{article}

\usepackage{mathtools}

\MHInternalSyntaxOn
\makeatletter
\newcommand*{\saved@mathgroup}{}% better safe than sorry
\NewDocumentCommand{\varunderbracket}{ O{\l_MT_bracketheight_fdim} O{.7\fontdimen5\textfont2} m }{%
   \edef\saved@mathgroup{\the\mathgroup}% save current \mathgroup
   \underbracket[#1][#2]{\mathgroup\saved@mathgroup\relax#3}% use saved \mathgroup
}
\makeatother
\MHInternalSyntaxOff

\begin{document}

\begin{gather*}
\mathsf{foo + ba\underbracket{r - ba}z} f\underbracket{oo \mathrm{bar} baz}\\
\mathsf{foo + ba\varunderbracket{r - ba}z} f\varunderbracket{oo \mathrm{bar} baz}
\end{gather*}

\end{document}

---

¹⁾ Well, actually completely different, but you get the idea.