Old theory? The homotopy type theory community claims to be "rediscovering" Frege, and, Martin-Lof found some motivation in Russell.
Russell's paradox relies upon a specific interpretation of the membership relation motivated by logicism -- namely, that it be irreflexive. Under the assumption that it is the sole primitive of a language, an elimination of the universal quantifier yielding a reflexive occurrence of membership will generate an uninstantiable class abstract ( { x | ... } ).
Instead of speaking of "mathematics", one should speak of presuppositions with respect to a paradigm. In category-theoretic foundations, membership can be reflexive. Lawvere maintains that logicism diverged from standard mathematical practice. In addition, category-theoretic set theorists hold that the logicist interpretation is not faithful to Cantor's views. That is something to think about when being told that Cantor gave us set theory.
Russell's foundation for mathematics does not have a metatheory because he actually developed the groundwork for recognizing its possibility. There is a term generation procedure called definite description. Frege used it to speak of "the extension of a concept". This seems to have been a referential use of descriptions. But, Frege's system of logic did not seem to have a classical form. In particular, he preserved the law of identity by taking the empty class as the denotation for fictions. So, names with different intensions would be set equal to one another. Russell attempted to repair this by interpreting definite descriptions attributively. This is the presupposition of the first-order paradigm.
Tarski, impressed by Russell's work, moved away from the Polish school and his mentor Lesniewski. His introduction of a metatheory effectively treats all of the singular terms of a first-order theory as intensions. This is a generalization of the attributive interpretation of definite descriptions.
It is somewhat inappropriate to compare logicist arithmetic with the arithmetic of formalists like Hilbert. As Russell pointed out, how does the formalist explain succession from first principles? Skolem the Great did nothing more than claim it to be "obvious". You might consider looking up the expression "honest toil" to understand Russell's opinion of formalism. To the extent that foundational inquiry is intended to give a clear account of the assumptions used in mathematical proofs, to say that something is simply "obvious" is little more than an insult to one's intelligence. In any case, Frege's account of arithmetic used inclusive disjunction to implement succession for the logicist paradigm.
As for the arithmetical metamathematics of the Hilbert program, if mathematics is restricted to formal axiomatics, the use of numbers external to those axioms is an application of mathematics. The credence one gives to metamathematical claims ought to be seen as an indication of what beliefs one holds concerning mathematics. There are subtle issues involved here.
Galileo, apparently, recognized that one could push reductionism to the spelling of words. He thought it to be nonsensical.
