arXiv: post supplementary files using external document and xr

Question

I have two documents in LaTeX, one called main.tex and the other supplement.tex. They are both linked together by the xr package and compile without error. (They have to be compiled in the order: pdflatex on main followed by pdflatex on the supplement and then pdflatex on the main again to complete the cross-referencing. Is there a way to put this on arXiv and letting the compiler there know that pdflatex has to run a few times on each docment? Many thanks! Should one put up the .aux files also in that case?

Answer 1

You can workaround a problem. Take your file.aux, create xx.tex and upload it to arxiv along with your latex file. the xx.tex is:

\begin{filecontents}{file.aux}
The content of file.aux
\end{filecontents}

In your latex file add (on top) \usepackage{filecontents}

and before \end{document} add

\makeatletter\@input{xx.tex}\makeatother

This way your latex file will create the file.aux that you can use with xr and \externaldocument

Answer 2

I would upload the compiled supplement.pdf, and supplement.aux with main.tex so the arXiv compiler can extract all the labels used in supplement.tex (via xr) when building main.tex, that should compile fine.

However I really would suggest including the supplementary in the main PDF (e.g. combining the tex sources or appending supplement.pdf using pdfpages) as finding supplementary through downloading the source is a pain and can leave the supplement's existence poorly advertised.

Answer 3

I had a similar situation: I had give papers in total, all referencing each other! I uploaded them all to arXiv on the same day. For simplicity, let's pretend I just had two. I had the following code on my preamble.

\usepackage{xr}
%\externaldocument{Paper1}
\externaldocument{Paper2}

Here I assume we're working in Paper1 so I commented it out because you don't need to self-reference with xr. Indeed, if you do, then it will give you multiple defined labels warnings. Comment out the second line in Paper2.

Very importantly, you must make sure that labels are referenced in a unique way across all documents. Eg, don't use \ref{sec:intro}. I used \label{sec-p1:intro} and \label{sec-p2:intro}. I believe xr package has an optional arugument. I am not sure of the exact details, but if you write something like \externaldocument[p2:]{Paper2} then any label label from Paper2 will be imported as p2:label. Personally I preferred writing in the paper number manually, but either option is fine.

The following is my understanding of what happens. It might not be 100% accurate.

• If the current document is Paper1.tex, then latex creates Paper1.aux with all the label references. This file is used for the cross-references. Eg, if you type \ref{label} then it looks in Paper1.aux for the label label and, if it exists, prints out the appropriate thing.

• The xr package makes latex look not just in doc.aux but in all the referenced documents. So above it will look in both of Paper1.aux and Paper2.aux.

• Importantly, the xr package does not create Paper2.aux. We are running latex on Paper1.tex, so it only changes Paper1.???.

Now I describe how to do the arXiv submission.

• Make a new folder, arXiv_Submission_Paper1 and move your Paper1.tex file to this folder, along with anything else you need to compile---pictures, premable, etc.

• Incidentally, it is good practice to make a full copy of your submission and place it in a folder which you don't edit. This way you can see in the future exactly what you uploaded.

• Also, place Paper2.aux in this folder. You don't need Paper2.tex---you aren't going to compile that. You only need the .aux file.

• Now just upload Paper1, uploading Paper1.tex and Paper2.aux, along with anything else you may need! The xr package reads the Paper2.aux file exactly as you want it to.

• Swap the numbers 1 and 2 to do the same for Paper2.

This can easily be extended to more papers by just uploading more .aux files. If you have lots, you could create a subfolder called aux_files and use \externaldocument{aux_files/paper#}.

Alternatively, you can just merge all the other .aux files, eg Paper1.aux, Paper2.aux, Paper3.aux, Paper4.aux and Paper5.aux into aux_all.aux. You then just use \externaldocument{aux_all}. They might seem scary because of the .aux file extension. But they're just plain text files. You can edit them with Notepad, for example. Make sure to include a linebreak at the end of each file! You will get multiple defined labels warnings if you include Paper1.aux. It's up to you whether you are happy with this warning---it is only a warning and will not stop tex from running.

I can confirm that this worked on arXiv both in November 2019 (first upload) and in February 2021 (revision). The arXiv system changed significantly between these two times. arXiv used to just run latex once. This is one reason why you need to upload a .bbl file, rather than the .bib. These days arXiv will run latex many times. The above method worked, unchanged, both times.

Answer 4

Just for reference (and because I was proud of myself for finding this), I found another way taking inspiration from Overleaf's instructions to use xr. This can equally be achieved by adding the following to the head of your main tex file:

\usepackage{xr}   
% The following is to make it work on Arxiv. The trick is to include the
% build/*.aux files in the zip, they are then hidden to Arxiv which does not
% delete them.
\makeatletter
\newcommand*{\addFileDependency}[1]{ % argument=file name and extension
  \typeout{(#1)}
  \@addtofilelist{#1}
  \IfFileExists{#1}{}{\typeout{No file #1.}}
}
\makeatother

\newcommand*{\myexternaldocument}[3][SI]{%
  \externaldocument[#1]{#2/#3}%
    \addFileDependency{#3.tex}%
    \addFileDependency{#2/#3.aux}%
}
% Invoke \externaldocument with an additional parameter setting
% the directory where you will leave your .aux file (build/ here)
\myexternaldocument[SI]{build}{SI-filename}

and including the directory where you left your .aux file (here build/) when you upload files during your submission.

---

For clarity, here's a MWE: imagine you have two files, a main text (main.tex) and supplementary text (supp.tex), that you want to cross-reference. What you want to upload to arXiv is a zip with the following file structure:

├── main.tex
├── supp.tex
├── style.sty
├── build
│   ├── main.aux
│   ├── supp.aux 

You can either manually move the .aux files resulting from the compilations in a build/ directory, or, as I do, configure this directory as the output directory of your compilation (using latexmk, providing the option -outdir=build, or for pdflatex, -output-directory=build/).

Now here's the content of each .tex and .sty files:

style.sty

\usepackage{xr}
\makeatletter
\newcommand*{\addFileDependency}[1]{ % argument=file name and extension
  \typeout{(#1)}
  \@addtofilelist{#1}
  \IfFileExists{#1}{}{\typeout{No file #1.}}
}
\makeatother

\newcommand*{\myexternaldocument}[3][SI]{%
  \externaldocument[#1]{#2/#3}%
  \addFileDependency{#3.tex}%
  \addFileDependency{#2/#3.aux}%
}

main.tex

\documentclass{revtex4-2}

\usepackage{style}
\myexternaldocument[SI]{build}{supp}

\begin{document}

\section{Section}
\label{sec:mainsec}

See \ref{SIsec:sec1}

\end{document}

supp.tex

\documentclass{revtex4-2}

\renewcommand{\thesection}{S\arabic{section}}

\usepackage{style}
\myexternaldocument[M]{build}{main}

\begin{document}

\section{Section}
\label{sec:sec1}

See \ref{Msec:mainsec}

\end{document}

And in the output PDFs, you should have "See S1" appearing in the main text, and "See I" in the supplementary.

Answer 5

the xr package (and packages like it, such as xref that rely on having an external aux file present) don't work in arXiv's autotex engine. You'll need to re-work you citations between the two documents to not be cross-referenced from the same aux file.

Answer 6

I messaged both arXiv and Overleaf directly, as the previous answers still led to wiping out all hyperlinks when used with zref and \zexternaldocument*{}. arXiv suggested that I could build a PDF directly and that an exception could be issued. Overleaf replied that they weren't aware of any additional workarounds. For now, I've switched to an alternative preprint server (in my case, ChemRxiv), although may consider trying either to build my own PDF or making an arXiv-specific .tex file with both main and supplementary in a single document.

My message to arXiv:

I've come to the same problem several times over the last year or so. I understand that "packages such as xr and xref that rely on the \externaldocument command will not work in arXiv." (https://arxiv.org/help/submit_tex#bibtex). It also mentions: "Instead we require that you update your .bbl files to include the appropriate references for both documents." However, I'm not really sure how to implement this, since I'm running into issues with cross-referencing Figures/Tables/etc. rather than citations. I've tried workarounds such as described in arXiv: post supplementary files using external document and xr, but even if I get the labels to display properly, then within-document hyperlinks don't seem to work at all. A reference to Figure 1 in the main text isn't actually hyperlinked at all.

arXiv Response:

Thank you for your question. The merging of two documents in arXiv is accomplished via the pdfpages package. This behavior is detailed in the pdfpages manual (presently on page 2):

[1]http://mirrors.ctan.org/macros/latex/contrib/pdfpages/pdfpages.pdf#page=2

"Links and other interactive features of PDF documents When including pages of a PDF only the so called content stream of these pages is copied but no links. Up to now there are no TeX-engines (pdfTeX, XeTeX, ...) available that can copy links or other interactive features of a PDF document, too. Thus, all kinds of links will get lost during inclusion."

You are welcome to attempt to build your own pdf using the method that they suggest (the experimental pax package, see details at the above link), and then upload it into a replacement if having a single clickable document is important to you. An alternative method that could produce working links is to use GhostScript: gs -sDEVICE=pdfwrite -dNOPAUSE -dBATCH -dSAFER -sOutputFile=combined.pdf main.pdf supp.pdf

Please be sure to contact us with the submit-id so that we can issue an exception for your pdf.