Pure mathematician here. The sequence of events described is very unlikely to happen, because of cultural forces in pure math designed to prevent it.
If the situation the OP described did happen, it would be bad for everyone involved:
The authors would rightly feel like they were scooped, since they proved these results first, and then the referee (who had non-public access to their work) proved the same results. If he had not been the referee, he would not even have been aware that the results were true.
The editor would rightly feel betrayed. This kind of behavior is a major breach of ethics on the part of the referee. The editor would talk, and the referee would get a reputation as someone who cannot be trusted.
The referee would suffer terrible reputational harm, when the story came out.
Now on to some more specific points:
I see no reason why this would not be possible as the reviewer is not technically plagiarising anything. Firstly because there is no published work to plagiarise (because the original paper was rejected), and secondly because their paper would just contain the same theorems but with entirely unrecognizable proofs (and superior ones, at that).
The referee's behavior is a clear breach of ethics. It does not matter at all whether the paper where the referee got the ideas was published, or even whether it was a public preprint or not. The Oxford dictionary defines plagiarism as "the practice of taking someone else's work or ideas and passing them off as one's own." Clearly, the referee learned of the statements of the theorems from the preprint, took those ideas, and passed them off as his own, even if he wrote different proofs for the theorems.
In math, sometimes the main contribution of a paper is to prove a known conjecture using new techniques or cleverness. Other times, the main contribution is to ask an interesting question, even if the techniques to answer it are standard. Sometimes, an unethical person could scoop a work in progress just by getting the statement of the theorem the authors were proving.
Their paper could even acknowledge that the original researcher made them aware of the result.
This is a step in the right direction but is not sufficient. The ethical thing is for the referee to write to the authors and suggest a joint paper where the contribution of the original authors was figuring out the theorem that is true, and the contribution of the referee is a streamlined proof. If the referee had figured out the streamlined proof while refereeing the paper, he could have also given the proof to the authors as part of his referee report. Things like that do happen in math, and it's a nice, encouraging thing to do for junior authors when you find yourself refereeing their work.
If all of this is possible, how should the original researcher go about defending against it? An obvious safeguard is publishing a manuscript of the original paper to arXiv or a freely accessible personal web page, and to try and publish the paper in some other journal before the reviewer manages to publish theirs. Would this suffice?
Yes, a good first step is to put the paper on arXiv. In math, we have strong cultural barriers that prevent other researchers from scooping results that are on arXiv (though, of course, sometimes two teams will legitimately prove the same result at the same time). It is better to put the paper on arXiv than on a personal web page, because the arXiv timestamp is widely respected in the field. The authors should also reach out to the referee to make sure he knows about their previous work (because we're a single-blind field, the authors would not know that the referee had been the referee for their paper), and to coordinate how the papers cite each other. It would be fair if the referee's paper explicitly says the results were first proven by the authors (and cites their preprint on arXiv) and if the authors point out that subsequently to their work, shorter proofs were found. The authors' paper should still be published, but indeed it's good not to delay submitting it to another journal.
My concern is that the main contribution of the original paper is the novelty of the results themselves. Once the reviewer (or any other experienced researcher in the field, for that matter) was made aware of them and studied them carefully, it is indeed likely that they could come up with a superior proof, or even with an improvement on the theorems themselves. I would assume that this would render the original work unpublishable and superfluous.
When I was just starting out in math research, I often felt that way about my preprints. Here are some things to keep in mind:
A. It often seems clear to you after you've figured it out but might still not be that obvious to others. We often underestimate the novelty of our own papers because we understand them well, whereas we often struggle to understand the papers of others.
B. Experienced researchers are incredibly busy with their own research and unlikely to go around writing down shorter proofs of other peoples' results. And, the cultural factors I wrote about above tend to prevent unethical behavior, so even if someone did quickly write a shorter proof of your result, they would acknowledge you and would not prevent your work from being published.
C. When you do a bit of math, you should want others to come along and build on it, finding a shorter proof, generalizing the result, working to strengthen the conclusion, etc. That is the nature of research, at least if the research you did was interesting.
D. Even if someone did build on your paper, that would not make your paper unpublishable. It has happened to me many times that I proved something basically at the same time someone else did, with similar proofs. In all cases, my work cited the other work and theirs cited mine, and both works were published. Editors and referees understand that publications are important for your career, and that research consists of incremental contributions. Furthermore, the fact that someone else was interested enough in your work to quickly build on it looks good for your work, and suggests to the editor that it's important and likely to be highly cited.
In conclusion, try not to get anxious about potential worst-case scenarios. Try to do interesting work, and try to come up with good, clear proofs that are not "lengthy, computational and difficult to follow." When you finish a paper, submit it to arXiv and send it to a journal a week or two later (giving time for comments you might receive as people read the arXiv version). Then, while the paper is under review, go write a different paper. Soon enough, you will be well-established and won't have to worry about others scooping you.
