How did Astronomers mostly(?) agree to publish arXiv preprints along with peer-reviewed Journals? Was there pushback?

Question

This excellent, thorough and well-sourced answer to Has a gravitational microlensing event ever been predicted? If so, has it been observed? includes four links to papers on adsabs.harvard.edu pages, and all of these have arXiv preprints.

In fact a large number of posts link to the arXiv preprint version of astronomical papers and I suspect most astronomy papers have arXiv preprints.

To my understanding, this allows other astronomers earlier access to and awareness of ongoing research results which may allow the field to progress faster, and also allows everyone access to the paper without penetrating a paywall. This may have implications for the folks who collect the money which is paid at the wall.

Question: How did astronomers manage generally agree to do this? Was there much pushback by astronomical journals and/or by some authors in the astronomical community? If so, how was that resolved?

note: This may also be on-topic in Academia SE, but I think that both the pools of potential answerers and of interested readers of this bit of history in the field of Astronomy here is substantial.

---

Obligatory XKCD, hat tip to @JamesK

Answer 1

This is an interesting question, and I think it can productively be split into a couple of distinct parts, separating the publisher side from the author side:

1. Was/is there any pushback from journals about posting pre-publication versions of papers on the web?

2. Why do research article authors in astronomy post papers on arxiv at a relatively high rate?

Part 1 is easier to answer, since there are fewer individual actors to consider. As far as I can tell, the answer to this is a flat "no". As @ProfRob notes in a comment, there was a long tradition of dissemination of preprints on paper before the web existed, so it likely would have been difficult for publishers to forbid that practice once electronic dissemination became a possibility. Many journal publishers are directly connected with professional societies, and even for-profit journals need to show support for wide dissemination of ideas. I don't know of any journal that publishes astronomy research that doesn't allow posting of preprints.

Contrary to what one often hears, the higher-profile journals like Science and Nature are on board with this idea as well; both of them are very explicit in stating policies that allow both pre-publication posting on sites like arxiv, and free discussion of results at meetings. (current Nature policy, current Science policy). What they don't allow is authors discussing results with the press before publication. Nature in particular seems to emphasize the importance of preprints; they published their first editorial on it in 1997, and they seem to publish articles every few years that say, in effect, "no really, it's ok."

Part 2 is harder to answer, as is any question about why a practice evolves in some particular setting that involves individual decisions by many different people to adhere to some unstated norm.

First I thought I'd check to see whether it's really true that astronomy is different from other physical sciences in its use of preprints. The article linked in the answer from @Thomas suggests that astro is on the high side (at least as of 2011) in use of arxiv. I did my own spot check with a few ADS queries. To keep it simple, I just picked a few journals. In the years 2015-2019, this ADS query shows that there were a combined 24,634 refereed articles published in The Astrophysical Journal and Astronomy & Astrophysics (to pick a major American journal and a major European journal). Of these, 21,568 have preprints available on arxiv. (You get this by adding the qualification "property:eprint_openaccess" to the query.) So that's almost 88% of papers, which seems high indeed. To compare to another sub-field, I chose the journal Physical Review B (which publishes papers on condensed matter physics, since I know there's a cond-mat section of arxiv), yielding 25,952 articles, of which 16,936 have arxiv preprints. So that's 65%, still pretty high, but a fair bit lower than astronomy.

So, why might this be? My guess is that part of the answer might lie in the creation of ADS, and in particular the ADS project to scan all of the previous astronomical research literature and make it freely available online. That made it possible to sit at your desk and do all of your bibliographic research, rather than going to the library. Once people got used to doing that, it was natural to want to do it for the newer literature, too. Of course those electronically published articles were likely available to many researchers directly from the publisher, through their home institutions, but that early digital access was pretty heterogeneous by journal and sometimes not so smooth. That might have contributed to people wanting to put papers on arxiv to make them more available.

It's certainly more complicated than this, but I do think the existence of ADS (including its indexing of arxiv) may have made a difference in earlier / more complete adoption of electronic dissemination in astronomy compared to some other fields.

A related thing that I think is interesting is that the use of arxiv in astronomy has now become so ubiquitous that some researchers consider posting on arxiv to be important not just for early visibility, but for visibility at all. By that I mean that the daily arxiv postings have become, for many, the way that they learn about new literature, supplanting the role of journal table-of-contents services. I don't know how widespread the practice is (and I'm sure there are some generational differences), but I have certainly heard people say that arxiv is the only resource they use for regularly keeping up on new papers that are published. In other words, there's a sense in which simply publishing your paper in a refereed journal is not enough to make it fully visible to the community, at least not in the short term (though people will likely find it in literature searches later).

To check this idea, I went back to the ADS astronomy searches referenced above and looked at citation numbers. The 21,568 papers with arxiv preprints have 532,493 citations total, while the 24,634 papers overall have 561,738 citations. Subtracting, that means that the 3,066 papers without arxiv preprints have 29,245 citations. Looking at average citations per paper, that means that the arxiv preprint papers have 24.6 citations per paper on average, while the non-arxiv-preprint papers have 9.5 citations per paper on average. That is such a huge difference that it actually makes me wonder if there is some confounding factor I'm not considering - I encourage others to check the numbers and/or to try other searches. But if that's true, it's a huge effect, and bears out the idea that arxiv plays an important role in the community in research visibility overall, not just in early access.

Finally, I'll note that the importance of arxiv is now so complete that you even see an open letter from the AAS journal editors to arxiv, asking them to include the Research Notes of the AAS (a new publication of short, unrefereed papers) in the arxiv. There has been an ongoing discussion of this, since arxiv moderators started (mostly) rejecting arxiv postings of RNAAS papers.

Answer 2

It is the usual policy of publishing companies (as per their terms and conditions) that you can publish preprints of the submitted manuscript on non-commercial websites, not only on arXiv but also university websites and personal websites etc. And you can keep it there after the paper has been published, although you should then provide reference to the journal publication and add a copyright notice (because normally the publishing company would then have the copyright).

Publishing your preprint on commercial websites like Researchgate and Academia would usually be in violation of their copyright agreement though.

This holds not only for astronomy papers but in other areas as well.

This has of course also the welcome benefit that you can have your paper 'published' even when it is rejected by the journal (which many manuscripts on arXiv appear to be).

So whilst there is nothing conspirational or even illegal about sharing your journal article on arXiv (as it is in line with the policies of the publishing houses) it seems to be true that Astronomy has one of the highest arXiv publishing rates (see the graphics in this arXiv article (published elsewhere as well)). One can only speculate about the reason for this. It is interesting in this context that Astronomy has also one of the shortest publication delays (as shown in the same link). Maybe these facts are related somehow. It seems the traditional rule 'Publish or Perish' holds particularly strongly in Astronomy, possibly because everybody wants to stake out their claim of being the first in case of some theoretical or observational discovery (for which a publication on arXiv should be enough).

Answer 3

Just thinking about the push back from journals (and thus is just speculation). The number of "main" astronomy journals is small, ApJ ( ran by the American astronomical society), MNRAS ( ran by the royal astronomical society), and A&A ( a pan European organisation). There are of course other journals (nature, science, Phys rev etc) but most articles i expect are published in one of these three groupings (I'm bundling the various letters and supplemental journals together).

Two of these journals are ran by astronomy societies, and the third was a merger of various European publishing efforts. The arixiv is a great way to get astronomy research out there and help the societies fulfill their mission in promoting astronomy. I expect also being ran by scientific societies means there's less pressure to generate large profits with the journals. Thus the arxiv would be seen less of an issue/competitor in "stealing" potential revenue from a journal.

Answer 4

Anecdotally, the following is from Scientific American's December 10, 2015 Astronomers Skeptical Over "Planet X" Claims:

That is, perhaps, until now. On December 8 researchers from Sweden and Mexico quietly submitted two papers to the prestigious journal Astronomy & Astrophysics, announcing their discovery of not one but two possible Planet X candidates. The quiet did not last for long. Even though neither paper has yet been accepted for peer-review and publication, the researchers uploaded both to the arXiv, a public online repository for preprint papers, where they appeared last night. Today, as claims of newfound planets in our solar system reverberate around the world in news stories and blog posts, other astronomers are reviewing the papers and reacting mostly with skepticism. The ensuing discussions between experts in public forums like Twitter and Facebook offer a rare, real-time glimpse of the sometimes messy scientific process as it unfolds.

“Normally I prefer to only upload accepted papers,” says Wouter Vlemmings, an astronomer at Chalmers University of Technology in Sweden and co-author on both studies. “This time, however, we had exhausted our ideas... With the arXiv upload we specifically wanted to reach the community that could tell us if we overlooked something, in which case we fully intend to withdraw the papers... What I personally did not count on was the impact it has had outside the astronomy community.”

I ran across this from What about the U planet? and then Wikipedia's (Object) U (TNO) which links to it.