Ethical implications of using scraped e-mail addresses for survey

Question

I have an interesting (in my mind) experiment in mind: it entails e-mailing hundreds or thousands of corresponding authors, asking them to voluntarily fill out a short survey about their work.

I would obtain the e-mail addresses by scraping manuscript data from Web of Science and send automated e-mails to them. I believe this is within scope of the terms of use:

For Research Projects you may use the API:

(a) to view, use, download and print such data fields for individual academic use;

(b) to perform specific research or numerical or statistical analyses on such data to produce reports in support of scientific endeavors (provided for abstracts you must have the relevant rights by law or from the copyright owner for such use); [...]

I'm not quite sure if this would be ethical though. This kind of bulk e-mail resembles SPAM and, even though the academic value of the survey would likely be tangible to the recipients, it feels like it is crossing a line.

How would this kind of research likely be perceived?

Clarifications:

• I will be approaching my organisation's research ethics contact as well, but I expect them to err on the side of caution.
• I am not concerned about the technicalities of sending such mails (e.g., I would not use the university's email servers).
• I am aware that this paradigm may introduce a selection bias; such is not what I am asking about here.
• I am not pondering just mass e-mailing thousands of people. I would carefully select my audience and send them individualized mails with individualized, specific questions.

Answer 1

I think it would be perceived as very shady and extremely irritating by all the people you are spamming. People spam legally all the time and it doesn't make anyone feel good about it that it's legal.

Don't forget that when you do human subjects research you need ethical approval, so it isn't just the terms and conditions of these websites that matter but also the opinion of your institution's ethics board.

As far as methodological issues, the problem with low response rates in a survey is not just having a small N, but also that when your response rate is low the risk of bias becomes very high.

Answer 2

Spam (noun): 1. irrelevant or unsolicited messages sent over the internet, typically to a large number of users, for the purposes of advertising, phishing, spreading malware, etc.

(source: Google & Oxford Languages)

What you would be doing is effectively advertisement of your survey to people who never gave their consent.

So yes, this is very much spamming.

And besides the ethical issues, there are a ton of serious legal issues. For one, I'm very sure Web of Science TOS explicitly forbids such a scraping for any purposes*, and especially survey/advertising purposes. For two, I'm pretty sure most legal systems, especially EU for example, would have a lot of legal issues with sending en-masse advertisements to emails whose owners never gave consent for this. EU data/privacy protection laws are especially stringent and you could find yourself in huge legal trouble.

So just don't do it.

There are accepted, morally and legally safe ways of conducting such surveys. Follow those. Spamming around like this, you will a) expose yourself to huge legal liability; b) will surely put your e-mail into every spam filter ever; c) obtain data with questionable methods, which will be a strict base of refusal for any reputable publisher; and last but nonetheless, d) piss off literally every single scientist on the world at once.

---

* Important distinction: analyzing the data is not the same as using the information in it.

Answer 3

Apart from varying ethical issues (and human-subject issues: yes, consult your IRB...), from a statistical viewpoint your sample will be biased by self-selection (or not). Also, despite their attempt to universality, "Web of Science" is not the universal repository for all references...

Answer 4

The ethical and effective way to do work like this is to work with various journals (or whatever Web of Science is, if you need to work with them specifically) directly, so that the email can come from them. It's never going to not be an unsolicited email, but if the email comes from the journal or organization, it will have a better chance of being opened, and can limit contacts to a reasonable number of times and or methods - for example, they could include a link to the survey in certain correspondence that comes already.

Answer 5

This is always a tight line to walk along and, as the majority of other answers have clearly stated, there are plenty of reasons why spam is bad. I get a lot of spam every day -- from penis enlargement pills (which actually are quite rare for me to read nowadays) to "please publish in this journal" or "speak at this conference" -- which are far more common. They don't usually get read and I never see them unless checking my spam folder.

I would like to briefly posit an example of this sort of research that I think was done well.

I was invited to fill in a survey by a group of social scientists (not my area at all!) by email. They highlighted how they were asking me to take part in a survey because I met specific criteria that they were looking for -- which they then outlined.

They also outlined the research proposal -- briefly, but effectively -- which was looking at differences in attitudes to publishing over the duration of an academic career, attitudes to rejection, and a distribution of outcomes. Some of these criteria were things like "you appear to be an early-career researcher because your first publication was within x years of the date we collected the data", and they stated briefly what areas the full survey would explore.

The email was fairly long, very detailed, and specifically mentioned that it was part of their ethics approval. I read it and, as an academic, felt like I understood the motivations for their work and therefore I contributed to it (and filled out their survey).

Note that:

• All of your concerns about self-selection and selection biases are not ameliorated by this approach. In particular, it is difficult to distinguish those who read the email and did not participate from those who did not read it.

• The authors of the previous study I took part in highlighted that they knew this was a problem, and I was invited as part of a stratified sampling approach.

• At the end of the day, emails like that are still spamming people, but I personally was not offended by the time I finished reading the specific, personal reasons why I was a useful data-point. They wrote it to their audience.

• The fact that the email itself was specifically reviewed as part of an IRB (or equivalent) gave me confidence in its authenticity and that it served a useful purpose.

Answer 6

I'm not a full-throated supporter of this idea, and I do think "spam" is a pretty accurate statement of how many recipients would receive it, but I can't help but bristle at the many other harsh, erroneous claims that other answers have made.

1. You don't need to contact institutional review boards to send people email messages. Contra Bryan Krause's answer, you don't need to obtain institutional approval to send people an email message.

2. Corresponding authors willingly take an obligation to receive queries about their work. That's the purpose of having corresponding authors. The idea that well, corresponding with these authors is automatically some kind of ethical violation seems suspect to me.
   If your survey or message obviously relates to the authors' work, it is less likely to be perceived as "spam". I think this is a tall bar for any mass survey, but I don't think the idea is obviously absurd.

3. The terms of use seem to say this is OK. You want to use the contact information in the Web of Science data for individual academic use. That's explicitly allowed, at least by the excerpt you listed in your question. So in my mind, as long as you invidually are the one who writes the survey and distributes it (n.b. this may be its own technical challenge), it seems like you are explicitly in the clear.

None of this is to say that your idea is good or worthwhile. It will be perceived as spam by many recipients. It's probably a scientifically poor way to design a survey that will not result in particularly strong data. But it is not inappropriate to send corresponding authors email messages , you don't need institutional approval to send people emails, and the terms of use you found do say that "individual academic" "use" of the data is OK.

Answer 7

At least in Canada, your work would be explicitly considered Spam: https://fightspam.gc.ca/eic/site/030.nsf/eng/home
(that's a government web page)

Note that that page includes the following:

The legal definition of spam also encompasses:
...
the harvesting of addresses (collecting and/or using email or other electronic addresses without permission)
...

Canada's anti-spam legislation includes very large fines.

Answer 8

How would this kind of research likely be perceived?

Negatively. If I were a recipient I'd be annoyed and refuse to participate - and would consider complaining about you to your advisor or university - depending on how annoying your email is.

I will be approaching my organisation's research ethics contact as well, but I expect them to err on the side of caution.

Hopefully, they don't like getting spam either, so they'll tell you not to do it.

I am not pondering just mass e-mailing thousands of people.

Don't mass-email people who have not expressly indicated they are interested in such emails.

This is not about the academic merit of your work, it's about abusing the possibility of mass-mailing. Though shalt not mass-mail except by authorization, or in life-and-death situations. Period. Just don't do it.

Answer 9

Well, in addition to the fine reasons already listed, WoS limits maximum requests per second to 5, and email addresses don't appear to be in the fields supplied.

It's an idea with issues to start, but if you really want to do it, buy an email list.