Ethics of scraping "public" data sources to obtain email addresses

Question

I am wondering whether the following research practice is ethical.

A software engineering researcher downloads source code repositories from Github, a large source of publicly available open source code. The researcher searches the git commit logs to find email addresses of software developers who have committed to a project, and uses these email addresses to send them an email asking them to participate in a survey. If the recipient clicks on the link to the survey, the survey contains an appropriate briefing and obtains informed consent. The researcher follows all institutional and legal requirements related to human subjects research. The researcher limits the number of emails sent to only the number of participants they think they will need to test their hypotheses. However, at least one recipient of this email is annoyed that the researcher obtained their email address in this fashion and sent them unsolicited email.

Is this an ethical research practice? In particular, what would be the relevant ethical principles or ethical framework for analyzing this question? I've read a bunch of papers and backgrounders on ethics in human subject research and in engineering research, but they seem focused on other issues. Are there accepted norms or guidelines relating to this sort of situation? Has it been considered in other fields, such as the social sciences?

A possible argument that the practice is ethical: The data source is publicly available, and the email addresses were collected from this publicly available data. Developers chose to make their software repository publicly available, and they should assume that any information contained in it are public. Developers who don't want to be contacted could have configured their git client specially to use a different email address. The research will benefit our understanding of the science of software development. Subjects have an opportunity to decide whether or not to participate in the survey. Participant confidentiality will be protected, and all responses will be treated anonymously. The research complies with all legal and compliance requirements. From a legal perspective, the emails are not "spam", since the unsolicited email was not sent for a commercial purpose.

A possible argument that the practice is unethical: Software developers probably would not expect someone to scrape email addresses from the git commit logs. Their email address might be contained in a publicly available data set, but some developers might expect/consider the information private, or at least not public and free for unrestricted use. Some developers might object that it is one thing to use email addresses that are publicly listed on their Github profile page, but it is another thing to extract private email addresses that are provided as part of their git configuration, and that their understanding of social norms is that the email addresses automatically inserted into the commit logs by their git client were not intended for this purpose. Some software developers might object to having an unwanted email message in their inbox or find the practices "creepy".

---

Please note: I am not asking about IRBs, legal requirements, or compliance. I am super-familiar with those considerations. Assume that the researcher has complied faithfully with all of those requirements that are applicable in their country. I'm not asking about that aspect. In my view, researchers have an independent obligation to conduct research in an ethical manner, and to exercise their own judgement in avoiding unethical behavior, even if is legally permitted or approved by an IRB.

Answer 1

A relevant guideline from the Council of American Survey Research Organizations' Code of Standards and Ethics for Survey Research:

Research Organizations are required to verify that individuals contacted for research by email have a reasonable expectation that they will receive email contact for research. Such agreement can be assumed when ALL of the following conditions exist:

1. A substantive pre-existing relationship exists between the individuals contacted and the Research Organization, the Client supplying email addresses, or the Internet Sample Providers supplying the email addresses (the latter being so identified in the email invitation);

2. Survey email invitees have a reasonable expectation, based on the pre-existing relationship where survey email invitees have specifically opted in for Internet research with the research company or Sample Provider, or in the case of Client-supplied lists that they may be contacted for research and invitees have not opted out of email communications;

3. Survey email invitations clearly communicate the name of the sample provider, the relationship of the individual to that provider, and clearly offer the choice to be removed from future email contact.

4. The email sample list excludes all individuals who have previously requested removal from future email contact in an appropriate and timely manner.

5. Participants in the email sample were not recruited via unsolicited email invitations.

It would seem here that GitHub users do not have a reasonable expectation that they will be contacted for research.

Answer 2

Being curious about this question, I asked GitHub about such uses. Here was their response:

Thanks for reaching out! We always encourage users who have questions about our Terms to contact GitHub Support directly, so we can learn more about their specific situation.

In the meantime, I'll be happy to point you to GitHub's Terms of Service and address the general issue of sending unsolicited emails.

Section G10 of our terms states:

You must not upload, post, host, or transmit unsolicited email, SMSs, or "spam" messages.

So, it appears that they also interpret section G.10 as banning this practice, though they encourage anyone with a question about what uses of their service are allowed to contact them directly.

---

April 2022 update:

As polm23 pointed out in a comment, Github's terms have been updated since my answer was originally written. This concern is now addressed in section 7 of Github's Acceptable Use Policy, which includes the following (emphasis mine):

You may not use information from the Service (whether scraped, collected through our API, or obtained otherwise) for spamming purposes, including for the purposes of sending unsolicited emails to users or selling User Personal Information (as defined in the GitHub Privacy Statement), such as to recruiters, headhunters, and job boards.

So, sending unsolicited e-mails to users is still banned, just under a different section of their policies.

Answer 3

In my opinion, the specific case you described is ethical with the following qualification:

1. It depends on what the survey is about and what it is for.
2. It must be the case that the scraper allows for easy opting-out of any further email from them.
3. This is a bit into the ethical "gray area" and a slight changes of circumstances of the scraping and its use might put it past the line of acceptability.

When I put my name and email as a contributor in a file on some public repository (ignoring the case of the email getting in thee against my wishes), I am making myself somewhat available, to be reached for issues regarding that source file. Now, it's true that a survey about developers' habits is not something I had in mind; but if it's for an arguably-socially-beneficial cause - I don't believe it's an abuse of my putting my email there in the file.