Ethical concerns regarding the review of a paper on autonomous armed drones

Question

I would like your suggestion on a delicate matter involving a paper that I am reviewing. I will avoid giving too many details to preserve anonymity.

In this article the authors describe an algorithm for the (unmanned) control of a military armed drone. The work has been done completely in a simulator, but the authors show clear pictures of the type of real-world military drone they reproduced, explicitly saying that the real drone can be armed with different type of bombs. The performance of the algorithm is scored based on the number of times the simulated drone autonomously hit the target. The authors never discuss possible ethical concerns of this technology, neither they explicitly say if this technology will be used on real drones to hit human targets with possible involvement of civilians (e.g. in counter-terrorism).

My personal position is against the development of lethal autonomous weapons. Even though I have several years of experience in reviewing papers, this is the very first time I have to deal with such a problem and I am not sure how to move. I think that this kind of ethical problems are common in medical-related fields, but rather uncommon in engineering, and that they are catching our community off guard.

Note that, from a formal point of view the situation is fuzzy. Even though there has been an international petition against the development of autonomous weapons, the formalization in an international ban (like the one for chemical weapons) seems to be far away. Moreover, the ethical guidelines of the journal just apply to the use of animal in research, and to experiments on human subjects. This let me conclude that from a formal standpoint the paper is legit since it does not break any rule.

I have considered writing to the editor expressing my concerns or pointing out these concerns directly in the review, so that the authors can give an answer to my queries. As last solution I am also considering the option of withdrawing, but I would prefer to set up a dialogue with the authors and the editor since this is more constructive for everybody. However, I think it is also important to consider the hypothesis that my ethical position can bias my review (even though I am trying to be as neutral as possible) and this may be unfair for the evaluation of the technical quality of the proposed method.

My question is: how should I proceed in the review process? There are a few options: withdraw for ethical concerns, just review the article, review and express my ethical concerns to the editor (and maybe to the authors).

---

Update: 5 March 2020

Thank you everyone for your answers, they really helped me to better understand the situation. I decided to withdraw as reviewer, and I sent an official letter to the editor. In the letter I expressed my ethical concerns without giving any comment on the technical quality of the paper. The paper did not break any formal rule, therefore recommending an official rejection would have been like imposing my personal ethics over the review process. In the letter I have also reported those sentences that clearly framed the drone like an autonomous lethal weapon.

The assistant editor wrote me saying that the editor liked my academic attitude and wanted me for a second review. The editorial team has contacted the authors, who said they did not wanted to create a misunderstanding and that they are willing to modify the expressions. This seems to imply that they have forwarded to the authors the list of sentences I confidentially reported to the editor, and that the authors want to adjust those sentences in the revised version.

My personal opinion is that just rephrasing the paper without changing the substance would not be enough. You can substitute "target" with "goal" and "attack" with "reach" but that remains an autonomous armed drone.

Answer 1

I suggest you withdraw and send a formal letter to the editor expressing your concerns. Cite a larger moral universe than the narrow one covered by the journal's policies. Suggest that the research itself is unethical and recommend, without detailed review, that the paper be rejected.

I've been in a similar but less fraught case. I was shepherd of a paper that crossed a less dangerous boundary. I had repeated contact with the authors and tried, over several communications, to convince them that the actions they were proposing were wrong. I failed to convince them. The paper was rejected. I was actually given an award for my efforts to convince them, even though I failed.

My experience suggests that the people who do this sort of thing are not going to be convinced by a dialog of any kind. They are too invested in it to bother to listen.

Don't wait for "the powers that be" to decide this stuff is immoral and leads to a bad end. If people like reviewers at the bottom of the pyramid don't complain about it, this sort of research will continue and will get implemented. There are too many historical exemplars of this to ignore the consequences.

Answer 2

The OP is undoubtedly aware that this is a difficult question with some grey areas. It may also be difficult to provide an single objective answer, so I will first highlight some courses of action and then make a personal suggestion.

(1) Reject on grounds of the work being unethical- a reviewer has the moral right to do this. Of course, if there's no supporting legislation or precedent, this can become quite subjective, and the reviewer should be prepared for rebuttal.

(2) Reject on academic grounds- the approach would be to point out that the authors have not done a rigourous job since they haven't dwelt on the possible consequences of such technology, and this is an important part of technology development.

(3) Review the paper on purely academic merits, but in the review, convey explicitly to the editor that this work may have unintended negative consequences, and your review does not consider these.

(4) Scrutinise the journal and see if it publishes articles that are (in the reviewer's opinion) problematic. If so, withdraw from the review AND raise awareness about this journal publicly through media, academic networks and so on. (Write to the journal as well, since that is an expected courtsey.)Without the second action, the first is sterile and almost evading responsibility.

Yes, none of the options are easy, and they involve either inviting backlash or carrying guilt. One must take a decision based on one's fortitude and appetite for criticism. I personally cannot recommend taking the softer way out, so my recommendation would be either (3) or (4) depending on how strongly one feels about the issue.

EDIT:

Following a suggestion in the comments, it may be worth differentiating the background of (3) and (4).

In (3), where one conveys one's apprehensions through a private communication to the editor, the moral burden is placed squarely on the editor/journal. The reviewer has done a purely academic duty and is satisfied with communicating apprehensions privately. There could be some good reasons to adopt this stance- trusting the ethical/moral core of the editor/journal, believing that this grey area should be dealt with within professional confines and not publicly.

In (4), one does not absolve oneself on any moral responsibility, and chooses to tackle ethical concerns personally. This means accepting responsibility not only for backlash, but also for the successful dissemination of the message. It is inherently the more difficult, but more responsible and possibly satisfying approach.

Answer 3

If it really troubles you, withdraw from reviewing the paper and tell the editor about the ethical concerns.

If you reject the paper for anything other than academic reasons, you're implicitly imposing your ethics on the authors. Ethical statements that make perfect sense to you might not make sense at all to someone else; the very fact that the authors did the research indicates they don't find autonomous armed drones ethically abhorrent. You could claim that your ethics are "better" than theirs, but that is not certain, and a reasonable person could take the reverse position. If you want nothing to do with the paper for ethical reasons, decline to review it, and let someone who is less troubled by the ethics review it.

That said, you can point out the issue to the journal and make sure they are OK with the ethics. It's possible the journal will agree with you that autonomous armed drones are unethical and will reject the paper without review. If they publish anyway, you can distance yourself from the journal by not reviewing for them and/or publishing with them in the future.

Answer 4

You may be overthinking this. A reviewer does not accept or reject a paper. A reviewer recommends the editor to accept or reject a paper.

If you believe the journal should not want to be involved with weapons research, recommend rejection based on moral grounds. If the editor thinks moral grounds should not be considered, or believes weapons research is ethical, they may ignore your recommendation.

The difficult decision here lies with the editors¹, not with the reviewers.

---

^{¹Coming up with more extreme examples of research that has not directly harmed anyone in the methodology but still has strongly unethical implications is left an an exercise to the reader.}

Answer 5

TL, DR: You should just ignore your moral questionings for the purpose of reviewing this paper.

For disclosure: I work in a defense related field.

In summary:

1. The questioning you may be proposing is probably much more complex and involves more hard-to-draw-lines than you may be thinking.

2. Any political stance that may be held by the journal would be highly questionable, and also, not up to you to decide on them.

3. It is out of scope to have an ethical discussion on every such technical paper. Even though codes of ethic should be known and adhered to by all professionals in every field.

4. As far as established ethical guidelines on this kind of publication exist, none have been breached according to the present question (as you've pointed yourself).

5. You as a person and human being, have the option to refuse reviewing the paper. But don't expect or bully others into doing the same.

In detail/ranting mode:

Despite this being an engineering paper, you kinda should think about the framework of science in general. "Science" and "ethics" are different things, which intersect within "actions". The paper does not claim to kill people for the purpose of testing, it only talks about simulations, hence, due to the lack of actions, the science part is distinct from the ethics part here, so in my book there is absolutely no ethical concern whatsoever that a reviewer should care about while reviewing the paper (other than fake simulated results, dishonest reports and other stuff applicable even to pure mathematics).

I see a lot of technical challenges within the field of autonomous weapons/drones/cars and so on, and those are fields in which people do active work and develop their careers on, for the good and for the bad.

If you are against the development of autonomous weapons, this is a political stance of yours that (IMHO) should not be endorsed by any scientific journal as an entity. You could check if the journal you that requested your review does have some related policy, but I doubt this is the case. Hence, it is not part of your review task to make political assertions about the papers you review.

Where exactly do you draw the line? Autonomous killing weapons are off to you, but what about face recognition AI? There are also concerns about unregulated development of AI applications, and even proposed temporary bans on usage of this technology. Do you expect every AI journal cease publishing until further notice? Maybe you'd like all weapon performance related publications to cease as well because weapons in general are bad? I get thousands of results from searching "missile efficiency" on google scholar, all in the first pages talking about engineering with no ethical discussion. Should all those papers have been rejected as well?

Would it ease your conscience if the "theme" of the paper was a bit more disguised? Maybe you'd have nothing against a purely mathematical essay on Monte-Carlo methods, but guess what: They trace back to the project of the atomic bomb. Back when I was an undergrad student, I've worked in a project where a simulated drone would take pictures of a target. Oddly, the camera would always break after the first picture was taken. Took me a few minutes to notice it was meant the simulate a bomb. Though it was the only time I've seen this topic be treated with no unaware people being a bit shaken, apparently it was an empathy move on the part of the teacher.

Bluntly, if my (technical/theoretical only) paper was denied due to a reviewer thinking that "this technology should be banned", I'd see this as no different that him/her saying "my religion disagrees with what you are saying". Which did happen in the past, stories tell that Napoleon once questioned Laplace about not dedicating any word to God on one of his treatises. Of course this gets blurred in humanities fields where the contents of a poor publication may translate into explicitly forbidden things like hate speech, fighting words, and so on. There is also a problem in medical/biological sciences where any decent publication needs empirical results (i.e. while I can simulate a drone strike, I can't simulate a poison acting on the human body). But this questioning is not about the scientific merit of the paper.

Also, don't ask the authors to include an ethical discussion on their paper. It's just out of scope, and frankly that's a humanities field of science, not an engineering field. At one hand, I'm completely equipped to write and read the scientific literature about weapon engineering (in the fields I work with), and I did have to take a course on ethics during undergrad school, I am aware and adhere to the codes of ethics applicable to my profession, but I'm a complete noob when it comes to research level ethical/political and legislative sciences. I cannot and should not be demanded to jointly review and discuss the latest literature on the ethics of autonomous weapons and their technical inner workings every time I write about algorithms for weapons. I just do the latter. An exception could be given to adding some standard boilerplate text like "Disclaimer: This paper does not endorse murder". But only only lawyers care about boilerplate text and I'm guessing you are not a lawyer.

Look, I work in a defense related field and know people who work with some products that may eventually (deliberately) kill people. Really, the ethical discussion gets old, you may spend a few days deciding if you are okay with it but once you're done, then discussion is over, just move on and either do the job or refuse it, revisit your values once you feel like it, but not every day (or on every paper for the current example) and don't go around questioning your colleagues about this every chance you get. I'm not minimizing it, companies that develop such products should and generally do have psychologists to support folks who may grow uncomfortable with this situation. And they do ask if the potential employee is okay with working on this kind of product, because this should be given some thought. But these questionings reach conclusions, and insisting on the discussion is just bullying others into adhering to your world view.

While some people have objections about working with weapons (particularly consumer available guns), and do "vote with their feet" by refusing to work in this field, the practicalities of life impose themselves into there existing a very thriving "defense" industry, armed police officers, armed forces and so on. There is just much more votes in favor of weapons existing and being developed than otherwise, like it or not, agree with it or not. And all the more demand for more weapons production and development.

Answer 6

tl;dr: Show backbone and reject the paper.

@Applied academic presented four courses of actions in their answer; I'm referring to those and you could go read that first (but you don't have to).

My recommendation is:

• Start with (4): Check whether the journal is a problematic venue. You might have made a mistake if you've associated yourself with a journal which occasionally legitimizes assassinations and militarized lethal policing using drones, or other similar pursuits.
• If it is a problematic journal, just follow through on (4).
• If the journal rarely, or never, publishes such work: Do (1). That is, reject the paper (or rather, recommend it be rejected) on ethical grounds. The fact that there are ethical guidelines and accepted bans means you have enough ground to stand on in claiming that such research should not be condoned legitimized and disseminated via the journal. Try to make your argument both on general principle, and - if you have credible information about the specific research you're reviewing - examples of abuse of drone technology by the authors, their institute, their funders or their government. Unfortunately, such examples abound in our world today.

I'm against option (2), because complaining about lack of an "ethics section" suggests implicitly that the paper overall is close to being legitimate. I'm also against option (3), because that means you've "passed the buck" - you've done your part in getting that paper published, and you're now hoping the editor has more backbone than you have. It's doubtful they will, after seeing you've gone along with it.

Note: @DanRomik emphasizes in a comment that a reviewer does not actually reject or accept a paper, but rather recommends acceptance or rejection. That's well worth remembering even if we (and this answer) use the terms "reject" or "accept".

Answer 7

You have a number of viable choices, obviously. My personal bent is that if you choose to recommend turning the paper down based upon the ethics of weaponized drones that you do it, state that opinion in the review, and get off the stage, refraining from reviewing on scientific matters.

This way your objection is noted, you haven't made yourself part of the bad ethical situation, and you give the editor opportunity to follow the recommendation or seek further review.

The other extreme is to recommend turning the paper down, not saying it's because of ethical reasons, and searching for scientific justification for your recommendation. I would personally consider this scientific sabotage, and thus recommend against it.

The middle ground is to state your conviction, but offer a fair scientific review. I believe this is hard to do, and it brings your review into question. I would refrain from doing this.

Personally, if I had this particular moral conviction, I would simply contact the editor and decline to review the paper, stating why.

Defense industries have driven a whole bunch of technology. There are those who further this effort. I can understand refusing to be a part of it, but draw my personal line where it comes to sabotaging the field with disingenuous reviews. There are ways to have your convictions come through well short of this-- turn down the review, write a letter to the editor for publication, work on the leadership of professional organizations to end the policy of publishing such stuff, recommend to your librarian to discontinue subscriptions, etc.

Answer 8

Summary: ethical concerns come in degrees.

1. Some are sufficiently standardized in the academic (or even general public) world-wide that violating these standards means the paper should be rejected on ethical grounds.

2. Other ethical concerns are more personal in the sense that a reviewer may hold them personally, but they are not so widely agreed upon/standardized the authors should be held responsible to them.
   OP should IMHO tolerate that others may have a different point of view for these concerns (2.).

   Still, general personal ethical concerns (2.) often include ethical concerns that are widely accepted (1.) but on a narrower scope. These again are on topic for reviewing the paper.

---

I'd suggest to analyse the (un)ethical points a bit further.

I've never had to do with arms research, but in my field animal experiments are relevant, so I'll use them as an example for which I've considered my personal ethics, and for which I have also encountered other people's ethical considerations.

Here's my point of view:

• Is the unethical point something that is "universally"* agreed upon as unethical by academia or even the general public? In other words, is there a current mainstream consensus that says it's unethical, and possibly even standardized formal ethical requirements that are violated? If so, reject on ethical grounds.

  Examples would be plagiarism, presenting falsified results or observations or a study design that is considered unethical by the rules ethics committees work on (if there's relevant local variation, I'd judge by the local rules where the authors are).

  * I'm after a level of agreement that is stronger/wider than "my professional friends agree with me" and also stronger than "my local community/society" or "my religion says": e.g. my country has stricter laws (which I take as surrogate to ethical agreements) on experiments on humans than some other countries. But these differences do not mean that that other society thinks causing harm to humans is ethically fine, the differences result from nuances in where exactly the line is drawn, how much suffering a particular procedure is judged to cause, how much suffering is thought bearable and how much suffering is considered worth while. So there may be some cases where the judgment differs, but on many cases, the ethical judgment will agree. And that is what I'm after here: a "current mainstream consensus" (thanks @ObscureOwl) across continents, countries, religions, ....

  (I don't think one can claim that every single person agrees on any given ethical point - but asking for such an absolute agreement is pointless for the practical purpose here since it could not be used to determine whether someone's acts are unethical: as soon as they'd disagree, the required agreement could not be reached.)

• But I may have ethical objections that result from my (OP's) world-view/philosophy/religion/... and where I'd say while it is perfectly fine to personally object, but here I (OP) have to tolerate that other people have other points of view. Here,

  • a rejection as above is not possible. It would be abuse of power for a reviewer to force their personal opinion onto the authors and therefore unethical.

  • I may say "I won't be associated in any way with any such research" and refuse being a reviewer.
    Unfortunately, this won't do the least in terms of diminishing the unethical research.

  • Sometimes, digging a bit deeper in why exactly you object to this research can help to arrive at a more constructive solution in that I do the review and in a fair and relevant manner make my concerns heard.

  General personal concerns that are expressed on a more detailed level may translate to ethical concerns that are universally agreed upon (on a narrower scope than the more general concern) or sometimes also to scientifically relevant points.

---

Example from medical research

I once got a paper for review where more than 100 rodents had been used and killed for research. The paper reported proper ethical approval and proper handling (so so far fulfilled the universally agreed upon ethical requirements). They had studied a whole lot of different conditions.
The many different conditions meant that the supgroups in the experiment had only very few animals. While it may be good at the first glance to not make more animals suffer, it really meant that the whole study was worthless because no reliable conclusions could be drawn. This could have been known in advance.

Thus, we have

• the scientific concerns of a no proper sample size planning, leading to
• the study results being too uncertain to be of any scientific value and this also leads to
• the universally agreed upon ethical concern that needless suffering was caused to the animals.

In other words, valid reasons to recommend rejecting the study.

---

 Application to the paper in question

One obvious difference between this example and the automated arms is that the concerns with animal research are about what has been done already whereas I guess your automated arms concerns are more about future real-life consequences.

Considering as a thought experiment that it is acceptable for you to think about more specialized concerns such as that the proposed technology poses a danger to civilians.

While as you say a ban of automated arms is not (yet) universally agreed upon, AFAIK it is universally agreed upon that civilians should not be put into needless or unnecessarily high danger.

Starting form this narrower but universal ethical concern, I do suspect from your description that there may be several related scientific and engineering concerns with the paper. Which are serious because of the ethical concerns behind them.

The performance of the algorithm is scored based on the number of times the simulated drone autonomously hit the target.

Any such figure of merit can at most be part of the verification and validation of the algorithm.

Valdiation must include estimating and reporting the risk to civilians: if we don't know whether the algorithm leads to unnecessary danger to civilians, we cannot possibly consider it being fit for purpose.

Personally, I'd suspect that moreover there is a trade-off here in that hitting more targets may very well come at the price of also hitting more civilians. If that is possible, a scoring function that does not penalize hitting civilians may well be considered inherently unsuitable even for development purposes.

I frequently find flawed validation procedures in papers with far less dangerous topics. If the topic is not too important, and the experimental effort to perform a proper validation is out of proportion now, I'm often OK if the resulting limitations are honestly discussed.

The manuscript in questions doesn't even seem to fulfill this much weaker requirement:

The authors never discuss possible ethical concerns of this technology,

(And with such a critical topic I'd tend to insist on proper validation.)

---

neither they explicitly say if this technology will be used on real drones

... this they may not know.

Answer 9

In the context of reviewing an article for publication, it is not your job to Save The World. Setting up a dialog with the authors would be to place yourself as arbiter of the actions of a group of people. The editor of the journal should, quite rightly, reject such a plan as part of the review of an article. Though it is just possible such a program might be acceptable outside the context of a review of an article, the journal is by no means obligated to follow such a plan.

You basically have two choices. You can withdraw from review. Or you can give an objective review of the paper based on the content and the publication policies of the journal.

If the journal in question routinely publishes such information, and you cannot accept such items as moral or ethical, then you should inform them, immediately, that you should not be considered for reviewing such articles.

The cure for speech you disagree with is more speech. If you think the work in this article should not have been done, then you should publish your work explaining why. There will be outlets for your positions and arguments. If they are sound arguments based on sound reasoning and valid data, then they will be accepted.

Answer 10

To me, the answer is quite simple.

The matter at hand is not considered unethical by a clear majority of the scientific community (if it were there would be rules against it). Consequently, whether or not you consider it to be unethical is of little relevance, your views should not impact the ability of the scientific machine to make progress.

So your choices are withdraw and make no recommendation as to what happens to the paper or to review the paper on its merits and leave the discussions on ethics to their proper forum which is not the review process.

Answer 11

I would review the paper and point out that the very practical limitations of the work are not clearly stated.

The ultimate goal of this research is clearly to construct an autonomous weapon system which can function on the battlefield. Now, all warfare is subject to law. This includes the principle of proportionality. The principle of proportionality allows for collateral damage and civilian casualties as long as such losses are not excessive compared with the direct military advantage resulting from the attack.

It is clear that the authors have not considered the principle of proportionality, because their scoring algorithm does not consider the value of the military target and has no concept of collateral damage or the loss of civilian life. It follows that any autonomous weapons system controlled using their algorithm will have a high potential for violating the laws of war. Therefore, their research has limited practical value and as scholars they are obligated to state such limitations clearly.

If you step aside, then there is no guarantee that these very practical limitations will be mentioned. More importantly, the readers will not be reminded of the laws of war.

There is a good summary of the legal principles of warfare in this article. The main focus is on the principle of proportionality.