We have collected metadata of scientific publications (in a bilingual English-French context) from several international platforms (OpenAlex, Scopus) and French platforms (Hal, Idref, etc.). Many publications are referenced multiple times, with minor variations, either within a single platform or across different platforms. After eliminating trivial cases (such as those with a unique identifier or exactly identical metadata), there remain a significant number of non-trivial duplicates, and we aim to train a model to detect them.
To achieve this, we have loaded thousands of pairs of suspected duplicate publications into an online collaborative annotation tool, and professionals in the field of scientific publishing will annotate them as "duplicate" or "non-duplicate". The distinction is subtle, for example, two successive editions of a book, or an article and its preprint are considered a single work, hence duplicates. Conversely, successive reports on an archaeological dig, or an article and a conference presentation with the same title and contributors are not duplicates. As the source platforms have different data models, the metadata has been standardized to a common denominator, including title, description, keywords, author, document type, publication date, journal, etc.
Here is my question: we need to develop guidelines for the annotators. These annotators are asking if, when it's difficult to make a decision, they can conduct internet searches to check the source platform (which may contain additional information) or to find contextual information (such as about the research project).
Would the use of external information by annotators hinder the training of the model? Or would it, on the contrary, enable the model to detect patterns that humans may not necessarily notice at first glance? We are tempted to disallow internet use and introduce a "uncertain case" tag, recommending its use when the presented information is insufficient to make a decision.
But the question is a matter of debate among us.
