Scientific writing 3.0: A reader and writer's guide - Jean-Luc Lebrun, Justin Lebrun 2021

Trap 3 — The trap of references
Introduction Part II: Popular traps
Paper structure and purpose

Incorrect reference

How many references do you list at the end of your paper? 20, 30, 50? Did you read all the corresponding papers? Where do these references come from?

Let us just take one of the references you added to the reference section of your paper. Where did it come from? A paper you wrote? A review article? A citation index? Let’s imagine it comes from the list of references you found at the end of a paper you read, and let’s follow the trail of that reference like a bloodhound. Where did the writer find that reference? In an older paper they wrote, a review article, a citation index, or in the reference section from another paper? Do you get my drift? The reference that you are using is as reliable as the process used to capture the reference by all the writers who used that reference before you. This process may have been manual or electronic. If the reference was extracted from an older printed paper through OCR (optical character recognition), it may contain errors. If the error occurred during manual entry, it is then propagated electronically, unless someone got hold of the original paper, spotted, and corrected the reference error. But why bother to check? Can’t we trust what comes to us electronically?

The news story of Philip Ball in the December 12 2002 issue of Nature created quite a stir when he reported the findings of the paper entitled:“Paper trail reveals references go unread by citing authors,” by Mikhail Simkin and Vwani Roychowdhury. In the original Simkin paper,⁴ entitled “Read before you cite!”, the authors claimed that “only about 20% of citers read the original.” The percentage may be too low and the conclusion hasty because one cannot logically infer from the copy of an erroneous reference that the writer has not read the paper. But the problem is real.

The point is this: Even though one cannot infer that the writer has not read the paper when an reference error is found, the reviewer may still have that suspicion. If the writer is suspected of taking shortcuts in his references, then the list of references is no longer indicative that the writer is knowledgeable. The overall credibility rating of the writer goes down.

Imprecise reference

But the problem of bad references does not stop there. Often times a reference is wrong not because it is incorrect, but because it is at the wrong place in a sentence. Imagine this:

Paper [6] proves that the boiling temperature of water is 100° Celsius.

Author John Smith writes the following sentence:

“It has been proven that water boils at 100° Celsius and freezes at 0° Celsius [6].”

Of course, this is wrong. Paper [6] does not even mention the freezing temperature of water. You would perpetuate the error if you wrote the following in your own paper which references John Smith’s paper:

“The freezing and boiling temperatures of water are known [6].”

Had John Smith placed the reference at the correct place, the problem would not have occurred.

“It has been proven that water boils at 100° Celsius [6] and freezes at 0° Celsius.”

The guideline for proper referencing is given in Scientific Style and Format: the CSE manual for authors, editors, and publishers, now in its eighth edition. It is very clear: “A reference immediately follows the phrase to which it is directly relevant rather than appear at the end of long clauses or sentences.” Anything else may introduce imprecision, or worse, error in attribution.

Checking the references takes time. Can’t we just trust that other people check them first? Can’t we just trust the citation tools in Endnote? Well, rubbish in, rubbish out applies in this case also. In short, check all your references and read the original papers you reference.

Unnecessary references

Writers know that supporting a text with too few references makes it less credible, so they make sure to build out their reference lists. One of the easiest places to add references is in the first few sentences in the introduction when the author establishes the problem. It is easy to pack references at the end of a sentence that expresses a common sentiment such as “pollution is a major disruptor of marine environments [1—12]”. While all twelve references may be individually relevant, bulk delivery to the reader is not acceptable. This writercentered approach to unfiltered referencing creates problems for both the reader and the publisher.

Publisher perspective: The reference list at the end of every article is essential, but it quickly grows long, especially in longer articles with 50—80 references! Publishers only have so many pages available per journal issue to publish research, so they would rather publish meaningful text than reference lists. In fact, many journals now reject reference stuffing in their instructions to authors, limiting to 5 or less references per point. Make sure you specifically check the instructions for your targeted journal before publishing!

Reader perspective: The writer may think they are helping the reader by giving them a plethora of documents to choose from, but there truly is such a thing as too much information in today’s timepressed, results-driven world of research. Expecting the reader to chase down all 12 references for complete understanding is unrealistic. At best, an extremely motivated reader may choose to pursue a handful of these references. But which ones will they pick? Will you leave it up to chance, and hope that they choose the best ones, or the ones that are most relevant to your paper? Shouldn’t you, as the writer, carefully present your reader with only the best references from that list?

Unbalanced references

So what makes a reference worth mentioning? References typically come in two flavours. The first type adds credibility to introductory statements. It informs the reader that what you have written is backed up by peer reviewed research. The problem is, one could easily fill out a reference list with references of this type. The second type points to a paper that is closer to yours. It likely shares analysis, insights, or methodologies with your own paper. These references are likely to show up in the introduction, but also throughout the rest of your paper, for example in the methodology, results, or discussion sections. These references are far more valuable to the reader. They show how your work and results contrast and compare with the rest of the scientific field.

Make sure that your introduction contains high quality references that are used more than once in your paper.

Plagiarized references

What is a plagiarized reference? Imagine you’ve just read a recently published paper on a topic similar to your own, and you believe that they’ve done a great job of setting the stage in their introduction. They clearly establish the problems faced by the field and have sourced out relevant references already. Why go through all the hard work of finding original sources when you could just re-use their references? It’s so easy to just control-c and control-v, and voila, much of the hard work of the literature search disappears. On the face of it, this doesn’t seem like it should necessarily be bad behaviour. After all, if the original author has done a good job of choosing relevant references, why is there a need to re-invent the wheel? Let’s take a look at why it’s still best to put it some hard manual work.

This behaviour is akin to undergraduate students who, when faced with writing on a new subject, head to the nearest wikipedia page and use the references listed there as their sources. The worst offenders never even read the referenced material, they trust that the subject matter they are quoting is relevant to their own specific context. That is quite a gamble. As we stated under the previous subheading of unbalanced references, every author should choose references that resonate well with their own research. The more selective they have been, the less transferable these references should be.

This said, references from other papers are excellent sources for you to start your own research, and many references may end up being reused after all. But the odds that you will end up re-using the same references in the same order to make the same points are minuscule.

Missing references

Restraining from citing the work of your closest competitor in research is tempting. You may be competing with them for grant funding or publication. Why boost their citation count and potential visibility on the stage of research? However, not citing your competition is a strategy that is likely to backfire spectacularly.

A reviewer who knows the field is familiar with both you and your competitor already. The reviewer may explain the lack of citation in one of two ways. 1) You are purposefully omitting the competition for strategic and borderline unethical reasons, putting your needs above the needs of your readers. 2) You don’t know your competitor. You may be free of any unethical conundrums, but you come across as poorly read.

Between these two choices—unethical or incompetent—you can’t win. So cite your competitors. They will need to cite you too!

Courtesy references

Harassment in Science

Today, more than usual, Vladimir stared pensively at his evening coffee. This did not escape Ruslana’s notice, who delicately asked him if everything was fine at work.

“Perceptive as always, my dear! No, your husband is just fine. It is my friend Pyotr who is in trouble.”

“Trouble? What kind of trouble? Nothing too serious, I hope?”

“That depends on your definition of serious. After leaving our institute, Pyotr found employment in a new lab, and the work habits there are quite different. Within just a few days of starting there, Helena — that’s his supervisor — walked up to him and more or less demanded that Pyotr always cites her papers in his own. Can you imagine? She said it was an ’unspoken policy’ in the lab. What nose!”

““What cheek”, dear. You still need to work on your English idioms.”

Unrepentant, Vladimir continued. “Eh, regardless, Pyotr is quite lost. He does not wish to offend his new employer, but he doesn’t feel it is ethical to cite his boss as a matter of policy. But it gets worse!”

“How so?”

“Apparently one of his junior colleagues approached him and asked to also be cited in Pyotr’s paper! He said that in exchange, he would cite Pyotr’s paper in his own paper and in this way, everyone in the lab could grow their careers together! You know like “you scratch my head, I scratch your head!”

“It’s “You scratch my back, and I’Il scratch yours”, Vlad. So your friend Pyotr is feeling the pressure from all sides!”

“That’s right, Ruslana. You hit it on the cheek!”

“On the nose, Vlad, on the nose.”

What a predicament Pyotr finds himself in. Courtesy references are references that you are asked (nicely or not) to include in your paper, regardless of their worth to the reader. It seems pretty clear cut that this behavior is objectionable, and yet it exists. Dealing with a coworker is a little easier than dealing with a boss, so let’s start there.

First of all, it isn’t always wrong to cite a coworker, even if they ask for it! Many references in the introduction of a paper are interchangeable, so if your coworker’s paper could do the job as well as another’s, why not support each other! If, however, their paper is not or mildly relevant, the waters grow murkier. You could gently stand your ground, justifying your denial through your readerfocused approach to referencing. Because you are objectively in the right, ideologically and ethically, it is difficult for them to argue back. Be gentle. Just because you are right does not relieve any of the pressure they are feeling. Offer to cite them in the future whenever it does make sense, and allow them to see that you are not unfriendly, but that your hands are tied by your concerns for the readers.

A boss asking you to cite them in all your papers is a much more clear cut case. It is unethical, but it is also much harder to say no. After all, they may have considerable say in whether or not you keep your job. Your first line of defence should be to offer to place them in the acknowledgements. This may mollify their demand for recognition in an ethical way. If however they insist on citations, you have fewer options.

Fortunately, after much thought, we the authors have come up with a solution to this conundrum: play it off as a joke. If you receive the command to do something clearly unethical, pause for a second, then smile and offer a small laugh, accompanied by a variation on the following words: “Ahaha, I see what you’re trying to do here, but I wouldn’t do that, I’m an ethical researcher. Was that a test? Did I pass?” It doesn’t matter how bad your acting skills are or if your boss can see through your bluff. By phrasing your reply this way, the boss now has a choice to make. Do they still demand to have you cite them, even though it would now be clearly exposed as an unethical act? Or do they back down, using the pretense of humor to back out of the situation without losing too much face? Many would choose the easier option, the second one. As you’ve also shown yourself to be someone that is difficult to manipulate, they are also unlikely to involve you in such schemes again.