A manual for writers of research papers, theses, and dissertations, Ninth edition - Kate L. Turabian 2018
Evaluate sources for relevance and reliability
Finding useful sources
Research and writing
You will probably find more sources than you can use, so you must evaluate their usefulness by skimming quickly for two criteria: relevance and reliability.
3.3.1 Evaluate Sources for Relevance
Once you decide a book might be relevant, do this:
✵ ▪ Skim its index for your keywords, then skim the pages on which those words occur.
✵ ▪ Skim the first and last paragraphs in chapters that use a lot of your keywords.
✵ ▪ Skim the book’s introduction, especially its last page, where authors and editors often outline their text.
✵ ▪ Skim its last chapter or conclusion, especially the first and last several pages.
✵ ▪ Skim prologues, summary chapters, and so on.
✵ ▪ Check the bibliography for titles relevant to your topic.
Be sure that you’re looking at a book’s most recent edition. Researchers change their views over time, refining them, even rejecting earlier ones. If your source is an e-book, still follow these steps, but you can also search the whole text for your keywords. If you are an advanced researcher, read book reviews of promising sources (see section 4 of the bibliography of resources in your field).
If your source is a journal article, do this:
✵ ▪ Read the abstract, if it has one.
✵ ▪ Skim the introduction and conclusion; if they are not marked off by headings, skim the first six or seven paragraphs and the last four or five.
✵ ▪ Skim for section headings, and read the first and last paragraphs of those sections.
✵ ▪ Check the bibliography for titles relevant to your topic.
If your source is online, do this:
✵ ▪ If it looks like a printed article, follow the steps for journal article, and also search for your keywords.
✵ ▪ Skim sections labeled “Introduction,” “Overview,” “Summary,” or the like. If there are none, look for a link labeled “About the Site” or something similar.
✵ ▪ If the site has a link labeled “Site Map” or “Index,” check it for your keywords and skim the referenced pages.
✵ ▪ If the site has a search function, type in your keywords.
3.3.2 Evaluate Sources for Reliability
You can’t judge a source until you read it, but there are signs of its reliability:
1. 1. Is the source published by a reputable press? Most books and journals published by university presses, whether in print or in electronic editions, are reliable, especially if you recognize the name of the university. You can also trust some commercial presses in some fields, such as Norton in literature, Ablex in sciences, or West in the law. Be skeptical of a commercial book that makes sensational claims, even if its author has a PhD after his name. Be especially careful on contested social issues such as gun control or climate change. Many books and articles are published by individuals or organizations driven more by ideology than by evidence.
2. 2. Is the book or article peer reviewed? Most reputable presses and journals ask experts to review a book or article before it is published; this is called peer review. Essay collections published by university presses are often but not always peer reviewed; sometimes they are reviewed only by the named editor or editors. Few commercial magazines use peer review, and fewer still check an author’s facts. If a book or article hasn’t been peer reviewed, use it cautiously.
3. 3. Is the author a reputable scholar? This is a hard question to answer if you are new to a field. Most established scholars are reliable, but use good judgment: even reputable scholars can have axes to grind, especially if their research is supported by a special interest group.
4. 4. If the source is available only online, is it sponsored by a reputable organization? A website is only as reputable as its sponsor. You can usually trust one that is sponsored and maintained by a reputable organization. Some sites run by individuals are reliable; most are not. Do a web search for the name of the sponsor.
5. 5. Is the source current? You must use up-to-date sources, but what counts as current depends on the field. In computer science, a journal can be out of date in months; in the social sciences, ten years pushes the limit. Publications have a longer shelf life in the humanities: literary or art criticism, for example, can remain relevant for decades, even centuries.
6. 6. If the source is a book, does it have notes and a bibliography? If not, be suspicious, because you have no way to follow up on anything that the source claims.
7. 7. If the source is a website, does it include bibliographic data? You cannot judge the reliability of a site that does not indicate who sponsors and maintains it, who wrote what’s posted there, and when it was posted and last updated.
8. 8. If the source is a website, does it approach its topic judiciously? Your readers are unlikely to trust a site that engages in heated advocacy, attacks other researchers, makes wild claims, uses abusive language, or makes errors of spelling, punctuation, and grammar.
The following criteria are especially important for advanced researchers.
1. 9. If the source is a book, has it been well reviewed? Many fields have indexes to published reviews that tell you how others evaluate a source.
2. 10. Has the source been frequently cited by others? You can roughly estimate how influential a source is by how often others cite it. To determine that, consult a citation index (in the bibliography, see section 4 in your field). If you find that a source has been cited repeatedly by other scholars, you can infer that experts in the field regard it as reliable and significant. Such sources are said to have an “high impact factor.”
Those signs don’t guarantee that a source is reliable, but they should give you reasonable confidence in it. If you can’t find reliable sources, acknowledge the limits of the ones you have. Of course, you may find an exciting research problem when you discover that a source thought to be reliable is not.