Monday, 9 January 2017

How to (seriously) read a scientific paper | Science | AAAS


How to (seriously) read a scientific paper

Adam Ruben’s tongue-in-cheek column about the common difficulties and frustrations of reading a scientific paper broadly resonated among Science
Careers readers. Many of you have come to us asking for more (and more
serious) advice on how to make sense of the scientific literature, so
we’ve asked a dozen scientists at different career stages and in a broad
range of fields to tell us how they do it. Although it is clear that
reading scientific papers becomes easier with experience, the stumbling
blocks are real, and it is up to each scientist to identify and apply
the techniques that work best for them. The responses have been edited
for clarity and brevity.

Do you have your own tips or other questions you’d like answered? Leave them in the comments section.

How do you approach reading a paper?

I start by reading the abstract. Then, I skim the introduction and
flip through the article to look at the figures. I try to identify the
most prominent one or two figures, and I really make sure I understand
what's going on in them. Then, I read the conclusion/summary. Only when I
have done that will I go back into the technical details to clarify any
questions I might have.

- Jesse Shanahan, master's candidate in astronomy at Wesleyan University in Middletown, Connecticut

I first get a general idea by reading the abstract and conclusions.
The conclusions help me understand if the goal summarized in the
abstract has been reached, and if the described work can be of interest
for my own study. I also always look at plots/figures, as they help me
get a first impression of a paper. Then I usually read the entire
article from beginning to end, going through the sections in the order
they appear so that I can follow the flow of work that the authors want
to communicate.

If you want to make it a productive exercise, you need to have a
clear idea of which kind of information you need to get in the first
place, and then focus on that aspect. It could be to compare your
results with the ones presented by the authors, put your own analysis
into context, or extend it using the newly published data. Citation
lists can help you decide why the paper may be most relevant to you by
giving you a first impression of how colleagues that do similar research
as you do may have used the paper.

- Cecilia Tubiana, scientist at the Max Planck Institute for Solar System Research in Göttingen, Germany

If I’m aiming to just get the main points, I’ll read the abstract,
hop to the figures, and scan the discussion for important points. I
think the figures are the most important part of the paper, because the
abstract and body of the paper can be manipulated and shaped to tell a
compelling story. Then anything I’m unclear about, I head to the

If I want to delve deeper into the paper, I typically read it in its
entirety and then also read a few of the previous papers from that group
or other articles on the same topic. If there is a reference after a
statement that I find particularly interesting or controversial, I also
look it up. Should I need more detail, I access any provided data
repositories or supplemental information.

Then, if the authors' research is similar to my own, I see if their
relevant data match our findings or if there are any inconsistencies. If
there are, I think about what could be causing them. Additionally, I
think about what would happen in our model if we used the same methods
as they did and what we could learn from that. Sometimes, it is also
important to pay attention to why the authors decided to conduct an
experiment in a certain way. Did the authors use an obscure test instead
of a routine assay, and why would they do this?

- Jeremy C. Borniger, doctoral candidate in neuroscience at Ohio State University, Columbus

I always start with title and abstract. That tells me whether or not
it’s an article I’m interested in and whether I’ll actually be able to
understand it—both scientifically and linguistically. I then read the
introduction so that I can understand the question being framed, and
jump right to the figures and tables so I can get a feel for the data. I
then read the discussion to get an idea of how the paper fits into the
general body of knowledge.

I pay attention to acknowledgement of limitations and proper
inference of data. Some people stretch their claims more than others,
and that can be a red flag for me. I also put on my epidemiologist hat
so that I can try to make sure the study design is adequate to actually
test the hypotheses being examined.

As I go deeper into the argument framing, figures, and discussion, I
also think about which pieces are exciting and new, which ones are
biologically or logically relevant, and which ones are most supported by
the literature. I also consider which pieces fit with my pre-existing
hypotheses and research questions.

- Kevin Boehnke, doctoral candidate in environmental health sciences at the University of Michigan, Ann Arbor

My reading strategy depends on the paper. Sometimes I start by
skimming through to see how much might be relevant. If it is directly
applicable to my current topic, I’ll read the paper closely, apart from
the introduction that is probably already familiar. But I always try to
figure out if there are particular places or figures that I need to pay
close attention to, and then I go and read the related information in
the results and discussion.

I also check if there are references that I may be interested in.
Sometimes I am curious to see who in the field has—or more likely has
not—been referenced, to see whether the authors are choosing to ignore
certain aspects of the research. I often find that the supplementary
figures actually offer the most curious and interesting results,
especially if the results relate to parts of the field that the authors
did not reference or if they are unclear or unhelpful to their
interpretation of the overall story.

- Gary McDowell,
postdoctoral fellow in developmental biology at Tufts University in
 Medford, Massachusetts, and visiting scholar at Boston College

When reading papers, it helps me to have a writing task so that I am
being an active reader instead of letting my eyes glaze over mountains
of text only to forget everything I just read. So for example, when I
read for background information, I will save informative sentences from
each article about a specific topic in a Word document. I'll write
comments along the way about new ideas I got or questions I need to
explore further. Then, in the future, I’ll only need to read this
document instead of re-reading all the individual papers.

Likewise, when I want to figure out how to conduct a particular
experiment, I create a handy table in Excel summarizing how a variety of
research teams went about doing a particular experiment.

- Lina A. Colucci, doctoral candidate at the Harvard-MIT Health Sciences and Technology program

I usually start with the abstract, which gives me a brief snapshot of
what the study is all about. Then I read the entire article, leaving
the methods to the end unless I can't make sense of the results or I'm
unfamiliar with the experiments.

The results and methods sections allow you to pull apart a paper to
ensure it stands up to scientific rigor. Always think about the type of
experiments performed, and whether these are the most appropriate to
address the question proposed. Ensure that the authors have included
relevant and sufficient numbers of controls. Often, conclusions can also
be based on a limited number of samples, which limits their

I like to print out the paper and highlight the most relevant
information, so on a quick rescan I can be reminded of the major points.
Most relevant points would be things that change your thinking about
your research topic or give you new ideas and directions.

- Lachlan Gray, deputy
head of the HIV Neuropathogenesis Lab at the Burnet Institute and
adjunct research fellow in the Department of Infectious Disease at
University in Melbourne, Australia

What I choose to read is based on relation to my research areas and
things that are generating lots of interest and discussion because they
are driving the way we do psychology, or science more widely, in new
directions. Most often, what I am trying to get out of the papers is
issues of methodology, experimental design, and statistical
analysis. And so for me, the most important section is first what the
authors did (methods) and second what they found (results).

It can also be interesting to understand why the authors thought they
were doing the study (introduction) and what they think the results
mean (discussion). When it is an area that I know a lot about, I don't
usually care much about these sections because they often reflect the
authors' theoretical predilections and one of many ways to think about
the method and results. But when it is an area that I know very little
about, I read these closely because then I learn a lot about the
assumptions and explanatory approaches in that area of research.

- Brian Nosek,
professor in the Department of Psychology at the University of Virginia
and executive director of the Center for Open Science in

First I read very fast: The point of the first reading is simply to
see whether the paper is interesting for me. If it is I read it a second
time, slower and with more attention to detail.

If the paper is vital to my research—and if it is theoretical—I would
reinvent the paper. In such cases, I only take the starting point and
then work out everything else on my own, not looking into the paper.
Sometimes this is a painfully slow process. Sometimes I get angry about
the authors not writing clearly enough, omitting essential points and
dwelling on superfluous nonsense. Sometimes I am electrified by a paper.

- Ulf Leonhardt, professor of physics at the Weizmann Institute of Science in Rehovot, Israel

I nearly always read the abstract first and only continue on to the
paper if the abstract indicates that the paper will be of value to me.
Then, if the topic of the paper is one I know well, I generally skim the
introduction, reading its last paragraph to make sure I know the
specific question being addressed in the paper. Then I look at the
figures and tables, either read or skim the results, and lastly skim or
read the discussion.

If the topic is not one I know well, I usually read the introduction
much more carefully so that the study is placed into context for me.
Then I skim the figures and tables and read the results.

- Charles W. Fox, professor in the Department of Entomology at the University of Kentucky in Lexington

It is important to realize that shortcuts have to be taken when
reading papers so that there is time left to get our other work done,
including writing, conducting research, attending meetings, teaching,
and grading papers. Starting as a Ph.D. student, I have been reading the
conclusions and methods of academic journal articles and chapters
rather than entire books.

- Rima Wilkes, professor in the Department of Sociology at the University of British Columbia, Vancouver

As editor-in-chief of Science, I have to read and comprehend
papers outside of my field all the time. Generally, I start with the
corresponding editors’ summaries, which are meant for someone like me: a
science generalist who is interested in everything but dives deeply
only into one field. Next, I check to see if someone wrote a News
article on the paper. Third, I check to see if there is a Perspective by
another scientist. The main goal of a Perspective is to broaden the
message of the paper, but often the authors do a great job of extracting
the essence of the article for non-specialists at the same time.

Then I tackle the abstract, which has been written to broadly
communicate to the readership of the journal. Finally, I move on to the
paper itself, reading, in order, the intro, conclusions, scanning the
figures, and then reading the paper through.

- Marcia K. McNutt, Editor-in-Chief, Science journals


What do you do when there is something you don’t understand?

I like to read online so that I can easily cut and paste words I don’t know into a browser to check what they mean.

- McNutt

If it's only a few things in the article, I'll make a note to look
them up later. If I am really struggling to proceed through the paper, I
try to look up a review article or a textbook chapter to give me the
necessary background to proceed, which I generally find much more

There are a lot of acronyms and jargon that can be subfield-specific,
so I usually don't wade through the details unless it's for my own
research. But I always try to take my time to really understand the
methods being used.

- Shanahan

I will typically pause immediately to look up things I don’t
understand. The rest of the reading may not make sense if I don’t
understand a key phrase or jargon. This can backfire a bit, though, as I
often go down never-ending rabbit holes after looking something up
(What is X? Oh, X influences Y. … So what’s Y? etc…). This can be sort
of fun as you learn how everything is connected, but if you’re crunched
for time this can pull your attention away from the task at hand.

Sometimes, all the jargon in a paper can cloud the whole point of the
experiments in the first place. In such cases, it helps to ask
yourself, “What question were the authors trying to answer?” Then you
can determine whether they succeeded or failed.

- Borniger

It depends on how much the non-understandable bits prevent me from
following the main ideas. I usually do not try to understand all the
details in all the sections the first time I read a paper. If
non-understandable parts appear important for my research, I try to ask
colleagues or even contact the lead author directly. Going back to the
original references to get all the background information is the last
resort, because time can be limited and collaborations and personal
contacts can be much more efficient in solving specific problems.

- Tubiana

Sometimes, you can just read through a paper and any terms you're not
familiar with will become clearer by the end. If it is very heavy
going, then stopping and seeking additional information is usually the
way to go. I do a quick Google search on the topic, theme, method,
jargon, etc. If it is a very dense article, sometimes it will require a
few read-throughs before it all starts to make sense.

- Gray

The question I ask myself is, “Do I need to understand what that
means in order to get what I need from this paper?” I now read articles
in research areas well outside of my expertise, and I often don't need
more than superficial knowledge of the substantive content. If I can't
do anything with the paper unless I don't understand that depth, then I
do more background research.

- Nosek

Lately, I have had to read a number of papers outside my area of
expertise with a lot of unfamiliar jargon. In some cases, I am able to
directly extract the information I need from the results or figures and
tables. In other cases, I use Google searches to define terms and
concepts in the paper or read the cited references to better understand
the points being made. Occasionally, papers are so incomprehensible (to
me, at least) that I don't bother reading them.

- Fox

Do you ever feel overwhelmed reading papers, and how do you deal with that?

All the time. If the paper is relevant to a problem I am trying to
solve, you can be sure that there are key things in the paper that I do
not understand. That confusion is not a threat; it is an opportunity. I
am ignorant; I need to become less ignorant. This paper may help me.

Simultaneously, some papers are written terribly and are not worth
the effort. Someone else has surely written about the concepts more
clearly so that I can keep my confusion focused on understanding
substance rather than poor grammar.

- Nosek

I especially get overwhelmed if it's not in my subfield, if it's
long, and if it's full of technical jargon. When this happens, I break
it down into chunks and will read it over the course of a few days, if
possible. For really difficult papers, it also helps to sit down and
work through it with a colleague.

- Shanahan

Yes, many times. This is why I developed my own reading strategies,
by talking to other scientists and by trial and error. I also have
thrown up my hands in frustration and tossed the offending papers away,
never to read them again.

- Boehnke

Yes, and in these cases you have to realize that some papers are the
result of years of work by dozens of scientists. Expecting to digest and
understand everything in it in one afternoon is a far-fetched idea.

- Borniger

I have often felt overwhelmed! But certain sections might not need as
deep an understanding as others. You also need to know your own limits:
Are there some parts of the paper that you would like to emulate but
are not part of your expertise and might become “accessible” through

- Tubiana

If I feel the paper is very important to what I’m doing, I’ll leave
it a while and go back to it again a couple of times. But if it’s too
overwhelming, then I have to leave it aside, unless someone among the
colleagues I have contacted has been able to interpret it.

- McDowell

Do you have any other tips you’d like to share?

If there is a seminal paper I want to thoroughly understand, I find
some way to give a journal club-style presentation about it. Speaking
about a particular paper and answering questions is the best way for me
to learn the material.

Also, get a good reference manager. Mendeley helps me do my research, read literature, and write papers.

- Colucci

At the beginning, new academic readers find it slow because they have
no frame of reference for what they are reading. But there are ways
to use reading as a system of creating a mental library, and after a
few years, it becomes easy to slot papers onto your mental shelves. Then
you can quickly skim a paper to know its contribution.

- Wilkes

Be patient. Don’t be afraid or ashamed to use Wikipedia or other,
more lay-audience sources like blog posts to get a feel for your topic.
Ask many, many questions. If you can’t get a clear understanding of the
paper, talk with people in your circle. If you are still confused and
it's really important to understand the concepts, email the authors.

- Boehnke

Don’t hesitate to talk to more experienced scientists. You will be
doing THEM a favor by having them explain to you in terms you understand
what a complex paper means. All scientists need more experience
translating complex concepts into common terms.

- McNutt

If at all possible, read often. Try to keep a bibliography file with a
summary of the article, any important points, even a figure or two,
along with citation information. Pay attention to different ways of
structuring an article, and pay attention to different styles of
writing. This will help you develop a style that is effective and also

- Shanahan

DOI: 10.1126/science.caredit.a1600047

Elisabeth Pain

Elisabeth Pain is contributing editor for Europe.

How to (seriously) read a scientific paper | Science | AAAS

No comments:

Post a Comment