Column: Ask Marta

Dear Marta,

I'm very curious to know how established scientists approach reading other people's research papers, how they go about understanding and later recalling the content. What do you do when you read? (Do you skim? Do you focus on certain sections? Do you take notes?) Please share any strategies you might have.
Sincerely,
Addled After the Abstract

Dear Addled,

Read other people's papers as you would your own! Seriously -- perhaps the most important factor in determining how to read a paper is your purpose in reading it. If you merely want to leave a minimal trace in your memory banks to remind yourself that a paper on some particular topic exists, it may suffice to read the title and the abstract. If the abstract is well-written, it will leave the requisite dent in your grey matter, assuming it's still dentable. You shouldn't have to go to an NIH study section (as some academics have) to learn that how abstracts are written is critical. It bears repeating: ABSTRACTS ARE CRITICAL, so do write them well. Indeed, I often find that if (or perhaps more accurately when) I can write a clear and concise abstract for a paper then the rest is just a matter of specifying details.

Of course, if you want to replicate the studies reported, or to cite them then these details are critical and there's no alternative but to read the paper, and to do so with care and thought. For many people this means in a quiet, non-distracting place, when you are in a relaxed and positive mood, with your mind open and focused on what you are reading rather than on all the delectable choices that the world has to offer (at minimum, no readily accessible email, as the writer hangs her head in shame, at advice better given than followed). Once may not be enough (or it may be too much) but you won't know until you read the paper from beginning to end, or at least skimmed it in order to decide just how careful a read it deserves (though experience leads me to say that there is a U-shaped function here with both the best and worst papers often necessitating the most careful readings). You can take notes (or not), via notepad, computer, in the paper margins – whatever benefits your study habits and memory and storage/access wonts. Underlining also can be useful as long as you underline judiciously, and you are consistent in the choices you make for underlining (or not). Summarizing what you read in your own words is especially effective not only as a memory aid but in helping to insure that you actually understood what you read. It really helps when you return to your notes at some later date, although occasionally you may eloquently be wrong. Also, occasionally you may want to copy a chunk of text verbatim to guarantee that you got it right (make sure you write down the page number as well) and didn't distort it through your personal eye-brain filter. Believe me this happens even to established researchers. I'm often amazed (or appalled depending on my mood) to read reviews in which authors get the facts (much less the interpretation) of a particular study wrong.

As an aside, I should add I've always found writing chapters (like grants) to be especially useful exercises because they force me to read all the papers in an area carefully and to decide whether or not they are worth citing, elaborating, praising, criticizing, emphasizing, replicating or ignoring. Moreover, although chapters don't typically amount to more than a hill of beans in the publication count (such as for tenure), they nonetheless can be very important in helping to organize your thoughts on some topic and to influence others and/or research for years to come – assuming it is thoughtful and well-written.

When evaluating a paper I pay especially close attention to the methods and the results sections. The methods because they often matter with regard to the specific pattern of results obtained (just try replicating various (neuro)cognitive or behavioral phenomena, you'll witness this firsthand); if you don't know the details you will never be able to discover which are essential and which really don't matter. It is not unusual to find that certain methodological choices are based on (scientific?) superstitions or on the past (without added consideration). And, I give special attention to results, because these too matter, and are often different from what you might believe the results to be according to the abstract or even at times the discussion. When reading results, it is very important to scrutinize the analyses, the statistics (including degrees of freedom, means, ranges, extent to which data violate assumptions of the tests applied). The numbers are there (or not) for a good reason and it's not just to please advisors, reviewers, and/or editors.

A well-written scientific paper coheres – a story about some aspect of science unfolds much like a good whodunit. There's a query, a deliberate quest, a finding or discovery of some sort, and a retelling of the tale for those who may be on a similar quest or for those who relish a good tale for whatever reason. Given that, it is sometimes both satisfying and rather pleasurable to read a paper from beginning to end: Introduction, method, results, and discussion/conclusion. However, the better the writer, the greater the likelihood that the tale is very seductive, the framing so well done that you may become blind to the logic of the experimental design or interpretation or even to certain aspects of the data or analysis. Thus, I recommend that occasionally you consider reading a scientific paper differently. For example, you might first find the one or two sentences that state the question/issue under investigation. In a well-written paper this might be found in the first or last paragraph of the introduction or discussion. Then read the methods and results with this question in mind – front and center, arrive at your own conclusions and see what the author(s) conclude(s) in the discussion and abstract. Or, alternatively, you could skim for the question and the conclusions (usually beginning and/or end of discussion section) and then read the methods and results to see if the question has been addressed with the appropriate methodology and the conclusions are supported by the data. Yet another approach is to read the conclusions and consider what question has been addressed and what sort of data is needed to allow the conclusions, followed by a thorough reading of the method and results sections. When there are figures and tables in a paper, I often start by looking at those and seeing what I think the paper is about, what the data say to me. If there are others like me (in this respect), this would imply that figures are also quite important and preferably should be drawn so as to be able to stand alone.

As already mentioned, summarizing what you read in your own words is one of the best aides to memory. To that end, after reading a paper, first put it away, and then write a brief summary keeping the following questions in mind (though you can come up with your own): Why was this research done at all? In other words, what were the author(s) goals? What was found, and how, if at all, do the findings relate to the stated goals or if there were serendipitous but noteworthy findings that were tangential to the stated goals what issues do they relate to? Are there any obvious strengths or weaknesses to the study? Are there any questions you'd like to ask the authors (and if so, don't hesitate to drop them a line and ask). Are there any loose ends? And, by all means, if you have an idea for potential follow-up experiments, write them down -- even if it is outside your current research area. Who knows what the future will bring. Perhaps include a 2-line or so precis of what you would say about this paper to someone if asked or if you were to cite it in a paper of your own, including a sense of what you thought about the quality and import of the paper. It should go without saying that you should write this in such a way that at least you understand what you meant by what you wrote (consider checking by re-reading it a few days later when intentions are no longer hanging around to ease comprehension and/or have sometime take a quick read and see what they get out of it; do this 20 times and you'll have a good parlor game; and yes, I know you don't realistically have time to do this, but it's still a good suggestion!).

Finally, no matter who you are (personality or level-wise), some papers are impossible to understand fully the first time through because you just don't have the appropriate background knowledge. If you want to or must understand this particular report, then you need to go get that knowledge – and sometimes this could take years. Remarkably, this same paper usually will seem so much easier on subsequent readings. Other times, you need only read a few papers – perhaps previous ones by the same author or some of the ones cited before finishing or re-reading the paper. And, given that your reason for reading a paper is probably the most important determinant of how you should read it, don't be hesitant (at least occasionally) to skim a paper looking for some particular finding, fact, or idea. It will always be there for another reading.

And, last two bits of advice: (1) if you plan to cite a paper, then please read it – out of respect to the author, the scientific community, and yourself!; (2) when you have time, read your own papers preferably before submission (though any time is better than not at all) as if it were someone else's.

Read and consider
Don't let your mind wither or dither
Delve in depth and link in breadth
Read and consider - probe
Look beneath the verbal robe
Search hither and yonder
Among the words you critically ponder
Read and consider – with a clear goal in mind and a critical eye
So as to find what you seek,
To help decide what to keep and what to toss
To help locate the beauty and essence (of truth?) in
the ever-mounting fluff and floss.
Read and consider – with care
And avoid being blinded by flare
blind-sided, or chided
for so eloquently citing what wasn't there.

Acknowledgements: Special thanks to Jeff Elman.

back to top

In this section of Cognitive Science Online, UCSD Professor Marta Kutas answers questions from our readers. If you have any questions you'd like to ask Marta, please contact the editors . While personal questions relating to life in academia or graduate school may be appropriate, please keep in mind that this column mainly serves to provide advice and guidance on professional matters such as teaching and academic issues. Also keep in mind that this is an advice column and neither Dr. Kutas nor the journal will assume any responsibility regarding the consequences of following or disregarding the advice provided. Take advice responsibly!