How to Read a Paper

Andrich van Wyk -

A few years ago I came across a method for reading academic papers which I've kept coming back to as a reliable systematic approach to efficiently read important papers of varying complexity.

The method itself comes from a paper by Prof. Srinivasan Keshav, an ACM Fellow and researcher at the University of Waterloo. I recommend reading his paper, but I summarise the system here.

The Three Pass System

The system uses a top down three pass approach with each pass delving deeper into the details of the paper. Each pass has a specific goal. Depending on what you need to obtain from the paper, completing all three passes may not be necessary.

The First Pass

The goal of the first pass is to get a high level overview of the paper:

  • Read the title, abstract, introduction, section and subsection headings and the conclusion.
  • Glance at the references, noting whether you might have read any of them.

After the first pass you should be able to categorize the paper, understand the paper's context, validate the basic assumptions for correctness, note the main contributions and be able to determine the paper's clarity.

The first pass is sufficient to determine whether you are interested in the paper, whether it is relevant to your research area and whether there are any questionable assumptions made which may deter your interest.

Also note, if you are writing a paper, a first pass is perhaps all a reviewer will give you. Pay special attention to the parts mentioned above. Strive to be clear and concise in your headings, introduction, conclusion and abstract.

The Second Pass

With the second pass the goal is to understand the content of the paper to the point where you could explain it to someone else:

  • Carefully read the paper, but ignore details such as proofs or very technical details.
  • Make comments and notes on important points.
  • Study any figures or graphs, note details such as the axes, labeled points and whether statistical variance is indicated etc.
  • Note all unread references for further reading.

Doing a second pass is appropriate for papers that you are interested in, but aren't necessarily directly related to your work. After the second pass you may or may not understand the paper. If it is critical to understand the work, or you are reviewing the paper, move on to the third pass.

The Third Pass

The idea of the third pass is to understand the paper with such detail such that you could re-implement the paper.

  • Read the paper with great attention to detail, identifying and challenging every assumption.
  • Given the same assumptions, think about how you would reproduce and present the result.
  • If novel techniques or methods are used, make sure you understand them to the degree where you could use them yourself.

Comparing your idea of implementing the paper with the actual paper will highlight areas where the paper excels or falls short. After the final pass you should be able to reconstruct the structure of the paper from memory, be familiar with the techniques used and identify implicit assumptions and missing references.

Conclusion

For more detail on the system and its motivations and related work, please read Prof. Keshav's paper. It also includes a step based approach for doing a literature survey.

References

  1. Keshav, S., 2007. How to read a paper. ACM SIGCOMM Computer Communication Review, 37(3), pp.83-84.