Systematic reviews are an increasingly popular academic research method and manuscript style, often garnering many citations when published. In fact, the most recent bibliometric analysis of more than 1,200 published systematic reviews found they have cited an average of 26 times over a 4-year period after publication, or 6.6 citations per year. While publishing a systematic review can certainly add to your academic profile, with 85% of these manuscripts being rejected by journals at submission, success is far from guaranteed.

Although systematic reviews originated in clinical medicine, other disciplines are now finding the method effective to synthesize evidence as well. The following tips will help you avoid the common pitfalls that lead to high rejection rates of systematic review submissions.

1) Clearly differentiate a systematic review from other types of reviews

A common misunderstanding is that any organized review of the literature is a systematic review. In fact, there are more than a dozen types of reviews and although they may have names or processes that sound similar to a systematic review, they differ in important ways. For example, a literature review is a generic term that refers to a narrative summary of primarily current literature and often encompasses a broad sweep of the literature on a certain topic. A scoping review looks at the size and scope of the literature and can include ongoing research as well. This type of review may look somewhat at the quality of the literature but does not include a formal quality assessment. For more information on, as well as pros and cons of the different review types, see Grant, M and Booth, A.: “A typology of reviews: An analysis of 14 review types and associated methodologies,” Health Information and Libraries Journal, 26, pp.91–108.

When deciding which type of review is applicable to your project, it is important to review the journals in your field and note which types of reviews are accepted. Most journals do not accept every type of review article. Also, closely read the Journal’s Guidelines for Authors, which will detail the requirement elements of the review articles they accept for submission.

2) Assemble your optimal systematic review team

Another way that systematic reviews vary from a basic literature review is the need to assemble a core team of individuals who bring unique skills to the project. First, you will need subject matter experts who can guide construction of the research question, assessment of the literature, and interpret the meaning of your findings for your field. Second, you need at least one team member with prior experience conducting systematic reviews. The steps in systematic reviews are very detailed and must be followed precisely to have a valid review. Finally, adding a research librarian to your team is essential. A quality systematic review relies on using targeted search terms specific to each database to be searched. One cannot simply put a scattering of basic search term in to Google Scholar or PubMed for a systematic review. As the quintessential experts on constructing optimal searches and understanding the myriad of databases available, research librarians are a necessity.

3) Prospero and PRISMA will help guide your way

Once you have assembled your optimal team you will want to start drafting the protocol for your systematic review. Prospero is an international prospective register of systematic reviews. On the Prospero website you can search for existing reviews that may be similar to yours and see the protocols for other current or completed reviews. You may also register your own review protocol on the site. Unlike registration on clinicaltrials.gov, which is required by most medical journals for manuscript submission, requirements for Prospero registration are not as widespread at this point. However, with increasing concerns about protecting intellectual property, registering on Prospero gives you record of when you established your precise line of inquiry. Your second essential resource is PRIMSA, short for the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. PRISMA is your go-to guide for the detailed steps and documentation needed for conducting and writing up a systematic review.

4) Use PICO to write a strong research question

Every quality systematic review relies on a strongly constructed research question that aims to provide evidence to inform a clinically or practically important issue for the field. Traditionally, systematic reviews were designed to inform medical decision making, but have just as much application in other fields of study. Precisely defining your a) population of interest, b) intervention of interest, c) comparison (if any), and d) outcome(s) of interest is key. Closely consider the kind of question you are trying to answer.

Some examples of question variations include:

• What is the effect of different interventions on a particular outcome?
• What is the effect of a single intervention type on a given outcome?
• What is the association between one factor and a given outcome in a specific population?
• Which test is a better predictor of a particular outcome in a specific population?

5) Explore different methods for grading study quality

A key difference between systematic reviews and other types of reviews is a formal assessment of the quality of the existing body of literature. In the medical field, GRADE, a systematic approach to rating the certainty of evidence in systematic reviews and other evidence syntheses, is most commonly used. However, GRADE may not be the best choice for your field or if your review does not center on literature that primarily consists of randomized controlled trials. For non-randomized studies you might use the ROBINS-I or Methodological Index for Non-Randomized Studies (MINORS). There are tools for grading the quality of qualitative reviews as well.

In short, undertaking a systematic review can be a somewhat arduous but potentially rewarding type of academic writing. By following these five tips you will be off to a great start on setting the foundation for your systematic review.

---

Julie Reeder has been an Associate Editor for the Journal of Nutrition Education and Behavior for six years, where she makes decisions regarding 150 decisions manuscripts each year. In her role as Senior Research Analyst with the State of Oregon WIC Program she has conducted numerous interviews and focus groups over the past two decades.