Systematic reviews – what can they do for us?
Systematic reviews – what can they do for us?
Looking at a systematic review is a way of seeing an overview of the evidence on a particular topic. This can be helpful because, as practitioners it is impossible for us to keep up with all developments, and be aware of all the research, on every clinical problem we are likely to encounter.
In a systematic review, the authors have methodically identified, appraised and then summarised evidence from all available studies relevant to a defined topic of interest. The method used for finding the studies needs to be transparent and reproducible (i.e. systematic), and the criteria for including individual studies in the review (e.g. they must be randomised controlled trials; reported in English) must be explicitly defined before searching for the studies. This helps to reduce selection bias that might otherwise influence the results of a review. The trials need not be the same in all respects; they should broadly seek to answer the same clinical question, but details such as particular patient characteristics and drug doses used may differ. Ideally data from all relevant trials, not only those for which results have been published, should be included. This is because of publication bias – the tendency for only ‘positive’ or ‘interesting’ results to be published. Obtaining the results of all trials, both published and unpublished, is a laborious task.
If appropriate (i.e. the included trials have similar enough characteristics), the results from different studies can be combined mathematically to give a summary statistic giving an overview of the results for a particular outcome measure. Meta-analysis is feasible with as few as two studies. Sometimes, it is only through the power of a systematic review that an overall picture of clear benefit of a treatment becomes clear. A summary statistic (meta-analysis) can give a more accurate indication of the effects of a treatment. But beware of a meta-analysis that is not based on a systematic review of evidence (i.e. is a collation of data from selected studies).
Systematic reviews are well established in human medicine. One of the most well-known organisations producing systematic reviews is the Cochrane Collaboration, an international network of individuals and institutions committed to preparing, maintaining and disseminating systematic reviews of the effects of health care. Systematic reviews (called Cochrane reviews) are prepared by a group of collaborating authors (a Cochrane Review group). So far, there are relatively few published systematic reviews on veterinary topics (and no veterinary equivalent of Cochrane) but the number is growing fast. VetsSRev is a database of veterinary systematic reviews.
Finding a good-quality systematic review that answers your question is extremely helpful. But systematic reviews do not always have the answers you are looking for:
systematic reviews ask a very specific research question about a particular intervention in a clearly defined group of subjects with have a health condition or problem; the review may not deal with the specific problem in which you are interested;
often they do not look at the long-term effects of the intervention;
reviews more often assess the benefits, rather than the harms, of treatments.
All too often systematic reviews report that there is no good-quality research on a topic. But even knowing that no reliable evidence for a treatment exists can be helpful:
it shows when it is appropriate to go to the next level in the hierarchy of evidence such as non-randomised controlled trials, case series, case reports or personal experience;1
it can be a relief to find out from a systematic review that you are not missing some important new evidence about a treatment of a particular disease and that it is all right to carry on with your favourite treatments gleaned from clinical experience; but absence of high-quality evidence means being more cautious in claiming that the proposed treatment is the best choice;1
documenting uncertainty is a key step forward in prioritising new research.
Finally, it is important to be aware that the quality of systematic reviews can vary and so it is useful to know how to judge a systematic review (e.g. using a checklist, such as one developed by the Critical Appraisal Skills Programme [CASP]).
1. Williams H. Evidence-based veterinary dermatology – better to light a candle than curse the darkness. Vet Dermatol 2010; 21: 1–3.
Our purpose...
......is to provide busy veterinary professionals with impartial information on veterinary medicines with which to make treatment decisions in the best interests of animals, their owners and the environment. We mainly do this through the Virtual Veterinary Medicines Academy where our evidence-based peer-reviewed appraisals are the result of a rigorous research and editorial process and are presented succinctly in our multi-media CPD modules. We’re independent: we don’t sell ads, or receive commercial support. We’re funded by subscribers so you can be sure the information we provide is completely objective. Subscribers get unlimited access to the Virtual Veterinary Medicines Academy.