Systematic Reviews (Piloting ( The pilot study is a chance to test the…
Why systematic reviews?
- Limits of previous approaches to summarising evidence.
- Expert opinions
- Literature reviews
- Systematic reviews attempt to bring the same rigour to the review of research evidence as that used to produce empirical research evidence.
- Aims to identify, critically evaluate and summarise findings of all relevant studies found using a systematic methodology.
- An explicit, reproducible scientific methodology.
- Clear objectives and pre-defined eligibility criteria for studies.
- An aim to identify all studies that meet eligibility criteria.
- Assessment of the validity of study findings.
- A systematic presentation and synthesis of the characteristics and findings of included studies.
- Explicit methods limit bias in identifying and rejecting studies.
- Conclusions are more reliable and accurate
- Large amounts of information can be assimilated quickly by clinicians and researchers.
- Delay between research discoveries and implementation of effective diagnostic and therapeutic strategies may be reduced.
- Findings can account for some of the biases and contradictory findings found in individual studies
- Provide an account of what is known about a particular topic and what research may be lacking
Types of Reviews
- Quantitative reviews
- Use statistical procedures (typically meta-analysis) to pool together statistical findings from a range of studies.
- Appropriateness depends on:
- Quality of previous research studies (methodology)
- Heterogeneity of research studies
- Inappropriate use can lead to misleading results
- Qualitative/narrative reviews
- Use narrative methods to synthesis the results
- Used when quantitative review is inappropriate
- Not just describing the studies, but taking a particular approach to bringing together results
- Comprehensive/mixed method reviews
- A good protocol is essential to a good review…It’s a recipe book!
- In the review protocol state the plan from the review question to synthesis of the data
- A poor protocol means greater chance of losing or changing direction too much and that the review will not be systematic
- It is the basis for the methodology section of the dissertation
- A protocol would be submitted to peer review
- Review question
- Criteria (inclusion/exclusion)
- Search strategy
- Data Extraction
- Quality Assessment
- Data synthesis
- In the introduction set out the reasons for the review
- In the introduction:
- Explain why you are doing the review
- Give an introduction to the literature
- Define terms or explain a theory if applicable
- Explaiin terms and definitions
- The protocol introduction can inform the introduction of your dissertation. Have a clear threat
Review Question, Aims and Objectives
- Questions may be specific or broad
- Depending on the question they may be broken down into sub questions. (with one being the main one)
- The questions have to be clear! The PICO Framework :
- Interventions (whats being evaluated)
- Comparators (what is used as comparison)
- Outcomes (also tools and measures)
- Study design (depending on the quesition)
- PICo (for more qualitative reviews)
- P = Population
- I = Phenomena of interest
- Co = context
- Aims & Objectives
- What does the review hope to achieve
-What steps will be taken to achieve the review aims
Inclusions and Exclusion Criteria
- Criteria should capture all the studies of interest (with in and exclusion criteria)
- Not too broad or too narrow.
- Need to be practical
- They should link to the research question and it should be logical why they were chosen
- They will vary in complexity depending on the research question
- They should state publication types to be included and the language of studies
Identifying and Selecting Papers
- Clearly document the methods for selecting papers
- Range of methods:
- Electronic databases – e.g. medline/pubmed, psychinfo
- Scanning reference lists
- Search of a particular journal (browsing or searching)
- Citation searching
- Grey literature databases
- Government reports and reviews
- No one approach to identification, so consider:
- What strategies will you use and why?
- Which will you not use and why?
- Will you only use published literature or also be searching “grey literature”?
- Practicalities and time
Developing Search Approach
- The importance of an inclusive search
- Search strategy should be methodical and combine all the relevant terms to ensure a thorough search
- See a librarian and learn to use electronic databases
- It will take time and getting to know the literature to develop a good search strategy (that you don’t miss out important ones, but also don’t include some that are too broad or not relevant
- Make sure you record your search strategies for write up
- Two stage process
- After searching for all potential results…
- Stage 1: Screen all titles and (where possible) abstracts
- Compare against inclusion and exclusion criteria
- For some reviews it will be easier to exclude at this stage than others (e.g. a tightly focused review)
- If clearly irrelevant can be rejected. If relevant but fails on one or more criteria, consider recording your reason for rejection
- Be inclusive if in doubt
- Stage 2: Read papers
- For all potentially included papers get the paper and read it in full considering criteria
- Record any decisions that are not straightforward
- Be aware that results may be duplicated in other publications
- Find all the papers you include that you possibly can. Those you cannot, include in your selection diagram.
- Sources for papers:
- e-journals – don’t expect all papers to be available online
- Senate House library – good selection of print and some ejournals
- All other London academic libraries – you can access and photocopy an article
- Inter-library loans – Good service, but they cost money
- Contacting authors – May not respond
Have to document everything within the thesis
- Process by which key findings are extracted from the papers in an unbiased way
- Normally use more than one rater (but not for the thesis)
- Extracted data will vary depending on study. Consider PICOS
- Extracted data should allow you to answer the research question.
- Clarity about data to be extracted will save time
- Consider using a data extraction form:
- Yes/no categories
- Numerical data
- Free text
- Research varies hugely in terms of quality
- Assessment of quality helps with the synthesis of the data and final conclusions
- Definition of quality will vary depending on study
- Range of quality assessment approaches:
- See CRD guidance on quality
- If appropriate, see types of quality assessment used by other bodies (e.g. NICE)
- Look at similar reviews (look at critical appraisal guides)
- Search for quality assessment approaches
- Develop or adapt existing criteria logically (to fit your question)
- Data synthesis refers to the process of combining and summarising the findings of individual studies.
- It follows a planned approach specified in the protocol
- Initial description
- Table of details about the studies (e.g. name, design, sample, main outcomes) – general descriptives of the studies
- Several approaches to synthesis, but it should be systematic
- Unless you are very good with stats, for the dissertation you will be doing a narrative review
- The pilot study is a chance to test the methodology
- For a pilot there needs to be a clear and well thought out methodology
- Each part of the methodology should be tested at this stage. Aspects of the method in need of modification should be documented and discussed. (as part of the pilot study)
Undertaking the Review
- If you have a good protocol, undertaking the review should be quite straightforward.
- Think about your approach and organisation as this will save time. e.g.
- How to keep track of references
- How to keep track of themes for synthesis
- How you will do the synthesis
- Be practical and aware of time. All research has flaws
- Try to avoid changes at this stage. But changes that do occur can be documented and discussed
Can organise data into categories, can use thematic analysis, or clusters
- Presentation of extracted data
- Narrative synthesis
- Developing a theory of how the intervention works, why and for whom
- Developing a preliminary synthesis of findings of included studies
- Exploring relationships within and between studies (what factors might explain differences, e.g. impact of quality of studies)
- Assessing the robustness of the synthesis
Look at review in relation to other reviews… point out differnces or similarities
- Principle findings (direction and effect observed…)
- Meaning of the review findings
- Conclusions, Implications and recommendations
Possibly consider policy makers (can I suggest something)