Week 2 - Module 2 (Lec 6) (Internal validity ( Non-causal explanations …
Week 2 - Module 2 (Lec 6)
Approach to appraising evidence
Describe the evidence presented in the study
Assess the internal validity of the study
Assess the external validity of the study
Compare the results of the study with other available evidence
What relationship was being evaluated and what
hypothesis was being tested?
What were the exposure variable(s) and what was the
What was the study design?
– case study, case series
– clinical trial
– case-control study
– prospective or retrospective cohort study
– cross sectional study
Definition of the subjects that were studied in terms of:
– source populations
– time frames
– eligibility criteria
Summary of the main result
– what is the result in terms of the association between exposure
– should be possible to express the main result in a simple table
and obtain from the paper the means to calculate the appropriate
measure of association
describes the truthfulness of inferences about the study population (i.e. those that took part in the study)
– for the subjects who were studied, does the evidence support a
causal relationship between the exposure and the outcome or
just an association?
Five aspects of causal explanations should be
Is there a correct temporal relationship?
Is the relationship strong?
Is there a dose-response relationship?
Consistency of the association?
Specificity of association
The order of these non-causal explanations is important
– if there is severe bias, no analytical manipulation of the data will
overcome the problem
– if there is confounding, then appropriate analysis will (in most
cases) overcome the problem
– assessment of chance variation should be made on the main
result of the study, after considering issues of bias and
Non-causal mechanisms which could produce the
– chance variation
– bias in ascertainment (Selection)
• surveillance, diagnosis, referral, selection, non-response, length of
stay, survival bias
– bias in the estimation of exposure (Misclassification)
• recall, interviewer, prevarification, improper analysis
Confounding (the formal definition)
– the effect of an extraneous variable that wholly or partially
accounts for the apparent effect of the study exposure, or masks
an underlying true association
A variable is a confounder if:
It is causally associated with the outcome; and
It is noncausally associated with the exposure; and
It and the exposure variable are on two separate causal
The strength of the association between the exposure and
outcome changes when you account for it
– was a relationship between exposure and outcome identified by
– type I error: null hypothesis is rejected when, in reality, it is true
– if we perform 20 statistical tests using an alpha level of 0.05, we
will make a type I error on one occasion
can the results be applied to
populations other than that which was studied?
Three aspects of external validity should be considered:
Can the results be applied to the eligible population?
Can the results be applied to the source population?
Can the results be applied to other relevant populations?
Comparison with other evidence
Three aspects of comparison should be considered:
Are the results consistent with other evidence?
Are the results plausible biologically?
Is there coherency with the distribution of the exposure and the