A. Studies Found in Medical Literature

1. Descriptive Studies
2. Observational Studies
3. Interventional Studies - Clinical Trials
4. Meta-Analysis
5. Need to consider internal and external validity in evaluating trials
6. Internal Validity
   1. Role of chance hypothesis testing and confidence interval
   2. Bias - systematic event in allocation of exposure or outcome
   3. Confounding - are there other uncontrolled features that could be associated with outcome or exposure to explain results
7. External Validity - can the findings be generalizable
8. Levels of Evidence to Support Treatment Decisions [1]
   1. Level 1: randomized clinical trials (or "all or none" data)
   2. Level 2: cohort studies, outcomes research
   3. Level 3: case-control studies
   4. Level 4: case series
   5. Level 5: expert opinions without explicit critical appraisal; opinions based on physiology, bench research, or pathophysiological principles used to direct clinical practice
   6. "All or none" data are data that show all patients died before availability of a certain treatment; after the treatment became available, some or all of the patients survived

B. Descriptive Studies

1. Characterize disease in terms of person, place, and time
2. Useful for alerting medical community, hypothesis generation, resource allocation
3. No comparison group - cannot be used to assess cause and effect relationship
4. Case Report or Case Series
   1. Reports an atypical condition or disease for an individual or series of individuals
   2. May alson include typical condition with atypical presentation(s)
5. Correlational study
   1. Population level data used to associate disease and exposure
   2. People with disease may not be people who were exposed
   3. Subtle relationships may not be apparent (for example, "J" curve-type responses)
6. Cross-Sectional study
   1. Population examined at a defined time to assess exposure and disease status
   2. Useful for inalterable characteristics such as blood type
   3. Does not provide information about duration of disease or exposure
7. Level 4 evidence

C. Observational Studies

1. Case Control
   1. Subjects (cases) selected based on disease status / prevalence
   2. Controls have similar exposures but do not have disease
   3. Good for rare diseases, diseases with long latency, analysis of multiple exposures
   4. Recall bias regarding exposures
   5. Selection bias because disease and exposure occur before study begins
   6. Level 3 evidence
2. Retrospective Cohort
   1. Exposure and disease have already occurred
   2. Subjects selected for exposure to agent being studied
   3. Controls selected to be similar except for exposure being studued
   4. Good for rare exposures with high attributable risk to outcomes
   5. Subject to recall bias regarding degree of exposure and disease
   6. Subject to selection bias because disease and exposure occur before study begins
   7. Reduced selection bias if cohort is predefined
   8. Less resistant to confounders than prospective cohort
   9. Level 2 evidence
3. Prospective Cohort
   1. Exposure has occurred but disease has not
   2. Subjects selected for exposure to agent being studied
   3. Controls selected to be similar except for exposure being studied
   4. Good for evaluating multiple effects of exposure
   5. Subject to losses to follow-up which can undermine validity of study
   6. Subject to ascertainment bias
   7. Expensive and time consuming to conduct
   8. Level 2 evidence
4. Nested Case Control
   1. Case control study within a predefined cohort
   2. Reduces sampling bias but selection bias may occur (if cohort defined later)
   3. Example is Nurses' Health Study case control groups (not originally chosen)
   4. Level 4 evidence

D. Interventional Studies - Clinical Trials

1. Studies in which subjects are allocated to receive 1 of 2 or more of the exposures being investigated, exposed, then observed
2. Can only be performed when exposure being investigated is suspected to be beneficial and not suspected to be harmful (or much more beneficial than harmful)
3. Allocation may be random or systematic
4. Random allocation always preferrable because it controls for both known and unknown confounders if the sample size is large enough
5. Selection bias may be a problem when allocation is systematic
6. Study may be conducted blinded or double blinded
7. Effects of Blinding
   1. Blinding minimizes recall bias from subject but investigator aware of allocation
   2. Double-blinding minimizes both interviewer and recall bias
   3. Double-blinding increases internal validity of trial and is always preferred
8. Considerations in Evaluating Results of an Interventional Trial
   1. Number of subjects in study - enough for results to be statistically significant ?
   2. Number of endpoints achieved - enough to draw conclusions?
   3. Power of study - a function of the magnitude of the exposure's effect and the number of endpoints
   4. Losses to follow-up (did people who left trial share any characteristics ?)
   5. Characteristics of study participants compared to decliners
   6. Characteristics of subjects allocated to exposure compared to those allocated to placebo - especially important when allocation is not randomized
   7. Compliance of exposed group compared to unexposed group
   8. Magnitude of placebo effect - must be subtracted from effect observed for exposure to determine effect attributable to exposure being studied
9. Level 1 evidence

E. Meta-Analysis [1]

1. Attempt to combine results of 2 or more studies looking at a specific disease
2. Used to improve power of studies, particularly in low prevalence diseases
3. Problems with combining data often arise
4. Discrepancies with randomized, prospective trials have been observed [2]
5. However, number of these analyses is increasing
6. The kappa statistic overall has been in the 0.35 range, showing borderline agreement compared with randomized, controlled studies

F. Interpreting Statistics in a Medical Study

(Abbreviated List of Common Tests and Their Uses)
G. Statistics Found Frequently in Medical Literature

1. P (Probability) Value [3]
   1. Indicates the percentage likelihood that the results observed are due to chance alone
   2. By convention, a P value of 0.05 or less is considered significant
   3. Significant means the result is unlikely to be due to chance alone
   4. P values are affected by sample size and magnitude of effect
   5. If a sample is very small, the P value may be >0.05 although the magnitude of the effect is large
   6. If a sample is very large, the P value may <0.05 although the magnitude of the effect is small
   7. Caution must be exercised in interpretation of P values without consideration for scientific and medical issues relevant to a study [3]
2. Confidence Intervals (CI)
   1. Usually a 95% confidence interval, but can be set at higher or lower levels
   2. Means that a range of values within which the result will fall 95 times if an experiment if repeated100 times
   3. Alternately, a range surrounding the result, within which the true result lies with 95% probability
   4. CI are affected by both the sample size and the magnitude of the effect
3. Clinical Efficacy of Therapies [4]
   1. Experimental Event Rate (EER) and Placebo Control Event Rate (CER) are measured
   2. Relative Risk Reduction (RRR) = EER-CER /CER is effect of intervention (I) relative to control (C) rate ( is absolute value)
   3. Absolute Risk Reduction (ARR) = EER-CER for interventions which reduce bad outcomes
   4. Absolute Benefit Increase (ABI) = EER-CER for interventions which show benefits
   5. Number Needed to Treat (NNT) to have one (1) additional good outcome = 1/ABI
   6. In general, ARR and NNT are the most helpful in assessing the clinical efficacy
4. Kappa Statistic
   1. Measures agreement beyond that due to chance alone
   2. Value <0.40 considered fair-to-slight agreement
   3. Maximal value is 1.0
5. Likelihood Ratios (LR) [5,6]
   1. LR is ratio of probability of test result among patients with target disorder to probability of that same test result among patients without the disorder
   2. Positive LR (LR for positive test) is calculated as sensitivity/(1-specificity)
   3. Negative LR (LR for neative test) is calculated as (1-sensitivity)/specificity
   4. LR has also been called the Bayes Factor
6. Probabilities and Odds Ratios
   1. Pretest probability is prevalance of disease: proportion of patients who have the target disorder before the test is carried out
   2. Post-test probability is proportion of patients with that particular test result who have the target disorder
   3. Pretest odds: odds that the patient has the target disorder before test is carried out, calculated as pretest probability/(1-pretest probability)
   4. Post-test odds: odds that the patient has the target disorder after the test is carried out, calculated as pretest odds x LR

---

References

1. Dodson TB, Caruso PA, Nielsen GP. 2004. NEJM. 350(3):267 (Case Record)
2. LeLorier J, Gregoire G, Benhaddad A, et al. 1997. NEJM. 337(8):536
3. Borzak S and Ridker PM. 1995. Ann Intern Med. 123(11):873
4. Goodman SN. 1999. Ann Intern Med. 130(12):995
5. Sackett DL and Haynes RB. 1997. ACP Journal Club. 127(1):A15
6. Goodman SN. 1999. Ann Intern Med. 130(12):1005
7. Katz MH. 2003. Ann Intern Med. 138(8):644