Please enable JavaScript.

Coggle requires JavaScript to display documents.

Scholarly practice, Research Design notation; ROX - slide 29 in L4,…

- - - - Reflection in action – reflecting in the moment, thinking while doing,
    - - The OT process (our habits of thinking and acting) is shaped by the contexts (cultural, social, institutional) in which it is embedded.
    - - Presence to experience; One can develop one’s ability to be present in the moment
        
        Experience is at the heart of inquiry - when you experience something, you act upon it/do something with it and then suffer the consequences. 2 elements
        
        interaction between person and environment
        
        Through interaction with the world, we both change it and are changed by it (example: interacting with ideas)
        
        Experience may be what happens to you, but what you do with what happens to you is directly dependent on thoughtful reflection and the meaning you make of it --> meaning gives experience value
        
        continuity - experience is linked to the past AND the future; you carry with you what you learn in one situation to the next
      - Description of experience/problem (implies holding at bay spontaneous interpretations until analysis, where they can be more closely examined in light of the ‘data’ gathered
      - Analysis of experience; formulating hypotheses possible explanations for the question; spending enough time with the data. - thinking thru the consequences (requires imagination)
      - experimentation: acting and undergoing the consequences; intelligent action
    - - wholeheartedness - open-mindedness - responsibility
        
        One can think reflectively only when one is willing to endure suspense and undergo the trouble of searching
        
        Not linear
- - - - our own biases
      - research
      - expectations
      - personal experiences
      - environment
      - assumptions
    - - 3 types of theories... in notes
  - - - Implementation and evaluation go together (thoughtful reflection on beliefs which requires present to experience)
  - - - double loop - takes us from action to consequences to our problem in the first place - question assumptions (deeper level of learning)
      - white arrow - represnts single-route learning - what actions we took - does not lead to reimagination of what the need might be
- - - - emancipatory
      - Purpose? design logic? contributions?
        
        unsettles our beliefs in the givenness of things - can lead to transformative action
        
        unveiling hidden coercion, systematically distorted perceptions; as creature of habit, we question little and critical research really shakes this up
        
        not what they appear and what we believe is true/natural may not be the case
        
        Critical research tell us unintended consequences about beliefs
    - - cultural/ practical/ quallitative/ intepretive
        
        approach of choice to understand the complexities of being human
        - the study of meaning and meaning-making, and historically conditioned and culturally variable patterns of social interaction.
        
        qualitative reasonates with OT philsophy - L7 pg 6
      - applied ethnography
        
        qualitative methods; phenomenology, grounded theory, narrative, etc
      - Purpose? design logic? contributions?
        
        research supplies tools for understanding new concepts/ideas/theories, which enables us to coordinate our actions with others
        
        understanding identity, lived experience, social processes
        
        multivocal - 'many voices'
        
        having or open to many different meanings, interpretations, or applications.
        
        many contradictions and different perspectives
        
        sometimes the problem is already defined, but generally we have to create a view of the problem (define proble situation)
        
        supply new concepts or lens for understanding
        
        have to understand purposes/intentions of the person
    - - technical/ experimental/ instrumental/ analytical/ empirical
      - experimental research
        
        RCTs
        
        Explanatory vs. Pragmatic RCTs
        
        Explanatory trials = whether a treatment has any efficacy (often compared to placebo) under ideal, experimental conditions.
        
        placebo controlled; blinded; homogenous group of patients; high internal validity, low external
        
        Pragmatic trials = effectiveness of a therapy or treatment (usually compared with care-as-usual) in routine, everyday practice.; real world effects
        
        active comparison; not blinded; heterogeneous group of patients; high external validity but low internal
        
        start with a story and then go to data
        
        LIMITATIONS:
        
        Generalizability (external validity) from RCT is problematic; (see pg 34 L4 for more details)
        
        RCT serves a specific purpose but is not the end all be all
        
        Many questions about cause-effect relationships are not amenable (cannot be answered) to the RCT design: limited applicability
        
        RCTs show comparative efficacy for an average randomised patient and not for pertinent sub-groups: treatment effects are almost always heterogeneous. (gives us an average, but most people aren’t the average)
        
        RCTs often compare a new intervention with a passive (no intervention/ placebo) control condition --> What practitioners most want to know is whether the new intervention is more effective than what they currently do.
        
        The short and long term adverse effects of a new treatment or intervention are often overlooked.
        
        the researcher assigns conditions, or manipulates exposure to some hypothesised cause (e.g., new treatment)
        
        (1) two or more conditions (experimental vs control group) (2) Random assignmnt of subjects (3) observation/ measurement (4) between group analysis (compare outcomes)
        
        Goal: causal inference = outcome is attributable to the exposure (REMEMBER CORRELATION NOT CAUSATION)
        
        3 conditions need to be
        
        Covariation/correlation: variation in exposure (i.e., independent variable) is associated with variation in outcome (or dependent variable)
        
        Precedence: The exposure (or hypothesised cause) comes before the ‘outcome’.
        
        1 more item...
        
        Plausible alternative explanations for the relationship between exposure and ‘outcome’ can be ruled out.
        
        2 more items...
        
        random assignment
        
        a method for assigning participants to the different conditions --> The main purpose of random assignment is to rule out plausible alternative explanations (e.g., selection threat / pre-existing group differences).
        
        if subjects aren't blind, there are theats to in internal validity
        
        Diffusion of treatment: This occurs when subjects in different groups communicate with each other and learn information not intended for them.
        
        Compensatory equalization: This occurs when those providing the treatment attempt to give some of the advantages the treatment group has to members of the control group.
        
        Compensatory rivalry: This occurs when members of the control group try to gain some of the benefits of the treatment group.
        
        Resentful demoralization: This refers to control group subjects underperforming because they resent being denied the treatment.
        
        Blinding
        
        If not done can lead to ascertainment bias/ blinding rules this out
        
        Occurs when the results or conclusions of a trial are systematically distorted by knowledge of which intervention each participant is receiving. Ascertainment bias could be introduced by the assessor, the person receiving the intervention, or the investigator analysing outcomes.
        
        Also occurs when some members of a population are more likely selecred in a sample than others
        
        1 more item...
        
        Leads to threat of external validity
      - observational studies
        
        Cohort studies
        
        looking forward in time; studied over time
        
        Studied at regular intervals – survery and interview data
        
        ex. pg 42 in L4
        
        , information is collected about the outcome of interest and exposure to hypothesised ‘predictors’ (risk & protective factors), but in cohort studies subjects are followed over time.
        
        Can tell us about mediating and moderating variables
        
        mediating: mediating variable explains the relation between the predictor (independent variable) and the criterion (dependent variable). A mediator can be a potential mechanism by which an IV can produce changes on a DV.
        ex. rumination makes depression worse
        
        moderator: moderator is a variable (e.g., age, sex, SES) that affects the direction and/or strength of the relation between an independent/ predictor variable and a dependent/ criterion variable. ex. social support in anxiety
        
        Obervational studies are categorized based on time – may be retrospective, cross-sectional (snapshot in time), looking prospective (forward in time )
        
        want to find what the story is so they start with the data
        
        cross sectional study
        
        carried out at one time point (eg survey) = s snapshot in time
        
        Purpose: to estimate the prevalence of some condition or variable of interest in a given population and to identify potential ‘risk’ factors.
        
        limitations: they are carried out at one time point and give no indication of the sequence of events (and plausible explanations)
        
        case-control study
        
        retrospective/ thinking backwards
        
        compares patients who have a disease (cases) with patients who do not have the disease (controls), and looks back retrospectively to compare how frequently the exposure to a risk factor is present in each group to determine the relationship between the risk factor and the disease. ex. 44 L4
        
        no less interested in prediction, or cause and effect, and are necessary when conditions, such as exposure to some kind of traumatic event, cannot be assigned. can't manipulate exposure!
      - Purpose? design logic? contributions?
        
        knowledge of means, strategies and techniques (help us to discriminate more helpful and less helpful tools and stratgeies)
        
        just cause there is no evidence, it doesnt mean its ineffective; its all about learning the theory behind it --> some techniques have little evidence but are used
        
        design the experience to be had, under ideal conditions, the research is made without certainty but confidence. what they experience was consequent not just subsequent
        
        literature is full of statistically significant but trivial effects… so not always meaningful
        
        Analytical involves plausibility – predication is made and then tested – start with a story and then go to data
        
        identifying invariant, universal laws, cause-effect = prediction, manipulation, control, testing theory
        
        knowledge of ‘what works’, which may guide us in the intelligent selection of possible interventions or strategies of action.
        
        analytical is limited in that the hypothesis/ outcome has to be known in advance and it has to be measurable
        
        Practice should be based on the best available evidence, and the best or most trustworthy evidence comes from controlled clinical observations (e.g., RCT).
      - circle of methods
      - RCTs arent enough, rather information from multiple sources ( case studies, ethnography, observations, etc) to determine causal relationships
    - - 5 steps
        
        Ask - formulate clinical question
        
        PICO
        
        Patient/problem (e.g.,adults with osteoarthritis) Intervention (e.g.,hydrotherapy) Comparison/ control (e.g., care-as-usual). Outcome (e.g.functional independence)
        
        Acquire - search for evidence
        
        identifying appropriate databases, and revising our search terms - identifying key concepts and terms - combine search terms with Boolean operators - review and refine search results
        
        Appraise - being critical
        
        carefully and systematically assessing the outcome of scientific research (evidence) to judge its trustworthiness, value and relevance
        
        Only 'claims' can be valid (not methods). The purpose of critical appraisal is to assess the validity of the claims that are made (are the claims warranted?), based on the appraisal of the research or evidence upon which the claims are based.
        
        Rate the quality of the evidence using evidence hierarchy
        
        systematic reviews vs CAT vs meta analysis vs evidence-based guidelines
        
        2 more items...
        
        looks at the way a study is conducted (is there another plausible explination, is the reported effect size clinically significant, are the findings generaliable)
        
        Apply - apply evidence to practice
        
        integrate! - Applying 'best evidence' is not straightforward. Practitioners must integrate multiple sources of 'data', including research evidence, clinical expertise and patient preferences in decision-making
        
        Assess - evaluating evidence
        
        observe outcomes --> analyze/reflect on outcomes (was the result what was intended?) --> revise definitions of the problem-situation and devise new actions/activities
      - the conscientious use of current best evidence in making decisions about patient care
      - 3 principles
        
        Practice should be based on the best available evidence, and the best or most trustworthy evidence comes from controlled clinical observations (e.g., RCT).
        
        Science is cumulative: claims about ‘what works’ should be based on systematic reviews that summarize the best available evidence.
        
        Evidence is necessary but not sufficient for decision- making: Clinical decision-making requires consideration of patients’ values and preferences.
      - Transdisciplinary model of rehab
        
        turning research into contextualized evidence-based practices. Considers environmental context
      - Issues with EBP
        
        The nature of research-based knowledge
        
        Research knowledge usually takes the form of generalizations and applying these to particular situations is rarely straightforward.
        
        what works is not the same as what matters!!!
        
        The nature of professional practice
        
        OT practice involves more than selecting a line of action from a smorgasbord of so called evidence-based options to treat clinical problems.
        
        The authoritarian nature of EBP
        
        the conceptual distinction between thinking and doing
        which increased a power distinction (in old time doers where slaves...) --> creates dogma/stigma
      - John Dewey - see Lecture 2 Q card for more detail
- - - - the population parameter we want to estimate is the ‘true effect’ of some intervention (which is unknowable).
        
        the population is theoretical; it refers to the complete set of ‘effects’ observed over countless repeated trials.
        
        RCT produces a measure of effect which is a measure of the true effect
  - - - Sampling error - Pg 10 L5
        
        error caused by observing a sample instead of the whole population. it arises from estimating a population characteristic by looking at only one proportion of the population
        
        difference between the sample statistic and the "true" paramater
        
        No sample perfectly represents the whole population
        
        Sampling error is unavoidable
        
        Note that sampling error is theoretical, because we don’t actually know what the true population parameter is.
    - - backbone of inferential stats
      - central limit theorem
        
        the distribution of sample means approximates a normal distribution as the sample size gets larger/ more times an experiment is done. AKA falls close to the true population mean (although sometimes it strays quite far by chance)
      - the frequency distribution of a statistic (such as a mean) over many random samples drawn from a single population.
  - - - use the SE to construct a confidence interval
      - If SE is large than it is far from population parameter and if SE is small than it is probably pretty close and our estimate is probably pretty good
  - - - null vs alternative hypothesis the hypothesis that you support (your prediction) the alternative hypothesis, and we call the hypothesis that describes the remaining possible outcomes the null hypothesis.
      - History of P value - pg 38 L5. Fisher: P value was used to see if an experiment was worth repeating
        Neyman&Pearson: inductive behaviour - pg 42
        
        It was created to be a rough numerical guide of the strength of evidence against the null. For R.A. Fischer, the operational meaning of a small p-value was merely that one should repeat the experiment.
      - 0.0001 would be must stronger than 0.05
        
        As probability of p gets smaller, the possibility of chance creating difference becomes less possible. Determine the plausibility of results could have just happened by chance!
        
        If the observed effect size is large enough to be meaningful, then a large p-value should only lead you to the conclusion that the sample was too small, i.e., the study was ‘under-powered’.
        
        When you observe a large p-value, all you can do is reserve judgment.
        
        cannot assume that the treatment made no different if large P value
        - P value is confounded by sample size
      - P DEPENDS ON: Observed affect and size of the sample (the larger the sample, the smaller P)
  - - - clinically vs statistically signficant
        
        A statistically significant result is one that is unlikely to be the result of chance. clinically means i is important in the real world
        
        It is quite possible for a result to be statistically significant and trivial in real world
        
        It is also possible for a result to be statistically non- significant and important.
      - Risk Ratio
        
        measure of the risk/ the ratio of a probability of a certain event happening in one group compared to the risk of the same event happening in another group (dichotomous)
        
        Titantic example
      - if not dichotomous/ if outcome is numerical rather than categorical
        
        Raw mean difference
        
        Posttest mean for treatment group – posttest mean for control group
        
        doesn't tell us a lot
        
        Standardised effect size/ standardized mean difference
        
        Cohen’s d, Hedges g
        
        Converts unit of measurement into standard deviations
        
        calculate using online calculator
  - - - A way of quantifying uncertainty/imprecision
        
        A 95% confidence interval is a range of values either side of the estimate between which we can be 95% confident that the true value lies
        
        Only if the confidence interval excludes values that could be deemed clinically significant (ie effect size) would it be reasonable to conclude that a study has demonstrated no effect.
        
        When no difference falls within CI, sometimes conclude that it had no effect which isn’t the best thing to do. Need to repeat the study if this happens in a larger sample!
        
        https://select-statistics.co.uk/calculators/confidence-interval-calculator-odds-ratio/