Please enable JavaScript.

Coggle requires JavaScript to display documents.

研究法期末考, image image, image - Coggle Diagram

- - - - Have a standard set of questions to ask each participant or group. Because interviews and focus groups tend to be conversational,
        it is easy to vary the topics of discussion. While researchers want to tailor follow-up questions to participants responses and contributions, this lack of consistency makes comparisons with other participants difficult.
      - Make sure questions facilitate discussions. Yes or no questions should be avoided. Structured questions are beneficial, but should be preceded by semi-structured or unstructured questions.
      - Like with questionnaires, avoid making assumptions.
      - Also avoid expressing your own views. This can be very difficult to do when engaging someone in a conversation.
        However, remember that, as a researcher, your goal is to gather information from your participants. This requires active (attentive and responsive) listening.
      - Like with questionnaires, avoid leading questions. Rather than starting questions with “Do you think we should …” start with “What do you think about …
- - - - data must be first-handed
      - retrospectively not introspectively.
      - self-report methods, such as interviews, questionnaires or focus groups.
      - focus on perceptions of the experience rather than the event itself or facts.
      - participants are expected to share common experiences
      - Epoche:
        refrain from bias
        
        horizontalization:
        give same credits to all the data
        
        imaginative variation
        
        Bracketing:
        define and develop the theme well in an educated position
        (the researcher would have to do the questionnaire at first to be evaluated from a third party to decrease the bias.)
        
        structural description
  - - - Fewer data collection restrictions than other qualitative methods. You can still use interviews, questionnaires or focus groups; however, archival records, such as books, paintings, manuscripts and pottery may also be useful in interpreting signs and symbols.
      - Participants (or other sources) must use metaphors, similes or symbolism in language
      - Your focus should be on asking “what does that word or gesture mean?”
- - - - Parametric statistics:
        normal distribution
        
        t-tests, ANOVAs, Pearson correlation, simple and multiple regressions.
      - Non-parametric statistics
        
        Spearman correlation
        
        Mann-Whitney U
        
        Chi Square:
        同一樣本中，兩個變項(自變量與依變量)的關聯性檢定，是否為相互獨立，或者是有相依的關係存在，若是達到顯著，則需進一步查看兩個變項的關連性強度。
        https://www.yongxi-stat.com/chi-squared-test-of-independence/
        
        Kruskall-Wallis.
    - - meaningfully significanteffect is one that may not have a high level of confidence, but the improvement seen in an experiment is substantial for that particular field of study.
        (which is normally determined by the effect size)
      - absolute difference is when there is any
        difference between treatment groups within a sample
      - statistically significant
    - - Count data: is another scale of measurement that is not as neatly classified into either continuous or categorical scales.
      - Continuous:Interval, Ratio values
      - Categorical or Discrete:
        Nominal values have no quantitative properties, but may be represented as numbers
      - Distribution Moments:
        Distribution, uniform, binomial, and Poisson.
        Moment, mean, variance, skewness, and kurtosis.
- - - - naturalistic observation: 完全自然界之觀察
      - contrived observational: 創造研究環境
    - - The advantages
      - Data are often easy to obtain
      - Researchers can often get large sample sizes
      - Sometimes we don’t know a phenomenon exists until evidence
      - accumulates over time (i.e. school violence, grade inflation)
      - The disadvantages of conducting archival research include:
      - Types of information are limited. Because you are relying on other people to gather information, the data collectors may not collect specific information that you need.
      - Information may be incomplete or missing. Records may be lost over time or record keeping may be inconsistent as people or procedures for collecting information changes.
      - You may not know the method of data collection. If you do not have a detailed description of how the information was collected, you may not be able to make certain claims. For example, if I do not know whether or not all the participants took several abilities tests in the same order I will not know if the reason the scores on the Wechsler Adult Intelligence Scale are higher than those from the Woodcock-Johnson because one test is easier than the other or because one test served as a practice for the other.
  - - - Advantages
      - They can be qualitative and/or quantitative measures
      - They can be used with a variety of designs
      - They can test causal, correlational, and descriptive hypotheses
      - Disadvantages
      - Most of them rely on self-report data
      - It can be difficult to get random samples from them
      - Many are not tested for validity or reliability
      - Considerations for Collecting Data with Surveys
        Cross sectional studies are those that collect all of the data on a single occasion.
        
        Available resources
        
        Type of population
        
        Knowledge about the research question
- - - - Comparison
        Evaluation vs researcher
        
        Generalizability of results
        
        Standards
        
        Who sets the agenda?
        
        Preparation
        
        Purpose
    - - Internal vs. External
        An internal/external evaluation is one that is conducted by an evaluator who is also/not a stakeholder.
        The disadvantage is that an internal evaluator is likely to be biased in favor of or against the program and may be less likely to investigate what he or she doesn’t know.
        An external evaluator is less likely to be biased in her report. However, she may have a harder time establishing a rapport with stakeholder and getting honest appraisals of the program.
      - Needs Assessment, Process & Outcome
        Assess and make recommendations for reducing the problems. The recommendation is usually a plan or is used to make a plan for a new program.
        A process (or monitoring) evaluation determines how well the program is being delivered.
        Outcome evaluations are concerned with determining how the program changes program recipients or the community.
        反正就是一個計劃分成初始、過程、結果，三階段來做評估。
      - Formative vs. Summative
        Formative evaluations improve program performance by influencing immediate decisions about the program or part of it.
        Summative evaluations judge program worth by assessing program effects in light of relevant problems. This is a stricter form of evaluation and less popular among stakeholders.
      - Formal vs. Informal:
        pretest, posttest,
        Data for this type of evaluation are often collected using surveys.
  - - - Data-driven:
        approach to research attempts to let the data “speak for themselves.” This follows a Skinnerian approach to research – that is to simply, but accurately, describe outcomes as they are without bias. Rather than focusing on whether cognitive or behavioral patterns follow a particular theory, our job is to observe without speculating or making assumptions.
      - Theory-driven:
        A theory-driven approach to research seeks to extend, support or refute existing research. This follows a Bayesian approach to research methods – that is to start with what we already know. To make solid educated predictions, we must be experts in what we are predicting
    - - Steps
        
        • Aims or goals of a study –
        
        • Research question or hypothesis.
        
        • Description of the educational setting and participants
        
        • Time period the study will cover
        
        • How data will be collected and in what form
        
        • A description of the intervention.
        
        • What is the dependent variable and how it will be measured.
        
        • A description of the research design clearly explaining what is being compared and how these comparisons will be made.
- - - - Selecting Studies
        Studies must have some similar characteristics to make them comparable. Researchers specify criteria for inclusioninto the meta-analysis to control this balance between heterogeneity of utos and homogeneity of UTOS.
      - Meta-Analysis Coding:
        Coders gathering information from single studies may interpret results differently. Using multiple coders to evaluate the same studies and compare their ratings (using interrrater agreement) is one way to improve estimates.
      - Effect Sizes:
        the way how it is measured is like t-test
        Common effect sizes include correlations, Cohen’s d, eta squared, rsquared, and an odds ratio.
      - Limitations:
        One common problem is that some researchers may compare different constructs.
        how the experiments are implemented may vary from one to another.
        Another concern is the file drawer problem in which many single studies that do not find significant treatment effects are not published.
  - - - Baseline, AB, ABA, ABAB, washout periods
        using when:
        studying a rare or unusual phenomenon
        trying to get detailed information
        you can make many observations over time
        testing new, experimental treatments
        the treatments need to be customized for each participant
        focusing on individual differences
        it is not necessary to generalize to broad populations
- - - - Threats to Statistical Conclusion Validity:how well researchers can conclude that cause and effect variables are statistically related
        解釋事實的效力
        https://webcourses.ucf.edu/courses/1346335/pages/statistical-conclusion-validity?module_item_id=13053250
        
        Violation of Statistical Tests: when statistical results are not as usual one as we see.
        
        Fishing: repeated research on the same population/subjects may result in statistical possibility due to chances instead of factual difference.
        
        Low Statistical Power: when the sample size, effect size or alpha level is not large enough
        
        effect size: how effective the intervention/treatment is, as compared to control group.
        it's manifested as a degree.
        
        Alpha level: how the Type I or II　error is defined
        
        Reliability of Measurement: tools or the way to measure is not reliable or not true (as compared to the fact)
        
        Reliability of Treatment Implementation: interventions are not standardized and reliable
        
        Restriction of Range: when the data is not collected with wide range so that the research cannot truly reflect the desired effect
        
        Heterogeneity of Units: subjects in the group are so different that statistics are meaningless, e.g., personality between super rich guys and poor guys.
        
        Inaccurate Effect Size Estimation: Inaccurate effect size estimation is when the statistic used to analyze the data tends to over- or underestimate treatment effects. This most often occurs when the sample size is extremely high or low, as many statistics are sample size dependent – that is they are easily influenced by the sample size.
        
        Extraneous Variance in the Experimental Situation: Extraneous interventions that are not predictable/explanable
    - - Interaction of the Causal Relationship with Units
        Especially when cultural influences apply.
      - Interaction of the Causal Relationship with Settings:
        Interaction of the causal relationship with settings is when the results of a study cannot be generalized to another setting.
      - Interaction of the Causal Relationship with Outcomes:Interaction of the causal relationship with outcomes is when the same results of a study cannot be found if the researchers use a different operation or measure of the outcome or dependent variable.
      - Context-Dependent Mediation:
        the causal relationship will differ because of the presence or absence of the mediators.
      - Interaction of the Causal Relationship over Treatment Variations: similar treatment cannot be directly applied to each other. Most of the time they must be implemented together.
    - - Inadequate Explication of Constructs
        definition of a research construct is not clear
      - Mono-Operation Bias: only one source of participants and instrument are used for operation
      - Mono-Method Bias: similar to the above but different in terms of method/approach
      - Confounding Constructs with Levels of Constructs: When an inference is made about a construct based on a restricted range of levels of that construct (i.e. only high or low values of performance)
        (受測範圍(依變量彼此接近)過小，以致無法得出適當結論)
      - Treatment Sensitive Factorial Structure
      - Reactive Self-Report Changes
      - Experimenter Expectancies
      - Reactivity to the Experimental Situation
        When participants respond differently under experimental conditions because they are aware that they are being evaluated
      - Novelty and Disruption Effects
        Novelty and disruption effects is when participants respond especially well or poorly to a new intervention, simply because it is different from their usual routine.
      - Compensatory Equalization: Compensatory equalization is when the treatment, program or intervention provides desirable services and an administrator or staff member provides similar services to compensate the control group.
      - Compensatory Rivalry
        Compensatory rivalry is when the treatment, program or intervention provides desirable services and the control attempts to find a similar treatment or services.
      - Resentful Demoralization: Resentful demoralization is when the control group resents not receiving a desirable treatment, program or intervention and responds more negatively on outcomes or dependent variables.
      - Treatment Diffusion
        Treatment diffusion is when the control group learns about the program or intervention from the treatment group and also begins to use it.
      - Hypothesis guessing: the testees change the answer in order to support or refute the hypothesis
    - - Threats to Internal Validity
        
        Ambiguous Temporal Precedence: 因果倒置/不明
        
        Selection: Selection most commonly occurs when it is not feasible or ethical to randomly assign participants to treatment conditions. (不當選擇受測者)
        
        History: there are multiple influencers (normally an extraneous one) simultaneously as intended interventions
        
        Regression: when the testees are tested in the extreme period, scores would regress as they go back to normal period.
        
        Maturation: natural development occurs at the same time.
        
        Attrition: participants no longer stayed in the research.
        
        Testing: when the testees "learned" the test by receiving multiple tests, the effect may change
        
        Instrumentation: measurement tools are replaced and the effect is mistakenly achieved.
        
        Additive and Interactive Threats to Internal Validity: abovementioned validity measures are compound.
        
        Evaluation apprehension:
        When participants try to provide more favorable responses because they are anxious about being evaluated