Please enable JavaScript.

Coggle requires JavaScript to display documents.

Mary Rider - Coggle Diagram

- - - - Standardised questionnaires or interviews used to collect data on preferences, attitudes, beliefs, behaviours, and factual information
      - Suitable for descriptive, exploratory, and explanatory research
      - Often uses individuals as units of analysis, but can use key informants for organisations or groups
    - - Measures unobservable constructs (e.g., political orientation, self-esteem, attitudes toward policies)
      - Efficient for large, geographically dispersed populations via postal, email, or phone surveys
      - Unobtrusive and convenient—respondents can answer at their own pace
      - Enables large samples that detect small effects and support subgroup comparisons
      - Economical in researcher time, effort, and cost compared with experiments and case research
    - - Vulnerable to non-response bias, sampling bias, social desirability bias, recall bias, and common method bias
      - Key challenge: keeping gap small between “true attitudes” and answers given (response bias; Tversky & Kahneman, 1973)
    - - Postal (mail) questionnaires
        
        Advantages: inexpensive, unobtrusive, wide geographic reach
        
        Disadvantages: very low response rates, long delays, risk of lost surveys
        
        Requires follow-up reminders and careful response tracking
      - Group-administered questionnaires
        
        Respondents gathered in a room, complete survey independently
        
        High response rates, opportunity for clarification
        
        Limited to accessible groups (e.g., a workplace or class)
      - Online / web surveys
        
        Advantages: extremely low cost, automated data capture, easy modification
        
        Disadvantages: sampling bias (overrepresentation of younger, connected populations)
        
        Challenges estimating response rate if link posted broadly rather than emailed
        
        Dual-media (postal + online) can improve coverage and choice for respondents
    - - Personal (face-to-face) interviews
        
        Rich data, opportunity for probing and clarification
        
        Higher response rates if interviewer is skilled
        
        Costly and time-consuming; potential interviewer effects
      - Group interviews / focus groups
        
        6–10 respondents interviewed together with a facilitator
        
        Useful for exploring complex issues and triggering new ideas
        
        Risk of dominance by strong personalities; reluctance on sensitive topics
        
        Better for exploratory than for descriptive/explanatory research
      - Telephone and CATI interviews
        
        Phone contact plus computer-assisted scripts and recording
        
        Random digit dialing improves sample selection
        
        Not ideal where phone density is low or where visuals are needed
    - - Preparation
        
        Understand the study purpose, sampling, and potential interviewer bias
        
        Rehearse the interview guide and timing
      - Locating and Enlisting Cooperation
        
        Work around respondent schedules; “sell” the importance of the study
      - Motivating Respondents
        
        Show enthusiasm, respect, and attentiveness
        
        Maintain neutrality while encouraging full responses
      - Clarifying and Probing
        
        Address confusion, objections, and ask follow-up questions as needed
      - Observing Response Quality
        
        Note body language and inconsistencies for later interpretation
    - - Pre-interview
        
        Prepare kit: cover letter, instruments, ID, contact phone number
        
        When possible, schedule appointments in advance
      - Beginning the Interview
        
        Confident, clear opening; brief explanation of study and confidentiality
        
        Avoid jargon; seek permission for audio recording
      - During the Interview
        
        Ask questions exactly as written, in order
        
        Use probing techniques
        
        Silent probe (pause)
        
        Overt encouragement (“okay,” “uh-huh” without approval/disapproval)
        
        Ask for elaboration (“Can you tell me more?”)
        
        Reflection (“What I’m hearing is…” then pause)
      - After the Interview
        
        Thank participant sincerely
        
        Immediately record notes and observations
    - - Structured vs unstructured questions
        
        Structured: fixed response options, easy to code and compare
        
        Unstructured: open-ended, rich qualitative data, harder to analyse
      - Dichotomous responses
        
        Two options (yes/no; true/false; agree/disagree)
      - Nominal responses
        
        Multiple unordered categories (industry type, race/ethnicity, etc.)
      - Ordinal responses
        
        Ordered categories (education level, frequency categories)
      - Interval / scale responses
        
        Likert scales, semantic differential, Guttman scales (approximate equal intervals)
      - Continuous responses
        
        Fill-in numeric values (age, income, years in job), ratio-scaled
    - - Use simple, active-voice language
      - Avoid jargon unless targeting specialised professionals
      - Maintain consistent style and reading level throughout the instrument
    - - Negative wording and double negatives
        
        Example: “Should your local government not raise taxes?”
        
        Causes confusion and inaccurate responses
      - Ambiguity
        
        Vague or multi-interpretation terms (e.g., “income”, “big,” “any”)
      - Biased or value-laden language (Rasinski, 1989)
        
        “Assistance to the poor” vs “welfare”
        
        “Halting rising crime” vs “law enforcement”
        
        Subtle wording changes can shift support dramatically
      - Double-barrelled questions
        
        Combine two concepts in one question (hardware AND software satisfaction)
        
        Must be split into two separate items
      - Overly general vs overly detailed questions
        
        General: “How big is your firm?” → better: employee count, annual revenue
        
        Detailed: only ask additional detail if needed for research purpose
      - Presumptuous questions
        
        Assume agreement or benefits (“What do you see as the benefits of a tax cut?”)
      - Imaginary / unrealistic questions
        
        “If you won a million dollars, how would you spend it?”
        
        Yield random, non-scientific answers
      - Knowledge assumptions
        
        Asking respondents about information they are unlikely to know (e.g., CEO on daily operations, high-schoolers on Bay of Pigs foreign policy)
    - - General rules
        
        Start with easy, non-threatening items
        
        Move from factual/behavioural to attitudinal questions
        
        Progress from general to specific topics
        
        Use transitions when changing topics
        
        Use filter/contingency questions (skip logic) where appropriate
      - Video guidance integration
        
        Ask open-ended questions before related closed-ended ones to avoid priming
    - - Respect respondent time – keep surveys as short as possible
      - Use clear, legible fonts and consistent formatting
      - Provide clear instructions and examples when needed
      - Avoid too many different question types (cognitive load)
      - Assure confidentiality and explain use of data (academic, organisational, etc.)
      - Promise to share aggregate results when feasible—and follow through
    - - Desk check
        
        Researcher completes the survey as if a respondent
        
        Identify confusing wording, ordering problems, and missing items
      - Pilot survey
        
        Trial with small group similar to target population
        
        Observe difficulties, ask for feedback on clarity and length
        
        Check whether responses align with research goals
      - Iterative process
        
        Write → Critically evaluate → Pilot test → Revise → Re-test if needed
        
        Aim: minimise response bias and improve validity and reliability
    - - Non-response bias
        
        Low response rates (15–20% typical for postal surveys)
        
        Non-respondents may systematically differ (e.g., dissatisfied customers more vocal)
        
        Strategies: advance notification, relevance, short and friendly questionnaires, endorsements, follow-ups, incentives
      - Sampling bias
        
        Coverage problems with phone, online, or literacy-based sampling frames
        
        Wrong population sampled (e.g., parents vs students on learning outcomes)
        
        Hurts generalisability and external validity
      - Social desirability bias
        
        Respondents avoid embarrassing or negative admissions about self or others
        
        Especially strong for sensitive topics (illegal downloads, office politics)
        
        Hard to eliminate in questionnaires; interviewer judgment helps in interviews
      - Recall bias
        
        Inaccurate memory of past events, behaviours, or motivations
        
        Mitigation: anchor questions to specific, recent events where possible
      - Common method bias (Podsakoff et al., 2003; Lindell & Whitney, 2001)
        
        Spurious covariance when IVs and DVs measured at same time with same method
        
        Diagnostics: Harman’s single-factor test, marker variable technique
        
        Remedies: longitudinal designs, multi-method measurement
    - - Informed consent and voluntary participation
      - Avoiding harm, especially with sensitive topics
      - Data protection and secure storage
      - Transparency regarding sponsorship and funding sources
  - - - Research design where one or more independent variables (treatments) are manipulated
      - Subjects randomly assigned to treatment and control groups
      - Outcomes (dependent variables) compared across groups
      - Strength: high internal validity, enabling causal inference
    - - Observational studies
        
        Measure variables as they naturally occur
        
        Show associations/correlations but not causal relationships
        
        Vulnerable to lurking variables (e.g., IQ, motivation, sleep)
      - Experiments
        
        Apply treatments and provoke responses
        
        Aim to control or balance lurking variables via design
        
        Provide stronger evidence for causation when well-designed
    - - Experimental units and subjects
        
        Experimental unit: entity being treated (person, animal, object)
        
        Subject: experimental unit that is human
      - Treatment group
        
        Receives experimental stimulus or intervention
      - Control group
        
        Does not receive the treatment or receives placebo/standard condition
      - Explanatory variables (factors) and levels
        
        Factor: independent variable manipulated (e.g., brushing time, instructional type)
        
        Factor levels: specific values/categories of a factor (e.g., 30 seconds vs 2 minutes)
      - Treatment combinations
        
        In factorial designs, treatment = combination of factor levels applied to a unit
      - Response variable
        
        Measured outcome (e.g., oral health, test scores, attitudes)
    - - Correlation ≠ causation
        
        Example: study time and GPA positively correlated but not purely causal
      - Lurking variables
        
        Hidden variables (IQ, motivation, stress, sleep) that influence both X and Y
      - Experiments attempt to neutralise or balance lurking variables via random assignment and control
    - - Randomization
        
        Randomly assign experimental units to treatments
        
        Helps prevent bias and distribute lurking variables across groups
      - Replication (repetition)
        
        Apply each treatment to many units to reduce random variation
        
        Increases precision and reliability of treatment effect estimates
      - Control
        
        Compare two or more treatments (including control or baseline)
        
        Control for confounds by design (e.g., same setting, same instructions)
    - - Laboratory experiments
        
        Artificial settings, high control, strong internal validity
        
        Lower external validity due to lack of realism
      - Field experiments
        
        Real-world settings (organisations, schools, communities)
        
        Higher external validity; more difficult to control extraneous variables
    - - True experimental designs
        
        Require treatment manipulation and random assignment
        
        Strongest internal validity
      - Quasi-experimental designs
        
        Treatment manipulation without random assignment
        
        More selection-related threats to internal validity
      - Non-experimental designs
        
        No manipulation or random assignment (e.g., surveys, correlational studies)
    - - Random selection
        
        Sampling mechanism; each population unit has a positive chance of selection
        
        Primarily affects external validity (generalisation)
      - Random assignment
        
        Design mechanism; randomly allocate sampled units to groups
        
        Primarily affects internal validity (causal inference)
    - - History
        
        Outside events influence outcomes (e.g., exam preparation)
      - Maturation
        
        Natural changes over time independent of treatment
      - Testing
        
        Pretest affects posttest responses (practice, memory)
      - Instrumentation
        
        Changes in measurement tools or difficulty between pretest and posttest
      - Mortality (attrition)
        
        Differential dropout between groups, especially of low performers
      - Regression to the mean
        
        Extreme scores tend to move closer to average at retest
    - - Pretest–Posttest Control Group Design
        
        Notation: R O1 X O2 / R O3 – O4
        
        Treatment effect: E = (O2 – O1) – (O4 – O3)
        
        Controls many threats (maturation, testing, regression) by symmetry between groups
      - Posttest-Only Control Group Design
        
        Notation: R X O1 / R – O2
        
        Treatment effect: E = O1 – O2
        
        Avoids pretest sensitisation; still controls key threats via random assignment
      - Covariance Designs (ANCOVA)
        
        Include covariates measured at pretest (C) to control extraneous variance
        
        Notation: R C X O1 / R C – O2
        
        Analysed using analysis of covariance
    - - Factorial designs
        
        Manipulate 2+ factors simultaneously (e.g., 2×2, 2×3, 2×2×2)
        
        Each cell = unique combination of factor levels = treatment
      - Sample size rule of thumb
        
        Aim for ≈20 units per cell (Cohen’s medium effect power guidance)
      - Main effects
        
        Overall effect of one factor averaged across levels of other factors
      - Interaction effects
        
        Effect of one factor depends on level of another factor
        
        When interactions are significant, interpretation of main effects becomes limited
    - - Randomised block design
        
        Subjects divided into homogeneous blocks (e.g., students vs professionals)
        
        Within each block: random assignment to treatment/control
        
        Reduces noise due to block differences
      - Solomon four-group design
        
        Combines pretest–posttest and posttest-only structures
        
        Tests for pretest sensitisation effects on outcomes
      - Switched replication design
        
        Two-phase implementation with crossover of treatment and control roles
        
        By the end, all participants receive treatment
        
        Common in organisational program evaluation
    - - Non-equivalent Groups Design (NEGD)
        
        Pretest–posttest structure without random assignment (N instead of R)
        
        Vulnerable to selection–history, selection–maturation, selection–regression threats
      - Regression discontinuity design
        
        Assignment to treatment/control based on cut-off score on preprogram measure
        
        Ethically attractive when resources go to those most in need
        
        Internal validity depends on careful modelling of the cut-off
      - Proxy pretest design
        
        Pretest measured retrospectively or via archival data after treatment has begun
      - Separate pretest–posttest samples design
        
        Different samples at pretest and posttest in each group
        
        Allows some quasi-experimental inference without repeated measures
      - Non-equivalent dependent variable (NEDV) design
        
        Single group with two DVs: one expected to change with treatment, one not
        
        Control DV helps identify extraneous influences
        
        Pattern-matching NEDV compares observed vs theoretical patterns across multiple DVs
    - - Atheoretical experimentation
        
        Hypotheses not grounded in theory → ad hoc and hard to interpret
      - Weak measurement
        
        Instruments not tested for reliability/validity; reduces comparability across studies
      - Inappropriate designs and missing controls
        
        No consideration of interactions, covariates, or relevant control conditions
      - Inappropriate tasks and subject populations
        
        Unrealistic tasks (e.g., undergraduates role-playing expert managers)
        
        Threaten internal and external validity simultaneously
    - - Treatments are the core of experiments and must be carefully constructed
      - Use prevalidated tasks when possible
      - Conduct manipulation checks to see if treatment had intended psychological effect
      - Pilot test tasks repeatedly to refine clarity and difficulty
      - Prefer simple, familiar tasks over complex, unfamiliar ones when in doubt
    - - Experiments best for testing causal mechanisms suggested by theory
      - Can complement survey research (Chapter 9) by:
        
        Using survey results to identify candidate causal relationships
        
        Following up with experiments to test those relationships more rigorously
      - Non-experimental, quasi-experimental, and experimental designs form a continuum of control vs realism