Rigour:

• being extremely thorough & accurate
  
  • enhances believability, trustworthiness and quality of the research
  • required to ensure the study findings are transferable (qualitative) and generalizable (quantitative)
  • tries to get rid of confounding variables, deal with bias, use different designs to get quality results
  • TERMS: (similar terms for the different paradigms or methodology)
    
    • Qualitative (credibility = true value; transferability = applicability, "fittingness"; dependability = consistency; confirmability = neutrality, audit-ability)
    • Quantitative (internal validity; external validity/generalizability; reliability; objectivity)
      

Credibility: How accurately have you recorded the findings?

• most important one
• to increase probability of high credibility (prolonged engagement, prolonged observation, triangulation of sources, methods, investigators); peer debriefing (talk to experts in the field about the results)
• negative case analysis (look at outliers and try to understand them)
• member checks (reconfirm with what participants are telling you)
• always look at credentials; minimize bias, researcher reflectivity
  

Transferability:

• extent to which findings can be applied to other situations (but not generalizable)
• need to make context very clear to increase transferability
• Detail every step of the study !!!
• readers interpretation and application to their own life - does it have relevance for them?
• compare and contrast

Dependability:

• examination of process by which study was done (external auditor)
• ensuring results come from study participants and not from characteristics or preferences of the researcher
  acknowledges limitations so people are aware
• diagrams of audit trails (keep good results and stay organized)

Confirmability (auditability)

• audit trails (details of research design, implementation, operational details; implementation of study etc)
• are the theories that are developed coming from the data or from researchers reflections on data?
• reflective appraisals (memos, field notes, researcher journal) --> want other researchers to be able to follow your way of thinking and a way to get rid of bias

Reliability Coefficients:

• indicate magnitude of relationships between variables
• range (0 = very high error, no reliability) // (1 = no error, high reliability)
• for a tool to be considered reliable, a level of 0.70 or higher should be reported
• a good article will tell you what reliability and validity is

Measurement Tool:

• whole thing together (ex. survey or questionnaire)
• you have scales within instrument (may have 1 or multiple scales)
• scales are groups of items asking about the same thing (most range rom 6-8 items)

Stability:

• extent to which scores are similar on 2 separate administrations of an instrument
• evaluated by:
  
  • test-retest reliability (focus on this one as more common; giving the test twice)
  • parallel form reliability (requires participants to complete same instrument or parallel form on 2 occasions; correlation coefficient computed between the 2 scores; appropriate for relatively enduing attributes - ex. self-esteem)

Internal Consistency (Homogeneity):
- extent to which all the instrument's items are measuring the same attribute (are we sure they are testing the same and proper thing that we are wanting to test?)

• evaluated by administering instrument on one occasion; appropriate for multi-item instruments
• Methods:
  
  • Item-to-Total Correlation: statistical test to create a matrix to see how correlated each item is to the total (want correlated between 0.3-0.8; balancing act to get them correlated but each item is getting at a slightly different aspect; want different approaches to your construct)
  • Split-Half Reliability: splitting all the items in the scale in half and comparing the halves – are the scores for each half no matter how you split them about the same? (no matter how you split them)
  • Kuder-Richarson Coefficient: used when questions have a dichotomous response (dichotomous is one or the other); ex. yes/no; true/false; reliability scores usually above 0.70
  • Cronbach's alpha: compare each item in the scale to other items in the scale – correlation matrix; comparing in a scale to other items and see if correlated; good alpha is 0.7-0.9 (0.9 could be either be excellent or indicate duplication of items)
    
    
    
    

Equivalence:

• the degree of similarity between alternative forms of an instrument or between multiple raters/observers using an instrument (want to be sure we are getting the same score no matter the process)
• most relevant for structured observations
• assessed by comparing observations or ratings of 2 or more observers (inter-observer/inter-rater reliability)
• numerous formula and assessment methods
  

Validity:

• the degree to which an instrument measures what it is supposed to measure
• are we measuring what we are believing we are measuring
• 4 Aspects:
  
  • Face Validity: have a group similar to your participants use the measure and get their feedback (did they think it made sense? were they confused?)
  • Content Validity: want to make sure that the instrument being made is correct and the content it is giving is correct
  • Criterion-related Validity: agreement between responses on measure and actual behaviour (does the person actually do what they said they do on the measure?)
    
    • concurrent = two different measures at the same time (same construct) and trying to understand if measurements have the same constructs
    • predictive = measure it now and at a later time and see if you are getting the same thing
  • Construct Validity: How well the tool measure the theory behind it? Does the tool validate the body of evidence underlying the tool development?
    
    • 1. Hypothesis testing (a little more complex)
    • 2. Convergent Validity (use an already validated tool that measures the same construct and give both to participants)
    • 3. Divergent Validity (try to see if you constructs are different than other similar constructs; use another measure that measures something similar but different and see if you get similar or different responses); if we are getting at the same thing then something is wrong as that is not what we are wanting to get at
    • 4. Contrasted Group Approach (give tool to 2 groups and see if responses match expectations)
    • 5. Factor Analysis (statistical way to measure the construct validity)