Reliability & Validity
Reliability
Definition
degree to which a research instrument produces consistent results
instrument is another word for survey
Examples of Reliability
EX1: Guy at carnival tries to guess your weight
you go to him for that week and guesses you weigh 110 pounds
next week you come and guess he's 130 pounds
the guess isn't consistent nor reliable
EX2: Bathroom Scale
when you step you weigh 120 pounds
couple of days later you weigh 120 pounds again
more consistent then EX1
not possible to exactly calculate the reliability or consistency of a research instrument
but we can estimate it in a few different ways
1st: Test Retest Reliability
when the same instrument is given to a group of people at two different times
results from time 1 are compared to results from time 2
this is in order to determine how well the instrument consistently gets the same results
2nd: Parallel Forms Reliability
when a group of people complete 2 similar versions of an instrument
each version of the instrument is trying to measure the same thing
then the results from the 2 versions are compared in order to determine the consistency of the results between the similar versions of an instrument
3rd: Internal Consistency Reliability
when different instrument items that are trying to measure the same construct are compared to see how they produce similar results
item is another word for question on the survey
1st Type: Average Inter-Item Correlation
when a group of people complete an instrument and then all of the items on that instrument are measuring the same construct are compared to each other
2nd Type: Split-Half Reliability
then the items are compared overall to create an average of those comparisons
a group of people complete an instrument and then all of the items on the instrument that are measuring the same construct are split in half to form 2 "sets" of items
then the 2 sets of items are compared to each other to see how well they consistently measure the construct
4th: Inter-Rater Reliability
when the decisions from different raters are compared to each other to see how consistent the rater's decision's really are
Rater are researchers who rate or judge a variable in the research study
Example of Validity
Validity
Definition
how accurate an instrument is a measuring what it is trying to measure
we can determine the accuracy of an object in different ways
EX1: Weight Example
you hop on the bathroom scale everyday every it says you weigh 120 pounds
lets say the scale is off by 50 pounds because you actually weigh 170 pounds
even if the scale is reliable it may not be valid or accurate in measuring
1st: Face Validity
this is when researchers simply look at the items on the research instrument and give their opinion if the items appear to accurately measure what they are trying to measure
least scientific way of all measures of validity
just the researcher's opinion if the items look valid or not
2nd: Construct Validity
when we are able to generalize about our construct of interest because we are accurately measuring that construct
Think of construct validity as being truthful in how we are labeling our construct
Examples of the Different Types of Validity
Example of Construct Validity
EX1: Love of Coffee
all instrument items are enjoying the smell of coffee
but are we really measuring the love of coffee...no
3rd: Criterion-Related Validity
when the results from the instrument accurately relates/predicts some kind of external variable
Example of Criterion Related Validity
EX1: SAT Scores
designed to measure literary skills, writing skills, and how test takers analyze or solve problems
they have the validity when they compare it to an external variable thats related to it like College GPA's
those who score well tend too get good grades in college
4th: Content Validity
how well an instrument covers the range of meanings included within a concept that is being measured
Example of Content Validity
EX1: Love of Coffee
we need to cover the wide range of meanings and indicators regarding the love of coffee
include items not just about enjoying the smell of coffee but also enjoying the taste or the effects of coffee
we can only generalize or talk about our findings
🚩 Example of Reliability
🚩 Examples of Validity
🚩 Example of Construct Validity
🚩 Example of Criterion Related Validity
🚩 Example of Content Validity