Survey evaluation / Ethical issues
Two components design
what to measure
designing and testing
-
systematic review
cognitive interview
put questions into survey
drafting questions
pretesting
focus group
Defining objectives
what should be measured for those accomplishments (variables)
analysis plan
paragraph: purpose/what to accomplish (excludes unnecessary questions)
sample design
IV, DV, CV
reality vs abstract concepts
refine objective (what do ppl know )
ppl from pop
perception, experience, feelings
use list of issues to evaluate
goals
issues: voluntary, paid, laboratory, only few
comprehension + vocabulary
think aloud and ask follow up questions
don't provide: evaluation of specific wording + how difficult it was
consistently understand and answer
needed info available
answer describe what is meant
answers valid measure
make easy as possible
finding problems
what are problems tho?
clear instructions + questions?
how does it go in reallife
how long does it take
criteria length:
- cost
- response rate effects
- ability and willingness
if survey too long:
cut questions
subsets to subsample
Informing respondents
purpose
confidentiality
sponsorship
voluntaring
name organization
skip any question
Signed consent
limits of confidentiality
access to info such as medical report
very sensitive: embarrassment if public
Protecting respondents
how given info is treated
minimize breach of confidentiality
life's shouldn't be affected
names deleted
not accessable non project members
no link respondent - answer
identify by answers, no access
researchers sign for confidentiality
data is encrypted
Responsibilities to interviewer
give them full and accurate info for respondents
safety and fear of crime
debugging computer assisted instrument
proofreading printout of program
testing it
those 2 not adequate for long instrument and complex questions
therefore: at beginning of survey: check distribution of answers to check whether contiguency instructings work as intended