Please enable JavaScript.
Coggle requires JavaScript to display documents.
PSY 390 STUDY UNIT 2 Research Operations (Chapter 5: Measurement Issues…
PSY 390 STUDY UNIT 2
Research Operations
Chapter 4: Problem Definition and Literature Review
FORMULATING RESEARCH PROBLEM AND HYPOTHESES
Sources/motivation of Research Ideas or Topics
There are many sources of research ideas and topics.
Casual observation
You may identify a research idea by doing casual observation.
i.e. Seeing beggars on the street may trigger your interest in studying poverty in Singapore
Personal experience or commitment
i.e. Being a mature student, you may be interested to learn more about the academic aspirations and stress of part-time university undergraduates like yourself
Previous research
may also inspire you to fill the
knowledge gap.
Cross-cultural studies frequently originate from previous research findings
Testing Claims
You may also think of a research problem to test a claim made by someone.
For example, a researcher may want to compare the rational-logical ability between males and females to verify a teacher’s claim that girls are ‘idiots’ in mathematics.
Qualitative vs Quantitative research topic
Qualitative Research
topics of qualitative research may appear rather
general and loosely structured.
Qualitative used to
formulate theory thereafter
Quantitative research
usually requires a
concrete research topic
to be clearly set at the beginning of the research process.
Topics are usually
precisely and specifically defined
.
-
Hypothese
s are well formulated
before the study
is carried out.
Formulation of Research Problems
Research problem
A research problem is a
specific topic
that you want to investigate.
The aim of formulating a research problem is to clearly state the specific issue
to be studied
The two major activities in formulating research problems are:
Selecting Research Problems
Narrowing Research Problems
Selecting Research Problems
This requires extensive information gathering through lit.review
However, in order to research these questions realistically, you need to
narrow your research topic or problems
to a researchable and manageable issue.
A
good research problem is limited in scope
. It should be bound to certain times, places, and a social context.
Narrowing Research Problems
This can be done by asking basic questions:
What are the main elements in the research problem?
The process goes on until you select some aspects that you want to study.
i.e. Suppose you limit the problem by:
specifying
the drug-taking phenomenon by the types of drug preferred
limiting
adolescents to those studying secondary schools in Singapore.
basic and specific questions
Basic question
These questions usually take the form of why, what, how, when, where, who.
They help
limit the scope of research
i.e. Why are the drugs taken?
specific questions
specific questions are related and within the scope of the basic questions, but they ask more
specific details about the basic elements
of the research questions than basic questions do.
i.e. Do any family factors contribute to drug-taking?
Is drug taking a reaction to excessive pressure from school?
Writing Research Hypotheses
5 Characteristics of Casual Hypotheses
Can be expressed as a prediction or an expected future
outcome.
Are
logically
linked to a research question and a theory.
Express a causal or cause-effect
relationship
between the variables.
Are
falsifiable
; that is, they are capable of being tested against empirical evidence and shown to be true or false.
Have at least
two variables.
5 Potential Errors in Causal Explanation
Ecological fallacy
The empirical observations are at too high a level for the causal relationship that is stated.
i.e. New York has a high crime rate. Joan lives in New York. Therefore, she probably stole my watch.
Spuriousness
An unseen third variable is the actual cause of both the independent and dependent variable.
Hair length is associated with TV programs. People with short hair prefer watching football; people with long hair prefer romance stories. (Unseen: Gender)
Teleology
The cause is an intention that is inappropriate, or it has misplaced temporal order.
i.e. People get married in religious ceremonies because society wants them to.
Reductionism
The empirical observations are at too low a level for the causal relationship that is stated.
Because Steven lost his job and did not buy a new car, the country entered a long economic recession.
Tautology
The relationship is true by definition and involves circular reasoning.
i.e. Poverty is caused by having very little money.
Categories of research question
The following are some categories of research questions
Comparative questions
focus on the
similarities and differences
of the variables or their stability over time. (why)
Causal
relationship questions investigate whether one variable causes another
Descriptive questions
describe the properties of entities or phenomena according to obvious qualitative and quantitative characteristics such as shape, colour, size, etc.
Associative questions
examine whether and
how
variables are associated.
LITERATURE REVIEW
Purposes of the Literature Review
The major aim of a literature review is to
establsih what is known
about a topic.
There are three main reasons for conducting a literature review:
There are
three main reasons
for conducting a literature review:
Avoiding duplication of effort in research activities.
Other researchers may have already done something that answers your research problems.
If you are satisfied with the available information, then there is no point in putting extra effort in conducting similar research.
If you are not satisfied with previous research, reviewing the literature informs you about possible gaps in research wherein you may put your research effort.
Avoiding conceptual and procedural mistakes.
You may find a researcher’s own evaluation of his or her work at the end of the research report.
Researchers may also criticize and defend one another’s work in separate articles.
You can
learn much from mistakes made in previous research
, and you should try every means to avoid these same mistakes in your own research work.
Placing the current research into a scientific perspective.
The literature may inform you about
how your research
may
add to the current knowledge
base.
Selecting Information for Literature Review
Here are some useful criteria to help you select the best information from the sources that you locate.
Relevance
This concerns with the extent to which pieces of information relates to your research problem
Importance
Comprehensibility
Importance
i.e. impact on social science literature?
Comprehensibility
i.e. can the material be easily understand, especially when the research topic is totally new to you
recency/ Date of Publication
Research material should reflect the current state of thinking on the concerned topic
Identify
the most recognized/
influential authors
in the field.
Their publications are usually more inspiring and informative than others.
credible information,
(i.e. based or refrencing classical studies) which helps you conduct good research.
Assembling and Summarizing Information
To help you
organize
a vast amount of information, you need to
summarize
relevant information and
store
it in an easily retrievable form. This include:
catagologing research material (i.e. name, publicatiton)
Organizing and Using Information
An important aspect of a literature review is to
critically examine, organize, synthesize
and then use the selected relevant literature
To do this you need to:
Skim
through reports to gain an overview,
and decide
whether information is relevant and in what ways it is important to the research project.
Read critically.
This requires you to have some familiarity of the subject matter (as you read more about a particular topic, you will be more familiar with it) and some basic knowledge about research principles and methods (which you will gain from this course).
Chapter 5: Measurement Issues & Research Instruments
What is measurement
Measurement is the
means
that researchers use to qualitatively or quantitaively
classify observations
into units
Quantitative and Qualitative Measurements
Quantitative measurement
involves primarily numerals and mathematical concepts.
It aims to provide a
precise
and simple measurement that allows for easy checking of accuracy and relevancy
Qualitative measurement
is names, labels, and categories. It aims to provide insights and
detailed analysis
of the origin, process, and meaning of social phenomena.
Measurement Decisions
What to measure:
This ensures that what you measure is
relevant and answerable
to your research
problem
how to measure.
There may be more than one way of measuring a particular phenomenon.
This ensures that you select the mode that
best suites
your measure
CONSTRUCTS AND OPERATIONAL DEFINITION
Constructs
Constructs are hypothetical
attributes
or mechanisms that help to explain and
predict behaviour
in a theory.
Attitudes and personality are examples of constructs that are theoretically important, but are
not directly observable concepts
, and therefore difficult to measure.
(i.e. intelligence, emotions)
Operational Definition
An operational definition is a procedure for indirectly measuring and
defining a variable
that cannot be observed or measured directly.
An operational definition
specifies a measurement procedure
(a set of operations) for measuring an external, observable behavior and uses the resulting measurements as a definition and a measurement of the hypothetical construct.
Issue with operational definition
The primary limitation of an operational definition is that
there is not a one-to-one relationship between the variable that is being measured and the actual measurements
produced by the operational definition.
Consider for example, the familiar situation of an instructor evaluating the students in a class. In this situation, the underlying variable is knowledge or mastery of subject matter, and the instructor’s goal is to obtain a measure of knowledge for each student. However, knowledge is a construct that cannot be directly observed or measured. Therefore, instructors typically give students a task (such as an exam, an essay, or a set of problems), and then measure how well students perform the task
The indirect connection between the variable and the measurements can result in
two general problems
Second, operational definitions often
include extra components
that are not part of the construct being measured.
For example, a self-report of depression in a clinical interview or on a questionnaire is influenced by the participant’s verbal skills (ability to understand questions and express feelings and thoughts) as well as the participant’s willingness to reveal personal feelings or behaviors that might be perceived as odd or undesirable. A participant who is able and willing to describe personal symptoms may appear to be more depressed than someone who withholds or conceals information.
First, it is easy for operational definitions to
leave out important components
of a construct.
for example, we can define depression in terms of behavioural symptoms (social withdrawal, insomnia, etc.). However, behaviour represents only a part of the total construct. Depression includes cognitive and emotional components that are not included in a totally behavioural definition.
One way to reduce this problem is to
include two or more different procedures
to measure the same variable.
Chap 5: Levels of Measurement
It is important to specify the level of measurement to
determine
:
Types of data collected
How we organise data
Types of statistical analysis that can
be undertaken
Levels of measurement can
be organized in two ways:
Continuous vs. discrete variables
The four levels of measurement
(NOIR)
Continuous and Discrete Variables
Continuous variables
Measured
on a continuum
with an infinite number of divisions (i.e.
unspecified/ can be subdivided
).
Examples: time, temperature, age
Example of age as a continuous variable: 25 years, 10 months, 2 days, 5 hours, 4 seconds, 4 milliseconds, 8 nanoseconds, 99 picosends…and so on.
You
could
turn age into a discrete variable and then you could count it. For example:
• A person’s age in years.
Discrete variables
a.k.a. Categorical variables
Measured in terms of
distinct, separate
and mutually exclusive categories, which are often finite.
Examples: Nationality, race, religion, gender, marital status
Comparing both variables
Continuous variables can be reconceptualised as discrete variables but not the reverse.
i.e., age can be reconceptualised as age groups but not the other way round. (will invalidate the study/results)
Measuring continuous variables as discrete is discouraged.
Detailed information about the construct is lost during reconceptualization from more precise levels of measurement to less precise levels of measurement.
Reduces reliability and validity of measurement.
More practical to define and measure continuous variables
Possibility to reconceptualise into discrete data later during data analysis/discussion phase.
The four levels of measurement
(NOIR)
Discrete
More hone in on Quali
Nominal
Nominal measurement involves the
classification
of events/information into categories The categories should be:
Distinct:
The categories are different from one another in at least one attribute.
Mutually exclusive
: Each should belong to only one category.
Exhaustive:
There must be an appropriate category for each case.
Example
Sex: Male/Female
Ethnicity: Chinese/Malay/Indian
Accuracy: Correct/Incorrect
Ordinal
Ordinal measurement involves the ordering of data and the
ranking of variables
in a
continuum according to their magnitude
Classfies and orders
the variables
Example
Degree of friendliness: How Friendly are you? – Not at all/somewhat/average/very/extremely
Education level etc
Gradings of assignment
Income level
Continuous
More on Quanti
Interval
Classifies, orders and specifies
the degree of
difference
between each
interval
on an equivalent scale (from low interval to high interval)
Example
Temperature: i.e. range of 94C and 96C
(however, note that as far as interval measurement is concerned, 94C and 96C is the same as 100C and 102C).
Ratio
Classifies, orders, tell us the exact value between units, AND they also
have an absolute zero
–which allows for a wide range of both descriptive and inferential statistics to be applied.
These variables can be meaningfully added, subtracted, multiplied, divided (ratios).
Example
Income: $0 - $100000
Years of work experience: 0 – 50 years
scores
Income
Chap 5: Reliability and validity
Reliability
Reliability refers to the
degree of consistency
.
A reliable measurement
means is a measurement that
is consistent
when repeated under identical or very similar conditions.
It does not vary due to the process of the measurement or the instrument used to measure the construct.
Types of Reliability
Stability reliability
Representative reliability
Equivalence reliability
Stability reliability
This refers to
reliability across time.
This means that measurements remain consistent over a period of time
Test-retest method.
A measure’s degree of stability reliability can be verified using the test-retest method with the same group of people.
Test-retest method: measuring results from the same group of people multiple times across different time periods
Example
an IQ test with a high degree of stability reliability means that you should get the
same scores if you give the same test
(or a very similar test to the original) to the same group of people
each time
.
Representative reliability
This refers to
reliability across different subpopulations
or groups such as class, race, gender, or age group.
This means that the measurement of a construct delivers the same result when used in different groups.
Subpopulation analysis.
Representative reliability can be verified using a subpopulation analysis.
This compares the measure across different groups and gathers independent information to verify the results of the measure
Example
a Singaporean researcher wants to test the degree of representative reliability of his IQ test.
He conducts a subpopulation analysis to see whether his IQ test works equally well across the different races: Chinese, Malay, and Indian.
He then
obtains independent measures
(e.g. give a universally standardised IQ test)
and verifies
whether the performance in his IQ test are equal for the Chinese, Malay, and Indian groups, as they should in a universally standardised IQ test.
His IQ test would have a
high degree of representative reliability if the three groups score similarly in his IQ test.
Equivalence reliability
This refers to the
reliability across different measures of the same construct
.
This means that the different measurements of a construct deliver the same result.
Split-half method
The degree of equivalence reliability across different measures of the same construct can be verified using a split-half method.
This randomly divides or
splits the different measures
of the same construct into two parts
and determines whether both
groups
give the same results.
Example
the researcher wants to test the equivalence reliability of the different test questions that are all supposed to measure IQ.
Using the split-half method, he
randomly divides the 50 test questions into two groups of 25
test questions.
If his test questions have equivalence reliability,
his participants should score the same
on the first group of 25 test questions as with the second group of 25 test questions.
We can calculate a special statistical measure such as Cronbach’s alpha to determine the degree of equivalence reliability across different measures or tests of the same construct.
Inter-rater reliability
Equivalence reliability can also be applied when researchers use different observers, raters, or coders of information in their research.
In this case,
each observer/rater/coder is a “measure”
. Researchers would want to
ensure that there is equivalence reliability
across the different observers/raters/coders,
i.e.,
they agree with one another.
This is called inter-rater reliability or inter-coder reliability. The degree of inter-rater reliability can be verified by having several raters measure the exact same thing independently and then comparing the results across the raters.
Example
A researcher, who is interested in studying fluency in speech across different contexts, can have four different research assistants watch the same video of a speaker and rate the fluency of the speaker.
If all four ratings are the same, the researcher can be confident that there is high inter-rater reliability of his measure of fluency.
Validity
Validity is defined as
how well a measure correctly measures
the construct of interest (i.e.
how relevant
are the measurement results). (accuracy)
For example, an IQ test might be a valid measure of an individual’s general intelligence but an invalid measure of an individual’s personality
Types of Quantitative Validity
Face validity
Content validity
Criterion validity (CP)
Construct validty (CD)
Face validity
This refers to the
use of face value
(Cursory/superficial results) to ascertain whether the measure really measures the construct.
i.e Does an indicator
makes sense as a measure
of a construct
Example
a researcher who is looking to administer an IQ test, may look through the test items and ensure that the items have face validity, i.e., they really test the participant’s general intelligence, such as those that ask a participant to solve puzzles.
He would exclude items that do not have face validity, such as those that ask a participant about his family background.
Content validity
This refers to how well a measure captures all aspects of the definition of a construct,
i.e.,
Does an indicator represent all aspects of the definition
of a construct
Improving Content validity
Just like how we can use multiple measures to improve the reliability of our measurement, the use of multiple measures can also improve the content validity of our measurement.
By combining multiple measures
of the construct that we are interested in, we will be able to capture as much of the definition of our construct, thus we can get a more
accurate and holistic measure of our construct
Example
A researcher, who is interested in studying the interactions between family members, may define interactions between family members as family members communicating with one another using traditional modes of communication, such as conversation, and contemporary modes of communication, such as texting and social media.
He may create a measure of interactions between family members via two survey questions
1) How often does your family discuss financial issues together?
2) Does your family eat dinner together?
This measure has
low content validity as the two questions only capture two aspects of the content specified in his definition
of interactions among family members. They ignore interactions via contemporary modes of communication as well as other aspects of traditional modes of communication. To improve the content validity of this measure, the researcher must either make his construct definition more precise and narrow or include more measures.
Criterion validity
This refers to the
use of a well-established standard
(criterion)
to assess
how well a measure is measuring the construct.
i.e.
Does an indicator rely on some independent (standard)
, outside verification
to assess
how well it is measuring a construct?
Concurrent validity
refers to
how well our measure agrees with pre-existing measures
of the same construct and whose validity have already been well-established either by ourselves or other researchers.
Example
A researcher, who wants to create an intelligence test suitable for Singaporeans, will want to ensure that his intelligence test has concurrent validity,
i.e., participants’
performance
on his intelligence test should be
similar to their performance on a standardised IQ test
.
Predictive validity
refers to
how well a measure predicts future behaviour
that is related to the construct it is measuring.
Example
In Singapore, students in junior colleges take the A-level examinations as a measure of their academic aptitude, i.e., their ability to perform in university.
If the
A-level examinations
have a
high degree of predictive validity
, students who do well for the A-level examinations
should also do well in university
.
Otherwise, the A-level examinations have a low degree of predictive validity if students who do well for the A-level examinations end up performing poorly in university or similarly with students who do poorly for the A-level examinations
Construct validity
This refers to
how consistent multiple measures of the same construct are performing
.
Multiple measures that have a low degree of construct validity should not be combined into a single measure of the same construct
Convergent validity
refers to the idea that multiple measures of the
same construct should produce similar results
Example
A researcher, who is interested in measuring the construct “academic success”, can use multiple measures instead of just relying on “O-level academic performance”, such as “level of education”, “performance in school tests”, and others.
If these measures converge
, i.e., people who performed well in O-levels examinations have a high level of education and performed well in their school tests, then this combined measure of “academic success”
has convergent validity.
Otherwise, this combined measure has weak convergent validity and the multiple measures should not be combined into a single measure.
Discriminant validity
It refers to the idea that
measures of different or opposing constructs
should produce different or
opposing results.
Example
A researcher, who is interested in measuring friendliness, may include survey questions that directly measure friendliness, such as “Do you make friends easily?” and “How likely are you to go up and talk to a stranger?”
For this combined measure to have
discriminant validity
,
responses
to these questions should be
negatively associated
with responses to questions that ask the opposite, such as “How shy are you?” and “Do you only have a small group of friends?”
Validity in Qualitative Research
The following are some criteria suggested as measures of validity in qualitative research:
Presentation of evidence.
There should be
enough raw data presented
in the research report that allows the reader to evaluate the interpretation that is being made.
The raw data should be clearly
distinguished from the interpretation
in the report.
Independent audit.
An
independent researcher
(who plays no part in the project) is employed to
check the credibility of the final report
in data collected. It is also to make sure that there is a logical progression that runs through the chain of evidence
Example
(e.g. interview questions, audiotapes, transcripts, coding and classifications) all of which should have been clearly and systematically filed for checking.
Internal consistency
.
Researchers have to make sure that the
research
is internally consistent and coherent.
They should
represent a coherent argument
, making interpretations that are warranted by the data presented.
(minimise ambiguity)
If there is any ambiguity, then it should be dealt with in a coherent and ordered manner.
Triangulation.
By
using more than one method of data collection or source of information
, it is believed that a more accurate picture of the examined social phenomenon can be obtained.
(Triangulation is a navigation concept that refers to the notion of locating the position of an object from
two different locations in order to increase the accuracy of its position.
)
Example
For instance, a researcher who studies gambling behaviour may conduct separate interviews with gamblers and their family members.
He may also obtain further information by actually participate in gambling activities with the gamblers (i.e. acting as a participant observer).
With all these data at hand, the researcher can be more confident about the description and interpretation of the gambling experiences.
Member validation.
This is to take the analysis of responses back to the research participants (the ‘members’) and ask them to check or comment on the interpretation. If the research aims to understand participants’ own perceptions about different things and situations, it would be sensible to ask them to check the interpretation about their own account about themselves.
Chap 5: TYPES OF RESEARCH INSTRUMENTS
Questionnaires
It is a research instrument which seeks to
acquire data through standardised questionnaire items.
Researchers usually ask the same set of questions to all research participants (except for minor modifications according to the research design and characteristics of different target groups).
Respondents typically give written responses to standardized questionnaire items. This means that their structure and content is fixed before data collection so that all respondents are exposed to the same set of stimuli (the questions).
Differences in their responses can, therefore, be attributed to the differences between respondents rather than to the differences in the process (e.g. the wording of the question) that produces the answer.
Developing a Question Plan
This involve drafting the questions for a questionaire.
consists of three main steps
List relevant broad categories
.
Outline
important variables to be investigated in the research, which
enables you to visualize
the general format of the
questionnaire.
Develop questions relevant to the outlined categories
.
Devise specific questions that
measure the concerned variables
as prescribed by the broad categories.
Identify specific objectives
. Identify specific purposes related to the research problem.
This is important for
determining relevant variables
to be measured in the research.
Formulation of Questions: Techniques and Strategies
Having decided on what questions to ask, you need to concentrate on how to ask
them.
Open-ended vs. closed-ended questions
Open-ended questions
do not provide options for respondents to choose from but allow respondents to give their own answers.
They are
useful in obtaining unanticipated information
from the respondents and understanding the respondent’s own opinions.
However, answers to open-ended questions are usually
difficult to compare and interpret.
Researchers have to break down the answer into smaller parts and code them in systematic and useful ways to facilitate data analysis. This could be
more time-consuming
and taps heavily on researchers' energy and expertise.
Types and forms of questionnaire item
The questionnaire items can take the form of statements and questions.
Statements may be
tailored to measure sensitive topics but in a
non-personal, general, and thereby
less threatening fashion.
i.e. The respondent is
asked to react to somebody else’s idea rather than answer a pointed question
about his or her own views’ (Hessler, 1992, p.98).
Using both statements and questions in the same questionnaire helps to capitalize on the advantages of each.
Basic Principles for Constructing
The basic principle for constructing questionnaire items is to
formulate questions
in a way that respondents can
easily and accurately understand.
This means that they can provide adequate information for you
To do this, you need to:
Know the Objective
Make sure that each questionnaire item
achieves the research objectives.
Sometimes it is easy to confuse ‘important items’ with ‘relevant items’.
If an item is very important but does not relate to any concepts identified in your research problem, then it is useless to your research and you have to delete it from the questionnaire.
Know the respondents.
Understand their background (e.g. education, culture) so that you
use appropriate words and context that they understand
or are familiar with.
By knowing about your respondents, you can also make sure that your questionnaire is asking something they are likely to know about.
Know basic rules
Follow some basic rules in formulating the questionnaire items
Bernard (2000) lists the following 15 rules for constructing questionnaire
items:
Package items
in self-administered questionnaires.
Response
categories should be exhaustive and
mutually exclusive.
Use
clear scales.
Keep
threatening questions, short.
Construct contingent and
filter questions carefully.
Provide choices
when appropriate.
Make sure that each
item is related to
the
research
goal(s).
A
void loaded questions
Make sure that the
respondents know enough to respond
to your questions.
Avoid
putting
false premises
into items.
Don’t use affective
ly worded items.
Be
unambiguous
Don’t
use
double-barreled items.
Provide a reference when
asking for opinions on
controversial issues
.
Use
vocabulary
that the respondents understand and are
comfortable
with.
Interview Guide
Interview Guides are used when data (normally qualitative date such as feelings) is collected through personal depth interviews.
As opposed to survey questionnaires, the interview guide is designed using either statements or
open-ended questions, instead of standardized structured questions
.
interviews are usually taped-recorded and the interview transcript is produced after the interview to become an important component of the research data used for qualitative analysis.
useful for exploring complex and emotionally loaded social phenomena, such as delinquency and poverty
Structured Interview
The goal here is to
minimize interviewer bias by standardizing the research procedure
.
The number, sequence, wording, and even response categories of questions are fixed.
The
questions are asked in a specified order
and way
Unstructured interview
The main goal is to
explore all possibilities in order to gather information
and allow the respondents to take the lead to a great extent.
In this way, areas important to the respondents and the examined issue are identified.
There is no fixed schedule of questions, that is,
no definite order or specified wording
.
Semi-structured interview
The aim of this kind of interview is to
obtain in-depth information more naturally
(to the respondents) at the same time obtaining information about the same questions from all respondents.
Interviews in which the respondents are asked the same questions, but the
question order and way in which they are asked may vary
.
This is particularly
useful in studying respondents with different ages or social backgrounds when wording and sentence structure need to be adapted.
Observation Guide
Observation is one of the data collection methods used
mostly by qualitative researchers
. Depending on the types of observation used the researcher relies on
an observation guide (for
unstructured observation
such as participant-observation and other ethnographic studies) or
an observation checklist (for
structured observations
) to record quantified data.
A
checklist
is a specific kind of observation schedule in which different levels/values of the variables are listed in a table so that researchers only need to use checkmarks
to record the observation
. The development of observation schedules is rather simple. Having refined the research problem, you need to identify empirical indicators that measure the variables to be studied. The categories of empirical indicators are then listed in the observation schedule.
CHAPTER SIX: SAMPLING DESIGNS
Basic Concepts of Sampling
Population vs. Sample
Population
it is the
total group of elements
that the researcher wishes to study.
The elements may be observations, people, objects, households, documents, or any specific entity.
It is the sum total of all elements that the sample elements supposedly represent.
Sample
It is a
part of the population
that is used to show what the population is like.
The sampling units can be individuals, families, school classes, geographical area, organizations, city governments, books, media programmes, minutes in an hour, stones, etc.
Sampling Frames and Sampling Units
A
sampling frame
is a
list of all the units wherein the sample is drawn.
Theoretically, the sampling frame should contain all units in the target population. But in practice, it usually contains only a portion of the population.
For example, the telephone directory includes individuals who subscribe to a telephone line, but it does not have records of people who do not subscribe to a telephone line.
Sampling units
are
elements that make up the population
and are chosen at the sampling process.
Parameters vs. Statistics
parameter
parameter is a
measurable characteristic of a population
.
It is usually unknown.
An exact value of a parameter can be obtained by studying the whole population.
An estimate of it can be obtained by studying samples selected from the population.
Statistic
A statistic is
a characteristic of a sample
used to estimate the corresponding parameter of a population.
Since different samples may be drawn from a population, different statistics may be obtained. Although a single population parameter corresponds to a particular characteristic, there can be more than one corresponding sample statistic.
For example, a researcher may draw samples from different parts of the country or even another sample from the same region in the country.
Each sample may yield a different statistics, for example, the mean age.
Sampling Errors
A sampling error is the
discrepancy between a sample statistic and its corresponding population parameter
.
Since samples are estimates of the population, you may expect that a sample can represent the population only to a certain extent.
However, since not all samples are the same, sampling errors also vary from sample to sample.
Probability
Probability is
the likelihood of an event to occur
.
It is expressed as a fraction, the ratio of the number of target cases (or outcomes) to the total number of possible cases (outcomes).
probability plays
an important role in inferential statistics
. Probability allows you to predict what kind of sample is likely to be obtained from a population. It forms a
bridge between populations and samples.
Representativeness of Samples
‘Representativeness means that what you know from the sample is the same as what you would know if you studied the entire population’
(level of external validity)
Specifically, the
important characteristics of the population
must be
represented by the sample
For example,
diversity
in a population
must be taken into consideration
in sampling.
For a sample to be representative, it should tap the diversity among the population sampling units.
Sample Size
The sample size is the
number of sampling units to be included
in a sample.
The general rule of sample size is: The
larger the sample size, the lower the sampling error
Determining Sample size
the following factors serve to guide the determination of sample size
Characteristics
of the population.
Population characteristics such as homogeneity and size need to be considered in determining the sample size.
Generally,
the more diverse a population is, the more the required sample size to tap its diversity.
Similarly, the larger the population size, the larger the required sample size
Desired
precision
.
The
more precise estimate
of the population parameters,
the larger the sample size
.
Available
resources.
Without adequate resources
(e.g. time, money, human resources), the plan or decisions to have a
large sample size cannot be executed.
Number of
comparisons or breakdowns in data analysis
.
If multiple comparisons are required for a study, then researchers need larger samples, for instance, if the researcher wants to compare groups according to social class, age, gender, education level, and ethnicity.
The sample should be
large enough
to include a sufficient number of sampling units
for each of the categories.
Response and spoilage rates.
The number for selected sampling units should be larger than the expected sample size.
Researchers normally need to identify a
larger number
of eligible respondents than the required sample size
to allow for non-response
, such as unwilling to participate in the research, unable to follow instructions on questionnaires, drop out from an experiment, terminate an interview before it is completed, or fail to maintain a contact with the researcher whose study extends for a long time.
Strength of effect
.
The
stronger the expected effect
of a particular variable, the
smaller the required sample size
to demonstrate the effect.
Reliability of measures.
Measures with low reliability are associated with more measurement errors, which in turn reduce the accuracy of estimating population parameters.
A
larger sample size
is therefore
required for measures of low reliability
than those of high reliability.
Sampling (Participants) (To add from Chap six of PSY 390)
The
goal
of sampling is to create a sample (of participants) that
closely represents the population
of interest so that the data obtained can be generalised back to the population.
Sample size
Large and random samples are not necessarily representative samples of the target population.
A researcher can have a
larger sample size, it may not necessarily overlap
much with the population
if there are errors in sampling.
A
smaller sample size
with
fewer errors in sampling
may overlap much more with the population and
be more representative.
Sample size is linked to the target population
Sampling Methods
Non-probability Sampling
All participants have an unequal chance of being selected. I.e intentional selection (bias is present).
Convenience sampling
based on ease (such as word-of-mouth) to participate in your study.
A researcher who does convenience sampling
risks sampling bias
and/or not getting a representative sample of the population
Quota sampling
based on the filling up of
“quotas” in a particular category which are set by the research design (intentional)
.
For example, a researcher can decide to have certain proportions of the ethnicity of his sample so that it will be similar to the proportions of the ethnicity of his population.
Purposive Sampling
known as judgmental sampling, is defined as
a method in which researchers use their expertise to select individuals
to represent the population for their specific research purposes.
They may select typical cases, extreme cases, or theory based cases for the sample.
However, there is no way to safeguard the representativeness of the sample.
Extreme cases
are units or individuals that have extreme characteristics. The tallest and shortest people in a class,
Theory-based cases
match the characteristics specified in a theory. For instance, a researcher may want to investigate the cognitive development of children under different situations
Typical cases
have the general characteristics of the target population. For instance, a typical housewife would be a married woman with children and no paid job.
Snowball Sampling
In snowball sampling, researchers first identify a few research participants who have certain characteristics relevant to the study. They then ask the participants to recommend any other people who meet the research criteria
The key thing here is that the target good here is very hard to gain entry to.
Probability Sampling (i.e. random or systematic)
All participants have an equal chance of being selected. However, this may
result in a non-representative sample of the population.
Types of probability sampling:
Simple random sampling
: sampling based on a purely random process where everyone has an equal chance. Absence of Bias. No sample error. Easiest and fastest.
Systematic sampling
: sampling based on specific intervals, e.g., every third number on the list
Stratified sampling
: sampling based on strata or categories
Cluster sampling
: multi-stage sampling. divide your population according to (geographical clusters) clusters and select your participants randomly within those clusters. sample within a cluster an sample across.
Random-digit dialling
(RDD): sampling was done randomly by phone number
Sampling Errors
A sampling error is the
discrepancy between a sample statistic and its corresponding population parameter
.
Since samples are estimates of the population, you may expect that a sample can represent the population only to a certain extent.
However, since not all samples are the same, sampling errors also vary from sample to sample.