Intro to Hypothesis Testing
Logic of Hypothesis Testing
Hypothesis Testing: statistical method that uses sample data to evaluate a hypothesis about a population
1.) State hypothesis about a population
2.) Set the Criteria for a Decision
3.) Collect Data & Compute Sample Statistics
4.) Make a Decision
The Unknown Population
Begin with a set of individuals as they exist before the treatment
Goal is to determine what happens to the population after the treatment is administered
Basic research situation for hypothesis testing
Assumed that the parameter means is known for the population before treatment
Purpose of the study is to determine wheater the treatment has an effect on the population mean
The Sample in the Research Study
Goal of testing is to determine whether the treatment has any effect on the individual in the population
Usually concerns the value of a population parameter
Hypothesis is about unknown population
Stating two opposing hypotheses
1.) Null Hypothesis: states that the treatment has no effect
2.) Alternative Hypothesis: there is a change from the general population
Predicts independent variable (treatment) does have an effect on the dependent variable
Determine which sample means are consistent with null hypothesis & which sample means are at odds
Determine which values are near the mean & which are very different
Distribution of sample means is divided into 2 sections
1.) Sample means that are likely to be obtained if the null hypothesis is true
Sample means that are close to the null hypothesis
2.) Sample means that are very unlikely to be obtained if the null hypothesis is true
Sample means that are very different from the null hypothesis
The Alpha Level : probability values that is used to define the concept of "very likely" in a hypothesis test
To find boundaries that separate high probability samples from low probability samples
Must define what is meant by "low" & "high" probability
With rare exceptions, the alpha level is never larger than .05
Critical Region : composed of extreme sample values that ate very unlikely to be obtained if null hypothesis is true
Boundaries are determined by the alpha level
If sample data fall in critical region, the null hypothesis is rejected
Compute z-score that describes exactly where the sample mean is located relative to the hypothesized population mean from the null hypothesis
z-score = (sample mean - hypothesized population mean) / the standard error between the sample mean and population mean
TWO possible outcomes
1.) Sample data is located in the critical region
Sample is not consistent with the null hypothesis
Treatment has an effect
2.) Sample data is not within the critical region
Sample data is reasonably close to the population mean specified in the null hypothesis
Fail to reject null hypothesis
Treatment does not appear to have an effect
Reject null hypothesis
Uncertainty & Errors in Hypothesis Testing
Hypothesis testing is an inferential process which uses limited information as a basis for researching a general conclusion
Type I Errors : occurs when researcher rejects a null hypothesis that is actually true
Type II Errors : occurs when a researcher fails to reject a null hypothesis that is really false
Researcher concludes that a treatment does not have an effect when in fact it has no effect
Probability of type I errors occurs when researcher unknowingly obtains an extreme, non-representative sample
Alpha level for a hypothesis test is the probability that the test will be a type I error
Determines the probability of obtaining sample data in the critical region even though the null hypothesis is true
Hypothesis test failed to detect a real treatment effect
More Information About Hypothesis Testing
Significant or Statistically Significant Results: if the results are very unlikely to occur when the null hypothesis us true
Result is sufficient to reject the null hypothesis
Treatment has a significant effect if the decision from the hypothesis test is to reject the null hypothesis
Factors that influence hypothesis testing
If the z-score is large enough to be in the critical region then reject the null hypothesis and conclude there is a significant effect
Difference between the treated sample mean and the hypothesized mean from the null hypothesis influences the z-score
If there is a big difference then the sample is noticeably different from the untreated population
Usually supports a conclusion then the treatment has a significant effect
Size of z-score is influenced by the standard error
Determined by variability of the score & the number of scores in the sample
Assumptions for hypothesis testing with z-scores
Random sampling
Independent observations
Value of standard deviation is unchanged by treatment
Normal sampling distribution
Directional Hypothesis Tests
One-Tailed Test: statistical hypothesis specify either an increase or decrease in the population mean
Makes a statement about the direction of the effect
1.) In first step of hypothesis testing - directional prediction is incorporated into the statement of the hypothesis
2.) In 2second step of hypothesis testing - critical region is located entirely in one tail of the distribution
Comparison of One-Tailed vs. Two-Tailed Tests
Difference between criteria used for rejecting the null hypothesis
One-Tailed: reject when the difference between the sample and population is relatively small
Provided there difference is in a specified direction
Two-Tailed: there is a large difference independent of direction
Used when there is no strong directional expectation or when there are two competing predictions
Concerns about Hypothesis Testing
Limitations
Effect Size: measure intended to provide measurement of the absolute magnitude of a treatment effect independent of the size of the sample(s) being used
1.) Focus is on data rather than the hypothesis
2.) Significant treatment effect doesn't necessarily indicate a substantial treatment effect
Hypothesis testing simply establish results obtained in a research study that are very unlikely to have occurred if there is no treatment effect
Reaches conclusions by
1.) Calculating standard error
2.) Demonstrating obtained mean difference is substantially bigger than the standard error
Makes relative comparison
Measures by Cohen's d = the mean difference / the standard deviation
Statistical Power
The power of a statistical test is the probability that the test will correctly reject a false null hypothesis
Probability that the test will identify a treatment effect if one really exists