FUNDAMENTALS OF RELIABILITY
The term reliability suggests that it can be trusted, in whole or in part on the basis of test scores.
It's based on
The consistency and accuracy of the results of the measurement process.
It's important to put attention on
Test scores do not have to be totally consistent or error free.
TRUTH AND ERROR IN PSYCHOLOGICAL MEASUREMENT
One of the most enduring approaches to the issue of reliability.
Is
The classical notion of TRUE SCORE test theory.
Although true scores do not really exist, it is possible to imagine their existence
The true scores are the hypothetical entities that would result from an error-free measurement.
Methods for estimating the reliability of scores provide a way to estimate true scores, or at least the limits within which true scores might lie.
The concept of real score in indivtidual data
An individual's actual score is conceptualized as the mean score in a hypothetical distribution of scores.
It would be obtained if the individual performed the same test an infinite number of times.
In practice, it is obviously impossible to obtain such a score even for one individual, let alone many.
Instead of actual scores, what you get from the tests are the observed scores.
THE RELATIVITY OF RELIABILITY
If a test is qualified as reliable.
This implies that
Its reliability has been permanently established, in all respects, for all uses and with all users.
Even when applied to test scores, reliability quality is relative.
Rather
The score obtained may be more or less reliable due to the examiner's own factors.
For example
Tiredness, lack of motivation, the influence of drugs, etc.
Either
To the conditions of the test situation
For example
The presence of distracting noises, the personality of the examiner, the rigor with which time limits are applied, etc.
All these factors can affect, separately or jointly
To
Score obtained to a greater or lesser extent, even to the point where the score is so unreliable that it has to be discarded.
SOURCES OF ERROR IN PSYCHOLOGICAL TESTS
Error can enter psychological test scores for a huge number of reasons.
Many of which are outside the realm of psychometric estimates of reliability
Nevertheless
Errors entering test scores can be classified as coming from one or more of the following three sources
(b) The examiner
(a) the context in which the test is performed.
(c) The test itself.
Including factors related to the test administrator, test scorer, and testing environment.
FUNDAMENTALS OF
R I A B I L I D A D
Some of the errors from these sources can be minimized or eliminated.
As long as
Parties involved in the test development, selection, administration, and scoring process follow appropriate assessment practices.
RELIABILITY IN THE USE OF THE TESTS
Reliability of scores is an ongoing consideration in psychological testing
Because
The ever-present possibility of errors from various sources entering test results.
Nevertheless
The way in which the reliability of scores is considered differs in several
test development process points
As well as
in their actual application.
Reliability estimates should be carefully considered and applied at the stages of
(a) Test selection.
(b) Interpretation of scores.
The Item Response Theory Approach to Reliability
There are more sophisticated methods to estimate reliability through item response theory (IRT)
With IRT methods, reliability and measurement error are addressed from the point of view of the information function of individual test items.
As opposed to the test as a whole
Given to
The level of difficulty and the discriminating power of the individual items.
Regarding
The trait evaluated by the test can be more carefully calibrated by IRT methods.
The information provided by the response of each examinee is more precise and, therefore, more reliable.
RELIABILITY CONSIDERATIONS IN THE INTERPRETATION OF TESTS
Once you have:
-Chosen one
-Administered
-Passed a test
Reliability data is applied in the test interpretation process for two distinct but related purposes.
- It is recognizing and quantifying the margin of error in the scores obtained in the test.
- It is to evaluate the statistical significance of the difference between the scores obtained
To help
Determine the significance of those differences in terms of what the scores represent.
THE RELATIONSHIP BETWEEN RELIABILITY AND VALIDITY
From the psychometric perspective.
Evidence of the reliability of scores is considered a necessary, but not sufficient, condition for validity.
In fact
The two concepts are intrinsically related.
In the sense that
The reliability of the score can be considered in itself as a minimum test
That a valid measure of a behavioral sample has been obtained.
Evaluation professionals generally agree that
Reliability tests of scores are not a sufficient basis
For
Make valid inferences about their meaning
Nevertheless
There is some room for disagreement as to the extent
In what
Reliability tests are considered essential for the valid assessment of all types of behavioral samples.
That can be collected through tests.
For example
When test scores are derived from behavioral samples.
That are unique or idiosyncratic may not be repeatable or consistent.