FUNDAMENTALS OF RELIABILITY

The term reliability suggests that it can be trusted, in whole or in part on the basis of test scores.

It's based on

The consistency and accuracy of the results of the measurement process.

It's important to put attention on

Test scores do not have to be totally consistent or error free.

TRUTH AND ERROR IN PSYCHOLOGICAL MEASUREMENT

One of the most enduring approaches to the issue of reliability.

Is

The classical notion of TRUE SCORE test theory.

Although true scores do not really exist, it is possible to imagine their existence

The true scores are the hypothetical entities that would result from an error-free measurement.

Methods for estimating the reliability of scores provide a way to estimate true scores, or at least the limits within which true scores might lie.

The concept of real score in indivtidual data

An individual's actual score is conceptualized as the mean score in a hypothetical distribution of scores.

It would be obtained if the individual performed the same test an infinite number of times.

In practice, it is obviously impossible to obtain such a score even for one individual, let alone many.

Instead of actual scores, what you get from the tests are the observed scores.

THE RELATIVITY OF RELIABILITY

If a test is qualified as reliable.

This implies that

Its reliability has been permanently established, in all respects, for all uses and with all users.

Even when applied to test scores, reliability quality is relative.

Rather

The score obtained may be more or less reliable due to the examiner's own factors.

For example

Tiredness, lack of motivation, the influence of drugs, etc.

Either

To the conditions of the test situation

For example

The presence of distracting noises, the personality of the examiner, the rigor with which time limits are applied, etc.

All these factors can affect, separately or jointly


To

Score obtained to a greater or lesser extent, even to the point where the score is so unreliable that it has to be discarded.

SOURCES OF ERROR IN PSYCHOLOGICAL TESTS

Error can enter psychological test scores for a huge number of reasons.

Many of which are outside the realm of psychometric estimates of reliability

Nevertheless

Errors entering test scores can be classified as coming from one or more of the following three sources

(b) The examiner

(a) the context in which the test is performed.

(c) The test itself.

Including factors related to the test administrator, test scorer, and testing environment.

FUNDAMENTALS OF
R I A B I L I D A D

Some of the errors from these sources can be minimized or eliminated.

As long as

Parties involved in the test development, selection, administration, and scoring process follow appropriate assessment practices.

RELIABILITY IN THE USE OF THE TESTS

Reliability of scores is an ongoing consideration in psychological testing

Because

The ever-present possibility of errors from various sources entering test results.

Nevertheless

The way in which the reliability of scores is considered differs in several
test development process points

As well as

in their actual application.

Reliability estimates should be carefully considered and applied at the stages of

(a) Test selection.

(b) Interpretation of scores.

The Item Response Theory Approach to Reliability

There are more sophisticated methods to estimate reliability through item response theory (IRT)

With IRT methods, reliability and measurement error are addressed from the point of view of the information function of individual test items.

As opposed to the test as a whole

Given to

The level of difficulty and the discriminating power of the individual items.

Regarding

The trait evaluated by the test can be more carefully calibrated by IRT methods.

The information provided by the response of each examinee is more precise and, therefore, more reliable.

RELIABILITY CONSIDERATIONS IN THE INTERPRETATION OF TESTS

Once you have:
-Chosen one
-Administered
-Passed a test

Reliability data is applied in the test interpretation process for two distinct but related purposes.

  1. It is recognizing and quantifying the margin of error in the scores obtained in the test.
  1. It is to evaluate the statistical significance of the difference between the scores obtained

To help

Determine the significance of those differences in terms of what the scores represent.

THE RELATIONSHIP BETWEEN RELIABILITY AND VALIDITY

From the psychometric perspective.

Evidence of the reliability of scores is considered a necessary, but not sufficient, condition for validity.

In fact

The two concepts are intrinsically related.

In the sense that

The reliability of the score can be considered in itself as a minimum test

That a valid measure of a behavioral sample has been obtained.

Evaluation professionals generally agree that

Reliability tests of scores are not a sufficient basis

For

Make valid inferences about their meaning

Nevertheless

There is some room for disagreement as to the extent

In what

Reliability tests are considered essential for the valid assessment of all types of behavioral samples.

That can be collected through tests.

For example

When test scores are derived from behavioral samples.

That are unique or idiosyncratic may not be repeatable or consistent.