Please enable JavaScript.
Coggle requires JavaScript to display documents.
Module 12: developing tests for maximal performance (item writng (true…
Module 12: developing tests for maximal performance
tests of
maximal performance
are intended to measure if the student is giving all of his effort. these types of tests typically have one correct answer.
the test developer must figure out the
domain
or possibly
construct
that the test is to be administered in. for psychometrics this would be a knowledge test in a classroom
speed tests
provide a large number of relatively easy items. how someone performs is measured by the amount of items gotten correct in a relatively short period of time
power tests
provide an ample amount of time for completion. here, how someone performs is assumed to result from differences in understanding or ability
many tests use some combination of
speed
and
power
to complete there assessment. the wonderlic classic cognitive ability test is an example. is uses cognitive ability to measure how an individual will perform in personnel selection.
level of cognitive objective
Bloom developed a taxonomy to assess what and how a certain test is measuring
Knowledge
: these items are the most concrete and include memorization of fact. key words may include
define
,
list
, and
name
comprehension
: these items assess understanding and interpretation of information. key words may include
summarize
,
discuss
, and
distinguish
application
: these items measure the test taker's ability to use information to solve novel problems. key words may include
apply
,
demonstrate
, and
calculate
analysis
: these items assess the test taker's ability to see patterns and organize components of a whole. key words may include
analyze
,
order
, and
classify
synthesis
: these items assess the test taker's ability to draw appropriate conclusions from information and to use old ideas to create new ones. Key words may include
generalize
,
combine
, and
create
evaluation
: these items at the highest level of abstraction require test takers to compare and discriminate between ideas. test takers are required to substantiate their choices based on rational argument. key words may include
assess
,
convince
, and
recommend
alternative knowledge dimension to Bloom's taxonomy
factual knowledge
conceptual knowledge
knowledge of theories
procedural knowledge
metacognitive knowledge
item format
constructed response
multiple-choice
true-false
matching
free response
short-answer
essay format
item writng
true-false items
express the item clearly, concisely, and simply as possible
assess only important knowledge; avoid assessment of trivia
create items that assess understanding-not just memory
avoid double negatives
use more false statements than true statements in the test
word the item so that superficial logic suggests a wrong answer
ensure that the intended correct answer is obvious only to those who have good command of the knowledge being tested
make the wrong answer consistent with a popular misconception or a popular belief irrelevant to the question
are typically used by people who have little experience with test making
matching items
are used to determine if an individual can distinguish between similar ideas, facts or concepts
provide additional plausible response options to avoid a "process of elimination" approach
keep response options short
choose item stems and response options that demonstrate the test taker's ability to distinguish between similar things
denote each item stem with a number, and each possible response option with a letter
provide clear instructions regarding what the test taker is intended to do. for example, specify whether response options can be used more than once
multiple choice items
can be used to assess lower and higher order cognitive ability
make sure the items assess important, significant ideas
pose a question (or statement) that has a definite answer
avoid giveaways as to the correct answer
when using a negative in the item stem (e.g. not), use capitalization or underlining to ensure that the word is read
consider using two sentences in the stem, one to present necessary background information, and one to ask the question
use gender and ethnicity in the inclusive fashion. alternate between "she" and "he." proper names should reflect ethnic diversity
tips for writing quality response options for multiple-choice items include the following
place words that appear in every response option in the stem
arrange response options in a logical order
include plausible distracters
include some true statements in the distracters that so not correctly answer the question posed in the stem
ensure that response options are parallel-that is, of approximately equal length and of equal complexity
write brief response options, rather than long ones
across the test, ensure that the correct response is balanced roughly equally across the possible response options (i.e. A, B, C, D, and E)
create distracters that include familiar phrases that would be attractive to those with only superficial knowledge
avoid the use of "
all of the above
" Once a test taker has determined that at least two response options are correct, the correct response must be "
all of the above
"
use "
none of the above
" only if this is sometimes the correct response option. Further, avoid the common mistake of including the option "
none of the above
" only in items for which it is the correct response
avoid use of "
A&C"
- type response options. while item difficulty increases, item discrimination does not improve
short answer and essay items
this type of question is typically used to test higher cognitive ability
ask only questions that produce responses that can be verified as better than other responses
provide terminology in the questions that limit and clarify the required responses
provide multiple specific questions rather than a very limited number of long questions
do not provide test takers a choice among several questions. this, in effect, is providing different exams to different students the equivalcy of alternative essay items is highly suspect
specify the number of points for each part of the essay item
test each item by writing an ideal answer prior to administration
tips for scoring short answer and essay items
when you are familiar with the test takers, ask test takers to record a confidential code rather than names on the test
develop a set of criteria for the scoring of each item
read several responses before assigning grades
select papers to serve as excellent, good, nominal, and poor models of the standards by which you are grading
assign global, holistic grades to each question rather than multiple grades on such elements as content, originality, grammar, and organization
test-wise test takers
the ability to answer the question based not on knowledge from the studying but rather from clues given away in the question.
the process of item modification
having subject matter experts (SMEs) review the items
pretesting the items (
pilot study
)
in pilot studies, they are normally conducted second. the order of operation usually goes as such; a small group of SMEs review the test and if it passes or modifications are made then the next step is to have a larger group of SMEs review it, this second larger group is called the pilot test
making any necessary modifications