Assessment and Feedback
Assessment Item Writing: Clear Questions, Distractors, and Review Checks
What changed
This revision replaces broad advice with a blueprint process, paired weak and improved items, a distractor review, a scoring rule, post-use decisions, and a copyable item-review checklist. It also separates ordinary classroom review from formal psychometric claims.
Start with the purpose and decision
An assessment item should elicit evidence about a named learning objective. Before drafting, write the decision the evidence will support: reteach a prerequisite, give feedback, assign further practice, report achievement, or revise the assessment. The purpose determines the content, response format, scoring rule, and level of review required.
Write the intended inference in one sentence: “A correct response will support the conclusion that the student can ___ under ___ conditions.” Then list demands that are part of the objective and demands that are not. If the objective is mathematical reasoning, for example, unfamiliar idioms and unnecessarily dense reading can obscure the intended evidence.
Build a small assessment blueprint
A blueprint prevents a quiz from overrepresenting whatever was easiest to write. Use the applicable standards and curriculum sequence, then map each objective to the student action, item count, format, and scoring weight.
On a small screen, scroll this table horizontally to see all columns.
| Objective | Required student action | Planned evidence | Items and weight | Access demand to review |
|---|---|---|---|---|
| Identify a claim supported by a source | Select and cite relevant text | Two selected-response items | 2 items; 2 points | Source length and vocabulary |
| Explain why evidence supports a claim | Connect evidence to reasoning | One constructed response | 1 item; 3 points | Writing load unrelated to the explanation |
| Compare two claims | Apply criteria and justify a judgment | Short comparison | 1 item; 3 points | Amount of information held on the page |
Count points as well as items. A single long response may carry more weight than several multiple-choice questions. Check that the final weighting matches the priority of each objective.
Weak and improved multiple-choice item
Worked multiple-choice example
Objective: Given a plant investigation, identify the variable the student changes.
Weak item: “Which of the following is not an incorrect statement about the experiment?” A. Plants are interesting. B. Light. C. The student watered them. D. All of the above.
Why it is weak: The negative construction is difficult to parse, the experiment is not described, the choices are not parallel, and more than one answer could be defended.
Improved item: “A student places identical bean plants under 4, 8, or 12 hours of light each day. Each plant receives the same soil and amount of water. Which variable does the student change?” A. Type of plant. B. Hours of light. C. Amount of water. D. Type of soil.
Key: B. The stem supplies the needed context, asks one positive question, and makes the changed variable distinguishable from controlled conditions.
Review each distractor
A distractor should represent a plausible error related to the target, not a trick, joke, or clue. Each choice should fit the grammar of the stem and be similar enough in length and specificity that formatting does not reveal the key.
On a small screen, scroll this table horizontally to see all columns.
| Choice | Reason a student might choose it | What it tests | Keep or revise |
|---|---|---|---|
| A. Type of plant | Confuses a condition held constant with the changed variable | Distinguishing variables from controls | Keep |
| B. Hours of light | Identifies the value deliberately varied | The intended objective | Key |
| C. Amount of water | Notices a measured quantity but misses that it stays constant | Distinguishing variables from controls | Keep |
| D. Type of soil | Confuses another controlled condition with the changed variable | Distinguishing variables from controls | Keep |
Replace a distractor when it is indefensible, duplicates another choice, depends on irrelevant background knowledge, or is almost never selected after a sufficiently informative classroom use. A rarely chosen distractor is a review signal, not automatic proof that the choice is defective.
Weak and improved constructed response
Worked constructed-response example
Objective: Explain how a quoted detail supports a stated claim about a character’s decision.
Weak prompt: “What do you think about the character? Explain everything you know.”
Problem: The task is unbounded, invites opinion without evidence, and gives no basis for scoring.
Improved prompt: “The character decides to return the lost wallet. Quote one detail from paragraphs 6–8 and explain how that detail supports the claim that the decision was difficult.”
Three-point scoring rule: Award 1 point for an accurate, relevant detail; 1 point for connecting that detail to the stated claim; and 1 point for an explanation consistent with the passage. Do not award the evidence point for an unrelated quotation or the reasoning point for merely repeating the claim. Accept an equivalent spoken, signed, scribed, or assistive-technology response when permitted and when it preserves the same evidence.
Run wording, background, and access checks
On a small screen, scroll this table horizontally to see all columns.
| Check | Question to ask | Revision when the demand is irrelevant |
|---|---|---|
| Negative stem | Can the question be asked positively? If a negative is essential, is the negative word visually clear without relying on color alone? | Rewrite positively or emphasize “NOT” in text and verify that the item still asks one clear question. |
| Reading load | Are sentence length, vocabulary, and source volume necessary to the objective? | Remove extra narrative, define non-target terms, or separate source material from directions. |
| Cultural background | Does success depend on an experience, idiom, place, household practice, or cultural reference that was neither taught nor assessed? | Supply the needed context or use a setting that does not carry the answer. |
| Accessibility | Can students perceive the content, navigate the item, and use approved accommodations without changing the target? | Add text alternatives, improve layout, permit the approved response route, or replace inaccessible material. |
| Unintended clues | Does grammar, answer length, repetition from the stem, or choice pattern reveal the key? | Make choices parallel and remove clues unrelated to knowledge. |
| Scoring | Would two reviewers apply the same stated rule to the same response? | Define point criteria and test them on sample responses before use. |
Use psychometric terms within classroom limits
Item difficulty is the proportion or percentage of students who answered an item correctly in a particular administration; it is not an inherent label for all learners. Distractor analysis examines which answer choices students selected. Item discrimination describes how responses to an item relate to performance on a broader score under a specified method. Reliability concerns consistency of scores for an intended use, while validity concerns the evidence and reasoning supporting an interpretation and use of scores.
In an ordinary class, small groups, uneven instruction, absent students, collaboration, and a short quiz make these statistics unstable or incomplete. A teacher can inspect patterns and flag an item for review, but a single classroom percentage should not be treated as proof of bias, reliability, mastery, or a student trait. High-stakes or standardized uses require qualified technical review, adequate samples, documented procedures, and the standards that govern the program.
Review items after use
- Check the key and scoring: Rescore a sample and inspect whether the stated rule handled reasonable responses.
- Compare item evidence with the objective: Look at student work, not only total scores, and identify the reasoning each item actually required.
- Inspect response patterns: Note distractors chosen, omitted items, unexpected interpretations, and access problems reported or observed.
- Choose a disposition: Keep the item when evidence is aligned and interpretable; revise it when wording or format interfered; remove it when no defensible repair is available.
- Protect the decision: When an item is flawed, reconsider any grade or instructional conclusion materially affected by it under local policy.
Copyable item-review checklist
ASSESSMENT ITEM REVIEW
Assessment purpose and decision:
Learning objective:
Required student action:
Item format and point value:
ALIGNMENT
[ ] The item elicits the knowledge or reasoning named in the objective.
[ ] Point weight matches the blueprint.
[ ] Difficulty comes from the target, not a trick or unrelated demand.
WORDING AND CHOICES
[ ] The stem asks one complete, direct question.
[ ] A negative stem is necessary and visibly clear.
[ ] Each option is plausible, distinct, parallel, and grammatically consistent.
[ ] There is one defensible key, or the scoring rule covers valid variation.
FAIRNESS AND ACCESS
[ ] Reading and language demands are necessary or supported.
[ ] No untaught cultural background is required unless it is the target.
[ ] Images, layout, navigation, and response mode are accessible.
[ ] Approved accommodations preserve the intended evidence.
SCORING AND POST-USE REVIEW
[ ] Point criteria identify what earns each point.
[ ] Sample responses have been checked against the rule.
[ ] After use, response patterns and access problems will be reviewed.
[ ] Disposition: keep / revise / remove.
Revision note:
Completed checklist example
The checklist below shows how one teacher completed the review for the improved plant item from the example above. The comments show the evidence behind each check, not just the check itself.
ALIGNMENT
[x] The item elicits the knowledge or reasoning named in the objective.
(Comment: a correct response requires identifying the changed variable.)
[x] Point weight matches the blueprint. (Comment: 1 of 2 selected-response items.)
[x] Difficulty comes from the target, not a trick or unrelated demand.
(Comment: reading load is one short paragraph; no idioms.)
WORDING AND CHOICES
[x] The stem asks one complete, direct question.
[x] A negative stem is necessary and visibly clear. (Not used.)
[x] Each option is plausible, distinct, parallel, and grammatically consistent.
(Comment: all four options are noun phrases of the same length.)
[x] There is one defensible key, or the scoring rule covers valid variation.
(Comment: key B; the controlled variables are stated in the stem.)
FAIRNESS AND ACCESS
[x] Reading and language demands are necessary or supported.
[x] No untaught cultural background is required unless it is the target.
[x] Images, layout, navigation, and response mode are accessible.
[x] Approved accommodations preserve the intended evidence.
(Comment: read-aloud and scribe routes keep the variable-identification target.)
SCORING AND POST-USE REVIEW
[x] Point criteria identify what earns each point.
[x] Sample responses have been checked against the rule.
[x] After use, response patterns and access problems will be reviewed.
[x] Disposition: keep / revise / remove. (Comment: keep; review after first use.)
Revision note: none needed before first use; recheck item-level responses after administration.A completed checklist is useful only when each check names the evidence that supported it. An unchecked item with a written reason is a deliberate decision, not an incomplete checklist.