This document provides a brief overview of the different types of items that can be authored in Item Bank. This lesson guides item authors through understanding what the different item types entail and when it might make sense to use each, and to provide some best practices for authoring different item types.
Item Type Overview
The Multiple Choice family. In all of these item types, the author defines answer choices, and the student selects one or in some cases more than one of them.
- Multiple Choice
- Multiple Select
- EBSR (Evidence-Based Selected Response)
- Inline Dropdown
The Constructed Response family. In all of these item types, the student “constructs” a response from scratch, whether with the keyboard (Explicit Constructed Response and Constructed Response), an on-screen math editor and/or the keyboard (Math Equation Response), or a set of mouse-based drawing tools potentially in combination with the keyboard (Drawing Response).
- Explicit Constructed Response
- Math Equation Response
- Constructed Response
- Drawing Response
The Interactive Family of Item Types. These item types are more interactive. They range from Hot Spot, where a student is shown an image with clickable regions that they can toggle on and off, to Drag in the Blank, where a student drags objects into a response area.
Click-to-Select Item Types:
- Hot Text
- Hot Spot
Drag-and-Drop Item Types:
- Drag in the Blank
Multiple Choice Family of Item Types
Multiple Choice is the most fundamental and familiar item type in this family. Students are presented with several answer choices, and must pick one of them.
- It is usually best to offer 4 or 5 answer choices, although it is also possible to use Multiple Choice items for true/false or similar questions.
- Incorrect answers known as “distractors” are ideally chosen to represent common misconceptions or errors, so that students’ answer choices will provide insight into their knowledge and understanding.
The following information applies to the answer choice order of Multiple Choice, Multiple Select, EBSR, and Inline Dropdown items:
- By default, answer choices can potentially be shuffled when a question is delivered. This won’t always happen when an assessment is assigned. The administrator decides whether or not answer choices should be shuffled. If, however, a particular question has had “Lock Choice Order” set to true, then answer choices will always be fixed, even if shuffling is otherwise being used on the assessment.
- One reason to lock answer choices is if there is a “none of the above” or “all of the above” answer choice. It wouldn’t make sense for such a choice to appear anywhere but last. (There is not currently a mechanism to lock only the last answer choice.)
- Another reason to lock answer choices would be if they are in a particular order that should always be used. For instance, when the answer choices for a math question are numbers, it is a common practice to list them in numerical order.
Multiple Select is like Multiple Choice, except that students can select more than one choice. In the language of computer interfaces, these items use square “check boxes” (which are “multi-select”) rather than circular “radio buttons” (which are “single-select”).
- It is often a good idea for a Multiple Select question to tell the student the number of answers to select. E.g., “Choose TWO”. This is especially true if partial credit is not going to be awarded. Questions that don’t indicate how many choices should be selected may be quite difficult to answer correctly, and this is particularly true of reading comprehension questions where there is some amount of judgment involved. When answers are completely objective and unambiguous, as is common in math, it may be more reasonable to omit this information. (e.g., “Choose all expressions that represent an even number.”)
- It is usually best to avoid having a “Multiple Select” question where only one answer choice is actually correct, which would likely be viewed as a “trick question”.
- We generally recommend offering 4 to 6 answer choices, of which no more than n/2 + 1 are correct—i.e. no more than 3 of 4 or 5, and no more than 4 of 6.
- When partial credit scoring is used, scoring is based on the total number of correct answer choices. For instance, if there are two correct answer choices, then ½ point will be awarded for each that is selected; if three, then ⅓ point will be awarded for each that is selected. Deductions for incorrect answers are only taken if the student marked more answer choices as correct than was in fact the case. For example, if the correct answer is ABC, then both AB and ABD would be scored as ⅔, whereas ABDE would be scored as ⅓.
Partial Credit is not currently available in New Item Bank.
EBSR (Evidence-Based Selected Response). This item type essentially consists of two linked multiple choice questions, but scoring is different from stand-alone multiple choice items. Generally the second question should ask the student to identify the evidence for the first question; usually these questions are based on a passage. If a pair of questions do not adhere to this structure, EBSR may not be the right item type to use. For instance, the first question might ask the student to choose a character’s emotional state, and the second question might ask the student to choose the textual evidence from which this can be inferred.
- If all-or-nothing scoring is in effect, then both parts of an EBSR item must be answered correctly for credit to be awarded.
- If partial credit scoring is in effect, then a student can be awarded half credit if only the first part is correct, but will receive no credit if only the second part is correct.
- It is also possible to make the second part a Multiple Select question rather than a Multiple Choice question.
Inline Dropdown. In an inline dropdown item answer choices appear in-line with text, in a dropdown menu (as the name suggests). An inline dropdown item can have a single dropdown menu (“response area”), or several. In each case, just one response will be correct (as with Multiple Choice items).
- It is usually best to keep Inline Dropdown answer choices somewhat succinct. A word, or a relatively short phrase, is ideal. For longer answer choices, especially those that might require more than one line, Multiple Choice may be a more appropriate option.
- Compared to Multiple Choice items, it is more common to find situations where it makes sense to offer only two or three answer choices, at least for Inline Dropdown items with several response areas.
Constructed Response Family of Item Types
Explicit Constructed Response (ECR) is appropriate for questions that have very well-defined answers, such that a student can reasonably be expected to type the correct answer so that it exactly matches the “official” correct answer specified by the item author.
- For an answer to be recognized as correct, it must EXACTLY match the answer specified by the author. The ability to make validation indifferent to capitalization is planned, but is not currently available, so if the primary correct answer is “minute”, and you want “Minute” and “MINUTE” to also be recognized as correct, they would each have to be added as alternate answers.
- ECR can be used with numerical answers, although in some cases Math Equation Response may be a better fit.
- ECR is also appropriate for names, and single-word answers where there is no ambiguity. If the correct answer is a multi-word phrase, consider whether this is really the best item type for that question.
- ECR response areas are usually embedded inside of text. E.g. “The capital of New York is [ ].” An ECR item can have one or more response areas.
Math Equation Response (MER) is similar to Explicit Constructed Response (ECR), but designed specifically for answers that are mathematical expressions. The biggest differences relative to ECR are as follows:
- Students are provided with an on-screen math editor that allows them to enter mathematical expressions like fractions and radicals, as well as mathematical characters like the multiplication and division symbols. (Students can also enter their responses via the keyboard.) Various editors are available, with buttons to support symbols and notation commonly used in different grade levels (e.g. “Grade 6-7”) and topic areas (e.g. “Statistics”). The student cannot change the editor, so choose one that is a good fit to the question.
- Answers are validated as correct or incorrect by a validation engine, which provides more flexibility than simple string matching. An overview of MER validation options follows.
- There are two validation options, Literal and Symbolic.
- Literal Validation is similar to the basic string validation used by ECR, but with several features that make it more flexible.
- Leading zeroes are always ignored. So 0.5 ⬌ .5. (Read ⬌ as “is equivalent to”. In other words, if the correct answer is specified as either 0.5 or .5, the other will automatically validate as correct.)
- Spaces are generally ignored. So “a+b” ⬌ “a + b”.
- Thousands-separator commas are ignored. So “1,000” ⬌ “1000”
- The multiplication sign and dot are interchangeable. So “5×a” ⬌ “5⋅a”
- Symbolic validation, by contrast, assesses whether the expression entered by the student is mathematically equivalent to the correct answer. So if the correct answer specified by the author is 1 1/4, symbolic validation will also accept 5/4, 1 2/8, or 1.25, all of which would be rejected by literal validation.
- Literal validation also offers two optional settings that tweak its behavior.
- If Allow Trailing Zeros is set to true, then trailing zeros will be ignored (as leading zeros always are) . So 5 ⬌ 5.0 ⬌ 5.00 ⬌ 5.000, etc.,
- If Ignore Order is set to true, then
- ab ⬌ ba, a+b ⬌ b+a, ab+c ⬌ c+ba
- a=5 ⬌ 5=a
- 0<x≤5 ⬌ 5≥x>0
- MER items authored in New Item Bank have a single stand-alone response area. It is not currently possible to create an MER item with multiple response areas, or with a response area that is inline with text.
- For numerical answers, when is it better to use MER vs. ECR?
- For simple whole-number answers under 1,000, ECR is simpler.
- For decimals, MER has some advantages (e.g. treatment of leading zeros)
- For fractions MER is the preferred choice, since students can format them properly, rather than just seeing a typed expression like “1/16", which is what you would get with ECR.
- For numbers over 999, MER will automatically recognize commas used as thousands separators, without needing alternate correct answers.
- For dollar values and the like, MER can recognize responses with trailing zeros (e.g. “4.00”) without needing an alternate correct answer (depending on how the validation is configured).
- ECR allows response areas to be embedded within other text, which is not currently possible with MER, although that is under development.
- CR items must be manually scored using a rubric. For that reason, it is recommended that they be used only in situations where other item types cannot adequately assess the skill in question.
- For math CR questions where a student is asked to explain their work, consider turning on the “Equation Editor” setting, which will allow the student to access a math editor when writing their answer. If you do this, choose an editor consistent with the demands of the question.
- These items must be scored by hand using a rubric. For that reason, even though this is a very flexible item type, we suggest only using it in situations where other item types cannot adequately assess the learning objective.
- It often makes sense to include a background image, which students will draw their answers on top of. For instance:
- The background image could include shapes that students are asked to divide into thirds, or to draw lines of symmetry on.
- The background image could include a diagram, where students are asked to label certain parts. (The drawing tool supports basic text entry.)
- Bear in mind that the drawing tools provided are rudimentary, so item demands should be consistent with their limitations.
Interactive Family of Item Types
- This item type takes as its starting point a block of text. The author designates portions of that text (words, sentences, etc.) as “tokens”, which students can select, and then designates one or more of the tokens as the correct answer.
- Sometimes, an item will have a small number of tokens (perhaps 4 to 6), only 1 or 2 of which are correct, making for a student experience quite similar to a Multiple Choice or Multiple Select item. In other cases, every word, or every sentence, will be “tokenized”, potentially making for a quite different student experience. For instance, a student might be asked to identify all the misspelled words, or all the adjectives.
- Students can select a token by clicking on it, which highlights it. Subsequent clicks toggle the selection state. Students can always select at least as many tokens as the number of selected tokens in the correct answer, but may be restricted from making additional selections beyond that number. For full credit, the student’s selections must fully match the selections in the correct answer.
- Hot Text has a setting called Highlight Choices.
- When Highlight Choices is on (as it is by default), each token will be surrounded by a dashed box, making it obvious on sight what the tokens are. This may be a good option for items where just a few words have been tokenized, since it will otherwise take the student a few seconds just to identify the available answer choices.
- When Highlight Choices is off, tokens are not called out, but mousing over any part of a token will highlight the entire token. This is generally preferred for Hot Text items where every or almost every word or sentence is a token, since drawing a box around every word can make the text difficult to read.
- This item type takes as its starting point an image. The author designates “hot spots” by click-and-dragging to create rectangular regions on the image, and then select to designate one or more of them as the correct answer.
- Hot spots appear as semi-transparent colored rectangles; selected hot spots are indicated by a heavy border. The author can choose from three colors (red, yellow, and blue) for both the semi-transparent regions, and the borders. For black and white or gray-scale images, any color choice is fine, but for color images it is recommended that you try the various options to find one that makes the hot spots clearly visible without obscuring important features of the image itself.
- Similar to Hot Text, there can be just a few hot spots, or a great many. For instance, if the image consists of an unlabeled map, the author might designate every region as a hot spot.
- The default authoring behavior is to limit the item to just one correct answer, in which case clicking on a hot spot will select it (designating the correct answer for an item author, or the student’s answer choice for a student). To change this, the Multiple Correct Responses setting must be turned on; in this case clicking a hot spot will toggle its selection status. When this setting is on, there is no mechanism to limit how many answer choices a student can select. To receive full credit, a student must select all the correct answers, and no incorrect answers.
- In this item type, the author creates unique “choices”, draggable chunks of text, and “response areas”, containers, into which choices can be dragged (one choice per response area). Initially, students see all the answer choices grouped together; to answer the question, choices are dragged to response areas.
- Response areas are embedded inside of a text-based template. Although the template can include images and math markup, neither of these can itself hold a response area. To answer an item, students must drag the correct choice(s) to the matching response area(s).
- Choices can include mathematical markup, but cannot currently include images.
- Each response area needs to have a unique correct choice. For instance, it would not be a good idea to have an item that says “To create purple, mix [red] and [blue]” since if a student dragged “blue” to the first response area and “red” to the second, that would be marked as incorrect.
- By default, each choice can only be used once; once a choice has been placed into a response area, it will no longer be present in the “pool” of available choices. To change this behavior, turn on the “Allow Duplicates” setting. Then, answer choices will remain in the “pool” even after they have been used, such that the same answer choice can be dragged to multiple response areas.
- It is fine to have more choices than response areas, such that there will be “left over” choices.
- Although it is currently possible to create an item with unused response areas, response areas into which no choice should be placed, this is generally not a recommended practice.
- By default, choices appear below the template; to change this, select a different Choices Position setting.
- By default, the order in which choices appear to students will be randomized. To change this, turn on the Lock Choice Order setting, and then answer choices will be presented in the order in which they were created. For the most part, randomizing the order is recommended. If you choose not to randomize, pay attention to the order in which you create answer choices, to make sure that they aren’t “pre-sorted” into the correct order.
- If an item has multiple response areas, the student must place the appropriate choice into each of them to receive full credit.