How to Write a Smart Test

By Beth Richardson, RME High School Mathematics Coordinator

My career in education began as a high school math teacher. Throughout my teaching, I wrote countless math “questions” to check my students’ understanding, from daily bell-ringers to full-length tests. However, it wasn’t until I became immersed in the world of assessments that I learned some important components of a well-written test. First of all, the “questions” that make up a test are commonly referred to as items by researchers in the field of assessment, which is what I’ll call them from here on.

Test Math Knowledge in Different Ways

There are many different levels in which the brain engages in mathematical concepts. The book Adding It Up: Helping Children Learn Mathematics (2001) identifies five specific types of thinking that together determine a person’s proficiency in math. Here’s a brief explanation of each and example of items with the same skill (slope) assessed at the different proficiency levels:

• Conceptual Understanding – comprehension of mathematical concepts, operations, and relations
• Procedural Fluency – carrying out procedures flexibly, accurately, efficiently, and appropriately
• Strategic Competence – ability to formulate, represent, and solve mathematical problems 
• Adaptive Reasoning – capacity for logical thought, reflection, explanation, and justification
• Productive Disposition – habitual inclination to see math as sensible, useful, and worthwhile, coupled with a belief in one’s own efficacy. 

Sample Conceptual

Sample Procedural

Sample Strategic

Sample Adaptive

As teachers, we can only test the first four of these in a traditional test setting. However, productive disposition is something you can learn about each of your students as you interact with them daily.

Multiple-Choice Tests

Basic Multiple-choice Item Components:

• Skill: Comes from TEKS, district curriculum, etc.
• Mathematical Proficiency Level: Procedural, Conceptual, Strategic, or Adaptive
• Stem (Text/Graphic): Make sure the text and graphics you use are purposeful and relevant to the underlying mathematical skill/concept being assessed
• 4 Response Options: 1 correct response and 3 distractors that are well-thought out - no throw away distractors!

Write Plausible Distractors

For multiple-choice tests, the responses you provide are just as important as the question you ask.

Take the time to write distractors that are based on students’ common mistakes and misconceptions. To help ensure the distractors are plausible, write a rationale for each distractor. Also, avoid using give-away distractors that do not relate to the item. Here’s an example of a spreadsheet that can be used when writing a test. This spreadsheet can easily be copied and changed to create multiple forms of the same test. The specific details in the stem can be changed, but the same distractor rationales can be used. This will allow you to analyze the knowledge of all students even across different test forms. You can also use this spreadsheet for free response items (ex: items 11 and 12).

Where to find the most common mistakes your students will make:

1) Your students: Daily: During class discussion or student activities, take note of how students explain and talk about concepts. Previous Assessments: While grading homework, quizzes, and tests take note of the most common errors your students make and misconceptions your students have about particular operations or topics.

2) Research-based resources: IES Practice Guides; Adding It Up; And many more…

Summing it All Up:
When writing any assessment, it is important to include items that test students’ conceptual understanding, procedural fluency, strategic competence, and adaptive reasoning skills because each of these components is equally important in their overall math proficiency. When writing items for multiple-choice tests, make sure to be purposeful in the response options you include.

National Research Council. (2001). Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford, and B. Findell (Eds.). Mathematics Learning Study Committee, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Siegler, R., Carpenter, T., Fennell, F., Geary, D., Lewis, J., Okamoto, Y., Thompson, L., Wray, J. (2010). Developing effective fractions instruction for kindergarten through 8th grade (NCEE 2010-4039). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/practiceguide.aspx?sid=15