Analyzing Assessment Items

By Dr. Pooja Shivraj, RME Educational Assessment Researcher

Much of the work we do at Research in Mathematics Education involves the development of assessments used by educators to identify students who may be struggling with algebra-readiness knowledge and skills, so that teachers can provide additional instructional support. The research process we use is rigorous and begins with an assessment blueprint, then item writing, internal reviews and external expert reviews, followed by a pilot test and finally the development of the test forms. The pilot test is given to a large number of students in order to determine the validity of the assessment items. Our researchers receive the results of the pilot and perform an extensive statistical analysis to determine if an item is good, psychometrically speaking.

The point of obtaining item statistics is to develop a pool of items that function well from which future tests can be designed. There are two kinds of analyses that can be performed: a Classical Test Theory (CTT) analysis, which is sample-dependent and non-model based, or an Item Response Theory (IRT) analysis, which is sample-independent and model-based. Regardless of the type of analysis performed, three primary statistics are used to determine if an item is psychometrically good. The ranges listed below are the acceptable norms found in the literature.

(1) The item should have a strong correlation between each item score and the total score. In other words, the correlation should show that the test-takers choosing the correct answer on the item are likely to receive a higher score. This statistic is measured by the point-biserial correlation (CTT) or the point-measure correlation (IRT). A good item would have a point-biserial correlation of >0.2 or a point-measure correlation of >0.25.
(2) The difficulty of the item, measured by the proportion of students answering the item correctly (CTT), should be between 30% to 80% of the test-takers. In IRT, the difficulty parameter, b, should be between -4 and +4.

An item characteristic curve depicting the discrimination parameter
(a) and the difficulty parameter (b) in an IRT model

(3) The discrimination of the item, also measured by the point-biserial correlation (CTT) should be higher for the correct response than the distractors. In IRT, the discrimination parameter, a, should be between 0.5 and 1.5. The greater the discrimination, the better the item discriminates between lower ability and higher ability students.

What can you do with items that don't function well? For the items that don't function well, reviewing the data would be the first step. Are the items functioning poorly because the majority of students are choosing the correct answer? Is one distractor not being chosen at all? Are the majority of students choosing a single distractor more often than other options? These data would all be red flags. The next step would be to review the content of all the items that don't function well, especially the items that were flagged in the previous step. What about the content led students to choose or not choose a particular response choice?

Using this process of analyzing data, reviewing items, and adjusting the content of the items, a pool of items that function well can be developed for use in the future.

Note: Many other statistics (e.g., fit statistics in IRT like Chi square, infit, outfit, etc.) could be used to determine if an item functions well in addition to the ones described above that could also provide information at the test level. Please feel free to email me if you would like more information at pshivraj@smu.edu.

Project PAR: Promoting Algebra Readiness

By Dawn Woods, RME Elementary Mathematics Coordinator

Many students appear to be on-track for mathematics achievement in 4th grade, but exit 8th grade without having developed critical skills in the area of rational numbers (National Center for Education Statistics, 2009). Conceptual understanding of rational numbers, as well as, their symbolic representations is a critical component for understanding everyday situations in algebra. Students must master these foundational skills and concepts at the elementary and middle school levels. Project PAR: Promoting Algebra Readiness is an intervention curriculum designed to build this rational number understanding.

Project PAR is a three-year, Institute of Educational Sciences (IES) funded research study that is working to develop a strategic intervention on rational number concepts that use evidence based strategies. The purpose of this project is to promote algebra readiness for sixth grade students by developing students’ conceptual understanding of rational numbers on a number line. The project team consists of researchers and curriculum experts from the University of Oregon (UO) and Research in Mathematics Education (RME) at Southern Methodist University who have extensive experience designing math interventions for a range of student learners as well as vast teaching experience in the mathematics classroom.

Project PAR is completing the development phase of this study where curriculum writers from the UO and RME have designed the scope and sequence for the intervention, developed approximately 100 print-based lessons, and conducted preliminary feasibility testing of individual lessons. The project is now moving into the implementation phase where classroom intervention teachers in Texas and Oregon are teaching the lessons. At this time curriculum writers and researchers will determine if the lessons have realistic expectations and goals for classroom use as the teachers use the lessons and provide critical feedback. This summer, curriculum writers will revise the curriculum based on the results of the feasibility study in preparation for the pilot study scheduled for the 2014-2015 school year. During the pilot study, the potential promise of the intervention increasing student achievement will be examined.

RME would like to thank the sixth grade math teachers Bush Middle School in Carrollton-Farmers Branch ISD who opened up their classrooms for the preliminary feasibility testing as well as the sixth grade math teachers at Fowler Middle School in Frisco ISD who are implementing the PAR curriculum during the feasibility study. We could not do our work with out the support of great teachers at great schools who are putting evidence- based strategies into practice!

National Center for Education Statistics (2009). The nation’s report card: Mathematics 2009. Washington, DC: National Center for Education Statistics

Insight to ‘Teaching Math to Young Children’

By Sharri Zachary, RME Mathematics Research Coordinator

The National Research Council (NRC) and the National Council for Teachers in Mathematics (NCTM) describe two fundamental areas of mathematics for young children: 1) Number and Operations, and 2) Geometry and Measurement. According to the NRC (2009), conceptual development within number and operations should focus on students’ development of the list of counting numbers and the use of counting numbers to describe total objects in a given set. It is recommended that teachers provide students with opportunities to “subitize small collections [of objects], practice counting, compare the magnitude [size] of collections, and use numerals to quantify collections” (Frye et al., 2013). Conceptual development in geometry and measurement should support the idea that geometric shapes have different parts that can be described and include activities that model composition and decomposition of geometric shapes.
The Institute of Educational Sciences (IES) released a practice guide recently on Teaching Math to Young Children. The recommendations put forth in the IES practice guide are:

1. Teach number and operations using a developmental progression
2. Teach geometry, patterns, measurement, and data analysis using a developmental learning progression
3. Use progress monitoring to ensure that math instruction builds on what each child knows
4. Teach children to view and describe their world mathematically
5. Dedicate time each day to teaching math, and integrate math instruction throughout the school day

These recommendations are intended to:

• Guide teacher preparation that will result in later math success for students
• Provide descriptions of early content areas to be integrated into classroom instructional practices
• Assist in the development of curriculum for students in early grades

To access/download the full IES practice guide, please visit http://ies.ed.gov/ncee/wwc/PracticeGuide.aspx?sid=18

Frye, D., Baroody, A. J., Burchinal, M., Carver, S. M., Jordan, N. C., & McDowell, J. (2013). Teaching math to young children: A practice guide (NCEE 2014-4005). Washington, DC: National Center for Education Evaluation and Regional Assistance (NCEE), Institute of Education Sciences, U.S. Department of Educa-tion. Retrieved from the NCEE website: http://whatworks.ed.gov

National Research Council. (2009). Mathematics learning in early childhood: Paths toward excellence and equity. Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press

How to Write a Smart Test

By Beth Richardson, RME High School Mathematics Coordinator

My career in education began as a high school math teacher. Throughout my teaching, I wrote countless math “questions” to check my students’ understanding, from daily bell-ringers to full-length tests. However, it wasn’t until I became immersed in the world of assessments that I learned some important components of a well-written test. First of all, the “questions” that make up a test are commonly referred to as items by researchers in the field of assessment, which is what I’ll call them from here on.

Test Math Knowledge in Different Ways

There are many different levels in which the brain engages in mathematical concepts. The book Adding It Up: Helping Children Learn Mathematics (2001) identifies five specific types of thinking that together determine a person’s proficiency in math. Here’s a brief explanation of each and example of items with the same skill (slope) assessed at the different proficiency levels:

• Conceptual Understanding – comprehension of mathematical concepts, operations, and relations
• Procedural Fluency – carrying out procedures flexibly, accurately, efficiently, and appropriately
• Strategic Competence – ability to formulate, represent, and solve mathematical problems 
• Adaptive Reasoning – capacity for logical thought, reflection, explanation, and justification
• Productive Disposition – habitual inclination to see math as sensible, useful, and worthwhile, coupled with a belief in one’s own efficacy. 

Sample Conceptual

Sample Procedural

Sample Strategic

Sample Adaptive

As teachers, we can only test the first four of these in a traditional test setting. However, productive disposition is something you can learn about each of your students as you interact with them daily.

Multiple-Choice Tests

Basic Multiple-choice Item Components:

• Skill: Comes from TEKS, district curriculum, etc.
• Mathematical Proficiency Level: Procedural, Conceptual, Strategic, or Adaptive
• Stem (Text/Graphic): Make sure the text and graphics you use are purposeful and relevant to the underlying mathematical skill/concept being assessed
• 4 Response Options: 1 correct response and 3 distractors that are well-thought out - no throw away distractors!

Write Plausible Distractors

For multiple-choice tests, the responses you provide are just as important as the question you ask.

Take the time to write distractors that are based on students’ common mistakes and misconceptions. To help ensure the distractors are plausible, write a rationale for each distractor. Also, avoid using give-away distractors that do not relate to the item. Here’s an example of a spreadsheet that can be used when writing a test. This spreadsheet can easily be copied and changed to create multiple forms of the same test. The specific details in the stem can be changed, but the same distractor rationales can be used. This will allow you to analyze the knowledge of all students even across different test forms. You can also use this spreadsheet for free response items (ex: items 11 and 12).

Where to find the most common mistakes your students will make:

1) Your students: Daily: During class discussion or student activities, take note of how students explain and talk about concepts. Previous Assessments: While grading homework, quizzes, and tests take note of the most common errors your students make and misconceptions your students have about particular operations or topics.

2) Research-based resources: IES Practice Guides; Adding It Up; And many more…

Summing it All Up:
When writing any assessment, it is important to include items that test students’ conceptual understanding, procedural fluency, strategic competence, and adaptive reasoning skills because each of these components is equally important in their overall math proficiency. When writing items for multiple-choice tests, make sure to be purposeful in the response options you include.

National Research Council. (2001). Adding it up: Helping children learn mathematics. J. Kilpatrick, J. Swafford, and B. Findell (Eds.). Mathematics Learning Study Committee, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

Siegler, R., Carpenter, T., Fennell, F., Geary, D., Lewis, J., Okamoto, Y., Thompson, L., Wray, J. (2010). Developing effective fractions instruction for kindergarten through 8th grade (NCEE 2010-4039). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/practiceguide.aspx?sid=15

All Resources are Not Created Equal! A Closer Look at Algebra I Textbooks

By Dr. Candace Walkington, Assistant Professor of Mathematics Education & Elizabeth Howell, RME Research Assistant

Quite often teachers inherit the textbooks that will be used as a primary teaching tool and resource in their classrooms. Textbook choices may be district-level decisions or may be decided in a multi-year cycle. As a classroom teacher, the decision may be out of your immediate control. However, as the ultimate instructional leader in your classroom, it IS possible to use current research about textbooks in order to improve outcomes for your students, regardless of the required curriculum.

Background
Experts agree that algebra is a crucial course in students’ mathematical trajectory, and success in this course has been identified as important to college and career readiness (Stein, Kaufman, Sherman, & Hillen, 2011; Cogan, Schmidt, & Wiley, 2001; Kaput, 2000; Moses & Cobb, 2001). A key learning goal in algebra is the use of symbols to represent and analyze situations and problems, and many textbooks are written with a heavy emphasis on symbolic usage and presentation.

However, working with symbolic representations is challenging for students (Walkington et al., 2012), and evidence suggests that students actually have improved learning when they first learn about a new concept using concrete and familiar formats (Goldstone & Son, 2005) - like verbally presented algebra story problems. By giving students story scenarios first, instead of symbols alone, we can draw on the things they already know and understand. Over time, these verbal supports can be faded, as students begin to understand and work with symbols more.

Textbook Classifications
Current Algebra I textbooks are classified as being traditional or reformed. Traditional texts have the lion's share of the textbook market (Holt, Pearson, Saxon, etc.), and introduce concepts by showing definitions and worked examples, and then presenting problem sets. These texts are typically expected to go along with a teacher-directed approach to instruction. Reformed textbooks are more rare in Algebra I, and often follow NCTM standards for reformed teaching, taking a student-centered approach. They may present students with more complex, open-ended problems or mathematical investigations, and accentuate the use of problem-based learning. Differences have emerged in the way that traditional and reformed curricula introduce the use of symbols in Algebra I, and many reformed textbooks in mathematics have taken less of a symbolic approach and adopted a more verbal presentation style.

Example of problem presented in VERBAL format:

• Maria just got a new cell phone, and on her phone plan each text message she send costs $.10. Write an algebraic expression that relates the number of texts Maria sends to the cost in dollars. How much will it cost to send 7 texts?

Example of problem set presented in SYMBOLIC format:

• Solve for y when x = 7:
  y = 3x + 5
  y = 0.25x
  y = 2x - 3

Current Research
In a recent study, researchers found that the presentation format of the examples and homework problem sets in commonly used Algebra I textbooks varied depending on the type of textbook, traditional or reformed (Sherman, Walkington, Howell, 2013). Reformed texts favored a verbal presentation first, and this verbal first approach was faded over time in the text. Thus in reformed texts, when students are first learning about algebra, they get a lot of verbal problems, but as their expertise develops, they get more symbolic problems.

Traditional texts favored symbols first - in each section, symbolic problems were presented to students before verbal problems. Traditional texts also had fewer single format only sections - there were fewer sections that had only verbal problems, or only symbolic problems. Most traditional texts contained a mixture of symbolic and verbally presented problems in the homework, yet the instructional examples provided in the text trended toward symbolic. Other research has also suggested that in traditional texts, the student recommended exercises in the teacher’s edition sometimes excluded the verbal problems from the students’ assignment.

Implications
Because many schools are using traditional textbooks, it is likely that your district adopted text has a heavy prevalence of symbol-first presentation. Be aware of the challenges that this approach may present to your beginning Algebra I students, and use verbal presentations in your initial classroom examples whenever possible. When assigning homework, be aware that the recommended exercises may exclude all of the rich verbal contextual problems, and add a few back in to your homework assignment. Better yet, choose a few to discuss and work on together as a class.

The choice of an Algebra I textbook may not be a decision that you can determine, but how to best use the examples and problems presented in the book is always your choice as the teacher. What type of textbook are you currently using? Take a closer look, and be aware of the presentation style that the book favors. If verbal scaffolding is not prevalent, it is easy to add those supports back in to help your students to succeed. A closer look will help you to see those places in the curriculum where a verbal presentation could be beneficial.

Resources:
Texas State Adopted Textbooks: http://www.tea.state.tx.us/index2.aspx?id=2147499935

References:
Cogan, L.S., Schmidt, W.H., & Wiley, D.E. (2001). Who takes what math and in which track? Using TIMMS to characterize U.S. students’ eighth-grade mathematics learning opportunities. Educational Evaluation and Policy Analysis, 23, 323-341.

Goldstone, R., & Son, J. (2005). The transfer of scientific principles using concrete and idealized simulations. Journal of the Learning Sciences, 14(1), 69-110.

Kaput, J. J. (2000). Teaching and learning a new algebra with understanding. U.S.; Massachusetts: National Center for Improving Student Learning and Achievement.

Moses, R., & Cobb, C. (2001). Radical Equations: Math Literacy and Civil Rights. Boston: Beacon Press.

Sherman, M., Walkington, C., & Howell, E. (April, 2013). A comparison of presentation format in algebra curricula. Presented at National Council of Teachers of Mathematics Research Pre-session, Denver, CO.

Stein, M.K., Kaufman, J. H., Sherman, M., Hillen, A.F. (2011). Algebra: A Challenge at the Crossroads of Policy and Practice. Review of Educational Research, 81(4), 453-492.

Walkington, C., Sherman, M., & Petrosino, A. (2012). ‘Playing the game’ of story problems: Coordinating situation-based reasoning with algebraic representation. Journal of Mathematical Behavior, 31(2), 174-195.

Help Your Students Experience Fractions Conceptually

By Dawn Woods, RME Elementary Mathematics Coordinator

Many students find fraction concepts difficult to understand yet the understanding of fractions is essential for learning algebra and advanced mathematics (National Mathematics Advisory Panel, 2008). As an elementary mathematics educator, I noticed that many of my students struggled with fraction concepts across the curriculum. I wondered how I could help my students experience and understand fraction concepts conceptually so they could succeed not only in my classroom but also in advanced mathematics.

My search for answers began with research. I discovered that to understand fractions means to recognize the multiple meanings and interpretations of fractions. Furthermore, I needed to explicitly present these different constructs in a contextual way to build understanding. Mathematics educators generally agree that there are five main fraction constructs and that they are developmental in nature.

The first construct presents fractions as parts of wholes or parts of sets. Research suggests that this construct is an effective starting point for building fractions (Cramer & Whitney, 2010). However, it is important to realize that the part-whole relationship goes way beyond the shading of a region. For example, it could be part of a group of animals such as (¼ of the animals are dogs), or be part of a length, (we ran 1 ½ miles) (Van De Walle, Karp, Bay-Williams, 2013).

Researchers such as Cramer, Wyberg, and Leavitt suggest that the fraction circle manipulative is a powerful concrete representation since it helps to build understanding of the part-whole relationship as wells as the meaning of the relative size of fractions (2008). Here, they use fraction circle models to help build mental images that aid in the ability to judge relative sizes of fractions. It is also important to remember that the fractional parts do not need to be identical in shape and size, but must be equivalent in some other attribute such as area, volume, or number (Chapin & Johnson, 2006). However, it is important to teach beyond this first construct to include other fraction representations and models.

The second construct presents fractions as measures. Measurement (Van De Walle, Karp, Bay- Williams, 2013) involves identifying a length and then uses that length to determine a length of an object. The number line plays an important role in this construct by partitioning units into as many subunits that one is willing to create (Chapin & Johnson, 2006). For example, in the fraction ¾, you can use the unit fraction ¼ as the selected length and then measure to show that it takes three of those to reach ¾ (Van De Walle, Karp, Bay-Williams, 2013). Research suggests that students who develop an initial understanding of rational numbers as measures, develop ideas of unit, partitioning, order, addition and subtraction (Cramer & Whitney, 2010) while using the number line as a model. Essentially, this powerful construct illustrates that there are an infinite number of rational numbers on the number line as it focuses on how much rather than parts of a whole.

Fractions can also result from dividing two numbers. This construct is often called the quotient meaning, since the quotient is the answer to a division problem (Chapin & Johnson, 2006). Think about the number of cookies each person receives when 15 cookies are shared between 3 people. This problem is not a part-whole scenario (Van De Walle, Karp, Bay-Williams, 2013) but it still means that each person will receive one-third of the cookies expressed as ¹⁵⁄₃, ⁵⁄₁, or 5. Connecting division to fractions enables students to feel comfortable with seeing division expressed in multiple ways such as 16 ÷ 3, ¹⁶⁄₃, and 5¹⁄₃ and is important for continued success in advanced mathematics.

The fourth construct presents fractions as operators. In this construct, a fraction is a number that acts (or operates) on another number to stretch or shrink the magnitude of the number (Chapin & Johnson, 2006). For an example, a model of a car may be 1/16 the size of the original or a cell maybe magnified under a microscope to 400 times the actual size demonstrating a multiplicative relationship between the quantities. This construct takes fractions beyond representation to a place where students know how to use fractions to solve problems across the curriculum.

The fifth and final construct characterizes fractions as the ratio or comparison of two quantities. A ratio such as 1/3 can mean that the probably of an event is one in three (Van De Walle, Karp, Bay-Williams, 2013). Or a ratio can also represent part-whole relationships such as 11 children at the park compared to the total number of 18 people. We could write this part-to-whole relationship as the fraction ¹¹⁄₁₈. However, it is important that realize that all fractions are ratios but not all ratios are fractions (Chapin & Johnson, 2006). Part-to-part comparisons such as the number of children to the number of people at the park, 11:18, is not a fraction because this comparison does not name a rational number but presents a comparison of two numbers.

Fraction understanding, although a challenge to students, is a critical mathematics concept. For students to really understand fractions, they need to experience fractions across all five constructs in meaningful ways that build conceptual understanding. This conceptual understanding, in turn, provides students with mental representations that enable students to connect meaning to fractions across a variety of contexts.

Chapin, S.H., & Johnson, A. (2006). Math matters, 2nd edition. Sausalito: Math Solutions Publications.

Cramer, K., & Whitney, S. (2010). Learning rational number concepts and skills in elementary school classrooms. In D.V. Lambdin & F.K. Lester, Jr. (Eds.), Teaching and learning mathematics: Translating research for elementary school teachers (pp. 11-22). Reston, VA: NCTM

Cramer, K., Wyberg, T., & Leavitt, S. (2008). The role of representations in fraction addition and subtraction. Mathematics Teaching in the Middle School 13(8), 490-496.

National Mathematics Advisory Panel (2008). The final report of the national mathematics advisory panel. Jessup, MD: Education Publications Center. Retrieved from http://www2.ed.gov/about/bdscomm/list/mathpanel/report/final-report.pdf.

Van De Walle, J.A., Karp, K.S., & Bay-Williams, J.M. (2013). Elementary and middle school mathematics: Teaching developmentally, 8th edition. Boston: Pearson.

Focus on Research: A Discussion on Learning Progressions for Instruction and Assessent

By Dr. Deni Basaraba, RME Assessment Coordinator

The need for differentiated instruction to meet the needs of all learners is one source of evidence that students’ learning is not linear and that not all students follow the same learning pathway to mastering content. Learning progressions can be used to describe the successively more sophisticated ways student think about an idea as a student learns, providing a description in words and using examples of what it means to move over time toward a more “expert” understanding of a given topic or content area (Duschl, Schweingruber, & Shouse, 2007).

In addition to including descriptions of students’ understanding as they move from novice to expert understanding, learning progressions also often include descriptions of common misconceptions students may have about the content of interest that may hinder or impede their understanding; these misconceptions can then provide the focus for targeted instruction (Alonzo & Gearhart, 2006).

The complexity associated with learning new content, because it is not linear or the same for every student, is best represented graphically as a complex map or network of connections and interactions rather than a linear path; this complex map allows for the fact that there is no “best” pathway and that some students may take one path in their learning than others to attain proficiency with the same content. A map of a sample learning progression will show not only the development and sophistication of students’ thinking as they move in the learning progression (i.e., increasing in sophistication of their skills and understanding) but will also represents an interaction and integration of knowledge.

In addition to relatedness among constructs in the learning progression, there are also connections of the knowledge and skills between one skill and the next. For example, if the target strategy for a level of a learning progression is the ability to recall multiple
strategies for single-digit addition (e.g., making tens, doubles), the perquisite skill might be a count on strategy whereby students can count on from an initial term (e.g., 5) to make a larger number (e.g., 5, 6, 7, 8). Finally, the most foundational skill in this hypothesized learning progression might be the ability to count all, that is, start from counting at 1 all the way to the desired sum (e.g., When asked what 5 + 3 equals the student starts counting from one – 1, 2, 3, 4, 5, 6, 7, 8).

How can learning progressions inform instruction and assessment?
Learning progressions can be a critical cog in the machinery of instruction and assessment. If, for example, we know that learning progressions provide ordered descriptions’ of students’ understanding, we can then use that information to help identify the “landmarks” or essential knowledge and skills students will need to learn as part of the math content, which can be used to help with instructional planning (e.g., what content to teach and when to teach it).

In addition, because learning progressions often include descriptions of the target knowledge and skills as well as common misconceptions or errors in students’ thinking we hypothesize may be interfering with students’ acquisition of a particular skill or mastery with specific content, learning progressions can provide valuable insights to how students think about the content of the learning progression. Together, these pieces of information can be used to help determine an appropriate sequence for the content of instruction (e.g., focusing first on foundational, prerequisite skills that gradually increase in complexity) as well as to develop classroom-based assessment items that focus on knowledge and skills that have been taught during instruction.

Alonzo, A. C., & Gearhart, M. (2006). Considering learning progressions from a classroom assessment perspective. Measurement: Interdisciplinary Research & Practice, 14(1-2), 99-104.

Duschl, R. A., Schweingruber, H. A., & Shouse, A. W. (Eds.) (2007). Taking science to school: Learning and teaching science in grades K-8. Washington, DC: National Academies Press.

Invented Strategies

By Saler Axel, RME Research Assistant 

In traditional classrooms, students are often taught one or two strategies for whole-number computation. They memorize rules to compute different operations. At first glance, teachers may mistakenly think their students “get” how to compute. What is often the case though is that students may be able to compute using a tried and true method, but cannot explain why it works. When students attempt cousin items that do not read exactly the same as what they are used to seeing, they may struggle or may not correctly calculate an answer.

When students learn how to compute algorithms, but do not learn the concepts behind them, they miss important stepping stones. By teaching students how to invent strategies, they learn what methods work best for them and which will better serve them in the “real-world.”

According to John Van de Walle in Elementary and Middle School Mathematics: Teaching Developmentally, invented strategies positively impact students’ academic success.

1. Students make fewer errors. When students compute with strategies they understand, they make fewer errors. When students make errors and do not understand the concepts behind their actions, they may have a far more difficult time fixing their efforts.
2. Less re-teaching is required. Teaching conceptual understanding is time consuming, but worth the effort! Not only can students gain the strategies necessary to be more successful in mathematics, the time spent teaching them is meaningful. When students know the “how” of computation but not the “why,” more re-teaching is necessary to help students develop computational skills.
3. Students develop number sense. Students’ development and use of algorithms provide a deeper understanding of the number system.
4. Invented strategies are the basis for mental computation and estimation. Mental computations are invented strategies. When students are taught how to use invented strategies, they are being taught mental computation. There is therefore little need to provide direct lessons in other computational formats or how to do mental math.
5. Flexible methods are often faster than the traditional algorithms. Van de Walle provides the following example to clarify: Consider the product 64 x 8. An invented strategy may be to calculate 60 x 8 = 480 and 8 x 4 = 32. Then find the sum of 480 + 32 which is 500 + 12 which equals 512. A student that uses a traditional algorithm will likely spend more time than someone that uses an invented strategy such as the one above.
6. Algorithm invention is itself a significantly important process of “doing mathematics.” When students invent successful computation strategies, their confidence in mathematics is strengthened. Younger students have been traditionally taught to compute algorithms without understanding why they work or being given the latitude to create their own methodologies. Van de Walle suggests that by opening the door to invented strategies, elementary students gain a valuable view of “doing mathematics.”

Van de Walle gives some examples of invented strategies with multiplication, such as useful visual representations, complete-number strategies (23 x 6 = 23 + 23 + 23 + 23 + 23 + 23 = 138), partitioning strategies, compensation strategies, and using multiples of 10 and 100.

By shifting your practice from teaching students traditional methods to increasing students’ awareness of how computation works, you can provide a solid foundation to enabling the use of invented strategies in mathematics.  As you teach, remember that more math drills is not the answer. Find which facts the student is struggling with and what current strategies they are using on the facts they do know. Break students into teams and challenge them to come up with multiple ways to solve a problem while always explaining how they got the answer. Being able to explain how students came to their answer is essential.

Consider your students. What strategies will you use to best encourage their mathematical thinking and “doing?”

Van de Walle, J. A., Karp, K. S., Bay-Williams, J. M. (2013). Elementary and middle school mathematics: Teaching developmentally. Upper Saddle River, NJ: Pearson Education, Inc. 

Math Educational Apps for the iPad

By Savannah Hill, RME Professional Development Coordinator

Apple iPads have taken the US by storm. One of the major markets they push for is education. If you are new or unfamiliar to this technology, here are some additional resources to help you get started.

You may ask, "Is it really a valuable tool for me to use in my class?" There have been several research studies done on this topic, but here are a couple.

• In a study done by Houghton Mifflin Harcourt in California showed that students using iPads saw their math test scores increase 20% in one year compared to students using traditional textbooks (Bonnington, 2012).
• A study centered on Motion Math has shown that the iPad can help with fundamental math skills. Fifth graders who regularly played the game for 20 minutes per day over a five-day period increased their test scores by 15 percent on average (Riconscente, 2011). Click here for the final report.

During the lunch hour at the RME Conference, we explored some mathematics educational apps for the iPad and had many requests for more information about these. Here is a little more information about them!

• Motion Math: Hungry Fish (Free version) Students can practice mental addition and subtraction with this app. The fish are hungry for numbers. Students can make sums by pinching two numbers together - instant addition! Keep feeding the fish to win a level and unlock new colors and fins. Also check out Motion Math: Wings - This game is for children ages 4 and up to develop a conceptual understanding for multiplication. Students play by tilting your bird to the bigger number. Students will master multiplication in 6 different visual forms: rows of dots, clusters of dots, groups of dots, a grid, a labeled grid, and symbols.
• Click here for Motion Math's website to learn more about all of their apps.

• MathBoard (Free version) MathBoard is a highly configurable math app for students in kindergarten through elementary school, addressing simple addition and subtraction to multiplication and division. More than just standard drills, MathBoard encourages students to actually solve problems, and not just guess at answers. This is done by providing multiple answer styles, as well as scratchboard area where problems can be worked by hand.

• Chicken Coop Fraction Game In this hilarious educational game you will be shown a fraction and your job is estimate the decimal equivalent by placing a nest on a number line. Our hens are mathematical experts and they will fire their eggs towards the correct answer. If your estimate is good the eggs will be caught in the nest but if you’re too far out it all gets very messy.
• Teach Me 2nd Grade ($1.99) This app keeps children engaged with a unique reward system where children earn coins by playing learning games at the school. The simple and intuitive user interface is designed to be child friendly, which allows children to play with help from the teacher. An animated teacher gives verbal instructions and feedback to encourage the child to learn and succeed! In addition, the learning screens are colorful, fun and rotate between six different subjects so children don't get bored.

For a table of all of the available apps to use for mathematics education, visit TCEA's iPad List and click the mathematics tab. This list is organized into concepts and gives the price of the app with a description.

Bonnington, C. (2012, January 23). iPad a solid education tool, study reports. CNN Tech. Retrieved from  http://www.cnn.com/2012/01/23/tech/innovation/ipad-solid-education-tool 
Riconscente, M. (2011) Mobile learning game improves 5th graders' fraction knowledge and attitudes. Los Angeles: GameDesk Institute.Tweet

Using PLC's to Strengthen Math Content Knowledge

By Dr. Janie Schielack and Dinah Chancellor

Professional Learning Communities can strengthen a group of educators in various capacities providing a framework intended for professional growth but also an environment that fosters personal learning that effectively improves their individual teaching practice. This level of engagement with fellow teachers encourages a trickle-down effect, whereby teachers apply the same process as students work through an activity. When this level of engagement with students is consistently practiced, a deeper level of understanding by the student is achieved.

• Do you currently use PLC's? How are they useful to you? 
• If you do not, how do you think they could benefit your own teaching practice?
• How might students benefit from a teacher who is in a PLC?

What’s in a Name? Think-a-Loud Protocol

By Dawn Woods, RME Elementary Mathematics Coordinator, and Marilea Jungman, RME Project Specialist

In January, Research in Mathematics (RME) conducted think-aloud protocols with second, third and fourth grade students at Nebbie Williams Elementary School in Rockwall ISD. The school volunteered to participate in the think-aloud protocol study that was designed to enrich the development of the ESTAR Universal Screener.

Background: What is the ESTAR Universal Screener?

The Elementary Students in Texas Algebra Ready, or ESTAR, is the latest intitative within the Texas Algebra Ready (TXAR framework) to support elementary students in the state of Texas to achieve a high level of preparedness in mathematics. The ESTAR Universal Screener is being designed to help educators identify students who may need additional support in becoming algebra-ready in the elementary grades and will be aligned with algebra-readiness knowledge and skills articulated in the revised Texas Response to the Curriculum Focal Points.

This document, based on the revised TEKS adopted in April 2012, identified critical areas of mathematics instruction in a framework for sequencing and developing curricula at each grade level. This document provides the content of the ESTAR Universal Screener and will be organized around foundational, bridging, and target knowledge and skill levels and simultaneously includes items written to target four levels of cognitive complexity - research indicates 4 areas critical for mathematics success: procedural understanding, conceptual understanding, strategic competence, and adaptive reasoning. Data generated from the screener will be reported in a format that helps teachers make informed decisions about the content and structure of mathematics instruction in the classroom.

Why is the ESTAR Universal Screener important?
Although performance standards are in process of being established for the State of Texas Assessments of Academic Readiness (STAAR), data from 2012 indicate that 3rd grade students responded, on average, to only 30 of 49 mathematics items correctly (61%) while 4th grade students responded, on average, to only 32 of 48 (or 66%) of mathematics items correctly (Texas Education Agency, 2012). These data speak to a need for early identification of students who may be struggling to learn critical mathematics content. One of the research steps in developing the Universal Screener is to conduct student interviews, also known as think-aloud protocols.

What is a Think-Aloud Protocol?
The purpose of a student interview or think-aloud protocol is to transform a student’s covert thinking process into an observable behavior so that the thinking process can be documented and analyzed (van Someren, Barnard, & Sandberg, 1994). Basically, we ask the student to work through a small number of math items appropriate to their grade level, and to “think-aloud” as they work. This concurrent data capture is maximized through the notes and reflections of the interviewer, the use of audio/visual, and a field observer dedicated to recording the student’s thoughts, hesitations, and gestures verbatim.

Once the student solves a math item, the interviewer asks the student to reflect on his or her thinking process after the task is completed. This is called retrospective data collection. Here, the interviewer uses questioning, prompting or dialogues to encourage the student to talk about his or her thoughts about the math item. The repetitive nature in the questioning allows student’s initial thoughts to be repositioned, and in many instances, a student alights on the correct answer after a first-round wrong choice.

We use the student interviews to verify and provide validity evidence of misconceptions captured in item designs as well as learn how metacognition (Flavell, 1979) plays a role in the planning and strategies students use in mathematical problem solving. Furthermore, student interviews also provide valuable information about students’ sense of self-efficacy, which may be correlated to student’s academic achievement (Hackett & Betz, 1989) and predicts later success for elementary school students (Bandura, 1997; Joet, Usher, & Bressoux, 2011).

Summing It Up
Think-aloud protocol is a valuable qualitative research method that enables researchers to uncover and map thinking processes. As we begin to analyze our data from our project at Nebbie Williams Elementary School, we hope to not only provide validity to our items for the ESTAR project, but gain valuable insight into how students think about math and how metacognition and self-efficacy play a role in students mathematics achievement.

Thank you again to the wonderful teachers that helped us with our project!

3rd Grade
Ms. Jennifer McCurry and Ms. Melody Carrilo

4th Grade
Ms. Christine Gregory and Ms. Lana Edwards

2nd Grade
Dr. Marcella J. Hodges and Ms. Kathleen Elam

Bandura, A. (1997). Self-efficacy: The exercise of control. U.S.A.: Macmillan.  

Flavell, J. H. (1979) Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist 34(10). 906-911.  

Hackett, G. & Betz, N. E. (1989) An exploration of the mathematics self-efficacy/mathematics performance correspondence. Journal for Research in Mathematics Education 20(3). 261-273.  

van Someren, M. W., Barnard, Y. F., & Sandberg, J. A.C. (1994). The think aloud method: A practical guide to modeling cognitive processes. London: University of Amsterdam, Department of Social Science Informatics. 

Tweet

Differentiate Instruction for All Students

By Savannah Hill, RME Professional Development Coordinator

In the article 7 Steps to High-End Learning in Teaching Children Mathematics, M. Katherine Gavin and Karen G. Moylan describe seven steps to help teachers present differentiated instruction to all students. They have been working with the National Science Foundation on Project M² to research how this is best executed in the classroom.

From Microsoft Office

From field-testing high-level, differentiated geometry and measurement curriculum units on diverse populations of kindergartners, first graders, and second graders, they have found that all students are capable of developing deep understanding when each lesson is differentiated to accommodate the variety of student abilities, interests, and prior experiences.

They share seven steps teachers can implement in their classroom to help differentiate instruction for all students.

1. Select an appropriate task. Since research has shown that students are able to justify thinking at high levels, begin with an advanced concept in order to allow for opportunities to differentiate and support students.

2. Increase expectations for all students. Provide concepts that will challenge all students. Allow for activities that challenge high learners and can be differentiated with scaffolding for those who may need extra support. The National Council of Teachers of Mathematics (NCTM) advocates that challenging mathematics curriculum should be provided for young students (2002). Their study found large gains from pre- to post-testing on all students and found that they significantly outscored a comparison group.

3. Facilitate class discussions about the concepts. Encourage students to justify their reasoning and generate classroom discussion in order to help students work to understand each other’s ideas and come to a conclusion on correct answers. This will allow teachers to gain insight about students thinking, any misconceptions they may hold, and in turn, allow the teacher to better differentiate instruction.

4. Encourage all students 
to communicate their thinking 
in writing. When students create written representations of their work, such as in words, pictures, or tables, they are challenged to explain their thinking in ways that others can understand. This can also allow the teacher to have insight on the students’ thought process. Teach your students to practice “writing” out their math by having group responses on a question and then have students write with partners or independently, scaffolding where needed.

5. Offer additional support. Teachers can create “hint” cards to differentiate instruction for those students who are struggling when working a problem. The card, which teachers can drop off on a desk as students work, can include a definition with a picture, a question to connect prior concepts to current ones, or a way of modifying the activity.

6. Provide extended challenges. Teachers can also create challenge cards that can be shared in the same way as hint cards, but challenges those high performing students.

7. Use formative assessment to inform instruction. Make sure to analyze students’ thinking in case instruction needs to be adjusted to correct misconceptions before giving any final assessment. Gavin and Moylan suggest using open-ended questions focusing on the essential math concepts.

They encourage teachers to start small. Pick one or two current lessons and differentiate instruction using the hint or challenge cards. After you have tried it once, reflect on how it went, and try again!

Summing it All Up 
Differentiation is a way for all students to access high-level mathematics. It can easily be done in your classroom one step at a time. Check out their full article in the October issue of Teaching Children Mathematics and see examples of how they executed these steps in their research.

What are ways you have differentiated instruction for all students in your classroom?

Gavin, M. K. & Moylan, K. G. (2012). 7 steps to high-end learning. Teaching Children Mathematics, 19(3). 184 – 192.

National Council of Teachers of Mathematics (NCTM). 2000. Principles and Standards for School Mathematics. Reston, VA: NCTM.

Tweet

Welcome to Our Blog!

We are the Research in Mathematics Education (RME) unit at Southern Methodist University, located in Dallas, Texas.  Our goal as a research and outreach unit is to provide evidence-based support to improve students’ mathematics achievement in North Texas and across the country. 

Courtesy of Southern Methodist University.

We have launched this blog with the hope of cultivating a forum for discussing the critical issues that face educators and offering tools in the form of research, peer review and discussion, and professional outreach opportunities. You will hear from a variety of the faculty members, as well as research associates and master practitioners, all who have a deeply vested interest and broad experiences in mathematics education.

As a practitioner reading this blog, awareness of current national and international mathematics education issues will be discussed. We intend to provide practical applications in the classroom. You will be able to connect with other researchers and educators to share insights, offer tips, and ask questions. We invite you to visit, read, and comment often.

All comments and questions related to curriculum or assessment are appreciated and encouraged! We want to hear your feedback whether you agree or disagree; we just ask that you respect all opinions. You are also welcome to leave comments or give suggestions below. You can also follow us on Twitter for the latest RME news and updates about what is happening around in the world of mathematics education.

**RME does not endorse or advertise any published products.