Educational Testing Has Limits, Says Critic

WASHINGTON, SEPTEMBER 22, 2008--A more complete understanding of the benefits and shortcomings of educational testing is integral to developing robust accountability regimens in K-12 schooling, Harvard professor Daniel Koretz said at an AEI conference on September 22. "Test scores alone are never sufficient to evaluate schools."

Koretz, author of the acclaimed new book, Measuring Up: What Educational Testing Really Tells Us (Harvard University Press, 2008), offered a mostly critical view of the current state of educational testing, arguing that ignoring metrics of student achievement that cannot be captured by a single diagnostic test is "always the wrong thing to do." The result, he said, has been policy crafted with little appreciation for how tests might be better used to improve academic standards.

Koretz suggested that one of the major problems with test-based accountability models is that they often "exaggerate how well students are performing." He offered several explanations, including incentives to "teach to the test," the "narrowing the curriculum," and a singular focus on "bubble students" (those right near the line of proficiency.) To address these concerns, he underscored a need to get rid of unrealistic targets for student gains, establish varied and holistic measurements of student achievement, and temper an overreliance on standards-based reporting.

Other panelists echoed Korestz's pleas for reform, but they differed about how policymakers should design new tests or attend to flaws in the system. Roberto Rodriguez, senior education adviser to Senator Edward M. Kennedy (D-Mass.) on the Health, Education, Labor, and Pensions Committee, contended that accountability schemes need to do a better job of "accounting for trends over time rather than year-by-year analyses" and of "promoting formative assessments that are based on multiple measures and multiple indicators."

Bella Rosenberg, an independent educational consultant and former special assistant to the president of the American Federation of Teachers, said that while she favors test-based accountability in principle, the way policymakers have gone about implementing it is tantamount to "officially sanctioned malpractice." She explained that under current law, "the federal government has created incentives to retard the development of better testing" and called for instituting "a range of different metrics to evaluate school and student performance."

With the pending reauthorization of the No Child Left Behind Act (NCLB), efforts to develop fairer and more accurate educational testing systems have taken on increased significance. NCLB, which uses high-stakes testing in reading and math as the basis for doling out rewards and sanctions to public schools, mandates that all students must reach "proficiency" by 2014. Today, more than 11,000 schools are identified as failing to make "adequate yearly progress."

Koretz acknowledged that while NCLB has exacerbated many of his concerns over large-scale educational testing systems--specifically their reliability and potential sources of bias--simply eliminating the requirement is not the solution. While there are better and worse approaches to developing educational tests, he added, there is no one method that has proven demonstrably effective.

Part of the reason, he said, is that every effort to test children involves compromises. As Koretz writes in Measuring Up, "Any design that offers you gains on one front is likely to impose costs on another, and the task of sensible design entails carefully weighing the inevitable trade-offs. . . . Many people simply dismiss these complexities, treating them as unimportant precisely because they seem technical and esoteric."

Although Koretz has become one of the most forceful critics of educational testing, he was quick to refute charges that he is "anti-testing." Rather, he noted, standardized exams can serve a useful purpose in determining whether students are making progress across grade levels and subject areas--so long as they are viewed with a discerning eye toward what they can and cannot accomplish. "Tests are a unique and very valuable but a necessarily limited source of information about achievement," he said. "When we test, the first thing we have to do is get rid of a lot that is important about education."

      --THOMAS GIFT

For video, audio, and more information about this event, visit www.aei.org/event1799/.

For more information about education policy studies at AEI, visit www.aei.org/hess/.

For media inquiries, contact Véronique Rodman at 202.862.4870 or vrodman@aei.org.

###