February 2006

Federal Education Statistics: Charting a New Course

This past October, the U.S. Senate unanimously confirmed Mark Schneider as the new commissioner of the National Center for Education Statistics (NCES), the nation’s primary organization charged with conducting federal education research and collecting statistics on student achievement. The data NCES compiles will shape the direction of education policy in this new era of No Child Left Behind accountability. Schneider’s responsibilities include overseeing the National Assessment of Educational Progress--more commonly known as the “nation’s report card.” In his first public remarks, Commissioner Schneider shared his vision for NCES and explained the importance and impact of education statistics on school reform in the future during a February 6 address at AEI.

Mark Schneider
National Center for Education Statistics

The role of the National Center for Education Statistics (NCES) is to document the condition of all levels of American education. The commissioner’s role is to maintain and improve the many NCES data collection systems so that they are uniform, accessible, and relevant to current needs and trends.

In particular, four areas present significant challenges to NCES data collection. The first is improving the accuracy of the National Assessment of Educational Progress (NAEP), which accounts for almost half of the NCES budget and is the center’s most visible product. Collecting data for NAEP has been difficult because the number of schools participating has been declining. Schools are less willing to participate in additional standardized tests, particularly in topics not covered in mandatory No Child Left Behind (NCLB) act testing. The number of students participating also has been difficult to maintain, particularly for twelfth-grade data. Additionally, the older the student, the more difficult it is to ensure students are taking the exam seriously. Furthermore, NCES needs to update the reporting criteria for NAEP in light of technological and political developments. NCES is in the process of developing a more accurate index for measuring socioeconomic status, one that comprises several variables using census data. With the proliferation of schools due to increasing school choice, NCES faces new challenges in tracking students who change addresses and schools several times. As more states adopt computer-based testing, NCES must answer concerns that changing the modality of testing midway through long-term tests will alter results and that student familiarity with certain kinds of media will shape outcomes.

Collecting data on the effectiveness of American high schools has also presented a second challenge to NCES. There is increased focus nationwide on high schools, but the most important measures of high school effectiveness--dropout rate and graduation rate--remain difficult to calculate. NCES has recently reformed how it measures graduation rates. Additionally important to collecting high school data is improving methods of documenting the transitions between school levels--namely, the bridge from elementary to secondary school (the critical ninth grade) and the passage from secondary into postsecondary, or into the “real world.” Eventually, there is a wide range of outcomes relating to high school that NCES hopes to be able to measure--such as success in postsecondary school and trends in marriage, childrearing, crime, and civic participation.
  
NCES has two primary approaches to collecting high school data. First, the center uses longitudinal data sets--collection efforts that span several decades and draw information from a wide variety of sources. Currently, there is both an ongoing Educational Longitudinal Study 2002 (ELS#2002) test, which monitors students from tenth grade onward, and Early Childhood Longitudinal Study (ECLS), which tracks students beginning at a much earlier grade level. These studies are both limited in their scope, as they are student-based; in the future, NCES would like also to run a school-based longitudinal study that can illuminate more about the outputs of schools. Second, NCES collects high school data through state administrative data sets. NCES has been working to improve these sources in each state to make them more accessible to independent researchers. Collecting data through both resources is effective, though sometimes difficult in light of privacy concerns.

A third area of challenge for NCES is postsecondary education data collection. Information on postsecondary schools is grouped and measured separately from all other NCES data. Ultimately, the commissioner would like to reconfigure data collection to ensure that it is uniform at all education levels. But for now, the commissioner is focusing on how to bridge together the data that has already been gathered. The two data systems currently used for measuring postsecondary education are the National Postsecondary Student Aid Survey (NPSAS) and the Integrated Postsecondary Education Data System (IPEDS). Both data sets need general improvement; the commissioner has focused specifically on refining how these sets measure price and accountability. It is necessary to recalculate the price of postsecondary education because people need to better understand the variation between the “sticker price” of a postsecondary degree and the actual price, or “discount rate,” that many people end up paying.  It is also important to increase accountability in postsecondary schools. One of the best indicators of an institution’s effectiveness is graduation rate, but the current IPEDS measurements of graduation rate are lacking and must be improved to include more nontraditional categories of postsecondary students.

Finally, NCES needs to expand its efforts to collect data on teachers. There is surprisingly little data on teachers, even though they are the most expensive input into the education system. NCES is focusing first on better recording teacher compensation--namely information on salaries, which prove very difficult to isolate. Also, NCES is working on collecting data that would help us predict the rate of retirement and demand for teachers in the future. Other questions remain, such as how effectively the teacher workplace is managed, and how often unsatisfactory teachers are weeded out. NCES strives to collect the data that will provide these answers.

To conclude, there are five more general, overarching challenges that NCES is facing. First, NCES must decide whether it wants to take an inductive or deductive approach to improving data collection. The commissioner believes an inductive approach is most effective; he intends to respond to problems as they become relevant and central instead of trying to construct the perfect data model beforehand. Second, NCES faces challenges in constructing its workforce. It is difficult to attract highly skilled staff who are willing simply to present rather than interpret data, as is consistent with NCES’s mission, as well as people who have the potential ultimately to be strong leaders and managers of the center. The NCES workforce would also benefit from expanding the capacity of internal staff (instead of over-relying on outside contractors) and reforming the Flexi-Place and Flexi-Time system. Managing its resources is NCES’s third challenge, as it is relying on outside contractors and constantly balancing frequent budgets and escalating costs. Fourth, NCES confronts the problem of measuring units that are consistently changing. Making matters worse, the same units are described using different definitions, some of which change due to varying political motives. The commissioner continues to focus on making definitions consistent and data sets uniform. The fifth and final challenge for NCES is to create a system that is not at risk of becoming obsolete in a year. It is the commissioner’s job to help NCES remain flexible and forward-thinking.

AEI research assistant Rosemary Kendrick prepared this summary.