<html><body>American Enterprise Institute November 30, 2006 [Edited transcript from audio tapes] <TABLE cellSpacing=1 cellPadding=1 width="100%" border=0> <TBODY> <TR> <TD> <DIV class=BodyText>8:15 a.m. </DIV></TD> <TD> <DIV class=BodyText>Registration and Breakfast</DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>9:00  </DIV></TD> <TD> <DIV class=BodyText>Introduction: </DIV></TD> <TD> <DIV class=BodyText>Frederick M. Hess, AEI</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>9:10</DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Panel I: The Big Picture National Implementation and Capacity </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Presenters: </DIV></TD> <TD> <DIV class=BodyText>Michael Casserly, Council of the Great City Schools</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Jeffrey R. Henig, Columbia University Teachers College </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Paul Manna, College of William and Mary</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Michael J. Petrilli, Thomas B. Fordham Foundation </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Discussant: </DIV></TD> <TD> <DIV class=BodyText>Chester E. Finn Jr., Thomas B. Fordham Foundation</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>10:40  </DIV></TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Break</DIV></DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> <DIV class=BodyText>10:45   </DIV></DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Panel II: The NCLB Remedies in the States</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Presenters: </DIV></TD> <TD> <DIV class=BodyText>Julian Betts, University of California, San Diego</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Patrick McGuinn, Drew University</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Alex Medler, Colorado Children s Campaign</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Discussant: </DIV></TD> <TD> <DIV class=BodyText>John Winn, Florida Commissioner of Education</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>12:05 p.m. </DIV></TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Luncheon</DIV></DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>12:55   </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Panel III: The NCLB Remedies in the Districts</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Presenters: </DIV></TD> <TD> <DIV class=BodyText>Stephen K. Clements, University of Kentucky</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Jay P. Greene, University of Arkansas </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Jane Hannaway, Urban Institute</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>David Plank, Michigan State University</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Discussant: </DIV></TD> <TD>Stephen Jones, Norfolk (Virginia) Public Schools </TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>2:30  </DIV></TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Break</DIV></DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>2:35  </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Panel IV: Reconstituting Districts and Schools</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Presenters:  </DIV></DIV></TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Bryan C. Hassel, Public Impact </DIV></DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Joe Williams, Education Sector </DIV></DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Discussants: </DIV></TD> <TD> <DIV class=BodyText>Alan Bersin, California Secretary of Education </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Morgan Brown, U.S. Department of Education</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>3:50  </DIV></TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Break</DIV></DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>3:55  </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Panel V: Lessons Learned</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Discussants: </DIV></TD> <TD> <DIV class=BodyText>Kati Haycock, The Education Trust</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Diane Ravitch, Brookings Institution</DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD>Marshall (Mike) Smith, William and Flora Hewlett Foundation</TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR> <TR> <TD> <DIV class=BodyText>5:00   </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText>Adjournment and Reception </DIV></TD></TR> <TR> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD> <TD> <DIV class=BodyText> </DIV></TD></TR></TBODY></TABLE> Proceedings: Panel I: The Big Picture National Implementation and Capacity Frederick M. Hess:  Hi, I m Rick Hess, Director of Education Policy Studies at the American Enterprise Institute.  I would like to welcome all of you here today to join us for the conference Fixing Failing Schools: Is the NCLB Toolkit Working?   AEI is sponsoring this conference in conjunction with the Fordham Foundation.  I would like to thank my colleague in this, Checker Finn for being Checker.  Many of you in this room were here when we held a similar effort about three years ago in a volume that was published under the title Leaving No Child Behind by Palgrave McMillan in 2004, in which Checker and I had the idea of looking at how the remedy provisions of No Child Left Behind were actually working on the ground.  At that point in time in 2004, these provisions were quite new.  Many people were still trying to figure out what they actually entailed and what they could look like and in fact in most states, we had only seen the public choice and supplemental service provisions begin to be implemented.  At the time, Checker and I, as we are wont to, were hopelessly optimistic.  And we also thought that it would be useful to come back and repeat that effort once these plans had more of an opportunity to take effect, to be implemented, to unfurl.  That is what we are gathered for here today.  Now, No Child Left Behind has been a source of much discussion, analysis, and heated rhetoric in the past five years, of course.  It is no surprise; it is a big law with grand ambitions and a lot of moving parts.  NCLB addresses just about everything we care about in education scientific research, reading, teacher quality, a host of issues.  But it is hard, of course, at least at the federal level pioneering a testing and accountability framework.  Now today, we are not trying to offer a comprehensive assessment of No Child Left Behind.  That is not what the authors were asked to do and that is not what any of them attempted to do.  Instead, we are looking at one half of the testing and accountability framework, the often taken for granted remedy half, and trying to make sense of how it is working and how it might work more effectively.  For those of you who want a broader brush-up on the law, Mike Petrilli and I did a little volume earlier in the year, which you will find available out in the lobby called No Child Left Behind:  A Primer.   I m happy to point you to it.  There is also obviously a lot of other good work on the law more broadly.  That is not what we are addressing today.  So it would be unfair to challenge the authors on questions which do not actually relate to what they were asked to study or write about.  The testing side of the testing and accountability equation has received extensive attention.  State officials, federal officials, a lot of academics have spent the past five years arguing and examining cut scores, proficiency targets, N-sizes, value-added metrics and related concerns.  Now, receiving far less attention have been the accountability issues posed by No Child Left Behind.  What do we do when testing suggests that some schools are failing to make adequate progress or that some districts are failing to do so?  This is a question that has bedeviled reformers for decades.  It is not a new question.  The response adopted in No Child Left Behind was to create a cascade of remedies that would give students in these schools new options while addressing the shortcomings of failing schools.  There are really at least three strands of thought embedded in the strategies of the No Child Left behind remedies.  One was to create new opportunities to serve children stuck in troubled schools.  A second was to pressure low-performing schools and districts to improve their performance.  And the third was to provide guidance and technical support to schools which were deemed to need that kind of assistance to improve.  In asking is the NCLB toolkit working, then, we find ourselves asking how well it is accomplishing these three sets of tasks.  The challenge is that we do not have good or clean data on any of these questions.  In fact, the lassitude with which local state and federal officials have approached the task of collecting data on NCLB interventions is remarkable, especially in an era marked by ceaseless talk of research and data.  As you will hear throughout the day, simply locating descriptive data on the remedy provisions is no easy task and trying to assess the impact of any of them has barely been contemplated.  What exactly are the NCLB remedies?  Now, many of you in this room are familiar with these and familiar in great detail.  Some of you do this every day -- Morgan.  On the other hand, I realize everybody in this room does not necessarily have a whole lot of time to delve through the particulars of NCLB, so it might be useful for us briefly just to run through the issues at play.  NCLB cascades are to be implemented in a series of steps once schools are identified as having failed to make adequate yearly progress.  Ballpark, about one quarter of the schools in the nation this year are deemed not to be making AYP and about one in eight have been identified as needs improvement.   Once schools have entered needs improvement status year one, they are supposed to get technical assistance from the local education agency and they are required to offer public school choice to a particular set of students in those schools.  In year two of the remedies, they are required to make available supplemental educational services.  It is useful to think of it really as after-school tutoring, in most cases, to qualified students.  In year three, they are required to create a corrective action plan that complies with a menu of available options.  In year four, they are required to begin the process of restructuring, and in year five they are required to implement restructuring.  If after restructuring in year five a school continues to fail to make adequate yearly progress, there is an open question as to exactly what happens.  This will be one of the topics we will speak about today.  Now, today s intrepid authors have set out to provide a deep and scholarly examination of how these NCLB remedy provisions are actually working.  The authors were not asked to evaluate these provisions.  In the language of the education profession, what we are looking at are formative rather than summative assessments of the NCLB remedies.  The authors have sought to offer a careful nuanced look at how the NCLB remedies are playing out in schools, districts, states and in the US Department of Education.  This research is instructive not only for deliberations regarding NCLB but because NCLB remedies include popular high-profile reform ideas such as public school choice, school restructuring and charter schooling that are actually relevant to broader debates about school improvement.  Finally, a popular current line is that NCLB is well-designed but in implementation has sometimes stumbled.  That analysis deserves careful scrutiny and should very much be on the table as we look at these questions today.  As Checker and I wrote back in 2004, operationalizing any statute as complex as No Child Left Behind brings inevitable headaches.  Different agencies and levels of government must learn to work in new ways.  Officials must take on unfamiliar roles and educators must author ingrained routines.  As these arrangements are negotiated, a certain amount of confusion is to be expected.  Such problems are normal, usually diminished with time and experience, and are mainly of interest to students of government process.  However, some laws also summon more fundamental woes by perverse incentives, incompatible interests or unworkable expectations.  These do not go away with aspirin and a night s rest.  They may in fact require surgery.  Identifying successes and challenges in various locales reflecting on which problems may be alleviated with time and which require more fundamental action - this is our charge today.  Today will unfold in five panels.  The first, already seated up here, is going to look at the big picture, offer a broad, descriptive look at what is taking place in the states and major urban districts as well as with supplemental services and with Department of Education implementation.  The next panel will look at the NCLB remedies at the state level.  The third panel will look at them at the district level.  The fourth is going to look particularly at the questions of restructuring, both in cases of districts and of schools, and the final panel is going to see if we can synthesize what we might take away from everything that is presented today.  Hard copies of the conference papers are available in the lobby where you entered.  They will also be available on the AEI website today, and the AEI website link will be up there throughout the day.  The revised collected analyses will be available next year in a volume that is going to be published by AEI Press next summer or fall.  But in the interim, the papers will be available in their entirety on the AEI website, so feel free to help yourself.  Before we get started, I just want to make two requests.  One, please keep the day moving along.  We have limited between-panel breaks to about five minutes.  I ask that you please assist us in getting sessions started in a timely fashion.  Second, when asking questions from the audience, please wait for the microphone, identify yourself by name and affiliation, and actually ask a question.  We have this proud DC tradition of long orations with the question mark appended.  Anyway, with that we are going to go ahead and get started with the first panel.  The first panel is going to offer us a broad look at how these issues are playing out nationally.  Four authors will be presenting.  We will be going in alphabetical order.  First up, writing about how the remedies have been playing out in major urban districts, is Mike Casserly, Executive Director of the Council of Great City Schools.  Prior to becoming Executive Director, Mike served as the organization s Director of Legislation and Research for 15 years.  Most of you know Mike.  He is probably one of the most influential voices on urban schooling in the country today.  So we appreciate him taking the time and actually going out and collecting these data which is a nightmare, to put it simply.  Speaking second will be Jeff Henig.  Jeff is a professor of political science at Teacher s College, Columbia University.  His books include the Color of School Reform: Race, Politics and the Challenge of Urban Education and Building Civic Capacity: The Politics of Reforming Urban Schools.  Jeff s work is focused on the boundary between private and public action and tackling social problems.  His piece today looks at the political economy of supplemental education services.  Again, not attempting to evaluate this in any what is the impact on learning outcomes sense, but trying to help us understand: what does this whole new sector look like that we have created under the legislation?  Third is Paul Manna, an assistant professor in the Department of Government and a faculty affiliate with the Thomas Jefferson Program in Public Policy at the College of William and Mary.  Paul is the author of the recent book School s In: Federalism and the National Education Agenda, which is the first serious treatment, actually, of how NCLB has been implemented in the states and considering questions of state capacity.  Paul is going to be talking about the challenges in implementation of the NCLB remedies at the state level.  Finally, it will be Mike Petrilli, Vice-President for National Programs and Policy at the Thomas Fordham Foundation.  Mike served in the Department of Education.  Under the Bush administration, he was Nina Rees deputy at OII.  He helped coordinate and, in fact, helped launch some of the NCLB public choice provisions and supplemental service provisions.  He is also, as I mentioned, the author of No Child Left Behind: A Primer and an editor with Education Next.   And finally, serving as discussant on the first panel will be Checker Finn.  Checker, President of the Thomas Fordham Foundation and the Thomas B. Fordham Institute, a Senior Fellow at Stanford s Hoover Institution, Senior Editor of Education Next.   Previously, Checker has served as former Assistant Secretary for Research and Improvement at the US Department of Education and was a founding partner and senior scholar with the Edison Project.  The way the first panel will work is each of the authors is going to take about 10 minutes to just present the key take-aways.  We will then have an extended period for conversation about what we might take away from this initial broad look.  Mike, would you please start us off? Michael Casserly:  Thank you very much, Rick.  Can everybody hear me okay?  Thank you.  I m Michael Casserly.  I m the Executive Director of the Council of the Great City Schools, what you may know as a coalition of the nation s largest urban public school systems across the country and an organization that actually supports No Child Left Behind, although after scanning some of the papers I m starting to wonder why.  I m going to keep this very brief and give you just the highlight of the status of No Child Left Behind in the nation s major urban school district.  We have considerably more data in my paper than I can possibly present here in 10 minutes so I m just going to give you some of the briefest highlights.  The data that we are presenting and releasing today actually, for the first time - come from a new survey that the Council of the Great City Schools did of its 66-member urban school districts, 36 of which have responded to date.  There is more data coming so over the next several weeks these various figures will change somewhat.  These 36 responding districts have enrollments that are 65 percent-free in reduced-price lunch, 18 percent English-language learners, 13 percent students with disabilities, and about 80 percent students of color.  In addition, these 36 districts have approximately 7,500 schools enrolling approximately 5.1 million students.  About 5,900 of these schools or about 80 percent of all of these schools are Title-One schools.  The number of these schools that are in NCLB s sanction, that is, the actual sanction process, not the warning status, has grown steadily since 2002-2003 despite significant gains in the cities in reading and math on state tests and on the National Assessment of Educational Progress.  The number of schools in school improvement one, two, corrective action and restructuring increased from 975 in 2003-2003 to approximately 1,500 in 2003-2004 to 2,045 in 2004-2005 and 2,200 in 2005-2006.  Conversely, another 531 schools made enough progress in 2005-2006 to get out of sanctions or to have their sanctions put on hold.  About 30 percent of all the schools in these 36 cities are in sanction status.  This constitutes about 26 percent of all sanctioned schools in the country.  Finally, there appears to be no particular relationship between the number of schools in sanction under NCLB and their performance on the National Assessment of Educational Progress.  Boston and San Diego for instance have similar reading and math scores on NAEP, but San Diego has 15 percent of its schools in sanction while Boston as 47 percent.  That is one of the reasons why our organization last week called for the institution of national standards.  The data also indicated that the majority of the schools, approximately 2,043, had not made AYP in reading.  This was the largest category in which we had not made AYP, although a substantial number of schools had not made AYP in math as well.  We also asked about the number of schools not making AYP because of a single sub-group.  This has been an issue of debate over the last couple of years, and the answers varied by sub-group.  Approximately 70 schools did not make AYP solely because of their African-American students.  About 15 did not make it because of their Hispanic students; 32 did not make it solely because of their poor students, and 94 did not make it because of their limited English-proficient.  About 155 did not make it because of their disabled students and about 94 schools did not make it because of their 95 percent testing requirement.  This also means, conversely, that most of the schools that are in sanction in the cities are in sanction because of more than one sub-group at a time.  Now, for the sanctions themselves, starting with choice.  The data indicate from our survey that the number of students transferring to a higher performing school under NCLB increased from 11,292 schools in these 36 cities in 2002-2003 to about 22,553 students in 2005-2006.  This translated into the huge leap of participation from one percent to 2 percent.  The percentages are still pretty low in terms of choice, but we did ask, in addition to NCLB choice participation, about the numbers of students transferring for non-NCLB reasons.  Here, the numbers were much higher.  About 325,000 students, or about eight percent of the enrollment, actually transferred in these districts for reasons other than No Child Left Behind.  If one combines the two-percent No Child Left Behind plus the eight-percent non-No Child Left Behind to non-district charters and private schools then the portion of urban school students using some form of choice or another exceeds about 30 percent.  We also have considered data on the length and numbers of windows during which parents have to choose the numbers and types of methods of communications with parents, the numbers of choices, school capacity and other information in this paper.  In general, the data shows some progress on each of these fronts since 2002-2003 - longer windows, more choices et cetera.  Next, supplemental educational services.  Here, the data showed that the number of students participating in SES has increased substantially.  The numbers in the responding districts grew from approximately 110,000 students in 2003-2004, mainly the first year in which SES was really offered, to approximately 180,000 in 2005-2006.  About 16 percent of all eligible students are now served by SES.  This next number is going to surprise, I suspect, everybody in the audience, but the data also indicates that approximately 95 percent of all students in these 36 cities now receive their Supplemental Education Services from a private provider, not from the district.  That is 95 percent.  Only six of these 36 districts are now able to provide their own services.  For the most part, urban school districts are out of the SES business.  Despite the apparent increase in the numbers of students receiving SES, however, it also appears that the rate of participation remains flat.  The number of SES participants per school actually held steady at about 112 students between 2003-2004 and 2005-2006.  This suggests that the overall increase in numbers has been due largely to the higher numbers of schools that have to offer SES and not to the overall yield in the program itself.  We have not finished analyzing the data yet but we did look at the percentages of students served in the districts or in cities where the district could provide services and where they cannot; I suspect this will be of interest to the folks from the Department of Education.  As I said, the average was about 16 percent; but in cities where the district was able to provide their own SES services, approximately 29 percent of all eligible students actually received services.  In cities where the district was not able to provide services, only about 11 percent of the eligible students were being served.  If all of the cities were able to provide SES at the same higher rate of 29 percent, then participation numbers would almost double by about 150,000 kids.  Third, we asked about the strategies that districts were using with their schools in corrective action.  The most common strategies involved technical assistance, professional development, informing parents about the status of the schools.  The least common strategies involved charterizing, privatizing or giving the schools over to the individual states to run.  Finally, we looked at the strategies being used with schools in restructuring.  Here, the pattern of sanctions and the pattern of options is pretty much the same as we saw in corrective action; same kinds of things being used most often, the same kinds of things being used less often.  There are lots of ways, of course, to slice all of these data but in some ways we are really getting started on our analysis.  We have considerably more data than we are presenting here or even in the paper.  I would like to wrap up here by making three quick points.  One, we are starting to review various local evaluations of SES and finding very mixed results; modest at best on average and harmful in some cases.  The effects of SES on state test scores really depends on the provider you are looking at.  Two, we are getting better as cities at implementing the law but implementation, I have to say, is quickly becoming an exercise in compliance with rules and sanctions that are poorly designed to raise student achievement.  And finally, let me just make one quick observation about the cascading nature of the sanctions themselves.  This cascading process does not seem to be very effective.  It has school districts chasing an annually changing set of strategies without enough time to make any of them work.  We are making, we think, very good faith efforts to make this law work and we continue to support it.  But, in the end, Congress is going to have to recalibrate it in order to produce the results that they want.  Thank you. Frederick M. Hess:  Jeff? Jeffrey Henig:  Rick and Checker pushed us very had to do PowerPoint; it is not my m�tier, so we will see how this works out.  It is already showing.  So I m going to talk today about supplemental education services, and Mike already provided more data than I found looking.  This is a research-thin area, as he said, but nonetheless all of us have some memory of being in school and there was a kid there who was on the margins, always there; people vaguely knew who he was but no one remembered his name and I want to suggest in some ways SES is like that.  In the run-up to NCLB and, subsequently, the popular kids are disaggregated test scores, adequate yearly progress, highly qualified teachers.  People have not talked much about SES - I ll show you some data on that - and really did not think very much about SES leading up to the legislation [indiscernible] that much subsequently.  Very quickly, the basics on the program are familiar.  SES provides out-of-school tutoring, meaning tutoring in the afternoons after school, on weekends, on the summer.  It is tutoring for low-income students in failing schools, Title I schools that have failed to make adequate yearly progress.  Providers can be the districts but the law stipulates explicitly that the providers can also be both for profit and nonprofit providers.  The states are responsible, according to law, for approving providers and for monitoring their performance.  The funding for the program comes from the district s Title I allocation, and districts are required under the law to set aside at least 20 percent of their Title I allocation to cover the cost of combined supplemental education services and any transportation costs associated with the choice provisions of the law.  I think it is fair to say that at the time this was being talked about, again, the choice provision got way more attention.  I think most people involved expected that the choice part of the program was going to be a bigger chunk than it has turned out to be compared to SES.  By any measure, SES is a major policy initiative.  Compared to choice, it involves more children; in 2004-2005 nationally, over 430,000 kids.  It is a big-bucks industry, potentially; up to $2-2.5 billion a year, conceivably.  This is not what is actually being spent now, but what would be the case if you sort of maxed out that 20 percent on Title I.  And by the way, the 20 percent is not a ceiling.  It is a ceiling in terms of what the districts are required to provide but it can be more and the program is growing rapidly.  The program is growing rapidly, over 210 percent between 2002 and 2003 and 175 percent between 2003 and 2004.  It is a major initiative also because it represents a new kind of policy.  In some ways, this is at least interpreted by some people as the government endorsing the idea in an era in which privatization of social services is the scrimmage line for partisan battles between Republicans and Democrats, between the Right and the Left.  SES is an endorsement, at least in principle, of the notion that for-profit providers may be able to do a better job at providing basic services than government employees.  My paper suggests that this is a program that is better understood in many ways as the outgrowth of inter- and intra-party maneuvering rather than the culmination of a deep and sustained thinking about how kids learn, how families and schools interact in communities and how reform tools take root and succeed.  In served in an intra-party sense different functions for Republicans and for Democrats.  For Republicans, what was appealing about SES was that it was a way to assuage conservative Republicans who were distressed over the Bush administration s backing down on insisting on a voucher component in the law and SES as stepping in as kind of the replacement for choice had three benefits.  It looks like vouchers in some ways.  It is portable aid that families can carry with them from provider to provider in principle.  It is given to individuals and not to schools and it involves the private sector.  It is also a program that provided a nose under the tent, if you will, for profit providers, a growing industry and one that many conservatives wanted to encourage.  And it gave in just the bargaining and political landscape around the law; it gave conservatives on the further Right end of the party, a chance to claim that they have had an impact after they, in fact, felt somewhat marginalized by the elimination of vouchers.  The program served a different function for Democrats.  For the new Democrats, if you will, Lieberman and others, it was a chance to demonstrate that Democrats are not anti-market, that Democrats can and will pragmatically employ market solutions when they make government work better.  For the old Democrats, if you will, the traditional mainline Democrats the Kennedys, Miller and others, the program had three virtues.  First, and most importantly, it was not vouchers; secondly, SES because it is delivered out of school, it did not represent a direct confrontation with the teacher s union encroachment on their turf; and third, SES was in many ways a substantial expansion of public responsibility rather than a contraction or shrinking of public responsibility and a new way to get aid to Democratic constituencies low-income kids, minorities, central cities.  I mentioned before little attention.  I analyzed the coverage in major newspapers, both of NCLB, generally, and SES in particular over time.  This just maps out whether& the red on the top is whether NCLB was mentioned anywhere in the text of the articles.  The blue is whether it was mentioned in the abstract, an indication of a more substantial focus, and the two lines on the bottom are for SES.  And you can see, if you take being mentioned in the abstract of the article as being an indication of serious interest, there is a little blip of interest around 2001, 15 articles, really minor and fell off rapidly.  Jeffrey Pressman and Aaron Wildavsky, political scientists, have made the argument that policies imply theories, by which they mean that every policy, at least, implicitly, has a network of assumptions about causality, that if you press here, the following things will happen; this will lead to that this will lead to that, and so on.  For SES, you can look for the theory but there is not really one.  And instead, I suggest in this paper there are two simplifying images that in some ways animated the actors, and in the case of obsolescent localism worked similarly for both Republicans and Democrats as an analysis of the problem that needed to be addressed.  And the market metaphor - I will say more about this and second - was a clearer notion of how SES would fit into the solution in a way that was intellectually consistent with the Republican agenda, generally, and with the thinking behind the law.  So obsolescent localism, this is not a well-honed thesis but a collection of impressions based on demographics, socio-political trends that some people have come to believe are associated with higher mobility in a more contemporary American society that is weakening people s attachment to localities, weakening people s attachment to place and making localities in some senses structurally weaker.  The image of obsolescent localism suggests that policies at the local level tend to be driven by parochial values, interest group politics, over-bureaucratization and reflexive loyalty, the status quo.  Now there are two different kind of images that Democrats and Republicans draw on in thinking about obsolescent localism, I suggest.  For Democrats, there is a history of thinking of localism in part in terms of Southern resistance to integration, to Brown versus Board of Education, and in a more contemporary vein, to suburban exclusionism, suburban tendencies to retreat into their localities, put up high barriers for entry, to take care of their kids and their kids schools but make it tougher on inner cities to deal with the folks who are left.  For Republicans, the image is more one of Union-dominated bureaucracies in large urban central cities.  In both cases, the predictions, if you will, that seem to emerge out of this image of obsolescent localism is that we should expect the locals to resist SES, to resist encroachment on their control of their Title I monies, that there will be a need for strong federal and state role to enforce this to make it happen and then over time the locals will be gradually displaced at the local districts as a major actor.  The market metaphor very quickly assumes that what this program will do along with choice provisions is turn families into consumers who are free to shop around for providers who will best serve their kids.  This will create a competitive environment in which various providers will up the ante in whether to gain greater market share.  For some supporters of this notion, that includes the notion that the district will up the ante in terms of its performance as well.  The districts will just be one under this model among a diverse array of tutoring providers; and there are predictions that evolve out of the market metaphor.  One is that there will be an active demand, that there is a latent demand for tutoring services that will be activated by the addition of money vouchers that& essentially that families will be able to use, that there will be a vital supply-side response, that new providers will come out to engage in this market and that there is an edge for profit providers.  Very quickly, since I m out of time, what do we find?  I want to argue that what we find is something quite different from this.  The local districts are still the fulcrum of activity.  Rather than resisting outright, some districts have embraced SES, that there is a slowly growing demand side but it is a tough sell.  Getting parents to sign up and getting kids to go is much more difficult than many anticipated.  The paper includes an analysis of who the providers are.  Most of them - there are some 17,000 - are local, parochial providers if you will, Mom n Pop social service agencies and the like who will operate in only one district or certainly in only one state; 93% by my count operate in one state only.  I used the language dual market of these local Mom n Pop providers versus a multi-state corporate provider.  Corporations make up only about 2.5 percent of all the approved providers but of the corporations, over 70 percent are operating in multiple states whereas in contrast, social service organizations make up about 13 percent of all providers, but only less than two percent of them are in multiple states.  Finally, I argue the best way to understand this is of SES as a local contracting regime, like local governments contract for other services.  The locals are central; they are central not because they are central in the law but they are central because states and federal government lack the capacity to really get involved and monitor what is going on.  They are central also because locals control access to schools and teachers, and at least for the for-profit providers, it is proving very critical for them to get onsite provision or to get recommendations from teachers and to use teachers as providers for reasons I ll talk about in the question-and-answer period because I m already in deep trouble.  But I want to suggest in this contracting regime, access depends on not just demonstrated performance but traditional things that count in a lot of local government, which is political ties, support from active community organizations and the like.  I conclude with some implications for policy and you can read this in the policy.  I ll just summarize it in two sentences and that is the response can either be make the program fit the original image, get tougher on the locals, beef up the market sector, or work with the locals to improve their capacity to do this well and do this right; I side for the latter. Frederick M. Hess:  Thanks Jeff. Paul. Paul Manna:  Hey, good morning everybody.  My paper focuses on the importance of the state s role and state implementation of the No Child Left Behind remedies, and I think this is an important focus for this conference for this project.  A lot of people correctly state that NCLB is a major law.  It has expanded Washington s reach across the country into the nation s schools, but I think in saying that it is important not to overstate the federal role.  State policy in education remains absolutely crucial, and in the paper I used the phrase that often I think of state policy as the fuel that powers the NCLB engine, that if it were not for the states taking action, making policy decisions, et cetera, NCLB would basically go nowhere and the law really relies a lot on what is happening at the state level.  Further, if you look across the states, states are diverse places in terms of their educational governance structures, in terms of their political climates, and also in terms of the bureaucratic capacities that they have in order to make laws work.  This is true in education; it is true in all kinds of other policy areas.  And so as you look across the country at how NCLB is being implemented in the states, it is not surprising that you see a lot of variation when you examine the ways that states have tried to help and do their part in implementing the NCLB remedies.  So in my remarks to day, what I want to do is just briefly describe some of that variability that I have uncovered at the state level and then also discuss the impact of that variability.  What impact does it have on how the law is unfolding?  And the first issue I want to start with is timing.  As you know, state testing systems and accountability systems are the major instruments that are used to determine whether school districts and individuals schools have made adequate yearly progress.  In most places, students take their tests in the spring, maybe March, April or May, typically; and so it is crucial for NCLB and its remedies to work.  It is crucial for the states to swiftly and accurately process all those results so that people at the local level know which remedies are supposed to be implemented and so that those people have enough time to implement them.  Today, as this first slide shows here, actually, though, nearly five years since NCLBs passage, the states still struggle to determine and report out final AYP results on time.  What you have in this figure which appears in the paper, on the horizontal axis, on the X-axis, you simply have the rank-order of the states when they release their final AYP results.  And so the first state is the dot farthest to the left and then as you work your way across, the last state is the farthest to the right; and then on the vertical axis there you have the day at which the final AYP scores were released.  If you look at that, you see that most states are releasing these final numbers sometime during the month of August, this window, almost right as the school year is beginning.  In several states you can see it is bumping right up against late August, early September; some states, even after the school year has begun.  I think if you count the dots there it is about seven or eight are getting these results out the door.  If you are a quick study, if you maybe count cards in poker or something, you will notice that there are only 43 dots on the figure, and that was because as of late October 2006 when I gathered these data, there were still seven states that had not completed the final assessments of AYP that would affect the current school year.  I have to go back and see if they are done since then but when this was put together less than a month ago that was the status.  Clearly, there are big implications for implementing the law s remedies.  If you are a parent and you want to opt for school choice or you want to try to get your child enrolled in supplemental services, it would be great for you to know that you have those options in advance of the school year.  I mean, it is hard to fault a parent who would not want to uproot a child s routine, et cetera, early in the school year if the parent is only learning this information right on the eve of the beginning of school.  Further, think of the role that local districts play in implementing the remedies and especially some of the large urban districts that Mike talked about.  If there are remedies that need to be put into place that involve more dramatic action, the corrective action or structuring kinds of things, imagine having all that on your plate or learning about that at the same time that you are simply just trying to get school started.  If anyone in this room has been a teacher (I was a teacher before entering this life), you know the start of school is a hectic time and those last two weeks of August go like that.  So to have this additional knowledge coming in as you are trying to get the regular year started can be quite challenging.  Let me talk next about supplemental services.  I ll talk about what Jeff called the unpopular kid.   I ll give the kid some stage time here.  This is one other area that I examined in the paper, and as you know and as Jeff briefly summarized there, the main state responsibilities for supplemental services, there are really three things involved, right?  States are supposed to develop a list of supplemental services providers, the approved providers.  They are supposed to disseminate information about those providers so that local school districts know who they are and so, importantly, parents know who they are.  And also the states are supposed to hold providers accountable for results.  If you look out there at the provider list the states have put together -and this is one thing I analyzed systematically for the paper - I got a hold of all of the supplemental services provider lists that the states have put together.  So the document that a local school district or parent would see when it was trying to decide which provider to choose, I systematically analyzed those lists, looking for 21 specific provider characteristics; and when you look in the paper there is a logic to the 21 characteristics that I looked at.  I m just showing you a handful of them here just to give you a flavor of what is in the paper.  Here is just a quick summary of a couple of main points that come out and these particular elements illustrate the point.  First of all, most of the state provider lists do a pretty good job of giving sort of basic information about the provider - a phone number to call; obviously, you know the name of the provider; as you can see up here, the geographic area that the provider serves, right?  Some providers claim that they will give services anywhere in the state; others say We are only going to serve this particular school district or community, and the vast majority of the provider lists actually do contain that information.  I was surprised though that not every state SES provider list has a phone number for the provider.  Not all of them do that; it is over 80 percent.  About 85 percent have the phone number but not all of them.  I mean, that seems like a very basic piece of information that you would want to have in there if a parent is going to take advantage of this kind of remedy.  One area where it is clear that the states struggle is on the issue of holding supplemental providers accountable for the results, and if you see the sort of three lines in the middle there on the overhead, it suggests that in these provider lists, parents get very little information about how their provider is going to be reporting results back to the parent or the school, how often that will happen, and if there is any evidence of the provider s tutoring services having been effective.  What kind of data do they have to show that their work has been effective?  Now, that part is particularly troublesome if you think that one of the main things you would want a parent to look at when choosing a provider is whether the provider helps the child to learn material, helps the child to catch up.  What research base is there for that and how is the provider held accountable?  There really is not much there in explicit terms.  What most states -- or I guess not most; actually about half of the states do provide however - and that is the last line up there - they provide an opportunity for the provider to insert sort of a narrative description of the program.  And so you will often see sort of some nuts-and-bolts info and then sort of a paragraph that the provider describes itself.  Oftentimes those things do mention some of these issues about accountability and evidence of success but it is not presented systematically.  So imagine if you were a parent and you are trying to pick a provider, do you actually want to wade through maybe 30 or 40 paragraphs to try to spy on the ones that actually mention the accountability issue?  That might be a very difficult task, and like I said, most states do not actually single that information out and present it.  Okay, the last thing I want to talk about is corrective action and restructuring those remedies.  Really, one of the things that I found, and that I think others in the panel here and other panels have found, is that gathering systematic state-level information on these two remedies in particular is very difficult.  To know exactly what is happening across the states is hard to know.  Here are just some examples of things that in my own work it has been difficult for me to discern.  You would think that at the touch of a button, you would hope that at a touch of a button, five years into NCLB, we would know this information, but we actually do not.  This is really crucial, especially if you think about the options that are embedded within the corrective action and the restructuring remedies.  It is bad enough that we do not really know how many schools in each state are at each level of improvement or at these levels of improvement in particular.  If we want to know how these remedies are working, what we would really want to have is information on which particular flavor of restructuring has been implemented sort of most consistently in this state compared to this state.  What are the different remedies?  Because the remedies, like Jeff said, they are the levers that policy-makers think we can pull.  If we pull this restructuring lever in this way, we will hopefully get an outcome that we like.  But right now we do not even know systematically across the states which levers are being pulled in which areas.  So just let me wrap up with a few brief points.  The first one is, again, I think it is important for us not to over-state the federal role in the nation s schools; even though NCLB is a major law, a lot of what happens with this law depends on state choices and, as Mike was saying earlier, local choices.  So the law is still unfolding and being implemented in a fragmented system of governance that we have for American schools across the country.  And so given again these internal political climates in the state policy choices, et cetera, those kinds of things will continue to have a tremendous impact on how the law s remedies are unfolding.  So thank you very much. Frederick M. Hess:  Thanks Paul.  Mike? Michael Petrilli:  Well, I was going to start by saying that I was the kid in Jeff s class who was going to get in trouble for not doing what he was told because I do not have a PowerPoint presentation.  But at least when I get put in time out I ll have some company - Jeff would be joining me and that makes me feel better.  As Rick said, that is popular in Washington today, to declare support for a policy but complain about its implementation.  We certainly see this in the Iraq debate where even the neo-cons in this very building who advocated for the war and for President Bush s vision of establishing a beachhead for democracy in the center of the Middle East are now complaining that the mission has been botched.  And, of course, we also see it in the debate over President Bush s most ambitious domestic policy, the No Child Left Behind Act, which tries to establish a beachhead in the battle to close the achievement gap.  Now, members of Congress and advocacy groups and think-tanks who support No Child Left Behind have been increasingly vocal about its implementation.  For example, recently, Senator Ted Kennedy, soon to be chairman, recently said of No Child Left Behind:  The administration s implementation of the reforms has been inadequate and ideological.  Its ineffective implementation has undermined the reforms it said were so important.   Now, what all of these people seem to be saying and believing is that with the right people calling the shots in the US Department of Education, making good decisions and acting wisely, the law could work as intended.  So in other words, when President Clinton comes in to office in 2009, and she appoints Secretary Broad or Secretary Bersin or Secretary Haycock - I could keep going but I ll take charge of implementing the law, all will be well.  Now, this argument, of course, has huge repercussions; if it is true that all that is wrong is implementation and, especially, federal implementation, then the law itself does not need many changes.  So exploring this contention was the purpose of my paper, at least for some of the law s remedies.  Now, I should admit upfront that I am not exactly an unbiased outsider.  I served for four years under Rod Paige and helped to implement some of these policies; and since leaving the administration I guess you could say I have been a somewhat vocal critic of some Secretary Spellings decisions.  So keep all of that in mind when you consider my basic conclusion which is this:  At least when it comes to public school choice and restructuring, I have come to believe that this law is basically unimplementable.  So let us get to that.  So in the paper I looked at three policy paradoxes, the dilemmas that the Department of Education has tried to unravel, has had to try to unravel.  Number one, in order for the public school choice and free tutoring provisions to work, local school districts must take aggressive actions to inform parents of their options, yet districts have little incentive to do so; number two, students and schools in need of improvement are to be provided with options within the same school system, but in many big city districts there are not enough good schools to go around.  And paradox number three, school districts are supposed to restructure persistently failing schools, yet these districts rarely have the inclination or the political will to do so, and some loopholes in the law make this bold action even less likely.  I do not have time to discuss all three so I ll focus on the first and consider what the department tried to do to address it and why it has failed thus far, and this is informing parents of their options.  Now, it almost goes without saying, in order for parents to take advantage of school choice programs, they must know that they exist, and this is a huge challenge.  After all, we know that corporations spend billions of dollars trying to get us to buy soap or cars; cutting through the fog of information, especially on a shoestring budget, is never easy.  This is even more the case when the target audience is poor and overworked and, perhaps, even overwhelmed by life.  Doing it right means going into the community, knocking on doors, making announcements at churches, posting flyers everywhere, showing up at festivals, blanketing the airwaves, and on and on and on.  In other words, it is a whole lot of work to do this right.  So under even the best conditions, informing parents of their options under No Child Left Behind would be difficult.  But of course the construction of the law creates a unique problem; it requires the districts to inform parents of this option, yet doing so is at odds with their own interests.  Again, many big cities do not have any school choice options to offer to parents and when it comes to supplemental services, few are all that interested in having parents go to private providers or at least because the law is not use it or lose it in terms of the money for choice and SES, but use it, or if you do not you can use it for whatever else you want to use it on.  There is not a lot of incentive for districts to make sure parents are using these services.  Now, these problems were easy to predict as soon as the law s ink was dry.  In fact, at the last conference Rick and Checker held, these were all raised.  So what did we do to try to address them?  The first thing we did was to try to appeal to district s better angels and to encourage them to do the right thing.  So for example, we published these very colorful innovations and education guides, 50,000 each sent in to all the school districts in the country, held conferences, talked about concrete things districts can do to inform parents.  What did we learn from other school choice programs?  What have other districts learned from their own school choice programs?  But let us be honest; it was not really for lack of know-how that most of these districts failed to inform parents effectively.  Again, those pesky perverse incentives and there own struggles bureaucratically to inform parents really about anything had not gone away.  So we adopted a second strategy - if you cannot work through the districts, work around them; in other words, empower the outsiders.  Through the Secretary s discretionary fund, we made some grants to several advocacy organizations dedicated to informing parents of their options such as BAEO and CREO.  So BAEO, for example, launched an aggressive outreach campaign in cities like Detroit, Atlanta, Philadelphia.  They used a mix of radio ads, grassroots communication and media relations.  The project had some success.  Knowledge of the NCLB options increased in their target cities from 37 percent to 72 percent over three years.  But these funds for these kinds of projects were quite limited, could not have much of an impact.  That is when we discovered the parent information and resources centers, parent information and resource centers program or PIRCs.  This certainly sounded like a program that was about informing parents of their options.  It actually was not, but that was okay; we kind of used some application language to get these PIRCs to do the work of informing parents of their options and soon about 70 of these organizations around the country were doing so.  Now it is hard to know whether any of these non-traditional methods going around the districts worked and even again in such a big country with so many children eligible, even these activities have to be seen as mostly symbolic.  Now it soon became clear to us that what mattered most was whether districts bought in to the choice and tutoring provisions and decided themselves to launch aggressive outreach campaigns; after all, they have particular advantages.  They know who exactly is eligible for choice and tutoring and nobody else has that information.  They have the power to send information home in students backpacks or, even more important, to instruct principals and counselors and teachers to tell parents about these options at back-to-school night or at report card night.  And as our innovations guides found out when we talked to districts, it became very clear that parents mostly trusted information that was going to come from their teacher or their principal.  So with the arrival of Secretary Spellings, the department tried a new tack.  It tried to replace some of the law s perverse incentives with different incentives, those that would encourage districts to play ball.  In other words, it said let us make a deal.  It launched two different pilots; first in August 2005, it allowed four districts in Virginia and later in some other states to flip-flop the order of public school choice and supplemental services.  And then, second, four urban districts that were in need of improvement were allowed to provide the tutoring services directly.  In both of those cases, the deal was you have this new flexibility; in return you need to do much more aggressive parental outreach and show that you are getting better participation rates.  Evaluations on those programs are not due until February at the earliest, so we do not know yet if they are working.  And I think as some others will argue, allowing these districts to serve as tutoring providers could have some other deleterious effects, though I know Mike would disagree.  But no matter what the impact in this handful of cities, still the small pilots are not going to have much of an impact on the larger national picture.  Now, one strategy, of course, the department has not adopted is getting tough with wayward states and districts.  Now, it is not necessarily true for the law as a whole.  There have been instances when the administration has taken away money for states or for districts for not implementing parts of the law.  So for example, states that are not testing new teachers as is required under highly qualified teachers provision or there is not testing LAP students as required.  In some cases, the department has withheld administrative funds.  So why did we not take the same action when it came to choice and SES?  The answer is pretty simple; the examples above are pretty much black-and-white.  This is a matter of gray.  In almost all cases, districts were following the letter of the law.  They were sending letters home to parents.  Those letters might have been full of jargon and written in eight-point type and really done everything they could to discourage parents from taking advantage of the choices, but they existed.  And at the end of the day, going through the motions is not illegal.  In other words, the department did not have legal grounds to take action.  All right, so let us recap very quickly.  None of the department strategies appear to have made much of a dent, though I would argue we tried to be relatively proactive about trying to make this work.  So let us return to the original question posed by the paper:  Was implementation, especially federal implementation, the problem?  Now, to be sure there have been mistakes.  We could have moved faster to model the kind of parent information and outreach expected from districts or worked harder to do something about getting states to release the information sooner.  We could have issued a regulation that would have made it harder for districts to roll left-over money intended for choice and tutoring into other programs, creating stronger incentives.  And we could have made an example out of some districts, if we could find them, that were clearly not following the letter of the law.  Now, these actions might have helped at the margin; I do not think they would have changed the basic story line though because as far as I know, nobody inside or outside the administration has figured out ways to fundamentally solve these problems.  So I think it is hard to argue that implementation, or better implementation, is going to really change the story.  And it does not, though, mean that changing the law is an answer either, though I m happy to offer a few suggestions when we get to Q&A.  The hard truth is that there may be no solutions to these problems because they are inherent in our federal system.  If there is one lesson policy makers and all of us should take away it should be this:   While it is hard to get recalcitrant states and districts to do things they do not want to do, it is impossible to get them to do those things well.  So in other words, we can coerce districts and states to follow the letter of the law but we cannot coerce them to follow the spirit of the law.  And when it comes to these things like informing parents, creating new schools of choice and overhauling failing schools, going through the motions is not good enough.  Thank you. Frederick M. Hess:  Thank you Mike.  Checker? Chester E. Finn:  Well, in order to say something, let me thank Rick and his terrific team for pulling this conference together as well as they have done and also our very fine authors for yeoman work on these papers.  As I re-read the papers for this panel, and most of the papers for the other panels, I began to think that instead of handing out coffee and sweet rolls over there, we should be handing out mood-altering pharmaceuticals, the kinds that deal with depression, because, fundamentally, this collection of papers tells a fairly depressing story five years out.  It is fair to ask whether five years out is the right timeframe in which to begin to pass judgments, and it is five years.  If you remember, it was those crazy months after 9/11 that Congress finally returned to and buckled down to and finished work on the No Child Left Behind Act, which was in late 2001 and it was then signed into law in early 2002.  So it is five years out.  We know that any large, complex, ambitious federal law requires fine-tuning and revision and is not going to work perfectly out of the gate.  That I think goes without saying but I think the question as we move into reauthorization season for No Child Left Behind, and we have no idea how long that season is going to last, but the question as we move into reauthorization season& I do not have any answer to this one today but the question is can the problems be fixed through amendment?  Or are we, as Mike Petrilli just suggested, dealing with something much more fundamental that goes to the very nature of American educational federalism and a kind of large over-reach by the federal government aspiring to accomplish something which the federal government is actually incapable of making happen no matter how it writes the statute.  I think that is the question and if you go back to what it was we were trying to accomplish here, what is the country trying to accomplish, let us do keep in mind this was because kids were not learning enough, are not learning enough; poor kids especially are not learning enough.  Achievement gaps are way too wide and a lot of schools are not every effective in solving those problems is what this was all about.  So the appropriate criterion against which to judge all this is whether those problems are getting solved, not just how our various sort of programs working in a kind of bureaucratic or implementational sense.  There is a little bit of good news in terms of student achievement among poor kids in the country over the last five years but it is really unclear whether it is attributable directly to No Child Left Behind or perhaps to the spirit of standards-based lifting all boats that No Child Left Behind is part of across the country and that actually pre-dates and may survive No Child Left Behind.  There is not a lot of progress, however, in getting schools and districts in need of improvement off those lists.  In fact, Mike Casserly s paper makes pretty clear that the number of schools and districts on those lists of needing improvement keeps growing, not shrinking.  There is incredible variability by state, a point made not just by Mike Casserly but also by Paul and Jeff.  America s basic public education governance structure explains why that is the case but in the context of a national law, it actually seems sort of insane that there is this much variability across the country by state.  There has been very little use of the public choice program for a host of reasons though, interestingly, there is considerable use of other kinds of school choice in American education today.  There are a variety of reasons; one that pervades almost all of these is how late in the year the NCLB results come back.  That goes to a different problem which I do not believe any of our papers even gets to, which is the complete inadequacy of the American testing industry and testing enterprise to anywhere near live up to the challenges of speedy, accurate reporting that this law presumes will happen, and that has to happen in order for this law to work as intended.  The SES utilization is greater than public school choice but it, too, is plagued by late notification as well as by the kind of underlying confusions that I think Jeff s paper unpacks as to where did this come from and what is it supposed to do.  The corrective actions and restructurings, I think it is clear from everybody s papers, have been relatively mild.  Mike Casserly uses the word softer, kind of friendly interventions at both the school and district level, not drastic, dire, disruptive interventions.  Why is this working so unevenly, clumsily and for the most part ineffectually?  Well, part of it is, no doubt, ambiguous and loophole-riddled statutory language; that, at least in principle, could be corrected through amendment.  Part of it certainly is implementation at the federal level and everywhere else.  But I really think the overriding problem, and Mike Petrilli was there at the end of his comments, is the federal government actually lacks the leverage to effect the changes contemplated by NCLB except in situations where states and districts want to make those changes, actually want to make those changes.  In other words, the law itself is a huge over-reach, given the limits of the federal government s ability to alter the behavior of states and districts in America, leading to Mike Petrilli s word, unimplementable.   If we were living in France or Singapore or a variety of other places we would be having a completely different kind of conversation; then we really would be able to focus mostly on whether kind of the central administration of the Ministry of Education was doing its job right.  But when you are living, when we are living in as loosely coupled a federalist structure as this one, with as little leverage as there is coming from Washington and as many other moving parts at the state level, the district level, the school level and then all those other things, like for example the testing industry, it will all have to kind of work really well and work in harmony and work in sync with each other in order to change.  And this is an enterprise famously full of institutions and places that are kind of set in their ways and are not really keen to change - you can begin to understand why this is not working.  Let me quote from Paul s paper because I think he says this really quite crisply in his paper:  Absent a major revolution in educational governance in this country, both federal and local officials will continue to rely upon state policy and leadership to improve the educational fortunes of thousands of American children who attend struggling public schools.  The continuing persistence of state and local control of education means that NCLB s ambitions and federal officials promises to strictly enforce the law continue to collide head-on with the primary institutions that control American schools.   Jeff s very insightful case study of the political economy of the SES program helps explain sort of how it came to be that Washington lawmakers created a program, and that is one program within the law, of course, that probably cannot work as intended.  He writes of the SES program s thin and tangled intellectual roots, his phrase.  His paper, in fact, reminded me a little bit of Pat Moynihan s famous book about LBJ s community action program titled Maximum Feasible Misunderstanding.   So many disparate notions and expectations and theories came together in one program, sort of co-mingled there by an over-ambitious or accommodating or compromising or na�ve Congress.  So many disparate notions came together; there was actually no possible way that the resulting program could in reality be implemented to the satisfaction of any one of those theories or any one of those interests.  In NCLB as in community action, I think it is fair to say successful implementation would require the active and sophisticated coordination and subordination of many, many different adult interests and power centers and roles.  And yet it is not really in their collective interest& it may be in their collective interest, but it is not in their individual interests to make this work.  The collective benefit would only be that of children, and they, of course, do not have power centers.  They do not have lobbyists.  They do not have bureaucracies.  There is no children s bureaucracy in America.  There are only adult bureaucracies and it is those that would have to change in order for the kids to benefit. I do not think anybody in this whole equation has the leverage within our system to actually make change, force to change, cause to change if they do not want to change.  If they do want to change, this can go great, and there are places around the country where it is going pretty well.  But there are places that for their own reasons want to do it right, want to change, want to do it differently.  So let me just finish where I started.  I think the big question, and I hope we can return to this at the end of the day in the closing panel where we have Kati and Mike, Diane and others up here, I think the fundamental question is whether the problems being surfaced today can be solved through reauthorization, or whether they go to something far more fundamental than anything Congress can or is likely to do as it amends this law over the next few years?  Thank you very much.  Frederick M. Hess:  Jeff, let me go to you to kind of start the conversation.  In some sense, your paper ends in a more optimistic note than Mike s, particularly when you talk about capacity building or investing in local ability to make SES work.  You actually seem to suggest there is more room for refining this thing than Mike or, say, Paul s paper seem to suggest.  You want to speak about that? Jeffrey R. Henig:  Yes.  I do not want to be a sort of an apologist for all elements of the legislation but I do think it is important to think clearly, as Checker said, about the difference between evaluating an immature and a mature policy regime.  I think there are many instances in which federal government or State governments have overreached and, by overreaching, have kicked into action responses that ultimately play out to the benefit.  And I think there is potential & just speaking in terms of supplemental education services.  So part of the problem is lack of infrastructure and data management.  Part of it is lack of capacity in terms of understanding how to evaluate provider performance, things like that.  I mean, there is a whole history - and Paul has written about it well in his book - about how federal governments have borrowed capacity from the lower levels of government; and so the fact that the Feds cannot do it themselves I do not think is definitive.  There is just one other point which is one thing that I think the law could do better, but there is an element of it as well, is thinking about how do you actually improve the political environment at the local level in ways that lead to greater willingness to try to take advantage of the positive aspects of the law?  I think the Federal Government from Washington can do that sometimes.  There were elements of the community action program and things like that in terms of public meetings and public information requirements that empowered groups that were somewhat disempowered.  I think the disaggregated data aspect and the public test board aspect of the law has that capacity and probably already has that effect on the ground; that it is a new weapon for local groups to use to bring pressure on the local districts also. Michael Casserly:  If I could pick up on one thing that Checker mentioned, I think he is absolutely right about this kind of issue about the said cohesion of the law.  I mean, if you think back to when this thing was first authorized, in many ways what really happened here& and anybody who has ever worked at Capitol Hill before kind of understand this dynamic.  In an attempt to get a pretty bipartisan bill, both sides, Republican and Democrat, had a number of different options on the table that they wanted to put into the legislation.  But rather than thinking through how all of these various pieces would fit together, everybody simply said, Yes, let us take all of them.   And consequently, nothing locked together in any coherent way and it, I think, ultimately became the seed of a very unimplementable program.  Second thing I would mention though that we have not mentioned yet here is the law really in many ways, both from its beginning other than its grand intent about raising achievement and closing achievement gaps, never had any theory of action behind it around how to actually improve student achievement.  And there really was very little capacity in the U.S. Department of Education about how the goals of NCLB might be met.  And there is certainly very little capacity in the individual states about how to raise academic performance.  So in some ways, I think what ended up happening was that much of the action at the federal level and at the state level and sometimes at the local level, too, really became about compliance with the nuts and bolts provisions of the law and not about what it would take to actually raise student achievement, an issue that we thought the legislation was supposed to be about but is hard to find at any place in its implementation. Frederick M. Hess:  There is an obvious irony there, which was the entire logic behind 1994 reauthorization people were concerned that ESEA had become a compliant framework, which was getting in the way of actually making these dollars help. Michael Casserly:  And the response was let us do some more of it. Frederick M. Hess:  Mike? Michael Petrilli:  Well, I think what is important to understand about, especially the choice pieces of this, is that in fact, the theory on one hand was that it would be another hammer, another way to [indiscernible] certain behavior to break through some political barriers to making important change but the only people that could actually implement it and enforce it were those same people whose behavior we are trying to change.  And I want to be clear; when I say that these parts of the law are unimplementable, these specific parts, I did not say that about supplemental services.  I am a little more optimistic there.  I certainly think that the testing and accountability framework is implementable.  Every state breaks out data by race; that is a huge and important innovation and I think it has had some good benefits.  The question is, what can we do to provide political cover to the local superintendents and, to some degree, the State officials who want to, in my opinion, do the right thing, push for a change, push through the political barriers to change without creating this overreach, without over-promising?  So when it comes to choice, for example, I think we have to perhaps now admit that, as compelling as it is to say every child in a failing school will be given a choice, we cannot deliver on that promise.  So maybe we have to say, okay, let us think about a grant program that provides funds to school districts that want to do public school choice and then can be held accountable for doing it; it is only 100 of those districts instead of 15,000, but if they are the right districts we might get more leverage.  It is not as compelling, exciting.  The rhetoric is not as powerful but I think it would be more doable and, again, might get the results that we are looking for. Frederick M. Hess:  Mike? Michael Casserly:  This issue has been brought up before in lots of different forms about kind of the conflict between the sanctions in the bill and who has to administer them and what that conflict is all about.  And I think that adds to why this thing is so hard to do, and why its implementation is so slow.  But there is no particular reason why you could not take the sanctions in their cascading fashion and redefine them around instructional interventions based on good research, maybe make this a multi-year school or district improvement process in which you have to use various instructional interventions based on good research.  And then follow it at some time certain by your sanctions, rather than having this cascading, ever changing, hard to implement, poorly-timed set of things that just [cross-talking]. Frederick M. Hess:  So what would that look like?  What do you have in mind? Michael Casserly:  We have not thought of it all the way through yet but we did suggest to the NCLB Commission that maybe what they could do is, in terms of the reauthorization, was, like currently, have a year's kind of planning period and then if you didn't make your AYP targets go into, say, a three-year school improvement or corrective action phase whereby the school district or the schools had to implement various instructional interventions, more scientifically-based curriculum, more coaching, better professional development aligned with the curriculum and the standards and the like, and include maybe choice and sub-services from the get-go.  But define them around the kinds of strategies that you would need to raise achievement.  Then if you could not get & if you could not make your targets by the end of that specified period, three years, four years whatever it is, then think about either closing the school or reconstituting or something like that.  But it would give people the time to actually implement something before you started to phase in all of the sanctions and made them space it out.  It would also help you solve this problem that Paul mentioned about the late submission of data that would occur, basically, at the beginning of that cycle.  But then you would have three years or four years to adjust and to design your program, and the like. Frederick M. Hess:  Though it seems to me that there are two kind of operating theses embedded in what we are talking about.  One is the capacity thesis.  So when you are talking about the three-year phase, then the notion is that the technical assistance is not working, that people do not have a clear sense of what constitutes an effective scientifically-based curriculum and such, and that giving them the three-year window and the additional lag time, but then, I guess, the second is the Petrilli thesis is, well, that is nice, but it is not a problem of capacity so much as design and will.  Michael Casserly:  Now, I really take issue with that. Frederick M. Hess: Okay. Michael Casserly:  Every piece of data that we gathered suggests that the school districts have been making better efforts at providing choices earlier in the school year.  Opening their windows for longer, having rolling windows and the like, providing the information in more languages does not mean that it is everything that the choice advocates want, to be sure; it is not.  But on the other hand, it does not suggest that the school districts all over the country are deliberately dragging their feet to make this happen because, as I have pointed out in our research, the number of kids who are actually availing themselves of one choice option or another, NCLB or not, is really quite high.  And one of the main problems for the NCLB choice, as Checker pointed out, was that its timing is after all of the other choices have been made and the parents have kind of the worst options and the fewest numbers of schools to pick from.  One way to solve this would be to sync that up a little bit better, but it does not signal that somehow the low participation rates are the result solely of resistance; they are not. Frederick M. Hess:  Paul, so at the state level, help us think through these kinds of competing tensions.  Take, for instance, the identification of school year.  How much is the capacity an issue?  And how much of it is actually an issue of will that states and/or testing companies could get these schools identified much earlier if they actually wanted to push. Paul Manna:  I guess one of the things to think about in terms of capacity (and someone made a comment a minute ago that we wrote this law that was supposed to not be based on just sort of compliance and filling out paperwork and it was supposed to actually have real results on the ground) well, if you think about the thousands of people who work in the education industry, public sector across the country, for years and even still with lots of Federal and other grant programs, those people were trained to use James Q. Wilson's term, critical tasks.   The critical tasks for people in those agencies were sort of compliance-oriented kind of processing.  So the key task for a state education department for years was make sure the money is going out to the schools and keep track of where it goes and then make sure it is spent correctly.  Do not necessarily worry about the result at the end but just audit the books to make sure it was spent the way it should.  And that is the reason why you had the invention of these pull-out programs Title I because it is a great way for a local district to prove that it spent the money on the right kids.  Whether it had anything to do with learning was anyone s guess at the time.  I think we now think that it really did not help necessarily at all [indiscernible].  And so to ask compliant people who are trained and have for decades been working in organizations that are designed to work on compliance-oriented activities, it is not surprising that that is where everyone kind of migrates to; they kind of migrate to their comfort level.  And so it is hard.  Some of the state officials that I have talked with for this sort of commented on that, that they make the comment that it is hard for them to work with local districts sometimes because principals, even, are sort of the managerial types on the ground.  They are not instructional leaders.  They are not people who wrap their head around the curriculum and what is going to make the students learn.  Some do, but by and large that has not been how we have trained them in this country, and so we need a real kind of shift in those capabilities.  When I think of capacity, I think that is one of the main issues. Frederick M. Hess:  Okay, Mike.  Then we will open it up to questions. Michael Casserly:  I think one of the great ironies here is that NCLB was really at the outset billed as kind of a follow-through to the standards movement, kind of the next steps in the standards movement.  But the great irony here in lots of ways, in the standards movement again, was in a lot of ways about defining our work, about the results that we got and the outputs, rather than the input.  Ironically, NCLB has unfortunately devolved to a place where it is now about the process and not about the results so much.  It has become very compliant-driven in ways that the standards movement was supposed to transcend and did not. Frederick M. Hess:  All right, let us open it up to the audience.  You will find Juliet over here and Rosemary over here with microphones.  Again, please wait for the microphone; please identify yourself by name and affiliations, since many in the room may not know each other, and again please actually ask a question or else I will truncate your remarks.  Jay Green:  It is a little bit of cheating for me to go since I am going to be speaking later.  I am Jay Green of the University of Arkansas, but I liked all the analyses I heard.  But I heard this incredibly gloomy conclusion about the impossibility of the Federal government to do certain things.  And it seems to me that is inconsistent with other evidence.  The evidence is that the Federal Government actually does lots of things quite successfully, including in education, where it imposes things upon schools that they do not want to do.  One example I can think of is special education.  How was the Federal Government able to succeed in getting local districts to provide services to disabled students when previously they did not always want to?  How were they able to succeed at that but they cannot succeed here?  And that is my question. Frederick M. Hess:  Okay, so Petrilli then Casserly. Mike Petrilli:  So I guess I would ask Jay if he thinks that schools are providing services to kids with disabilities as well.  I mean, obviously that is an enormous compliance framework, and as Mike said this is now supposed to be about results.  But the trick is& I wrote a piece a while ago about how No Child Left Behind includes the what-works and the whatever-works theories of action.  And the whatever-works people say, Look, we are just going to focus on results and get however you want to achieve them, and that implies that people could get better results but they do not want to.  And they do not want to because of their own self interest, political interest, the bureaucracy, the union, et cetera et cetera, but that we are going to change that dynamic by having some kind of hammer.  I guess what I am arguing is that it is clear that the choice hammer is not working because the people that are supposed to pull the trigger on that are the same people we are trying to change the behavior of.  I am more optimistic that the accountability hammer can work if done right, but that is still an open question mark.  So if you want to go to the special ed. model, I think that is all about compliance.  I think what is tricky is how to make the results-based accountability work in this system and who is it that is going to pull the trigger. Michael Casserly:  I will make my comment real short, Jay.  I actually end my paper optimistically, and I think NCLB can work, could work, if we do it right, and in some ways there was really something of a paradigm change.  A lot of people have talked about what NCLB was supposed to represent.  I guess I m not terribly surprised in the grand scheme of things that we did not get it right the first time out, and that it is going to take a couple of more iterations before we really fine-tune this in a way that is effective. Frederick M. Hess:  Paul? Paul Manna:  Just one quick thing.  I think one thing that IDEA does is it empowers parents and their lawyers to get the things from school districts that, maybe, the districts might not be willing to apply and to provide.  And in the NCLB context, one thing that it seems like the Federal government could do a very good job of if they are really committed to it would be the information provision issue.  The fact that now we see breakouts of test scores by student subgroups is a powerful thing.  We see now that the Richmond public schools in Virginia are doing better with African-American kids than the Fairfax County Schools in Virginia.  That would have shocked people I would say 10 years ago before we saw the numbers.  And so if the Federal government could take seriously the information gathering and dissemination function so we could know where these remedies are happening, that would be a really powerful thing that I think it could do which sort of teeters toward what I think Rick has called suggestive accountability in another context.  It provides information, but that information can empower people to put the kind of pressure on that might be really useful.  I think that would be a great thing for the re-authorizers to think carefully about in the next round. Frederick M. Hess:  Checker? Chester E. Finn:  Well, let us keep in mind that special ed. is still fundamentally about providing services, not about delivering results.  There have been some efforts at results accountability layered on top of all the service-providing mandates, requirements, rights, litigations and so on in special ed.  But it is still fundamentally not about are the kids learning; it is fundamentally about are they getting services?  NCLB is fundamentally about are the kids learning. Michael Casserly:  It is supposed to be. Chester E. Finn:  It is supposed to be.  We do not have a 40-year tradition in the elementary and secondary education act of machinery that focuses that way.  I think I am seeing slightly differently what everybody else up set up.  In 1965, when this law was first passed this is the successor to the original ESEA of 1965 Frederick M. Hess:  And when Checker was already a young man. Chester E. Finn:  I was already an old man.  It was about getting money out for additional services to be provided to disadvantaged kids, and the logical way to do that was to give the money to the existing hierarchy - state education agencies, local education agencies because they were the ones that had the other money; and this was a perfectly logical way to get money out.  But it was not about results; it was certainly not about interventions and restructurings and corrective actions and all these other things.  We are still using the Federalism Architecture of 1965 with the federal government assuming that state education departments can do something, which, in turn, rely on local education agencies to do something.  That was one way to hand out money, but it is not a very good way to bring about structural change in the very agencies that are in charge of the implementation of the program. Gerry Bracey [phonetic]:  Hi, I am Jerry Bracey, independent researcher.  Most schools have 37 subgroups to report out.  If the school is working for 36 of them& let us say it is working for everybody, 36 of them are making AYP and maybe the special ed. kids are not, why should the other 36 groups have the capacity to take the choice option?  That would surely simplify that hammer. Michael Petrilli:  I think it is a reasonable question.  When the law was going through, the concern was that if you narrowed the choice option to only kids who were doing poorly, you would create an incentive for schools to try to push those kids out.  So they would say, Hah!  If we could get rid of those 10 kids we would make AYP.   That was the conversation.  I do not know if those figures are founded or not but that is why the law was written as it was.  I guess I am done. Kwesi Rollins:  Kwesi Rollins with the Institute for Educational Leadership.  I am curious about national standards and America's testing capacity.  It seems to me that& well, I would like to hear from the panel if that is really part of the solution, long-term.  It seems if you had national standards and you had the capacity to have all test results back at a reasonable time, the immediate result would be more schools not meeting AYP and kind of a huge crisis.  It seems that part of the solution is scope and sequence of some of the concepts in the tool kit.  I wonder in a system like if you use DC for an example where currently maybe 2/3 of the schools are not meeting adequate yearly progress, you have a fairly robust choice system, at least on paper in terms of the number of charter schools.  But they are not doing well either, so parents do not have choice.  So where does national standards fit into all of that? Frederick M. Hess:  Mike? Michael Casserly:    I think national standards will, maybe, cure some things but it is not going to cure everything by any stretch of the imagination, and certainly not going to solve this problem about the late submission of data.  As a matter of fact, it is conceivable it could make it a little bit worse.  On the other hand, if the United States has any pretense at all about being first in the world in reading and mathematics and science and in other subjects, leaving it to 50 individual states to define what proficiency means is really nonsense.  It is no way for the country to get there.  And it is impossible for any individual state or district to really compare themselves across state lines in ways that will help you figure out what works and what does not work.  The standards alone probably would not do it.  You would have to require the states tether their individual tests to those standards with common definitions of what proficiency are, and then you could maybe have some hope of creating some logic and some coherence around a national education strategy. Frederick M. Hess:  I am not going to let Mike and Checker address this because it will go on forever.  All right, one quick sentence. Michael Petrilli:  Thanks Rick.  That is what I get for& you know, he is my Podcast buddy.  Okay, so very quickly, we could imagine a system whereby there were national standards and tests, a real national accountability system that measured every school the same that determined which schools were in need of improvement; and then stop.  And the states and the local districts would decide, then, what to do about those schools that need improvement.  That might be a more workable federal role, to be the performance measurement system but not to try to get involved in the actual sanctions and remedies. Jane Hannaway:  Jane Hannaway of the Urban Institute, and, again, I am cheating also because I ll be speaking later.  But I m intrigued by this discussion of the lack of congruence of objectives across levels in the system, but no one got down to the level of the teacher.  And I want to talk a little bit about something I observed and get your reaction to it.  In some schools, teachers, in fact, are the providers; they are often hired by corporate entities.  When that happens there, it is a win-win for the teacher because the teacher is earning more money and the teacher is working with their kids. Frederick M. Hess:  And you are referring to Supplemental Services, right Jane? Jane Hannaway:  Yes, Supplemental Services.  And the principals like it, too, because you do not have to train staff.  They know what the standards are so it could be that we just are not quite pushing this far enough, that there is a natural coalition between some of these sub-services providers and what teachers want for themselves and their kids. Jeffrey R. Henig:  There is no doubt that that is what is going on and that for a lot of the for-profit providers, with the exception of the ones who are providing online distance learning options, there is huge appeal to working with the schools and with the teachers, not just in terms of having a site which they can do it on-site, but because what the providers know is that teachers are a trusted source of information in the community.  It is when the teachers tell the parents who to go to or to get services that it happens, and it is an easy way for the providers to respond to the issue of alignment with the state curriculum if they say that it is the teachers who are providing.  So there is clearly a common interest between the for-profit providers out there and the schools and teachers in that sense.  So I think that that is part of the reason why the way we typically think of these things in terms of market versus government is misleading in many respects.  I mean in that sense, I would argue in the program overall and other aspects as well, what you really have is a mixed public and private delivery system evolving.  That is part of what I was trying to get at with the contracting regime.  Those kinds of things can work well; in other words, it is two distinct sectors combining specific expertise and resources.  Or, it can become comfortable and an arena in which teachers get some extra money but basically do a lackadaisical job and there is no pressure on it.  So it is not an answer in and of itself, but it is clearly a part of what is happening. Frederick M. Hess:  Okay, Michael& Michael Casserly:  And I think this situation is actually going on much more extensively than people think it is; and that is why there is a great irony here, that sometimes the private providers are looked at as being somehow more affected than the school districts that failed the kids during the regular school year.  But exactly the same people are providing the services for both. Frederick M. Hess:  Paul? Paul Manna:  This is one area where a couple of the people [audio glitch] states that I have talked to said it complicates the accountability part for them on supplemental services because if you are trying to know whether the supplemental services part created some benefit for a student, how do you parse out sort of the service that the teacher gave there maybe with the 15 minutes of after-school help that that same teacher gave that same kid outside of the supplemental services arena, and just the fact that the teacher knows the kid better, right?  So is it really supplemental services?  You could imagine someone who is in charge of evaluating performance across the state trying to figure out how do you design the research so that you know that it is working and it is not when you have all these things happening simultaneously? Frederick M. Hess:  Okay, one final question.  Martin Torres:  Hi, I am Martin Torres.  I am with The Alliance for Excellent Education.  You cannot see me.  I am around the corner here.  At the end of the discussion, Paul Manna talked about the lack of capacity that SEA has had in carrying through the responsibilities, whether it is because of lack of expertise or personnel.  What can the Federal government do to better assist SEA s in this regard, given that state legislatures do not want to create larger bureaucracies and already feel that they are carrying the brunt of the cost [indiscernible] law? Frederick M. Hess:  All right, and let us keep the responses brief, please. Paul Manna:  Yes, I think one problem - I talked about this in my book a little bit - is that for the history of the Federal distribution of aid to states, it has basically been channeled through the agencies so that has created a lot of animosity in some cases and some sort of incongruence between agencies, legislatures, governors and state boards.  And so, if the states feel like the feds are bankrolling their agencies and the governors or legislators do not really like what the agencies are doing, they are going to try to kind of work around that bureaucracy in some ways, maybe create some of their own institutions.  And so, to say what can the Federal government do, that might be the source of the original problem, the fact that there has been such a tight link between the agencies and the Federal government. Frederick M. Hess:  Mike Petrilli, Mike Casserly? Michael Petrilli:  One thing I say in my paper is that the Federal Government is better at following than at leading so it has been more successful in areas where states have already taken the lead and had some success.  If we have some examples of states that were using their capacity to do good things in terms of remedies and improving schools, that would help the states then make a case that more money would help all the states do that.  I do not know that we have that many states we can point to that have done that effectively, and I think that is an important first step. Michael Casserly:  Let me just pick up on that last point by Mike.  One thing that the Federal government could do better is in the area of research on what actually raises student achievement for many of our students and then go about the process of developing some capacity at the federal level for providing technical assistance to states and to major school districts on what actually works.  Currently, the Department of Education is, for all intents and purposes, a regulatory agency and a bank.  It does not actually provide technical assistance on how it is you reach the underlying goal of the act - better student achievement. Frederick M. Hess:  Jeff? Jeffrey R. Henig:  Just real quickly, what the states need is, really, technical capacity and data management systems and evaluation.  One of the capacity issues that is getting worse is with the growth of the for-profit education sector, the job market for people with those technical skills is quite different, and there is a sucking away of talent and ability from the states.  And that is a critical problem because the people who are overseeing this, even if it is on a contracting arrangement, need to have enough know-how that they can tell whether they are getting what they are paying for. Frederick M. Hess:  I think one point that came out of this that I hope we will touch on the rest of the day is simply how little we know about some of these issues; the authors, like Paul and Mike, are trying to give just a snapshot of what is taking place.  The fact that one has to go out and hand-collect data, and that so much of what was just presented is new we actually do not know that Paul's slide on reporting dates which is not readily available makes it hard for us to talk thoughtfully about what is working and how to make it work.  So I hope that we will talk about that with more specificity in the other panels.  I would like to thank the panelists on this first session for a good job.  We are just going to take just five minutes and then we are going to convene with the second panel, and thanks.  Panel II:  The NCLB Remedies in the States Chester E. Finn:  We are starting Panel II now.  Panel II goes to State Case Studies of NCLB Remedies.  We have three states that are the subject of our case studies.  They are, specifically, California, New Jersey and Colorado.  Additionally, our respondent is the Commissioner of Education in Florida.  So we have in effect four states in front of you, three with papers and one with experience.  The apropos of the final question in the previous panel about what can be done about state departments of education, John can address himself to that if he wants to when his turn comes.  Julian Betts of the University of California at San Diego is going to talk about California.  Pat McGuinn is ill, and so our very own Rick Hess is going to deliver his conclusions and slides on his behalf regarding New Jersey.  And Alex Medler, who is Vice President of Research and Analysis at Colorado Children's Campaign based in Denver, is going to talk about Colorado.  The respondent will then be John Winn from Florida as I said.  The presenters have, if I have got this right, 12 minutes.  And then if anybody thinks they are going to get away in this panel with what Jeff Henig got away with in the last panel, you are wrong.  We are going to end 12 minutes each and have a little bit more time for discussion.  So without further ado or longer introductions, Julian, you are on and it is six minutes until 11. Julian Betts:  In conclusion& I'm going to start by talking about the lay of the landscape in California.  I'll talk about choice, SES district improvements.  I want to focus a lot on in a state as big as California where the State Department of Education has the capacity to handle this and then I'll end with some policy prescriptions.  I believe the land is pretty simple.  The most obvious part is it s a very large state; very large number of English learners.  Both of those represent challenges for successful implementation for obvious reasons.  California implemented its own accountability plan in 1999.  It is quite different from the federal system, and it is a gain [phonetic] system but at the school level, and it has led to a lot of confusion.  The state has pretty high proficiency standards.  I have a couple of citations to provide evidence on that.  What the state is doing is setting for AYP the percentage of students who are required to be at that proficient level; very low, with only one increase in the first six years.  So you can see the starting in spring 2008; things really ramp up.  So the number of schools that are deemed in need of improvement is going to skyrocket starting in 2008-2009.  Already, though, we see a lot of schools in program improvement.  That is the terminology we use in California for failing schools.  Thirty seven percent of schools are in need of improvement.  A lot entered just last year but you see schools all the way up to year 5 of program improvement.  There are very few schools that are about to be in program improvements; that is the black bar at the top there.  The laser isn t quite working.  That is your head, I think.  [Cross-talking]  Schools that were in trouble have got into trouble a long time ago is the basic problem.  I think, though, we see a lot of red flags in these data in a sense that there are a lot of school here that are going to be lifers.  It is going to be very hard for them to get out.  The requirements for AYP right now are quite low.  Thirty seven percent of Title I schools are in AYP; twenty five percent of Title I schools are in year 2 or higher.  So they are in trouble, they have been persistently in trouble.  Last year, only six percent of schools got out of AYP.  It is going to get a lot worse starting in '08.  Male Voice:  You mean, got out of improvement status? Julian Betts:  Yes, sorry, got out of improvement status.  That is right.  Choice is required at all schools in Year 1 or higher out of PI status.  Choice in California has really much to do about virtually nothing.  You see that the number of students enrolled has risen from 3,000 to 8,500; this is in a state with 6.3 million students.  The one year where the state was able to give me data on the number of students who are eligible for choice 2003-2004, we saw participation of 0.3 percent.  I think the right hand column is probably in a way the most relevant; of all students in California, the percentage participating is a small fraction of 1 percent.  So, why is the participation so low?  In the conference three years ago here, I claimed that a lot of the problems in San Diego had to do with timing problems.  In California, schools have roughly one week after they are deemed in PI status to organize busing routes, to advertise them and enlist parents; obviously, a hard thing to do.  Today, that is not a very good excuse.  Most of these PI schools are in Years 2 through 5.  The US Department of Education strongly feels that California districts are not doing a good job with parental notification and is actually doing an audit of the 20 largest districts requiring the California Department of Education to send it templates and actual signed letters from individual principals, and also setting up a time line.  There is the Clint Bowlik [phonetic] legal complaint against Compton and LA unified, as well.  Under federal supervision, the State Department in California has put together a template.  It is very accurate and it is describing to parents what choice is and why they are being offered choice.  It has absolutely no marketing savvy whatsoever.  It does not get to the fact that choice is option until Page 3.  A real possibility here is that some parents see absolutely no advantages to shifting a kid across town to another school in the same district.  Chuck Weiss, who is Ventura County Superintendent, said exactly that; he said that it is not really a true choice.  And also he feels a lot of parents believe that there are not going to be good supports for struggling students, including EL students in more affluent schools across town, which might be right.  In some cases, preexisting school choice programs dampened the demand for NCLB choice.  In Hayward Unified, a mere 10 students are participating in NCLB choice.  The Associate Superintendent, Christine Quinn, told me that it is because of the open enrollment program, primarily, and there maybe something to that.  This NCLB choice boost achievement, we know virtually nothing about that.  An ongoing Rand [phonetic] study using a represented sample of districts nationally is going to be coming out in the future and will give some pretty strong evidence on that.  We just put out a book through the Public Policy Institute of California where we are using experimental evidence to look at the existing choice programs, and we are finding basically the lottery winners show no differences compared to lottery losers one to three years after the lottery date.  And that is pretty important for NCLB choice because NCLB choice in San Diego has been largely meshed in with the preexisting choice programs.  Supplemental education services, I'm a little bit more optimistic here.  You see 98,000 students were participating in supplemental services by 2005.  Remember, that compares to a mere 8,500 in choice.  The participation rates of eligible was up to 7 percent by 2003-2004.  It is probably about double that by 2004-2005.  So this is a program that is starting to have some bite; but, still, the participation rates are low.  What can we do to increase participation rates?  Again, the US Department of Education is keying on lack of good parental notification letters and has asked the state to create a template letter and oversee the largest districts.  The manager of SES in San Diego told me that it is exactly these letters that are the primary barrier to parents getting their kids involved in SES.  And you see here that she says, basically, any parent is going to toss this letter because it is written in legalese.  One SES provider told me that at the high school level, they basically have backed out because in spite of great cooperation from the school, parents and kids were simply not interested in staying after school and doing more work.  About one-sixth of local education authorities in California are deemed in need of improvement.  In the California context, what is going is that there is a state-mandated survey the district fills out; an external evaluator then comes in, vets that process and helps them implement reforms.  I have some concerns about whether external evaluators can be both well informed about the district and truly arm's length.  Chuck Weiss of Ventura says that in the case of Oxnard High, he is basically convinced that the use of this survey got the district out of PI status; so, it can work.  But he named many other districts where he felt the district essentially was lying to itself about what it was doing and how successful these programs were, so no guarantees here.  Is Sacramento an information bottleneck?  Information is going both ways - from districts to Sacramento and back down, and the same between in Washington.  I was somewhat alarmed to learn that in a state with more than 1,000 school districts, there are only 11 staff in Sacramento working on non-financial aspects of Title I.  There are basically 49 aspects of Title I to administer here; choice in SES are two out of 49.  Ann [phonetic] just told me that their staff could spend the entire year just on that and they still could not keep up with federal demands and local district demands.  There are a lot of quotes you can find in the paper where districts confirm that it is just really hard to get information out of Sacramento.  Policy prescriptions.  First of all, for Sacramento, this is going to be ironic; I'm arguing for expanded staff in Sacramento.  Here I am at the American Enterprise Institute calling for bigger, better government, and I am sitting beside Rick.  I also think that the staff needs better training; we heard a lot of this from school districts.  We need improved coordination with County Offices of Education, which are really an underused resource, I think, in California.  What about at the federal level suggestions for reforms to NCLB?  I do not know how many of you read the magazine the Economist, but, allegedly, their style guide has only three words in it simplified and exaggerated.   So, I'm an economist, your guess.  My number one suggestion, first thing we do, we kill all the lawyers.  Obviously, I'm exaggerating here for a fact that  where it all clearly does not belong in there.  But I really think that if we do not replace these overly legalistic letters of parental notification, we need at least to supplement them with something glossy, something that the advertising industry would put out.  San Diego was so disgusted with its own letter that it spent a lot of money coming up with a very glossy brochure that includes a two-page comparison of every single supplemental service provider and what they offer.  This is what should be required. If you want the parental letters of notification, that is fine; this should be required as well.  I think we should make it a norm rather than the exception.  The school district should be able to offer SES rather than choice in Year 1, unless we can somehow fix the testing industry.  And, finally, if we really are concerned about low participation and most districts as far as I can tell not spending their 20 percent allotment on Title I, why not get rid of the income criterion for participation in SES and choice?   Allow failing students who are high- income to participate as well; that might be a good signal for low income families.  By all means, if and when these programs get oversubscribed, give first dibs [phonetic] to low income students.  But, really, if we are serious about leaving no child behind, that should include struggling students who are not low income.  Thank you. Chester E. Finn:  That was a model of concision, as well as humor, as well as actual constructive recommendations.  Thank you very much.  Rick, would you like to put on your McGuinn hat and pretend you are in New Jersey? Rick Hess:  I will.  And I will try to do none of the things Julian did just because I like it when you look frustrated.  There you go.  Pat McGuinn, as many of you know, is a professor at Drew University.  Pat had a medical emergency so I am stepping in.  If you have comments, queries or feedback, please e-mail him.  You will find his contact information available in your packet.  Since he is not here to actually get suggestions or criticism, please send them to him.  All right.  Pat's piece is on implementation in the State of New Jersey; it is titled Equity Meets Accountability:  The Implementation of No Child Left Behind in New Jersey.   Four features of the New Jersey Education Policy context are distinctive - the historical attachment to municipal home rule and over 600 small independent districts, enormous disparities in wealth across districts and the increasingly active role of the courts through their mandates as state in school finance and governance.  Spending gaps [indiscernible] the state judiciary to mandate aggressive school finance reform - New Jersey is famous for this as many of you know - and the 1998 Abbott versus Burke decision, which led to the creation of 31 Abbott districts with extensive judicial and state oversight.  In 1987, New Jersey also passed the nation's first takeover law for public schools and it has been used first in Jersey City in 1989; since then, in Patterson in 1993 and Newark in 1995.  Like many states, New Jersey had to devote considerable time and energy initially into aligning its existing state standards, assessment and accountability policies with NCLB.  This was particularly difficult in the state's Abbott districts which were operating under detailed court-ordered school improvement plans based on the whole school reform model.  Many of the state's worst-performing schools are supervised directly by the state's division of Abbott implementation, while NCLB implementation is run out of the state's Title One office.  A further complicating factor initially was that the State Department of Education tried to implement a number of new state reform initiatives simultaneously with the rule out of NCLB.  New Jersey also faced a number of major data collection or reporting problems that delayed its ability to identify schools and districts in need of improvement and to initiate mandated remedies.  The percentage of schools making AYP in New Jersey has fluctuated since 2003 from 75 percent in 2003-2004 to 65 percent in 2004-2005 to 73 percent most recently.  The number of schools identified as in need of improvement has grown marginally from 496 schools in 2003-2004 up about 25 percent to 574 schools in 2005-2006.  Okay, then.  With regards to the public choice provision, due to the unusually small size of most districts in New Jersey, given the state's 600 plus existing districts, the NCLB choice provision has had little impact.  Most suburban and rural districts are so small they have only one elementary, one middle, and one high school.  In the majority of districts in the state, if the school is labeled In Need of Improvement, there is no other school in the district to which a transfer might be made.  The state's larger urban districts of multiple schools at each level but most are failing to make AYP in the majority of these districts, sharply limiting available options.  School choice in New Jersey is further constrained by a difficult charter conversion process, the limited number of charter schools in the state - there are currently 54 - and the absence of a state inter-district choice law.  In Newark, over the past three years, just 10 children have utilized the public choice provision.  Statewide in 2002, 844 transfer requests were made and 504 of those, or about 60 percent, were granted.  In 2003, the figures were 557 requests statewide and 363 transfers granted statewide.  In 2004-2005, the eligible choice population was 105,000; of those, 978 students actually applied to transfer; of those who applied, 735 did transfer, so something under 0.7 of 1 percent.  Data for 2005-2006 does not yet exists.  As best as can be determined, the State Department of Education has made a limited effort to oversee district compliance with either the public choice or SES provisions.  A federal audit in 2005 criticized the New Jersey Department of Education for delegating this responsibility to the county superintendent offices without any written policies or procedures on monitoring for compliance.  The report found that school districts in New Jersey had done an inadequate job of notifying the eligible families of their SES and choice options.  In response, the State Department of Education has initiated increased supervision of choice and SES across the state.  The results of this effort at this moment are not yet clear.  A 2006 survey of Abbott district websites, for instance, revealed that none of the websites investigated contained SES or public choice forms or lists of SES providers or public choice schools.  With regard to supplemental educational services, the state's management of SES was also criticized by both educators and providers.  Educators expressed concerns about the rigor of the state's approval and supervisory process and the reporting requirements for private providers, worrying that they were not able to get the information they needed to ensure quality or to fully make families aware of choices.  For providers, on the other hand, the biggest concern has been the discretion left to school districts and the state's unwillingness to hold districts accountable for the small percentage of eligible students actually enrolled.  The state has tolerated tremendous variance in SES policies from one district to another.  SES providers also expressed concerns about a lack of transparency in the registration and approval process at the state level and half-hearted district efforts to inform eligible families at the local level.  Some private SES providers report that districts viewed them as ineffective providers of educational services, as a threat to district interests and, consequently, have worked to restrict their operations.  The number of approved SES providers has grown significantly in the state from 70 in 2002 to 159 this past November.  In 2006, 40 percent of the providers, however, were LEAs, or high performing schools; 33 percent were private for-profit companies, and 23 percent were non-profits.  The number of students utilizing choice in New Jersey has remained only a fraction of those eligible for services.  [Cross-talking] Excuse me, thank you.  That is right; the number of students utilizing SES has remained only a fraction of those eligible for services.  During the 2003-2004 school year, 63,000 children were eligible for SES and 19,000 enrolled; that was about 30 percent.  In 2004-2005, 82,000 students were eligible but the number enrolled actually declined to 16,000, which meant that the rate of enrollment had dropped from 30 percent in 2003-2004 to about 20 percent in 2004-2005.  Enrollment for 2005-2006, that is last school year, has not yet been compiled.  The number of students receiving SES in Newark, the state's largest district, has grown from 1,640 in 2003-2004 to 4,325 in 2005-2006.  So, just to be clear, the data is available for at least some districts but has not yet been collected and compiled at the state level.  The number of providers operating in Newark has increased from 13 in 2003-2004 to 20 in 2005-2006.  The biggest SES provider in the city, however, is the district itself.  Newark's after-school youth development program enrolled more than 2,400 students of the 4,300 [indiscernible] enrolled in 2005-2006, or roughly 55 to 60 percent of all students receiving tutoring.  While the State Department of Education has been less active in promoting choice and supplemental services, it has been much more active in assisting struggling schools.  In 2004-2005 the state initiated what it calls Collaborative Assessment and Planning for Achievement teams; that is CAPA teams - and they have an acronym so, therefore, they are neat - which are based on the Kentucky Scholastic Audit model and conduct week-long school reviews in low performing Abbott and Title One schools.  There are concerns, however, about whether the Department of Education has sufficient capacity to ensure that the CAPA teams are actually equal to the challenges they are seeking to address.  While many observers enthusiastically praise the CAPA teams, their effect remains entirely unclear.  When pressed after giving initial hosannas, the New Jersey Department of Education could point to no data whatsoever demonstrating the impact of CAPA teams.  It is also unclear how binding CAPA recommendations actually are.  Schools and districts appear to have great discretion in determining whether and how to implement the CAPA recommended reforms.  Most schools that have reached restructuring appear to have adopted the broad catch-all option with the focus on data analysis, curriculum, and professional development [indiscernible] release nice options rather than structural or organizational change.  Apparently, it is not yet possible to get the definitive numbers on this in the state because nobody has collected them, so it is entirely undetermined with any precision exactly what remedies are being utilized at different schools.  For political and philosophical reasons, the New Jersey Department of Education has hesitated to overrule districts and either mandate or more aggressively suggest particular interventions.  Conclusions.  The implementation of NCLB has been decidedly mixed in New Jersey as the state has struggled to comply with the law while preserving its historic tradition of local control.  The Department of Education initially failed to provide timely AYP determinations or adequate guidance regarding NCLB's remedies.  The state appears to have rectified some of these initial speed bumps and has taken steps to improve its oversight and align its assessment and accountability systems with NCLB.  For details of all of this, obviously, see the paper.  Pat's paper is actually extensive and quite detailed and I do not think I am doing it justice in this quick overview.  Particularly by adopting the forthcoming New Jersey Quality Signal Accountability system, of course, the single accountability system has not yet been implemented, so exactly how this will work remains to be seen.  When it comes to NCLB remedies, however, it is clear that they have had an uneven impact and been implemented unevenly in the state.  Finally, the small size of districts in the state has combined with the lack of an inter-district choice [indiscernible] and a difficult charter conversion process to make choice virtually nonexistent five years into the NCLB regimen.  And despite steps by the state to increase the supervision and standardization of the SES process, it appears districts and schools have used their considerable discretion over the process to constrain the entrance and expansion of private SES providers and the utilization of these services by students.  New Jersey has expanded significant effort on intervening in failing schools.  The CAPA process, especially the focus on early intervention, has provided unprecedented guidance to schools and districts and has encouraged them to embrace a data-driven approach to improvement.  Whether or not this actually will yield any tangible evidence, however, despite the remarkable enthusiasm of the state for it, is entirely unclear. Chester E. Finn:  That was exquisitely timed and nicely done for a paper you had nothing to do with writing.  We bounced from the West Coast to the East Coast, and now to the Mountains.  Alex, another local control state, if memory serves, [indiscernible], Colorado. Alex Medler:  Okay, although I might want to have Rick synthesize my paper - he did such a good job - I probably would not be able to do it as well for my own.  I am going to talk about Colorado; as Checker mentioned, it is a strong local control state, which complicates things.  But I would like to back up a little bit and say that what I'm going to talk about is the power of NCLB.  I would like to define power for these purposes.  For me, power is getting someone do something they would not otherwise have done.  So I could command you all to sit and take credit for it, but that would take no power.  And for the folks that are standing there, I am no more powerful than you if they ignore me.  So, in NCLB, what we have is a lot of activity that could be justified as compliant with NCLB in doing what is required.  In fact, there is nothing going on that the states and districts would not probably have done otherwise, so it remains largely powerless when it comes to the interventions.  In terms of defining the problem, it is actually quite effective and it has done a good job of changing how the ESEA works.  Previous iterations of the ESEA were basically compensatory programs that compensated districts for the inconvenience of having poor kids show up.  And NCLB articulates a civil right that the children in those schools deserve a better education and we need to act on it.  The powerlessness comes in getting any change in the action.  Just for background, Colorado has 178 districts, three quarters of a million kids and 1,700 schools.  It has many small rural districts where choice is not really viable or an option.  Most of the students are located in the Denver Metro area in 14 large districts.  It has a history of and a constitution talking about strong local control, as well as a lot of public choice going on already.  In terms of the problem definition, if you look at the state system for identifying low-performing schools and compare it to NCLB, there are inconsistencies and they have been the same throughout the five years.  The yellow are schools that the state calls excellent or high.    Then there are the last two categories that are, in fact, not making AYP.  So NCLB - is it effective at pointing out that there are schools that the state and the communities think are great schools where there are kids who are not learning?  In the problem definition, it succeeds.  The demographics of those kids are as you would expect, if you look on the left, that is the state as a whole; on the right, the NCLB identified schools.  So they have about a third as many White kids, about three times as many Hispanics, and about twice as many Blacks as the comparison schools.  That is true if you look in and outside of Denver.  Colorado has basically 100 real schools identified for school improvement; 54 of them are in Denver public schools, one district, 46 spread throughout the rest of the state.  The non-Denver schools also have the same disproportionate enrollment of low-income and minority kids.  And they are extremely concentrated; about two-thirds of those 100 schools have 80 to 90 percent minority enrollments.  Looking now to the interventions, let's start with choice.  It is the same in Colorado as everywhere else; there is not much of it going on at all through NCLB.  Out of 53,000 eligible kids, almost 900 actually exercise choice but a full 800 of those were in Denver.  So in the other 46 schools, we have a total of 72 kids benefiting from choice; that is less than two students per identified school.  So in the spirit of Milton Friedman, if choice is supposed to generate pressures that might lead to change at the school level, there is clearly not enough choice taking place to generate that kind of pressure.  That is not to say that Colorado does not have a lot of choice.  This is where all the people in Colorado are sitting because of the command.  About 18 percent of Colorado students attend the choice not controlled by their local district, either by attending charters, crossing district boundaries to a state-mandated inter-district law, home-school [indiscernible] and private schools.  If we add the inter-district choice programs which districts run quite a bit, we actually have about 200 times as many kids exercising choice outside of NCLB, as within it.  If we look at supplemental services, we will see that there is some participation and 67 schools where it is in play.  About 28,000 kids are eligible, and about 3,000 participate.  So, this only seems like a reasonable amount of participation because we all talk about choice first.  If we just had a conference on SES participation, I think the conclusion would be participation is totally inadequate and insignificant.  But because we talk about the two, it seems like it is a fantastic success.  The participation rates are also quite uneven; about a third of the schools really do not have much at all, a third or somewhere between 5 and 10 percent.  And a third actually have significant participation, largely a function of district size within Colorado.  Just like choice, there are other programs in Colorado that do a lot more supplemental services.  The 21st Century Learning Centers in the small district I looked at in Colorado of Cortez served 15 times as many kids as the supplemental service provider in that same district.  Denver itself has 178 sites with over 10 different supplemental programs competing with the NCLB version.  So even though they are embracing it pretty seriously and trying to do it, it pales in comparison to the other things that they already want to do.  If we look at restructuring, we see an equally bleak story.  So far, Colorado, out of those 1,700 schools, has restructured three and they have another 12 that might face it coming up pretty soon.  We keep sort of expecting the NCLB curves to catch more and more schools; it has not really happened yet so there is not much to say about that restructuring.  The one thing I would say is that almost all of it is in Denver;  Denver public schools has seven other different restructuring initiatives underway in one form or another that also compete with NCLB.  And if you do any of those, you will comply with the letter of the law, if not the spirit.  And for the& there is also& the thing that NCLB did for restructuring in Colorado was provide political cover to water down a stronger state law.  Colorado preceded NCLB by mandating the state chartering of long- term chronically failing schools.  That was the only option.  They did it with one school, which was also one of the NCLB schools, and the process worked out so badly that the state's reaction was to model the new law on NCLB and allow any option anybody wants to qualify as restructuring.  So, there is not much evidence, but the evidence there is that there are more options at the local level than NCLB mandates.  And then if push comes to shove, no one has the political stomach to actually do the profound stuff.  So this is a case where it is not adding strength; NCLB is adding cover to what the districts actually want to do.  In terms of a performance link, there are too few students exercising choice for it to matter.  Even if all 72 of those kids improve their performance dramatically, or the districts were somehow motivated by their absence, it just would not really matter.  The SES evaluation is just getting started.  The state department is concerned about the quality of those providers, and I think based on implementation of other reforms it is easy to say a quarter to half of them might be effective and that would be optimistic; so that 10 percent is going to turn into 2 or 5 percent that actually are improving in performance, so I would keep that in mind.  We will know more about it the next few years.  The question will be about SES, not is SES effective, but which SES providers and which contexts is effective?  For restructuring, basically, the question is, how do you provide cover for people to do hard things, not how do you provide political cover for people to do easier things.  I did do a cost analysis to look at it.  If the idea is that you are going to drive action from the federal level by creating incentives by movement and new options, what are the costs to the district?  So I multiplied the number of choosers and the number of kids getting SES by the costs of, let's say, $400 per transportation and the cost of the SES contracts at the maximum rate.  And then I added in the amount of new money that schools got through school improvement grants; the state gives out about $150,000 to your grants.  So if you count that as $75,000 a year for the schools that get them, and then count as costs what they might have spent on the transportation and the supplemental services, outside Denver, being identified for school improvement is actually a net gain for the schools of about $15,000.  In Denver, they lose a whole $60,000, which is not really a significant portion of their budgets.  So if the power of the federal government is to drive change that creates financial incentives for districts to respond, that is clear evidence that there is just not enough change happening to really pressure the districts to do anything different.  So we should not be surprised that the only ones that respond and do stuff are the ones who wanted to do so in the first place.  So to summarize my findings, let's say choice is only happening in places where it was already happening but at a much smaller scale because of NCLB.  In Denver, for example, about two percent of the eligible kids practice NCLB choice whereas the district has a 32-percent choice rate going on already.  In SES, it happens a little more but not much and is also small compared to other alternatives besides NCLB.  And in restructuring, there is yet to be anything that is happening& that has happened yet and that the district's initiated activities are probably more significant.  I would actually say that in this case, NCLB is probably a very effective tool for supplanting local activity.  So for the fiscally conservative, you say this is a good mechanism to get federal money to replace state money for activities that the districts already wanted to do.  And to conclude, you are really not getting enough action to affect the kids who are actually participating in the intervention and you are not getting enough incentives produced by that level of activity to get the districts to do change.  So what should we do?  Clearly, additional interventions would have more impact, and I say additional because I think you could& it is only five years into this.  I think Mike Kirst s work on Title One would tell us that 10 or 12 years is probably a better time to have formative or final judgments about this so we can make improvements.  But in the meantime, we should look at additional incentives if we actually hold these goals dearly.  And we should judge those on the ability to impact student performance rather than sort of ideological suppositions.  We should focus on the ability to affect large numbers of kids.   And the federal mandate - the power - should actually be used in ways where the power produces change.  So how could the feds do stuff that makes locals, whether it is the states, districts, schools, or parents, do something they would not have done?  Otherwise, we are wasting that power to the extent that it exists.  So I have three recommendations, and they are aligned with how the states are dealing with a P-16 approach to education.  First off, I would expand the choice to include post-secondary open enrollment and dual enrollment for the secondary schools.  I would also expand it to mandate inter-district choice like Colorado has, and that is where the federal authority would be good because that is not something districts do on their own; it is something states sort of enable districts to do and then the districts will cannibalize each other and the choice will take place.  And then the last one, which I'm most excited about, I believe we should actually offer high quality, full day preschool for every three and four-year old in the neighborhood of an identified school and full day kindergarten funded by the feds for these low performing schools.  There are a couple of reasons I bring these all up.  For the preschool, keep in mind we look at participation rates in new universal subsidized preschool measures as we were considering a ballot initiative in Denver.  Their participation rates are 60 to 65 percent on average where it happens.  So we are competing with zero for choice, 10 percent for supplemental services with 65 percent.  Now, we can also have another debate - and this place would be glad to house it - about the efficacy of choice and whether it improves performance for the choosers or those that are left behind.  But I think we would have a much less controversial debate about early childhood education in a quality setting and full-day K.  You re standing, so I am done.  Look forward to your questions. Chester E. Finn:  Thank you very much, Alex.  John Winn deserves special thanks for being here today.  Not only is he the sitting Commissioner of Education in a huge state but he is also in the middle of a transition from governor to governor and he is, according to the Florida press, the only member of Jeb Bush's cabinet who was not been asked to resign by the incoming governor.  We think he has got his hands full.  We are very glad he is here today.  On the other hand, he might welcome the respite. John Winn:  Thank you.  We still have air conditioning on in Florida.  I never thought I had sweaty palms until today, so you will have to excuse me.  But our editorial writers in the major newspapers in Florida have noticed that Governor-elect Crist has not asked for my resignation, so they have decided to do it for him.  We like to try to keep a good healthy dialog going in the State of Florida.  First, let me say how much I enjoyed reading the three papers and the material was& the observations presented very well, very well-articulated and went from not only utilizing studies and research that have been done but also utilizing the personal kind of interview and thoughts of people because this is far more than the kind of policy stuff that is out there in the stratosphere somewhere.  What I would like to do, although I do have a certain point of view as the sitting Commissioner of Education, that I will have a tendency now and then, if not throughout the entire discussion, to defend states as the ones not doing the right thing.  However, I'll preface that by saying I have no sympathy with any state that is not zealously working for school improvement.  And in working with& regardless of the history and traditions of the state, regardless of whether it is& a state is local control, strongly local control or not& most are in some way or another.  But I do believe that it is absolutely the duty of every State Department of Education to the citizens of their state to ensure that struggling students, wherever they may be, struggling schools wherever they may be, improve; and that is not necessarily synonymous with faithfully implementing No Child Left Behind.  There is no doubt that No Child Left Behind, as Checker says, is about outcomes; it is about changing the hearts and minds of educators and citizens about new ways of approaching education to try to do things differently in order to reach chronically underachieving schools and the students that they serve, whether they be chronically underachieving themselves or whether they be performing very satisfactorily or above satisfactory.  That being said, I have few observations on why states are not uniformly implementing No Child Left Behind, and I do think some of it has to do with the law itself; some of it has to do with the implementation, and some of it has to do with pure human behavior that in many cases has not, I think, been contemplated in the implementation of No Child Left Behind.  So to put it simply, No Child Left Behind, I believe, is having limited capacity in terms of changing hearts and minds for a couple of reasons.  One, it grew out of Title One.  Title One should have been totally repealed and it should have had no vestiges of Title One present in it; and I'll tell you why.  It is permeated with Title One mentality, from heavily-laden process to going through all the rules and making sure that they work and so on and so forth with very, very little passion outside the US Department of Education, and, frankly, independent groups who have been more passionate about the implementation and the possibilities of No Child Left Behind and the education establishment itself, forgetting that it is the education establishment that is going to in the end make the change.  So, while we celebrate a focus on results, the No Child Left Behind, does not in fact have a focus on results in a way that is going to change the behavior across the state unless reauthorization brings that; and I'll tell you why.  Number one, it gave away the most power that it had.  It gave away the most potential power, which is setting standards.  When No Child Left Behind Law allowed states to set standards under the banner of flexibility, it gave away the one thing in lieu of 100 process steps that are, in some cases, meaningless.  It turned the implementation over.  It did not set a clear boundary that implementation of No Child Left Behind should not be done by Title One bureaucrats because in most places it got picked up by Title One bureaucrats, and if you have ever been in a Title One meeting, God bless them, you are lucky to get to Day 2 or 3 before you start talking about student outcomes.  Well, I do not have to go there.  Title One IDEA did not have any better implementation than No Child Left Behind because we did not pay any more attention to the human dynamics of what we were trying to accomplish in this country in education; and I'll explain that.  We simply asked state commissioners of education, state legislatures, and state boards of education to remove too many articles of clothing at once, too many articles of clothing at once.  If you want something happen uniformly, and happen well, you keep it simple, okay?  You say to states, Hey, you do not reduce your speed limit, you do not get any money.   Okay?  A case of not hard to understand, right?  So what we misplaced our local control and state control concept by throwing away the number one thing that we had to do, which is set high standards and enforce those high standards.  And it also provided a great disincentive for states that had already gone out there and taken the risk to do exactly what No Child Left Behind was supposed to do by saying, You are going have one accountability system.   Well, that was, as we all know, the first victim of No Child Left Behind because it became pretty clear that no state is going to give up its entire accountability system for a federal mandate.  I mean, nobody wants to lose money but it just cannot be done politically.  After you invest 10 or 15 years for some states that have been out there, you cannot just completely give it up, and so nobody did.  So we got this convoluted kind of deal where in most states, except those that did not have an accountability system, which No Child Left Behind was revolutionary, they needed all those pieces.  And so, in that respect, it was good, but the states that have been out there had tried some of those pieces and they did not work.  Had tried school takeover in New Jersey; it did not work.  I'm sorry, but I do not think the State Department of Education did any better at running the school than & perhaps less well.  But anyway, so what we did was we said, Okay, you are going to measure results, and for a lot of states that had no measurement, fine.  But it should have contemplated where the states were in making that progress and even though US DOE tried to do it& who said, Fire the lawyers?   The lawyers were & the answer always came back.  Lawyers would not let us do it.  We do not have the authority to do it.  So states like Florida did, had already had a status model, year-to-year model, and it failed.  That is why we went to an individual student growth model and a proficiency model, both, so that your high school that has 80 percent of entering ninth graders on second and third grade level gets credit for bringing those students up two or three grade levels, far better than a middle class suburban school that does not have those challenges.  Yet they get punished out of No Child Left Behind, except for the Safe Harbor provision.  But in a state large like Florida, like California, New Jersey, you have got diversity, you do have a great deal of exposure.  So we asked them to all of a sudden implement the system with great vigor; and then we handed them a system that defied common sense in some ways.  Number one, pass/fail.  Who gives pass/fail?  What state in this country has sustained an accountability system that is pass/fail?  Nobody.  So now, I'm Commissioner of Education; we have a special relationship in Florida to the federal government, as you know, and it has not always worked in our favor.  However, our system, we have been in our foxholes and getting a push back for having tough standards, for having a tough intervention and requirements; and now we have got to defend the system that we& common sense, everybody knows is not going to work.  So we started using, at least showing our schools, when you line up our school grading system with the percent of criteria in No Child Left Behind that schools met, it lined up perfectly.  All of our schools that met 90 percent or more of the No Child Left Behind were all A schools; it was like a perfect match.  And we tried to at least to give some public exposure to that to parents and all that, but, of course, the confusion is incredible and it is difficult to ask somebody to go out and do war when they are confused, and when they see some things out there that basically do not make sense like 100 percent proficiency for each subgroup.  And when they see an aspirational goal turned into an operational goal with little or no flexibility, not measuring student growth when they have already experienced that as a much more robust way of school accountability, we ask them to go back.  We ask the entire state to go backwards on that.  And so, you have states that -- and I do believe that in many times, the way to change hearts and minds is to have them go through certain steps and kind of develop a habit that they had not developed before.  So, I understand that as well.  But if we want aggressive state implementation, we have got to understand the human dynamics of what we are asking people to do to make those kinds of things happen.  And, Checker, if you will just give me one more minute, I would like to comment on SES and choice.  Choice, I believe, has ultimate power only when it is private school choice.  There is a world of difference in our experience in the State of Florida as to how local educators respond to public and private school choice.  They yawn on public school choice and maybe that is because we have had it for years.  But they are scared to death of their kids leaving, and that has been documented by Jay Greene, and it is an important operational situation.  What we are asking state commissioners in a place where choice & to get behind a politically-charged battle for a program that they are still trying to see how it meets.  In SES, Florida will have 100 percent of its eligible-funded students on SES this year.  Why did nobody have very little SES is because, number one, public school systems do not know how to market [indiscernible] prices.  Number two, we are asking them to market their competitor.  Ask them to market for their competitor, okay?  So no wonder we have foot-dragging.  How did we solve it?  We passed a state law and we have 100-percent compliance.  And so, states can in areas that we want to bring a broader array of services to students, there are things that states can do and we are holding our SES providers accountable for individual student learning gains.  And if they do not make them, they are off the list; same thing with schools.  So there are things that states can do on their own to do that, but it takes some time; it takes a lot of help and it takes, I think, above all, incredible common sense and understanding the difference between legitimate concerns and whiners.  Thank you. Chester E. Finn:  Thank you very much.  Isn t Florida henceforth going to have a special relationship to the Lord? John Winn:  Yes, we do. Chester E. Finn:  How does the new governor spell his name? John Winn:  C-R-I-S-T. Chester E. Finn:  Okay. John Winn:  Crist. Chester E. Finn:  Thank you all very much four terrific presentations.  I think everybody was really totally riveted and I know I was.  Let me first ask whether NCLB should basically give up on choice.  It seems to me everybody is documenting how little NCLB choice is occurring and how much other kinds of choice are occurring.  And is choice simply a different policy arena that NCLB should have nothing to do with in the future because it obviously is not working within the NCLB constraints?  Or alternatively, should NCLB insist on more radical choice whether it is the private school choice that John was suggesting or the inter-district choice that a couple of you have alluded to or other ways of opening up the choice spigot beyond what the current law requires?  Comments& Alex? Alex Medler:  I ll just say that regardless if it does not get a deep penetration for those students that it does reach, it is a good thing.  In the civil rights version of the law it is a noble thing to pursue even if does not get too many kids. Chester E. Finn:  You mean, the right to leave? Alex Medler:  The right to leave a school that is not educating them.  And then if we can figure out more options and then within some context there is not much choice, so maybe that is an addition.  We should not be na�ve to think that it is going to compete with all the other pressures within state politics or local politics that produce choice.  But if we ought to consider it, one of NCLB's hallmarks is that it sort of articulated that right. Chester E. Finn:  Other views from California? Julian Betts:  I do not think we should give up on it quite yet.  We are close, but not quite.  We have to recognize that a lot of districts already have a lot of choice, so any contention here that parents do not care about choice is clearly wrong.  In San Diego Unified, about 30 percent of students are on non-local schools already.  That is a penetration about 200 times greater than NCLB choice statewide; so parents do care about this.  I think it has got to do with the implementation.  Parents do not understand what program improvement means; I think parents are quite concerned that if their local school suddenly makes adequate yearly progress a couple of years in a row, they are going to lose the right of choice.  So allowing some sort of grandfather clause I think would generate increased demand as well.  Inter-district choice could work but probably more on the East coast than in the Midwest because distance is such a very big problem. Chester E. Finn:  John... John Winn:  I definitely think we should not give up on choice but I think we are misguided if we try to evaluate choice on participation.  If [indiscernible] when nobody is participating, it is not working.  It has got nothing to do with that.  What is not working in many cases is how it is being offered and delivered and so on because one of the things in Florida& we have about 40,000 students on private vouchers; students with disabilities, poor students.  We have a great deal of choice but I think the basics for choice is what No Child Left Behind should do is just to say any state, any school district, any school receiving federal funds gives choice regardless of the record of the school's improvement because it should be a right, and not make it up your credit nightmare.  Although SES is the largest choice voucher program in this country, it is a voucher program, albeit not the full-day school.   But it is an incredible choice program for parents and maybe it will help get them engaged into and hunger for additional choices. Alex Medler:  If I could offer one other clarification, I think it is probably a bit of mislabeling too to talk about NCLB choice.  What it is, is NCLB busing and transportation for choice.  So there is lots of other choice.  The unique thing within Colorado about NCLB-driven choice is that it is about transportation paid for by somebody else.  So we do not talk about as busing.  That is not a word we can use in political circles, but that is what it is, and it is for a different group of kids. Chester E. Finn:  Okay.  At the risk of heroic oversimplification, I am going to divide the NCLB potential sources of action, sources of change into three big headings, one having to do with sunlight and data; one having to do with market-triggered changes like choice in SES and the third having to do with what I think I'm going to call top-down interventions in schools and districts that are not doing a good job, that are not succeeding.  I heard everybody almost stipulate that the data and sunlight is a good thing to have out there and that it might trigger some positive things locally and at the state-level.  We have just been talking about the choice pieces, the market pieces.  Now come to the top-down outside interventions in failing schools and districts.  I do not think I heard any of you supply any cause for optimism that the top-down interventions by district at the school level or by state at the district level are working at all well or even could.  Tell me I'm wrong.  John?  John Winn:  You are wrong. Chester E. Finn.:  Prove it. John Winn:  This is purely from Florida's experience.  And I read with interest because we do the same thing.  We have interventions.  We have intervention teams, although I have not seen any evidence that any of them have ever worked.  We have intervention plans, which I have seen a ton of evidence do not work.  Let me tell you, the only intervention, in my opinion, that is going to work with struggling schools is if you are State Board of Education or Commissioner of Education, whoever has the authority has very, very broad authority to sanction school districts with funding.  If they do not actually implement their plans, and here is where you go to capacity because we do not have the capacity, probably no State Department of Education has the capacity.  We have in our F schools have required them to submit an improvement plan for years now. And guess what?  Some of them & actually we have got the greatest improvement and some of them get off the F list and kind of hang around the D list and drop back down.  So what we found was we decided this year, enough is enough.  Thank God we only had seven multiple F schools, so we said, Okay, for seven, we are going to be on your ground to see if you implement your plan.   And the State Board required no less than 26 interventions for those schools, everything from proving your principal as high-achieving, removing underperforming teachers because we have learning games on every student by teacher, and sanctioning them, not making them ineligible for state grants and lose lottery funding if they did not comply.  And we have people on the ground every two weeks in those schools to do it.  Now, is that fun? Chester E. Finn:  Every two weeks? John Winn:  No, but I'll tell you what.  It is eye- opening. Chester E. Finn:  Every two weeks in the school? John Winn:  Yes. Chester E. Finn:  From the state? John Winn:  I report to the State Board of Education the monthly progress of each of those seven schools discreetly in open State Board meeting and we vote on whether or not that district is in compliance with those 26 interventions.  And, we are seeing change... Chester E. Finn:  And say again, what the club is?  Besides embarrassment, if it is not doing those 26 things, then what happens exactly? John Winn:  If they do not one of the 26& I'm talking about pass/fail, right?  If they do not do one of the 26 they can be out of compliance; they can lose lottery dollars.  The two sanctions that we have, in fact, imposed are become ineligible for state grants with the exception of students with disabilities, for students with disabilities or struggling students, struggling schools.  We have withheld the superintendent's pay for a month... Chester E. Finn:  A thought... John Winn:  Well, it only took a month to get back in compliance. Chester E. Finn:  So let me just... John Winn:  So my advice to states is if you ever expect in the schools that 30 years ago were the lowest in your district - and Florida has big districts - if you ever really expect, then you get a superintendent who is going to hold everybody accountable in his district, who is going to make a change and that will see it.  Or you are going to have to move in on the ground and work with them. Chester E. Finn:  Let me... John Winn:  But do not try to take them over because it is their responsibility. Chester E. Finn:  Let me just see if I have got this right.  You have got seven schools and 26 criteria and every two weeks you go look, and the district loses funding if its school is not in compliance with the 26 criteria and you have enough fine-grained control over the district's funding from the state that you can, for example, withhold the superintendent's pay? John Winn:  We have. Chester E. Finn:  Okay.  And how much of the district's money is potentially at stake through this system that you are describing? John Winn:  Here is the problem that States have, and State Commissioners.  Nobody wants to withhold money from children.  You know, education, that is not a good thing, not a good thing.  So we have not gone to the level of withholding lottery.  We have a state lottery that goes into enhanced education so we have shied away from that.  But we have found that our school districts get a heck of a lot of money through grants - blow-through, discretionary, whatever.  And we had one district lose about $12 million the first month and the superintendent had the salary withheld the first month and they were in compliance the next month. Chester E. Finn:  And you have how many schools in Florida, public schools in Florida? John Winn:  Thirty-five hundred. Chester E. Finn:  Thirty-five hundred, and currently, you are working on seven? John Winn:  Currently, we are working on seven persistently F schools. Chester E. Finn:  Alex, your grounds for optimism...? Alex Medler:  Well, I guess I go back to the first distinction between the three groups.  And I would say that there is a sort of a false distinction, and that one of the best things you could do to encourage top-down changes would be to change the data.  And so, I think what the Feds can do is create a hostile political environment for resistance to change by shining that flashlight.  We have an effort in Colorado now to get a unique teacher identifier in the data system, like some other states, so we can track student performance to their teacher back to an ed. school.  And the legislators say, Well, that it is okay if it is a flashlight, but we do not want it to be a club or a hammer.   And I just said Well, if you have ever met a cop at night, you  know that five D-cells and a flashlight turns into a club pretty quickly.   So, if you have the data on the performance and you affect the way the media talks about graduation rates and achievement gaps and everything else, then when there are superintendents, they will support not top-down but middle-down; let us say district-initiated reform because of political pressure.  And that it is a top-down reform driven by federal power but it is initiated through the flashlight part. Chester E. Finn:  Did you have something on this point?  [Cross-talking]  No, I have got two different questions.  So, if you want to speak on this top-down intervention, do it. Julian Betts:  I, too, feel that maybe you are a little bit too gloomy, Checker, in the sense that a lot states would not have any accountability system right now if it were not for No Child Left Behind, and I think that is a salutary effect.  I also think that it is really important that No Child Left Behind disaggregates by race and other categories.  And that is very important because it has really made the public understand how widely acute the gaps are.  I m more on your side when it comes to the compliance, enforcing compliance, either from the state capital or from DC; it just does not seem to be working very well.  I think top-down can work reasonably well when the top is defined as the district administration.  We did a quasi-experimental analysis of reforms in San Diego of the reading reforms, and apart from the high school level, reforms actually worked quite well.  There were political issues associated with that which were problematic but the reforms actually worked.  When you were giving your list of three things - data, market triggers and top-down interventions - where I thought you were going with this was Is it not inconsistent to have top-down intervention and market mechanisms at the same time?   And I think that is something that we are going to have to look at in the future. Chester E. Finn:  Or maybe not... Julian Betts:  One of the examples that I have seen in San Diego is that one of the reasons San Diego got out of the SES business was complaints from the external providers.  There was going to be no Title One money left over for them, which was simply incorrect because even now, the district is unable to spend fully its five percent allotment for SES.  What the district was providing was the most successful element of the reading reforms, which is the extended day reading program.  So it is basically out of that business partly because of complaints that it was rigging the system which actually were not, in my opinion, true. Chester E. Finn:  I have two last questions.  It ought to be simple and factual.  Julian, I think your numbers said, if I remember, that 12 people in the California Department of Education worked on this program statewide? Julian Betts:  Including the director, yes. Chester E. Finn:  Including the director, okay.  Do any of the other three of you have the ability to name a number of state employees who are working on NCLB implementation and enforcement in the states that you are dealing with?  John and...? John Winn:  We have, oddly enough, 12 as well. Chester E. Finn:  That is another big state. John Winn:  But our implementation& and, Checker, I do not want to give the wrong impression that 3,500 schools and we are having to go to this level of intervention for seven is I think, a testimony to the fact that we have had incredible student achievement improvement in Florida and in schools responding to our state system and a good strong history of continuing improvement.  So I did not want to give the wrong impression that you have to be that overbearing for every school, but we have 12 but when No Child Left Behind was enacted, I put together a 20-person task force, including our data people, including our assessment people, including our public school people, our Title One people, our exceptional student education people because to us, it was just like our state accountability and assistance program.  Everybody had a piece to play in it.  So we have maintained the integrity of that group through the entire process so that we& because I did not want to turn it back up, just turn it over to Title One and have it be largely process-driven. Chester E. Finn:  So it is 20 people on the task force and 12 full-time employees?  Did I hear it right? John Winn:  Correct. Chester E. Finn:  Okay.  Alex, Colorado, I know, has a famously small Department of Education. Alex Medler:  The famous small Department of Education with no state funding but it survives on federal money, so it actually has about 8-10 people. Chester E. Finn:  Eight to ten... Alex Medler:  The system that only serves, like, 780,000 people. [Cross-talking]  And I have to say, with truth in advertising, I had to talk to Rick in the middle of this one.  My wife shifted from one job in the State Department of Education to become the Title II director for the state.  So I'm glad we are not talking about highly-qualified teachers today. Chester E. Finn:  Rick, do you have any way of knowing what New Jersey does? Rick Hess:  No, I do not. Chester E. Finn:  Okay, enough said.  It is the best answer I have ever had from Rick, actually, on anything.  John, last question.  Would you just unpack for one more minute your statement that they should have repealed Title One?  What should they have then done? John Winn:  Well, I say that because if you are about outcomes and about hearts and minds and focused on improving student improvement, you do not bury yourself in process, and a clean break from Title One would have sent a message to everybody that this is not about process.  Because No Child Left Behind& we basically do not need any of the other parts of Title One because No Child Left Behind goes further.  Now, I do not know how the federal government initiates nationwide change of this nature in a high-quality manner.  And I would prefer seeing a system where you do have parameters on what your standards are.  You do have parameters on your student performance and that states have consequences and things to do; make sure that they are playing their role to make sure that those students in all schools are getting appropriate service, as opposed to providing steady work for a lot of people that are handling process. Chester E. Finn:  But you would have kept& or you would not have kept the $10 or 12 billion flowing that is currently in the Title One program? John Winn:  Yes. Chester E. Finn:  Yes, flow it? John Winn:  Yes. Chester E. Finn:  Okay.  Having used up almost all the airtime for this panel, are there any questions from the gathering?  We have time for one or two and I cannot see, for some reason, to the extreme left.  You.  Yes, stand up, seize the mike, and introduce yourself. Gina Piek:  Hello, my name is Gina Piek.  I'm an education legislative assistant for Senator Mel Martinez from Florida.  Recently some colleagues and I have been discussing an idea that has been floating around about keeping some of the good things with No Child Left Behind but allowing a charter state option where states with existing systems that have proven results, like Florida or California or Colorado, could have the option of running their state system the way it was intended.  How do you see that playing out within your states and what potential portions of No Child Left Behind would you prescribe for the remaining states that are under the federal system because they do not have a system of their own? Chester E. Finn:  You are, in effect, describing a complete state opt-out or waiver for a state which can somehow prove that it has got its own system that is working? Gina Piek:  Yes. Chester E. Finn:  And then the question is, what would you make the other states do? Gina Piek.  Yes.  I guess I'm saying, what in your states would you then prescribe to be left in the federal legislation for those states that are not able to produce their own successful program? Chester E. Finn:  Okay. Rick Hess:  This is the thing that Krista Keyfer [phonetic], for instance, has floated at Heritage.  It is probably the most thoroughly written-up version of this.  I mean, part of the problem is it reflects a little bit an incomplete understanding of how NCLB is actually written and how it works.  In some sense, that authority actually exists.  The Department, in approving each of the 50-state plans, has actually had a fair bit of discretion in terms of what constitutes compliance with the various statutory provisions of No Child Left Behind.  And, in fact, there is some degree of discretion right now on determining what these things have to look like in a state.  So what you are really talking about is, in reauthorization, broadening the language to give the Secretary more discretion to make judgments of what constitutes compliance with the spirit of the law and then&   so essentially, so it is hard to kind of identify specific pieces.  Everybody is going to be up here today.  I think we will probably give you a different set of answers on what are the non-negotiables.  But really, in terms of thinking about the idea of the charter state option, the real question is what are going to be the criteria that states have to meet?   What do you wind up with as kind of non-negotiables?   And that is going to be a legislative compromise that [indiscernible].  And then what are the procedures for figuring out what constitutes good faith compliance? Chester E. Finn:  That was an excellent answer and I do not think I ll let anyone else speak to this one.  Do you have other& are there other questions in the group?  Here's one.  Somebody can make it up or you can start speaking.  You are? Female:  [Inaudible] Chester E. Finn:  Okay.  The media are here. Female Voice:  Could the panel talk about data quality capacity in terms of staffers who are& we had some things in California how few people are actually working on it at all -- state and district level people who are providing this data and what training states have done as far as that goes? Alex Medler:  I would say in terms of the criticism that the states are not returning the data soon enough for choice to matter, I would say the choice is irrelevant.  The states are not returning the data soon enough for instruction to be effective.  It is an infinitely more important question.  So the states have to actually figure out how to return data that it useful and do so in a timely basis and they do not have a willing& certainly, in Colorado they do not have a will to do so.  Now, I do not know about capacity because they will not let you in.  So we have had to go back to the legislature year after year just to sort of have some leverage to get them to share.  The problem, I fear, that right now the education establishment in Colorado is responding by saying Stop asking for all these data.  You are not giving it back.  It is a waste of our time.    And the answer is they might win and will stop gathering a bunch of that data.  The better answer is Let us deal with this purpose stuff.  Let us get the data back, turned around to researchers like myself as well as the districts and the educators in a much more open and transparent fashion. Chester E. Finn:  Good.  One more question before we break?  Cindy Brown? Cindy Brown:   Well, it is kind of a simplistic question.  A lot of folks are talking about doing something to beef up state capacity through some dollars, and I take it you all agree that adding reporting requirements so that all the data that is missing that you laid out would be exchanged.  They would have to agree to that in order to get these additional dollars.  I mean, would that help at all? Julian Betts:  Yes.  That was really going to be my answer to the last question.  I had not had a chance to answer it so I'm going to, anyway.  In California, the real problem is we have student-level data but they are not linked across time.   Any state that comes up with a reasonable plan that is student level and is value-added, though it would make a lot of sense, the real problem in California is that the accountability system, although it is a growth model, is school-level.  And that is really problematic, especially in poor neighborhoods where there is a lot of movement of kids from one year to the next.  So you do not know whether scores are going up or down because of teaching or because of the children themselves moving. Alex Medler:  Whereas we have now data that goes from pre-school through college with unique student identifiers that tracks it over time. And there are eight people in that State Department of Education who might be able to get access to the file. Chester E. Finn:  Anybody else?  Rick? Rick Hess:  One point.  It is just& and this is a general data question about No Child Left Behind, the enthusiasm for the test collection and desegregation and such is driven on the [indiscernible] agenda;  it has driven NCES;  it has driven data collection of the states.  One of the things that this research suggests is that it is not that we have been pushing on that front and other important fronts.  It is that part of the enthusiasm for collection of test data has actually not been accompanied by similar efforts to understand how reforms are playing out so that we can actually systematically examine and understand the impact of reforms.  So one of the takeaways, I think, out of this state level, and there is various strategies to encourage that and support it,  but a lot of this data has existed.  I mean, the fact that Mike Casserly and the council could collect this stuff from the districts but that nobody has, either federally or at those state levels, suggests in some sense that there is this bottleneck in terms of getting information that already exists, and collating it and making it usable.  And thinking about how to tackle that is an important issue in [indiscernible]. Chester E. Finn:  John, you want the last word? John Winn:  Just one statement.  We also have a longitudinal database and we follow student learning games from year to year and propose that in our growth model.  I think one of the best things that the reauthorization could do with regard to data is to fully authorize a robust growth model, but only, rather than watering it down and letting people call things a growth model, require it to track individual student growth over time as an alternative to the status model for proficiency.  And I think you will see every state in the country get behind their data and, to my way of thinking, it is absolutely necessary to get your data back in town.  We get ours and report to schools, school grades and AYP the first week in June and... Chester E. Finn:  When you give the tests? John Winn:  Yes. Chester E. Finn:  When do you give the tests? John Winn:  We give the test in middle-March. Chester E. Finn:  Okay.  The next forty minutes represent the only actual break in the day.  We are going to be back at 12:55 to start the next panel.  In the meantime, there is a sort of a picnic over there that you can enjoy and please join me in thanking a terrific panel. Panel III:  The NCLB Remedies in the Districts Frederick M. Hess:  All right, we will go ahead and get started please.  If folks could& if my unruly panelists would& folks, please go ahead and take your seats, we will go ahead and get started.  We are now shifting from the state level to the district level.  Obviously, this is& when we are talking about No Child Left Behind, given the nature of the remedy provisions, it is hard to draw clear and distinct lines.  Certainly, in the previous session, there were times when we talked about things at the school level or the district level and certainly in this session, it is going to be hard to talk about districts, distinct of talking about activity at the state level or at the school level.  But, the thrust of this one is trying to think about the same issues that we have been wrestling with this morning, but with a focus that targets the district level.  We have four papers on the session, our discussant, unfortunately, is brought down by the flu, Stephen Jones of Norfolk is apparently not able to join us, which Checker and I agreed is actually a good thing, since we have more or less collared the audience thus far to give us more opportunity to have you actually raise questions and issues that you think need to be on the table, so it works okay.  The four authors will be speaking alphabetically.  The first up will be Steve Clements.  Steve is a policy research associate with the University of Kentucky's Institute for Educational Research.  Steve's projects have included Grant Program Coordinator Duties for Title II, Teacher Quality Enhancement Grant at Kentucky's Education Professional Standards Board, and an education data system improvement project for the Kentucky Department of Education.  Speaking second will be Jay Greene.  Jay is a senior fellow at the Manhattan Institute s Education Research Office, and Head of the Department of Education Reform at the University of Arkansas.  Jay is a prolific author and it is hard to find somebody who is not familiar with at least some of his work.  It was referenced earlier today by John Winn.  Jay is the author of the book, Education Myths:  What Special-Interest Groups Want You to Believe About Our Schools and Why it Isn't So.  Jay has conducted evaluations of school choice and accountability programs in Florida, Charlotte, Milwaukee, Cleveland, and San Antonio.  Speaking third will be Jane Hannaway, who directs the education policy center at the Urban Institute.  Jane is the most recent Editor of Educational Evaluation and Policy Analysis, one of the leading journals in the field of educational policy.  Her work focuses on structure reforms and education, particularly reforms promoting competition and choice.  Jane has led for several years the research project down in Florida and her work, in this case with Miami-Dade, comes out of that larger body of work that she has conducted.  Finally, David Plank is Director of the Center for Education Policy in Michigan State University.  He has co-authored the piece he will be representing with Chris Dunbar, who is also a Professor in Michigan State.  David is the author of Choosing Choice:  School Choice in International Perspective.  He has worked as a Consultant in Education Policy Development for the World Bank, the US Agency for International Development, and the United Nations Development Program.  So we always appreciate David when you turn your eye back to domestic issues from abroad.  Each of the authors will speak for about 12 minutes.  Because we do not have a formal discussant, I will probably pose a couple of questions to give them a chance to elaborate a bit, and then we will open up the rest of the session for questions from the audience.  With that, Steve, would you please get us started? Stephen K. Clements:  I really appreciate Rick and Checker for providing me the opportunity to be part of this project.  I'm here primarily to talk about the experiences of three very rural, very remote districts in Kentucky, their experiences with NCLB.  This larger discussion is incredibly valuable to me, but doing these case studies in the districts, I have been roaming among the trees of NCLB and this is a nice opportunity to step back and see what the larger forest looks like.  When I moved to Kentucky 13 years ago, the state was just in the early stages of a very important systemic school reform initiative.  So I have been able to watch a state level reform initiative in its implementation up close, and I have seen a lot of its effects on local schools and even on the education of my own kids.  I have also seen, though, in the last few years how NCLB demands have ratcheted up pressure on these local schools, so part of the story in my mind in Kentucky, as many of the other states, is a sort of competing our overlapping accountability systems.  We have had a system in place for over a dozen of years now that has generated a considerable amount of legitimacy, and it is a system that people in the schools are very familiar with, and No Child Left Behind is a kind of imposition on top of that.  In the limited time I have here today, I want to do two things very quickly.  The first is just to provide you some basic information about the three school districts that I visited, and then I want to summarize what I found in these districts in terms of remedies for failure.  You'll have to consult the chapter for some of the more interesting details, I think, for example, about how Kentucky's accountability system works, and its interaction with NCLB.  I think all of these things are important to understand in terms of this larger picture.  In a way, I'm doing something that is very different from all of the other presentations we have heard so far.  I do not have a lot of high level statistics about NCLB, about supplemental services and choice, but I'm going to look at exactly what this & I'm going to illustrate how some of these dynamics that folks have discussed here so far play out in individual districts.  It is good that you did not ask me to provide a lot of high-level data on the state because it does not exist.  In terms of these three districts, I chose them in consultation with the State Department and they represent different geographic parts of the state and very different socio-economic settings, but the common denominator is that all of them have faltered somewhat under NCLB.  The first district is in Martin County.  This is on the eastern border of Kentucky adjacent to West Virginia, way over in the corner, and in a lot of ways, Martin County is a stereotypical Appalachian district with very high levels of poverty, very low levels of formal education.  One of the statistics I did not cite up here is that the median family income in this county is about $22,000 a year.  So, keep these median family income figures in mind as we go through some of these counties.  Martin has a total of 2,200 students and six schools.  Four of them - two elementaries, a middle, and a high school - are located in Aeneas, which is the county seat, and two of them - another elementary and a middle school - are over across the mountain in a community called Warfield.  The economy here is tied very closely to the coal industry, so as that industry waxes and wanes, so wax and wane the economy of the county.  About 400 miles away, on the opposite end of the state, at the tip down near Missouri and the Mississippi River lies Fulton County which is a completely different, it is a rolling agriculture area.  Only about 7,000 people live in this county.  It has a 25 percent poverty rate.  It has a slightly higher formal education level both in the K12 and post-secondary level, and it has about a quarter of the kids are African-American.  That is unusual for the counties across the state.  Most of the minorities in Kentucky are concentrated in about eight or 10 mostly metropolitan communities, and the farther out from those centers you go, the fewer you find.  And now, coming back roughly to the middle but southern part of the state is Monroe County.  It is less economically troubled and less academically troubled than either of the other counties, but it is by no means prosperous.  It is closer to the size of Martin County with about 10,000 residents.  It has slightly lower poverty rate but it also has a lower formal education rate.  Monroe County has five schools - three elementaries, a middle, and a high school �- and four of them are located in Tompkinsville, the county center, and one is located about 10 miles away in a little community called Gamaliel.  I'm going to speak for most of the rest of my limited time here off of this one slide, and I want to just make some general points about what I found when I investigated life in schools on the ground in these districts.  One of the interesting things you find about these districts is that while they are not meeting the goal set for them by the state's accountability system, they are not in assistance under the state's accountability system either.  They are in a kind of gray zone that we call Progressing.  So from the standpoint of the school community, they are moving in the right direction.  They are just not on a trajectory that is going to get them to where the state says they need to be by 2014.  Now, under NCLB, however, they have all slipped into tier three at a district level and several of them have, especially Fulton and Martin, serious problems with the middle and high schools regularly failing to make AYP.  The schools and the districts as a whole, I think, have responded with a fair amount of intentionality to No Child Left Behind, as well as to some of the sanctions or remedies that the state system has provided.  Someone mentioned in the previous panel about Kentucky's scholastic audit system.  That, sort of, is the heart of what our state's accountability system has used to try to get school communities focused on the kinds of changes they need to make.  That audit system brings in an external team either to a school or to a district, and there is a fairly well-developed protocol that goes into a lot of detail over the course of the week, and looking at the school's academic environment, its organizational capacity, and its culture.  And then that external audit team provides an array of recommendations for improvement.  All of these schools and districts have made substantial improvements not just in the previous three or four years before NCLB came along, but in direct response to NCLB.  The kinds of improvements that these schools have focused on have been fairly traditional.  They focused on putting new leadership in place.  They focused on curriculum alignment, increasing use of formative assessment, and placing reading and math specialists courtesy of Title I and Title II into key positions in the school.  I think, arguably, these are the kinds of changes even if they are top-down directed from external people that NCLB envisioned.  So in some senses, I do not think this is a reason for pessimism about the potential for No Child Left Behind districts to do certain things with their schools.  Now, having said that, choice is largely absent from the tool kit.  Fulton and Martin Counties were the two with schools in high enough tier to have & mixing up choice and supplemental services.  Maybe I should have spoken with supplemental service as first tier as for the Colorado suggestion.  All three of them have had to promote choice within their district.  Fulton County has obviously, with three schools, it has no inter-district choice options.  The Fulton County people, when their district started falling into - when their schools started falling into various tiers contacted the other districts in the geographic region, and all of the districts refused to allow Fulton County students to transfer.  In Martin County, on the other hand, most of the other districts did not have schools that were healthy enough to take their students, and the one district that did decided that it would allow their students to come on a case-by-case basis.  So, in other words, we will take the ones that we think will help us and we will leave the rest there behind the county, exactly right, the rest behind.  Now, one of the things I did not do was parse the letters that were sent home informing people in the districts about the choices.  But Martin County did this, sent home the obligatory letters, multiple iterations of these letters, and none of the middle school families elected to try to get their kid in an adjacent district.  Keep in mind that the closest school that this district would allow Martin students to transfer is a 45-minute drive, and there is an extent to which, with the family connections and so forth, there is a lot of suspicion about whether students from one community would be treated well in schools in a different community.  That is obviously something that deserves a lot of attention.  So Fulton County had essentially no choice.  Martin County has extremely limited choices.  This year in 2006-2007, the high school in Martin County reached tier one, and so far there are seven of its students that are interested in transferring to the district 40 minutes away.  I do not think that has actually taken place yet, but they are working on that.  In Monroe County, in the center of the state, ironically, the one tier one school, the one Title I tier one school, is an elementary school but Monroe County already allows inter-district transfer among elementary students, so the choice has already been exercised in one of those & those families were offered it; there were no takers, no additional takers.  Supplemental services is more present in these districts and it is gradually coming on, but it is going to take a long time to develop a significant participation, I think.  Statewide about 9 percent of eligible students, I'm told by the state department, are availing themselves of supplemental services.  I saw even smaller percentages of that in these districts.  In Martin County, seven students participated the first year, nine participated in the second, and so far this year, 11 students - this is out of about 175 or 200 eligible students - are actually taking part in the supplemental services.  In Fulton County, there were four students that participated the first year, 20 the second year, and about 24 are enrolled right now.  That is actually a larger percentage than Martin County's participation.  I did not offer any epiphanies while I was out visiting these school districts, so I do not have any grand policy prescriptions.  But the logistics of providing both choice and supplemental services in these places make it daunting and I doubt very seriously that we will see those items being a significant part of school improvement process, and that focuses us back again on what the state can do in terms of directed school improvement efforts. Frederick M. Hess:  Thanks, Steve.  I think we will get back.  I would love to hear a little more detail, too, about some of these developments, so we will get a chance to talk about that.  One public service announcement I'll make is we are all aware that it is warm in here and I see people fanning themselves.  They assure me that we have already turned down the AC as low as it will go.  I guess we have got too many. Female Voice:  [Indiscernible]. Frederick M. Hess:  Well, one of these directions. Male Voice:  There is just so much hot air. Frederick M. Hess:  So, yes, so apparently we are doing everything we can and so we are aware of it, and we will do what we can.  And with that, Jay, the hot air introduction. Jay P. Greene:  Thank you.  Let me see if I can& there we go.  I'm only as productive as I am because I have a lot of people who helped me.  On this paper, I had excellent assistance from Jonathan Butcher, Laura Israel Jensen, and Catherine Shock [phonetic] at the University of Arkansas.  The topic of this paper is to examine the choice options under NCLB and the willingness of schools to inform parents of their choice options.  Everyone knows that there has been very little participation in the choice option under NCLB.  We have heard of this all morning.  Somewhere around 1 or 2 percent participation rates among those were eligible.  The question is why do we see such low participation rates?  And a number of explanations are offered in the literature.  One is late notification, that schools do not tell parents in time for them to choose, and we saw some evidence about that this morning that notification does not come until quite close to the start of the school year or sometimes later, and so parents could not possibly exercise choice if they do not hear about it in time.  Although that, in some ways, begs the question why does it take so long.  Some of it may be capacity, some of it may be will, some of the capacity maybe related too will.  That is not a fully satisfactory explanation.  Another explanation that is offered is insufficient capacity, that there is nowhere for students who are eligible for choice to go.  There are numerous accounts from big cities - Los Angeles, Chicago - that there are basically no schools that are not AYP, that are making AYP, to receive all of the students who are eligible for choice.  But this explanation is also not fully satisfactory since the US Department of Education has made clear that it would accommodate districts that wish to make SES available in lieu of choice, if choice options were not available.  Yet apparently, only something like eight districts, as of a year ago, were pursuing this option.  It is also possible for districts to make choices available in neighboring districts.  But again, as of a year ago, according to the GAO, there were no districts or almost no districts taking advantage of this.  And so again, insufficient capacity is not fully satisfactory as an explanation.  Some people suggested low demand.  The parents do not want choices.  They are quite happy where they are.  There may be some truth to this, but this also does not seem like a fully satisfactory explanation given that one might expect that the parents would want options if they were going to schools that were performing quite horribly, and we do see when other kinds of choice programs are introduced, that parents do pursue them at higher rates than are observed under NCLB.  So maybe there is another explanation more fundamental than any of these explanations, which is maybe parents do not exercise choice under NCLB because no one told them that they can or they are not properly informed that they can.  Now, there has been prior research on this, William Hall [phonetic], for example, did a very interesting survey in Massachusetts where he surveyed parents who would have been choice-eligible, and the vast majority have no idea that they are eligible for choice even though they are, but it is possible that schools have made every effort to inform every reasonable effort to inform those parents, but the parents do not care.  So maybe it is just not the responsibility of the school at that point, just the responsibility of parents.  That is possible.  So we wanted to test this more directly than this prior research.  Our idea was to send e-mails to choice-eligible schools, asking for information to see what kind of response we would get.  We identified choice-eligible schools.  We excluded Louisiana because hurricane Katrina may have made everything more difficult for those schools and we did not want to pressure them in any additional way.  We also excluded any school that had made AYP in 2004 and 2005 because those schools potentially could get out of being choice eligible if they made AYP the next year.  We wanted to have set of schools that we knew, for certain, were choice eligible.  We then searched for e-mail addresses for these schools so that we could send them an e-mail.  And we found 2,887 e-mails.  We then sent them an e-mail of ambiguous origin.  We had to work with our human subjects committee at the University of Arkansas.  It is not like the cool old days Milgram and Zimbardo where you could do really great stuff.  Now, you have to not torture people.  So we came up with a plan that they approved where we would send an e-mail, where we did not falsely represent ourselves as parents, but it was ambiguous so that schools might believe that we were parents, so that we could see whether they would respond to us.  So we created e-mail addresses with some commercial e-mail providers that were havalina@.  This is a thinly-veiled reference to our beloved razorback, the mascot of the University of Arkansas, which at least amused us, [indiscernible], I think, so, now we sent these e-mails out.  Now, technically, this is not a measure of legal compliance because it is, as I understand, not the legal responsibility of the individual schools.  I believe that legal responsibility belongs to districts.  But schools are the local representatives of districts, and one might reasonably expect that they would have procedures in place to inform parents that the districts would help them develop.  So we sent out these e-mails and this was the text of the e-mail, and again this ambiguous slightly odd language has to do with the necessary ambiguity of our e-mail.  So, it just had, I have heard something about there being options for students under No Child Left Behind.  Can you tell me about what those options might be?  Thanks, and from <A target=_blank href="mailto:havalina@something" target=_blank>havalina@something</A>.  This was sent to every choice-eligible school for which we could find an e-mail address.  And here is what we got back.  It is a little small.  I'll help you through it.  Actually, some of the e-mail addresses we have were invalid e-mail addresses.  We ended up with 2,448 valid e-mail addresses.  The biggest finding was that the vast majority of schools do not respond at all.  The 85.2 percent of the schools did not respond.  The 14.8 percent did respond in some way.  Half, roughly half, of those schools that did respond in some way asked, Who are you?  Why are you asking? but failed to provide anything beyond that.  We had actually a protocol that when they asked us who are we and why are we asking, that we then send a response e-mail where we said we are researchers trying to find out about schools and the information they provide about NCLB, and this 7.4 percent of the schools then just did not respond to that.  So even when we told them that we were researchers studying this, they did not respond.  One-point-two percent of all schools that we contacted gave us an unresponsive response.  I'll give you illustrations here in a second.  Point-five percent gave us inaccurate responses saying that we are not eligible for choice, and we had narrowed the list down to all schools that we knew were eligible.  Two-point-two percent said, We would be happy to give you information.  Call this number, and we simply assumed that they would give us the information if we call the number.  We did not actually call those numbers.  And 3.6 percent actually responded by e-mail, providing us with some factually accurate information about what options were available.  So basically, 94 percent of schools that we contacted did not answer our question.  So let me give you some examples.  These were examples of responses where they only asked for information and did not provide any.  We got some that were string of question marks, that was it.  We got, I'm confused.  Do I know you?   We got, I'm happy to receive& we got some that were more thorough and courteous like, I would be happy to respond if you will confirm who you are, if you are a parent, or a family of certain school.  I would appreciate knowing your specific needs or concerns, et cetera.  We also got this non-responsive responses.  This is where people told us what they thought about No Child Left Behind.  The first one was, The question concerning the options for the NCLB is a rather vast area.  I would suggest you go to Google on the website and type in the No Child Left Behind.  I'm sure you will be able to find out just what you are looking for.  Be prepared there is a lot of material.  Good luck.   This was an e-mail sent to a choice-eligible school.  Some people said things, what they felt about the program, and so NCLB's reality has been to provide little, if anything, for students that has mostly added to our bureaucracy and provided us with nothing more paperwork that I assumed then paperwork to do.  Another one that is not very grammatical says, Since this study did not go any further than - this is in response to our follow e-mail.  Since the study did not go any further than the original e-mail that did not identified the sender, I consider it unreliable in its data with a very big margin of error.   And perhaps, they are right.  We did, however, get accurate responses.  And here is an example of an accurate response, a helpful and accurate response.  Students were enrolled to schools identified as in-school improvement status, have the option to apply and to attend another school.  The schools that our students would be able to apply to attend under the NCLB legislation are x and y.   That was a great answer.  This consisted of 3.6 percent of the schools that we contacted.  Now, is this a reasonable test?  Well, for some people, e-mails are the preferred form of communication.  Some people work during the day and they cannot easily call during working hours, so they might want to send an e-mail to the school and hope to get an answer.  This is the kind of answer they might expect to get.  Sure, our request was ambiguous, that is true, and maybe our request seemed a bit silly.  But businesses regularly receive odd and silly questions and they respond to them courteously.  We, in fact, did a little test where we sent out to eight businesses, including three education companies, similar e-mails, almost identical e-mails with similarly odd language.  And for example, we sent to Starbucks, I have heard something about your coffee, can you tell me about getting it.  Thanks.   Not only did they respond with this lengthy e-mail with links to website where you could click on a store locator and describe the different kinds of coffee that they will sell, but there was also a quality control link at the bottom that you could click on to provide feedback to the company about whether the answer was responsive to the question.   Or similarly, I referenced the Lazlo Letters, in case any of you are old enough and obscure nothing or entertainment to know this.  Don Novello, who played Father Guido Sarducci, had a series of books called the Lazlo Letters where he used to write letters to businesses and politicians with bizarre things that he would say and the great thing about the book where they printed his letters and the responses is that people always responded courteously no matter how inane he was.  For example, he e-mailed the Mr. Bubble Company and said, Your box says keep dry.   How am I supposed to use it if I keep it dry?   And they sent him back this very nice letter about how you keep it dry when you are not using it, and they gave him coupons for his mother to buy, and he got very irate that he does not live with his mother and he wrote back to them, and they are nice back to him.  The point is why the businesses answer inane questions nicely, while reasonable questions to choice-eligible schools go unanswered at a high rate.  It is different incentives.  Starbucks wants to sell me coffee.  The school is actually uninterested in my leaving and so it is not particularly responsive, which is essentially what we were talking about all this morning.  I, however, have to just & maybe I'll save it for Q&A because I have run over my time, but I disagree with Mike and Checker's gloominess that it is impossible to get schools to comply.  I think actually, whenever you hear people say that you cannot do it, if anyone say that it is not worth doing it or it is not feasible, that it is unlikely to happen but we could certainly do it.  If we can get people to pay income tax by voluntarily reporting their income, we can get schools to comply.  We just may not want to, and that is a different debate. Frederick M. Hess:  Thank you, Jay.  That is a nice closing note.  Jane? Jane Hannaway:  Okay, the title of this paper is, (Trouble Even in Choice Paradise:  NCLB Options in Miami-Dade County Public Schools).  My co-author on this is Sarah Cahodis [phonetic].  Raise your hand, Sarah.  Sarah is one of our terrific research assistants that do everything they can to make the senior researchers look good.  Let me tell you why you should be interested in what is happening in Miami-Dade.  There are a number of reasons.  First, it is the fourth-largest school district in the country, so there are a lot of kids there, and a lot of poor kids there.  Secondly, the district and the state have striving choice programs.  Choice is part of the system in the state.  It is something that is strongly advocated by the previous governor.  There are a number of pieces of legislation that afford variety of choice.  Thirdly, there is an accountability system that in many ways is similar to NCLB, and in other ways different.  So, we have a population that is accustomed to choice, accustomed to accountability system, an accountability system that grades schools and then also offers choice.  So, one's priors would be that this would be very fertile ground for NCLB choice options, that if you are going to introduce choice into a district, you are introducing it into a district with a population that is accustomed to making school choices.  It is also a district in a state where there is a very active State Department of Education, a very data-intensive State Department of Education, and I have been doing work with Florida over a number of years and I might say, and I would say this even if John Winn were not here, that a very impressive competent State Department of Education.  So, for all those reasons, I think Miami is an interesting case to look forward.  It is the best-case scenario.  If it is going to work anywhere, it should work in Miami.  This is what we see.  This is enrollment in the different choice programs operating in Miami.  As you can see, over 30,000 students are making choices via the Magna [sounds like] program.  Over 18,000 are exercising choice through charter schools.  There is a control choice program; the control choice program is part of the state legislation where school assignment & there are school choices built into school assignment.  Almost 10,000 students I choose is with the grant from the Department of Education, where they are creating greater supply of alternative-type schools in Miami and encouraging choice, about 10,000 student there; McKay Scholarships, 4,000 this year, that is the special education voucher in Florida; and the OSP scholarship, that is the Opportunity Scholarship Program, that is associated with the state accountability system.  Now, to put this in even greater perspective, about 22 percent of students in Miami-Dade exercise school choice.  This is greater than the total enrollment in Boston, in Denver, in DC, in San Francisco -  I have a whole list here - in Cleveland, in Austin.  So this is big, this is big type choice.  It is larger than all these big city enrollment total.  And so I think it is a real good question.  What about NCLB?  This is choice heaven.  I consider five explanations, and some of these are the same ones that Jay considered, and some of these are the same ones that have come up in other panels, and that will probably be discussed over the course of the day.  The first one is that there are not enough eligible schools available because of high rates of schools not making AYP.  Miami-Dade is a big district.  There are 180 schools in Miami-Dade that are not in school improvement; 115 of these schools had space.  In order to transfer into a school, the school could not be enrolled greater than 115 percent capacity.  There are 115 schools in Miami-Dade that were available for its students to exercise choice.  So it does not hold as an explanation.  Furthermore, because with No Child Left Behind, you can choose any school basically in the district.  The number of schools available in the NCLB choice exceeded the number of schools available in the other choice options.  There is a smaller number of magnet schools, a smaller of charter schools, that control choice happens within school zones.  There may be 30 or 40 schools and some of these that are available, so a large number of schools were available through NCLB.  So if I were more talented with PowerPoint, I would put a big red X over number one.  It just does not carry the day.  Secondly, and this is something that has come up a lot, that the district does not sufficiently inform parents and I pushed this when I was down there.  Letters go out April 15, letting parents know if it is likely that their school will be in school improvement and they will have options.  So they get informed prior to -- the ink on the test is still wet, but they know which schools are at risk and they are letting parents know April 15th when it is likely that they will have options.  They do all communication in multiple languages - Creole, Spanish, and English.  They have school fairs, fairs for providers in the SES program where all the providers come and basically hawk their tutoring programs.  They do television and radio spots across town, and do these repeatedly and they do it in multiple languages.  They even enlisted Miami Heat players to do phone calls to people's houses.  So you pick up the phone and there will be Miami Heat players saying, Free tutoring is available for your kid, and they continued doing that to the level that they were afraid they were sort of badgering people so they had to back off a little bit.  In addition, they even went so far as to enlist the help of local large employers.  There is a big hospitality industry in Miami and the district enlisted the major employers and would go, and the employers would help disseminate the information - five minutes, okay - and they would make presentations to the employers.  So, they are out there.  Big red X, district does not sufficiently inform parents.  In addition, and I should add, each of the schools that is eligible, where parents are eligible to make a choice, each of those schools has in the school, the list of schools that those parents can choose.  So they do not have to go very far to find the list, as well as a very active website.  Logistical barriers; Miami-Dade is big.  It is about 2,000 square miles, but choice mainly happens within zones.  There is a fairly sophisticated transportation system because they have all these other choice programs going on.  So logistical barriers, red X.  Decision timing; this gets a little bit trickier.  Let me just talk about this one for a minute.  Only 1 percent, if you look at 2006, 2007, only 1 percent of the eligible students requested school transfer.  It is important to recognize that 100 percent of students who requested school transfer in Miami got it.  Even if there are constraints somewhere, it is far from that, far from the constraint, but we are still at this 1 percent level.  These are the decision windows for the different choice programs and as you can see, the NCLB choice window and the OSP, that is the state accountability choice window, is generally later than the other options.  Magnet schools happen in the fall, I Choose happens in the fall, McKay Scholarship happens pretty much anytime.  Administrative transfers generally happen over the summer.  I think this is important, and I think it is a point that Mike Casserly brought up this morning.  This has nothing to do with the school district.  This is a structural feature in No Child Left Behind, as well as in OSP, where you become eligible for choice on the basis of school performance and even though the state is Herculean in terms of turning its data around and getting it back out there June 15th, even though the district is really upfront letting people know you are likely to make choice, you still have to wait until the data come in and as fast as you can do it, it is still later than most of the other options.  Other explanations; AYP is not a good signal of school quality, and I think this is something that Jerry [indiscernible] brought up this morning a bit, and I think it is something we really should discuss more.  If you look at the 2005-2006 box on the far right, what you can see there is 61 percent of the schools on the state accountability system got a grade of A to C, failed AYP.  If you just look at the A-B schools, over half, a majority of schools that got an A or B in the state accountability system failed AYP, and we all know the number of hurdles you have to clear to make AYP.  So there is a conflicting system here of signaling about what school quality means.  As we were writing this paper, we heard late and I heard from someone in Miami that they actually had some data at the individual student level which could show what school kids left and what school they transferred into by choice programs, and so we really pushed& Rick was ready to kill me, I said, You have to wait until I get the data, after we get the data.   I flew down there trying to get the data, I could not carry it back.  We finally got it the Saturday before this is due and Sarah, it arrived at her house on Saturday.  It turned out to be 1,000 pages of writing in a PDF file.  I was like, Holy smokes.   Anyway, Sarah deserves a lot of credit because she cracked the PDF file and got in and linked it up and changed it into Excel.  No really, really.  It yielded some very interesting information, which I think buttresses the point that AYP indicator is not a good indicator of school quality.  How could you expect that therefore to be a guiding choice or even a stimulating choice?  So if you look at the yellow-marked numbers, that is the percent of students that were choosing in the NCLB program and then in the Magna program, and then administrative transfers, which includes I Choose, that were choosing schools that were failing AYP.  I should stop for a minute and explain what AYP is.  The state requested, the US Department of Education and said, Hey, we got all these schools that are getting A, B but they are failing AYP.  Cannot we have this provisional AYP based on their scores on the state test that would then not be& have to go through with their choice options?   And the department basically said no.  They said, You do it on an experimental basis, you cannot do it statewide and state [indiscernible].   I think that is accurate.  Anyway, if you look at here in the NCLB program, 65 percent of the parents who are exercising choice are choosing the schools that is failing AYP.  The pattern is similar with Magna schools and with administrative transfers.  A majority of those parents exercising choice are choosing schools not making AYP.  If you look over at the same schools now with this state grade, you see that a majority of the parents are choosing A schools.  So you have two grading systems there - you have the state accountability system, you have the AYP accountability system.  It looks to me from these data that parents are relying on the state grades to guide their choice, or the state grades comfort with their more detailed judgments about what good schools are.  That has got in their choice.  But they are not using the AYP's data to guide their choice.  So in terms of implications, I think there are basically two implications.  One is the structural problem that Mike talked about, I mean, choice just cannot work but not in that sort of time length.  Secondly, if you really want to induce parents to make choices about schools and to make good choices about schools, we have to put a lot more thought into what the information signal is that we give parents about the performance of those schools because at least in Miami-Dade, they are not buying this AYP grading. Frederick M. Hess:  Thank you, Jane.  David? David Plank:  I would like to recognize my co-author, Chris Dunbar, who is also my colleague in Michigan State University, so an equal partner in this project.  And for those of you who are sleepy after lunch, I m going to give you an overview of the paper, so if you want to take a nap, you can sort of pay attention to the first slide and then you can snooze off for a few minutes.  In Michigan, lots of schools have failed to make adequate yearly progress.  We have over 500 schools, about 15 to 16 percent of all schools in the state.  The implementation of the mandated cascaded sanctions under NCLB has been extremely weak.  I ll give detail on this, but there is very little evidence that the sanctions have really been taken seriously in Michigan.  There is a strong observed preference at both state and district levels for gentler approaches to schools that are not performing well.  Despite that, the NCLB goals are being met in Michigan, particularly for those schools that are in the most trouble.  More than half of the schools in Phases Three, Four, and Five of NCLB sanctions have made AYP for two consecutive years and reset their sanctions clock to zero.  So you may wake up for the end when I ask what is the lesson about this, what can we learn from the Michigan story.  In contrast to a few years ago, No Child Left Behind is no longer really a political issue in Michigan.  The governor is a strong supporter of No Child Left Behind, the superintendent and the state board of education are focused on implementation rather than resistance.  There are some technical issues that arise in Michigan as in other states about confidence intervals, about English language learners, but the political obstacles are no longer that salient.  One thing that is worth observing is that like Florida, like many other states, Michigan has its own state accountability system.  Michigan s accountability system, to put it in sort of vernacular terms, is lousy.  Parents take AYP, take NCLB very seriously, but do not pay any attention at all to the state system.  The implementation of NCLB sanctions has been very weak.  This is partly attributable to reasons that we have heard already today.  One is that the Michigan Department of Education has very limited capacity for oversight and enforcement of NCLB sanctions.  The administrator who is responsible for supplemental educational services refers to the system as chaos.  She is a part-timer in the Department.  After five years of implementation, they are trying to hire a consultant to come in and clean up the mess.  But there is really almost no one at the state level who is paying attention to supplemental education services.  Because the Department does not have capacity, there is heavy reliance on good faith in the districts.  Just to give you an example of how this works, a couple of years ago the Detroit Public Schools reported that all of their schools in Phases Three, Four, and Five had replaced their principals.  They all adopted exactly the same strategy of replacing principals.  Two years later, which is to say this year, the Department of Education went down to visit some of the schools and the principals were still in place.  It is nice to rely on districts good faith but not always wise to rely on districts good faith.  And there is a general reluctance in Michigan at both state and local levels to confront unions.  I ll talk in a few minutes about some schools that in Michigan are in Phase Seven.  The Department has begun talking about what they call hybrid charters, where their schools will be transferred to the authority of charter authorizers in Michigan.  But if you talk to the charter authorizers, they say there is no way that the Department will be willing to confront the unions, be willing to sort of override the prevailing governance arrangements to actually transfer those schools to us.  We heard a lot about school choices today, so I will not say much about it, except to say that we do not have an exact number for the number of students who have availed themselves of school choice under NCLB in Michigan, but zero is within the margin of error.  Supplemental educational services--once again the administrator in charge of this program in the Michigan Department of Education declares that the system is characterized by a complete lack of clarity or responsibility.  There is virtually no oversight from the state.  Nevertheless, there are students who are participating in the system; most of those are in Detroit, a substantial in number in Flint.  It is worth noting that there are no students participating in the system in many districts where there are eligible schools including Benton Harbor and Lansing, which means that those districts are spending none of their ESEA money on either choice or supplemental educational services.  And there is wide variation in the character of services across districts.  Michigan is further along in the implementation of No Child Left Behind than some states, and we have quite a large number of schools that have entered the restructuring phases of No Child Left Behind.  The strategies proposed by NCLB have generally not been adopted in Michigan.  There is strong resistance to closing schools or replacing staff.  We have seen no district-sponsored charter schools, no contracts with management companies, no state takeovers, not that the state could take over but they have chosen not to do it.  We have seen some governance changes in some smaller districts which were documented by the Center for Education Policy.  What we have seen is a preference for alternative strategies, for gentler strategies, and we just have some quotes from our subject districts at Flint, Grand Rapids, and a Michigan charter school which tell a fairly consistent story.  In Flint, a senior administrator said, We are not here to beat you up or take your job.  We are here to support you.   Grand Rapids, We are here to help but there will be consequences, and in the Bayard Ruston Academy, they characterized the two keys to their success with NCLB as the commitment of their current staff and the hands-off attitude of their school board, allowing the principal basically to take charge of the process.  So what do the alternative strategies look like?  Those have cohered around basically five relatively familiar ideas.  The first is alignment of instruction with clearly defined learning objectives.  We have examples in each of those three districts: frequent assessments of student performance, ranging up to almost daily assessments in schools that are not performing well in Grand Rapids, and regular practice in the charter school; the use of those assessment data to group and regroup students and target instruction; the employment of turnaround specialists or coaches.  This is an idea that has received a lot of enthusiastic support in Michigan, particularly at the district level but some at the state level as well.  The Flint public schools have provided coaches for all of their principals in schools that are not making AYP.  Grand Rapids began with individual coaches for principals but has moved now to a model of lead principals where principals in successful schools are responsible for working with their colleagues in less successful schools.  Central Michigan University, which is the authorizer for the Bayard Ruston Academy has sent a turnaround team of coaches, not a single coach, but a team of people with different specializations, and the charter school has also relied on peer coaching by teachers who are exceptionally effective in that school.  We see enhanced public accountability, which is to say that the principals are required to participate in public meetings with often the superintendent or with sort of regional supervisor, with a representative from the Intermediate School District, often with parents, to talk about both the data as the data come back, and about their plans for improvement.  And the head of the school improvement office in the Michigan Department of Education said, Well, there is nothing new here.   She characterized these, in fact, as vapid and obvious.  These are things that school should have been doing all along, they are just doing the obvious things.  But the fact is that in Michigan, at least, the obvious, if you like, has produced results.  Bayard Ruston Academy had failed to make AYP for five consecutive years.  It has now gotten itself off the list after making AYP for two consecutive years.  There once were 17 elementary schools in Flint that were on the list in need of improvement; there is only one left.  Likewise, only one elementary school in Grand Rapids is still on the list, and the rest of the schools in these districts have made AYP for two consecutive years and reset their sanctions clocks to zero.  It is worth noting that the success that has been achieved in elementary schools does not extend itself in general to middle and high schools, which represent a different set of challenges.  This is just a picture of the schools in Michigan that are in need of improvement and it is much too complicated to go through in detail.  If you look at the teal-colored column on the left, the bottom three cells, those are schools that were in Phases Three, Four, and Five of NCLB sanctions that are now off the list entirely.  That is more than half the schools in each of those categories.  The other number that is worth looking at is the four in the bottom right-hand corner of the table, which are the schools that have sort of passed beyond the horizon of No Child Left Behind.  They are schools that have been through the restructuring phase, have failed to make AYP, and the Department is going to have to decide what to do about those schools.  But so far, they have not taken dramatic any steps.  So what is the lesson?  I think in Michigan, No Child Left Behind has genuinely focused administrators attention on the neediest schools, both at the state and the district level.  Their persistent failure is now publicly visible and no longer publicly acceptable.  It is clear that the threat of sanctions, the threat of punitive sanctions has provided leverage for both the Department and for district administrators.  But what we have observed in Michigan is that many roads lead to AYP and not just the road laid out under NCLB that the sanctions cascade.  There are no particular reasons that the sanctions cascade would have led to better results than the alternatives that were implemented in Michigan.  With all of that said, it is important to recognize that getting over the first hurdle for many of these schools is relatively easy because it is a very low bar, and the second hurdle is going to be significantly higher.  The other thing that is worth noting, as I have said, is that this appears to be working reasonably well in elementary schools, but not at all in middle and high schools.  Thanks. Frederick M. Hess:  Thank you, David.  When I first read these papers, one of the things that struck me was the way in which they complement each other and cover so many of these issues - the restructuring, piece, the choice piece - but I think it came out even more clearly in these presentations. A couple of things I want to ask about to clarify and expand a bit, and then we are going to open it up to the audience.  First thing, Jane, could you think a bit out loud about the whole public choice question?  Something that struck me this morning in the conversation was, I do not think any of us are entirely clear on what public choice is for.  One of the conversations obviously they were having at the Department through 2002 and 2003 was, How do you increase capacity at schools that are making AYP? There are a lot of ways to do it.  You can double-shift schooldays.  You can add trailers.  Of course, most of these things also entail hiring new teachers, and in fact perhaps adding a second faculty essentially, as if there is some magical assumption that having it in this building with different teachers would nonetheless yield superior results to having it inside.  So as we think about public choice, how should we think about this?  Are parents making a mistake if they disregard both the Florida system and AYP?  How do we understand the factors that are in play in why people are choosing schools? Jane Hannaway:  Thank you.  Most parents, I think, would want their kid in a school where they are going to learn and be happy.  I think they are willing to make choices and assume some cost to make that happen, like transportation costs.  But I think it is very, very difficult to sort out what the exact match is between a kid and a school.  Frederick M. Hess:  But the NCLB premise is that in the schools not making AYP, these children will be better served moving to a school which is making AYP.  And that certainly seems at least an arguable premise. Jane Hannaway:  I think they are more likely to learn in Florida if they move to a school that is an A or B school, not necessarily if they move to a school that is making or not making AYP. Frederick M. Hess:  For the folks who might not be familiar with the Florida system, why is that?  Why is the A or B a different kind of indicator? Jane Hannaway:  Well, for a couple reasons.  Number one, it looks at level of performance of the school on average.  It looks also at gain, so it gives schools credit for gain.  It is actually very complicated and it has changed over time so the system, which is a whole study in and of itself, these accountability systems are something that can be tinkered with in order to get the incentives right.  Different weights are given, for example, to math and reading.  Different weights are given to writing.  So it is a sort of best professional judgment about what the characteristics are of the school that is being productive.  The main difference, I think, between AYP and the Florida accountability is this taking into account gain. Frederick M. Hess:  And so in the case of Florida, where we see 70,000 kids exercising choice, how much sense does it make for us to even be concerned about whether NCLB choice is being utilized or how effectively it is utilized? Jane Hannaway:  I do not think it makes much difference at all.  I do not think & clearly, AYP and NCLB are not driving choice behavior in Miami-Dade.  I mean there are relatively few exercising choice through NCLB and those that are do not appear to be using AYP as an indicator of what school to go to or what school not to go to. Frederick M. Hess:  Steve, so one of the things that this strikes is, you see little or no evidence of choice behavior obviously, but at the same time you seem reasonably optimistic with the impact NCLB is having in these districts in terms of motivating reform.  What are the kinds of changes, when you talk to the folks in these schools, what is it that has changed the consequence of NCLB and does any of that have to do with the NCLB remedies or sanctions? Stephen K. Clements:  Well, let me say a couple of things.  First of all, it is kind of hard to tell because we have a lot of changes that were going on in response to the state s accountability system and the remedies that were already being moved in place in some of these same schools.  So disentangling the effects of NCLB in Kentucky s system is very difficult to do.  One of the things, though, that NCLB has done interestingly is that it has revealed some of the deficiencies in the states accountability system, and I will echo Jane s sentiment here that in Kentucky, the results on our system are a lot more meaningful to most parents and school communities than AYP results.  But one of the things about our system is that it is porous.  It gives a lot of points to kids who do very well on the state assessment, the rough equivalent of the advanced kids.  They get extra points for doing that and what that does is it kind of masks the kids at the lower end of the spectrum.  So in that sense, NCLB has prompted a lot of our schools that are failing to make AYP to focus on those kids more specifically than they did before. We have spent a dozen years trying to figure out how the state should provide technical assistance to failing schools and districts, and it is a long, hard slog.  I think we have come up with some pretty good ideas and some pretty good solutions, but I think a lot of other states are just in the early stages of grappling with what it means to provide the kind of assistance that schools and districts need. Frederick M. Hess:  Jay, if sympathetic superintendents or a state chief is sitting here and they read your paper and they buy it, are there concrete things that they could do or that need to be done in reauthorization that would create incentives or pressures on districts and schools to take this more seriously? Jay P. Greene:  Well, I have three ideas for reauthorization.  And again, this is kind of a response to Mike and Checker s cannot-be-fixed sort of view.  The three things are, if you have private choice, you will have more utilization of choice.  Why?  Because then you have someone who has an interest in marketing that choice.  Right now, the choice could be marketed by someone who does not want you to exercise it, so they are not particularly interested in doing it very well.  Second, we could have some sort of legal liability of the schools for failure to perform to individual families.  Again, like Special Ed, and actually I have to disagree with Mike and Checker that there is not a difference between Special Ed and NCLB.  NCLB does not demand results for SES or for choice options.  It simply demands that those services be provided, just like Special Ed.  For Special Ed, the services are provided but for SES and choice, they are not being provided.  We could clearly do it if we wanted to and part of why it happens under Special Ed is we have legal liability if you do not.  Third, we could have auditing like the IRS where there is someone who is in charge of making sure that people comply, and like with the IRS if you violate the law, you go to jail.  Now the question is do we want to do these things?  So, whenever you hear people say It cannot be done, you are almost hearing a disguised version of I would rather not, or I do not think it is likely it will.   I think these are things that we could consider and we ought to decide.  Are we willing to do these kinds of things or not? David Plank:  Just to respond to that comment, we have no evidence that these things work.  Why would we institute those kinds of draconian strategies for making people do things that may not improve the performance of their kids? Jay P. Greene:  I m simply responding to the claim that it cannot be implemented.  I agree that it is a legitimate question.  The real discussion would be do we want this or not, but that is a whole separate conference. Frederick M. Hess:  David, one of the things that your paper seems to suggest is that when we talk about the remedy provisions of NCLB, we are basically looking in the wrong direction.  So what you are suggesting is a lot of good things have happened in Michigan but it is not through the supp-services, or public choice, or what we think of as mean-spirited restructuring, but it is simply because it has gotten people to focus on the nuts and bolts? David Plank:  That is the conclusion that we are driven to by what we observe in Michigan, but I think that the key to the deal, nevertheless, is the threat of sanctions.  These are indeed things that schools ought to have been doing all along, that good schools have been doing all along, that bad schools failed to do.  You need at least the possibility of more severe consequences for schools that fail to do them.  We have not seen those consequences implemented, but they remain available to the Department and to other decision-makers.  I think that is the key part of the deal.  They have to be available. Frederick M. Hess:  Now if I read your graphic right, there are about four schools that have gone over the horizon?  Is that it? David Plank:  Right. Frederick M. Hess:  What happens to these schools once there? David Plank:  No one has decided yet.  The Department is talking about two things.  The first of those is simply closing the schools.  What they mean by that is basically dismissing all of the staff, replacing the principal, and bringing in an entirely new team.  That has not happened in Michigan at all.  We have schools now in year seven, so that is still a challenge before the Department.  The second is this idea of hybrid charters, that they would re-open the schools as charter schools.  That runs into some legal obstacles because we have a cap on the number of charter schools.  There is a lot of legislation that would have to be implemented or adopted to put that into effect.  But those schools are really the test of whether the Department is serious about even the threat of sanctions.  As schools move into year 6, 7, 8 and so on, if nothing happens, then the other schools will probably stop paying attention. Frederick M. Hess:  Okay, with that, let us open it up.  Do we have the first question? Gerald Bracey:  Hi, Gerald Bracey, independent researcher.  A couple of years ago in Fairfax County, a couple of elementary schools did not make AYP and the choice option kicked in.  One of the things that was found was that the kids who were transferring were among the highest performers in the schools that they were leaving.  I have not seen anything else about that, but it is not the way it is supposed to work in theory.  I just wondered if any of the panel has encountered that same phenomenon. Jane Hannaway:  No, unfortunately.  I was hoping the file I received would have individual identifiers like that, but it was just the ID and the school leaving and going to.  But, consistent with that observation, when you look at the total choice going on within Miami-Dade, 27 percent of the choice is going from A school to A school.  So there are some consumers out there, parent-consumers that are sort of fine-tuning.  Twenty-seven percent, that is the biggest chunk.  So I would expect that there would be a bias in any sort of choice program favoring parents who are more knowledgeable.  Pete Wright:  My name is Pete Wright.  I m an attorney and I m one of the lawyers who represent the kids with disabilities.  We have the Wright s law website and a bunch of books about Special Ed.  The question is that kids with disabilities have the right to a free, appropriate, public education.  Can we let the kids that do not have disabilities have some of the same rights?  For example, under Special Ed law, parents are the enforcers of the law.  The National Council on Disability made it very clear and took it back to civil rights.  The Feds do not enforce the law, the states do not.  What is happening now with No Child?  Same.  But we know back in 1972 right here in DC, if you were in a wheelchair and you had mental retardation, you were not allowed to attend school.  It was too expensive.  A big case, the Mills v. DC case and the Park case turned it around with litigation.  What if we gave parents the right of enforcement under No Child?  Do you think we would see some school districts get fearful of litigation, and is that not what really has driven change in Special Ed, the parents?  Certainly it has not happened with the Feds.  So my question, I guess, really is if we give parents more rights, they have no right of action if the teacher is not highly qualified.  If we gave parents more rights, could we not start to see some implementation? Jay P. Greene:  In fact it was one of my ideas.  If we wanted to enforce this, we would expose schools to legal liability for failing to provide an adequate education.  Now, I should also emphasize that for some of you out there, you have some nightmare in your head of tidal wave of litigation and incredibly burdensome costs.  That is how you imagine Special Ed.  You are wrong.  As it turns out, just like IRS auditing, you do not have to audit very many people to get really high compliance.  Similarly, very, very rare lawsuits actually produce very good compliance in general for Special Ed.  So for example in California, just 0.6 percent of students with disabilities filed a formal complaint, 0.6 percent in a year.  I mean, we are not talking about huge percentages of students filing lawsuits.  They just do not.  If you are thinking that Special Ed is just outrageously expensive, and that if we were to expand this model more broadly that it would be unreasonable, Special Ed does not cost nearly what you imagine it does and there is actually a piece forthcoming in Education Next that I have written on this topic, that actually the costs are not growing very rapidly as percentage of school revenue, and are not nearly as big as people think. Mike Petrilli:  Mike Petrilli, Fordham Foundation.  I have a question for David.  We know that AYP means something different every year in most of the states.  So, do you have confidence that those schools that made AYP, maybe because of these provisions, that it was because they improved and not just because it became easier to make AYP in Michigan over the course of time? David Plank:  I do not have data to back it up so I cannot tell you with confidence, but I do have personal confidence that the large investment that Michigan has made in NCLB-related investment has been in their tests and assessments.  They have hired a number of people in that operation, very good people.  They have really tried to beef them up.  The Fordham Foundation is not very happy with Michigan s standards, but we in fact previously had a high school graduation standard that required one course in Civics.  We now have grade-level expectations across the curriculum.  So I think that both standards and assessment have improved in Michigan, not deteriorated. Mike Petrilli:  Were there changes in confidence intervals, or all those things that could make it easier for schools to make AYP as far as you know? David Plank:  They are in dispute, I think, with the Department of Education, but they have not achieved what they wanted to achieve in terms of confidence intervals, for example. Roberto Rodriguez:  Hi, Roberto Rodriguez with the US Senate Health Committee.  I have a question for David as well.  The first, have you seen in your research a correlation between the degree of the states kind of hands-on involvement in those types of school improvement activities and the success there?  Is the state really directly involved in these strategies?  And then also if you could comment on whether there were outside resources brought to bear on these strategies that you think might have helped succeed? David Plank:  The answer to the first question is yes and no.  The yes is that the state pays very close attention to schools that are in years Three, Four and Five of NCLB sanctions, so that as schools moved further up, sort of down the performance ladder.  The state pays closer attention to them and the state focuses its attention on about 25 of the lowest performing schools.  With that said though, the state has no capacity whatsoever to reach out to the district or the school level, so that there is no direct engagement by the Michigan Department of Education in those schools.  It is left to the intermediate school districts and the local school districts to deal with those schools.  In terms of external resources, there has been some sort of technical assistance that has been provided by outside consultants, by Michigan State University, by other organizations that have sort of prepared coaches that have sort of gotten people oriented to the things they need to do to make AYP, but in terms of financial resources, nothing significant. Lynn Olson:  Hi, Lynn Olson, Education Week.  I wanted to push a little on this question of hard versus soft accountability because reading across the papers, you could say, states and districts have enforced hard accountability, and therefore in reauthorization maybe the law should focus more in making hard accountability real.  But a couple of people said there is no evidence some of those hard accountability measures actually raised student achievement.  And David, your paper suggests maybe some of the soft accountability measures do.  So, the question is what evidence do we have about this hard versus soft approach, and as you go towards reauthorization, which do you want to invest your efforts in beefing up? David Plank:  The way I framed this in the papers is that we have competing hypotheses; we really do not know very much about how to make low performance schools improve on a consistent basis.  We observe it happening but we do not have a real, sort of tight formula for how to do it.  So we have competing hypotheses.  If we have to put a name on it, the Rick Hess hypothesis, that sort of being mean would drive schools to improve their performance.  And we have, if you would like the Richard Elmore hypothesis or others, that what they really need is technical assistance, and then if you give them the support that they need, then they will improve their performance.  What we have right now are a series of provisional tests of those competing hypotheses, but we do not have strong evidence.  We certainly do not have strong evidence that the remedies required under NCLB are effective in consistently raising performance across schools.  If you think about the Michigan case for a second, even the people who are responsible for implementing the soft accountability say this is what schools should have been doing all along.  They just have not been doing it.  And the question is, if you kick them, will they do it more?  Or if you tell them how to do it, will they do it more?  And I think that is really the fundamental question. Jay P. Greene:  Actually, I m not sure it has to be either/or.  It could be kick them and tell them.  And we do have some experience.  Oddly enough I think we have a lot better evidence on how to get failing schools to improve from outside of NCLB than within.  For example, we have some good evidence from Florida s accountability system before NCLB, or independent of NCLB.  And so we could try to imitate more successful accountability and choice systems but the federal legislation did not do that. Frederick M. Hess:  There is the irony, of course, in a piece of legislation that mentions scientifically-based research so frequently that none of the remedy provisions have anything that approaches even a lukewarm level of scientific backing.  These are reasonable intuitions people had about how to tackle these schools and so there is that questions as well. Morgan Brown:  Morgan Brown with the Education Department.  I just had two very quick questions that relate to both the studies.  Jane, you talked a lot about choice in Miami-Dade but you did not talk as much about SES, and my understanding is Miami-Dade went from quite low participation rates in SES to quite high participation rates recently partly because of some of the actions the Florida Department of Education took, and so I wanted to hear a little bit about that, and then very quickly for Stephen, I recently read an example of a rural school district in Kentucky that had some of the typical role challenges on providing a choice option under NCLB and they chose to start a virtual school to provide that choice option.  I cannot remember the name of the district but I would like to know whether you heard anything about that. Jane Hannaway:  I can say a little bit from the data I got from Miami about what is happening there with the supplemental education services.  What they did is they really did an all-out effort to try to get parents to increase the take-up rate on supplemental education services with a variety of ways, and they did.  From 2004-2005, 7.8 percent of the kids who were eligible for SES requested SES.  In 2006-2007 as of October 31, 23.5 percent, so that is a three-fold increase in the percent requesting, and this can go up over time because they may still request more SES.  Of that 23.5 percent requested SES, but only about 8 percent were actually receiving SES and I did not get too much into it in my presentation because there were so much else going on with school choice to compare it to, but some of the issues associated with SES that I got from the district was a huge management problem.  I mean, you have these multiple providers out there that are impossible to monitor.  They are all over the place, you just cannot be everywhere.  They want to use the schools.  The providers want to use the schools so you have to schedule which provider is using the school or when.  Apparently, there are kids that are sometimes unsupervised, waiting for their provider to have the classrooms to do it.  The payment system is one where, as I understand it, where the provider gets paid per hour of instruction per kid.  So the accounting system that the district has to go through to pay the providers is a very complex one.  There is a lot of infrastructure associated with SES.  Another thing that concerns some people down there is that they had these fairs where some of the providers are actually quite aggressive marketers of their services, and they all advertise this sort of free tutoring for kids so parents would sometimes go to these fairs and they would sign up for three or four providers, which really sort of screws up the data and then the district is trying to figure out where is the kid going.  Is he going to this provider or that provider?  Of course, for the parent it is free. So it is a huge management issue and if there is going to be an SES system that is going to develop, I think there are going to be a lot of management hurdles that are going to have to be overcome.  I do not see this operating as a sort of simple market, because in many ways it is actually a very regulated market because the payment comes through the school district responsible.  It is not Let the buyer beware.   It is not let the buyer beware because you have this intermediate in there, so I just think it is a huge management problem. Stephen K. Clements:  Once the outreach was done, there were certainly signs of increased parental demand for tutoring services. Jane Hannaway:  Yes.  Well, let me give you a numbers as well.  The percent requesting went up to 23.5 percent, so it tripled from 2004-2005 with a lot of effort.  A lot of effort that the district should be commended for.  The percent actually receiving services went up from 2 percent in 2004-2005 to 5 percent in 2005-0606, to only 8.1 percent in 2006-2007 as of October 31.  You have only 8 percent actually receiving services, even though 23 percent signed up for it.   Part of this, in a big district like Miami, I think SES is very complex.  A lot of this could shake out over time since it is at the early stages of learning, but it is hardly going to be a simple system especially when we start monitoring performance.  If we are going to be monitoring performance of the SES providers, I suppose you could do it if you can get the individual payments records, so you know how many hours of instruction an individual kid got.  It is something that they are struggling hard with; it is something that they are taking very seriously; I m talking about the district now, they are really trying to make it work.  It is something where they clearly did get marching orders from the state and they take that very seriously.  I said why are you doing this?  Number one, it is the law.  We have to do it.  Number two, the state really wants us. Frederick M. Hess:  Steve, if you could also just say a point, has NCLB as a public choice and SES provision, pushed the people in any of these three districts to start talking more aggressively about virtual options? Stephen K. Clements:  Well, let me just say that some of the supplemental services that have been provided in these districts have been through the Kentucky Virtual High School Program.  That is on the table for some of these districts, but in some districts it is easier to accomplish than others because a lot of these poor families do not have computers at home and it has to be done on the school property under supervision and so forth.  I have not heard of a virtual school, virtual choice idea.  I think it would be very fruitful for the Feds or the states or some entities to put money into looking at the potential options in places where they do not currently exist, whether this is schools within schools, or virtual choices, or other kinds of things.  I think it is going to be a long-term but useful project to try to break up the very traditionalistic views of rural communities toward schools. Frederick M. Hess:  And were these tools mentioned or raised when you spoke to principals or administrators in these three districts? Stephen K. Clements:  No.  The grammar of schooling idea is very, very powerful in these communities.  Many of these communities, the people who were still there are the people who went through these schools 25 years before.  So it just does not occur to many of these families that they might consider going someplace else. Frederick M. Hess:  Okay, John and then Jeff.  John? John Winn:  I wonder what we should think about or what our response would be to an accountability system where you have more schools fail every year than the year before? Frederick M. Hess:  And the answer is?  We will treat that one as a rhetorical.  Tom? Tom Toch:  I am Tom Toch with Education Sector, and we have heard today that there are probably thousands of schools in restructuring or likely to be in the coming years, schools that have failed for multiple years in a row, schools that are in real trouble that are largely dysfunctional.  At the same time, we have heard today that the states have the capacity to perhaps turn around, or do the hard work of fixing these severely troubled schools in the hundreds, that is, if they can manage maybe a dozen or two each.  How do we resolve this dilemma, this conflict, given the fact that turning around the truly failing school is such hard work, that it takes resources of lots of different sorts and time? David Plank:  I will not presume to have an answer to either question; I think they are the same question, which is that we have designed an accountability system which is going to identify more and more schools over time as not making adequate yearly progress.  What we have observed, at least in Michigan, is that with concerted attention from the state, from district officials, it is possible to get schools over the first hurdle, which in some cases may be a very, very low bar.  What is it going to take to get all schools to 100 percent proficiency in the next seven years?  I guess I ll be the first to say publicly in this meeting, it is not going to happen.  That is not an achievable goal.  It is not even clear that we can get schools over the next relatively low hurdle. Frederick M. Hess:  We have time for one more question - I will take Julian. Julian:  I wanted to talk to Jane a little bit about her contention that AYP is not a good measure of school quality.  I think that can well be right, but could you do a little more to work with your own value-added measures of test scores to see what it is that parents care about a lot more than achievement?  The work of Schneider, Teske, and Marshall all suggests that non-whites care about safe schools, and work in Texas suggested that distance matters a lot to lower-income parents, and what we have done in San Diego suggests just about everything but test scores matters.  Could you say a little bit more about anything you have been able to do so far with alternative measures of achievement and whether that matters? Jane Hannaway:  Actually, we just started working with these data so we are going to be working with them more, and we know what the schools are so we can add in other descriptors of those schools.  The unfortunate part is that we do not have more information on the characteristics of the individual kids, but the thing that struck me and I mentioned this earlier with this 27 percent of all the choice that is going on, 27 percent is from A school to A school.  It is sort of the largest percent, so there is a very large fraction of parents who are making choices on the basis of some more fine-grain measures or indicators of school quality, as well as factors that are other than straight school achievement.  But we are going to be pushing these data more to try to get at some of that.   Panel IV:  Reconstituting Districts and Schools Chester E. Finn, Jr:  Thank you to the persistent and interested members of the audience.  This panel, which has two terrific papers and two terrific discussants, is specifically focused on reconstituting schools and reconstituting districts.  Bryan Hassel and Joe Williams have written two excellent papers and Alan Bersin, who is almost a free man, and Morgan Brown, who is not and shows no signs of becoming one any time soon, are going to be discussants from the district and the state perspective and from the federal perspective.  Take it away, Bryan--and you have a co-author, right?  Bryan C. Hassel:  My co-author, Julie Kowal is here, at the back.  So, when I think about restructuring in the context of this, or restructuring of any kind, the first thing I think of is my 10-year old daughter s teeth which are about to be in serious need of restructuring.  I personally had my teeth restructured when I was a child and I can tell you, it was painful both physically and psychologically because I had to give up some of the most important human activities like chewing gum and crunching ice.  I think it was also painful to my parents, and so I m going to have that pain, too.  And yet, apparently, teeth need to be straightened.  I think this is the way the debate about restructuring in No Child Left Behind often goes, too.  Proponents say there are schools that need to be straightened out and this is a necessary thing that needs to be done and the restructuring provisions are going to get the job done.  Opponents say this is unnecessarily punitive, unnecessarily painful.  It is not going to get the results that we desire and we have other courses of action that we need to follow.  We thought in this paper that we would look at what is happening in some actual schools that are engaged in restructuring to get that view of this picture because it is easy to get lost in the kind of big-picture policy.  You pick up some interesting nuances when you get to the school level.  We have talked a lot about the law so I m not going to go through these, but we have these five options for schools that have reached this point of No Child Left Behind - four that are thought to be more dramatic, and one that is more unknown.  There were about 600 schools in restructuring in 2005-2006, but so few schools are making it out of improvement status each year generally, that it is probable that 2,000-3,000 schools will be in some form of restructuring over the next few years if current trends continue, and that can be even higher as the No Child Left Behind AYP metrics get more and more difficult to meet.  As you would expect, about 90 percent of the schools in restructuring were in urban districts.  Fifteen districts, in fact, had half of all the restructuring schools, this according to AIR s data about what is happening around the country with restructuring.  And just a few states had nearly 70 percent of the schools in restructuring in 2004-2005.  Demographics are what you would expect, as is always the case when you look at schools that are having trouble.  Now, as everyone has pointed out today, most restructuring that is going on is this mild and moderate option, option five.  The charter school option is almost never used; contracting out almost never used.  Replacing staff, a little more common, but this other option is the most common and under that you see everything from extending the school day, outside expert advice, and other kinds of more incremental reforms being pursued.  But that national picture does not really give you the flavor of what is happening at the school level, and that is what our paper aimed to do.  We profiled the restructuring experience of four schools from Michigan and California.  We picked those states because they have been studied extensively by the Center on Education Policy, and we also have David Plank s paper on Michigan and a California paper from Julian Betts.  There is a lot of information about what is happening in these states.  We selected two schools in each state from the states lists.  We interviewed people involved at the district and school.  We reviewed all the documents we could find about restructuring.  And of course, these are not any kind of representative sample of schools in restructuring.  We are not making any kind of claim like that.  They are just a way to get some interesting flavor, anecdotal information from on the ground.  If you look at this slide here, you can see that the four schools were similar in some ways.  They are all schools with what you think of a high-need populations, high-low income, high in many cases, English language learners, high minority populations, but then they pursued pretty different restructuring strategies.  So, a couple of them, Buchanan in Grand Rapids, Michigan, and Balboa out in San Diego pursued option five.  They did things you think of in conjunction with option five.  They adopted new instructional model, all-day kindergarten, increased use of data in the case of Buchanan.  Balboa enhanced its professional development and it reorganized the school day.  In fact, they reorganized the school week so that Friday would be a day where teachers would have a lot of time to do planning together and other kinds of work.  The bottom two schools here, Milwood, which is in Michigan as well, in Kalamazoo, and Gompers, which is in San Diego, did things that are more dramatic.  Milwood had a new principal, replaced 65 percent of the teachers and changed its overall instructional focus to a magnet program and even added a grade, so they changed their grade configuration.  Gompers in San Diego became a charter school.  So they replaced & 75 percent of the teachers.  They had a very different focus of their instructional program and of course, along with having a charter, gained all kinds of control at the school level with their budget and other things.  That is the kind of overview of these case studies.  The paper has a lot more detail about all the different things that went on in these schools and I commend all those stories to you.  What I thought I would do in my presentation, given the strict time limits and given the fact that Checker is the moderator so I actually had to adhere to them, right?  Go straight into some of the lessons and findings that we drew from these four cases, but then use examples from the cases to illustrate them so you will get a sense of what we saw in these cases without having me sit up here and talk through them one at a time.  So the first one, what we observed in these four schools is No Child Left Behind providing an interesting opportunity to make changes, but then giving really wide flexibility on how to proceed.  We did not conclude from looking at these cases that nothing is happening in schools under restructuring.  Even the schools that were doing more moderate reforms were using the No Child Left Behind restructuring moment as an occasion to say, Okay, what are we going to here?  Our school is not working as well as we want to.  Let s figure out something to do.   Of course, they ended up doing very different things.  And in some cases, we found that No Child Left Behind really did empower the leaders to make changes they otherwise probably would not have made.  They said things like, We probably would not have gone to this extreme without No Child Left Behind, or it would have taken longer.   The main thing that we saw in that vein was the pretty large-scale staff replacement that took place in most of these schools, except for Buchanan.  Buchanan is an example of where& perhaps maybe not a lot went on that we might think of as restructuring in the sense of dramatic changes.  Most of the things that happened there were preexisting reforms like a new instructional model, or a school improvement team.  The kinds of things that you would expect to see in schools in the second or third year of No Child Left Behind identification.  But in the other three, especially Milwood and Gompers, pretty significant things happened at these schools; whether they are things that are going to lead to good results or not, of course, is another question that I ll come back to.  What we concluded at the bottom of this first lesson is that this Act does put something of a tool in the hands of local leaders that they might not otherwise have.  But how or whether they employ it remains a local decision, and therefore is a very indeterminate situation.  So back to my little debate at the beginning about proponents saying, Restructuring is the answer.  We are going to straighten things out with it, versus opponents saying, This is a punitive system that makes schools do all these things that they should not be asked to do.   Either of those situations really pertains here.  The Act does not really require anything in particular to happen at schools, but it does give a tool that could be used by people that have the will and the capacity and all those things to get things done.  Our second lesson is that we saw that No Child Left Behind was just one of the many, many forces that are acting on the schools in these areas.  Let me give a couple of sides to that.  One is in the incentives that they face.  It is true; No Child Left Behind makes a slight change in the incentive picture because these schools are now being required to do something called restructuring.  That is on the table.  But there are many other incentives that work when you look at these schools and in particular, in the cases of Milwood and Balboa, both of these schools had endured huge enrollment declines.  They were losing students.  In Balboa s case, it was because of, I think, demographic enrollment shifts in the district.  In Milwood s case, it was a matter of competition from private schools and charter schools.  And so these were the forces that were making some people within the school community really say, We have to do something here.  We need to change something.   In fact, probably we are much more dramatically impactful on the school than No Child Left Behind was.  No Child Left Behind was probably a minor factor compared to those other forces.  So those incentives are coming from all different directions, not just No Child Left Behind.  At the same time, schools also are operating in this incredible web of constraints that No Child Left Behind does not really address at all.  For example, when Balboa started going about its restructuring work, it faced a collective bargaining agreement that strictly limited exactly how it could go about handling the staff aspects of the work it was doing.  When the other school in San Diego, the Gompers that was going charter school, which of course is the way you think you escape constraints the most, even it ran up against a lot of constraints.  It had to attain approval from 51 percent of the tenured faculty even to become a charter school, and it had to overcome, I m sure we can get some personal perspective on a pretty serious bid of opposition from the school board to the whole idea of becoming a charter school.  And so even going charter does not free you from the kinds of constraints that schools are under.  No Child Left Behind is one of just many, many forces operating on schools in these situations.  A third area of lesson that we came to had to do with the role of the district in this.  No Child Left Behind says the district is responsible for restructuring but it does not exactly say how that is supposed to work.  We found a wide variety of district roles in these cases, ranging from, in the Buchanan case, more or less a mandate, the district saying to all its schools in restructuring, Here is what you are going to do.  Here are the changes you are going to make.   All the way over to San Diego where school-level teams were put together that involved parents and staff and community members that spent time working through what they wanted to do in the way of restructuring, ultimately subject to the approval of the district, but generated in its grassroots way.  And if you think about it, neither the kind of strict dictatorial across-the-board-uniform approach, nor the let-the-school-decide-what-it-is-going-to-do approach makes a whole lot of sense conceptually.  The schools are different; they have different needs.  It makes sense to have some tailoring of strategies.  On the other hand, allowing the school community itself, which has had years to reform under other accountability systems and hasn t to just simply come up with a plan also probably seems like a poor idea.  So San Diego, perhaps, was a good hybrid of district involvement and yet, school-level thinking which, at least on this issue, could be thought of as a model.  And so district role is something that is very much influx in these schools, and it will be interesting to see how it plays out in other cases.  And then finally, this relates to that last point.  We saw in some of these cases the restructuring moment again providing an opportunity for engaging the school community, parents, and people in a broader community, the way that they might not have been engaged otherwise.  In a couple of the schools that we interviewed, their principals were accustomed to very little parent involvement at all, and yet the chance to say, Here, we are going to restructure this school, led to a chance for people to get involved in ways that they would not otherwise.  And then again, that San Diego School level work group is probably the best example of that.  Now, whether that is a force for good or ill or neither is another matter because parents might be more conservative than they need to be in the case of school restructuring.  Parents are often really satisfied with their schools, according to polls, even when they are not so great.  And so this is not necessarily to say, This is good, but it is an interesting part of the picture in these cases.  And now I want to conclude by saying a word about results.  We know not so much as we would like to about what it takes to turn around successful schools, but we do know from turnaround literature in the business sector, for example, that usually, when organizations turn around successfully, they get pretty dramatic results early on.  The early winds that they get spark a cycle of improvement that goes up and up.  There is not a lot of evidence of that kind of dramatic gain in year one in these schools.  Ironically, the exception here in the math case is Buchanan, and which did see a large increase, but the rest of the scores are pretty flat; reading looks even flatter.  There is not evidence that, at least in the first year, we saw the kind of start of that cycle that you might expect to see in turnarounds.  And it just leads me to think that restructuring mandate is perhaps necessary as part of an overall policy.  It is necessary to say this must be done.  But clearly, far from sufficient to get the kind of virtual cycle going that we need in these schools.  Chester E. Finn, Jr:  Thanks very much, Bryan.  Joe, taking us to Baltimore, among other places.  Joe Williams:  Sure.  A few years ago, the Seinfeld television show had an episode that sort of encapsulated what the whole Seinfeld phenomenon was about.  Jerry and George Costanza went to visit some television execs and pitched a sitcom to them.  They asked him what the show was about, and they said, Nothing.  It is a show about nothing.   I have been asked to write about all of the radical forms of district restructuring under No Child Left Behind, including takeovers, turning over districts to outside entities, hanging school board members in effigy.  And so essentially, like Seinfeld, my paper is about nothing - 30 pages, double spaced& about nothing.  It was almost about something.  Baltimore is the sort of case study for this.  Many of you watched this closely last spring.  When the state moved to takeover a handful of persistently low-performing schools, which involved individual schools.  The rationale for the state involvement was the fact that the district itself had been under corrective action in the state of Maryland and had not lived up to some of its own program improvement plans that it devised.  We saw what happened in Baltimore.  I will not dwell on it -- it s in the paper quite a bit.  But this sort of issue gets raised that regardless of what the law says, this stuff happens in a political context and that things may be difficult.  I also just wanted to say, this is something that still could happen in the evolution of No Child Left Behind.  A lot of the focus today has been on individual schools.  There are some states that are only a year into identifying districts in need of improvement.  So this could be something that we would see playing out to a greater extent in the future, although I think the evidence so far about the general sentiments that play out when this does come to head, suggests we will not see that much.  I also do not want to suggest that I m taking a position either way on whether states should be getting involved.  I think there are arguments on both sides about this.  They are the kind of arguments that will play out in a case-by-case basis.  This is originally what we are supposed to be dealing with.  The same concept as it applied to schools will be applied to districts in terms of whether or not they are meeting the various objectives, what the involvement is supposed to be, and making sure that things change.  At this point, talking about a very small number of school districts that fall under improvement, the states are required, I think, December 1st, to transmit the latest numbers on this, so we should have more updated numbers soon.  Frederick M. Hess:  Tomorrow.  Since everybody has said they are so prompt.  Joe Williams:  We are talking about a small handful.  And, as we will see in a couple of slides later, districts, for example, districts like Philadelphia, Harrisburg, they have all been involved in other things, in other various reform strategies that predated NCLB.  So it is to the extent that they are in compliance with the stuff, it is things that they were doing already.  So very, very quickly, I ll run through this stuff.  State education agencies are required to identify districts that have failed to make AYP for 2 years as in need of improvement.  Basically, if they are still on the list after two years, they will be going to corrective action.  Districts themselves, in many of the states that I was able to look at, are pretty much given the first kick of the cat at trying to develop how they are going to get off of this, which, in the spirit of local control, may be a wonderful thing, but it could also explain why we do not tend to see drastic action.  We do not see many superintendents who are putting forward plans that put themselves out of work.  So what happens if a district does get to the point where they are in corrective action?  States are required by law to be one of six things:  Defer programmatic or administrative funds.  I think the Florida example we heard before is the first time I have heard of that, and I feel for Rudy Crew and his family for not having money to eat that month that his paycheck was withheld.  Institute a new curriculum.  This tends to be overwhelmingly what we end up seeing happening in district restructuring to the extent that any restructuring takes place.  Replace district personnel, remove individual schools from district control, and take over an entire district, or abolish or restructure the district.  In some cases, those six options are limited on a state-by-state basis, if laws in the particular state do not allow some of the actions to take place.  So 36 states do not allow for contracting with third parties to operate schools, and 27 states do not authorize state takeovers of schools or districts.  This was a quote that I found in one story from 2004 which sort of implies there are not a lot of people, like Rudy Crew, who are worried about not getting their paychecks for a month.  This North Dakota Superintendent, after his district was placed under improvement status, said, It actually means very little for the school district.  Now, the biggest thing for us is that it is embarrassing and we would like to remedy as quickly as possible.   So why do we not see more radical forms of restructuring, that whole idea of this local control works against it?  Now the Feds pretty much looked to the States, which looked to the districts to determine initial plans for reform.  What you end up seeing is it is up to the state then to come in and say that the plan the district devised on its own is not going to cut the mustard, and that presents another level of the kind of political activity that we see at the state level.  One thing, I cannot quantify this but it kept coming up as I continued to talk to people at various state education agencies.  The idea that there is a good working relationship between districts and states seems in many cases to be enough to prevent anything drastic from happening.  The state, once it becomes involved in working intimately with the district during implementing some reform plan, has a vested interest in the plan that is rolling out, and I think there is a comfort level for the most part between districts and states that are working closely.  In the Baltimore case, we saw that it was one of the reasons.  The state had lost confidence in the City of Baltimore s ability to deal with its low- performing schools.  My own understanding is that the new leadership in Baltimore is working more closely with Maryland officials.  It may not be as heated this year when it gets revisited.  The other thing that ends up jumping out is districts that end up in corrective action, they do not just end up there out of the blue; there are generally all sorts of things that have gone on at the state level, at the local level, including just even electoral politics, school board people changing hands, and new superintendents brought in new plans they have ushered in.  Some of the examples in New Jersey and New York, in Patterson, all of these, they are all part of the district.  Many of them had already been taken over by the states.  So what else are you supposed to do with those districts other than put them on double secret probation.  And this is another point, and I think this is an interesting one, and I think it resonates with a lot of people -- but a lot of people actually involved in the day-to-day work of this stuff, tell you that imposing the same work for districts that you have for schools ignores the reality that it is a completely different process, it is a completely different piece.  Another one, I think this gets to whether it is reality or just political.  The last thing you want to do with struggling districts is take away their money, and the politics that brings up, if you are dealing with urban areas, it would be political suicide for somebody at the State House to suggest taking money away.  But this is from Roy Romer of Los Angeles -- people like me like Romer because every time he opens up his mouth a useable quote comes out.  He is getting at this idea that if a district needs help, it needs help.  It does not need to be bashed over the head.  I look forward to any questions or feedback, if you have individual examples of things, like withholding paychecks, I would love to hear them so I can include that in the final draft.  Chester E. Finn, Jr:  Thank you very much, Joe.  Alan.  Alan Bersin:  Luckily, we picked up where Joe left off in timeframe in which we analyze these matters, and some people will find this very uncharacteristic of the way I approached my work in San Diego and some of the things I have said at the State level, but the fact is that we have to put in perspective the relatively short history of public education in the United States.  What has happened with No Child Left Behind?  When Thurgood Marshall won the Brown case in 1954, after about 30 years of litigation strategy, he predicted that all of the schools in America would be integrated within five years.  That was his prediction and someone as saged of an observer of American society and its strengths and weaknesses as Thurgood Marshall could be so wrong about public education and the pace at which it changes should give pause to those of us who are getting unduly pessimistic five years out of the gate, a little after one year after a traumatic restructuring has taken place, at least at Gompers.  I can tell you that compared to 10 years ago when I first entered the sector, the conversation is almost entirely different throughout the education world.  Now, for people who are developing policy at the Washington level or the Sacramento level, for example, that may be less than satisfying in terms of being able to capture evidence of significant progress, but it is a sea change in terms of the way the dialogue is going forward at school levels, at district levels, and indeed, at state policy levels.  Much of the conversation is garbled.  A lot of the conversation is pointless, and even worse, but the point is that the conversation is different and the way in which social change happens in our country requires that it be given time to play out.  The interesting question in the context with public education is whether or not there will be sufficient time, given the conservatism of most human instincts.  The question will be how do we accelerate change?  It should come as no surprise, what Bryan and Joe report to us, that districts and schools, and state educational offices are peculiar candidates to order revolutions.  One who is in control of power typically is not very good at designing, let alone implementing revolutions designed as one seeking for power.  So that the notion and one of the deficiencies is in the Act, although it comes with the territory of large-scale political change, is there really was not an effective theory of action with regard to No Child Left Behind.  How it would actually play out overtime, except at the level of more ideological choices than actual hard or savvy political choices, is a little bit like Terry Moe s analysis of the role of labor unions.  We should not look to districts or states to act in ways in which they are constitutionally and genetically incapable of acting, which sets the stage for much of what we have to do in the next generation of No Child Left Behind.  But it should come as no surprise that districts typically do not adopt more than the most gentle forms of restructuring.  My all-time favorite was the one that I heard a year ago as Secretary of Education, when one district in Northern California reported that it was restructured, and it was going to focus on math instruction next year.  So the wisdom of, at least as I take it from the first five years, is there is much work to be done in terms of fixing the legislation.  It is fairly crude in many respects, but that the general direction is one that we need to preserve, and we should not tamper with that basic purpose.  The worst thing that could happen to us is that Diane Ravitch would write a book three years from now, called Right Back Where We Were.  We do not want this simply to be the latest in a series of reforms that people within the established school order simply wait it out because they know that this, too, will pass.  The main contributors, the main levers of change in the sector are accountability, choice in competition, and transparency, and we should not lose those in whatever we need to do to fix No Child Left Behind.  A wise lawyer once observed to me that hypocrisy is the homage that vice pays to virtue.  Hypocrisy is the homage that vice pays to virtue, and there is a certain amount of hypocrisy to have a law say, here are the five steps that you need to take and at the end of the day, you need to restructure and provide leadership to a new school situation.  That is often the way in which we move the ball in terms of changing.  The balance that we have to watch, I think, over the next 5 to 10 years in this country is that people stopped believing in the law, and they disregard it because they see so little enforcement so that it lapses into the kind of old relic on the books that people point to and say, Oh that does not matter.   And then we will lose some of the benefit of the changed conversation that I think the law has led to so far.  So what are the lessons so far as we can gather at this juncture?  First, I would hope that the research community would do a lot more case studies in an attempt to develop the stories that serve as existence proof.  Gompers Middle School, which became a charter in San Diego stands for a very important proposition, which is that when you actually do what all of us know in the education sector that we need to do to provide for good schools; it happens and it happens very quickly.  The good news is that when you put together a terrific principle, and teachers who choose to be at the school, and parents who are genuinely welcomed into the school discussion, and you provide resources, a steady flow of the change that can happen a year is dramatic.  Gompers was the worst gang-ridden school described well by Bryan and his colleague, even in greater depths in a piece that Joe Williams has done called Extreme Takeover for Education Sector.  A year after that school had teachers who wanted to be there, a principal who was excellent, and parents who are engaged, you see the kind of change in a school that suggests to us that we really do know how to fix schools in America.  The question is why we cannot do it in scale and why we cannot do it in an accelerated fashion.  And in accessing that progress, I think we need to be careful. And I do not think Bryan was doing this.  He was suggesting that there is no miracle reflected in standardized test scores in one year.  But a little like the broken window theory of law enforcement, when you start attending to the small things then the momentum builds and the culture changes.  So, the first lesson is yes, we have to continue to have these sanctions.  We have to continue to call for restructuring and for some of the reasons that Alex outlined in his presentation.  It is important that the standard be up there.  But again, what we have to be careful about is that we pay so little attention to actual enforcement that people stop believing in it and the system loses legitimacy.  Chester E. Finn, Jr:  Thank you very much, Alan.  School reform is safe for at least another few hours because Diane is 225 miles from her computer.  The moment that we have been been waiting for, the first chance for a living, breathing current Federal official charged with the implementation whom we can all beat up on.  Welcome.  Morgan Brown:  Thank you.  I want to start by saying how much I, since I am the only Fed here on these panels, how much I appreciate that Checker is okay with me having twice the normal speaking limit to respond to all of the things that have come before me.  Right, Checker? I will not attempt to respond to all the things that have come before, although I do want to mention a couple and then I will leave the rest for Q&A and you can sling some more arrows in my direction at that point.  There have been a number of very legitimate questions about the lack of data in this area, particularly for Choice and Supplemental Services, and that is something that is a great concern to the Federal Department.  It is unfortunately the old story of since the reporting of that data was not required specifically in the statute, it is not happening.  And when we went to the states to ask if we can have that data, they said, Well, you did not let us know that you want us to collect that data, and so we do not have some of that data.   Slowly, things are changing and I will not go into a long description, but the people who care about data over there have actually, with what is called now Ed fax, which is the way we are collecting data from the states and via the states from the locals.  It has actually undertaken a massive effort to work with states.  Starting this school year, we should have much better data about Choice and SES, and if that is not the case you can invite me back next year and berate me for that as well.  The other thing I just wanted to do before I go into talking about a couple of things that I think really do need to be priorities as we look at reauthorization.  A small disclaimer, particularly for media that are in the room, because I m mentioning proposals for consideration.  I m offering these proposals up because they have been in discussion not just within the Department, but outside of the Department, but I would appreciate if you did not report back that Morgan Brown said the Secretary Spellings was committed to putting this under the reauthorization bill because then it will make it more difficult for me to hang out with people like Checker and Mike Petrilli than it already is, and I probably would not be able to come back.  Let me go through some of these.  I know this panel is mostly about restructuring, but I wanted to say some words about Supplemental Education Services, because I think the SES option and school restructuring requirements really do have to be the two priorities.  I think that it is the perspective of the Department, both in terms of improved implementation which I m not going to talk a whole lot about, but also in terms of statutory changes for reauthorization.  As many people have talked about, there has been some modest improvement in participation rates for SES.  According to GAO, there actually was from 2003-2004 to 2004-2005, an increase of 12 percent to 19 percent in participation even at the time -- those are actual participation rates in terms of eligible students, so that was at the time that the number of eligible students was dramatically increasing.  Having said that, as someone pointed out before, those rates only look remarkable when compared to the choice data, and I think the perspective of the Secretary is that those rates are unacceptably low, and so we really do have to do something to dramatically increase SES participation rates, and if people are interested in hearing about some of the things we are doing on the enforcement and outreach side currently, I would be happy to talk about that during Q&A.  The other thing we need to make progress in, which has been alluded to, is the evaluation -- tracking and evaluating students that are receiving supplemental services, and the ability of states to do this varies greatly, both in tracking the data as a first step but even more importantly in setting up an evaluation system.  This is an issue that I think is going to have to be solved more on the technical assistance side, and we do have some initiatives under way.  I do not think there is much to be dealt with there on the statutory side but again, I would be willing to discuss that more in Q&A.  For SES, for NCLB reauthorization proposals, let me mention two key ones that are certainly out there and have been discussed.  One is the idea that supplemental services should be made available to students in the first year of school improvement where choice is now.  We have a pilot project that I think was referenced.  We are right now in about five states, in about 16 districts, districts that had been allowed to flip the consequences so that they can offer supplemental services the first year and choice the second year, and there has been some positive results of participation rates from that.  Some people, including some good friends in the Choice sector, get into a lot of consternation about the idea that Choice would be in the second year, so let me just say I think it is perfectly acceptable to discuss Choice and SES both being offered in the first year just as they currently are in the second year, and frankly from my previous perspective as a state department bureaucrat, education bureaucrat, where I actually had to oversee Choice and SES, I was sometimes frustrated particularly in a Choice-rich state like Minnesota, where I was, where the parents would come to me and say, Hey this choice option does not really offer me anything I do not have now.  What I really want is the supplemental services, and you are telling me I have to stick around.  I have to stick around the school for another year until I get supplemental services.   That is something that I think will definitely be on the table.  The other one, which has been referenced before, is the idea that the 20 percent set aside of Title I funds that were the equivalent thereof that districts were required to set aside to implement Choice and Supplemental Services rather than having the unused portion simply go back and be available for regular Title I expenditures; it should be required that any unused portion to be rolled over to the following year and continue to be dedicated to Choice and Supplemental Services.  I think people out there would argue that it creates a whole different set of incentives for districts to the extent that there is a perception and in some cases a reality that districts are dragging their feet on implementation because they hope this leftover funding would be available.  If it is not, that perhaps they would get more serious on the implementation side.  I m not going to talk a lot about the Choice Option right now, not because I do not think it is important, but I think there is a lot more complexity to that and I am probably more optimistic about the Supplemental Service prospects.  But some of the many challenges that we have heard about with the Choice option, both in identifying the data piece about the fact that just because the Choice participation rates are low does not mean that families are not using Choice, but also some of the fixes to the problems that are out there, one of which I do believe in a certain district s lack of choice capacity.  It can be solved but it is a longer-term problem.  Let me get to the restructuring requirements, which is what this panel is about and I should start by saying, I do have a little sense of inadequacy sitting next to Secretary Bersin and talking about restructuring.  I liken it to being on a panel that talks about professional football with Tom Brady just because I m a New England Patriots fan.  One of us has actually been in the game, and the other has done a lot of armchair quarterbacking about the issue of restructuring, but let me launch into it and see what I can come up with.  I certainly agree with the points that have been made in some of the papers that we need to ensure that actions taken under the restructuring requirement yield real reconstitution or other major governance changes and that is clearly what restructuring was intended to be.  And so again, I think one of the proposals that has been discussed and will be up for grabs during reauthorization is the idea that the number 5 option is simply eliminated from the list of options that districts can choose when restructuring schools.  We found in our Department research for the national assessment of Title I, that most Title I schools in restructuring status actually implemented one of the corrective action categories rather than one of the restructuring categories.  That is why in some cases I have heard, restructuring being kind of derisively referred to as corrective action year two or corrective action year 3, as opposed to restructuring, planning, and restructuring implementation.  Combined with that, I think there will be some discussions especially if there is agreement to eliminate that fifth option, to the extent the federal government is providing a school improvement grant funds with the Formula Grant Program, that there needs to be a requirement that states focus at school improvement funding much more on schools and restructuring and providing technical assistance for schools in restructuring partly because there is an issue of districts and schools not having the leaders who really know how to do restructuring or reconstitution.  And so that if you are going to eliminate that fifth option for those four remaining options, there should be an emphasis that the states have to really focus on working with districts who are instituting one of those four remaining options and also that the money could be used to have states more closely monitor efforts and restructuring plans that are being implemented to know what is really happening.  I want to say something about other barriers to restructuring, because I think we often do not& maybe we do talk about them enough, but the situation with districts, even when there is the political will to do reconstitution is that many districts and certainly many school areas have no idea how to effectively plan and implement it.  Not just in terms of how you replace staff and things like that but how you do community engagements, as Secretary Bersin was talking about, like Gompers School in San Diego, which has done an amazing job of engaging the community.  And what we heard from some parent leaders just blew me away in terms of the role they now take inside the school on a daily basis in directing and overseeing and having input into that school.  I want to give some credit here to the National Association of Charter School Authorizers, which hosted this conference in San Diego which has just released a series of guides called Starting Fresh, that do not just focus on reopening persistently failing schools as charter schools but look at restructuring in terms of guidance for districts across the board and provide a lot of practical guidance on such topics as engaging parents in the community, selecting the right education providers if they are going to play a role in reconstitution, establishing the right relationship terms with new schools.  These are a lot of topics for which there has really not been much practical guidance out there, and I think that is as much of a challenge as any concerns about the statute.  Greg Richmond, the president of NACSA, deserves some credit and he knows a lot about this because his previous job was being involved with the Renaissance 2010 initiative in Chicago, which for those of who are familiar with it, is committed to essentially opening 100 new public schools over a period of time.  So they really have had to go through the process of closing schools and opening schools and have the bruises to prove it, but have learned a lot of lessons in the process.  One of the quotes from this NACSA guide was, Whether through NCLB or districts own initiative, and whether through a charter contract or some other legal structure for powerful change to happen, a start fresh school must have the freedom to redefine every aspect of the school and be held accountable for performance.   I m not going to tackle a lot here about reopening persistently failing schools as charter schools.  I think that is an option, but I have some words of caution about that as well.  My time is up.  I m going to ask if Checker will just allow me to say a word about the school district restructuring piece, because I think the points that Joe made about it being somewhat early to know to decide whether that has been successful or not or valid, but I also think it is fair to say that there seems to be very little evidence that states have been successful in taking over districts and even to some extent taking over schools.  We heard about the example New Jersey, many of us know what happened to Detroit, so personally I would say if we are going to spend a lot of political will and effort on this issue of states getting involved in districts, there are two other options that should be looked at:  Takeovers by mayors, which at least in New York City and Chicago show some promise, and what I ll call the open sector approach, where we focus on the creation of new public schools, and let parents to the education marketplace decide which district schools should be closed.  And if any of you have been to Indianapolis and watched what Mayor Bart Peterson is doing there as a Democrat, to open 16 new public schools, essentially a parallel school system and open sector school system, and almost 4,000 parents have voted with their feet, and that is causing the district to have to make some tough changes.  I think it should remind us that in addition to top-down efforts with regard to restructuring and reconstitution, there really can be in certain cases some powerful grassroots bottom-up efforts and maybe that is in some ways what we should focus on as Mike Petrilli mentioned with grant programs, including, we have one already, the charter school program that helps expand school choice options in that way where there is local support for doing so.  Thank you very much.  Chester E. Finn, Jr:  Bryan, one of the things that was not crystal clear at least from the oral presentation is in these four schools which adopted these four different approaches, who decided which approach to adopt?  Was it the people within the school, or was it their district, or was it an interactive thing? Bryan C. Hassel:  Well, it was different in different cases.  So in Buchanan, one of the schools in Michigan, the district basically said, Here is your restructuring plan.   In Milwood, the other Michigan school, there was a back room deal worked out between the guy who is going to become the principal and the superintendent to move forward with a certain kind of reform, so it was kind of both ends.  Gompers, that school-level team, developed the plan but there was huge district involvement in that planning process.  And the school board had to approve it.  In the final case, the school, again, it was in San Diego, so the school-level team developed it with district sign off.  That is very diverse and I think that is probably true nationally. Chester E. Finn, Jr:  But at least from the three out of four, it sounds like in your little sample, these people in the school played a very substantial role in deciding the new fate of that school.  Is that right, Alan? Alan Bersin:  Yes.  At least in San Diego the principal was this positive.  Leadership started with the school leader but it does not detract from the collective action that was taken in those schools to say that the principal was key to building the political consensus and having a dialogue. Chester E. Finn, Jr:  Tell me about this other point Bryan made about people saying they did things under this that they would not have done otherwise.  Is that because they got any new powers or authorities or simply because it changed the politics of the circumstance in which they found themselves such that it emboldened them to do things they could not otherwise have done?  Is there any change in what was technically within their control or is it a change in what was politically doable? Bryan C. Hassel:  It was largely a change in what was politically doable in the symbolic politics of the matter.  We heard cases where people seem to think that No Child Left Behind required what was done in these schools, even though of course No Child Left Behind is very vague.  People would tell us that we have to do this because No Child Left Behind said we did, and in fact that is not the case but it was a symbolic value.  The one case where there was real power given was in the Gompers case where once they had devote to become charter then they did gain a lot of specific legal authorities that they did not have prior to that.  But in the other cases, it was less a matter of authority than symbolic political picture change. Chester E. Finn, Jr:  Joe, clarify one thing on Baltimore that was not clear to me even as a resident of Maryland and a reader of some of the local news.  When the state tried to move in on those schools within Baltimore, it cited NCLB, invoked NCLB, but I was under the impression that the state superintendent was actually utilizing preexisting state accountability authority. Joe Williams:  It was both.  They cited NCLB as the justification.  The fact that they were able to do it earlier than other states is because they had the existing accountability framework. Chester E. Finn, Jr:  Under Maryland? Joe Williams:  Exactly, under Maryland. Chester E. Finn, Jr:  And the rebuff that they got from the state legislature? Joe Williams:  One year reprieve, so that it would be revisited next spring, theoretically.  They stopped the state from taking any action at all with regard to those schools. Chester E. Finn, Jr:  Is that equivalent to the legislature overriding NCLB?  Is this massive resistance to a federal mandate? Joe Williams:  For a year and for changing the government structure of those individual schools, yes.  And then you can argue that they were able to get some restructuring in Baltimore at the time since there is a new superintendent or CEO, I guess, that there has been some change in leadership that you can argue was prompted by. Chester E. Finn, Jr:  But the state board of Ed and superintendent did something I believe was warranted under the federal law and their on state legislature said, No, you may not do that. Joe Williams:  That is right. Chester E. Finn, Jr:  Okay.  Alan, has the State of California attempted to restructure any school districts under NCLB?  I know there are a couple of earlier examples of places like Compton that were in financial trouble. Alan Bersin:  No.  The only restructuring that the state has engaged in pertains to six schools that have run through the course of the state accountability system, and while consistent with No Child Left Behind, all of the action has been taken in connection with the state accountability system. Chester E. Finn, Jr:  Looking forward, you see it as likely that the state will attempt to do an extreme makeover of any districts under the federal authority that NCLB has assigned? Alan Bersin:  Not in the foreseeable future, no.  The politics of complicating California by the disharmonization between the state and the federal systems have created enough wiggle room for educational leaders to avoid taking any dramatic action at all with regard to restructuring school districts. Chester E. Finn, Jr:  So we are sitting up here with an example in Maryland, one arm of the state trying to intervene and being slapped by another arm of the state.  And in California, our biggest state, you are saying that this is not likely that happen as a state intervention at the district level.  Morgan, is there any place where NCLB has caused a successful or a state restructuring of some sort of a district? Morgan Brown:  Well, I think as Joe mentioned in his research to the extent it is happening at all, it is happening in milder forms of what we would call really corrective action and not through restructure and reconstitution, and I think the Baltimore example is instructive.  I mean, here is the state that at least had some political will to do it and actually legally can do it, which is another challenge for the majority of states frankly.  And you could not come up with a much stronger case than Baltimore unless you are thinking about New Orleans or something pre-Katrina for doing a restructuring of the district, and so I think that may be one of the reasons we need to look at if we are going to do a restructuring of districts, looking at other options. Chester E. Finn, Jr:  Alan, back to you for a second, you, on the one hand, were quite optimistic about the long term of this kind of reform in America paying off if we give it a long term.  On the other hand, I believe you are the one that said that NCLB expects districts and states to do something, do things they are constitutionally and genetically incapable of doing.  So are those two parts of your remarks not in conflict? Alan Bersin:  At least not on my mind.  The difference being that the specific rule, the explicit rule on reconstitution and restructuring to me, long term is less important than the cultural change to which it contributes in terms of focusing on student achievement, disaggregated student achievement and the like.  How we implement that will require, I think, and one of the interesting challenges we have as opposed to running away from federalism as a mechanism by which we introduce changes to figure out how we have each level of government play an appropriate role in terms of this educational reform.  Until 9-11, law enforcement and education, since the new deal were the only provinces of local activity that retain vitality.  With 9-11, we are seeing more and more of law enforcement being taken over and seen from a national prospective and since No Child Left Behind, that is the perception on education.  I think if we ever look to state departments of education, let alone the federal government, to do actual schoolwork, we will be inviting catastrophe for children. Chester E. Finn, Jr:  Okay.  It is a clear strong statement.  Joe? Joe Williams:  I just want to back up for a second on the Baltimore case & at one point, there was a brief threat from the federal government to withhold funding to the entire State of Maryland.  The threat was not taken seriously at all at that time.  The threat was put out there and Ray Simon sent the letters saying that your funding is in jeopardy.  There are enough lawyers working for the state legislature that looked at the statute and said they are not going to, so do not worry about it. Chester E. Finn, Jr:  I guess there was a gubernatorial election on the horizon at the time, was there not?  Okay.  Morgan, just to put you on the spot on one thing, and I do not really do this to make you squirm, but you talked about the need for reconstitution to be real reconstitution and the desirability of eliminating option five.  I believe I have read in comments from one of your colleagues, at least at the assistant secretary level, saying in effect, We do not have an opinion as to what forms of reconstitution districts ought to engage in. Morgan Brown:  In terms of the options that are out there, whether the current five options now, or where four remaining options, that is the official position of the Department because there is a certain amount of respect for the state role in this to provide some guidance to the locals.  I think, though, as we look at these case studies, it is instructive and looking at Brian s study that to the extent that number five was chosen, it seemed only to have impact when it was done in concert with one of the other options, such as replacing some staff.  So it was not number five alone but in concert with some other options, but you are right.  I do not think that the Department has taken official position that we think, for instance, that all schools should be reopened as charter schools. Chester E. Finn, Jr:  Let s open the conversation.  Phyllis, let s start with you.  You got your hand up first. Phyllis McClure:  Thank you, Phyllis McClure, unaffiliated.  For Mr. Brown, has the Department looked at the unexpanded balances of Title I money in the low-participation states?  And would it not be interesting to find out what they have done with that 20 percent that they have not spent on Choice and SES? Morgan Brown:  We have done so but only on a case-by-case anecdotal basis.  This is one of the major data pieces that is being collected this year through Ed fax, but I think it is safe to say that most states have not come close to spending their 20 percent set aside.  In a few cases and some cases, there may be legitimate reasons why they have not been able to it if there is a rural district with small numbers of students et cetera, but that has been a warning sign for us that many states have not come close to spending that 20 percent set aside. Chester E. Finn, Jr:  But, in terms of the second part of her question, you do not know what they have spent it on?  Morgan Brown:  No.  Obviously they are free to spend it on other approved Title I expenditures under the statute. Chester E. Finn, Jr:  Bill Holly. Bill Holly:  In the spirit of NCLB, I have a multiple choice question for you.  The answer options are strongly agree to strongly disagree.  I think we talked about this almost every session, so I would like to get your proposition to which you might respond, that the more schools we try to change significantly, the less likely it is we are going to change those schools that need the most changing in meaningful ways. Chester E. Finn, Jr:  Okay, agree or disagree and no elaborate explanations.  The more schools we try to change, the less likely we are to change the schools that most need it.  Starting with Bryan. Bryan C. Hassel:  Strongly agree.  Am I fast? Chester E. Finn, Jr:  Joe. Joe Williams:  I disagree.  Not strongly. Chester E. Finn, Jr:  Okay, Alan. Alan Bersin:  I strongly agree. Chester E. Finn, Jr:  Morgan. Morgan Brown:  Somewhat agree. Chester E. Finn, Jr:  We are not of one mind.  You.  Who are you? Carol Jackson:  Carol Jackson, GAO.  My question is for Morgan Brown, about the possibility of eliminating option number five.  The other restructuring options could be options that are left over are charter schools replacing all school staff and private management organizations that they take over.  And in a number of states, not all of the options are available.  So, how are you going to respond to that? Chester E. Finn, Jr:  You mean because of state law? Carol Jackson:  Because of state law, yes. Morgan Brown:  Right, and I think there are two issues.  One is for those people interested in that option, to discuss it.  This may part of the discussion, this pressuring states to at least make one of those -- I understand why in some states the issue of state takeovers is problematic for some of the reasons that Secretary Bersin mentioned, but certainly looking at the issue of districts removing principals and teaching staff does not seem to be out of the realm of possibility and I want to emphasize again, this is not as though that would be the only thing they would do.  They could still choose to do many of the more mild restructuring things or corrective action things they would do under option five, but they would have to make sure at least part of the plan was one of the four major options. Chester E. Finn, Jr:  Bryan. Bryan C. Hassel:  One thing that is worth remembering is that option five does not say, Do whatever the heck you want.   It says, Engage in another form of major restructuring that make significant changes to the school staffing and governance, so another thought is just to actually enforce that provision so that if the school is doing option five, they have to do that rather than change the time of day they do math instruction. Chester E. Finn, Jr:  Somewhere on the far left, I m told there is a question. Kate Neville:  Hi, I m Kate Neville from Cross & Joftus.  I had a question primarily for Mr. Bersin and for the whole panel.  It does not seem like we have heard a lot of evidence that states have the capacity do this, to take over schools or districts and run them well, so what is the argument that the state is better suited to come in than the district in terms of restructuring school within the district.  And also then, if there is no capacity at the state level, there is no capacity at the district level, who knows what to do?  When we did here the one example Commissioner Winn said about withholding Rudy Crew s salary which stands out, as an example of actually taking drastic action.  Wilmer Hutchins in Texas, Joe that might be another example for you, they ended up being annexed to Dallas because they had been taken over by the state time and time again, and people were banging their heads against the wall.  Who knows how to do this?  Is there a private sector option on the list?  But I have not heard much talk about that.  Is there a capacity in the private market to do this? Alan Bersin:  I was actually relieved to hear from John Winn that in Florida he had 12 people, in much the same way that California had only 12 people implementing this.  We constantly try to deny the fact that real change will come from changes in the local context.  To that extent, all of the real controlling politics are local.  The best we can do from the state capital or from Washington in my view is to create the conditions in which that is not precluded as matter of definition.  So that you say Who can redo, if not the state and not the feds?   It is not about going in and having bureaucracies further removed from the local context, actually try to micromanage or manage school change.  But there are very important policy differences that can be made at the state and federal levels to create conditions for local leadership to emerge.  Where the local leadership emerges is never in my view going to be determined in Sacramento or Washington. Morgan Brown:  If I could just follow up on that, I think there are some legitimate concerns about state capacity coming in and reconstitute or take over schools, but saying that there is not a lot of current capacity at the local level is the same thing as saying there could never the capacity at the local level, and indeed there are districts like Chicago, and to some extent, New York, that are doing these now.  And, of course, people like Rick Hess had written a lot about the way we train leaders for both districts and principals, and the kind of training we have does not prepare them for this type of thing at all.  So, one of the few things we have to do is take a look further back in the human resource cycle of who comes in to these districts, which I agree is a long term process.  But I do not think we should just throw up our hands and say, There is no way school districts will ever be able to do this. Chester E. Finn, Jr:  Next?  Okay, Alex.  Yes, Alex Medler. Alex Medler:  Alex Medler with the Colorado Children s Campaign.  Here is a question for Joe.  You talked about having only a thousand districts getting into restructuring. Joe Williams:  Not restructuring, needing improvement. Alex Medler:  Needing improvement.  My question is, in Colorado, all the sizable districts are there already right away, and everyone knew that would happen.  Are they the thousand biggest and 90 percent of the big ones?  And if so, then this is a bigger deal than you are letting on. Joe Williams:  No, I actually do not know.  That number came very late in the game from the US Department of Education.  In fact, after the deadline for the paper, I made the slide here but not for the paper.  We know some of the ones that have been identified specifically are the big ones, so you are right.  But I think it also gets to the point that these are all district set of doing things and for some time as well.  They are not popping out for the first time. Chester E. Finn, Jr:  Based on what Mike Casserly has stated this morning, I believe, about two-thirds of the districts in his sample are in this situation right now.  And those are big districts with hundreds of thousands of kids. Joe Williams:  One of the things that is playing out there is that the discussions between the state and the districts about why they are on the list& just talking to some state officials in Delaware that two districts that are supposed to undergo restructuring, both of them are on there because of failing to make AYP for disabled students.  So you can pinpoint why they got there.  It is easier to find whatever technical assistance they are providing, and therefore, why do we want to take over an entire district if we are just going to be dealing with disabled kids? Chester E. Finn, Jr:  It is a Special Ed problem.  Interesting. Cynthia Brown:  Okay, thank you.  Bryan, as a graduate of Milwood Elementary and Junior High School, I was disappointed to see its difficulty but glad it is making improvement.  I have a question.  As one who spent the first half of her career on enforcement activities, I have been a little startled particularly in the setting that we are not talking about incentives to get more radical change in school improvement.  And so I wonder if the federal government might bribe some more dramatic improvements by setting up some kind of carefully tailored incentives program to fund schools that are willing to say extend learning time like what is described in the New York Times magazine this week that keeps schools and few regular public schools that are doing it, if that might encourage change.  We see it in the teacher incentive fund.  There are some pretty dramatic proposals that are being funded with some big money under that program, and could we carefully design programs that could provide incentives for more dramatic change than we have been talking about today? Chester E. Finn, Jr:  This is also a plausible question for the next panel, which is supposed to do with takeaway lessons from the whole day.  So Alan, what is the prospect for bribery? Alan Bersin:  Well, I would not want to put it that way.  The answer is absolutely, and if you look in the post-Brown, using that reference point again, nothing happened after Brown v. Board of Ed until the Department of Education, the Office of Civil Rights started to use the lever of withholding funds, and then change happened very quickly.  I think earlier today, Checker observed that federalism has been a fairly sure structure in American history for distributing and allocating funds.  It is a natural to do that.  If, in fact, we can avoid all of the political pressures and seeing that it is not just spread around haphazardly by political rather than strategic design. Chester E. Finn, Jr:  She was also talking about new possible, additional& yes, not just withholding current. Alan Bersin:  Exactly to provide, I think with the point of specifically tailored grant programs along the lines that Mike Petrilli suggested exactly in order. Chester E. Finn, Jr:  We have about five more minutes for this session. Mark Vineis:  Mark Vineis, Mondo Publishing.  In looking at is this tool kit working and taking Alan s optimistic view & and some questions that were raised even earlier -- should we be thinking about the changing and the adding of some of the tools in the kit.  For example, so many of the tools and where compliance has been attempted to be made is heavily put on the classroom level.  Should more tools be added to the kit that take into consideration that classrooms are part of schools in building principal leadership and have tools in the toolkit at that point.  Schools are parts of failing systems in helping build capacity at the system level and thinking about in that way so that it is really about system reform and building capacity in scale, and thinking about the act in that direction so that it can really impact the classroom. Chester E. Finn, Jr:  I sort of think Mike Smith is going to talk to you about this in the next panel in some fashion, but what do any of you think? Bryan C. Hassel:  That seems like another potential use of funding, the incentive is one, capacity is the other, but I think both of them face the same challenge of how could you do it and not just throw the money down the drain and not make it to waste.  It is easy to say we are going to build capacity through a program but what would that really look like?  You have to have a pretty strong theory of what sort of capacity is going to lead to effective change.  For instance, leadership.  What kind of leadership so we want to foster and how can we foster?  If we have that, we could develop some good investments. Chester E. Finn, Jr:  So far, they have not scaled very well.  There is only one Alan.  Around somewhere out of sight? Michele McLaughlin:  Hi, Michele McLaughlin, American Federation of Teachers.  I cannot see you, so I m going to talk to the screen and pretend that you are talking to me.  I was glad to see that Bryan brought up the issue cause because we have heard a lot today about districts not being willing to take the meaner options, or tougher actions under restructuring, but we have not heard anything -- maybe the reason.  Maybe they have not done so because it is very expensive, extended time which has seems to work in some districts and in New York City, the zone schools in Miami are really expensive.  So it is interesting to hear Morgan talk about maybe having some targeting so that there would be more resources, because right now you get the same amount of money if you are in the first stages of improvement as you get when you are restructuring. Chester E. Finn, Jr:  This was the last question for this panel. Morgan Brown:  I think there is certainly some merit to looking at especially if you are looking at the tiered approach trying to help those schools most in need, which would potentially be the restructuring schools at some very kind of targeted financial help.  Although I think that some of the points the panelists have been brought up before about trying essentially to get schools to do something that their leadership does not want them to do just by giving them more money should raise some questions.  And that is why I m more partial to some of the ideas where you would do some grant making to help schools and leaders do things they do want to do, which is one of the benefits of the chartered school programs as you give money to start new schools because they want to have an extended day and they believe fundamentally in it as part of their program.  There is also the fundamental idea behind the President s America Opportunity Scholarship for Kids Program, which is a grant program to help expand private school choices of the state level, and that grant funding could go nonprofit organizations that want to provide scholarships, just like the Washington Scholarship Fund does here with the DC Choice Program.  That might be a more fruitful way to look at it. Joe Williams:  Michele, I think you have a good point.  Some of the state education agencies that I spoke with said that not only did they not have enough money to do more radical forms of restructuring, they do not even have enough money to do restructuring like the kind of technical assistance that they are being asked to provide.  They feel like they are already stretched in pulling that off in the offices where they are talking about 8 to 10 people, like we have heard before. Chester E. Finn, Jr:  Okay, our last five-minute biology break is going to start.  Join me in thanking the panel.   Panel V:  Lessons Learned Frederick M. Hess:  We are going to go ahead and get started if folks would please take their seats.  Okay, we are up to the last panel of the day.  We have assembled several authorities to try to help us make sense of the 13 analyses and the various discussions that we have heard thus far today.  This is a session where we are really trying to see if there are particular takeaways, broader and more general or more specific and implementable, that speak to the NCLB remedies.  This session will run for about an hour.  Each of our speakers will start off with some brief comments, and then we are going to open it up for them to have a conversation for a while about the questions on the table with some prodding by Checker.  And at the point that it seems appropriate, we will then open it up for questions and thoughts from the audience.  A couple of particular things I want to mention before we get started.  First, I want to thank the funders who have made this research and this conference possible.  That is the Koret Foundation and the National Research Initiative.  I would also like to thank the generous folks who underwrite the education program activities of the AEI on an ongoing basis.  Second, the papers are available on the web.  When you get to the office tomorrow, they will be available at this website on this slide.  There are also hard copies available for you to take with you in the lobby as you depart.  And finally, we will wrap it just after 5:00.  At that point, we actually need to ask that folks to migrate out of this room because they are going to need to start setting it up for a different event this evening, but there is a reception that is going to be held out in the lobby and we welcome you to stay and talk about the issues with the authors and each other.  With that, let me introduce this concluding panel.  First, we have Kati Haycock, Director of The Education Trust.  Prior to this role, Kati was executive vice president of the Children s Defense Fund and founder and president of the Achievement Council.  Diane Ravitch is a historian of education, a research professor at New York, a senior fellow at the Brookings Institution, and a senior fellow at the Hoover Institution.  Third, we have Mike Smith, Director of the Education Program at the William & Flora Hewlett Foundation and a widely published education researcher.  During the Clinton administration, Mike was acting deputy secretary and undersecretary of education.  During the Carter administration, he was chief of staff to the Secretary for Education and assistant commissioner for policy studies in the Office of Education.  And, finally, we have with us yet again, the inimitable Checker Finn.  With this, Mike, why don t you go ahead and start things off please? Marshall Smith:  I'm going to move very, very quickly through a bunch of slides and I would be delighted to come back to them during the questions and answers.  But as I got into this, it seemed to me a reasonable thing to do to really begin to rethink about it than to shape something that was somewhat different from the conversations over the last eight hours or so.  So the title of this begins to lead you down the path, should we be focusing on the toolkit or should we be focusing on children learning?  Okay, so it is a great set of papers.  And I actually -- even after reading all the papers, I listened today and did not get bored, so congratulations to the paper writers.  In some ways I think you did a great task, great job in some ways.  Nice for the reviewers, their conclusions all came out pretty much the same.  You read through these papers and you will find three things: that they say the implementation of these tools is weak, very weak in most cases.  The frequency of abuse is small rather than miniscule; and there is no discernible achievement in effect when in fact they have actually gathered some data about achievement, which is not very often.  And so, what did we expect?  Well, there had been dozens of reasons given today to explain why this phenomenon happened, why these conclusions.   These are difficult patchwork interventions.  They include many parties - private and public.  There are extra demands on time and skills of parents, little supply site development, little quality control, perverse incentives for schools and districts create new organization entities and relationships, et cetera, et cetera, et cetera, and very little even attempt to get evidence of the facts on these things.  One might expect and there had been arguments of this sort from some very smart people that we can overcome these things.  We can overcome these challenges.  By God, all we have to do is work a little bit harder and get a little bit tougher.  But you know, there is a big literature on these three sets of issues and the literature is very discouraging.  There was one discouraging thing about the papers.  Nobody went back and really looked at the literature and tried to understand in fact why this was not happening.  It is as though education research or education intervention started in 2002.  It did not.  As Checker reminds us, it did not even start until& well, who knows when it started.  It started before Checker and I were around, so that was a long time ago and if you look at after-school programs, you look at summer programs, you look at pullout tutoring, these are all nil effects unless they are tightly and thoughtfully connected to regular schooling, generally having the same people doing it.  On choice, I look at the literature of Magna schools, charters, vouchers on average no effect, lots of arguments about that.  But when you find a lot of arguments, you find very little effects because you get people arguing on both sides and all depends on the study that they are looking at.  On the third, when breaking up schools, creating charters, reconstitution, state interventions to date if they have all or tiny little effects.  So let s stop messing around.  The toolkit focuses really to confining the three tools or themselves piecemeal and are not be expected to have a significant effect.  Moreover, the focus on the toolkit takes as a given the rest of NCLB, including the continued focus on standard-based reforms.  The emphasis on reading first on early reading, the use of only punitive -- and the accountability system as well as a host of other issues.  So as a beginning, we should step back, review our assumptions, look at the evidence, and so on.  We will look at evidence from two different things.  One is recent achievement trends, which I find have been distorted in many of the discussions today, and the second is evidence about some changes that may make big effects rather than little effects.  Recent achievement trends.  All right, so I m going to look at NAEP subgroups.  Let me look at both the longitudinal and main NAEP.  We will start off by looking at 1994 to 2005.  I used 1994 because it was the beginning of the Clinton administration s new bill, which introduced the standards-based reform.  Just as a point of fact first, some of you have not looked at your history books recently.  Bush 1 started accountability.  If you look at the Bush 1 provision, it is very similar to the current provision.  It does not have quite as punitive a set of requirements on it.  But in Bush 1, they had it in the 94 authorization.  It was there in 2002; they just picked it up and made it a little bit tougher.  So let s think about 94 to 2005 then as being the time of standards-based reform.  And the math gains here are 1 to 2-plus grade levels at fourth and eighth grades.  So we have gone up in the fourth grade more than two grade levels over that period of time in math.  In reading, we are going up about two-thirds of the grade level to a grade level and-a-half.  Just look quickly at the grade level gains for Grade 4 and for age nine, which is the longitudinal for reading and mathematics.  Reading on the left-hand side, you see 15, 15, 15, 19 for African-Americans, and Hispanic Americans is about 10 points is about a grade level.  So grade level and-a-half, grade level and-a-half, grade level and-a-half, and then almost two grade levels; that is on reading.  On Math, you have got 27 points, two-and-a-half grade levels, 12 on longitudinal math.  I m not sure what happened there.  But then 24 and 20, these are substantial changes.  Let s go quickly to a second set of ideas.  I m not going to show a chart on this but here is the basic idea.  Does standards-based reform work as a concept?  We have got data on it between 1994 and 2005.  We also got data from the longitudinal NAEP before 1994.  We got it from 1978 or so up to 1994.  You see, we have two periods of time, right?  One is before standards-based reform, one is after standards-based reform.  Are the slopes the same or are they different?  It is a simple test of whether or not you have an effect of standards-based reform.  Turns out you got a substantial effect in reading and you have got a really marginal effect in mathematics.  In other words, the slopes of gains in reading went up dramatically.  In the period from 1994 to 2005, the slopes did not go up dramatically.  There were gains in math before and after as I showed you, but the slopes did not go up so dramatically as to say you have a really clear picture.  So you have a guarded positive result on standards-based reform; it moved the ball on the achievement scale.  All right, so let s look at achievement before and after NCLB.  You can do the same thing with this, right?  Now, I m using basically the main NAEP, okay?  We can compare achievement slopes from 2000 and 1994 to 2002 with 2002 to 2005; actually it had math.  You have to use 2003 to 2005 because there was not a 2002 math test.  The math slopes again are approximately the same.  So the slope of gains in the first 10 years are about the same as the slope of the gains in the latter three years in NCLB.  Now, what does that mean?  Well, it means that NCLB has not had any effect, right?  I mean it is the same slope as though it just was not there; it did not have to exist.  But let s take a look at the reading slopes.  I have four figures up on the board.  The gains between 1994 and 2002, now these are& both the dates& the dates of the reauthorizations, 1994 was the reauthorization in the Clinton years; 2002 was a date of the actual reauthorization signing of the bill for NCLB.  During that period of time, 1994 to 2002, the Clinton years let s call it African-Americans gained 14 points, Hispanic-Americans gained 13 points in reading skills course.  And during the period 2002 to 2005, it was a gain of one and two points.  The slopes are clearly different.  Look at it in grade 8; here the gains are smaller.  Nine and four for African-American and Hispanic-Americans and it is down between 2002 and 2005, which are the NCLB years.  So you cannot make any real judgments about that.  It is to really look at the effects of NCLB, but what you can say is there is no evidence of any clear improvement at all on achievements course.  I mean, it is unambiguous.  Okay, this just runs through what I just said, math and reading gains.  You should realize that reading gains are also much smaller in eighth grade.  SBR qualified success, no evidence of NCLB effectiveness.  Okay, so one option is stay the course with a few minor corrections; we have heard that before.  Some people argue standards-based reform.  So take the arguments that standards-based reform works is to really to see results.  Therefore, up the ante; tell locals to implement better, get tougher, tougher on sanctions on misuse, et cetera, et cetera, create and require a national test, I guess, because people think that they are going make people work harder.  I do not know what the theory of action there is exactly.  So what should we do?  We should not stay the course.  We should take bold steps but build on strength.  We are going to change the way we think about things.  Most educational policies on the margin -- okay, I ll get there -- throw out the add-ins, throw out the add-on programs; they are just getting on people s way.  The system wants us to stay on the margin, but we should not do it.  So we think about improving the system somehow.  So learning of function of the system, of the content of motivation of time, shoot for big effects.  Big effects, folks, really critical.  And the point here is that in the system idea, I should have in here keep standards-based reform.   Turn around and use some positive incentives.  There are lots of really good ideas that you can use on the incentive side.  Support information-based, continuous improvement on all levels.  I mean, that is basically what good businesses do.  You have to have good information systems and you have to support continuous improvement as you go through.  Keep the teacher qualification to this aggregation.  The district and state reports increases transparency as Alan was arguing and so on.  Remove the add-ons.  That is, language, language, language.  We are forgetting language; we are forgetting vocabulary; we are forgetting the analytic use of language; we are forgetting logic.  We are forgetting all those things that have to do, and that affects every subject; it stops us from reading.  I can tell you stories that are horrendous and we are ignoring language, folks, and until we do it every one of our scores after fourth grade is going to be in the pits.  Motivation and the desire to learn is a really easy one.  We should be insuring that teeth are healthy.  That alone would raise test scores in a substantial way.  Eyes, ears, nose, et cetera.  Increase art and music.  You have to spend time on the motivation, and you have to spend more time, and you have to spend time that is really tightly linked to the school system where I mentioned KIPP and we know how to accelerate learning now.  There is no question about using technology.  We can do things that are just astronomically different than we have before.  We are not doing any of them.  So we want to climb up the box and do not settle for small effects.  Thank you. Frederick M. Hess:  Absolutely, the best graphics of the day.  Diane? Diane Ravitch:  Well, I want, first of all, to thank Rick and Checker for organizing a really terrific conference.  I have to say I learned an incredible amount from reading these papers and I m far better informed about how NCLB has been implemented or not implemented than I was before this week.  So unlike Mike who thought out of the box, I thought inside the box and addressed the question, is the NCLB toolkit working?  And to judge by the papers, the answer I think has to be no.   I think that the bottom line of the NCLB is that the message goes out to the schools across the nation.  Do something, and lots of schools are focusing on low performing kids and are doing something.  But the papers suggest that very few students who are exercising choice.  In the rural areas there are no choices.  In cities, there are few available places and better schools.  There are many more kids who are eligible for choice than there are places in good schools, or there are districts that already have a choice.  As Jane Hannaway said, in Miami are references to other states where there are already choice policies in place, so NCLB did not really add anything there.  And even when the districts make a good effort to advertise choice when they write a good letter explaining it, not many parents are choosing a different school.  Many of them seem to want their kids to go to the nearest school or I thought about the list that I had seen for New York City where I live, and saw some of what I thought were some really good schools on the list.  And it occurred to me that they missed their target by one category and all the kids on the other categories do not want to change because they think it is a terrific school, and it is sort of like the Florida s story where there are eight schools that do not make AYP, and parents whose kids would like the school and are happy are not going to choose another school.  We also learned from the papers that very few kids are taking advantage of the SES, and the numbers only look good in comparison to the choice numbers.  The SES providers are a mixed bag with not much proof that they are superior to the school that the children are attending.  Some of them are fly-by-night operations.  Some of them are hucksters.  Some of them are using the same teachers available on the regular school.  And from the papers we read, I did not see any evidence that SES is working.  There were no numbers that said this is making a difference.  And again, as Jane Hannaway said, if we are going to incorporate SES into a new version of NCLB -- boy, I love all these acronyms -- what we need is a lot of MGT, management, management, because it is chaotic to have dozens and scores of providers that the public schools are responsible for overseeing and paying.  There is not much evidence from the papers that restructuring is working or that it is making any difference.  Not many districts have opted for radical restructuring.  Few of them if any have converted failing schools to charter schools.  When Colorado tried, they will always change so that they cannot do it.  Few of them have dismissed the staff.  Most of them are doing more of what they would have done anyway -- more professional development, more coaches, more mentoring, a new curriculum, better alignment of curriculum and assessment.  In other words, they are responding to the demand do something and they are doing something.  They are doing what they know to do, which is what you would expect and that is really& in most cases, the best you can hope for is that people do what they know how to do and do it better.  The question that I kept asking as I read the papers was how do we know that any of these tools in the toolkit are the right tools?  What reason is there to believe the choice is going to raise achievement?  As Mike said, there is a little evidence that it does.  It has in some places and it has not in other places, but in any event, it is hard to see this is a federal mandate when we have so little evidence that it is a reliable tool.  What reason do we have to believe that private sector tutoring companies are more effective than the regular school when so many of these companies have absolutely no achievement data to show that they are more effective?  What reason do we have to believe that schools will get dramatically better if they are converted to charter status?  Some will, lots of them will not.  Some of them will flounder and be worse in a regular school.  What reason do we have to believe that Congress knows how to fix the nation s schools?  I got to the question of capacity and there was a lot of discussion today about do the districts have the capacity?  Do the state education departments have the capacity?  I can assure you that if neither of them have the capacity, the US Department of Education has even less.  So I do not think that the American people voted to have the Department of Education here in DC running the nation s schools or to have the Congress making decisions that will then be mandated and imposed in the absence of any evidence that they work.  So I began to believe as I was reading through these papers that we might, in fact, be barking up the wrong tree.  And, of course, I kept coming back again and again to the feeling that I have had from the date that NCLB was passed, that a 100 percent proficiency is an absurd goal, unless by proficiency we mean something along the lines of basic, which is just minimal literacy.  But unless we redefine proficiency, it is not going to happen.  It is not going to happen in another lifetime, let alone by 2014.  And then maybe that part of the toolkit that we did not discuss which is accountability may be the piece that is working best, and that is the assessment of the desegregation so that the schools and districts can focus on the lowest performing kids and to try to help them become better than low-performing kids.  But I think that the flow with the assessment is the fact that we do not have accurate information, and any kind of action that districts might want to take and states might want to take requires accurate information.  So I keep coming back to the fact that if you have 50 states, 50 tests, 50 sets of standards, we are not going to get accurate information.  Mike casually talked about this in his paper.  So to begin to have any kind of theory of action, you have to know where you are.  We do not know where we are because we have all of these different proficiency scores, most of which are confusing, chaotic, and meaningless.  But I also ended up feeling, after having rather go through all of this, the state of confusion and thinking, that we know less about how to raise achievement today by policy or by fiat than we did five years ago.  It seems quite mysterious to know how you can change achievement in schools.  So I was very happy to hear Mike s presentation because now he provided all the answers that were eluding me.  One of the things that I feel quite certain about living in New York City is the test prep is not instruction and that is all our system has been doing intensively since we assumed mayoral control.  Test prep, test prep, test prep, which is why our scores have gone up in fourth grade, but they have not gone up in eighth grade.  In fact, they are lower than eighth grade, at least by the NAEP count.  Actually, they have not even gone up in fourth grade if you look at NAEP, so mayoral control has not Morgan, I m sorry to say provided a radical change at least in the NAEP s scores.  So then I m left with the question with which I ll conclude:  What should the proper federal role be?  I think that the focus should be on results.  I believe there should be national standards, certainly in subjects like science and math where there are already international standards.  I think that the value of national test is to provide accurate information about where we are and then research and equity redistributing resources so that there is an equitable -- so that the schools trying to meet their goals, it is not for lack of resources with which to do it.  But I think that the states and the districts should be the ones who figure out how to meet those goals and not the federal government because I do not think that federal government has the capacity to do that.  I do not think that the federal government -- and here I quote Mike Petrilli -- should be the agency that defines and directs the sanctions and the remedies.  Thanks. Frederick M. Hess:  Thank you, Diane.  Kati? Kati Haycock:  So Mike thought out of the box, Diane thought in the box, and I thought between the boxes.  Let me just start actually where Alan Bersin did because maybe it is because we are both Californians.  I m in somewhat of the same place he is.  I mean on the one hand, I feel, as I know many people in this room do, enormous sense of urgency about improving the education that poor kids and kids in this country get.  And other than perhaps Mike Casserly, I probably spend more time in schools serving these kids around the country than most anybody else in this room and you cannot do that and not feel an enormous sense of urgency about the need for change and improvement.  That said, the conversation has changed.  It has changed in hugely fundamental ways, far more people today to actually sort of take responsibility for what students learn than ever before.  And there were sort of fewer excuses, more focus.  Yes, people do crazy things and, yes, they don t always know what is the best thing to do, but there is a sense of felt responsibility.  There is a changed conservation that I think is really important not to underestimate.  Now, I was thinking just the other day, one of our staff members was on the state plan review for Georgia actually back in 94- 95 under the last reauthorization.  And when they did the site visit and talked to the chief and to her deputy and they asked about this, What are you doing to help low-performing schools?   And their answer was, Well, they know who they are and they will call us if they need help.   I mean, no sense of responsibility at all and, well, I do not think there are many chiefs who were quite where John Winn is in a sense of feeling a real sense of responsibility and overwhelming sense of responsibility that is the core of the work; is creating conditions to make that happen.  I think we are a long way away and Georgia is a long way from where it was in 94.  That said, I have some trouble with sort of operating assumption of this conference in perhaps somewhat the same way that Mike does and that is the sort of assumption is that the NCLB toolkit is choiced-up services and restructuring, and the basic idea seems to be that if those things are being used, if they are being used well, and if results are getting better that somehow NCLB is a success.  At least in my head, those three things really are not levers.  They are not tools of change.  They are tools from which change does not work.  And at least my sense is where we really need to focus is on the tools of change, and that is the tools that teachers and principals need in order to make the improvements so we do not have to use these sanctions in the beginning and that is& we know what they are.  We know that teachers want them.  We need rich, coherent curriculum.  People should not have to make that up.  State standards do not get even close to clear enough.  Teachers are teaching in the dark, beating them over the head and telling them to get better is not good enough.  We need good curricula.  We need formative benchmark assessments attached to that.  We need professional development that is ongoing that is attached to that.  We need extended learning time at the school but that is not just classroom-based that has all kinds of youth development base.  We know that stuff works.  We actually have the money in this law to do it but we do not, and we need, of course, teachers who know their stuff and who wanted to teach these kids.  I mean the interesting thing for me is that at some level, there was a hope in Congress that if they kind of sanctions were set up as they were, that states would do these things.  But I think what we heard today is not necessarily that states do not want to do these things, but their capacity is limited, and so they do the things they have to do and those really are the things we talked about in these papers.  I also think that what is important to acknowledge is that they are afraid of some parts of this.  I mean I have had a long running conversation with one state commissioner who will go on to remain nameless here about the importance of curriculum.  And I said, Your teachers want it.   And his response is, Yes, they want it.  But they want it so they can say, We taught it and the kids did not learn it.  So our responsibilities are over.   That is actually not why I think teachers want it.  But we have to get over that.  Core knowledge may not be the only -- I mean it should not be the only one out there.  Teachers should have lots of that kind of stuff.  So again, my question is, how do we put real tools of change in their hands?  I m not so sure the old ones need to go away, but we need to remember those are not the tools of change.  They are the tools when change does not work. Frederick M. Hess:  Thank you, Kati.  Checker? Chester E. Finn:  So I think there are three, crudely speaking, options on the NCLB future table up here.  One of them has been implied, I think, by most of the discussion today; we might call it, Fix the Toolkit.   Second one articulated, most recently by Diane, we might call, Turn It Upside Down and I ll explain it in just a second.  And the third one, addressed by both Mike and Kati, though in somewhat different terminology might be called Try a very different approach that has a lot more to do with instruction curriculum learning.   The likeliest thing to happen in Congress is Fix the Toolkit, because that is always the likeliest, or Try to fix the toolkit.   That is always the likeliest thing to happen.  And that would include some pretty obvious things that have gotten on the table today.  Faster data, clearer communications with parents, wider choices, more information about SES providers, better data overall, closing some loopholes on reconstitution, and things like that.  Those kinds of toolkit take tools to the toolkit.  Fix the toolkits is the likely see scenario.  The Upside down scenario that Diane has put forward and I think I personally subscribe to, though that is not really very important here, is change the federalism expectations, have the national standards and measures, and then basically free up the other players to produce those results in the ways they think are best instead of regulating them with ever finer tools.  And then the third approach, which I m going to have the most trouble articulating because it is sort of only appeared at this conference in the last 23 minutes is let s call it the instructional approach or the curricular approach, which is something I think quite different from the current NCLB toolkit and even quite different from an upside down version of the current NCLB Toolkit.  So I want to try those out as at least the kind of three ways.  We are thinking about this, this big federal law going forward, and I ll be delighted to have my colleagues up here disagree with that formulation of the options. Diane Ravitch:  I want to disagree. Chester E. Finn:  All right.  That will be the next thing to happen. Frederick M. Hess:  Diane? Diane Ravitch:  Well, the reason I will disagree with the formula is that I think that where the action is in change is curriculum instruction, so I m not on the other side.  It is just that the issue is who should do it and I do not think the federal government should do it because the worst thing would be to have Congress decide with the national curriculum should be, or to have Congress decide the right instructional methods, and I just think that we have to say, That is the most important thing and it does not belong at this level.   So if we are talking about reauthorization, I go for the more & however you characterize the approach of establish the standards, certainly in areas like math and science which you might say they are less contentious than other subjects, but as I said before, there are international standards, and free up to the states used the value of these standards and the testing to help people find out how they are doing in some realistic way as the urban district assessment has done for the cities so that they can see that there are places where they really need to focus in order to help these kids.  But I just think that it is not an opposite thing to say that, therefore, I m not on the curriculum instruction side; I just think it is a matter of who should do it, and it is not the Feds. Chester E. Finn:  Well, if you and Mike actually had not interrupted my train of thought, I was going to ask Mike and Kati to sort of explain what is the federal role in the approach that they were advocating.  But they can get& Diane Ravitch:  Was so rudely interrupted. Chester E. Finn:  Before I was so rudely interrupted, but they can get to that as part of correcting my formulation of the& Marshall Smith:  Yes, just a couple of thoughts on it.  I think I would almost blend to, and I think Kati and I may disagree slightly on this, I m much bigger on soft accountability on positive incentives rather than negative incentives on the federal government, providing some resources for states to try out in a serious way that the extended time, for example, or some of the technology, things that can go on or a variety of the other things that I mentioned.  But I actually fear a national curriculum, fear a national test even though as some of you know, Bill Goodling once called it Smith s folly on the House floor, yes, because I was that lead person for the national test in the Clinton administration and I took a lot of heat from a lot of Republicans and a lot of Democrats at that four years.  But my fear is a fear of the District of Columbia and people inside the beltway, and I think that is a fear that is shared when you get outside the beltway.  There is one perspective here, and there is a set of other perspectives on the outside, and I do not think there is a lot of similarity between them in many instances.  So I would have the federal government play a little bit of a softer role but very big on information, very big on incentives for doing what has at least empirically been tested in a variety of time as being effective. Chester E. Finn:  I just want to clarify just on this--  So how would you distinguish this or compare it, say, to the report card model?  So if you want to talk to 94, the mandatory test conversation. Marshall Smith:  I think transparency is critical and we tried some of that in 94 and attempted reauthorization later on in 98- 98.  We have a lot of it in.  But I think that ought to be strengthened; no question about that.  But I think it is more than that, they get resources.  I think using resources to give incentives, I mean, suppose we gave incentives to schools and districts to decrease the dropout rate.  If they actually got money if they move that dropout rate over at what has been the average for the last three years in that school, on that district.  Give them thousand bucks a kid.  I mean our dropout rates are abysmal.  They are just about the worst in the developed world and it is a national problem; it is not just a local interstate problem.  So there may be ways that those kinds of things can be looked at, but they got to be local and state strategies doing it. Kati Haycock:  I actually think that the basic framework of the law needs to remain intact.  I have lived in state for way too long to have plenty of disaggregated data and plenty of transparency and crappy results and no sense of responsibility, so I m definitely not a believer in just sort of this soft information.  I do, however, think that incentives are good idea.  As you know, they are permissible under the law but very few, if any states, have actually used resources to do that.  And that really is the question that is underneath all of these and that is, it is not that there aren t resources in this law to do some of the things that we have talked about, but they are not being used in that way and that is a trick for me.  If you look at Title II, for example, you got a $3 billion in there that is supposed to be about helping teachers to get better on what they do, the vast majority of which is not spent on that at all.  So this is the struggle.  So you put dollars out there and you are relatively permissive about how it gets used, then you know as well as I do it does not get used on the stuff that teachers need in order to get better.  So on the curriculum thing, I m puzzled.  I m quite sure the federal government should not do it, whether federal dollars could support either consortia of states or consortia of states and universities, so we can have five really powerful curriculum models for folks to choose from.  I mean I m not sure the right way to do this.  I know, however, that just handing over a little more money to states and saying, Oh, add this to list of things you could do, that will not work.  So if we feel a sense of urgency about getting teachers and principals the tools they need to avoid these other tools, there has to be something more direct there and I do not know how to do that. Chester E. Finn:  I want to push Mike a little on this very point because I m going to stipulate that everything you put up on the screen as something we know how to do has, in fact, worked somewhere.  But the notion of scaling it on a really substantial level at, let s say, big city level, citywide, or a state level much less a national level, I do not think I have seen that happen.  What is going to make that happen? Marshall Smith:  It did not happen either, and I do not know exactly what would make it happen.  But I do know attention to it by Congress and attention to it by the press across the country, attention to it by people in this room and by the various groups that are represented here could help make it happen, could change our orientation toward language, for example.  I mean we know exactly why the scores begin to plummet after fourth grade.  We know that.  We know it has to do with language.  We know it has to do with lots of these kids have never read a book in their life.  Never read a book.  I mean my wife is starting a charter school and they are starting with sixth grade and kids coming into that school, one of the things they did was they took them out and they had a rope climbing thing, build cohesion.  And they were all sitting around and one of the teachers asked the kids, So what book did you read this summer?   Nobody raised their hand.  This is 50 kids.  And finally, it came to the fact that many of them had never read a book in their life.  They were not expected to in the schools.  So this is crazy.  It affects science, it affects history, it affects math in a very big way.  There is some data which suggest that it is 50 percent of the reason for the kinds of areas that appear now in algebra test.  So these are things that are talked by a combination of thought leaders and people who work in education.  They could change that.  So I agree we should have accountability; no question about that.  What I meant soft approach to it, I do not want that kind of rigidity that has built into it.  I also worry about everybody in the country getting in a totally failing school by the year 2014.  That is exactly what will happen.  So if you are rigid, you invite gaming.  If you invite gaming, you invite the kind of dissatisfaction with the job and in the whole way you are doing business because you are not doing something as worthy of you.  But you have to game because everybody else is gaming.  So we want to figure out a system where the incentives are working for us rather than against us. Frederick M. Hess:  Why do we not actually open this up a little bit?  We can start right at the middle.  Again, do us a favor of identifying yourself by name and affiliation.  Please keep your questions short and actually make them questions. Pam Wright:  Hi.  My name is Pam Wright from Wrights Law.  We represent generally kids with disabilities but a lot are also the people that Ms. Haycock represents.  Talking about reading and language, and the law specifically states that one of the goals was to ensure that children will be proficient in reading by the end of Grade 3, and that states will be given money, and school districts would be given training for the teachers so that the teachers could teach the children to read and be proficient readers, and I do not think that is happening.  So you talked about language and you talked about curriculum.  We do training all over the country.  Frederick M. Hess:  So what is the question? Pam Wright:  How can you enforce these parts of the law that are already in there? Frederick M. Hess:  Okay.  Mike? Marshall Smith:  If you compare US with other countries on reading, by third grade US is one of the top 3 or 4.  We are actually teaching kids how to read but not how to understand what they are reading.  How to use the reading to learn something, they can sound out the words.  So they are meeting the letter of the law in many, many cases.  Not in all, obviously not in all but in many, many cases.  The English language learners, there is just a really lovely study, big book actually.  Diane, August and a bunch of other people put it out.  What it shows is that English language learners do just as well as native language learners in reading up to second grade or third grade.  They learn at the same level of competence and then they just fall apart because they have not gotten any language when they have been taught.  But everybody else falls apart, too; they just fall apart a little bit faster.  And so they go from &   I was just looking at a district in LA where around 50 percent are in the proficient level.  California s proficiencies are not too soft.  By middle school, they are under 20, by high school they are down to less than 10, and they have already lost about lower 50 percent of the kids in the school.  It is tragic and it is largely language. Ed Jones:  Ed Jones, Open History Project.  This may be your last chance for optimism maybe.  My question is has NCLB in any way changed the culture of the teaching profession?  Are we moving forward, not just affecting the state bureaucracies or these local schools, but are we changing teaching as a profession? Frederick M. Hess:  Kati? Diane Ravitch:  I think that it has had negative effects on the teaching profession for the reason that Mike Smith just mentioned.  This notion that everyone is going to be proficient by the year 2014 which everyone simultaneously knows is not going to happen creates a sense of pending doom, and also there is so much gaming of the system and so much dishonesty in the reporting of score, and then the inflating of scores and the excluding of low performing students from test and from other circumstances that I think it creates a fundamental dishonesty that is not good for the profession. Kati Haycock:  Given on what she said, let me respond to that then.  Because I actually feel a little more optimistic based on data, and it is not that I m unaware of the problems that teachers have with this law or with its implementation.  But if you look at what the data are telling us, first, if you look at teacher s satisfaction with the profession, it is actually higher today than it has been in the last 10 years.  If you look at who is entering the profession and I m talking here not just at through traditional routes, but at the caliber of people we are drawing through alternate programs now, especially the mid-career switchers, a fabulous set of folks who are attracted by these challenges.  So I do not think it is quite as all negative as Diane suggested.  I think the picture is more mixed.  You would actually, we are going through a huge transition from, I taught it and if only 20 percent of my kids learn it, that is okay because that is all that was expected.   We had a whole bunch of people who were trained up with that as the expectation.  We are now saying, that is not even close to good.  You got to get from 20 to a 100 and you are going to have people say, I m out of here.   But that is not all bad, right?  And the fact that we are attracting more and more people again, the numbers coming through traditional routes are up too, but the numbers coming from these mid-career switchers who are by and large high-end, more people of color, who are really attracted by this challenge and I think that is really exciting.  The question really is how to build on that by supporting them instead of just the hammer.  Again, I do not think the hammer should go away, but the tools have to be there for these folks to actually succeed. Frederick M. Hess:  Kati, I guess, and I would like to pose this to the whole panel.  In light of the conversation we have today about these toolkit components, are there ways that these can be used more effectively to effectuate the kind of cultural change you are talking about, to effectuate the kinds of broader changes that Mike is advocating?  So it seems to me, one thesis is that if we can get public choice to work and supp-services to work, whatever work means, that that is a good thing.  But it strikes me the way you and Mike have laid out is that is missing the point.  These are supposed to facilitate a broader cultural organizational change in schooling.  Sounds like from the papers we have read and heard today that there is little evidence that much of that is going on, if any.  Are there things that we might do that might be done in reauthorization, that might be done to the Department that would help this existing toolkit push forward on the kinds of changes that you sketched would be helpful? Kati Haycock:  It is an interesting question.  I actually do not think that there has been a clear impact from SES from what I can see.  That whereas some years ago people were just sort of dismissing KIPP, for example, as just a freak, I m now having far more conservations with regular public school teachers who were looking at that stuff saying, How can we do some of those things, too?   So I actually, I mean, I think the question is, how do you support that kind of learning?  The charter world and the public school world tend to be very, very separate worlds, and so the question for me is it is really about learning at all levels, about how can we learn in the public systems from the private schools for that matter or charters.  It is about how can states that have wrestled with district improvement learn from each other.  It is about how districts can learn.  I mean, what the great city schools is doing in terms of the academic audits and the roadmaps they are providing is very powerful, but mid-sized and small school districts have nothing like that.  So I mean it is really about reorienting, I think, around a set of support for learning. Frederick M. Hess:  Checker? Chester E. Finn:  Well, that most obvious toolkit item that points in this direction is in fact the charter style piece of the reconstitution tool.  Every single one of those schools profile on the Sunday Times was a charter school.  And that was a case for reason, which is they needed the freedom that comes with being a charter school in order to be a kind of school they wanted to be because the kinds of hours and changes and stuff, that they end when KIPP announced just last week that they are going to come to Columbus, Ohio.  They said in effect, We can only come as a charter school because that is the only way we will have the personnel freedom and the schedule freedom and the budgetary freedom and so forth and so on to run the kind of school we want to run in Columbus, Ohio.   Now, it may well be that you working with districts can persuade them to evolve their district schools into these kinds of instructional models.  In the meantime, though, in the short run, creating the opportunity for more &   I ll guess I ll call them non-district schools of the good kind.  I mean, there are plenty of crappy charter schools, too.  Non-district schools of the good kind.  I assume the reason that Mike Smith s wife is involved with a charter school in California is because there is greater freedom that way to do things right. Marshall Smith:  We are actually doing a study of KIPP.  There had been a lot of claims that, well, what KIPP does is it selects only the kids that are ready to learn or it eases out kids who are not learning, et cetera, et cetera.  I mean lots of arguments that, oh, they are not really making the kinds of gains that many of them are making.  And so we are looking at five KIPP academies in the Bay Area and they are terrific and they are so far, as far as we can tell, they are not doing those things.  They have the same kind of turnover rate as similar schools.  They are not pushing the kids out.  Some kids leave, but kids in these situations are moving all the time and they move for lots of reasons, and these reasons do not seem to be because they are being pushed out.  So KIPP is probably not coming back to California because it cannot afford to be in California.  We do not have enough money for it, which is really too bad.  They do two or three things.  They do not just do time, they have a culture that is very strong and very supportive of each other.  It is really very, very clear stuff.  They give a lot and they give lots of autonomy to the principal who is very well trained.  Don Fisher spends maybe $48,000 on every principal and trains them at either Berkeley or at Stanford for six months, and so they come in knowing heck of a lot about what they are doing.  I m on the new school s investment board.  Every charter school, group of schools, cluster of schools that we have looked at that is effective extends time by over 30 percent. Toni Cortese:  I m Toni Cortese from the American Federation of Teachers.  One of the things that I really find missing in the discussion here today is, as teachers and students are chasing after this ever-raising standard every year and we know we are going to get more and more schools again on the list, there is a deeper problem that I have under that, and what is in the toolkit to assure that we are going to end up with educated children?  And the reason I bring that up is I have teachers from around the country saying that they are spending double periods on reading and on math and when they can fit in some social studies and they can fit in some science, and maybe at the very tail end music and art, they try and do that.  So while we are looking at this toolkit, what could be in the toolkit or what could be changed about it so that students would get the thing that we owe them the most, which is the greatest equity, the ability to learn from a rich curriculum? Chester E. Finn:  Before anybody else answers that, on December 12 down the street is an all-day Fordham-sponsored conference on that very topic where Toni Cortese is going to be a panelist, Diane Ravitch is the kick-off speaker, and Kati is on at least one panel, and it is going to be one heck of a discussion all day on that very point. Kati Haycock:  First of all, I do not think there is anything in the toolkit as it exist that would promote what you describe, which is what we would consider a liberally educated person.  I went to public schools in Houston, Texas and we had art and music and science and history and social studies, and all of those things are just part of what you did in school, and it was not an add-on; it was part of the core curriculum.  It was not a school for the gifted.  It was this totally mainstream school for everybody.  And I fear that what has been happening over these past few years is, you know, the term is the narrowing of the curriculum, this relentless focus on test prep.  And in fact we are giving kids language but not the content and the context to make sense of the world, and the reason that they arrive in middle school unable to do well on the test is because they do not have any of the background knowledge.  And this is why I m a very strong supporter of E. D. Hirsch s Core Knowledge Program because it has all of that.  It starts in kindergarten or pre-kindergarten and extends all the way through eighth grade and probably through high school.  But it is sort of if you are an educator, you should believe that all of these things are part of what is education.  It should be part of your vision of what education is and we have a federal law that does not include these things. Gary Ratner:  Gary Ratner, Citizens for Effective Schools.  I would like to follow up on what Kati Haycock and Mike Smith referred to, particularly Kati started it.  You said we need to focus on the tools of change more so than the tools where the change does not work.  There seems to be a substantial agreement among the panel that the keys here are challenging and engaging curriculum, high quality instruction, and from Alan Bersin I would add parental support for high-level learning at home.  So here is my question.  We got the author of reauthorization coming up.  Why should we not shift to the focus of the whole accountability system in the law so that the schools and the districts and the states are held accountable for implementing the systemic changes that we know how to do to improve teacher and administrator preparation, to do staff development with peer collaboration and mentoring and those things, and the adult literacy and the parenting skills that we know a lot of these families of Title I kids desperately need if they are going to support their work?  We could keep accountability, but you will shift the focus from sanctions and testing in order to make the changes that really matter. Kati Haycock:  I mean I think the short answer, Gary, is that the last 12 times the federal government has done that, it has been an absolute mess.  And I think if we have learned anything from that, it is when the feds get into prescribing processes, that things go from worse to worst.  And again, there is nothing in the law that runs counter to any things we have talked about here and, in fact, the theory was that if you set up the accountability around learning, and you back off and give people flexibility of how they use the resources, that the things we have talked about today would happen.  Okay, so that was the theory.  Obviously, those things are not happening today.  But where the feds come in on that is a really, really tricky problem.  There is every opportunity to make it worse, not better.  A lot of times people have a hard time with the order of things here because NCLB feels sort of backwards, right?  People think, well, what you do, you start with standards, you add professional development, you do the stuff with teachers, and on, and on, and on, and eventually you have sanctions and, oh, the people will do it that way.  The problem is if you do it that way they never get there, right?  So the only way people ever, ever get serious about this stuff is by focusing on results.  And everything in my experience has taught me that and that government ought not to get into the details of these other processes.  So the question is though how to make it work better, and that is what we have to wrestle. Frederick M. Hess:  Mike? Marshall Smith:  Yes.  I agree with Kati on that.  You do not want to move away from focus on outcomes.  And you know, the problem with focusing on inputs at all, apart from the most generic level, is that then the Department goes out and regulates on that, and then each state regulates on the regulations, then big cities regulate on the regulations on the regulations.  And by the time you are down there, by the time you are at that level, you are at this very, very narrow interpretation of the law that has none of the spirit behind  Kati s evidencing and you have lost it all, you got this narrow curriculum, and so on.  In fact, they had one other thing on& we have not talked much about it.  I was a little surprised Alan did not talk about it.  I think there has got to be sea change in a lot of the districts.  Districts should be in the business as supporting the learning of kids.  They should not think of the principals and teachers as reporting to them and is doing whatever they are saying.  It should be just totally turn around.  You invert the pyramid in exactly the way that pyramids inverted in the Toyota plants and throughout the world now.  It used to be just Japan but now they are all over the world.  They inverted the pyramid.  The people on the line are calling the shots there.  The people who were suggesting the changes and in those changes are pulled together and mediated by a group a little bit above them but made up of the teachers themselves.  So there is a different way of looking at the world, I think, and we are not doing it.  You got to worry about HR and many districts do not worry about HR.  You got to have good budget systems.  Now, you got to have a damn good information system and you have to learn how to interpret information and use it effectively.  These are not easy tasks to do, as something like Alan or lots of others can attest.  The last thing is that school boards love to turn over superintendents.  You get elected to a school board, you got to do something so let s turn over the superintendents.  Superintendents that have& the places that have been effective, at least certainly in California, have all had the superintendents in there for six years or more and there are some very effective places in California at this point with that track record.  And yet on Boston with Tom being there for 10 years, you got this kind of track record of people who are really, really dedicated to staying in and making systemic change over a long period time.  That is the only way you are going to get there. Frederick M. Hess:  Do we have time for one more question? Kate Neville:  Hi.  I m Kate Neville.  I would be curious to hear how the panel responds to critics, and sometimes not critics but concerns expressed by parents of public school children that No Child Left Behind is different in terms of federal policy and legislation that affects public schools, that it affects every single child?  It is not special ed; it is not Title I.  Everybody in school is affected.  And what about the kids who can pass the test before they walk in the door?  What are the long-term ramifications of devoting resources almost exclusively to those who need the most help, and how do you help everybody maximize their learning potential and their contribution, particularly in light of the increasing global economy and our comparisons internationally are falling& ? Frederick M. Hess:  All right, okay.  And actually let me bend the question a bit for you.  Jessica, I think it could be a nice closing note.  In light of this conversation today, given the No Child Left Behind has this kind of broad gauge application to children and the vast majority of the nation s schools, how do we come up with remedies?  How do we make No Child Left Behind work in a manner that is focused enough, that is nuanced enough to tackle schools and tackle problems without creating the turmoil and the discontent and the backlash that emerges in some communities? Diane Ravitch:  My suggestion was the federal government should focus on the results, should establish national standards in key areas, should have a national test, and should have been sent lots of incentives for districts and states to improve instruction, and to develop powerful curricula key to the standards, and to leave the states and districts free to reach those benchmarks and focus on raising up the bottom, but also focus on helping the kids who need to aspire to even higher goals. Kati Haycock:  I think of two quick comments.  Number one, I think it is fair to say that we have a high-end achievement problem in this country in that too few of our kids including too few of our poor kids and kids of color are getting to high levels of achievement.  What I think is also, though, important to say is there is no evidence or whatsoever that in the post-NCLB era, high-end kids are somehow doing worse.  Their growth on the NWEA studies and others is exactly the same as it was in the pre-NCLB years.  So any sense of this huge priority that poor kids are supposedly getting, if only that were true, is pulling down high-end kids is simply borne out by the evidence. Frederick M. Hess:  Mike, any closing remarks? Marshall Smith:  Well, I agree with both Diane and Kati on this.  I spent a lot of time in Asia and I m going to be in Vietnam Saturday or Sunday.  In China and India, they are all talking about the flat earth, and they are all talking about creativity, and they are all talking about how we are going to catch up with the United States in terms of the way that the kids learn.  And they smile and say, No, I understand you are not learning that way anymore.   And I say, No, of course, that is not true.   But there are lots and lots of concerns on this high-end scale, and it is reasonably clear that we have a hell of a competitor battle on our hands over the next two or three decades. Frederick M. Hess:  Checker? Chester E. Finn:  If you took the model Diane articulated which I agree with and add to it, a value-added or growth model approach to analyzing the assessment data, then everybody s growth gets counted, not just the growth of those who are below some arbitrary line.  And so if you are already over that arbitrary line but you need to keep growing, and I do not know anyone who does not, your growth also gets factored into whatever judgment is made about the schools you attend. Frederick M. Hess:  With that, first, I would like to thank Rosemary Kendrick and Juliet Squire for the work that made today possible.  I would also like to thank the Fordham folks for helping out with this.  Second, I would like to thank all the authors and discussants for what I think was really an illuminating and interesting day.  Finally, I would like to thank all of you for joining us.  You will find a reception set up in the lobby.  Please feel free to have a drink and discuss the questions.  I look forward to seeing you all soon. [End of transcript]  </body></html>