<html><body><A class=eResources href="http://www.aei.org/events/eventID.684/event_detail.asp">Leaving No Child Behind: Options for Kids in Failing Schools</A> January 15-16, 2004 Unedited transcript prepared from a tape recording <TABLE width="100%" border=0> <TBODY> <TR> <TD vAlign=top align=left width="100%" colSpan=3>Thursday, January 15, 2004</TD></TR> <TR> <TD vAlign=top align=left width="16%"> 11:45 a.m.</TD> <TD vAlign=top align=left width="84%" colSpan=2> Registration</TD></TR> <TR> <TD vAlign=top align=left width="16%"> 12:15 p.m.</TD> <TD vAlign=top align=left width="22%">Welcome:</TD> <TD vAlign=top align=left width="62%">Frederick M. Hess, AEI</TD></TR> <TR> <TD vAlign=top align=left width="16%">12:30</TD> <TD vAlign=top align=left width="84%" colSpan=2>Developments on the Federal Level</TD></TR> <TR> <TD vAlign=top align=left width="16%">  </TD> <TD vAlign=top align=left width="22%"> Presenters:</TD> <TD vAlign=top align=left width="62%">Richard Lee Colvin, Columbia University</TD></TR> <TR> <TD vAlign=top align=left width="16%">  </TD> <TD vAlign=top align=left width="22%">  </TD> <TD vAlign=top align=left width="62%"> Siobhan Gorman, National Journal</TD></TR> <TR> <TD vAlign=top align=left width="16%">  </TD> <TD vAlign=top align=left width="22%"> Discussants:</TD> <TD vAlign=top align=left width="62%"> Michael Cohen, Achieve</TD></TR> <TR> <TD vAlign=top align=left width="16%">  </TD> <TD vAlign=top align=left width="22%">  </TD> <TD vAlign=top align=left width="62%"> Lisa Graham Keegan, CEO, Education Leaders Council</TD></TR> <TR> <TD vAlign=top align=left width="16%">  </TD> <TD vAlign=top align=left width="22%">  </TD> <TD vAlign=top align=left width="62%"> Nina Rees, Office of Innovation and Improvement, U.S. Department of Education</TD></TR> <TR> <TD vAlign=top align=left width="16%"> 2:00</TD> <TD vAlign=top align=left width="22%">Break</TD> <TD vAlign=top align=left width="62%">  </TD></TR> <TR> <TD vAlign=top align=left width="16%">2:15</TD> <TD vAlign=top align=left width="84%" colSpan=2>The State of State Implementation</TD></TR> <TR> <TD vAlign=top align=left width="16%">  </TD> <TD vAlign=top align=left width="22%"> Presenters:</TD> <TD vAlign=top align=left width="62%">Jane Hannaway, Urban Institute</TD></TR> <TR> <TD vAlign=top align=left width="16%">  </TD> <TD vAlign=top align=left width="22%">  </TD> <TD vAlign=top align=left width="62%"> Robert Maranto, Villanova University</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Alex Medler, University of Colorado</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">David Plank, Michigan State University</TD></TR> <TR> <TD vAlign=top align=left width="16%">4:00</TD> <TD vAlign=top align=left width="22%">Discussants:</TD> <TD vAlign=top align=left width="62%"> Mitchell Chester, Ohio assistant superintendent for policy development</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Jeff Cohen, Sylvan Education Solutions </TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Rep. Jane Cunningham, Missouri House of Representatives Tom Houlihan, executive director, Council of Chief State School Officers</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">John Stevens, Texas Business and Education Coalition</TD></TR> <TR> <TD vAlign=top align=left width="16%">5:30</TD> <TD vAlign=top align=left width="22%">Reception</TD> <TD vAlign=top align=left width="62%"> </TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%"> </TD></TR> <TR> <TD vAlign=top align=left width="100%" colSpan=3>Friday, January 16, 2004</TD></TR> <TR> <TD vAlign=top align=left width="16%">8:00 a.m.</TD> <TD vAlign=top align=left width="22%">Breakfast</TD> <TD vAlign=top align=left width="62%"> </TD></TR> <TR> <TD vAlign=top align=left width="16%">8:30</TD> <TD vAlign=top align=left width="84%" colSpan=2>District-Level Developments</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%">Presenters:</TD> <TD vAlign=top align=left width="62%">Julian Betts, University of California, San Diego</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Michael Casserly, Council of the Great City Schools</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">William Howell, Harvard University</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Douglas Reed, Georgetown University</TD></TR> <TR> <TD vAlign=top align=left width="16%">10:15</TD> <TD vAlign=top align=left width="22%">Discussants:</TD> <TD vAlign=top align=left width="62%">Keisha Hegamin, Black Alliance for Educational Options</TD></TR> <TR> <TD vAlign=top align=left width="16%"></TD> <TD vAlign=top align=left width="22%"></TD> <TD vAlign=top align=left width="62%"> John Liechty, associate superintendent for the Los Angeles Unified School District</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Gail Littlejohn, president, Dayton Public School Board</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Joe Williams, New York Daily News</TD></TR> <TR> <TD vAlign=top align=left width="16%">11:45</TD> <TD vAlign=top align=left width="22%">Luncheon</TD> <TD vAlign=top align=left width="62%"> </TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%">Conclusion:</TD> <TD vAlign=top align=left width="62%">Frederick M. Hess, AEI</TD></TR> <TR> <TD vAlign=top align=left width="16%"> </TD> <TD vAlign=top align=left width="22%"> </TD> <TD vAlign=top align=left width="62%">Chester E. Finn Jr., Thomas B. Fordham Institute</TD></TR> <TR> <TD vAlign=top align=left width="16%"> 12:45 p.m.</TD> <TD vAlign=top align=left width="84%" colSpan=2> Adjournment</TD></TR></TBODY></TABLE> Proceedings: MR. HESS: Good afternoon.  I'm Rick Hess, Director of Education Policy Studies at the American Enterprise Institute.  I'd like to welcome you today to AEI, the home of common-sense school reform--to this conference on Leaving no Child Behind: Options for Kids in Failing Schools. This conference is being hosted in collaboration with the Fordham Foundation, and has been organized and planned in collaboration with Checker Finn, president of Fordham. The No Child Left Behind Act, passed by Congress in December 2001, and signed into law by President Bush in January 2002, is the Bush Administration's signature domestic policy initiative.  It has occasioned extensive discussion, and has been at the center of national education policy debates for the past two years. The Act rests, really, on three pillars: the first are its wildly discussed accountability provisions; particularly the requirements that states make sure they test all students in grades three to eight, and student at some level in the course of high school, with a particular emphasis on literacy and numeracy. The second important element of No Child Left Behind--one we discussed here at a conference in October--is the Qualified Teacher mandate; the requirement that all states have a qualified teacher in every classroom by 2006. A third element of No Child Left Behind--that, at the time of passage, was heralded, and is a central part of it, but has received much less attention--are the remedies for children and schools that fail to make adequate yearly progress--AYP.  The law envisions a cascade of options for these children, and remedies for the schools that are educating these children that are at the center of the implementation efforts in states and districts, but have not received nearly the attention or notice of the testing provisions, or the qualified teacher provisions. Let me give--for the general audience--just a couple of background facts about No Child Left Behind. For 2002-2003, 36 states today have identified 23,000 schools as having failed to make adequate yearly progress; 5,200 schools have been identified as failing to make adequate yearly progress--AYP--- for two years running.  Students in these schools are now eligible for some of the remedial measures envisioned in the law. How big is the scope of what we're talking?  In Fiscal 2003, the Federal government is spending about $24 billion--about $24 billion on programs envisioned under No Child Left Behind. As you can see--flip it--as you can see from these recent headlines, No Child Left Behind has been drawing a lot of attention of late--in the education world, and more widely, especially on the campaign trail.  For instance, in an NPR interview--an NPR debate--Democratic Presidential candidate John Edwards identified his vote for No Child Left Behind as the one mistake he most wishes he could have back.  Democratic Presidential front-runner Howard Dean has been somewhat vociferous in his critiques of No Child Left Behind. It is likely that the various elements of No Child Left Behind--not just the testing provisions, but the entire array of elements of No Child Left Behind--are going to be central to the coming year's national debate over domestic policy and in the Presidential election. While immense attention has been paid to the accountability and testing provision and to their implementation--and especially to their cost--and while some attention has been paid to the qualified teacher challenge, little or no attention has been paid to the question of remedies for children in schools that fail to perform adequately in No Child Left Behind.  For instance, while groups like the Education Commission of the States, Education Week and the Education Trust have done fantastic work of tracking the implementation and design of testing measures, they have not yet made similar efforts to track the implementation of remedies for children in schools that are failing to make adequate yearly progress. On the one hand you can't do everything, on the other hand, it's important that we start to think about this other piece of the puzzle. The remedies included particularly--the remedies envisioned by Congress when they created this law--there were four of them--they particularly encountered, in the first two years, public school choice, and what Congress called "supplemental service provisions."  What these things mean in practice is that students in schools that consistently fail to make AYP--adequate yearly progress--are eligible for two things.  If their school fails two years running to make AYP, the students are eligible for the option to move to a better-performing public school.  If their school fails to make AYP three years running, they're eligible for tutoring services after school, provided with Federal monies to support disadvantaged students.  These remedies are both supposed to be the hammer that compels schools and districts to improve, and to ensure that children are able to escape demonstrably ineffective schools. Given that charge, it's essential that we know how children are being served, how these programs are being implemented, and what steps will render these programs more effective. Let's overview, for a second, the cascade of remedies, just so we're all clear on what we're talking about.  I know this can be somewhat technical stuff, and it's not something that we all talk about every day. If schools fail to make adequate yearly progress--which means for one of their identified sub-groups, by grade level, by race, by gender--if they fail in any of these identified groups to have all students make the state benchmarks two years in a row, students in that school become eligible for public school choice.  The district is required--it's mandated, I guess--to offer them choices in other schools that are meeting adequate yearly progress benchmarks. If a school fails to make AYP three years in a row, the district is required to provide students in that school with the option of enrolling in supplemental service programs.  These are after-school tutoring programs that could be provided either by the district, or by an outside provider, that uses Federal money to provide, essentially, free tutoring for the students. In the fourth and fifth years, the cascade of remedies no longer deal directly with the student, but start to become a matter between the district and the school.  For that reason, we will not be addressing them in this particular conference.  In the fourth year, schools are required to undertake corrective actions, and in the fifth years, schools become eligible for restructuring. Go ahead and switch. Two years after the passage of No Child Left Behind, it's clearly still too early to gauge how the options of No Child Left Behind mandates for students in troubled schools are affecting student learning.  However, it's not too early to see whether supplemental services and public school choice are being implemented in sensible ways, or whether the machinery is working as Congress intended. A year ago, when we were discussing this affair, Checker and I agreed that one of our hopes was that an effort like this would permit us an opportunity to talk about choice-based reform in a way that side steps much of the vitriol and many of the exaggerated claims that typically characterize the school choice debate. When NCLB was passed, it was by sweeping bipartisan majorities--as Siobhan Gorman will talk about in the first session.  It was Senator Joe Lieberman, a Democratic Presidential nominee, who came up with the notion of supplemental services.  Public school choice has received bipartisan support for the past decade.  Our hope is that this becomes a conversation about how to make these options work for children, rather than one more debate about whether or not choice is morally desirable. The conference, therefore, focuses on the pragmatic questions of making the NCLB remedies work, rather than on the more sweeping debates over school choice.  In particular, I'm hoping that we will keep in mind four particular questions as we engage in this conversation in the next two days. First, how is implementation proceeding? What's going on out there? Second, what appear to be the sticking points at the Federal, state or district level? Third, what are promising innovations that we're seeing--again, at the Federal, state or district level? And, finally, what changes to policy or practice would help ensure that public choice and supplemental services work the way they were intended to? The conference will unfold in three parts, from the broader to the more narrow.  We'll start at the federal level--the first panel--considering the central challenge of supplemental services, and the nature of public school choice at a national level, through the national lens. The second and third panels today will focus on exploring and analyzing developments at the state level.  First, researchers will discuss their findings on what has been transpiring at the state level.  And then the third panel of the day: participants and practitioners will react to the findings and share their insights and experiences. Tomorrow, the first panel will again present new research on developments a the district level, and the second panel will provide practical conversation and analysis of those developments. In the closing session tomorrow, Checker and I will jointly lead a discussion of the research findings and what the implications are for policy and for practice. Copies of the 10 research papers that are being presented, as well as in press, an 11th paper, analyzing parental awareness of No Child Left Behind, are all available outside in the lobby. We hope that you will help yourself to a copy of the paper. Selected papers are also available on the AEI website at <A target=_blank href="http://www.aei.org/" target=_blank>www.aei.org</A>.  And, for those of you who prefer a more manageable version of this collection, Palgrave McMillan will be publishing the collected papers as a volume in August, and all of the papers in revised form will be available at that point. Panels will run for 90 minutes, to be followed by 15-minute breaks.  Speakers will speak for about 10 to 15 minutes, with the final 20 to 25 minutes of each session are reserved for questions from the audience. During the Q&A, when you're recognized by the chair--whether it's myself or Checker--please be kind enough to use the microphone and to begin by telling us your name and affiliation.  Please be sure, given that this is D.C., the questions are actually questions--[Laughter.] --out of kindness to all of us. In the case of those questions that are actually speeches, that will eventually be followed by a question mark, the chair will reserve the right to kindly move on to somebody with an actual question. [Laughter.] Please keep this in mind. Before we get started, I'd like to thank a couple of folks who did all the hard work of making this thing come together--in particular, Emily Kleuver and Edward Kelly of AEI.  I'd like to thank Derrick Menschu of the Fordham Foundation for all the work they did in putting this thing together.  And I'd particularly thank the generosity of the folks at the Annie Casey and the Walton Family foundations, as well as others for their support and funding--both the research and this conference. With that, we're going to take about a two-minute break.  We're going to summon the first panel up here--and I'm delighted that you can all join us today, and I'm looking forward to getting started. Thank you. [Off the record.] MR. HESS: Okay.  That didn't take long. I'd like to introduce our first panel of the day, which is going to focus on developments and supplemental services and public choice at the Federal level. We have two papers being presented.  One is by Richard Lee Colvin, a director of the Hechinger Institute at Columbia Teachers College at Columbia University.  The Hechinger Institute--for those of you are not familiar with it--is a premiere organization in the nation that helps media and the journalists who are covering education on a daily basis learn about the issues and introduces them to the most recent scholarship.  Richard is one of the nation's leading education reporters. Over to his right, at the end of the table, you see Siobhan Gorman of the National Journal.  Siobhan's paper is on supplemental services, particularly focusing, on the supply side of the supplemental services market: how are we finding enough providers of tutoring to make sure that children who are eligible are receiving the tutoring services they need? We are fortunate to have with us three dynamite discussants on this panel.  First, next to Siobhan you'll see Lisa Keegan, CEO of the Education Leaders Council, former State Superintendent from Arizona.  Lisa was a party to most of these conversations during the time the legislation was being drafted, and has been working with state superintendents throughout in helping them find ways to put this stuff into effect. Closest to me you see Michael Cohen.  Mike is president of Achieve, organization devoted to helping states and districts design standards that are going to ensure that all students are prepared at the highest levels of performance.  Mike previously served in a couple of roles in the Clinton Department of Education, and was party to the Straight A's Plan and various other elements which wound up getting read, in many ways, into the drafting of No Child Left Behind. And Nina Rees, to Mike's right, is Assistant Secretary of Education, head of the Office of Innovation and Improvement with the U.S. Department of Education. Each of the Speakers is going to speak for about 10 to 14 minutes.  I'd like to ask that you please hold all questions until the conclusion, at which point we will open it up for questions and response. We're going to go in order that they're on your program, so we're going to start with Richard. Developments at the Federal Level MR. COLVIN: Good afternoon.  I want to thank Rick Hess and Checker Finn for asking me to be a presenter.  I'm not a researcher, I'm not an academic.  I do head this institute that works with journalists, and so I guess in that sense, I'm an educator.  But I still regard myself primarily as a journalist. And so when I took on this assignment, I took a journalistic eye to it, and I tried to look across the country and just stitch together as many different things as I could see out there, and to figure out if there was any conclusions that I could draw from them. Since everybody else on the program knows a whole lot more about school choice than I do, it's rather intimidating to talk about these things.  And it's particularly intimidating to have such folks as Mike Cohen and Nine Rees and Lisa Graham Keegan to comment on whether I got anything right.  So I'm anxious to find out whether I did. You know, if you recall, leading up to No Child Left Behind, the school choice aspects of it got a lot of attention.  And I was still a reporter at the Los Angeles Times in December of 2001 when there was a report that was put out by the House Education Committee--Education and the Workforce Committee--that triumphantly said that with the passage of this new law, 6,552 public schools, their students would immediately have access to school choice. And the language of it was rather interesting, because it was a very well written piece, and it was the kind of thing that a journalist jumps out.  It said, these children who are currently trapped in chronically failing schools, they'll be freed to go out and find better educational options. The report went on to say that the No Child Left Behind provisions would pave the way for other Federal efforts to expand choice.  And the examples it cited are interesting in retrospect, because it said that a voucher programs such as in Milwaukee and Cleveland and Florida would get a new burst of energy from this law.  And--although, of course, it did say that this was a public school choice, but it would pave the way for further Federal efforts to promote private school choice. The report also touted education saving accounts, and allowing tax-free withdrawals for educational expenses.  And it gave a passing mention to the Federal support for the start-up costs of charter schools. Now, when I took on this assignment, I went back to that piece, and it was very interesting, because what it strikes me now, in retrospect, is all the wide range of types of school choice that existed then and have continued to grow and flourish since then that are not mentioned, as if they are in some way invalid choices; that they're not real choices. It did not mention, for example, that there are 17 states where school districts are required to set up some system of intra-district choice, in which parents can choose from among the schools that are in their local school district.  Nine other states, those policies are voluntary. It's not easy to get a handle on how many students take advantage of that, but it's in the millions; in the range of four to five million kids are estimated to be taking advantage of that option already. Just in my new home city of New York City, K-8, 66,000 kids are taking advantage of various intra-district choices to attend schools that are not in their neighborhood.  And, by contrast, although 220,000 students in the first year of No Child Left Behind were eligible to transfer to other schools under its provisions, only 1,500 actually did so. The national data from NCES show these big patterns and this growth; there's 93-99--many of you are probably familiar with this data--there was a 46 percent increase in the number of students who were exercising public school choice; 4.7 million attended a public school they had chosen in 1993.  Six years later, that figure was 6.85 million. Percentage-wise, this represents a change from 11 percent of all students attending a chosen public school, to 14.5 percent.  And, interestingly enough, given the fact that we often say that it's most advantaged students in our society and our communities who exercise school choice, this trend towards public school choice is accelerating more quickly for poor kids, and more quickly for African-American kids than it is for White kids and middle-class kids. Interestingly enough, we often think of school choice as an urban phenomenon.  In rural America, the percentage of students who are exercising public school choice, from '93 to '99, expanded from under 7 percent to well over 10 percent of the students. So, it's not just intra-district choice, as I've mentioned.  It's charter schools--which I consider to be a public school choice; magnet schools--the kind of controlled choice programs that offer a little bit more limited choice in order to achieve other kinds of goals--integration goals, or--either economic integration or racial integration--such as just are being launched now in Hillsboro in Penales counties in Florida, and have existed elsewhere prior to that; massive increase, dramatic increase in the number of dual enrollment programs, which allow high school students to take college classes, either at their own--either with college professors coming to their schools to offer those classes, or actually going to the community college campus and taking classes there for credit, for both high school and college.  It seems to me that that's a real option. Virtual schools--by the year 2004-2005, it's estimated that a million students will be taking classes virtually via computer.  That did not exist even a few years ago. As you know, there's a huge private investment that's underway in creating new kinds of schools.  The early college schools-- the "middle-college schools" some people call them; the small schools that the Gates folks and the Carnegie Corporation and the Open Society Institute and others are funding.  All of those options in schooling are expanding rapidly. I just want to just mention--you know, take one state, for example--Michigan.  Not that many years ago, John Engler said there's an education monopoly in this state, and monopolies don't work.  Well, it hasn't been too long since he made that statement.  There are now 70,000 students who are attending charter schools.  There's another 35,000 students who are attending public schools outside their home districts--a figure that has grown 45 percent per year since it was created in the late 1990s.  There are thousands of students in Michigan enrolled in college classes that are alternative high school classes, as I mentioned; 7,000 kids are filling virtual seats in the state's virtual high school. Yet, as I point out in my paper--I'm a blowhard, I guess--[Laughter.] --Michigan, just to take an example, is not a free-wheeling education market.  In the year 2000, an effort to offer vouchers--private school vouchers to kids who attended low-performing schools was defeated by the voters.  There's been tremendous resistence to further expansion of the charter program; in fact, there was a philanthropist outside of Detroit who offered to pay for 15 new high schools in the City of Detroit, and that was defeated by the education establishment. There are many school districts in Michigan that resist allowing students to transfer in.  So it's a mixture. Rather than--I'm not going to talk about all the other examples of choice.  Suffice it to say that there's a lot of choice going on out there.  There's also great evidence that there's far more demand for choice than already exists.  We know that there's waiting lists for charter schools, there's waiting lists for magnet schools.  We know, for example, that 9,200 students in the state of Minnesota asked to transfer to other public school districts; half were able to do so.  We know that in the city of Chicago, 132,000 students applied for their magnet programs; 24,000 got in. I could go on.  Three's clearly not sufficient supply for the demand for choice.. So what do we do about that?  Well, some places are actually responding.  The city of New York, they're trying to create 200 new schools.  They also created this new system of matching students, so that they could try to maximize the number of students who would get into the high schools that they selected.  That's going to do some good.  But Chancellor Klein said frankly: "Look.  My problem is I don't have enough schools.  I don't have enough good schools."  It's a chronic problem.  He doesn't have enough good schools. I talked to the person who heads the controlled choice program in Cambridge, he said that, you know, the bottom line is you've got to have good schools, and you've got to have a wide range of programs to choose from.  Because if some of your programs are good and others aren't, and parents don't get into the one that they want to get into, they're going to be unhappy.  So there is a supply problem. So how do we create more good schools?  You know, of course, if school districts were businesses, they would see the models that were popular, and they would try to emulate them.  My experience as a parent with choice was in the City of Pasadena, where I moved from--an open enrollment district.  You get to go to any school you want. However, how many of those schools were ones that you really wanted to go to?  Well, there were a few magnet schools in the district; one was so popular that people would line up three days in advance.  And every year there was a story in the newspaper saying, you know, the campers have come; the tents are up.  People want to get into this school.  And I kept thinking, you know, as a parent--I actually had looked at that school and didn't want to go there.  But I just thought, well, why wouldn't you try to create more of those schools?  But, in fact, the way the district dealt with it was: these are some kind of, you know, subset of parents who have special needs.  They're a special interest group.  They're affluent.  They're middle class.  Why do we need to serve those?  We need to serve other kinds who have greater needs. I didn't ever understand that attitude.  I mean, if--and so what did parents do in Pasadena?  They would go to private schools instead, if they didn't get into one magnet school.  Pasadena also did some other weird things, like the deadline, when they told you that you would get into your school, came well after the deadline for private schools.  And so that didn't really make a lot of sense. I'm just going to wrap up.  So what are some of the lessons of this?  I think that, as No Child Left Behind does, you have to invest in transportation.  That's a big issue for folks.  You have to invest in school supply.  There's many Federal programs that supplies the money for that. But let me leave you with just one thought.  You know, when kids go to college, they have many choices--right?  They have private, public, selective, non-selective, so on and so forth.  And the public schools employ counselors to help kids make good choices.  And there's no bias as to which choice; it's just kind of matching kids.  And what do affluent parents do?  They hire a person to counsel them on how to get into the school that they want to get into. Well, with all of this mixture of many, many choices overlaid on one another in many districts, it seems to me that the only way that parents are going to be able to sort their way through that is through some help.  And I think that we ought to emulate the system by which we help kids make good choices to get into college, and help them make good choices within all the choices that are available out there.  That requires a big investment. But it's the only way in which we're going to have systems of choice that achieve not only the individual aims of the best possible education, but also the larger societal goals of having all kids have access to good education. MR. HESS: Thank you, Richard. Next, we'll hear from Siobhan Gorman of the National Journal. Presentation by Siobhan Gorman MS. GORMAN:   With that overview, I'll take you to one of the newest members of the school-choice club.  It's a politically brand of school vouchers, in the form of after-school tutoring.  And Rich sort of highlighted that in the beginning of his talk as well. In the No Child Left Behind vernacular, this program is called "supplemental educational services."  And with its largely free-market approach, this program is probably the most aggressive school choice effort we've seen yet, at least at the Federal level. But the supplemental services program is also the product of enormous political compromise and that appears, at some level, to be hindering its success at this point--sort of from the implementation perspective. I'm going to take you through from the campaign trail to the final negotiations of the program, and then we'll take a look at the gaps between sort of the theory and the reality of the program. So, first, on the campaign trail--education reform figured quite prominently in Candidate Bush's campaign strategy--as Nina well knows.  It was both a way to appeal to the middle, with sort of compassionate conservative themes, but it also included some details that were going to appeal to conservatives that were school choice elements.  And Bush wove both of these carefully into his approach to school accountability by setting up a new set of sanctions--which Rick talked about--in terms of looking at what you're going to do with schools that are failing to make state benchmarks each year.  And in the first year, Bush said, in his campaign proposals, those schools would actually get an infusion of cash into their schools to try to turn things around.  If they failed for two years, they would have the opportunity to transfer to another public school.  And in the third year, they would get a voucher to attend a private school, if they so chose. In his first week on the job, President Bush unveiled his education plan, which was actually based largely on his campaign proposal.  But from the start, Bush made it clear that he wasn't going to go to bat for vouchers, and he made it very clear that he was--quote-"open to other ideas." What he was going to push for was bipartisanship.  And in his first week, he met with key Democrats, like Senator Edward Kennedy, and Representative George Miller--and even nicknamed George Miller "Big George." So, when Bush's staff met with Kennedy's staff--I think it was sometime in February--Bush's staff actually nearly fell out of their seats when Kennedy tossed out this proposal that he might be able to accept--not money going to private schools for the actual school day, but he would accept, possible, a program that would allow children in failing schools to seek after-school tutoring through a private venue. But because this was a compromise that needed maneuver around a lot of ideological land mines, the solution ended up basically being--to put a lot of layers of government between the Federal money and the parents, and this was able to appease the Democrats, who were very worried that it was just going to be sort of private sector monopolization, suddenly, of Federal funds; and, at the same time, give the conservatives some sense that the private sector was going to be able to have access to this money, and innovate and things like that.  But, in the end, essentially parents--it was a system where parents would exercise choice, but they wouldn't actually touch the money. And so that gets us to, sort of, what supplemental services are.  And, essentially, all they are is tutoring.  It's a very complicated name for a really basic concept.  But it's just--it's a Federally-funded program that is made up of--the money that's earmarked for school districts for educationally disadvantaged children, but it does require that those districts divert money that they would spend on other programs for poor children, and divert that toward private tutoring. So let's take a look at how this program works.  In an effort to compromise, the legislators set up a system actually modeled after an existing program, because that would be less controversial. It was set up to model the Reading Excellence Act.  And essentially what would happen would be that states would be required to set up a list of approved providers, and the Federal money could only go to providers that were on a state-approved list.  Then districts would be responsible for administering that program, and they would actually set up contracts with some of the providers that were on the state list, and that would establish, sort of, hours of tutoring that would be provided, and the fees and so on. From there, the parents would have a list of the providers that were available in their district, and the district--and the parents would choose which provider they wanted, but the district would pay the provider directly. Well, this sounds like a perfectly reasonable solution.  There are a lot of political actors in the process, and that led to an awful lot of complications.  The first one really was that, because districts could be providers themselves, there was a great potential for conflict of interest there, and that has borne itself out in a number of ways.  The second major thing is that the funding is extraordinarily confusing, and without going into the details, essentially districts are required to set aside somewhere between 5 and 20 percent of what's known as the money for Part A of the Title I education program.  And what happens is, because the guidelines are quite vague, districts can set aside quite a range of dollar amounts, and that leads to a lot of confusion on the providers' parts, and probably as well from the state perspective, in terms of trying to figure out how these programs are being implemented. In addition, the incentives for the districts are a little bit misaligned in that districts can also keep any of the unspent money that they were supposed to set aside for these supplemental services, and they see this program as a punishment, so there's not a real incentive to bolster up these programs at the district level. And that sort of leads to the final level of complication that we've seen, at least initially, which is participation.  Very few parents have actually signed up for these programs, and that may well be because, at the district level, it's not always being encouraged. My task was to take a look at the marketplace for supplemental services, looking particularly at the provider side, to kind of view it as a new marketplace.  And on paper, it's a very attractive market for tutors.  The estimated size could be upwards of 1.5 million children, and $2.4 billion.  Now, keep in mind that these are sort of back-of-the-envelope calculations--actually, courtesy of Jeff Cohen, who's in the back over there somewhere.  And so these are definitely sort of broad.  But it does give you a sense of the scope. And also t give you a sense of the scope, Jeff estimate that the pre-supplemental services marketplace for private tutoring was also $2 billion.  So, in effect, it could actually double that market. But, at the same time, in the 2002-2003 school year, only an estimated maybe 30,000 to 40,000 enrolled.  We don't have final numbers on that.  But it's certainly a fraction of 1.5 million, if that really is the potential size. From the providers' perspective, more than 1,000 providers exist around the country right now.  And as best as I can tell, that may well be double what the amount was at the end of the 2002-2003 school year.  There aren't really good numbers on this.  And there are lots of different ways to calculate totals.  And the lists of state approved providers change constantly.  But that's sort of my best guess. And, finally, in terms of the breakdown, the Education Department did a breakdown of some of the different kind of providers that are there, and the large breakdown is 70 percent private and, you know, maybe 25-ish percent public.  And within that, about 60 percent are sort of straight local or national private providers; about 24 percent are public school districts, or public schools or some form of the public education system.  About 4 percent are on-line; 3 percent are religious; and about 2 percent are university-run. Also in my paper I took a look at the cast of characters that we have in this marketplace, and I'm going to take you through that--just briefly. One way to think of it is actually a lot like, sort of, high school cliques.  And the proportions that I have listed here are the proportions that each of these groups-- [Laughter.] --make of the thousand-plus providers.  This isn't a gauge of market share, because we don't really have any way to gauge that without enrollment numbers for these children. One of the first--and perhaps--certainly one of the most important groups, is the large corporate providers.  And I call this "the popular crowd," because their brands were generally well-known.  These are like Kaplan, and Sylvan, and Princeton Review.  And parents who seek these out certainly are, in a lot of ways, kind of buying a familiarity of brand. The second group--school districts are really big players in all of this.  And by characterizing them as "bullies," I don't necessarily mean that in a derogatory sense-- [Laughter.] --but they probably have the greatest amount of leverage in this whole situation at the local level. One of the more interesting elements of this group is actually what I label "the overachievers."  They're the small corporate providers.  I ran into one group called "Club Z," which actually isn't even that small, but compared to something like Sylvan or Kaplan, it certainly is.  And what Club Z is able to do is maintain a very flexible business model that actually allows it to send tutors to children's homes.  So this is an interesting group to watch for new ideas and approaches to tutoring. One significant group is what I call the "do gooders," and these are community groups and organizations, and even small businesses or local retired teachers that hang out a shingle.  And they make up a large proportion of the total group, but each individual provider doesn't necessarily command an enormous market share.  I mean, they may only be serving 10 students, or 50 students or 100 students. The last group is what I labeled "the geeks," and they're the on-line providers.  I'm sure the comparison is rather obvious.  But they are certainly searching for innovative ways to use technology to meet tutoring needs, which is something to keep an eye on. One of the things that was most striking, actually, in the research that I did was actually how much we don't know.  And as, perhaps, Secretary of Defense, Donald Rumsfeld, might characterize it, there are an awful lot of known unknowns out there. [Laughter.] And what was most striking was actually how obvious some of these questions are, and it was a little frustrating that I couldn't get numbers on them. But the first one was the number of children actually eligible just for supplemental services.  I give you a guess, but it was really a back-of-the-envelop calculation, and there hasn't been any real tallying of that number. In addition, we don't know how much money the Federal government has or will spend on the program.  Those numbers may be forthcoming, but we are a yare-and-a-half into this, and without a sense of how much this is costing the Federal government, and sort of the system in general, it's very difficult to do any sort of cost-benefit analysis, or really evaluate the program. Another thing we really don't have much a sense of is how many new providers are getting into the marketplace.  While we can look at growth in the marketplace, it's hard to tell whether they're new providers, as in they weren't offering tutoring at all before, or that they were maybe community groups or something that were providing tutoring locally, but now they're taking advantage of supplemental services.  And it would be a good idea to get a handle on that. And then, lastly, we don't have much sense of enrollment at all, either in terms of looking at what types of tutoring providers tend to have the highest levels of enrollment, or specific providers.  And national numbers on that would be very helpful in gauging what's really going on with the program. In addition, as I was assessing the marketplace and looking at what a number of the providers were dealing with, they told me about a lot of problems--and I'll just go through them quickly, because I"m running out of time. But the first one we talked about a little bit before was parent participation.  Right now, parents either don't know about the program--and maybe in some instances don't want it.  But we don't have a very good grasp of that. Implementation is also very uneven.  Districts--some districts may well be dragging their feet.  Others may just not understand the program.  But there have been a lot of stories of districts' only notifying parents of the program with a jargon-filled letter that's kind of tucked into a student's backpack. Another issue that I mentioned earlier was districts' wearing two hats.  There is a potential conflict of interest there, in terms of districts that are responsible for both administering the overall program and promoting their own tutoring program at the same time.  And Districts will acknowledge that they do have a leg up, in terms of having better access to parents, and often having better access to facilities. The last thing is that quality control is kind of an afterthought at this point, at all levels of government.  And right now everyone is really focused on getting it up and running but nobody's really to define "success" for this program.  And if results is what we're worried about, that's something we should really be thinking about. My time's up, but I'll leave you with a couple final recommendations, and it should be noted that the Education Department has made an extensive effort, really, to start upping participation levels.  But that is kind of a necessary but not sufficient solution so far.  They really do need to decide who's in charge, and who's responsible for what; whether it's the Federal government, state government or local government. They're also going to have to decide on the rules of the game, especially when it comes to districts, and what districts' responsibilities are.  And they need to hold them to that. And the final piece is that hey really need to collect more and better data to diagnose implementation and quality problems.  The known unknowns that I listed before really are a good start, but this program is simply not going to be successful unless we know how it is and isn't working. And that sort of leaves me with a final thought--because as I was casting about for statistics, I was really challenged.  I was looking for how many students are participating in the program, or how much money might be spent.  And the only person who could really give me a gauge of this was Jeff Cohen at Sylvan.  And I just was puzzled as to why he, of all the people in this country, seemed to have the best information on this, and why it is that Secretary Paige wasn't the source of that. And, you know, I was just--the last question that I guess I would have is: is that how it should be for such an untested program?  We really should be a little bit more focused on exactly what's going on with the program itself? MR. HESS: Thank you, Siobhan. I think we all ought to recognize, too, that Richard and Siobhan were willing to walk into probably the worst possible question of research, one that's still unfolding; is just underway, with broad national significance.  So, before I allow the discussants to comment on this work, I wanted to make sure that I recognize kind of that debt, and I appreciate you guys' being willing to take a first crack at this. We're going to go with discussants in the order they're listed on your program. So, first up will be Lisa Graham Keegan, the CEO of the Education Leaders Council MS. GRAHAM KEEGAN: Well, thanks very much, Rick.  First of all, these guys--did you get it right?--got it achingly right.  Yes--unfortunately--quite right, I think. And just a couple of comments on the papers, and then I want to sort of outline, as part of those papers, where we ought to go from here, and how No Child Left Behind has really changed our context--thank God.  And that's the bigger picture.  I mean, we're operating in a different context here.  And how you meet that, in a system that's built for old results, is very challenging. The figures that Richard gives in his paper on who transfers in open enrollment setting are poignantly true, and that equation always has to figure the amount of resistence that's there.  And he makes this point.  But, I mean, open enrollment in public school settings is quite marginal, simply because it's really not open, and there's really no incentive.  I mean the school that takes the child, if I'm a public school, and I get a hundred more kids, I probably get paid roughly what I did last year to do that.  I really don't have an incentive that's direct to do that--in most instances--because principals aren't controlling for student budgets.  The incentive would be huge if that were true. For example, we might have a different discussion on class size if teachers got paid per child in the classroom--right?  If 15 is the ideal number of students, and I get paid the same for 35 kids, that's one discussion.  If I get paid a bounty of $1,000 per child, I might think 27--I can do 27.  I mean, these things matter.  What kind of incentives are we offering for that change in behavior? Siobhan also puts up--and in the paper, I was so fascinated--70 percent private providers in supplemental education.  Interesting.  But if you defined who the providers are in the traditional public school in the same way we're doing it in supplemental services, you might give a similar breakout. The fact of the matter is, in supplemental services for the first time we're actually defining the provider a person that's providing the materials for the services.  In a public school, it is also largely private enterprise driven, but we don't calculate the number of textbooks from Harcourt or those kind of things, as a private provider within the public school setting.  It's just interesting to me that now that becomes a subject--even though a lot of those private providers are actually under the jurisdiction of the district when they're providing those services.  And so finally we're beginning to get this sort of tension between who's really out there. No Child Left Behind changes our focus intentionally from process to results.  And that just sounds interesting--oh, great.  And the fact of the matter is, our school systems are built not for individual student achievement.  Our school systems are built to handle massive numbers of kids coming through the system, and your job is to school them; have them in school. The incentives are not build around--traditionally--whether you succeed with them academically or not.  Your incentives are around getting them in the door, counting them, having them there and alive at five, home, back the next day.  That's what the system is built on as an incentive.  And it's important to remember that--that as this new demand comes, it comes on top of a system that's built simply for process.  That's what it's for. So, choice, now is a way to meet the new definition of public education per No Child Left Behind which is, success in public education means every child comes to a particular standard.  Completely new endeavor.  Completely new. And so what we say is, and so if you're stuck in a school that can't get all children there, what we need is a choice.  I mean, and this--Richard hits on this pretty poignantly in his paper: we need a choice.  And those choices are out there.  But choice by itself doesn't get it for you.  Choice provides you a new venue.  Choice--if it's done in a model where money follows kids into a school also provides you an incentive. That's why public school choice is so marginal, I think.  It's only effective if money is really following the kids, as far as incentive or an innovation.  If it doesn't do that, there's no reason for me as a principal to want to attract new kids.  I just don't have any upside there.  I just have more headache. And so you get a new venue.  You get an opportunity for a new venue.  At the same time, we finally have realized that our remedy is not the choice itself; the remedy is what happens in the school, between the teacher and the child.  The student's success is our remedy.  That might happen in the traditional system, and it might happen in the new system, under a new setting, in a private school or a charter school, or a private provider in supplemental services.  But the remedy itself isn't a foregone conclusion, simply because you now have choice. Accountability provisions, as far as measurement, matter enormously here.  The new system says--No Child Left Behind says--you've got to succeed with kids.  If you don't succeed with kids in your school, we're going to give them the opportunity to go elsewhere.  But simply the opportunity to go elsewhere didn't solve anything.  "Elsewhere" has to be better. And this is where, in supplemental services, for example, I think we have a much greater opportunity of getting this right on the up front than we did, for example, in how we handled original accountability provisions in the charter school movement.  I mean, the charter school movement came--timeline wise--ahead of when we got clear about accountability.  So the marketplace mattered--and matters--enormously, bringing new innovators in matters enormously, and accountability was coming along.  But it wasn't sufficient--we didn't have enough per-student measurement going on in schools where we could say "Success," "Not a success," "You're still in business," "You're out"--except in the most obvious of cases.  And this is why I think, right now, in supplemental services, the onus is on those of us who want these provisions in place to get to the accountability stuff right now.  Siobhan makes a great point: somebody's got to ante up.  I would hope that it's those of those of us who are advocating for these new venues; that we define this well. But in the charter school movement it was like a V-8 moment, when people who were running shoddy schools in the district system got up, walked over, started running shoddy schools that were charter schools.  Who knew? [Laughter.] I mean, the fact of the matter is, if people didn't necessarily change, in some instances--you know?  Bad practice is bad practice wherever you do it.  And "choice" is not it's own remedy.  Choice is a venue' choice is an option.  Choice is, and should be--can be--an incentive for both parents and the schools that operate.  But it only works to the extent that you have some feedback to the public about what the quality of the service they're getting is. That's critically important, I think, right now in all of the choice provisions of No Child, that they sit side by side with public information about the quality of the program that you move to.  New venue?  Cool.  Fabulous.  New walls.  Good looking teacher.  Good.  Let's go for results now. So that's critical. The implementation and the hang-ups on supplemental services, in particular right now, is so important.  Siobhan's paper is very clear here, and I think grasps all of the issues that we understand in these first couple of years are pushed back. A couple of sticking points: I think one of the biggest is, the dollars available per student--if a district does not sort of dole out the money available for supplemental services, they don't manage to spend all that--which is anywhere between 5 percent of their Title I and 20 percent of their Title I, it is rolled back into the operational budget.  If you want to just do the thought exercise there--lots of money rolling right back into the operational budget eventually. Now, that's partially easy to fix; partially not easy to fix.  But that reality needs to be dealt with.  And the way that I see that changing is if the pain of a lack of achievement is greater than the pain of not having the money, the services will get provided.  If the success of having supplemental services provided is so huge, and these programs are so great that schools cannot afford not to get their kids into them--that's one way to drive it. The other is simply to regulate it and say, "Look, if you don't spend the money, it stays in a pool and it goes into next year's deal.  And pretty soon your tutoring at $10,000 bucks a kid."  Jeff might like that.  I mean, that might be good. [Laughter.] But that is a tension right now in their budgets.  And I do think it has to be dealt with, because internal incentives matter.  I mean, those of us who believe in marketplaces can't simultaneously applaud them and then boo and hiss school districts when they exercise an option that increases their per-pupil per capita.  Well done, you!  I mean, that's what you do if you're running a school, you're trying to get your highest--you know, highest amount of funding available.  And Title I dollars--actually in a school district--are sometimes some of the most readily available dollars for a principal.  They get that in a pot.  Now, you may get that in a pot in ways they don't get their other money in a pot.  And you want to talk about a pot that they like?  They like their Title I dollars--because they have control over it.  And as a school principal, I don't get that very often.  And I'm loath to give it up.  And I"m not going to give it up unless I'm convinced there's a trade-off for me. Some promising innovations here, as to what's coming on.  One of the things--and our organization, the Education Leaders Council is engaged in--is a coalition of both supplemental services providers--the geeks--everybody that Siobhan was mentioning--and also parent advocates; the Black Alliance for Educational Options, for example, does work with parents, and has received support to go and notify parents: "Free tutoring is available."  And if we don't change the name of supplemental education services this year, I think we're not doing our job well.  Because it's a geeky term, and parents don't know what that is.  "Fee tutoring"--advertise that, and work with providers to try to understand what some of these barriers are. One of the biggest issues that we discuss, and that providers are discussing right now, is--beyond the barriers to whether or not the districts really want to participate in this--how do we--what kind of accountability measures are we going to put on ourselves and on everybody else who wants to come in.  Because those of us who have done choice in all its beautiful incarnations, and have lived through the front-page story about, you know, somebody having school under a tree somewhere, with no content whatsoever, when the charter school movement--even though that's one school out of a hundred, it doesn't matter the day it hits the front page of the paper.  And the same thing will be true with supplemental providers.  If you find out in an after-school program that you've attracted folks who are basically snacking, napping, hanging out for a thousand bucks a kid--not good for the supplemental services exercise.  Certainly not good for the children, if we can just start there.  Not going to be good for a whole lot of children, if the exercise goes down on the backs of bad ideas like that. And so I think the providers themselves, and those who are trying to get to parents, are most keenly focused on how do we guarantee quality. Another innovation--and I think--I think I would like to see a whole lot more of--are the few providers who are going retail, directly to parents--an unbelievably successful idea--and that is to simply sell directly to parents; buy phone lists in inner cities; call parents directly.  And call to get them into the program; call them when the kids don't show up; call them to know how the kids are doing.  There are companies out there now--Platform Learning, probably most notably, up in New York--that is exercising this new action, with full-time people in a calling center.  And they call, and they get the kids in; the call when the kids don't show up.  They found out that when 30 of the kids didn't show up and they called parents, and their parents said, "No, they're with you," and we said, "No, they're not, or we wouldn't be on the phone"--that the kids were at the Motown Week celebration that was being touted by the District as a good and fun thing to go to.  And when the program said to the principal, "Ahh--we got kids at the review over here, and we'd kind of like to be having them--at the Motown Review, we'd like to have them in class," the principal said, "Ohhhh"--you know, because it wasn't the principal's job to make sure that the kids were there, but the principal was amenable to being called.  And the parents are saying to them, "Nobody's ever called us like this about our kids before.  That's outstanding.  Just wait 'til she gets home." I mean, that kind of really high-touch relationship in poor communities that are not used to that is enormously important--I think.  And finding ways to do that is going to be probably, to me, one of the most exciting things about this innovation, that it provides the opportunity for that. And I'll stop. MR. HESS: Thanks so much, Lisa. Next up is Mike Cohen, of Achieve. MR. COHEN: Thank you, Rick.  I have to make one quick comment on Lisa's last idea of outreach to parents, which I think is really good.  But I couldn't help but wonder whether you were about to set up a conflict with the FCC for all those folks who put themselves on the do-not-call list. [Laughter.] Both of these papers are really quite important, and I think it's a good time to be addressing the question of how well both the choice and the supplemental services provisions are working.  Both papers highlight some significant obstacles to successful implementation. Richard's paper, in effect, says: there are lots of other choice provisions and programs out there already, and the No Child Left Behind provisions are a pretty small drop in the bucket compared to what's out there already, in terms of the impact we can see so far. Siobhan highlights a number of significant implementation obstacles, from unclear rules of the game, to needing to get through state certifiers and get contracts from districts--perverse set of disincentives, some of which I'll talk about in a minute. But it's clear that there are really significant obstacles to making this program work. I want to try to make three points here in the few minutes that I have available.  One is to underscore that I think both the idea of expanding choices and options for students, and providing for a richer set of providers in the system is a good thing to do.  So I come at both of these initiatives inclined to find ways to make them work.  We ought to be doing more of this.  In some cases the Federal government has funded other efforts to get at the same kind of effort--whether it's the charter schools program, or the magnet program--to expand choice and bring different providers in; whether it's the 21st Century Program, to provide at least a place where kids could get tutoring and the like.  These are things we ought to be supporting. Second point I want to make is that the rhetoric about getting kids out of failing schools that they're trapped in--right?--and the rhetoric about combining sanctions for schools with help for kids is awfully appealing politically.  But as I read these two papers and thought about what they were saying, I'm less clear that they provide the right design principle for making either supplemental services or public school choice work as well as we need it to.  And I want to spend a bit of time on that. And then the third point I'm going make is there may be some better ways to approach this, to get done what--I think there's a fairly broad agreement--needs to be done. Let me talk for a minute--so the first point I wanted to make simply is: these are good things to do.  And I won't--I think there's agreement on that.  I think we're not having an ideological debate here.  I won't spend much more time on that. In terms of the second point--that the rhetoric may be better--right?--than-it may be better as rhetoric than as a design principle.  Siobhan's paper points to a number of real obstacles that I think derive, in part, from the rhetoric being translated into--what?--in particular, by the way, the linking of access to either choice or supplemental services, to schools' AYP.  That's where the rub comes in.  This is a fairly complicated issue.  Done this way, it helps create disincentives for districts to use outside providers, because it's very clearly tied to--the Title I money they otherwise would have had for in-district operations.  The caps on how much Title I money can be spent on supplemental services means that at some point in time if the program is succeeding, that it will create either limits on the number of students who can be served--because there's not enough money to serve all the ones who need it, or it will provide incentives for providers to provide low-cost and possibly low quality services.  We don't quite know the relationship between the two, but at some level I'll be there is one.  Or it will mean that you get high-quality services for three tutoring sessions, and then you've used up your money. But the design here, again--taking the money out of Title I and using it for these purposes--creates some perverse incentives, I think, and some interesting constraints. A similar one: in her paper, Siobhan tells a little anecdote about Club Z--very interesting.  I don't now how widespread a phenomena this is, but the story basically is--Club Z, that's a home tutoring service, does such a good job with the students that they are tutoring as a result of the NCLB provisions, creates such significant gains, that it turns out the schools that those students came from made AYP.  The kids are no longer for services, and they've sort of, you know, eliminated their market. [Laughter.] I don't know how frequently we're going to see that, but it did remind me that there was a time before--I think the 1994 re-authorization of Title I--when there were incentives in the--disincentives in Title I for schools to improve. And some point, if you made too much progress--the thought at least was that you lost your Title I money, and that in part is why Title I is much more heavily--funds are much more heavily allocated on the basis of poverty, rather than achievement. We should be careful, as we watch how this unfolds, to not make the same mistake we apparently made before, and make sure we get the incentives right here. The other point I would make, based on the Club Z example, at least--but I think you'll see this in general, as a result of linking this to AYP--I think many students who need tutoring will need it for a sustained period of time.  If your access to tutoring depends on whether your school makes AYP--depends on whether kids do better next year than they did last year--right?--and it affects your school's eligibility, that's not the basis to provide the kind of assistance that people who are behind need. So I think there are--again, there are some things about the way we've designed this that don't work will for providers, don't work well for the role we put districts in, and don't work well for what we want to do for students. Similarly, I think the choice provisions suffer a bit.  Paul Hill and his colleagues on the working commission on choice issues an interesting report a few months ago.  And I won't go through detail on that.  But I think the clear lesson from that is if you want to have a robust choice program that really needs to be incorporated into the system--incorporated into how you design and operate your system, not something added on. As I think Lisa was making the point, a choice program will work if you allocate dollars on a per-pupil basis and if the dollars follow the kid; even to other public schools within the district.  Very important to do.  You need a robust parent information and outreach system; not just a letter stuffed in the backpack that quotes from the supplemental services guidance--which I'm sure was indecipherable--by design. [Laughter.] At least it was in my day. If you look at, at least, the press clips on how the choice program is working, and the apparently slow uptake work, you know that there are a variety of things that are getting in the way: delays in receiving test scores and making AYP determines, so somebody at the district level knows which schools are going to lose kids, and which schools could potentially receive kids; and lack of information for parents; space limitations, particularly in urban districts, even though that's not an acceptable excuse, the fact of the matter is, if you only have a small number of schools that are making AYP, and they have a small number of seats in it, there are going to be a small number of kids who occupy those new seats. The point I'm trying to make here is if you're serious about creating a public school choice system that maximizes choices and opportunities for kids, you would design a school system to do that, and it would be designed very differently than the way it is now.  Simply tacking some choice onto systems that are not designed to make it work will leave a set of Federal provisions that don't work very well. So I think that there are some tensions and contradictions built into the very compromises that were reached in order to enact this legislation.  And I think if you step back from it a bit--and I have the luxury now of not being in a position where I either have to get something enacted or oversee its implementation, so it makes it a lot easier to think about it.  But if you gave yourself that luxury and anticipated that at some time Congress will revisit this, I think there a couple of different ways to think about this.  If you want to maximize tutoring programs, for example, you might look at something like the 21st Century Program--not to do exactly what's in that, but this is an approach which first of all, adds money to provide additional services rather than puts districts in the conflictual position that they're in now. Secondly, it is available on a more nearly universal basis, at least as far as the dollars would follow.  But the eligibility to participate doesn't depend on how well the kids in the school from one year to the next, and so you can m ore easily design a program that says, "If you're a kid who needs help, whether you're in a high performing school or a low performing school, we can provide you with sustained, high quality help as long as you need it." So there are ways, again, of designing a Federal program that provides funds to the local level.  I could, frankly, care less who gets the money and who provides the services.  I think we can figure out ways to make that work--that will provide more access and more opportunity to more kids than the current provisions do.  And we would need to add, as Lisa indicated--we'd need to get the accountability issues right here, and I think we're a long way from doing that. On the public school choice side, it's a bit more complicated to me.  All of the things that ;you need to do to operate a good public school choice system are--for better or worse--largely under the control--at least many of the things are largely under the control of local school districts.  And if players in that district don't really want to make it work, there are lots of things you could do to not make it work.  So it is difficult for me to imagine how the Federal government, in whatever legislation, waves a magic wand and says "Everybody's doing this now."  But you could imagine fairly robust, well-funded demonstration programs and incentive programs that would help a lot more local communities do this right.  And over time, I think you'd add to the movement for public school choice, and more providers in the system in ways that I think would make a difference. So, if I could wave a magic wand, those would be the directions I would go in--although I wouldn't rush there.  Let me just close by saying that it is very early in the implementation process.  It's not too early to be thinking about where to go, but the most important thing to do now is to collect the data that will enable us to get a really good handle on how things work for awhile. MR. HESS: Thank you Mike. With that, we'll turn it over to the one who actually is charged with implementing these programs-- [Laughter.] --Nina Rees, Head of the Office of Innovation and Improvement at the U.S. Department of Education.  Nina? MS. REES: Thank you to Rick.  And I want to thank AEI and the Fordham Foundation for taking the initiative to commission these papers.  I's always good to have outside organizations weigh in and give us advice.  And I'm honored to be here with you today, even though I think that I could have used a few supporters--although Mike did a nice job. MR. COHEN: I was actually thinking of the guidance we put out. MS. REES: Before I start to talk about Siobhan and Richard's paper, I think it's important for us to take one huge step back to evaluate the reasons behind these provisions; why the President and the Secretary put them in place; why Congress passed them.  And a lot of people forget the fact that the reason why these provisions were put in place were really as an incentive for districts to do a better job of reaching out to parents, and to not take those parents for granted. So, in my opinion, every time you evaluate the success of public school choice, or supplemental services, it's also important to look at some of the things that the district has put in place in order to counter these provisions. Let me just give you one example: the School District of Seattle before these provisions were put in place.  I spoke to the superintendent, and he was telling me that the reason why a lot of his students are not benefitting, or taking advantage of this public school choice program that they have in place is because the district does a very nice job of reaching out to parents, and has basically given them all the reasons they need in order to keep the students in the system. Now, I'm saying that, understanding fully that some districts are going to take advantage of this, and there are ways to sell certain programs to parents--especially low-income parents--if you don't have a lot of information available out there.  That may not necessarily be good at raising student achievement, but at the end of the day, again, the purpose of the Federal Title I money, and the Federal role in education is to raise student achievement.  And one of the things that we have been noticing over the years is that districts are treating this money as funding for the school system, and not necessarily taking into account the needs of the parents in the system. So, again, looking at how these programs are being evaluated, I think it's also good to look at what districts are doing, and the reaction of the districts vis-a-vis these provisions. The second point I'd like to make--and this is something that Mike just mentioned--is that change takes a lot of time.  And I think we're right now--I mean, it's a good idea about the implementation of this program and look at it after just one year of implementation, but I would also venture to say that I think it's really premature to draw too many conclusions, or to take too many active steps at the Federal level to perhaps regulate the law.  I'm not saying we shouldn't do it, but for every regulation that we've thought of, there have been all these other reasons why the folks have told us it may be good and wait and see how this plays out.  It may be better for us to offer more technical assistance and see how that plays out, before we go in and over regulate the law. And one other point that I should make again before I answer the critics is that a lot of what needs to happen initially in order for these pieces to work is dissemination of good information to parents--parents who are not used to getting information; parents who are not educated consumers of education.  So, the report card provisions of No Child Left Behind make it clear that you have to make certain bits of information available to parents, but I think it's also very important for other individuals to take the initiative and make more information available to parents so that these choices are meaningful choices, and not just things that are happening because a lot other parents are also taking advantage of them. And some of these efforts, again, are on their way: GreatSchools.net, and the Burge Foundations have funded efforts with Standard & Poore's and other entities like Just for the Kids to make more information available on the web; but, again, to translate those in terms that low income parents can understand is a whole other venture in and of itself that I think individuals at the local level ought to take on, and it's not necessarily something that we can do at the Federal level in an easy fashion. So, with that, let me just go over very quickly Colvin's paper.  He did not really go into the details of the criticisms of No Child Left Behind in his presentation, so I'm not going to invite comments--[Laughter.]--on it. He was actually more critical than he was in his presentation.  One thing he did mention, though, is lack of funding.  He thinks we need to offer more money for transportation.  I would argue, again, that we offer funding.  IT's between 5 to 20 percent, but there's nothing limiting the district from adding more money on top of that if there is demand for these provisions.  They can transfer money from different pots: Federal pots, local pots, state pots of money.  So there is no cap, per se, on how much you can spend.  There are a lot of different sources of funding--Federal funding included--that states can use if there is a demand for public school choice. I also take issue with Lisa Keegan, who I respect, and I agree with most every time--with the fact that public school choice is not necessarily going to work, not so much because of the mechanics that she mentioned, but because I think if you just look at the research--the early research--on, you know, public school choice, again there is not that much out there, but I've read enough of Rich Hess's work, and Patrick Wolf's summaries of what's happened out there--I think that there is enough evidence to point to the fact that when you empower parents, and certainly when you look at the patterns of choice in districts where parents have the means to transfer their children out of schools and purchase a home in a neighborhood that has a good school, you notice that the quality of public education goes up.  So the more you encourage public school choice--again, you have to make sure the money is following the child and there are incentives in place--but the early research, I think, does point to the fact that the quality of education goes up; graduations go up; and, in some instances, also, the income of the students who graduate from these schools goes up. So we need to do more research.  This is one area that I don't think researchers have done a good enough job of doing the kind of research that we like at the federal level, which is the randomized field tests.  But I think as more people present the evidence, it's going to be much easier for districts to buy into choice as a mechanism to improve student achievement. Now, moving on to Siobhan Gorman's paper, the key thing that's come up over and over on this panel is the fact that there is an incentive for districts to not implement the supplemental services provisions of the law very well.  That is, in fact, true in most cases, but another thing that we have noticed in our discussions with folks at the local level is that fact that there are actually a lot of school districts that have valid questions about supplemental services and how they should go about implementing them.  These are not the districts that make the headlines in the newspapers, but they are the ones that are trying to do this and they are honestly not sure as to how this innovation is supposed to play out, and how they can best ensure that what these providers are offering is actually going to make a difference at a time where they feel, from a budgetary perspective, they may not have enough resources to allocate to truly implement these provisions in an effective way. One of the things we have been doing at the Federal level is we have a partnership with the Mott Foundation to encourage partnerships between after school programs and supplemental services.  In essence, we would like to see more of these after-school providers to also become supplemental services providers, so they're not solely relying on the flow of Federal funds, but they're also getting different sources of money in order to make sure that the program stays in place even after the need for supplemental services goes away. So, again, I think it's important for us to do a better job of understanding what the issues are and what it is that districts need in order to better implement.  I also think that it's hard for us to criticize--or I think it's unfair to gauge sort of this whole profitability issue, and attack the lack of national data and evaluation as the reason why we can't really invest in this market.  I think with any kind of innovation, or any individual who's ever done anything innovative or has tried to get into a particular market--especially the education market--there are certain chances you are going to take.  I mean, people are not always automatically going to like what you have to offer.  I think we do need to do a better job on the implementation end--and, as I said earlier, we may want to regulate the program and find ways to offer more incentives for districts to do the job well, but again, I want to kind of put this back on the side of the providers and get them a little bit more engaged in some districts, certainly, to mobilize, perhaps, folks at the local level to come up with these solutions rather than come to us constantly and ask us to come up with the solutions.  Because this is--I mean, this is a Federal program.  This is something that the Federal government has made available, but ultimately it's something that's being implemented at the local level.  And the needs of different school districts is always going to be different, so you don't want to come up with a one-size-fits-all solution that doesn't fit every school district's needs. In terms of lack of information, the consolidated reports that the Department has required--the report that states have to send to the Department--we have that information right now.  We're kind of going through it.  We need to clean it up, so to speak.  So we are going to be able to share some information about what has happened so far, to give folks a feel for what's going to happen next.  There's always a lag time, though, so, again, it's much better for folks on the outside to also try to track these numbers and publicize them. The Title I longitudinal study is also going to start tracking the benefits of these programs on student achievement.  Again, this is going to take a while to produce, but we are interested in finding out whether these remedies are effective at raising student achievement. And last, but not least--you know, Siobhan mentioned earlier, we have invested a lot of funding in doing more parent outreach.  Let me just highlight some of the things we've done.  We've given a grant to the Black Alliance for Educational Options, the Hispanic Creo [inaudible] Foundation on the parent front, but we've also made investments on the technical assistance front through a grant to the American Institute for Research.  We've also given a grant to the National Governors Association to help us get states to better understand public school choice and why it's a good remedy for them. So I think we're trying to do as much as possible at the Federal level to make sure that this law is being implemented properly.  We're going to have a booklet of best practices in the areas of public school choice and supplemental services coming out in March and April that will offer, again, more guidance on how to best implement.  But I think, as I said earlier, it's a little too early to criticize the law.  It's, I think, a little premature to start to regulate.  We need to think a little bit more about ways to do that in order to reach the goals that we're trying to reach. And I, for one, am very optimistic, and I think the outlook--especially for supplemental services--is very bright because, unlike some of the other sanctions, so to speak, under No Child Left Behind, on this one, we actually, in a lot of places, do have the support of principals and teachers, who see supplemental services as a benefit to their students, and don't always see it as competition. So, with that-- MR. HESS: Thank you, Nina. Nina, one question: we've talked a lot about the percentages of Title I money set aside for students.  Could you give folks just a quick illustrative example or two?  How much per pupil are we talking--nationally, or from a given district--just to give folks kind of a sense. MS. REES: Well--I wanted to mention something earlier--the fact that we don't have--you know, people are complaining that we don't have a full amount districts have to set aside.  If you look on the Department of Education's website, we have calculated the rough amount that school districts have to set aside.  Again, it's going to vary depending on how much money actually went down to the district level.  But we do have that number on our website, if you want to take a look at it. But as Siobhan mentioned, they have to set aside between 5 to 20 percent of their share of Title I funds, or something that equivalent.  So it doesn't have to be necessarily Title I money.  And then, in terms of how much money is going to follow the child, it's anywhere, I think, between $800 to $1,500 to $2,000.  But these numbers are going to vary.  And, you know, if you have a lot of demand, for instance, that number is probably going to go down. And it's also very hard for us to monitor, you know, the demand a the local level.  So, for instance, if a district has done everything that it can possible do before it decides to spend the money on something else, you know, how do you gauge what they did and how they did it, and then decide if they did enough or not. MR. HESS: Sure.  Okay, thank you. Andrew and Emily will come around with the microphones.  WE have about 20 minutes to take questions.  Please catch my eye or their eye. And we'll start right here.  Please identify yourself by name and affiliation. MR. LIEBERMAN: I'm Mike Lieberman with the Education Policy Institute. Fourteen years ago, almost to this very day, President-elect Bush and President Reagan had a news conference to announce the President-elect's education program.  Public school choice was the main platform in that program. I'm wondering if anybody on the panel can tell me why it failed, and whether those considerations have been taken into account in the existing program. MR. HESS: Thank you. MS.          : Mike, I can take it.  I would agree with Nina.  I wouldn't go to the "it failed" thing.  But it doesn't--as I said, the system is not built financially and incentive-wise--you know this--for money to follow kids under a public school program.  The way that we fund school districts is--just to be overly grotesque about it--the school district has a property tax--it levies a property tax.  It maybe raises half the money.  Then the state kicks in the other half.  There's a calculation based on who was in school for roughly a hundred days last year.  And we move that forward and you say, "All right, you had 10,000 kids, and the state amount says that you get $6,000 per pupil.  You raise this much tax, you need this much state aid."  That all goes to the central office.  Central office then distributes services and not dollars out to schools.  They don't say, "All right.  Westview, you get this.  Southview, you get this.  Northview, you get this."  That's not what they do.  They say, "Southview, here's your teachers and your principal and your books." So now, they say, "Oh--and choice.  Everybody choose a school."  Well, when everybody chooses a school, it doesn't--the dollars don't necessary change.  The services, you know, may or may not change.  It's all so muted, I think, Mike, that it--it just--it doesn't create enough of an effect, and the schools want to set--you know, "We only want so many kids," because they don't get more money for more kids necessarily and don't want to be overcrowded. But as Nina points out, in those environments where the choices are somewhat real somehow--either because the district has made dollar movement possible--and they can do that.  They could all do it tomorrow if they felt like it--or, for whatever reason, they somehow, they just did it right--in those more competitive districts, you do get a higher achievement rate, just like you get a higher achievement rate where districts are small and in competition with each other. The system isn't built for money to follow kids.  And I think until it is, choices like that--even outside the public sector--are not going to be as rich an option as they ought to be. MS.          : And can I just ask you a question?  When you say "It didn't work"--can you just elaborate on that? I mean, I would argue that the growth of charter schools is actually a testament to the fact that the call for public school choice has actually been heard, and people are creating new schools and making more choices available. I would also argue that the growth in the support for private school choice is a direct result of the fact that in some places public school choice is not enough. So I think it is actually, in some places, working just fine. MR.           : Can I just add to that?  I mean, you say "public school choice fails."  You know, if you look at the satisfaction that parents have who actually choose, it's overwhelming.  People who choose schools are satisfied with them.  And the point I was trying to make is that there's a lot of choice going on.  The people who are choosing generally are pretty satisfied.  It's just that the systems aren't in place to allow everybody to make good choices, and there are not enough good choices to go around. MR. HESS: Okay? Right here. MR. HOWELL: I have three sort of factual questions. MR. HESS: Please identify yourself. MR. HOWELL: William Howell from Harvard University. Do we have any sense--squishy numbers here--what percentage of the kids who are receiving supplemental services are getting them from districts--number one--from district providers? Of those, what percentage of these districts are starting up new programs, or are they just re-labeling existing programs that they have--tutoring--in order to get the money?  That's number two. Number three is: of those kids who are receiving the services from the district, who are attending underperforming schools, are they then receiving those services at the very schools that were identified as being underperforming in the first place? MS.          : I can answer the last one.  They cannot get the services in the same school, by the same entities that made that public school become a school in need of improvement. So--if the school is in need of improvement, that school cannot then also offer supplemental services after school hours.  But, an outside entity can come in and offer the services. MR. HOWELL: My understanding was as long as--if it's a district that's providing the service--the district as a whole, but the service is actually being delivered at the school, then it's okay--as long as sort of the funding is going through the district but not the actual school, that then it's actually--you've got the underperforming school giving the very-- MS.          : Well, it cannot be the teachers at that school, for instance. MS.          : To get at your first tow questions--the first one was that districts provide, maybe, 25-ish percent.  It depends on how you count.  It may be up to as much as a third.  But, again, these are pretty squishy numbers. And actually John Lipey, who's right behind you, is in charge of that very program for the L.A. Public School District.  So he may well be able to answer this a little bit better. Can you remind me of your second question? MR. HOWELL: [Off mike.  Inaudible.] [Laughter.] MR. HESS: Well, we can get it at the break.  We'll get to it at the break. Right here? MR. ARCHIBALD: George Archibald, with the Washington Times. In doing a story about No Child Left Behind this week, I called, you know, various of the players in town.  And Checker Finn was concerned about what has been mentioned here: that he thought one of the worst problems that needed correcting was the problem with the limited school choice provisions, and the fact that the group that Siobhan labeled as "the bullies"--the school districts--were resisting all efforts of people to have public school choice and what have you. And so, when I had an opportunity to interview Secretary Paige, I asked him about that, and he agreed it was a problem that needed correcting.  He was quite emphatic about that.  And I asked him if he agreed with Mr. Finn that it should be done by legislation--mindful, of course, that, Reg Weaver and the NEA also have a big agenda of legislative changes they want. I was not surprised when Secretary Paige responded that he did not see any need in any part of the law to open it up at this point for legislative changes, but that he thought that everything that was wrong--and that had been noticed to be wrong--could be handled through regulation. And now Secretary Rees is telling us that she doesn't see any reason at this point to over-regulate, or to re-regulate or whatever. I wonder if I could ask the question this way: to Lisa Keegan, the advocate for the education leaders, what sort of regulatory change would you see in this choice area--this limited choice area--to hold the feet of the school districts to the fire, so that they will have less opportunity to resist making this available to parents and students.  And, in response, I wonder if Secretary Rees would be willing to say whether she sees a way that this recommendation of Lisa Keegan could be implemented? [Laughter.] MR. HESS: Oh, it sounds good. MS. KEEGAN: You know, George, we were scheduled to have a drink later on tonight.  This may determine whether or not one or the other of us shows up. Nina already knows what I would say to this, actually. First of all--and I'm not simply saying this because now I'm on the spot--but I do think one of the most important things for the U.S. Department to do right now is hold on--hold tight--stay the course.  Do all those things--and collect the information that we've got about the current system and how it operates with this new overlay.  Because I can't imagine that we're much smarter than we were two years ago, except now we've got some really good data. I would tell you, I think one easy thing to do is to address this issue of the district benefitting from not using the dollars, George.  So, the district's supposed to set aside anywhere between 5 and 20 percent of its dollars for supplemental services.  We believe there must be some way to provide a statement to the districts on behalf of the--you know, the use that the kids are entitled to, that that's what those dollars are for.  And if they aren't used in the immediate first year or something, maybe there's another way to provide those for service.  But you can't just roll those back into your regular operational budget.  Because it seems to me that that incentive is too large for inaction. MR. ARCHIBALD: What about the public school choice--so Nina, did you want to comment? MS. REES: In order to regulate, we have to consult with a number of people first-- [Laughter.] --so I didn't want to prematurely call for regulations.  There are certain areas may need to be regulated.  We just haven't decided on that yet. The thing you need to worry about when you regulate is, are the unintended consequences on other districts that perhaps may be doing everything right, and not getting the results that they want.  Is it fair to ask those districts to hold onto this money throughout the year, even though parents, for whatever reason, are not taking advantage of these programs?  That's the big question that we need to see answered before we regulate. MR. HESS: Okay? MR. ARCHIBOLD: What about the public school choice. MR. HESS: Let's take that at the break, George.  Let's go to the next one--okay? I'm sorry--Emily, do have somebody back there? MR. RICE: Hi, Bill Rice, with the American Academy for a Liberal Education. I'd like to look ahead a few years, and ask this about the tutorial providers.  If the Department's emphasis on robust statistical evaluation holds; if the states in their approval process model their work on that same collection of pretty good data; if, in other words, the "n" under examination has to be pretty large, won't that inevitably favor the "bullies" and the "popular crowd"--that is, the large corporations, the Sylvans and so on--and drive out the mom and pops-the "geeks"- and others who won't be able to provide the sorts of numbers to make the quality of their work measurable? MS.          : I would say yes, but not even necessarily around the collection of numbers.  Even when I was talking with providers, I was talking with the person who heads up the tutorial services at Princeton Review and, in fact, he predicted that down the line what you will probably see is almost emerging of, let's say, the bullies and the popular crowd which even follows a current model that's going on in Los Angeles, whereby the districts are, in effect, the provider, but yet they will contract out with private providers, over which they can exert a lot more control than they could if they were just competitors with them at the district level. And what Rob Cohen at Princeton Review predicted was that the corporate providers--the large corporate providers would be in the best position to then take advantage of that contracting opportunity with the district, because they can scale up much more quickly; they already have a lot of capacity.  Both the large corporate providers and the schools districts--sort of regardless of whether or not they're partnering together--have kind of the greatest capacity to assume risk right now.  And so they are at an enormous competitive advantage. MR. HESS: Okay. Right here? MS.          : I'm sorry, could I just say one thing? MR. HESS: Oh, sure. MS.          : I mean, the law requires you to have research.  It doesn't say you have to have scientifically-based research--[Laughter.] --so--I mean, that's one of the big distinguishing factors of this provision.  And I think, because it's an innovation, Congress wanted to make sure that these innovators had time to adapt to the system.  And I would argue that some of these large, for-profit providers actually don't have the robust scientific-based research that, you know, our research office would require. So I think they're all in the same boat at this point. MS. FORTUNE: Hi, Iola Fortune from the Council of Chief State School Officers. We've heard a lot this afternoon about the role of the Federal government in regulating SES, and about district implementation.  And the question I'd like to pose to the panel is the other player in this, and that's the state education agencies, because we all know they have specific responsibilities with regard to implementing these provisions.  And I'd like to get your perspective on what you see their role is in implementing SES, particularly as it regards some of the district issues that we've been talking about. MS.          : I can at least start off with that. For my piece of it, I did talk to a couple of state officials, and also talking to people at the Education Department, it actually seems that, while it's unspoken, the states play a significant role in terms of quality control--potentially.  If it can be decided what makes up a successful tutoring program and what doesn't, it's up to the states to renew their approval for various providers.  So, through that capacity, in theory, states could be exercising a significant quality-control mechanism. But what states regulators, or folks at the state level, did say was that right now they don't have any guidance as to why they should accept or deny a proposal right now.  And so, on the whole, they're probably, you know, approving a lot more providers than maybe they would if they had stronger standards as to what constituted a good provider. MS.          : Rick, if I can-- MR. HESS: Sure. MS.          : Just--I don't know.  I mean, I don't know that that's true.  I do think that's a classic example of what Nina said before about the states' simultaneously wanting total freedom, and "tell us exactly what to do here."  I mean, how difficult is it to know what your job is when somebody says, "Approve a provider."  I just don't think that's hard.  I mean, I don't think you need a Federal regulation to understand what would be a good provider in your state?  They've got to meet the following non-negotiable criteria.  I just don't think that's that difficult.  Being good at what you do would be one, if you were going to set that up. And the other issue is that this is political like anything else.  The state office is either the first door or the first barrier to this program.  I've seen amazing, amazingly subjective reasons for approving providers.  You know, the person who's job it's been in Title I forever to decide who gets the grants now decides that those same people will become supplemental service providers, and anybody that's a for-profit organization probably doesn't know what they're doing.  You get that and you get the other side.  I mean--it's subjective. And, in my estimation, the set of criteria ought to be clear--that's a state role; say what you want.  And the state also, I think, ought to--as Michael said--fit this into their overall system, that when you're writing about how it you're going to achieve your goals in Arizona or Indiana, supplemental services is a part of that, and it becomes a part of the system everybody knows that the state is also advocating for and will make available. MS.          : I have one quick recommendation. A lot of districts are following the letter of the law but not the spirit of the law.  So anything you can do to encourage them to better implement the law is going to be very, very helpful in the long term.  You know, you are at an advantage, that you're closer to them, for one thing.  But, you know, making sure that they're doing it right, and they're actually fulfilling the promise of the law is very important.  And you're in the best position to make sure that happens. MR. HESS: Okay, we have time for one more question. Back here, please. MS. DAVIDSON: Barbara Davidson, with Standards Work. I'm wondering to what extent the panel--what you know about what's going on in terms of deadlines around the nation, and to what extent you--it's my understanding that under law, districts can impose a deadline, or cannot--and to what extent that's a problem? It seems to me that it would be ideal if parents could be signing up for this at least mid-semester, somewhat throughout the year.  So, I guess, what do we know about it, and to what extent it's a problem? MS. REES: We don't have district level information about enrollment periods.  Some districts have rolling enrollment, some of them open up the enrollment period one time a year and they don't open it again.  In a lot of places, the public pressure has prompted them to open it at least another time during the school year. So that's the kind of information that we don't have at the national level yet. MR. HESS: By way of keeping you interested in coming back tomorrow morning, of course, some of this information will be presented tomorrow morning, particularly for 46 of the districts in the Council of Great City Schools, Mike Casterly will be presenting some of this information in the a.m. MR.           : Rick, can I ask one question?  MR. HESS: Yes. MR.           :  I'd like to ask Nina a question. I'm just curious--you know, given all the choice that's going on out there--and, for example, I came across a story where the dozen least popular high schools in Chicago have lost between 66 and 72 percent of the students in their catchment areas.  So anybody who could choose--in fact a very high percentage of them got into selective schools, so the students who were capable of getting the test scores that would allow them to move, and who had the get-up-and-go to do so, and so and so forth, they had already left. And I'm just curious--what does No Child Left Behind add to the kids who are left behind for those--they've had many choices, they haven't chosen to operationalize them.  How does No Child Left Behind help them? MS. REES: Well, you're assuming that all districts have open enrollments and allowing children to move from-- MR.           : Just in that case--I'm curious. MS. REES: Which district are you talking about? MR.           : Chicago. MS. REES: I think it's just an added pressure point on the district to make sure that this funding that they're sending down to the district level to serve Title I students is there to serve their needs, and that the parents also have an option to go somewhere else if they feel that their needs are not being served. Anything you can do to let these parents know that they have the right to exit; anything you can do to let the folks out there know that there's a need to create better schools in a particular district is a step in the right direction. And, again, I wouldn't just look at the first year's implementation challenges and conclude that just because Chicago didn't have the capacity, this is not going to work as the reason why this law is flawed.  I think it's important to look at what happened in Chicago and try to bring more people to replicate successful models; build charter schools--lift the cap on charter schools, create more charter schools in a place like Chicago.  I mean, those are all options that would help for states to talk to districts about so that you are building room and capacity and meeting the needs of those parents who want to leave because of these requirements, or because of the districts open enrollment law. MS.          : Rick, just to answer Richard--I mean, this whole program--supplemental services--didn't exist before.  And in those schools where you say kids are still left behind, the school's still lousy and there's still kids in there--now, free tutoring; didn't exist before, it exists now. And we talk about vendors like they have no personalities.  I mean, the organizations that do this tutoring are actually very good, and have every incentive of going out and contacting parents and saying "This is something that's out there for you."  And that skill itself--the marketing skill, the contact with parents--is something that, traditionally, schools have not been good at.  And here's a whole new, fresh crop of people who can get paid to come in and help you do that. I don't think that's a small deal.  I think this is a much bigger provision than it's been given credit for--hopefully.  I mean, if we do it right. MR. HESS: Okay.  And we're going to have to make that the last word on this session. I'd like to thank the authors and the discussants. [Applause.] MR. HESS: Nina, I think we all want to thank you for your efforts to wrestle with this whole thing and make it work to the best of your ability. We're going to take a short break.  Checker is going to call us back--we're going to reconvene at 2:15 sharp.  And that will be the first of the state panels. [Off the record.] MR. FINN:  All right.  Everybody please take your seats.  We're going to try to keep this train running on time, at least for the next hour and three-quarters. I'm Checker Finn, with the Thomas B. Fordham Foundation, co-sponsors of this conference and project.  And delighted to be here.  Delighted that you are all here. I thought that was really a stimulating and spirited opening session.  Most conferences with panels, as you know, are deadly dull.  And you all are good to come and sit through ours.  Our panel is about to prove that, once again, that it doesn't have to be deadly dull. This is a discussion of the state level of implementation of the public school choice and supplemental services provision of No Child Left Behind. We've got four real scholars up here, with bona fide academic credentials.  They have done three case studies of individual states.  Three real, live, living, breathing states: Florida, Michigan and Colorado.  And we have an overview paper, by Robert Maranto. And because of the fact that we have an overview paper, we're going to start with the overview paper--with Robert--and then, after that, we're going to go alphabetically by author, which will have the effect of going to Florida, and then Michigan--sorry, and then Colorado and then Michigan. Everybody's going to have their 15 minutes and no more.  If everybody keeps to their 15 minutes, which they will, we will have pretty near 45 minutes for discussion, questions and answers at the end. I think this is a really interesting topic.  I've looked at the papers ahead of time.  You need to look at the papers, as well as listening.  You know the conference drill, which is to make everybody labor for months on an elaborate paper, and then give them 15 minutes to tell you what they found out--which is cruel and inhuman punishment for authors.  Unfortunately, the alternative is cruel and unusual punishment for audiences. [Laughter.] And having no very good choice here, we've sided with the audience--which gives the authors this unreasonable situation, made reasonable only if you also read their papers. So, with--and I'm not going to attempt to introduce these people, because they have mini=biographies in the programs, because they are all so distinguished that I wouldn't do justice to their lengthy resumes, and because I don't want to make any mistakes like Rick did. [Laughter.] So, with no further ado--Robert Maranto. MR. MARANTO: Thanks, Checker. It's an academic paper, but I'll try not to make it too terribly boring.  There's actually--if you dive through the mounds of statistics, there's some sort of fun stories. First off, I want to say--I want to recognize--back home, unfortunately a little under the weather, I want to recognize my spouse and co-author, April Rushing Maranto, and our son Tony--hi, Tony--who's there with her keeping her company and taking care of her--our four-year-old. Implementation, in the Federal system, is always incredibly difficult.  There's a huge line of political science research I won't bore you with, making the case that it's difficult and almost impossible.  In particular, the problem is that, in a Federal system you've got the Federal government and its varied organizations.  You've got the state governments, the local governments and their varied organizations.  And as the number of organization actors increases arithmetically, the number of agreements you need to reach with those organizations--with those actors--increases geometrically--exponentially--making it almost impossible to get anything done. In education, it's especially true.  We've got one U.S. Department of Education, but 51--including Washington, D.C.--state education authorities; 14,000 local school districts; 90,000-and-some schools.  And the history of ESEA--the Elementary and Secondary Education Act--demonstrates that.  There's a big literature on that from people like Paul Peterson and Michael Kurst, making the point that there's been a tradition, through the program, of the Federal government issuing regulations, states and localities ignoring the regulations, the Federal government issuing more detailed regulations--eventually, the states at least sort of buying in and using ESEA in part to sort of exert control over what they think of as lagging local school districts--and, as a result, over a period of years, some things actually being achieved. And I think that's maybe what we're seeing with No Child Left Behind.  In fact, I think, actually it maybe is working a little faster, at least on the school standards and accountability parts than one might think; although, unfortunately, slower, on the school choice parts of it, which is what I looked at--the school choice and supplemental services parts--or [inaudible]. Our charge is to look at what states are doing to get local school districts to implement those very tricky public school choice and supplemental services provisions of NCLB.  And so let me switch over to our PowerPoint--Jane, can you push it down one? It's actually a very difficult thing to do.  Like it says up there, like a Rube Goldberg mousetrap, the incredibly complicated school choice provisions of NCLB--school choice and supplemental services provisions--require six steps.  The state education authority, state department of education--SEA's the fancy name for it--the state department of education has to develop accountability measures.  Step two, it has to approve enough supplemental services providers--actually, that step comes later temporally, but for ease of presentation, we have it as step two. Step three: the state has to disseminate a list of schools needing improvement to the school districts and to the news media--and to the parents, ultimately, or get it sent out ultimately to the parents--in time for the eligible parents--that is, the Title I parents--to exercise school choice options. Step four: the local districts actually have to inform the Title I parents of their school's status--Title I parents in schools needing improvement--of their school status "in an unambiguous manner."  That's a big one--that's very hard to do--at least, hard for a lot of school districts to do; and, ideally, list possible alternative schools in a timely fashion.  "A timely fashion"--that is, before school starts.  If you tell the parents in October, gosh, I guess you can switch your kid from this school to this school, that's a very big sacrifice for the parents to make--yanking them out of one school into another one where they might start out weeks behind.  They're probably not going to do that.  In effect, school choice delayed is sort of school choice denied. Step five: the local district also is to inform Title I parents of their supplemental services providers and how to choose them. And, finally, the parents themselves have to choose to use or not use the school choice or supplemental services provisions.  Maybe they'll use them, maybe they won't.  But they're certainly not going to use them unless the state governments and the local school districts do those previous five steps. So this is a very tough thing to implement. Now, we wanted to see what the states are doing to get the districts to implement them.  So we did an e-mail survey of the 51 Title I coordinators in 50 states and Washington, D.C.--which henceforth will be called a state for ease of communication.  And we got 22 of them to return them.  I badgered them; actually, before we set out the survey I called each of them, and the non-respondents I called two additional times and also sent out two additional e-mails.  And through all that, either responding to the e-mail survey or agreeing to phone interviews, I was able to get 30 of 51--I was able to get respondents in 30 of 51 state Title I offices--so 59 percent, roughly. In addition, to try to get more sort of color to the survey, and also pick up states that I couldn't get otherwise, I interviewed 17 big-city education journalists in 17 different states.  And 11 education analysts/agitators--mostly from the state policy network--and so between those, I had 58 respondents; from 26 states I had respondent, from 19 states--I'm sorry, 16 states I had two or more.  And from nine states I couldn't get anyone.  I got all the big states except for New York.  Nobody would cooperate there. Now, one thing I want to emphasize about this is that I'm not a hundred percent sure of this data.  I can tell you what my informants told me.  And, in some cases, like in the numbers of supplemental services providers, I had my RA check it up on internet if she could to find them. But, basically, I'm a prisoner of my internet, as any kind of research like this is.  And we need to be up front about that.  And I'll talk more about that later. Okay--let's look at step one: the states' developing accountability measures.  States actually, to me, seem to be doing reasonably well at that, from my limited amount of data on it.  The mean state has designated 135-point-something schools as schools needing improvement.  That's actually about 7 percent of the schools in the states I had data for, which I think-- VOICE: Are we on the right chart? MR. MARANTO: Yes, that's it.  Yes.  Let's stay there for now. So, step one, they seem to actually be taking that seriously.  And some people told me interesting stories about chicanery going on in different states that I can talk about in the Q&A, but I have reason to think that states are actually taking that seriously. For example, the states with the greater percentage of schools needing improvement, that actually correlated pretty will with NAPE scores.  States with lower NAPE scores had more schools needing improvement.  So I think probably they are sort of doing that. Something I found interesting was the five state that had state elections last year were very slow to release their data.  Three of the five didn't release their list of schools needing improvement until November, after the election, that is.  And, in a couple of cases, informants said that that was related to the election.  I'm not too surprised at that.  And in an odd sort of way, that's a positive finding.  It says that people are taking these things seriously.  You know, all school reform is political.  Got to face it. Something I found interesting also was that the state--the SEA--the state department of ed officials I interviewed, 90 percent thought that No Child Left Behind was a lot harder to implement than previous versions of the Elementary and Secondary Education Act.  But, interestingly enough, 70 percent of them thought that it would actually improve education over the long term. And I got the sense, talking to them--I didn't get the sense, a lot of them told me this, although I didn't ask about it--a lot of them basically see this as a way to try to put lagging school districts in line; that this is a tool they can use to try to get people they think aren't quite doing what they should be to shape up.  And that's a positive thing. I found a lot of evidence they're taking the standards and accountability parts of this--not every state, but most of the states are taking it reasonably seriously. But the school choice parts of it, not so much.  Let's look at the steps here. State approved supplemental services providers: states are actually doing okay on that.  The average state--the mean state had approved 31 supplemental services providers--not a bad number, and some states do seem to be taking it pretty seriously; some local districts seem to be, others not so much. Steps three and four--not as positive.  You've got to get the list of schools needing improvement out early enough and see that parents are informed.  They don't seem to be doing as well at that.  Only 20 percent of the states I had data for actually released their list of schools needing improvement before August, and another 36 percent in August.  So only about half of states are getting their list of schools needing improvement out before school starts.  So, you know, that's going to make it very hard for parents--for Title I parents in those schools--to exercise school choice. Now, in fairness, a few states--Mississippi, Oregon, Nevada--got out the list for schools they thought would be below the line earlier, so those parents would have those choices.  But most states didn't. One-third of states give parents eligible for school choice under No Child Left Behind less than two weeks to make their choice.  That doesn't seem like enough time to me.  That seems like a very hurried system. And I might add, too, again, this data is not perfect.  A lot of the state informants would tell me, "Well, different districts do it at different times.  We think the average is about this."  So, you know, this is what they think it is.  You know, I think these shouldn't be considered really hard numbers. Five minutes--thank you.  Okay. What are states doing to try to see to it that districts are informing parents in a clear and understandable way?  Well, 19 state informants tell me that they check the letters that the districts send out to Title I parents telling them about their options.  And many of them told me that they've had to clean up some of those letters to make sure they're clearer. But 19 states don't--or at lease the informants I had indicated they didn't.  One state--Oregon--didn't initially, and after a crusading journalist got on their case, they started to.  So I gave them half a point for that. Twenty-two out of 37 states report that they do list sort of standard sample letters that districts should use to inform parents--Title I parents in schools needing improvement--of their choices.  So that's good. Now, the final thing--and I won't really--let's--I'm sorry, Jane, one more--hit it the last time. This is way too complicated, so I'll go over this really quickly now.  If you want more detail, you can read the papers. I developed a sort of an index--a measure--of how well states seem to be implementing the school choice and supplemental services provisions of NCLB.  And I basically gave them points--I won't run through it all now.  It's in your paper and it's up there--I gave them points for if they gave parents enough time--"timely announcement"; if they gave parents enough notice to pick schools--"parental notice."  If they--I had two different measures for if they oversaw districts' letters.  And a final measure for if they approved enough supplemental services providers.  I couldn't figure out if they were good providers or not, but at least I can count the number. And going through that measure, states are not doing terribly well.  The average state got about a 3.5 out of a possible 8 points of this measure.  And, now,  the states were graded more negatively--in states where I only have reporters as informants, they graded the states more negatively--not surprisingly--than where state officials were interviewed.  But even the state officials, their data, those states, I only had an average of about 4 out of 8. So, the states seem to be doing--my sense is they're doing pretty well at implementing school standards and accountability.  They're not doing terribly well--at least not yet--at implementing school choice and supplemental services, at least as I measure it. In fairness to the states, they've put so much energy--and I think they have put a lot of energy into developing standards and having lists of low performing schools and all that, there's maybe not much energy left over to implement the rest of the law. And I should also say that--again, long history of political scientists saying this, from school desegregation to 87 other policies--any time you try to get local governments to make a big change, it takes years and years.  And I think that the long-term--to make a very long story short--I think the long-term success of this, whether it works or not, will depend partly on whether local groups agitate to get school districts to do a better job; partly on whether the U.S. Department of Education pressures states and, ultimately, school districts to do a better job implementing this part of the law which is a big departure from past public policies. The final thing I'd say on that, there are some reasons to thing--I have in here--that even if the states and localities were doing a good job implementing this, maybe it wouldn't make that much difference.  But the fact is, until they implement it, we don't really know whether many parents would make use of it or not. And so I think that's probably a good point to leave on.  Thanks very much. MR. FINN: Thank you, Robert. Robert's grand table of state ratings can be found on page 35 of his paper, if you're interested in how the various states have done, according to his index.  And you may or may not be surprised to discover that the winners are Hawaii, New Jersey and West Virginia--on the Maranto index. They got six out of eight.  But it's a pretty interesting compilation of ratings of the 30-some states. And next, with no further ado, we will travel to Florida with Jane Hannaway. MS. HANNAWAY: On this cold day. MR. FINN: Thank you for taking us there. The State of State Implementation MS. HANNAWAY: This is a paper that I completed with Kendra [inaudible], sitting here in the front row, a research assistant at the Urban Institute. Why is Florida interesting?  Why it is an interesting case to look at school choice? Well, there are a number of reasons.  First of all, Florida has a long history of state accountability policies, and it has had, since 1999, an accountability policy that grades each of the schools, and that has consequences.  It has rewards for schools that perform well, and consequences for schools that don't perform well. It's an accountability system where all the kids in the state are tested; grades three through 10, and schools graded.  It also has a history of choice policies.  There's a number of choice policies operating in this state. It's also however, a year behind most of the other states in implementing the choice provisions and the supplemental education services provisions, simply because in the first year in which they were asked to grade schools for NCLB, there were no schools that were schools not making adequate yearly progress.  So this current year is the first year in which there were two years of schools not making adequate yearly progress.  So doing this paper was--you know, we were sort of sitting there on the front lines as the data was becoming available on line to try to put this story together. These are the grades that schools received from the state A+ Program.  This is the one that's been operating since 1999.  And if you look at this, you have to conclude that Florida must be doing something right.  I hope you can read these numbers--but let me point out some of the critical ones to you. In 1999, there were 8 percent of the schools received a grade of A.  In 2003, 47 percent of the schools received a grade of A. In 1999, 25 percent of the schools received a grade of D, and in 2003 only 5 percent received a grade of D.  So something--it looks like they're doing something right. Now, to some degree, these are corroborated by the NAPE scores, because the NAPE scores show that Florida is the only state in the country that, at least in fourth grade reading, made significant gains. Now, you can see the picture a little bit clearly without the numbers, here, where the pink line and line going down from 1999 to 2003 are the C and D schools, and the blue line going up are the A and B schools. These lines here at the bottom are the F schools.  So, relatively few schools in Florida have over the years received grades of F. That's the state accountability program, in place since 199; testing across the board; consequences for schools that don't do well; rewards for schools that do do well.  In many ways, it maps well onto the NCLB. In terms of choice policies--the kicker in the state accountability system is vouchers.  If a school receives an F in two out of any four years, then the students in those schools receive vouchers in the amount of the full state allocation for students that they can redeem at any public or private school in the state.  So far there are 619 students receiving such vouchers in Florida. Florida also has the McKay scholarships, which are scholarships for students with disabilities--special education students--and there are over 12,000 students in the state currently receiving those McKay scholarships. Florida also have corporate tax credits.  The way that works--and these are all state policies, go completely across the state--the way that that works is that corporations can get a tax credit for donating money to a foundation which then gives out scholarships to kids that they can redeem at other schools. So, when you look at this, Florida's the state that's been pushing choice for a long time; has one of the most comprehensive accountability systems in place; been pushing choice for a long time. By any measure, we would think this is a place where the choice provisions of NCLB are just going to slide right in to existing procedures and take off quickly. And I'll show you what happens--in a minute. Let me give you a little bit on the education profile of Florida.  Florida is a state that has large school districts.  The average district in Florida is over 30,000 students.  The average district nationally is only a little more than 3,000 students.  the average district in Florida is 10 times larger than the average district in the country.  Now, this is important when you're thinking about district-based choice, because it means within these districts, there is--theoretically, anyway--the possibility of a lot of different choices.  WE have large districts operating here. Florida also has a very diverse student population.  About half the students in the state are minorities.  About 25 percent Black; 21 percent Hispanic; and another 5 percent other minorities.  So about half the student population are minorities--very diverse; has important implications for NCLB.  So, both of these characteristics of the state again make it a very interesting case to look at when we're looking at choice provisions. This shows graphically the difference in Florida, nationally, with the size of the school district, where the blue bars are for Florida--and you can see that almost 40 percent of the districts in Florida have more than 25,000 students, and only a few percent nationally are so large. Again, when we get to the bottom of the distribution, most of the school districts in the country are small; important implications for choice provisions that are district-bound in Florida, not the case. Let me compare, in a little more detail, the similarities and differences between NCLB and the A+ Accountability Program--very similar, as I've said.  Both of them test reading and math.  In addition, Florida tests writing.  But both cover reading and math.  They both use the same test; the Florida test, the FCANT test, which has been in operation for a number of years.  SO they're all graded with the same metric. The proficiency threshold in both systems is the same.  You have to achieve a grade of 3 to be considered proficient in both the state accountability system and in NCLB. Both systems target needy students.  NCLB calls for separate reporting by subgroups of students; by racial-ethnic background, and by disadvantaged status; LEP status--students with disability.  Florida also targets--the accountability targets needy students, though they target low-performing students, not by racial-ethnic categories. Both systems have participation requirements.  NCLB requires that 95 percent of the students in the school be part of the system--part of the testing and accountability system.  A+ requires 90 percent, but you need 95 percent to get an A Both systems have choice consequences.  In NCLB, two years of not making adequate yearly progress, you get hit with choice.  In A+, two out of four years of getting an F, you get hit with choice. So they're very, very similar systems.  In fact, I'm sure that there was probably discussion, over family dining room table, about how to set up NCLB--[Laughter.]--and Florida probably influenced the structure. However, there are some critical differences.  The Florida system gives schools points.  And it gives schools points not just on the basis of achievement, but you also get points for learning gains.  NCLB just looks at the percentage achieving proficiency.  So you have both things going on in the Florida system. The students that are targeted, as I mentioned earlier--in Florida, it's the low-performing students that you get extra points for, without any regard to race-color.  Students with disabilities are not included in the Florida system.  The standard-curriculum students are in the accountability system.  Participation requirement is a little bit lower, except for the A schools; and the choice that you're allowed is a public-private school choice.  But the similarities, I think, are more striking than the differences. Let me show you how it plays out, however. You can see these numbers. Seventy-five percent of the A schools failed to make AYP--75 percent of the A schools; 87 percent--it says 85 here.  The counts are still coming in because of people questioning their status--appeals.  But any way you look at this--you know,  85 percent, 87 percent of the schools in the state didn't make AYP this year?  Seventy-five percent of the A schools did not make AYP this year?    You can imagine how heads were sort of spinning in Florida--at the school level, at the parent level, at the district level--about what is going on here. This shows t he reason that schools did not make AYP.  And, as you can see, the bar that is the highest are the students with disabilities.  The largest fraction of schools failed to make AYP because of that criterion. When I talked to people at the state level, asking them "Are you feeling a lot of pressure, a lot of conflict, a lot of confusion because of the differences in the results that you're getting from these two state systems?"  And they said, no we actually see them as somewhat complementary, because what NCLB is doing is highlighting pockets where we just aren't doing well in a different way. But, again, remember: that's the part of the system that's not being tested.  This is new--that's not being held accountable with A+. LEP students also, which are a large fraction of the population, are not doing well with AYP.  Florida is not doing well with LEP and AYP.  Their explanation for that, however, was that the way the state system works, they allow multiple tests; multiple forms of assessment for LEP students.  And they were re-thinking how they're going to do that testing in order to get it more streamlined and more effective. However, you can't ignore the fact that over 60 percent--about 65 percent of the schools that did not make AYP did not meet the math criterion for Black students. It may not be quite as bad as we think, because the number of criteria that are not met by schools is not that high.  So that the A schools, for example, failed on only one to five criteria.  Sometimes it's participation rate.  It may be, you know, students with disabilities.  But it will help them focus--there are very few hurdles for them to make it through. So it's quite possible that the A schools will pass next year.  And there are a large number of A schools that didn't make it; that this 769--that these A schools that are performing well by other criteria, and just missed it by--could be participation rate, will make it next year. But--and the other point I want to make is because of Florida's structure--because of the large district size and the diverse population--Florida schools, on average, probably have more hurdles to clear in order to make AYP than schools in other states. This is just looking at the A schools, and you can see that it's students with disability is where they aren't making it.  LEP, they aren't making it--and, again, with Black students. Okay, this year there were 45 schools--one minute--that did not make--where choice was required.  Next year, even if the 935 A schools that did not make AYP in 2003 make it in 2004, then about half of the 1,200 remaining schools will be required to offer choice.  So there still could be a tremendous amount of choice going on in Florida. We did two case studies to see how this is playing out at the district level.  We looked at a huge urban district--Miami-Dade.  Miami even has an assistant superintendent for choice.  It has 400 schools; fourth largest district in the country.  You know, if choice is going to work, it's going to work there--you've even got an assistant superintendent managing the thing--and a rural district with only three schools. Miami-Dade, without taking into account A+ or NCLB, 12 percent of the student population--and this goes to Rich's comments this morning--were already exercising choice through the district; 71 magnets, 25 charters, 14 controlled-choice, McKay tax credit, and they also have a voluntary public school choice program that's going to be taking off in 2005.  So there's a lot going on in terms of choice. I talked to the assistant superintendent, who said, with all the other programs going on, the effects of NCLB are negligible.  We looked even more closely at Miami Edison High School, where about a third of the student population were already exercising choice through these other mechanisms, and only 2 percent used NCLB.  And it's even unclear why that 2 percent were doing it, and why they didn't take A+, because this was also an A+ choice program where they were going-- MR. FINN: You're going to have to save the rural example for the Q&A. MS. HANNAWAY: Okay. MR. FINN: Sorry. MS. HANNAWAY: They have to use supplemental services.  There's no choice. MR. FINN: [Off mike.] There's no choice.  They have to use supplemental services. For more on rural choice, wait for the Q&A. Thank you very much, Jane. We are now going to journey to the chilly mountains of Colorado, with Alex Medler. [Comments off mike.  Inaudible.] MR. MEDLER: [Off mike.] Okay, first off, I'm going to be giving a case study based on the State of Colorado that concludes--oh, sorry. My case study is on the State of Colorado, but I included case studies of the Denver public school system--one of the state's largest districts--and of Ignacio, which is one of the state's small, rural districts.  So part of the story is about how different this is from urban to rural settings. I sort of have four points I want to go over.  First is a reiteration of what Jane has already said about differences of state accountability systems and identification.  Basically, I will say: ditto, that there are multiple systems in Colorado that appear to match with NCLB quite closely, but the identification patterns that are coming out are different, and there are more schools making AYP, and they don't always match up well with the state's other systems. Next--that there's uneven district-level implement, and the state may not have the capacity to correct for it.  We'll look specifically at supplemental services provisions, and at the choice challenges.  And then I'd like to basically use a little crystal ball with you, recognizing that this data is very rough, very early--as everyone has said.  But there are some anecdotal stories worth sharing, to give us pause to thing. So--first, in terms of accountability systems, the important part is that all these are in place now.  NCLB did not replace any of these.  In fact, Colorado tried to use NCLB to coordinate them and bring it together under a value-added pilot program, which the Federal government would not allow them to do. So they had hoped to use NCLB as a lever to make this more coordinated when, in fact, it's continued in a relatively uncoordinated fashion. This is based on preliminary data, but this looks at AYP and non-AYP schools, cut up by the way that the state's previous accountability system divided them. MR. FINN:  What's "SAR" stand for? MR. MEDLER: "SAR" stands for "school accountability reports."  So they divide schools into "unsatisfactory," "low," "average," "high" and "excellent."  Their threshold for "unsatisfactory" is very low.  It's the 2 percent of the low-performing schools.  But it's worth pointing out that the schools not making AYP--which are in pink--they do sort of follow the trends for how you'd want them to, but there are anomalies going on. Out of the "excellent" schools, there are 11 that they predict will not make AYP.  And out of their "unsatisfactory" schools, there's another 10 or so that will make AYP.  So, for the communities, that's a little confusing. In terms of the school improvement status, Colorado has a bit of lead because they jumped right into choice and supplemental services, using school improvement status designations from the earlier version of ESEA.  In 2000-2001, they had 154 schools on school improvement.  Last year they started out with 87 schools that remained on school improvement status.  Those were the ones that should have provided choice last year, and should have provided supplemental services this year. Just one heads up: they had seven schools that got off that list by taking away Title I funding from those schools, or by reconfiguring the grades that they served so that they no longer counted. But the important part now is that the number for this year goes up to 800 kids--or 800 schools, excuse me, which is roughly half the state's schools are estimated to not make AYP this year. If you divide the previous 80 schools--most of this analysis was done before this last data was released--but if you look at the 80 schools that were on school improvement last year, they're not evenly distributed.  Denver has 34 of them--so they have the vast majority.  Many of them--16 out of 27--actually only have one school not making school improvement.  So there's a lot of districts where the amount of choice or supplemental services they have to provide is pretty limited. This graph shows the percentage of school improvement schools they had before as a total of each district's schools.  Now, the ones on the left, with "0 to 10," or "11 to 20" percent of their schools as school improvement schools probably have another school serving the same grades that did make AYP, or that was not on school-improvement status.  The bulk of them, however, probably don't have any other, better performing school serving the same grade, or at least not very many. So, if those patterns continue with the next rounds of school improvement designations, there could be a lack of schools to choose form--just like in Florida. In terms of district implement, it really matters where you are and what you had in place.  It's exacerbated by an urban and rural split that's affected by existing policies for both choice and accountability.  And all of that depends on local support.  So, to look back to the stories about Colorado trying to do things using some flexibility, so far they haven't gotten much flexibility from the Feds.  Unfortunately, further implement is going to depend on good will, and I'm not sure the good will is being created by the state and Federal interactions as they've gone so far.  So that could be a problem. And, finally, those problems will create opportunities for resistence.  And there's some very creative resistence going on--I'll get to it later--but primarily dealing with the notification requirements. In terms of supplemental services, there's some relatively straightforward challenges.  The first panel talked about one of them, which is determining which providers are effective.  Colorado, with a penchant for following the Federal lead, actually does use the scientifically-based effect standard for its RFP--its request for proposals--for providers.  And they've had challenges in trying to figure what counts as effective for those providers, and they are looking for models and guidance.  And that would be one area they could get help from the rest of the education community. Next, the challenges in recruiting providers to serve the entire state.  They have far dispersed small districts, but most of the providers would like to serve the Denver metro area where there are lots of kids closely located.  And they're having problems getting the diversity of delivery strategies for rural areas--and we'll go into that a little bit more. And I'd also like to add that they had an excessive level of political pressure lobbying for particular vendors.  So they had questions about what made a vendor effective; they had calls from high places saying you certainly ought to approve these people, they look great.  I don't care if they don't have data. [Laughter.] The state will add that they did not cave in any way to this pressure, but the state did approve 23 out of 25 applicants in this last round. [Laughter.] So, either the applicants are all doing very well and the questions of effectiveness are an issue, or this political pressure could be coming into play. I've got some basic descriptive data on the providers.  First off, I mentioned it varies by region.  So, the Denver metro area's in the green.  They have about 21 providers.  Now, keep in mind it's the providers who decide which districts they will serve, not the state.  And the rest of the Denver metro area averages about eight providers.  If you're in non-metro Colorado, there are four. And then in terms of the services that they deliver, 23 out of 31 current providers use sort of hands-on tutoring; five use internet-based systems; and two use kind of a hybrid. Now, there was a lot of talk before about turning supplemental services into tutoring, because that would be more clear.  Unfortunately, for rural Colorado, it would also be a lie.  The rural providers--the three or four that will work outside of the Denver metro area--are all internet based.  So perhaps I'm giving short shrift to internet-based tutoring, but there is not a hands-on, person-to-person contact available for rural Colorado's schools. In terms of the base of operations: out of the 31, 17 are from the state as non-district providers; five are--as was mentioned earlier--districts approved to make provisions.  Most of those are in Denver, and they are existing programs, like a--but not the schools.  So they would qualify.  Another nine are from outside the state.  You might think that those nine are the internet providers.  That's not the case.  There's lots of those national models and big corporations that are hiring people, as well as a variety of delivery strategies.  So there's not too many correlations amongst the regions and the types, other than that rural communities have few and they tend to be internet based. Oh--and back up--in terms of grades, the largest number serve K through 12.  Then they also have a number serving K through 5 and K through 8.  The providers that focus on secondary schools are very few in number.  And the costs actually do vary a little bit with trying to serve all the grades. I think, though, there are some more interesting conflicts and challenges on the public choice front.  First off, the timelines are really quite difficult to deal with.  And there's problems of combining it with existing programs.  But these depend on what's in place locally. In terms of timelines, you take a state assessment in March.  You actually don't get AYP designation straightened out, right now, until just this last week of the following year.  You participate in an open enrollment program that spring, and you get to enroll in your choice school 17 months after your school was taking the test showing that you were in a failing school. Needless to say, the Feds did not accept that timeline and told Colorado to go back and adjust it, which they said they would, but obstacles include things like getting their multi-year, million dollar contract with the assessment firm to renegotiate it; or negotiating with districts over appeals.  There's also a lot of questions about which students actually have to be included.  WE assume it's all children, but it's all children who were attending that school the last year.  The assessment system isn't designed to track actually who attended where when.  So challenges like that. In terms of combining it with existing programs, it's mostly an issue of the notification; adjusting transportation or eligibility and timelines. The more significant challenges really are not so easy to fix.  These include creating choices where none existed--and particularly for rural Colorado. And then the next part is creating action where there's no support.  So most of these are things that are not going to be easily fixed, and they won't go away. In terms of having choices where none exists, the rural areas are very important.  Colorado's 178 districts have 124 in which there's only one elementary school.  So if your school is failing, I don't know what your other choices are going to be.  So this is a real problem for Colorado, and I think this case study is probably indicative of issues that will come up in other rural settings. So what they find is few supplemental services providers, and almost all are on line.  They find intra-district choice is logistically difficult and depends on rich families with SUVs to get over mountain passes or travel long distances in the plains.  And then there's few, if any, options in the district. Rural options do exist.  Colorado has 25 charter schools in rural areas--or 22, which is about 25 percent of the total.  The state has encouraged them to provide supplemental services right away, but they don't have many providers or many takers. So there are options in the rural areas.  I don't want to paint too bleak a picture.  But there are few, and they depend on cooperation. If you look at urban areas--Denver's a good example, and it's much like Miami-Dade.  They have 30 percent of their students already exercising choice under a variety of mechanisms.  What they had to do was basically spend about %50,000 on a new mailing that was actually quite customized; took four months of work by four people to pull it off, and the result was--and they had to adjust their transportation, and every time they had a question about how to provide choice, they answered it in the affirmative that would provide the most choice. The result was, 75 students in the first year, and 365 in the second.  That's roughly 2 percent of the districts choosers as a whole, and 2 percent of the students in those failing schools. If $50,000 seems like a lot, you should keep in mind, Denver's Title I funding went from about $15 million to $22 million.  So they have a $7 million increase, and they had a $50,000 out-of-pocket expense for the mailing, plus the cost of transportation. So it's not necessarily that there wasn't money to do there.  There wasn't willingness to provide it.  They already had a lot of things in place. But the key part, for me, to get to sort of the crystal ball in this is to share with you some of the opportunities for resistence that these stories will tell you.  I divided it into things that districts with the school-improvement schools will have opportunities to do, and those who might receive it. On the confusing front, we have this requirement to notify, and the state's jumping on that as fast as they can.  I have one letter to share with you, in which a school tried to argue that school-improvement status was a wonderful honor--kind of like a blue-ribbon status-- [Laughter.] --and they were really excited--so they can confuse people. Secondly, they can ignore it and just take no action.  There really are no legal requirements. And then, because of all the contradictions over time and deadlines and identification, the districts' version of shock-and-awe is delay-and-blame.  So if they don't do anything, and the state can't figure out how to tell them which schools need to provide choice or what options are good, they'll hope it will go away, and they can try to pass the buck. If you look at the receiving schools, there's a couple of even more disturbing things I warn about--but I'm willing to take bets for dinner that these things will happen in places in the next year. First off, there are disincentives for inter-district transfers.  If you're a small rural Colorado district and you had 29 LEP students, if you accept inter-district choice you might get the 30th, which will bring you to the point where your disaggregated figures have to be reported, and your schools or districts will not make AYP.  So, whereas Colorado now funds and it has incentives for inter-district choice, and 30,000 students exercise it, I'd posit that in the future that number would go down, and that districts will become reluctant. Secondly, in a nation where there's a nascent anti-testing movement, there's a real opportunity for a small minority to do some very powerful civil disobedience.  In Boulder there's already a group of parents who won't take the state test.  They think it's a waste of time and invasion of privacy and unnecessary.  The schools have to have 95 percent participation.  They have records now of 95, 97--somewhere around there.  So it's going to take about 2 percent of the parents to refuse to participate to put their schools into a status of not making AYP.  That will either provide them with choices, or allow them to deny those choices to anyone else.  And if you combine that with a motivated public and a small minority, I'm willing to bet dinner that some districts will have either one of those things go on, where people will start to protest NCLB by not taking the tests. MR. FINN: [Off mike.] Thank you, Alex.  That was a model presentation of a very interesting paper, and we appreciate it. To the chilly north, to the state of Michigan, with David Plank warming up in D.C. by leaving Michigan. MR. PLANK: Actually, I just came back from Brazil, so I'm actually freezing to death. Despite being a true scholar in Checker's characterization, I don't have a PowerPoint presentation--so I apologize. [Laughter.] There are advantages and disadvantages to being the last paper of the day.  And I think the disadvantages outweigh the advantages today.  My opportunity is really to reinforce themes that have already been presented by others. I'm going to say a couple of things that are new.  The first is, I just want to sort of draw a couple of things together, and say that NCLB--the success of NCLB depends on two assumptions.  The first is that schools receive clear messages about their performance; that they receive messages that they can make sense of.  And the second is that the sanctions to which they're exposed are predictable and effective.  And in Michigan, neither of these conditions has been met. And this is to sort of to provide an overview of what I'm going to say later--but these may be short-term start-up problems, or they may suggest that the incentives and sanctions that are part of NCLB are not going to work very well. We incline to the latter view for three big reasons.  The first is that the message that NCLB is sending to schools is weak and uncertain.  The second is that the sanctions are not, in the short run at least, very effective at all.  And the third that incentives for compliance are weak, as are sanctions for noncompliance.  And state oversight in Michigan is minimal. I noted from Robert Maranto's paper that we got a score of 4, but a 4 is virtually invisible, in terms of what the state is actually doing. I'll talk a little bit about school choice.  This is really to echo things that both Alex and Jane said.  Michigan has--that Richard said, also--that Michigan really has a flourishing market for school choice.  In Michigan, in contrast to most other states, all of the money follows the kid.  There is no local share.  And when kids move from one school to another, from one district to another, all the money goes with them. We have more than 180 charter schools, and very wide latitude for intra- and inter-district choice.  This is especially true in the major metropolitan areas in the state, where most of the failing schools are--97 percent of the failing schools are, of course, located. In Detroit, for example, kids can already choose from an array of regular public schools.  They can choose magnet schools.  There are more than 40 charter schools in the city, and a couple dozen more in suburban districts on the boundaries.  And there are dozens of neighboring school districts that accept inter-district transfers. So that over the course of the last six years, the Detroit Public School District has lost 20,000 kids; 20,000 kids have already exercised their opportunity to make choices.  All the other districts in the state are smaller, but the same thing is going on in Grand Rapids, Flint, Pontiac, Lansing and other cities. It's not clear that--the NCLB choice provisions really make a marginal contribution to the opportunities that kids face.  Whether or not they make a marginal opportunity to the opportunities that kids face, we can state as a fact that in Michigan no kids have been identified as moving under NCLB, and no money has been expended to transport kids. The market for supplemental educational services is just emerging in Michigan, as others have talked about in other states and other contexts.  I think we would argue that this is certainly the most promising feature of NCLB.  In Michigan there are 44 providers who have bene approved; 15 of those are in the public sector--intermediate school districts, regular two districts and two public schools.  And I'm not sure what the public schools are about, but there they are--and 29 private sector providers. As Alex, I think, talked about in the most detail, there's a great deal of skepticism on the part of educators about whether supplemental services are a good idea, whether they will work.  And there are many opportunities to resist sort of letting kids know about supplemental services and encouraging them to participate. We also found some wariness on the part of potential providers--private sector actors who are reluctant to get into the market for providing services because they're uncertain about the economics; whether they can put together a business plan, whether they can take advantage of economies of scale that would make it worth their while to provide supplemental educational services. There is some action on this front, though, in Michigan.  In Flint, about a thousand children will begin receiving supplemental educational services this month--I guess they would have started this week, in fact.  In Detroit, there are slightly over 3,000 kids who are participating in supplemental educational services.  That's of about 75,000 kids who are eligible for supplemental services. In our third case study district there are no children taking advantage of supplemental services, and the district has no plans to make those available. The second point that I want to talk about is the weak and uncertain message--and there's a lot about this in the paper, and I won't go into much detail now.  Michigan had the distinction, back when the Federal government published its first list of failing schools across the United States of having over 1,500 failing schools; something on the order of 20 percent of all the schools in the nation were in Michigan.  Negotiations with the U.S. Department of Education quickly reduced that number to 216.  The subsequent negotiations with the Department discovered that there were, in fact, another 540 schools that were failing, so the number then leaped up to 740.  But now some schools have made adequate yearly progress for two years, and the number is now 341--but, like Colorado, this year's test scores have not yet been released, so the real number--which will include high schools for the first time--remains a mystery. The fact is that several schools--many schools in the state--have been on and off the list more than once--for reasons which they, of course, will perceive as arbitrary.  And so, you know, the message that they're receiving from No Child Left Behind is not a strong and consistent one. Another quick point about Michigan is that there are strong penalties in the law for early compliance.  Michigan has been measuring adequate yearly progress since 1994, as a part of our state accountability system.  What that means is about half of what I would characterize as the "real" list of failing schools in Michigan, are in their sixth year of sanctions.  So that the second-third year--the idea that you would have to give choice in the first year, and supplemental services in the second year--these schools are facing reconstitution or closure.  And they're not thinking about supplemental services; they're not thinking about choice.  They're thinking about "Uh-oh--you know, the state's coming in to take us over."  So the sort of escalating sanctions are not operative in Michigan. I already mentioned the fact that, like other states, our scores have been dramatically delayed.  They were due in May last year.  We're now in--as I look at my watch--January of this year, and we still don't have the scores.  They should be out on January 30th. And, as with other states, there are competing education policies that undermine the effectiveness of No Child Left Behind.  We're under a state mandate to implement--a legislative mandate to implement our own accountability system which is, in many important respects, divergent from the requirements of No Child Left Behind, and integrating those two has proven to be an Herculean task. And our governor--our recently elected governor has made a priority of improving schools that are not performing well.  And her strategy for that is to bring a lot of social services into the schools, provide additional instructional resources and so on, which means that in schools that are on the failing-schools list--in many of the schools that are on the failing-schools list, enrollments have gone up rather than down, because parents can have access to additional services that they can't find elsewhere. The third point--the third big point is that incentives for compliance are weak.  Districts in Michigan are essentially deciding what they want to do about No Child Left Behind.  They're deciding how to comply.  The Flint school district, for example, is in the midst of an internal debate about whether the law really requires transportation or not.  Their current conclusion is that it does not.  As a footnote, Flint is also facing bankruptcy and has decided to eliminate transportation for all kids, including, one supposes, No Child Left Behind kids. So that the third--we've agreed to keep one of our districts anonymous because there's only one school, and different people in the district are talking about each other, and we didn't feel it was our right to say who they were--but that third district has done nothing to comply with No Child Left Behind.  Flint and Detroit have treated compliance largely as optional.  They have provided services for some kids; transportation for no kids; and have engaged in negotiations with the state and with the Federal government around those decisions. The final--gosh, I've got time, eh?--cool.  All right--tons of time.  I learned this from--I heard Checker Finn once say that, well, you know, if I've only got half as much time as I need, I'll talk twice as fast. And I think I learned that lesson. [Laughter.] State oversight in Michigan is virtually invisible, as I said before.  The state has been very careful to comply with mandatory provisions; sought, at one point, to avert a provision about limited English proficiency, and was slapped down by the Department, and has been very careful to comply fully ever since.  But we would argue that there really has been no serious engagement with the law, in terms of trying to encourage districts to comply, or to really change the kinds of services that they provide.  And there has been no meaningful oversight of district actions. There are a couple of reasons for that.  The first is a simply lack of capacity.  Under our previous governor, the size of the Department of Education was reduced by 90 percent.  There are about 150 people left in the Department, and they simply can't pay attention to this stuff; particularly when they're in the throes of a struggle to create their own accountability system which is different form No Child Left Behind.  So they have a bunch of--even--with only 150 people, they have a bunch of competing priorities. We're also in the midst of a very deep budget crisis in Michigan, which has resulted in disproportionate cuts--as they do--in state administrative services.  So the Department is fully preoccupied with trying to manage budget cuts.  The governor is bringing some services back into the Department which were taken out by the previous governor, which means they're in a turmoil of administrative reorganization.  And they're trying to reply to this state mandate which says that they have to have their own accountability system, which has turned out to be extremely complicated. One of my scholarly mentors--as I was becoming a true scholar, one of my scholarly mentors said that you should always end a paper with a smiley-face. [Laughter.] You shouldn't just give people the bad news, but you should always end on a cheerful note.  So I always try to do that. It's really hard to do in this case. [Laughter.] But I'm going to say three things.  Sort of--what would work?  What sort of steps might we take to move forward with No Child Left Behind?  I want to emphasize a point which is not one of the themes of the conference, but which is perfectly obvious to everybody in this room, which is that it is crucially important to get the measurements right; that the measurement systems on which we are relying to make judgments about schools are fragile and capricious.  And everyone knows that.  They identify the wrong schools.  They miss the boat frequently.  And getting that right--making sure that we're picking on the right schools--and there are schools that deserve to be picked on, but making sure we focus our attention on those schools is absolutely critical. The second is that we think that there really is great promise in supplemental educational services.  And that market really has not sort of blossomed in Michigan.  There's been resistence on the demand side, there's been resistence on the supply side.  And figuring out how to make that market work--how to get those services to the kids who need them, we think, should be the top priority for people working with No Child Left Behind. And, finally, I'll talk about another theme--I guess the choice and supplemental services part didn't work very well, in terms of the smiley-face--[Laughter.] --so the other part is that in Michigan, as in other states, the distribution of high-quality teachers is exactly opposite to what it should be if we were to make schools work well.  And working on incentives to get high-quality teachers into low-quality schools should be a top priority. As I close, I neglected at the beginning to acknowledge by co-author, who is here--Chris Dunbar, my colleague from Michigan State University.  And he and I worked on this paper together. Thanks. MR. FINN: Well, this has been absolutely fascinating--if I say so myself. What am I hearing in this set of papers?  I'm hearing that two years in--and it's only been two years, let's do keep that in mind--in three states that actually have what you might call sort of a culture of choice already going on their own, we're seeing, actually, very little choice occurring as a consequence of No Child Left Behind; very little change actually occurring as a consequence of No Child Left Behind--this despite, as I say, all three of these states having something of a history of being enthusiastic about choice policies of various kinds. I think we're hearing major clashes with states' own accountability systems, and the NCLB accountability system.  It sounds to me like we're hearing not a lot of integration of the NCLB choice provisions with the states' existing choice provisions--numerous in all three cases. We're obviously hearing about lots of resistence to the enthusiastic implementation of the NCLB choice provisions.  We're certainly hearing about lots of fumbling with the implement of NCLB choice provisions. We're hearing about lots of supply issues of alternatives, and in the case of the public school choice and of effective and operative providers in the case of supplemental services, these supply issues are obviously more acute in rural parts of the country than they are in urban, I think we're hearing. I think we're hearing about lots of unintended consequences; things that nobody ever exactly thought were going to turn out to work the way they're working.  And, finally, we're hearing, it seems to me, very weak incentives and sanctions for any of this to change. And having said all that, two years in, if I've got this at all right, based on these three limited case investigations--if I've got that right, the question we've got to ask going forward is: how many of those conditions are likely to change with the passage of more time?  Or will, two years from now, if everything sort of stays on its present trajectories, will we be coming to the same tentative conclusions?  Or are the conditions that we've been describing--you all have been describing--the kind of thing that is likely to change with experience and with time and with, let's say, the predictable amounts of good will and energy. So what I guess I'd like--we're going to have plenty of questions from the audience, but what I'd like to ask all of you, kicking this off, is, first of all, was I hearing you wrong with those kind of tentative conclusions; and, secondly, what's your prognostication about which of these--let's call them limiting conditions, if any, are likely to change going forward? Let's try tackling that, if you don't mind, in the original order.  So let's go back to Robert, then Jane, then Alex, then David.  And you don't have to give a long speech.  I'm just interested in your sort of top-of-the-head reactions to those: am I hearing this right, and what about it is going to change--do you think? MR. MARANTO: I have absolutely no crystal ball on this.  I mean, you know, political scientists are really good at predicting the past. [Laughter.] We don't--you know, I can tell you what caused a given war or something like that.  We're not good at predicting the future. I mean, I'd say if there's a good analogy, it might be school desegregation in the '50s and '60s, where you'd have varying levels of compliance versus non-compliance, district by district, state by state, and a lot of the success would depend on the activities of outside actors, including the U.S. Department of Ed, the state department of ed, including the business community; including groups like the Black Alliance for Educational Options; including the courts, potentially. I think that if, over time, you have more and more states, ultimately districts, moving in this direction it might create more momentum. Something that struck me in doing my interviews was that the state education authorities could speak very competently--they'd thought a lot--about standards.  And they supported standards, and they sort of believed in standards.  It seemed to be an idea--as political scientists say, an idea's whose time has come; that the policy has been softened up; people are used to the idea. They're not there on choice yet.  They don't have the same familiarity with it; they don't have the same belief in it.  And I think it would take some years of agitation for them, as a whole, to move along, because this is a very--and we've been debating choice, really, for 25 years--our standards, rather, for 25 years--which I'd say is about 15 years longer than choice.  So I guess I'll sort of leave it that. Long term--I'm cautiously optimistic. MR. FINN: The sobering part of your civil rights analogy, of course, is that 50 years after Brown, we're still fussing about desegregation issues. Jane? MS. HANNAWAY: I want to correct a couple things about Florida.  First, I didn't observe any resistence in Florida-- MR. FINN: Okay. MS. HANNAWAY: --at either the state level or the district level, nor did I observe any fumbling in Florida at the state level or the district level.  They turned around things fast. Schools were informed.  They do the tests in March.  Schools were informed--districts were informed in June.  They were given an early warning of which schools looked at risk; were given the final information in June.  July 1, Miami-Dade had letters out in Spanish, English, Creole--and all sorts of outreach going on.  So they were moving fast.  So I can't fault them on anything. What's going to change?  What should change?  I really worry a lot about the conflict between a state system and the Federal system.  And the reason I worry about it is the sorts of reports that I heard back from what parents were saying.  And I think is when parents are getting this conflicting information about whether their school is performing or not performing, what they are likely to do is just discount any performance information on their schools, which could undercut both state accountability systems and Federal accountability systems. What I would like to suggest to the Feds is that they go to states like Florida, and like Colorado, that have comprehensive systems in place; give them a little bit more flexibility.  Let them put in value-added measures, or give schools credit for learning gains, and use them as laboratories, rather than setting up these conflicting systems that, I think, hurt the whole movement. MR. FINN: So your emphasis is on the mis-match in the underlying accountability systems. MS. HANNAWAY: Yes. MR. FINN: Okay. Alex? MR. MEDLER: I'd agree with your summary.  It was adequate--or appropriate, for Colorado. I'd also say that the alignment for standards and identification between state and Federal requirements needs to improve.  But I think it's going to do what it has in the past, and that the Feds will be moving to the states' versions, rather than the other way around. I think NCLB is a codification of what lots of states were doing a few years ago, and there are states like Colorado and a few others that want to use value-added assessments, and other things that they think will improve their identification and the Feds are going to have to come around and say that's okay. So, I'd reinforce the idea of the Federal government going back to the states that are leading, or at least trying to lead, and let them experiment, rather than make them maintain multiple competing ideas at the same time. In terms of the mechanics of implementing choice and notifications, I think these are issues where we're being unfair to criticize people too early.  Out of the 14,000, 15,000 districts, they're figuring this out.  The states are figuring out how to get the people in line, and they'll do that over the next couple of years. So the real grotesque forms of resistence that are really obvious, I think those will be reduced considerably.  But there's lots of other forms of resistence that are more problematic.  That's why going back to the identification issue sooner rather than later I think is important. MR. FINN: David? MR. PLANK: Yes, my view on this is probably pretty obvious. I'm quite pessimistic about the law--and primarily because of the fragility of the measurement systems on which it's based.  And I think that that's going to require the kind of flexibility that Alex and Jane already talked about, where states' efforts to get this right have to be encouraged rather than discouraged by Federal authorities. I also want to say that I think that the school desegregation analogy is a useful one.  The goal of leaving No Child Left Behind is going to be a really hard one to achieve, and the idea that we could do it by opening a few charter schools, or providing a few hours of tutoring after school, I think, is--I won't use too strong a word, I'll just say delusional. [Laughter.] MR. FINN: "Won't use too strong a word, I'll just say 'delusional.'" Okay. We've got two wandering mikes, and we've got a [inaudible] audience, and the floor is open. Go ahead.  Remember to identify yourself, and speak clearly, and ask a question. MS. SEGAL: I'm Peggy Segal with the North Central Association Commission on Accreditation and School Improvement. My question is this: we've been talking about supplemental services and choice as two separate initiatives.  And I want to a what-if question. If the opportunity were supplemental services preceded choice, could that be a way to engage, collaboratively, school districts in working with providers to address the needs of individual students to try and build the capacity of school districts in a positive way, and then enable choice as a next-year alternative, rather than what we have now. Or, if you could address, maybe, the interchange between the two initiatives to build more of a collaborative--that, Jane, you were talking about--instead of a conflicting accountability system. MR. FINN: So you're also asking if it's backwards.  If the supplemental services-- MS. SEGAL: One, do you think it's backwards?  Could that be changed? And, two, could we use it strategically to build toward leaving no child behind, and capacity building of school districts, too. MR. FINN: Who would like to take a swing at that.  Alex? MR. MEDLER: I'd like to offer one piece.  I mean, the Colorado director of the supplemental services, her line was, "I'm so tired of people looking at this as a sanction.  Supplemental services is something we should want for all these kids." So, rather than reverse it, I just think the supplemental services ought to be universal and automatic--and certainly, as necessary in places where choice really is going to be an even harder option to provide. MR. FINN: Jane? MS. HANNAWAY: It's--when I first rad the bill--and I won't say I read every word of the bill-- MR. FINN: Only 1,100 pages long. MS. HANNAWAY: --I mean, it did strike me as odd.  You know, I thought, "What?  Choice is coming before supplemental services?"  But I think the reason is probably political, and I'm not sure we at this table have that much to say about it because the third option--the supplemental services--is one where the dollars leave the public school district.  In the choice option, all the dollars stay within the public school district. So, even though I think logically it does make some sense, on a political front I don't think it's a go. MR. FINN: On that side of the room--who've you got?  Cindy Brown. MS. BROWN: Yes, Cindy Brown.  I'm an education consultant.  And I'm doing some work for the Citizens Commission on Civil Rights, which is looking at the implementation of the choice provisions, as all of you are.  I think we may have a little more data, but we're not quite ready to say what our conclusions are. But I do have a question, for David especially, about Michigan. If you--what did you say? 20,000 kids have left the Detroit school system and are doing inter-district transfer?  Or charter schools. My question is: are they paying--these are poor kids, mostly low-income kids.  Are they paying transportation for these kids?  Have you looked at the race of the kids that are leaving and the income levels?  Because, frankly, I don't see how Michigan gets away without doing No Child Left Behind choice, because even if they were just going to pay transportation costs for the kids who are using the Michigan system--I mean, I don't quite get it, and it doesn't sound to me like they're following the rules of the law. And I'm interested--we're interested, especially, in some of the desegregation effects that might be happening, which is going to less where you can't do inter-district transfers.  But here, you can. MR. PLANK: Yes--just on the data question, we do not have data on the race or income of kids who move, because the state has not collected those data up to now. Those data have just become available this year, and we're beginning the analysis, but we don't know.  So the segregation-desegregation effects are not analyzable at this point. On the transportation question, there are concerns in Michigan, as elsewhere, about sort of two-tiered choice; that some kids are paying for their own choice--which all kids are doing now.  But some kids might have their choices subsidized by the state, and how would districts deal with that, and so on.  That has proven to be an obstacle to either districts or the state funding choice for anybody. MS. SEGAL: [Off mike.]--violation-- MR. PLANK: Well, that's--so the question then becomes does the state or the Federal government step in and say, no, we do need to provide support for this.  It has not happened yet. MR. FINN: Are you going to [inaudible] Alex? MR. MEDLER: Just briefly.  If we go back to the desegregation analogy, I'd argue that with NCLB, it's creating 50 different Coleman reports that will show the damage that's being done, and the people who have been hurt by something and there will be a raft of litigation for school quality, with choice as one of the remedies they seek from the courts. So I think that analogy is apt, and you're likely to see it play out again and again. MR. MARANTO: If I can just piggy-back onto that--I'm doing a lot of interviews now in affluent suburban districts--including the one I live in--and it's been interesting how many officials in those districts have told me--and these are districts with very high test scores, but they have a few minority students who are not doing very well, and some of them have candidly said, "Oh, my gosh.  We're going to have to start teaching those kids."  And that, I think is kind of a good outcome. MR. FINN: Yes--and, remember I can't see around the corner, so you're going to have to catch--if you're around the corner, you're going to have to catch Emily's eye so she can catch mine. Go ahead. MR. BASIL: Hi, I'm Dave Basil, from the Social Science Research Group. Democratic legislators have been arguing for some time that NCLB is underfunded, and that more money is needed. Now, obviously, the amount of money is an issue; what it's used for and how it's allocated are issues.  But looking at it from the state level, what do you make of that argument?  Would more many make a difference?  And, if so, in what ways? MR. FINN: All right.  Go ahead, Jane. MS. HANNAWAY: One of the big differences--interestingly--between the A+ Accountability System and NCLB, under A+ the failing schools--the low-performing schools--are getting in excess of $1,000 per student from the state.  In addition, their districts are pouring more money into them.  So the state system, which seems to be having beneficial effects--at least on the basis of the gross data that we're looking at so far, and on the basis of the NAPE scores--we seem to be seeing gains. Now, in the NCLB, the low-performing schools are going to lose money.  So it's a bit difference between the two systems. MR. FINN: Do-- MS. HANNAWAY: Can I just add a little bit? MR. FINN: Yes. MS. HANNAWAY: That new money that's going in is often very targeted money.  So it's money that's gone in for class-size reduction.  Or it's money that's gone in for--specifically focused on a reading initiative, in Florida.  So it isn't sort of general operational money which, I think--through which there can be lots of leakage, and you see very little effect.  So it's very targeted money going in. MR. FINN: We'll come back to money in just one second. Answer me a Florida question first.  On those four years of declining numbers of failing schools, or increasing numbers of A and B schools--are you pretty confident that the state has maintained a constant standard for making these judgments?  Or is there any possibility of grade inflation during that four-year period? MS. HANNAWAY: Yes, I think there's a possibility of grade inflation.  There are also ways to--there have been, and the state corrects them--there have been ways to sort of game it.  For example, the writing score--you know, originally, you'd have to get an F in reading, writing and math in order to get an F.  Principals that I talked to quickly learned that the way you get around is you can get--you can boost up writing.  If you get kids to write an essay, first paragraph starts "first," second paragraph, "then;" third paragraph, "finally" they would pass the writing test-- [Laughter.] --and the school would no longer be an F school. Principals figured this out real fast, and then the state changed the system.  So there's some gaming going on on both sides. But you know, there's the NAPE scores, too.  So it was the only state that we saw where significant gains in reading-- MR. FINN: An external measure. MS. HANNAWAY: --yeah. MR. FINN: Yeah--okay. It sounds like it's raising an entire state full of true scholars, if they are all going to be able to write essays like that. Back to the money question: David or Alex? MR. PLANK: I have two answers to the question.  One is that in Michigan we actually have a test going on of whether more money would make a difference, because the governor's strategy is quite the opposite to the NCLB strategy; which is that she's putting comprehensive social services and a lot of instructional support into schools that are not making good progress.  Because of budget constraints in the state, she's only able to do that in 18 schools.  We have 216 schools that are on the list.  So we may see some evidence about which strategy works better in the next couple of years. The second thing--I'll just reiterate a point that I made in closing, which is something that I think would make a tremendous difference would be getting high-quality teachers into low-performing schools.  That's likely to cost a lot of money, and generous funding for that would probably be a good use of money. MR. FINN: Alex? MR. MEDLER: I'd only add that the increases in Title I that happened when NCLB was passed will quickly recede into the minds of everyone in the states and districts--and especially if they're treated as sort of a one-time bump, rather than an ongoing increase. So, without further increases, I think the people in the states and districts will quickly come to think of what they got this year as the baseline for which there were no added amounts of money.  But if you look at it from before it happened, they did get an extra amount of money. MR. FINN: What were those Denver numbers? MR. MEDLER: For Denver it was from $15 to $22 million, for the state it was from $77 to a little over $100 million. MR. FINN: Over what-- MR. MEDLER: Over two years. MR. FINN: Over two years. MR. MEDLER: Yes.  So they're getting between 25 and 40 percent increases in basic Title I funding.  And then because the implement costs, specifically, of these--and sanctions--are relatively low, they're doing okay.  But if you look at all the other costs, or the state revisiting its contract with the assessment firm, there's a lot of other things here--or interventions we'd like to be in place, of course those are going to cost a lot more. But the nuts and bolts are not costing nearly as much as the increase that happened. MR. FINN: Next?  Right there with the tie?  The yellowish tie?  I'm color-blind, so we'll hope it's yellowish. MR. McGUINN: Patrick McGuinn, Brown University. In listening to the case studies that were presented today, it seems that the states you focused on really had fairly robust accountability and/or choice systems in place prior to the passage of No Child Left Behind.  And you all show how that leads to a particular set of implement challenges; the laying effects and the need to get two systems to coexist. But it strikes me that the real purpose of No Child Left Behind was to focus on those--and prod those states--that didn't really have much accountability or choice systems in place, and I wonder if you all could comment on what you've heard out of those kinds of states that really have to, in a sense, start from scratch, as opposed to deal with this coexistence problem.  What unique challenges are they facing? MR. FINN: Robert should start, and then others, from their general knowledge of other states. MR. MARANTO: I found a correlation between, basically, the states that were in compliance with the previous ESEA--the 1994 version--what was that called, Goals 2000?--whatever it was called-- MR. FINN: The Improving America's Schools Act of 1994. MR. MARANTO: That one--already forgotten. [Laughter.] MR. MARANTO: The states in compliance with that seemed to see implementation of NCLB as a little bit easier--at least the state officials there did.  Beyond that, I can't say a whole lot, although I would suspect--well, from interviews I sort of got this--I would suspect that states that didn't have a previous accountability program in place would be ones that would try harder not to implement this one. MR. FINN: Alex? MR. MEDLER: I did another piece looking at the Education Commission of the States' data on state implementation of the testing and assessment parts of NCLB, and then did a cluster analysis of it.  And it's really quite interesting. The states that are farthest ahead in implementing NCLB by that data tend to be controlled by Democratic governments, with larger minority representations, and larger school districts.  And the states that are farther behind had Republican governments, smaller districts, and whiter students. So I think it's interesting if we look at this as sort of a--if this is the Coleman reports of the coming desegregation analogy, then the Democratic states are going to have a lot more wedge issues, and some very contentious politics coming up if they continue to implement NCLB more aggressively than the stats that have Republican governments and primarily white populations and small school districts. MR. FINN: There will be some interesting politics coming up. MR. MARANTO:  Let me piggy-back on that too.  I found, kind of interestingly, the states that had the greatest percentage of low-performing schools were actually states whose officials were most optimistic about the impacts of NCLB.  So my sense is that a lot of states are willing to sacrifice some short-term embarrassment for what they see as potential long-term gains. MR. FINN: Next?  Go ahead. Is there anybody--Emily? MS. McCLURE: Checker?  Around the corner--Phyllis McClure, independent Title I-- MR. FINN: Hello! MS. McCLURE:  --consultant.  Hello. The law calls for a single state accountability system, but the Department lets states--or accepted dual systems in states that were very chauvinistic about their own system, and where the political costs would have been prohibitive for the Federal government.  But it seems the costs are pretty high for having a dual system. The problem is more fundamental than the choice system.  These state accountability systems from what NCLB requires.  They function to identify the schools that need help, whereas NCLB's function is to jack up student performance over years to a pre-defined level. So if you accept what the states are doing, you're not going to achieve the outcomes--the goals--of NCLB. MR. FINN: Was that a question?  Remember, we're only entertaining questions. MS. McCLURE: The question was about the costs. MR. FINN: Okay. Has anybody got a thought on this? MR. PLANK: I would just reiterate that what the states were doing was really quite diverse, and that I'm--I have to look at Colorado's existing model and say it's sort of flipped from the way you've described it.  So I think the states are really all over the map.  There's a lot of things going on, and it's hard to characterize it. MR. MARANTO: One word: federalism. MR. FINN: Federalism. Okay.  Mike Lieberman.  You're a patient fellow.  But you do need a mike.  Even you need a mike. This kind of mike, Mike. MR. LIEBERMAN: I have a question for David about the implications of union contracts. For example, if the union contract said, as some in Michigan do, that the district would not contract out any unit work with the permission of the union, I would guess that could be--being forced to implement it would be an impairment of contract. So I'm wondering whether you have run to any union contract issues in implementing the Act. MR. PLANK: We did not look specifically at that question, but I can give you two little anecdotes. One is that the Flint school district is strongly encouraging supplemental services providers to hire Flint schoolteachers to provide the services, because they know the kids.  That's the argument. And the second is that, at least we were told by Detroit BAEO--Nina Rees talked about the Black Association for-- MR. FINN: Black Alliance for Education Options. MR. PLANK:  --thank you--okay--the Black Alliance for Education Options--that at least one of their events, they were harassed by union members who were worried that they would encourage children to leave the Detroit public schools, which would cost jobs. But in terms of contract impairment, we have not looked explicitly at that, and don't know how that would work out. MR. FINN: Okay? MR. BARTON: Trent Barton--with myself. MR. FINN: With myself. MR. BARTON: With myself. MR. FINN: All right. [Laughter.] MR. BARTON: In the Commonwealth of Virginia we went through a very interesting battle in one of our largest counties, with reference to charter schools and the first wave of implementation of additional choices.  Sixty-six percent of the schools in that district failed to meet accreditation in the state, and they also failed to meet AYP. They learned a great deal of obstructionist and diversional attention techniques with the school staff interacting with the community. You mentioned the Colorado letter.  I find the Colorado letter to be humorous in nature, because it basically does promote themselves that they actually failed their thing. How can we change the--I guess, how can we work the best change of that mentality, without going in there, because they feel like we're going in there to tear them apart and attack them.  How best can we address that, and how best can we get around the political land mines that are currently there. MR. FINN: Remember these are true scholars, not political strategists. But if any of them would like to take a swing at this, I'm sure he would welcome it. MR. MEDLER:  Well, I'd back up a little bit to--a lot of school reform research that looks at the need for buy-in for people as far down the food chain as you can get in order for anything to work. And I think that the aggressive, hard-nosed approach, from the top down, is likely to alienate people farther down, which will make implementation harder.  And I believe that a little bit of softening, and a little bit of flexibility at the top would actually increase the potential--I'm not naive about it--but there might be, on the margins, more buy-in and more support at the local level; and that because enough people can delay-and-blame, or ignore-and-confuse, I think there would actually be better outcomes in the long run with greater flexibility than there are by trying to focus on avoiding shirking. MR. FINN: Any others? MS. HANNAWAY: Just going even further down the food chain to the parents--I mean what I observed in Miami, again, was not resistence on the part of the school people, but the parents were starting--there was a backlash that was starting from the parents. MR. FINN: On account of? MS. HANNAWAY: That they were getting conflicting messages.  They were feeling their school was picked on.  You know, they were saying, "Listen, we just jumped two grades in the state system, and now you're telling us--you're telling these parents, you're telling my friends that they should be leaving the school.  We have a terrific school." MR.           :  How dare some bureaucrat in the state say my school isn't good?  That's what I heard a lot. MS. HANNAWAY: And conflicting information. MR. PLANK: And just to add to that, we're about to identify high schools that are not making adequate yearly progress in Michigan, and the ones that going to be at the top of that list are the best high schools in the state, because of participation rates.  And those parents are not going to be happy when those letters come out. MS. HANNAWAY: Can I just say one other thing on high schools-- MR. FINN: Sure. MS. HANNAWAY: --where, actually, I did hear some resistence that was occurring. MR. FINN: Ah-hah--even Florida. MS. HANNAWAY: Even in Florida--and this was sort of--there were people at the state that were talking about it, and they weren't very happy about it.  And they weren't very happy about it, I might add. But it's--what they predicted, from what they heard, you know--scuttlebutt around--what they predicted is that we would start seeing very few high schools not making adequate yearly progress, because districts have discretion in how they do their economic survey to determine who gets the Title I money. And, historically, high schools have been in there trying to get the Title I dollars.  So they've been, you know, doing everything they can to show that they have low income kids.  Well, if it's coming with this clout, then all they have to do is change how they do their economic survey; they can have kids, you know, ask for free and reduced-price lunch.  Well, if you do that in a high school, kids aren't going to ask for free and reduced-price lunch, and you're no longer going to be a Title I school. MR. FINN: Very interesting. Emily?  Julian? MR. BETTS: I'm Julian Betts from U.C. San Diego, and the Public Policy Institute of California. MR. FINN: And a presenter tomorrow. MR. BETTS: And a presenter tomorrow.  I shouldn't be doing this. MR. FINN: Okay--as long as you're brief. MR. BETTS: I'll be brief.  Are those my seconds?  Okay. I want to talk about the potential for a race towards the bottom.  Somebody has put out a report--I think it was Achieve--showing vast discrepancies between NAPE test scores and the percentage of schools that are labeled as failing, basically because some states have higher standards. Did any of you hear talk about this at the state level; that, you know what?  We're setting our standards too high. MR. FINN: And wasn't there, at least, Alex, some scuttlebutt early on about Colorado fiddling with its standards; going to one notch lower than it was going to use for itself? MR. MEDLER: Right.  Yeah.  That was part of their initial negotiations. I mean, it's true that their first standard at the state for the school accountability report system is actually a fairly low one.  But they're trying to make progress, and I think there's going to be continuing conflict as they try to resolve it. I posed this to a lot of state-level policy makers; state board members and legislators.  And most of them thought, actually, that most parents so far were just disengaged from it, and if they could get them to engage with one system, that would be a great coup. So part of this, at this point, is still sort of a wonky issue, where we're like, "Oh, well, these policies are not meshing well together."  Well, for mom and dad, in a lot of schools that doesn't matter yet.  I think as it--in a year or two, it will more. MR. FINN: But Julian's narrow question is whether you're hearing anything about states' reducing their standards of proficiency in order to make it under NCLB. MR. MARANTO: I mean, I've heard from scuttlebutt about that from some states, but what kind of strikes me about this is: the state-level officials are setting the standards, and the local level officials will look bad if they don't meet them.  So, really, you know, until at least the politicians get involved, I think a lot of those state bureaucrats are going to want to have some fairly real standards out there, is my sense. MR. FINN: David? MR. PLANK: In Michigan it's not [inaudible].  The reason why there were 1,500 schools on the list in Michigan was because the state standard for proficiency was that 75 percent of the kids pass the test in four subjects.  The reason why that went from 1,500 to 216 was because the current standard is approximately 40 percent of the kids passing tests in two subjects. MR. FINN: Hmm. MR. PLANK: So, it's not a rumor.  It's true. MR. FINN: Jane? MS. HANNAWAY: I know--I was going to say "I know in Florida--"--I don't "know" in Florida.  It was reported to me in Florida that when the results first came out, especially about the great discrepancy between A+ and AYP, that there was discussion at the highest levels--the Governor and the Commissioner--about what to do about the standard.  And a decision was made: we're going to hold our standard.  And then they went public in saying, "We are holding our standard." So they're trying to turn it around to their political advantage.  But the consequence is that they're saying, "No, we're not going to lower them." MR. MARANTO: Let me jump in again.  States that try rigorous standards always lower them somewhat.  If you declare 70 percent of your schools as failing, politically you can't do that.  And if all states are sanctioned, then no states are sanctioned because there's no shame involved--frankly. You can't, in my view, politically--and education's political--public education is political--you can't sanction most of your schools.  You can't do it.  You've got to start with standards that will get, you know, some smaller percentage of schools.  Otherwise, politically, the whole system will come tumbling down. MR. FINN: You mean, if everybody is deemed to be failing, then nobody is failing, and we get into a version of Moynihan's "defining deviancy down." Alex, did you have another comment? MR. MEDLER: I was just going to add another analogy, that it quickly becomes the 55 mile an hour speed limit of education reform. [Laughter.] MR. FINN: Yes, bring your mike to him. MR. BRANDT: My name is Ron Brandt.  I'm currently part-time at George Mason University. I have a rather vague follow-up question to a comment by David Plank that says the problem is the fragility of the measurement. I live in the state of Virginia.  I've been opposed to the standards of learning and the tests that are used for a long time.  But when I come to meetings like this one--and I've been coming to some--and hear what people are saying, I being to say: okay, the state is purely limited in the amount of time it can ask students to take tests, and the amount of money that can be invested in testing.  This is about as good as the states can do. Under these circumstances, what do we do about the fact that these are very limited tests that are being used to make these major decisions?  I don't have--it's a vague question.  I don't have an answer to it myself. MR. FINN: So, you're going to the question of the adequacy of the testing instruments that are being used for making these judgements, which is a very specific question.  It's a little bit afield from today's conference, but I welcome comments. MR. BRANDT: David introduced it. MR. FINN: All right.  Then he can say something more about it. MR. PLANK: Well, the one thing I would say is something that others have already said, which is that we should at least begin to look at learning gains, rather than status on tests.  And I think that, you know, the Florida system includes that.  Alex, did you say Colorado does include it? MR. MEDLER: One pilot program. MR. PLANK: Michigan was moving towards it until No Child Left Behind, and now we've backed off because it would be costly and difficult to do under the circumstances where we have to do the other. MR. FINN: Yes?  Stand up.  He'll bring you the mike--if he can get there. MR. HENNIG: I'm Jeff Hennig from Columbia. My question goes to the notion--which I think is central to the legislation--that we need external incentives to kick states and other actors into action.  And yet we've heard, I think, in a number of ways today, that in a lot of respects the states have been ahead of the game here, in terms of putting choice into place, in terms of higher standards. So my question is: what's been the incentives that the states have been operating on previously? MR. FINN: Are you really asking did we need NCLB? MR. HENNIG: I'm asking about the premise that there are not other incentives in the system; incentives coming through the indigenous political institutions and democratic processes, and professional incentives of bureaucrats actually wanting to do a good job. MR. FINN: Robert? MR. MARANTO: Many bureaucrats want to do a good job.  I think most do--having been in government.  But not all do.  And sometimes doing a good job is not an adequate incentive.  Something that I noticed when I was in government was frequently senior executive would threaten to privatize part of their agencies, not necessarily really intending to do it, but just sort of put people on notice that they needed to shape up a bit.  And often that would have an improvement. My sense is that state-level actors are using NCLB as kind of a--you know, don't blame me, it's the Feds, but we have to do this.  Come on.  Let's get together. I see them sort of using it as that sort of wedge to develop the information systems, and to develop the changing cultures to lead to improved performance. So I think it's almost sort of a symbolic thing.  You could do a whole ritual of education reform thing about it. MR. FINN: Jane? MS. HANNAWAY: Yes, that's an interesting question, because it seems to me--and there are a lot of people in this room that know more that I do--but it seems to me that if you look over the last 10 years or so, it's the low-performing states that have been the most aggressive, in terms of education reform.  And I would put Florida in that box.  I would put Texas in that box.  I'd put Kentucky in that box. So there clearly was some, you know, residential pressure for improvement.  I suspect part of it was economic.  You know, the way you attract business in is you have to have a good school system. So the question is, what about the middling states?  Was anything happening in the middling states?  So I think there were some incentives.  I think they came form the top.  I think they were economic.  And I think there had to be political leaders that really bought into them strong as at least a means to an economic end, if not an end in itself. MR. PLANK: I think what No Child Left Behind is to kick the external stimulus up a level.  In districts and schools, the stimulus, through the '90s came from the state--at least in Michigan and the other states we've talked about--for reasons that Jane just suggested.  What we've done is to raise the level from which those external influences are coming, but the phenomenon, I think, is still the same. MR.           : And I'd go back and reinforce the sort of incrementalism of Federal change; and that this is the second version of Goals 2000; and Goals 2000 was the codification of America 2000.  And those were led by governors.  So it's right that it's bumped up from Governor Bush to President Bush, but that's also a reflection of 49 other governors and what they were doing.  So this has been more an aggregation and a bottom-up than people are willing to admit.  And that's, in part, because the states were doing these things already, or things like them. MR. FINN: That's actually a good not on which to end.  I've been admonished by the management that even though the schedule doesn't show it, you're supposed to have a six-minute break before the next panel starts promptly at four o'clock. So we will now declare a six-minute break.  Please join me in thanking a terrific bunch of panelists. [Applause.] [Off the record.] MR. HESS:  Okay.  Please find your way to your seats.  We're going to get started with the last panel of the day.  Please take your seats.  Take our seats, please. Well, it's been a long day.  And I know we've all been exposed to a lot of information.  And Checker and I had the privilege of reading these papers a couple times, in advance of the conference, and I still learned new information, especially during the last session. I'd like to thank Checker for doing a great job of chairing that. This final session of the day--we're going to go for the next 90 minutes--is going to be a discussion of the research we just heard at the state level. With us we have five folks who are eminently well qualified to share kind of their experiences and their reactions to the research--all of which has provided with them about a week ago. Personally, I'm looking forward to hearing what folks who actually are out in the field working with schools and districts and service providers, what their reaction is to the work we've just seen. I'm going to go down the list.  Starting at the far right is Jeff Cohen, the President of Sylvan Education Solutions.  Next, we have Mitch Chester, the Ohio Assistant Superintendent for Policy Development.  To his left, we have the Representative Jane Cunningham, who's Chair of the Education Committee in the Missouri House of Representatives, and is also Chair of the Education Committee for the American Legislative Exchange Task Force.  Next, we have Tom Houlihan, who is the Executive Director of the Council of Chief State School Officers.  And, batting fifth, but number one in our hearts--[Laughter.] --we have John Stevens, with the Texas Business and Education Coalition. We're going to give--each of our speakers is going to take about 10 to 15 minutes to share their thoughts and reactions.  We'll then have about 20 to 25 minutes to take Q&A.  At that point we'll adjourn for the day and we'll be able to continue the conversation in a more leisurely environment. We're going to go in the order they're on your program--so, Mitch, please begin. MR. CHESTER: Good afternoon. First of all, that you to the authors of the papers for very interesting and very important set of papers. I want to make reference Bob Maranto's paper, in particular, where he uses the term "bureaucrat" several times in the paper.  So I guess I'm the bureaucrat up here, being a state education agency official.  He also refers to "savvy bureaucrat," at one point, so I'm going to aspire to be a savvy bureaucrat this afternoon. I think I'd like to do a little bit of framing for talking about the papers and their focus on choice and supplemental services.  And I want to say a couple of things. First of all, these are studies of implementation.  They don't address the impact of the implementation on student achievement.  I mean, if we start with the premise that what this is all about--No Child Left Behind and accountability systems, remedies--they're in aim of improved student achievements; students' learning to read better, do math better. So I want to make clear that what we've looked at so far is implementation, not impact necessarily--unless we think of impact as getting a lot of kids in supplemental services as sort of the impact. The second thing--I want to say that accountability systems, when they're done well, create an opportunity.  They create a space and a time to rethink how we do business.  The challenge is to respond to that space and time in ways that are constructive and productive for the long run; not short-term fixes that may, in fact, slow us down.  And so some of the discussion in the last panel had to do with what we've seen in the first two years.  I think what will be interesting, particularly around choice and supplemental services is states are going to enter a new plateau next year--many of the states--depending on how they phase from the old ESEA into the new No Child Left Behind requirements around adequate yearly progress--many states will be in the position of going from the current year, where the number of schools that have to offer choice or supplemental services is relatively manageable--in Ohio it's 190 schools, to the next school year, where it could be as many as 600, 700, 800 schools.  And I don't think Ohio's unique in that respect. I want to postulate a frame for thinking about strong accountability systems, and then I want to locate choice and supplemental services within that frame.  And when I say "strong accountability systems" I'm thinking about systems that are successful in promoting improved curriculum, improved instruction, improved opportunities for students to learn, and thus improve learning. And I'm going to identify three touchstones.  One is the accountability system sets a good target.  The second is that the accountability system is symmetrical and aligned.  And the third is that the accountability is fair. I'm going to use this framework to unpack the four papers. Now, what do I mean by setting a good target?  The accountability system results in schools and districts focusing their efforts in constructive and coherent directions.  Whole-school accountability is part of setting a good target.  The whole school feels accountable for the progress of students, not teachers of particular grades where the state may test.  Disaggregation of results is another notion around setting a good target, and I think its a key feature of No Child Left Behind--an important and a right-on feature. Standards that are attainable; that require folks to stretch, that are nonetheless attainable are critical to setting a good target.  Once there's a critical mass of folks who have concluded that the standards are simply not attainable, the accountability system's going to quickly lose its ability to move things in the right direction.  There will be a lot of kind of game-playing and unintended consequences. And the last piece around setting a good target has to do with questions about the size and the nature of rewards and sanctions.  What kind of rewards and sanctions?  How intensive, how intrusive do they need to be before they are true motivators?  And this is core to some of the questions that were raised in the paper today? So one touchstone for the discussion of the papers is setting a good target. The second touchstone is that notion of "symmetrical and aligned."  And when we talk about "symmetrical and aligned," it has a couple of dimensions. First of all--and this is the one that, again, we focused on in the papers, it's symmetry of the state and the Federal requirements.  To what degree are those brought together in a unitary system, versus to what degree do they exist as separate requirements that result in separate inferences about schools?  The more they're brought together, the more aligned they are, and the clearer the message is to folks. We can also talk about symmetry in regard to who's accountable in the system.  Do students have a sense of accountability around the same measures that adults in the system have a sense of accountability?  In fact, one of the papers raised that question. And then, certainly the issue of the lumpiness of many of the accountability systems that we've fun up 'til now.  When we tested only in grades three in six, or four and eight, we tended to get these situations in schools where only the third and sixth grade teachers felt accountable, and you had principals moving staff around to try and put the strongest teachers in the tested grades.  No Child Left Behind, I think, does a very valuable service in spreading that accountability, that responsibility across the grades. The last of the three touchstones is that issue of fairness.  And fairness has at least three dimensions.  The one we'll focus on is opportunity to learn.  I mean, to the extent that I'm held accountable, do I have an opportunity to meet those standards? Second is adequate resources, and a third has to do with the quality of the measurements in the system.  The reliability, the validity of the measurements has to do with fairness also. So how can we use this framework--how might  we think about choice and supplemental educational services within the context of this framework? Well, much of what was discussed in the papers had to do with the notion of the size and nature of the consequences.  Do what degree is choice, or supplemental educational services, the right touch in terms of a consequence that's going to be motivating, going to be an incentive for folks to think about doing their business differently to try to avoid having to go down that road? In each of those three papers, we've got conclusions that say that probably the impact of choice--not so much supplemental services--is probably a lot less in the states than a lot of folks might have presumed.  My eyes were open in reading those three papers, to the degree to which choice is really infused throughout the system in those three states.  And it caused me to think about Ohio also in that same vein. There's a lot of different manifestations of choice going on in the states that were studied, as well as in Ohio.  So, to what degree is the implementation of the No Child Left Behind choice piece really something that puts some teeth into the system.  And in each of those three case studies, there were questions raised about that. In terms of "symmetrical," the alignment--certainly, in the Florida paper there was a lot of discussion about the fact that, under the state measures and the accountability system, states may earn A's, but under the Federal they may be identified as not meeting adequate yearly progress; the confused messages; the implications for parents who are trying to size all this up, understand their options--"should I move my child out of this school, shouldn't I?"  Under one set of measures, this school's measuring up pretty well.  Under another set of measures, I'm being told that I have options to leave this school.  Very confusing, very tough to sort out. And in terms of fairness--that third dimension of a sound accountability system--the notion of choice and supplemental services as increasing students' opportunity to learn is just so critical here, and just needs a lot of work.  I mean, we need to look at the degree to which choice and supplemental educational services really do provide for students who are not achieving as well as we would like improved opportunities to learn and thus leading to better student outcomes. I'd like to identify a couple of additional considerations, one of which was only peripherally touched on in the papers, and that's the issue of students with disabilities.  In many states--Ohio being one and, certainly, the data we looked at for Florida--when you look at the schools that did not meet adequate yearly progress, in a large proportion of those schools, students with disabilities are one of or the reason that they did not meet adequate yearly progress.  What are the implications around choice and supplemental services, if many of the students that we need to teach to read and do math better are students with disabilities?  And to what extent is the system geared up to address those needs? I think I'm going to stop at that point.  Thank you. MR. HESS: You make it so easy to be the moderator.  Thanks, Mitch. Next up is Jeff, I believe. MR. COHEN: I think I'm the only one with the PowerPoint.  Good afternoon.  I really am thrilled to be here, and I appreciate being invited to offer the perspective of a provider.  I also sort of relish the thought of being able to read professors' papers.  I don't get an opportunity to do this often, and it sort of felt like I get to exact some revenge on all the college papers-- [Laughter.]--and business school papers, and law school papers I wrote and had to defend. So I read the papers last weekend during the timeouts to the football games--[Laughter.] --assuming that that's when my professors used to read my papers--and actually was interested enough to go back and read them more carefully, and really found that I agreed from--again--our experience base, I agreed with most of the thrust of the papers.  And I will focus on not so much kind of pontificating on policy, but more our experience, naturally, in providing supplemental services. Quickly, just to let you know sort of what I wanted to discuss: I want to give a bit of a background--not the pitch on Sylvan--but I do think it's important that you understand sort of our perspective and background, and put my comments in proper perspective and context. Secondly, I want to talk about what I chose to call the "school transfer" versus "supplemental services," because I think they are separate, and the role of the SEA should be viewed as separate.  And I want to talk a little bit about that; maybe an observation, and then offer what I"m calling a practitioner's view on three assertions that I seem to have pulled out of the papers. First, just very quickly, the commercial on us, just so you know who we are--we are not the Learning Centers, by the way.  And that's why I think this is important.  We are a sister company, a related company to the Learning Centers, but much more focused on partnering with schools and school districts to provide services to address the needs of low-income, struggling students. We started the company about 10 years ago.  Last year we served about 70,000 students.  We commenced our supplemental services programs last year, and enrolled about 6,000 students.  And this year we expect to enroll about 28,000 students in supplemental services in 40 LEAs around the country.  You can see our profile: about 61 percent African-American, 29 percent Latino--just to give you some background on us. Now, school transfer versus supplemental services--these, to me, are not the same.  And I just wanted to spend a second talking about this, because it felt like, as I read the papers, they were painted with the same brush.  And I understand they're in the same provisions of No Child Left Behind, but as we look at our experiences and sort of watch the choice or the transfer experience unfold, they seem very different, especially in the context of asking the question of the panel, which is: where are the states and what are they doing? Our view is, with respect to school transfer, that the SEA role is much more of a policy implementator--if "implementator" is a word. [Laughter.] There are inherent structural limitations that the SEAs are struggling with and potentially, arguably, can't overcome: logistical barriers--moving kids around.  We've heard about the geographical barriers.  We heard someone talk earlier about one elementary school in a district.  Well, where are those students going to go? Capacity problems--we see all the time in cities that there's really nowhere for the kids to go, even when they raise their hand and say we want to transfer. When you compare that to supplemental services, we would submit that the role of the SEA is less of a policy implementation role, and could be more of what I would call a market regulator.  Because supplemental services is the establishment of a marketplace.  It is a brand-new marketplace, and at 20 percent--at full funding for districts that have to set aside their 20 percent, and if the district isn't using money to transfer kids, if you looked at 20 percent across all districts, that is over $2 billion--potentially--out there for tutoring.  That's a significant marketplace.  That marketplace will breed investment and innovation.  It will attract other folks like our company into the mix to think about how do we serve these kids, and how do we take advantage of this--what we see as a phenomenal opportunity to expand what we've been about for the last 10 years.  I mean, here is an opportunity for us to reach out in every single school district in America and offer our services, on a competitive basis, and the parents get to choose whether or not they think our program best suits the needs of their child. And I think, in that regard, that the papers somewhat underestimated the influence that the SEAs can have as a regulator.  There are some fine-tuning things that I believe the SEAs can do to make dramatic impacts and really cause great strides of improvement in the implement of supplemental services. Three assertions that I want to pull out specifically in the papers: one is this notion that there may be inadequate funding.  And I think primarily this came out of the Michigan paper.  Two is the notion that geography matters.  I think that is potentially a premature conclusion with respect to supplemental services; and then, third, that supplemental services suffer the same challenges as school transfer.  Again, I want to drill in, just for a moment, to this notion that we really ought to separate them and look at them quite differently. First, on the inadequate funding side: this is a quote from one of the papers.  As you can see, I really did relish the idea of going at these papers.  I thought that's what we were supposed to do. But I wanted to pull this one quote about funding, because funding, for us, has been an area of enormous confusion.  And you've heard it talked about today.  And I think states and, certainly, school districts are struggling and grappling with understanding the funding formula.  And here you have this comment about the per-pupil allocation, and that if more students start to take advantage of supplemental services, the per-pupil allocation would amount to only about $250. Our reading of the law, and our interpretation of the law, is that that is incorrect; that the per-pupil allocation remains constant--no matter what.  In Detroit, this year, the per-pupil allocation is approximately $2,000--regardless of how many providers--or, I'm sorry, how many students participate in the program.  It is a capitated amount.  Under that $2,000, we asa provider have to go to the LEA and negotiate a provider agreement.  The problem will come in when there are more kids asking for service than there are slots available at $2,000 a kid.  I do think there will be pressure on the LEAs to start to negotiate even more intensely with providers to bring that cost down, but the per-pupil allocation will not change.  It's constant, it changes per district, but it doesn't change based on participation.  Again--our interpretation of the law. So, if you believe where the Michigan paper was, then there was this notion that maybe the PPAs aren't sufficient to produce or generate academic gains.  And, again, we would differ with that.  At $2,000 a kid--even $1,000 a kid--especially once these programs are fully implemented, we've got years, and years, thousands of kids' worth of data to show that, indeed, we can make significant--statistically significant--academic gains with this type of funding. Again, whether there's a viable marketplace--there are more than a thousand providers approved across the country today.  It's only in its 18 month--second year.  I think there's absolutely a viable marketplace, and it's just getting ginned up. And that leads me to this last point: that in that regulator role, we would argue that SEAs can and should ensure the proper funding.  This is a very confusing aspect of the law.  And I suspect--and I actually know, in some cases--that providers have pulled out because it's confusing.  They look at it and think--if they read this and thought they're only going to get $250 a kid, I can see why providers wouldn't want to participate. On the issue of geography mattering, there was a suggestion in the papers that the rural students are at or will be disadvantaged.  Certainly, with respect to transfer, we believe that's true.  When it comes to supplemental services, this notion of limited universe of providers is true today.  I don't think--SES is "supplemental educational services" there--I don't think you can say that the quality is questionable.  I think that these conclusions are ultimately premature. The market for rural students is the most exciting aspect of this for us.  It is huge.  And this is an opportunity, through this program, to reach the rural poor like never before--in our eyes.  We've always wanted to work with kids in remote areas.  We're going to do it through online instruction.  But I will tell you that earlier it was, I think, suggested that the online instruction is not live.  We have a program that we've been sort of in the pilot phase of over the last two years--and we're actually bringing it out this year, both under supplemental services, as well as under contract agreements with school districts--that is indeed live.  It is online, live, real-time, with a certified teacher, through voiceover IP.  The kids wear a headphone.  They can write on a tablet.  They see what the teacher is writing on their computer.  The teacher can be anywhere in the country, and as long as those students have electricity--which is actually an issue, we understand, in the rural areas--so I don't say that lightly--and a phone line, which is another issue. But as long as they have electricity and a phone line, they can be part of this program. Our program will include sending a computer to the kid's home.  So they do not have to congregate in a common area.  They'll have flexible scheduling, and they'll get the same results that our students get in Sylvan Learning Centers. I emphasize that because I think, when you look at the rural market, the size is enormous.  And I don't think it should be lumped in that sort of "disadvantaged" category. Lastly on this issue: SEAs, I think, can be enormous catalysts to get rural implementation going.  We've seen that first hand.  We got approved in certain states, and designated the larger more urban school districts, because we knew it would take us a couple years to get ready to go out to the rural areas.  And we got phone calls.  There were states that actually called us and said, "We would like you to consider trying to reach some more remote areas."  And we're working in partnership with those states.  So that call has accelerated our entry into the rural market. Last issue--because I know I'm running out of time--this notion of supplemental services' suffering the same challenges as school transfer--and let me just put these points up--I think there's this notion that, one, the challenges are the same.  As I talked about earlier, I don't think they are.  Another notion that comes out of the papers is that there's a mixed level of effort on the part of the SEAs.  I think that's true, but we would argue that because of the market dynamics of supplemental services, there is an active role for the SEAs to play, but it's not that difficult. We have seen instances where one phone call from the SEA to the school district has clarified the whole funding issue, and opened up the marketplace.  And that's a point actually Lisa Keegan made earlier this morning, that it's not that much we're asking for the SEAs to be effective regulators. I see I'm out of time.  I just want to point you to the last bullets--that I think these are the things that we would argue that the SEAs ought to look at doing that really aren't that difficult: strong communication to notify parents that funding's available; clear communication about which schools are impacted.  We're in many, many school districts where we were setting up to reach out to the parents of schools, and then told that those schools are not on the list anymore.  Huge waste of time and effort and money for us, but also think of the confusion and the complexity in the marketplace when that happens. [Technical difficulty.] It's been a bit haphazard to date which, arguably is understandable, given, as I said, where we are in implementation but they have to take a real active role in setting for the criteria to look at what a quality program is, and what kind of results that kind of program generates. I'll had it back. MR. HESS: Thank you, Jeff. With that, we're going to change point of view, and we'll go now to Jane Cunningham and see what this looks like from inside the State House. REP. CUNNINGHAM: Thank you very much.  So I bring sort of a political perspective to this, as Chair of the House Education Committee in Missouri. I want to start out with the caveat--obviously, the papers that we heard presented previously showed us the issues that we need to deal with.  And I agree with that.  I recognize that we need to align some of these systems and coordinate them.  So I'm not going to go into those problems. But the thing that jumped out to me reading this papers, was the importance of attitude.  What is our attitude towards No Child Left Behind, and our acceptance of the principles that it produces? So I'm going to deal a lot with attitude. No Child Left Behind obviously breaks new ground.  And it does, from my perspective, in a couple of very important areas which we've talked about.  One of them is previously, where we have tested and gotten an average score, No Child Left Behind says we need to consider sub groups.  And those sub groups were originally buried in those average scores.  It's no longer acceptable to do that. The other area, of course, is choice and the tutoring services--I'm going to use Lisa's term "free tutoring"--go back to not the buzz words, but what people understand.  I'm going to put a little editorial comment in here about choice. You know we have expected public schools to be everything to everybody.  And I think we've expected too much of public schools, and put too much on them.  I think that what we really find is one size doesn't fit all.  And if we allow the opportunity for schools to find what is their right mission statement; what can they do really well, and then narrow that niche market so that they can provide it extremely successfully, we bring success and we bring options for students and their families. I'm going to go back to my term that I used first: attitudes.  And in the papers, they gave us some very good examples of those attitudes.  I wanted to cheer when Jane talked about Florida, and that people are jumping into this; they're engaging in it and they're giving the services that they're supposed. Positive attitudes--I saw in the paper--in the Denver public schools and, of course Miami-Dade--it looks like they're proactively getting out almost ahead of No Child Left Behind and providing to parents and to students what they need. In my own state of Missouri, there's a school that I visited last week, and this school is in the inner city.  Most of--I think all the students are free and reduced lunch.  All of them are minority students.  The school's name is Laclede elementary, and the president visited this school last week. In this district, 23 percent of the students are proficient in English.  In the state, 34 percent are proficient in English.  In this school, with all minority children, all on free and reduced lunch, 82 percent are proficient in English.  It can be done.  You walk in that school, it is very orderly.  They have uniforms.  It is most impressive.  There is a lot of collaboration with the community, and people come in and help provide incentives to help kids achieve.  It can be done. The negative things that I read in the paper: when districts purposely try to put letters out that show that--that look like they're achieving rather than there's a problem; when they purposely misinform parents.  I have a problem with that.  I also looked at the comment about the Boulder schools, and that reflects similar school districts that I have been in and that I have served on.  I have been on a local public school board. And I guess what I saw there is such a sad commentary, and I don't in any way--well, let's just say, my guess is it starts at the top.  Because we're going back to attitude.  And if the leaders in that school district have a good attitude, the parents are probably going to reflect that.  So when I see parents that want to boycott the test, I don't believe that started with the parents.  I believe it started with those who understand the system.  And if the people who understand the system will go to the parents and explain to them the reason our "good" school is labeled as not making adequate yearly progress, is because we have some sub-groups that need attention.  And maybe those parents can start volunteering to help those sub-groups, and that school district will not need adequate yearly progress the next year.  And, guess what--we will have done the right thing. In Missouri--I'm just going to tell you some examples of the same type of attitude that I see.  Just this week, an educator in a district in Missouri came to me and said, "Jane, there's not enough funding for all these mandates.  We have all these mandates, and there's not enough funding." And I said, "Tell me what is the fundamental mandate of No Child Left Behind.  What is the fundamental mandate?"  And she said to me, "To teach children."  To teach children.  Aren't we already funding to doing that?  Hasn't that been our mission forever? Secondly, I had, this past year--this past session--a superintendent come to me and he said, "Jane, isn't there any way that we can remove the minorities from the testing criteria?"  And I looked him and I said, "I hope I did not hear what you just said."  But it focuses on why we need no child left behind.  It does exactly what it says: no child left behind. I find that I label these two attitudes two ways: the positive one I label as "Focusing on children."  In our education committee, we have posters that we put up every time we have an education hearing.  And I invited some of our colleges and schools around the state to provide posters for us that remind us when we are voting that we are focusing on the kids. Those that have the negative attitude, I feel like they are focusing on self-preservation, and they tend to circle the wagons, give excuses, and give defenses as to why they're not making the grade. The realities: I found the realities in the paper interesting.  One of them, that when you ask education leaders if they believe No Child Left Behind is going to work, 70 percent said, "Yes, we believe it's going to help."  I thought that was really interesting, and something that we need to remember. The other thing that I read in the papers, that some people almost appreciate the top-down approach because they can say, "Oh, they're making me do it.  I have to make you reach these goals," and so it gives them an out of having to take the complaints to themselves. Let me give you an example in Missouri.  This past year I traveled the entire state of Missouri, meeting with education leaders, to get their ideas of things that we could do for a bill that we would be introducing.  One of the ideas was: please make the consequences, both positive and negative, of the students taking the MAP test, please make some requirements in there that there are consequences. Now, what they were saying is: around the state, high-school students are not working hard on these tests.  So they wanted us to be the bad guys and say, "You will get consequences if you either do well or you don't do well."  Now, in Missouri, every school district already has the authority to do this, but they want us to require it--top-down. So I think this goes back to out point: sometimes people are looking for the top-down.  And I guess, in reading all this and thinking about it on the plane coming down, our President is taking the hit for what a lot of us should be taking.  And this man has addressed this issue and I, for one, thank him for doing it.  And I guess I am inspired to go home to Missouri and say, "You know what?  This is about kids.  And I'm going to have to take some of those hits, too, if we're going to do the right thing politically." When we visited Laclede School last week, the 82 percent--it's interesting--I looked at the 82 percent and I was saying, "Yes, that is fabulous."  And the President in his comments looked at that, too, and he praised them.  But he very quickly went to the 18 percent.  And at first I was a little offended by that.  But you know what it said to me?  He cares about the 18 percent that aren't making proficiency yet and that we need to keep addressing. Policy recommendations that I would give: number one, attitude adjustment.  We need to decide are we going to do thing for kids, or are we not going to do things for kids?  That's number one. Number two: there are some easy things that we can fix that were mentioned, like those letters.  All we need to do is develop a letter that has fill-in-the-blanks: "Our school has been identified as--"--"That means you get to do this."  That's all we need to do.  That's an easy fix. Secondly, I do believe--like was stated before--parents are confused. Some of the better school districts in our state--well, about half of the schools in our state have been identified as needing adequate yearly progress, and some of our really good ones.  That is confusing.  And I think in our information to the parents, they need to be told, "All right, we do well when we score an average test.  We do not do well when our sub-groups are considered."  They need to understand that. With regard to rural difficulties, Jeff took exactly what I was going to say, and I was pleased to see that he saw there's some positive resources available to our rural districts.  I don't see this as negative--I agree with him.  And he's already said it.  I won't reiterate that. Problem with the No Child Left Behind identifying our good schools: politically what happens with that, the pressure is intense for us to lower the standards.  And I guess what I would like to see is if there's some way for the Department of Education to either point out and recognize two different measurements: average measurement that we're traditionally used to, and the No Child Left Behind measurements.  Maybe what they could do is look at the NAPE scores.  For instance, in Missouri we're going to promote this book.  This is put out by the American Legislative Exchange Council.  It is a report card.  It has some wonderful statistics in it on schools.  And on the NAPE test, Missouri ranked number seven.  One of the reasons I think it did is because we are keeping our standards high.  But I will tell you the pressure to reduce them is tremendous.  And politicians have a hard time standing up against that, and so do state boards of education. Someone brought up funding.  I might mention, in this book, that with regard to funding--and since we're in the District of Columbia--they are the second highest expenditure in the nature.  They are 51st in the nation on ACT test scores, and 42nd on the NAPE.  Missouri is about 40 in spending, but we are seventh on the NAPE.  It's not all about money. And I want to recognize the gentleman that put this together--Trent Baron, who is sitting over here.  He correlated a lot of this information, collected it, put it together.  So if you have questions, he is available. The other thing I would do as far as policy: clearly, we've talked about there are some problems and some barriers as far as supply.  We need to find ways to incentivize supply.  Perhaps some of the schools who will accept choice children need to be able to add on.  And, as far as supply, when we don't include the private schools, we're stamping out a whole area of the market that we need to look at. And I'll end there.  My time is up.  Go into questions with some of the other things. MR. HESS: Thanks so much, Jane.  Appreciate it. Next, Tom Houlihan, with the Council of Chief State School Officers. MR. HOULIHAN: Good afternoon.  It's a pleasure to be here.  And I'm going to take a little bit of a different tack. We have spent the last year, year-and-a-half, in the trenches, working with the vast majority of states' SEAs on the implementation of this legislation.  So I'm going to come at it perhaps from a little bit different perspective; from the reality of what people have experienced--good, bad and indifferent--as well as how all this relates to the papers we read. I have to tell you up front, when I received the papers I was a bit skeptical about what I was getting.  But I have to tell you, I was very impressed, after reading the papers, at the reality that was presented.  And I think that's really important.  And I also think it says a great deal that the states that have clearly been ahead in the choice issues and supplemental services and so forth, are struggling with all this. It makes me feel a tad bit better, because I know what other states have bene going through, who have no background and no experience. And so those papers really help from that perspective and, in fact, I'm going to share all those papers with our member states so that they can get a feel for what's going on. I thought the comment made by the first presenter about the idea that states are moving forward much more quickly--and probably more positively--on the standards and accountability end of this is very accurate.  Once again, I think people are far more comfortable with this.  They feel like they know what they're doing with this.  And in many, many states in this country--many local-control states--the idea of the state telling the local school systems what to do is not kosher.  So a lot of states are having to really work hard to bring about the implementation of all this, and it's kind of safer to start with standards and accountability measurements than it is the other items that we're talking about. I think in the future I think we're going to see supplemental services implemented probably more quickly, and perhaps less controversially--if there is such a word--than we are choice.  Because it's a little bit less complicated, in terms of supplemental services.  I thought, Jeff, what you talked about in terms of the rural areas and so forth, and where that's going, and the role of the internet and all that--that's going to be huge, in terms of reducing the complications that so many states are faced with.  And I'm talking about--you go to some places in North Dakota and South Dakota, we're not talking--I mean, when we talk about "rural," we're talking about real rural; and Alaska and some other places.  There are rural places like Jefferson County, Florida, and then there are rural places like places in Alaska and North Dakota and so forth--they have to have that internet approach or it's simply not going to work. I don't believe--I really, truly do not believe that there is anything sinister out there in terms of the implementation of choice and supplemental services overall.  I don't think that there is a sinister plot not to do that.  I do think states are struggling with a lot of issues related to that, and how to do it. Go back, once again, to the standards and assessment aspect--somebody mentioned the fragility of the measurement systems.  There have been states that have put out data that was inaccurate, and the reaction has been so overwhelmingly negative that lots of states saw what happened and pulled back, because they wanted to make sure that their data was correct. I heard some of the comments made about the fact that data's not being released as quickly as it should be.  I agree.  But people want to really make sure--especially those places that have not had the experience of releasing statewide data and so forth, there is a real hesitancy to move forward too quickly on this and, ultimately, that does impact supplemental services and choice. A couple of observations from the papers, and then I'm going to share a couple of quick comments, and then I'll turn it over to John. I think that the bill is complex, and is complicated, and is at times confusing.  I think the expectation of the legislation is crystal, crystal clear.  And I think that that's been pretty well documented in terms of what the papers have said.  It is about all kids.  Period.  No child left behind. But the implementation of such an enormous, 1,100 page bill, is not an easy task.  Yes, there are islands of excellence out there who have run with this, and who've done superb.  And, yes, there are attitude issues that are involved.  But across the country, in order to do this correctly, there are lots of questions that have to be answered.  I quite frankly think the Department has done a very, very admirable job in trying to implement an 1,100 page bill.  They have been reasonable in working with the states.  There have been issues that have come up that neither group even anticipated.  And so I think it's moving forward. So I wouldn't want us to leave thinking that people are intentionally dragging their feet--and I'm sure there are some, some places.  But overall, in terms of working with the states, I don't think that there is a sinister effort to keep from moving forward with No Child Left Behind.  There are, once again, maybe islands where that's the case but, overall, we certainly don't hear that from the people that we work with. I believe that the capacity issue, regardless of what anybody says, is an issue.  The capacity of state education agencies to implement some of this is a simple issue.  The comment made about the state of Michigan, and the fact that the department has been downsized by 50 percent--folks, that's an issue.  I think you see that in Texas, I think--well, I think you see that in most states as we struggle through budget issues the past few years.  The first place to cut is the bureaucracy in the state capital, and that's happened. There are some requirements of physical manpower to carry out some of this that cannot be simply thrown away.  In some states the requirements to certify supplemental services providers is not just as simple as making up a list.  Some state legislatures require detailed analysis of the companies' capabilities.  Folks, that does take time.  And there are some states--I think California being one--where there's a huge number of folks who want to serve as supplemental services providers.  So that capacity issue--I just have to say, based on working with our folks--that is an issue that is simply not going to go away, and that all of us have to at least take a look at. I think we sometimes underestimate the enormity of this effort.  Folks, this is ground-breaking change in the public school system of America.  Some of you have been advocating some of these items for years.  Some of you see these purely in political context, or political lenses.  But the reality is that this is a sea change in education across this country.  And, folks, it is simply not that easy to do.  Yes, there are people who don't like some of the policy issues--no question.  And there are some attitude issues.  But the cold, hard reality of changing the system of public education in this country--it is not a simple thing to do. And so I think sometimes we've got to remember the fact: we're only into the second year.  And let's face it, the first year the states didn't even have their plans in until about three or four months later than original.  So the point I'm making is: this is much bigger, and much more of a revolution than sometimes some of us who sit in Washington may realize.  This is a big, big change. A couple of quick comments--I think in all that we do, and all we talk about as we move forward, we've got to keep the concept of the law of common sense in mind.  There are some aspects of implement of this bill that, quite frankly, common sense-wise just don't make sense.  It's not placing blame on anybody, or trying to get out of anything.  But there are some issues that rules and regs have been made related to special ed, or LEP or whatever it might be, that common sense just tells you: "This is not going to work this way, but let's figure out how to make it work."  And I think as long as all of us keep this law of common sense in mind, I think we'll be far more successful in terms of implementing it. Finally, I think the thing that really struck me, that I really thought was terrific about the papers was the reality of what was presented.  Quite frankly, I figured I'd pick up and read four papers on an extreme, idealistic, ideological viewpoint, and anything that didn't agree or fall under that format was wrong.  That wasn't the case.  I saw a sense of realism in what I read, and I think that those papers can help enormously as we all try to really understand the complexities, the capacities and the difficulties all of us are going to have as we try to make this very, very important, and very, very, very critical piece of legislation a reality across this country. Thanks. MR. HESS: Thanks so much, Tom. And John? MR. STEVENS: This is not an enviable position to be in.  A couple of years ago I was the final speaker on a Friday afternoon at an education conference sponsored by the Federal Reserve Bank of Dallas.  And as I looked out in the audience, half the people who were still there were my friends. [Laughter.]  I don't--I lived here for 17 years, but I don't have my friends and neighbors in the room.  So I thank those of you who stayed for doing so. I have about 25 pages of notes, none of which--all of which are relevant, none of which I want to say anymore, because just about everything I intended to say has been said. So I'm going to try to put what we've done today, or what we've heard today, in a larger context, but also look forward a little bit. This really--if it's going to benefit the vast majority of the kids in America, has to improve the public schools.  Because whatever we do to offer choice, to offer supplemental services and so forth--the very vast majority of the kids are going to be served in the public school system.  And the broad strategic intent of No Child Left Behind is to cause statewide improvement, district-wide improvement, school-wide improvement that reaches every child. Increasingly, I've been interested in district-wide improvement.  And let me tell you, there are probably two reasons for it: number one, I've been working in Texas for 12 years, and there's almost 8,000 schools, and only 1,000 districts.  So it seems easier to get your arms around 1,000 schools than it is 8,000 districts.  The other thing is I get a year older every year, and I get more impatient. [Laughter.] But school-level excellence doesn't last, and very hard to be institutionalized.  And I've seen that district-level excellence is institutionalizable.  It can last over time.  Transitions can be made that will cause excellence to occur in districts that are large and serve large populations of disadvantaged kids. And so No Child Left Behind has to, in the end, drive really good work around curriculum, staff development, instruction; identifying and meeting the needs of individual kids and so forth.  And it can be, and is being done.  And that's very important. No Child Left Behind has the ability to drive that kind of work.  And that's really important. The second thing is: I'm really encouraged by implementation.  My friends in this administration are really serious about getting the states to do this stuff.  And, to their credit, the states, by and large, have responded positively because they sense a level of seriousness that maybe hasn't been there before.  And I think the responses that we've seen in a very short period of time have been good, and they've started with testing, assessments, and I will tell you--information systems. Don't ever underestimate the value or the importance of getting the information systems right.  A lot of the problems that the papers point to, in terms of delays in getting out the AYP and all of that stuff have to do with the fact that many states don't have good information systems--robust systems--to collect information, reliable information, from districts.  And districts don't have good information systems.  And so there is real serious questions about he quality of the data that they're processing. But implementation's moving ahead apace.  And I will tell you just as outside forces--and especially the business community--has been necessary to drive reform, I think outside forces--and especially the business community--is going to be important to follow, intervene, and to encourage good implementation of all aspects of No Child Left Behind. I talked a little bit about the overall improvement that's necessary in the system, and I don't want to lose sight of that, because that's really what's very, very important.  But I want to take a look at the future a little bit. Some people are really concerned about the fact that we haven't had a lot of schools identified in various places as failing to make adequate yearly progress, and identified as needing improvement.  I've looked at the AYP benchmarks, beginning in 2000 up to 2014 for Texas and a lot of other states, and I can tell you what most states have done: they've back-end loaded their benchmarks.  They start out expecting little, and they end up expecting a lot.  And I think, in some respects, state agencies and the people who've been submitting those plans have been driven by the idea that it isn't going to happen on my watch. [Laughter.] "We'll start out easy and, you know, by the time the trajectory looks like this, I'll be gone.  I'll be retired, and I won't have to worry about it." I can tell you, in Texas, until about 2008, the slope isn't very steep.  And it's two years at every level.  And then beginning in 2008, the slope all of a sudden goes like this.  And so when we look at what's required for every sub group, in terms of reaching that goal of 100 percent proficiency, I believe you're going to have a lot more schools--a lot more jurisdictions--in the future who don't make adequate yearly progress for one or more groups of students, in one or more subjects, simply because the way the system has been constructed.  And that means the issues of supplemental services and choice and everything like that will be, I believe, expanded greatly. Now, if public schools can improve their performance well enough to keep up with those goals, more power to 'em.  That's great.  But I just want you to realize that most states have back-end loaded the real tough challenges.  And so what we see right now is just a beginning of what they're going to have to face. Choice, certainly, and supplemental services have been discussed quite a bit here today.  I think too often they're seen as sanctions; they're seen as ways of punishing people in the public schools for not performing as we'd like them to.  I think we ought to look at them more as ways of serving the needs of kids in different, and unique and varied ways.  And until we look at it that way, and until we start evaluating their effect--not just how well we're implementing them, but their effect in terms of the results--until educators start seeing them as part of the solution to meeting the needs of every kid, they're going to look like sanctions.  They're going to look punitive, and they're going to resist them. Not in all cases--in some places people have really embraced this, with a notion of serving the needs of every student.  But until we get that kind of mentality there, that's not going to happen. And that comes down to leadership.  You can't do it all through compliance monitoring.  You get people who worry about rules and regulations and making sure that they get it all right.  I want them focusing more on teaching and learning, understanding the needs of each kid, meeting those needs and so forth, than worrying about how they--the extent to which they're following the rules and regulations that came out of the U.S. Department of Education and the state education agency. But that comes down to leadership.  It starts at the state level.  It exists at the district level. It's important at the school level.  But it's also important in the broader community--that what we're about here is the next generation of Americans; the next generation of Texans; the next generation of people in this community.  And the rules and regulations are back there, but what we're really focused on is teaching and learning and kids, because that will really matter for the future. Now, I'm a member of the National Assessment Governing Board, so I need to say something about NAPE. People are concerned about the level of standards and the tests that are used to measure performance against those standards.  One of the reason the NAPE is there as a confirming and validating instrument is that states are free to set their own standards, develop their own assessments, and measure progress--and use those to measure the progress of their students.  I can tell you, if the standards are too low, if the assessments are lousy, and if performance doesn't move on the NAPE--maybe not to the same magnitude--but if you see a lot of performance on state tests and you don't see any movement on NAPE over time, you have real reason to question the standards your state has set, and the efficacy of their assessments. So NAPE has a very important role to play.  NAPE also points consistently to the huge performance gaps that still exist.  We were making progress in the state in raising the floor; in bringing up performance from the bottom, and we need to start there, and that's important.  But we need to keep in mind the goal of the high levels of achievement that are necessary in today's world; what's really required to be a fully-participating citizen of this country; what's necessary to contribute to the economic development of our communities, our states and our nation.  And avoiding failure is not good enough.  We need to be able to focus much higher; have students, educators, parents and all of us reach to the levels of achievement that are necessary for people to participate fully and contribute in the kind of world we live in, and the kind of world they will live in in their future. And this is the kind of vision that No Child Left Behind, I believe, represents.  And if we forget, or set aside, to some extent, the problems of details, the complexities and all of that, focus on the big picture, there's broad agreement there.  Let's lead the country in that direction.  Let's play--all of us--the part that we can in bringing this vision into reality. Thank you. MR. HESS: Thank you, John. It seems to me that we've moved a bit from the more narrow questions of supplemental services and public choice, to thinking a bit more broadly about NCLB and how these pieces fit. So, before we open up to questions, I guess I'd maybe like to just pose Checker's earlier question to this panel, when Checker asked the previous panel for their general assessment of whether supplemental services and public choice were likely to work in their current incarnation.  We hear what I guess I would characterize as cautious pessimism. And I guess I'd be, you know, curious in a quick assessment from each of you as to whether you think, indeed, supplemental services and public choice are going to work significantly better with time?  Or are you as skeptical as some of the members of the previous panel. Let's go same order.  Mitch, why don't you start, please? MR. CHESTER: I think Jeff's comment about disentangling supplemental services from choice is right on target. I think that choice--I don't know that the prognosis is good in a lot of states for it being a viable policy tool, whether it's seen as a sanction that motivates folks to reassess how they do business, or whether it's seen as an opportunity to learn--simply because the projections on the number of schools that are going to be required to offer choice--and in many large lower performing districts, it's going to be the majority of schools, combined with the choice options that are already there, probably going to create tremendous restrictions on how viable choice really is. Supplemental services is a whole 'nother matter, and I think there's tremendous potential and possibility there. MR. HESS: Thank you. MR. COHEN: Well, I agree with that last comment, obviously.  I'll talk about supplemental services.  And as opposed to cautious pessimism, I would say cautious optimism--just the opposite. It is working.  It's happening.  You can go visit--we are in 140 Chicago public schools doing, I think, wonderful things with students that need help, and whose parents chose our program.  It's working.  That school district is just one of several that I could cite that has fully embraced supplemental services and worked with us in true partnership the way we want to. So, you know, I could sit here and tell you dozens of war stories about districts that we're, you know, going back and forth with.  But in the short time that we've been at this--18 months--we have seen dramatic improvement.  I mean, we are very encouraged.  And I said we've yet to tap in--I mean, we naturally started with the big urban school districts, because you can only go to so many places at one time. And I think you just have to keep remembering that it's the beginning.  It's the very infancy of these programs--at least we hope it's the infancy.  And if it is the infancy, then I think the prognosis for these programs is fantastic, and the future's very bright. MR. HESS: Okay. Jane? REP. CUNNINGHAM: I'm just absolutely full of hope.  Absolutely it's going to work. It has to work. Why are we here?  We're here because in 1983 the Nation at Risk Report showed us what we were about.  It showed us where we were.  And I think now we look at our test scores, compared to other nations, and we see where we are, and we know that a lot of our companies are hiring people from other countries because of their skill sets.  We-- must--make--this--work. And I'm also encouraged, too--I went to Milwaukee this summer with some of my colleagues.  And some of those colleagues were from the inner city in Missouri.  And I will tell you, I saw so much success in Milwaukee with the choice programs and what they're providing in the options for the kids.  And I think people like Representative Betty Thompson in Missouri--she represents the City of St. Louis and parts of it--and I think she began to see and get a vision for "My constituents need this.  I need to really look at the kids and what is best for them." And I was encouraged by what John said.  Why are we doing this all together?  We're doing it because we have to.  We need to make this country continue and progress and be an economic power.  And we must do it.  And I am filled with hope.  Yes--we--are--gonna--do it. MR. HOULIHAN: I would agree with Mitch and Jeff about the cautious optimism on supplemental services.  You know--I would, I would.  I think that there's real hope there.  I think that--I still think there may be some issues related to funding, about how much per child.  Because that is very confusing and it needs to be worked out. But I think that's very positive. I think the success of choice is directly correlated with the number of available schools that can receive students.  Period.  If they're there, I think it can be successful.  If they're not there, it raises a whole 'nother set of issues. MR. HESS: John? MR. STEVENS: I want say a couple positive words, mostly about supplemental services. As schools develop the ability to monitor the progress of each child, and understand their peculiar needs, and customize educational services to them. the kind of things that supplemental services mean will be increasingly important.  As they stop just educating the group, and really start paying attention, following the progress of every child. So, in terms of No Child Left Behind and the funds that are available in that way, that certainly is part of the picture.  But I believe that it will become a bigger part of the picture just across the board; not just in Title I schools, and not just in schools that fail to make adequate yearly progress, but as schools start identifying kids that need special help--in even the best performing schools--and as we start dealing with individual kids in the full spirit of No Child Left Behind, what supplemental services represents is very important to the success of the enterprise. MR. HESS: Okay.  Well, there we go.  The dour scholars and the more hopeful practitioners.  The world makes sense. Do we have any questions? Right here. MR. KORMAN: Steve Korman, Georgetown University's School Choice Demonstration Project.  A couple of questions about school choice. I was very interested to hear Mr. Mitchell say that he thought there was a lot less impact than the policymakers thought there would be with school choice.  And my question was just: would you advocate expanding school choice across districts?  Into private schools--like Ms. Cunningham just described?  Into charter schools outside of districts? And, in particular, some policy recommendations for those rural districts where there isn't any--effectively--any choice at all, would you advocate expanding across into other districts, or into private schools? MR. CHESTER: We're still on TV? [Laughter.] MR. CHESTER: Well, Ohio currently has a voucher program that allows students to attend private schools on public dollars.  Cleveland has charter schools which allow students to cross district lines to attend charter schools; allow virtual schools which kinds from around the state can attend. I think that the evidence is out.  If the question is whether or not school choice, and the provision of school choice improves the lot of all, I think the evidence is still out.  So if school choice is seen as a primary policy level for lighting a fire under the existing schools, ratcheting up the quality of the whole system, it's--as a viable option for kids who find themselves in schools that aren't performing well, I don't see the evidence that it's occurring that way.  I think it's occurring in spots and in places. So I am supportive of public school choice, giving kids options around where they attend school.  But if our hope is that that's the answer to ratcheting up the quality of the whole system, I'm not there yet. MR. HESS: Oh--back here? MR. SMITH: Thanks.  Nelson Smith, with New American Schools. There's a great line in Jane Hannaway's paper about seeing principals in Florida in sort of a sea change, walking around with clipboards with data and the test results from the disaggregated student groups, and thinking really hard about what to do with each of those students and groups.  And yet that seems to be a consequence of the A+ Program, not so much the consequence of the new No Child Left Behind reform. Since the direct beneficiaries of choice and supplemental services are actually students, not schools as such.  And since, as John said, much of the leverage seems to be at the district level, but not, again, inside schools, it seems to me that a lot of the attitude within schools around the country is that this is a burden, and that they're not reacting like those wonderful Florida principals and saying "This is good new stuff to work with.  Let's take the bull by the horns and do the best by all of these kids." How do you make that connection to the classroom tighter, as a matter of policy, or as a matter of the implementation of the Act? MR. HESS: Anyone? MR. STEVENS:  Well, let me try something here related to that. This really does come down to: there are faces behind all the numbers.  Schools that have small numbers of certain groups are less exposed to AYP.  The schools that are most exposed are those that have large numbers of students, and a very diverse population, so that they have a sufficient number of every one of those groups to avoid the consequences of No Child Left Behind in certain ways. But it doesn't absolve people in situations where they just have a few African-American kids, or a few special education kids, or a few LEP kids from educating all those students. So the policy will only go so far.  And that's where the kind of leadership I've seen in Texas in certain places is so very encouraging to me, because it really does reach down to the principle of No Child Left Behind, and they're not playing games about let's keep the numbers small enough so they don't do the calculation on us. And, you know, in the end it's got to reach that point of morality, in terms of the commitment that professional educators and the community make to the next generation. MR.           : Let me--can I-- MR. HESS: Sure. MR. HOULIHAN: Am I on?  Can you hear me? I think--picking up on what John said, there are two things that we need to think about. The first is, it is so easy in this country to place blame; to have winners and losers.  And if you're a winner, if you're a loser you're a--you get my point.  It's kind of our culture. What I think we need to do is approach this whole issue of what we're trying to do with sub-groups in a way that we initially don't place blame on individuals who don't succeed. Let me explain what I mean.  We are--many of you are aware of this--we are now talking about new levels of performance for all kids, regardless of sub-groups.  And every system--and education is a system--every system is designed to produce exactly what it produces.  And our schools for years have produced what certain policy set, etcetera, etcetera. We changed the rules.  And if you want to change performance, you've got to change the system--which means our people out there working every day must have the professional development and support to be able to make those kind of changes.  It is not just as simple as: "Tomorrow everybody will learn." And so I think one of the things we really can do to make this successful for all kids is to really get real about the professional development people need; where those weaknesses are; and at least initially, let's not blame individuals, let's try to help them get better to meet the needs of those kids. MR. HESS: Chester? MR. MITCHELL: Just to add--I'm very heartened--I mean, I tend to be an optimist on this stuff, and I'm a believer in the value of accountability systems, particularly if they're well crafted, and if we can get some sustainability. I'm very heartened.  The energy that went into the conversations I was having with school district officials in Ohio 18 months ago was strongly tipped toward discrediting the law; a lot of energy put in by school district officials around discrediting the law. Today, that balance has tipped.  Now, I'm not going to tell you there aren't people out there who are still putting energy into discrediting the law.  But a lot fewer; much smaller proportion.  And the tone of the conversation among those who are no longer spending their energy discrediting the law is now on: how do we put it to work?  It's here, we've got to live with it--how do we put it to work?  What are the implications?  How do we start to look at kids differently?  How do we start to re-think how we do school?  And those conversations are not real mature in a lot of places, but they're beginning. The challenge is to take this opportunity that's provided, of getting people's attention, getting some clear targets, and turning that into some productive, constructive kinds of changes at the school level that carry forward.  Part of it's sustainability. MR. HESS: John--before we get onto the next question--you had a point to make? MR. STEVENS: I just wanted to offer something really radical about public school choice in the context of Texas.  There is actually a high school in Houston that doesn't have a football team. [Laughter.] MR. HESS: There you go. MR. STEVENS: I say that--but I don't mean that facetiously.  I want you to think about that.  There are some people that would like their kids to go to a high school where that mentality doesn't drive the whole program. And so when we talk about choice, it's not just "good" and "bad," it's different characteristics of schools that parents would like to see their kids in, where kids would succeed.  And, you know, a good school is no choice at all, in my mind.  That's not what it's really about.  What we need to encourage within the public school system are schools that are really different; that serve different kinds of kids, who have different kinds of needs.  And we don't go there yet--except when we eliminate football in a high school. MR. HESS: Mike? MR. LIEBERMAN: I had an intern this summer-- MR. HESS: Please identify yourself. MR. LIEBERMAN: Hmm? MR. HESS: Please identify yourself. MR. LIEBERMAN: Oh--Mike Lieberman, with the Educational Policy Institute. I had an intern this summer from South Korea.  They devote one hour a week to extracurricular activities.  Imagine. But it raises a question for Jeff: Jeff, over half the families in this country spend, from their own pockets for tutoring, while their children are in elementary school.  And they're beginning to say, "Why is this necessary?" So here you are.  I presume you're not just adding on to what the schools do; you're doing something different.  My question is--assuming you're on the witness stand now, and you're sworn under oath--shouldn't the schools be doing what you're doing during the regular school day? MR. COHEN: As you can imagine, we hear that question all the time.  And, sincerely, on the witness stand, under oath, I think the answer is no. We're supplemental--I think that's where the word comes from.  We're supplemental.  We're not there to supplant what's happening in a school. I mean, the schools that we work in are the most challenged schools--forget "supplemental services."  Before this law was ever enacted, the schools that we've been working in are the most challenged schools in the country. We go into middle schools where the middle school is fashioned, is structured to present programming to sixth, seventh and eighth graders.  And the fact of the matter is, there are seventh and eighth graders reading on a second-grade level.  And, you know, I don't think it does anybody any good to sit here and say, so we're going to blame all kinds of folks in the school. I think the issue is: how do you address that very special need of a child that's in eight grade and is reading at a second-grade level?  Arguably, the school's not set up to do that.  That's not what they were structured to do.  They were structured to teach at an eighth-grade level. So, to us, we feel that that's at the point where that principal, that school district, has a dilemma.  And they have to make a choice.  And they can try to address that situation on their own, bring in a reading clinician, or look for a partner--private sector, non-profit, and that's where we come into play.  We're a partner.  We address specific needs of individual children.  We don't re-do what the school has done. That's why every one of our programs--and most of the providers, and our colleagues in the provider community--every one of our programs starts with an assessment--a battery of assessments--to determine what the child needs to learn. And we've seen this over and over and over again--one size does not fit all for our children.  So, take any random sampling of children and apply one curriculum to them, and you will find that that one curriculum is not successful because there are individual learning styles and differences, and I think you have to offer alternatives. And that's why what we are so thrilled about with respect to supplemental services is this full panoply of services.  And once parents become true consumers--and this law, to us, creates the opportunity for low-income parents to become true educational consumers--they can pick and choose, and they can shop, and they can recognize what's going to resonate with their children, and meet the needs of their children, and make that selection. MR. HESS: Okay. Back here? MR. FINN: Checker Finn.  This is primarily a question for Mitch and Tom, because you said something that went against expectations or predictions, and I'd like you to elaborate just a little bit. When Rick asked you to predict which is likely to be better assimilated and work better--supplemental services or public school choice--you both came down squarely on supplemental services as having the better odds of making headway. And the reason this is surprising, as some people said earlier in the day, is that supplemental services tends to take money out of the public school budgets, and put it somewhere else--like in Jeff's budget--[Laughter.] --whereas public school choice keeps the money within the public school budget, albeit moving the kids and money around among schools. And that's the reason why I--and I suspect others--was surprised to hear you say that it's likelier that supplemental services will have a bright future. Can you explain why the conventional wisdom here is wrong? MR. CHESTER:  Part of my response reflected on my lack of confidence that choice is likely to have a lot more impact on the quality of the system in general, and the learning that occurs across the board. So, in that respect, I think the potential for looking at supplemental services, ways of supplementing the instruction that students get--students who are falling behind--has the greater potential.  It has the potential to be targeted to individual students.  It has the potential to be doled out on an as-needed basis.  And I suspect the economics of it--I mean, right now, it's--there are some dedicated Federal funds for it which are driving its entree right now.  But I suspect the economics of it, as folks learn--get more experience with it, and figure out which providers are effective and really move achievement and which ones aren't; which kinds of packages a district can put together on its own that's effective, outside of the school--the normal school day--that as folks get more experience with that, the economics of it may work itself out.  Because for whatever incremental cost there is to that, if the impact, in terms of student achievement is substantial, I think it will have legs. MR. HOULIHAN: That's an interesting take that you took on it.  I can see where you're coming from.  But I think the reason why supplemental services will move forward--in part, Mitch covered the quality aspect--but also because I think supplemental services--at least in terms of working with folks that I work with--that that is seen as less volatile than choice; and also seen, perhaps, with less suspicion that somebody's trying to destroy public education.  So I think that's one factor up front. And second, I just think it's a practical thing.  If you work in--I really think the success of choice is going to be correlated to the spots that are available in a school system.  And if you look across this country, most school systems are fairly small--what's the average, $3000, or something of that nature?--and take the cities out of that mix.  But you look across all states, and there's just not going to be the level of opportunity for choice that I think would contribute to it moving forward as quickly as supplemental services. I also think that because we have had after-school programs for some time now that there is a natural acceptance of after-school programs as working.  And so it's easier to support those down the road. MR. HESS: Okay.  We have time for one or two more questions. Back here? MR. ROSE: Hi--Joel Rose.  I'm with the Edison Schools.  We're also a SES provider. One of the things we've seen in school districts a fair amount that really want SES to work; some that are focused on compliance more in the spirit; and then some that are sort of dragging their feet a little bit. And if you were to ask what one question can you apply to determine which category a school district might fit in, the question would be: do they allow providers to access their facilities to provide SES services.  That really becomes the gating factor, because without facility access, there's no choice for many parents.  They don't want to pick up their kid, drive them somewhere else, pick them up, so on and so forth. The question--and I guess it's really for Tom and for Mitch, is whether or not you all see a role at the state level in encouraging, or requiring or somehow getting school districts in the individual states to allow SES providers to access school facilities? MR. CHESTER: I'll take a nice straightforward answer to this question.  I honestly don't know. [Laughter.] MR. HESS: That's pretty straightforward. MR. HOULIHAN: And by that I mean that, given the various roles of state education agencies around this country, and state by state, that might be very appropriate in some places, that would be totally inappropriate in other--not talking about implementing the bill, but talking about getting into--forcing people to do something that someone things should be done. I mean, it just really depends where you go across this country.  So the answer is: I don't know. I think it's a great question, and it's one that needs to be explored. MR. CHESTER:  May I just add that this is a learning process for all of us.  The Council of Chief State School Officers under Tom Houlihan's leadership did a tremendous job in bringing states together around figuring out how to meet the initial requirements of the law, around the--particularly around the accountability requirements.  And a couple of papers talked about the kind of energy it took state agencies to figure out their accountability plans, and figure out where they had to go with testing contracts and so forth. I mean, I just think about the Ohio department--I mean, there were several full-time staff equivalents who worked on our application, and understanding the law, and figuring out where we needed to go, and intersecting with the field around this--outside of running day-to-day operations--for over a year. So, I think--I can see states coming together in the future, and sharing best practices findings, insights into where and under what circumstances supplemental services pays off.  And then, as Tom said, then it's up to each state as to how they grapple with that and deal with it in their own context. MR. HESS: I will take two quick questions, and then we'll let them be answered together, and we're going to call it a wrap. Right back here? MR. ARCHIBALD: George Archibald. Mr. Chester, you made the comment that you thought the evidence was out on the helpfulness of these choice projects that you talked about.  Do you think this is a rather persuasive argument that the Congress should accept Mayor Williams' request that the Congress fund this demonstration project for a voucher program in the District of Columbia? MR. HESS: Okay.  And-- MR. ARCHIBALD: To collect data--to get the evidence in. MR. HESS: Okay? MS. BROWN: Cindy Brown.  I just wanted to pick up on the question this guy from Georgetown asked, and ask John Stevens if he would comment--Texas has been a tremendous leader in moving forward on helping all kids--we all know a lot of its provisions under No Child Left Behind. Do you see Texas as being willing to explore the possibility of inter-district transfers, around some of its big cities and some of its suburbs, with certain incentives. I mean, you know, it would have to be done carefully, but could Texas ever lead on that? MR. HESS: Okay, Mitch and John, please--quick responses, and we're going to wrap it. MR. STEVENS:  Part of the problem there are just plain growth trends. We added 95,000 kids to the public school system last year.  And in the school districts that are losing enrollment, they lost 30,000.  So, the districts that were growing increased by 125,000. Many of the desirable places that people would want to go are in situations where every school is crammed. They're surrounded by portables.  And this is just not something that's in the cards.  The growth rate has accelerated.  Two years ago it was 70,000, then 85,000, then 95,000.  We're just really scrambling--in the suburban districts and in places like that that would be desirable--to keep up with that trend. MR. CHESTER: Well, I would put the same mantle on thinking about public funds to support private placements--vouchers--that I would on any research.  I mean, let's be clear about what the purpose is.  The Supreme Court has already settled the question of whether it's constitutional, so let's be clear about what the purpose is.  What are we trying to accomplish? If it's in the name of kids' learning to read and write and do math better, then let's design that demonstration in a way that we find out whether or not it does make a difference. MR. HESS: Okay.  Thank you. With that, we're bringing today's session to a close.  I invite all of you to join us right out here in the lobby to continue the conversation in a more relaxed environment. We'll be beginning tomorrow morning at eight o'clock with breakfast.  We'll start the first panel at 8:30.  We'll look forward to seeing you then.  And, again, papers are available in the lobby. I'd like to thank the discussants for the session. [Applause.] [Whereupon, the conference was adjourned, to reconvened on January 16, 2004.] MR. HESS:  Good morning, good morning.  If you'll take your seats, please.  I'm Rick Hess, Director of Education Policy Studies at AEI.  I'd like to welcome you all back for the second day of the conference on Leaving No Child Behind, hosting by AEI and done in collaboration with the Fordham Foundation. I'll be hosting my first session of the day.  My collaborator in this, Checker Finn, will be hosting the second panel.  And then over lunch, we'll be co-mediating a discussion about what we might take from the sessions we've seen and the implications for policy and for practice going forward. I'd like to welcome all of you back, all of those who were here yesterday.  I congratulate you on being able to make it through a second day of this.  For those of you who weren't here with us yesterday, welcome. You'll find copies of all the papers presented both yesterday and today out in the lobby, on the table, along with some other materials from the Department of Ed and what not related to sup services and public choice.  In addition, we have selected papers available on the Web at <A target=_blank href="http://www.aei.org/" target=_blank>www.aei.org</A>. For those of you who prefer not to shuffle around a bunch of papers, the volume with the entire set, handsomely bound, with insights from Checker and myself, will be available later in the year from Palgrave.  Dubious insights, some might say, but we prefer not to go that direction. Anyway, I'd like to introduce the first panel of the day.  If you recall, yesterday we started at the national level, thinking about supplemental services and public choice from a broad perspective. The second session, we looked--we then stepped it down to the state level and we looked at it empirically as to what's going on in the states.  And then we heard from a number of folks who are actually working with these issues at the state level. Today, we're going to take it down one more level of concreteness.  The first session is going to be on what is actually transpiring at the district level, and the second session that Checker is going to moderate is going to be, again, practitioners sharing their reactions and thoughts on the papers at the district level. So on the panel up here, the first paper is going to be the overview of what's transpiring at districts: Mike Casserly, Executive Director of the Council of Great City Schools, an organization of 60, I believe, of the nation's largest urban school districts. Second, we'll have Julian Betts and Anne Dannenberg [ph].  Julian is a Professor of Economics at the University of California at San Diego and at the Public Policy Institute in California. Third, we have William Howell, of Harvard University.  He'll be sharing his case study of Worcester, Massachusetts.  William also conducted, in concert with that, a fascinating study.  We were able--primarily, William was able to raise some additional resources for a survey of Massachusetts parents in ten communities? MR.          :  The ten largest school districts. MR. HESS:  The ten largest school districts in Massachusetts, using phone survey techniques to simply plumb their understanding of their schools' eligibility for No Child Left Behind, whether they could correctly identify their schools' status, whether they were aware of their options.  We're not presenting it.  It will be in the volume and you can find the paper outside on the table. Finally, we have Doug Reed, of Georgetown University.  Doug has done a piece on Montgomery County. We're going to start with Mike and then we'll go in the order I just introduced them.  That will take about an hour and then we'll have lots of time for questions. Mike? MR. CASSERLY:  Thank you very much.  I will be brief.  I'll try not to take my entire 15 minutes. I've basically drawn three conclusions from the work that we have done on this project.  First of all, we've concluded that the law, in general, is being put into effect at least in the large cities and is starting to take shape at the local level.  There are still lots of growing pains in the legislation and it looks very messy at ground level.  But when you step back and collect the data, there are clear patterns in the legislation and how it is emerging on the ground. Second, the participation levels in both choice and supplemental services are low even in this second year. And, third, as many of the presenters mentioned yesterday, it is still too early to tell if choice or supplemental services is having any effect on student achievement, although it is clear that student achievement in the major cities is improving. I draw these conclusions on the basis of a survey that we did of the 60 urban school districts that comprise the membership of the Council of the Great City Schools.  Forty-six of those districts responded, for a response rate of 76 percent.  All of the data are for this current 2003-2004 school year. Several of the questions that were asked yesterday but that could not be answered because of the lack of data at the federal and the state level we have at least preliminary data on at the local level and should be able to provide some insight from the local perspective. The 46 cities that responded to the survey include 8,266 schools, enrolling nearly 6 1/2 million students.  1,694 schools in these districts are in either School Improvement I, School Improvement II, corrective action, or reconstitution, meaning that approximately 20 percent of all of the schools in the nation's 46 largest urban school systems are in school improvement status of one kind or another, including about one-third of all of the Title I schools.  We did not ask for data on the number of schools in warning. These 1,694 schools enroll approximately 1,168,000 kids, of which only about 44,372 students requested a transfer this year.  Approximately 18,000 of these students were accommodated, meaning about 11 students per eligible school were transferring out into a higher-performing school this current school year.  While small, this rate is about three times higher than the number of transfers last year, which totaled only about 5,000 transfers in these same 46 cities. We also asked the districts for a variety of data on how they implemented choice, including how it is they communicated with parents, how many choices they provided, the length of the windows, the number of receiving schools, the number of inter-district transfers, the number of open seats, and lots of other data. First, it was clear to us that both the choice and the supplemental services provisions were being affected to a large degree by the timing and the accuracy of the data coming from the states.  Only eight districts received preliminary data from their states on their AYP status before the end of the last school year.  Fifteen cities received preliminary data by the end of July.  An additional 12 received their preliminary data by the end of August, and 6 received their preliminary data after the start of the current school year. Thirty-five cities out of the 41 that have to offer choice did not get their final data on their AYP schools until August or later.  Twenty cities informed their parents about their choice options before the end of July, and then another 21 cities informed their parents about their choice options in August or after the beginning of the school year. Two-thirds of the cities informed their parents about their supplemental services options after the beginning of the current school year, meaning that the time frame for the implementation of supplemental services is considerably behind the time frame for the implementation of choice. Second, all of the cities use letters of one kind or another to inform parents about their choice options.  In many cases, it was only a single mailing.  About a quarter of the cities also conducted some form of community meetings, parent meetings, or other forums at schools or other community centers.  And ten of the cities actually advertised or used some form of automated telephone calls a little bit like political candidates use. Third, 39 of the cities provided at least two choices.  No one offered more than six choices, and two districts offered only one. Fourth, 19 cities had a choice window of at least 3 or more weeks, and 14 had a choice window of a month or more.  Only eight of the cities had a choice window of between one and two weeks. Fifth, we saw no instance where a contiguous school district took any of the NCLB kids from the major cities, although a number of cities have inter-district transfers for reasons other than NCLB. In general, it appeared to us that choice appeared to be affected by a number of factors.  One the number of available higher-performing schools was a mitigating circumstance in these districts.  The level of communications, the number of communications vehicles, also affected the number of transfers, the attractiveness or lack of attractiveness of the options and the capacities of the individual schools. We did not see any relationship between the length of the windows and the number of schools or the number of choices and the number of transfers.  We did see a clear relationship, however, between the dates by which the data were returned and the number of transfers, and we saw a clear relationship between districts with open enrollment options and their ability to implement the choice options at an earlier date, and smoother and with a greater number of transfers. We also asked parallel questions about supplemental services.  Thirty-two of the 46 districts responding to our survey were required to provide supplemental services involving some 1,196 schools.  This is the first year in which supplemental services are being offered in the vast majority of the urban school districts and their respective schools. In these 32 cities, 134,000 students are currently being served this school year with supplemental services, or about 112 students per eligible school.  The districts used approximately the same methods for informing the parents about supplemental services as they did for informing parents about choice, but they were more likely in supplemental services to use school, parent, and community windows.  And we think the reason for this additional use of these forums was because the windows for supplemental services tended to be open longer than the windows for choice. The average city has approximately 24 private supplemental service providers.  They range in size and in origin, whether they are profit or non-profit.  There are also a number of different instances where there were a lot more providers than there were eligible schools, meaning that the providers were probably having to compete over a very small number of schools. I don't expect that this situation will last in the subsequent years, but at least this current year I'm not surprised that some of the providers are frustrated with participation rates because in many cases what we have is too many providers for the number of eligible schools in some of these cities. In addition, 23 cities, or a little more than half of the cities that are required to provide supplemental services, are their own providers.  But many of these school districts also hire outside providers to conduct their own in-house program, and in many cases they are the very same providers as the external supplemental service providers. An additional 22 cities, or about two-thirds of all of the respondents, allowed providers on school grounds.  I suspect the additional third will probably loosen up in subsequent years.  There are some cases where some cities don't allow for-profit firms on school grounds.  Again, I suspect that that will wilt away over time. And then there are a number of instances where the number of providers is so large that urban school districts didn't want to be in a position of picking and choosing which providers should be on school grounds and which ones wouldn't.  And rather than having everybody on school grounds, wandering the halls after the school day, they basically told people that nobody could be there.  But, again, about two-thirds of the cities allow the providers to deliver their services on school grounds. Most of the cities are still signing their contracts.  About two-thirds of the cities have signed contracts with at least some of the providers, but most of them are still signing contracts with at least some. The area of contracts, as anybody who is familiar with this business knows, is a rather frustrating one.  It is taking substantially more time than people had expected.  There is a considerable amount of detail to this. Finally, on the contracts there are still two kind of outstanding frontiers in the contracting business.  One deals with the alignment of the tutorial services with the curriculum of the district, and the second involves the evaluation of the effectiveness of the services. Finally, we looked at issues of money.  There is an extremely large amount of money that's being reserved out of Title I funds for both choice and supplemental services, not all of which is being used.  But the amount of money is really very large, and we've calculated all of that, although we haven't put it in the paper.  It'll appear in the paper in the final version.  We did not see more than two of the cities that were using anything more than their Title I money to provide supplemental services and choice under the NCLB program. One thing that I should also correct from the first panel yesterday--there were a couple of statements that were made about the use of money indicating that if choice and supplemental services money were not used that those funds reverted to the general operating budget.  That is incorrect.  That is not what happens with the money under this law.  It does revert to the Title I budget, but it doesn't revert to the general operating budget of the school district, as was indicated in the first panel. In closing, let me just indicate that when the law was heading to the House and Senate floors of Congress about two years ago, the nation's urban schools actually stood up and supported NCLB.  We didn't do it after the fact; we did it as a vote was in the offing. We did it for three critical reasons.  We did it because of its emphasis on student achievement, on achievement gaps, and on the accountability for results.  And while there are considerable logistical problems in the implementation of the law and a fair amount of frustration on the ground with all of the particulars, we have not changed our minds about NCLB and the merits of doing the program. Thank you. MR. HESS:  Thank you, Mike. All of that information, for those of you who aren't intimately familiar with what's out there, is new.  And I know it's a lot of information Mike just threw at you, so I'd really encourage you to help yourself to a copy of the paper.  It's just an extraordinary amount of knowledge that we haven't known previously. Also, Mike didn't mention it, but I'd like to.  One of the things that has come out of the recent NAPE [ph] data that hasn't received a lot of attention, of course, is that Mike's districts have actually been one of the bright spots.  And there was reasonably consistent and impressive growth in a number of the urban centers in the most recent NAPE.  And as I say, I think that's something worth mentioning. We're now going to go in the order we originally introduced them.  We'll start with Julian and Anne talking about implementation in San Diego. MS. DANNENBERG:  Good morning.  Thank you very much.  I'm going to be going through very quickly the San Diego context, implementing NCLB choice in San Diego, supplemental service implementation.  And then Julian will be talking about early challenges, emerging long-term problems, and he'll be drawing some conclusions. San Diego in many ways is similar to other large California urban districts and, I would submit, to other urban districts nationally.  In particular, it has large percentages of non-white students, limited-English-proficient students, low-income students.  And the bottom row here shows the percentage of students in program improvement schools in the large urban districts in California, the five largest, and California as a whole. San Diego has a long history of choice before NCLB.  They had at least four public school choice programs--their own form of choice, their VEEP [ph], magnets, and charter schools. Now, this shows a graphic representation of the percentage of students who are participating in NCLB choice and non-choice.  Now, the choice on this slide is not NCLB choice; it's the district's choice.  So almost 25 percent of San Diego Unified students are in choice even before NCLB. One of the things that San Diego Unified did starting in 1998 was they implemented a blueprint for success, devoting major additional funding, including much of their Title I funding.  The district believes that this is already consonant with NCLB goals. Next, I'm going to be talking about the actual implementation of choice.  In '02-'03--I'm sorry--in fall '02, only 72 students applied.  Of those, 24 actually changed schools.  By fall 2003, there is greater lead time, there's better information, and the applications are up to 480.  Of those, about 263 actually did change schools.  They were able to piggyback the choice onto their existing bus routes. This is a graphical representation by grade span.  As you can see, most of the action is in middle and high schools, and by December 2003, there were about 692 applications for choice.  Supplemental services-- MR. HESS:  Anne, out of how many eligible students is that?  How many students were eligible to apply? MS. DANNENBERG:  Thirty-eight thousand are in PI schools, both one- and two-year. For supplemental services, the main provider is San Diego Unified itself.  What they've done is used the funds to enhance a tutoring program that's been in effect as part of the blueprint. In '02-'03, there was only one other provider serving only 3 students, and almost 4,400 students being provided services by the district.  By this fall, it's up to 5 outside providers serving a little over 100 students, and they expect that this year there will be several thousand students in the district program. There are 103 total providers in the California list, but that means there are 102 non-San Diego providers.  Now, the list for San Diego shrinks because, first of all, the providers must be willing and able to provide services locally.  Some providers lack capacity.  Especially, limited-English-proficient services is one example.  And they also have to certify health and safety requirements. In addition to that, 36 of the state-approved providers are other local education agencies and San Diego can't use them. So why does the district provide supplemental services?  And I'm actually going to let this district administrator's words speak for the district.  I'm not going to read this to you, but we do acknowledge that funding is definitely an incentive, as well, but the district feels that it's very coherent, in essence, to have their own people providing services to their own students. MR. BETTS:  The next thing we'd like to talk about is early challenges in implementation, and at the risk of being a little bit self-indulgent, we'd like to talk about some of our challenges in finishing this paper. We scheduled two school-site visits on a certain day in October in the middle of the terrible fires that you've heard about.  You see in the foreground one fire; in the background, 12 miles distant, the cedar fire, the biggest wildfire in California history.  And wouldn't you know it?  The location of our receiving school that we were going to visit was right around here. [Laughter.] MR. BETTS:  The sending school was just to the right of this picture.  Both schools were evacuated.  Lots of homes around here were burned.  They were not the least bit interested in us visiting, rescheduling for later, but the telephone can solve some of those problems. So you didn't come here so early to hear all these stories about how the dog ate our homework, so let's get on with barriers facing the district. We identified three short-term implementation problems.  The district essentially had two months to implement NCLB in the first year.  We describe in great detail the frenzy that this produced in the summer of 2002 inside of the district.  Anne worked really hard to set up meetings with nine community-based organizations.  What we heard from these organizations, which deal with thousands of parents each year, is that they essentially had no questions at all about No Child Left Behind from parents. And administrators agreed with the central point there, which is that in the first year parents did not understand No Child Left Behind.  A lot of administrators also felt that the state did a very poor job in the first year of providing districts with information about supplemental service providers. Remarkably, the district felt compelled to file a Freedom of Information Act request to find out from the state the criteria it had used to qualify supplemental service providers and to find out which providers were willing to work in San Diego. Moving on to more permanent problems that we've identified, the most obvious one which has been talked about already is the problem of timing.  California releases its new list of program improvement schools each year in mid-August.  In fact, it was closer to late August this year.  That gives the district essentially two weeks to identify new bus routes for choice, to inform parents, and to solicit applications.  One administrator quite bluntly said that this system is, quote, "completely unworkable." A second problem we found and administrators talked about a lot was inconsistencies in the definitions of adequate yearly progress and the selection of program improvement schools.  That's important because it determines which students are eligible for choice and supplemental services. The first problem that administrators are really aware about is that schools that serve diverse populations are not playing on a level playing field.  There's a statistical bias against those schools. Second, the fact that the federal legislation has a single cut point in test scores for determining school quality is problematic, and you can either understate or overstate the improvement in a given school.  And the lack of a longitudinal component causes a lot of problems that I'll talk about more in a second. The third systematic, long-term problem we found is what appears to be inconsistencies between the federal legislation and the federal regulations when it comes to capacity constraints in busing.  I'll let you read this quote in red on your own, but basically you will see the mind of a very confused person there. The two constraints, first, are--the first one, the more important one is the funding constraint.  What happens when you come up against that constraint? The second constraint is what happens when a given receiving school feels that it can't take any more students?  The federal regulations say that a district cannot use that as an excuse.  The district is extremely worried about overcrowding problems and funding problems in the future. We also found some unintended consequences from the way that NCLB has been implemented.  The first is that the movement of students through NCLB choice could actually condemn both the sending and the receiving schools to program improvement status through falling test scores. What appears to be happening in San Diego is the students who are leaving PI schools are not the very lowest-scoring students.  They tend to be slightly above average.  So this is going to lower the test scores in the sending schools even further. At the same time, these students score far below the typical student at the typical receiving school, and we have one quote here from a principal who is more than afraid that it will lower test scores.  And there are implications here.  Already have the high schools in the district are PI status.  That number will increase because of this anomaly. The second unintended consequence really has to do with a subtle implication in the legislation that no districts have choice programs and no districts have remedial supplemental service programs either.  In this district, neither of those statements is true, and what came out of our interviews is that potentially we can see the choice provisions interfering with supplemental services. The district is the main provider.  These things happen after school, and we have a quote here from a principal at the receiving school that basically says that a lot of the kids who are bused in under NCLB will not participate in the remedial services offered in that school because they don't want to take the late bus home because they get home as late as 7:30.  It's a real issue. A related unintended consequence is interactions between NCLB choice and the existing choice programs in San Diego.  The district is taking NCLB choice very seriously.  Even though there are very few students participating right now, the district foresees that that is going to ramp up dramatically and they believe that they can run out of Title I money very soon. A likely outcome is that the existing choice programs, which have a history of over a quarter of a century in which they were very popular especially among disadvantaged parents, are going to disappear.  We also heard some stories about resentment by parents in the other choice programs.  The district is giving NCLB choice applicants top priority. Some other unintended consequences we heard about is the potential for "us/them" tensions at the receiving schools; more than potential, the fact that there's a great deal of public confusion in San Diego between the state and federal accountability systems, which in some senses are quite inconsistent. And one administrator talked to us about something we hadn't thought about.  There's potentially an adverse impact here on charter schools.  No matter what, a charter school can be affected.  If a charter school is high-performing and elects to receive students, it becomes subject to a lot of federal rules that it would not otherwise be subject to.  And in some ways, that obviates the whole purpose of a charter school, which is to escape the standard set of regulations. On the other hand, if a charter school is deemed program improvement--I think there are 2 out of 21 in San Diego--it now must depend upon the district to find busing to get students out of that charter school. In conclusion, we wanted to end on a fairly bright note if we could.  What are some options for solving some of the main problems?  So starting with the timing problem, where the district really has two weeks to design new bus routes once a school becomes designated PI status, the best solution we heard from administrators was something we were actually thinking about ourselves, which is to reverse the order of things. Right now, in the first year of PI status, a school is required to offer choice.  Only in the second year are you required to offer supplemental services.  We think that reversing this order would make a lot of sense.  Instead of giving the district and parents two weeks to make up their mind about choice and to plan new bus routes, why not give the district a year to plan the new bus routes and to advertise it in its fall glossy brochure on choice?  I think that would make a lot of sense. The second solution we have up on this slide is not nearly as good a solution.  That solution is to use the prior year's test scores to determine PI status this year. The third solution is to use computers to administer tests so that we could have real-time testing, and we might have results then in June instead of late August.  That's a very expensive proposition.  We're probably going to land on Mars before that happens. A couple of suggestions on how to adjust the definitions of adequate yearly progress and PI status.  It would make a lot of sense to use students who were--only the students who were in a given school for two consecutive years to determine growth.  That would help a lot in reducing the number of anomalies. Second, we really need to do something to reduce the statistical bias against schools that are serving diverse populations.  We need to simplify and align accountability.  In essence, there are three accountability systems in San Diego.  The district has its own tests and accountability, the state has its own tests and accountability.  We have the federal layer on top of that.  This is where really the title of our paper, "Too Many Cooks Spoil the Broth," comes from. We feel that in any reauthorization of the law, it would make a lot of sense to give much greater ability for states and individual districts to qualify their programs that meet at least the intent of NCLB, if not the letter of the current law. Improving information dissemination: There are a lot of suggestions in the paper coming from district administrators as well as the community-based organizations.  You'll have to read that on your own, I'm afraid. Overall conclusions: The glass is half full, I think, in San Diego.  Implementation is proceeding reasonably well--86 percent uptake rate for supplemental services, about a 1 1/2-percent uptake rate for choice, but that percentage is growing quickly. Unlike a paper you're going to hear in a few minutes, we could find no evidence that the letters that the district is sending home to parents are attempting to dissuade students from implementing choice or from using outside supplemental service providers. In a way, the biggest surprise to us is that really there's a lot of support in the district for the basic thrusts of No Child Left Behind, even if we did hear a lot of complaints about implementation.  And I think this final quote from one administrator sums up the feeling we heard many times: "We support the goals of the law.  They are the same as our goals." Thank you. MR. HESS:  [INAUDIBLE] Julian.  Thanks, Anne. Now, we'll go to the other coast, to Worcester, Mass., for a slightly different perspective on how a district is working on challenges. MR. HOWELL:  If the glass is half full in San Diego, the glass is shattered on the floor in Worcester, Massachusetts. [Laughter.] MR. HOWELL:  Worcester is a town 30 to 40 miles west of Boston that serves about 25,000 students, making it the third largest district in the state of Massachusetts, just below Springfield.  A blue-collar town, upwards of 50 percent of the students qualify for a free or reduced lunch, and there's a rising non-white population in the town. As of June 2003, 12 of the city's schools were deemed in need of improvement for 2 years, and hence qualified for the choice provisions.  And 5 schools were deemed in need of improvement for 3 years. Under the law, a total of 4,700 students qualified for choice; 4,700 students qualified for choice.  And 1,800, roughly, students qualify for supplemental services from a private provider.  As of today, although these numbers may be dropping, there is one student who is exercising choice and switching school--one.  And there is one other student--it's not the same student--who is receiving supplemental service from a private provider. So the question is why?  What is going on in this district?  And that's really what this paper is trying to get a handle on.  And I'm going to suggest five reasons why NCLB's choice provisions have failed to take hold in Worcester, Massachusetts. Let me start where the paper begins with Joseph Caradonio [ph].  Caradonio is the Superintendent of Public Schools in Worcester, and he's also the only superintendent to serve on the state NCLB implementation team.  He's a big figure and here's a sample of what he has to say about NCLB, and I'm quoting. "Reductio ad absurdum, you know, but this is what we're dealing with in terms of this insanity.  Oh, it's numbers and it looks great.  We've got numbers, simple, and it looks great.  And it's just killing--killing teachers and killing principals." He continues, "Well intentions gone awry, absolutely insane.  This law violates a lot of just simple things about testing and statistics and sampling and all that.  But people in Washington just kind of get that wild cowboy look in their eye and they just don't give a damn   ?    in Washington." And I start with Caradonio, who is obviously an outspoken critic of NCLB, not because he's got sort of the flair for language, which he clearly does, but because the federal government needs to take his views seriously, because what the federal government has effectively done is handed over the implementation of this law to him.  He's the big player in this district.  And having handed over the implementation of NCLB to him and to those who work for him, it's little wonder then that just one student has switched schools. Let me explain, just give you a sense of the system that's been put in place in Worcester and how parents can switch schools, what parents have to go through. First, a letter was sent in the spring of 2003 that explained to parents a number of things.  It explained that under-performing does not mean failing; that lots of exciting things are happening in the public schools, a lot of new reforms are being implemented; that by switching schools, students may forfeit their rights to supplemental services; that there's no guarantee that students will receive their first choice, that parents will receive their first choice, nor that siblings will be able to attend the same school as one another, but that if you're still interested, parents, you should talk to your principal. And, by the way, the district provides supplemental services as well, and the wonderful thing about the district providing supplemental services is that there's a coherence between the learning that goes on in after-day programs and the learning that goes on during the school day itself.  And so there's this fabulous alternative. So then those brave souls who decide to go meet with their principals to say I want to leave your school sit down with principals and principals reiterate many of the themes that were laid out in the letter, namely that the law is troubled, that public schools are improving, that district supplemental services are best, and that choice options are usually not in the best interest of kids. If you still want to exercise choice, you now need to go downtown and have another meeting with the head of the Parent Information Center, who reiterates many of these same themes.  You can see the pool mat quickly winnowing until we've got one child switching schools. The irony, it seems to me, though, is that these principals and the head of the Parent Information Center may well be correct in saying that your interests are not best served by switching schools, which leads me to my second reason why NCLB has failed to take hold in Worcester. Worcester public schools serve extremely mobile families.  During any given year, roughly a quarter to a third of Worcester public school students are switching schools.  And if you look at student in under-performing schools, those mobility rates jump up to upwards of 50, 55 percent.  Kids are switching all over the place in any given year. And the causes of mobility by all accounts have little to do with parents seeking better schools, per se.  Rather, as--and I'm going to quote here from Brian O'Connell [ph], who is the vice chair of the schools committee.  He notes, "Parents are moving from one apartment to another.  Husbands and wives are separating.  Mothers and boyfriends are separating.  Evictions are taking place." You can see, right, there's a lot of underlying causes that are leading students to switch schools in any given year.  And so rather than contribute to student mobility by encouraging families to take advantage of new choice opportunities under NCLB, principals and teachers for the most part are trying to contain it, and for good reason, for every time a parent switches his or her child from one school to another, two classrooms are disrupted and two school administrations have to adjust, further interrupting opportunities for student learning. So with such a mobile population, I would suggest it makes little sense to conceive of individual students being trapped in failing schools, a condition that NCLB intends to correct.  Schools may be failing and students may well benefit from switching schools, but wholly divorced from NCLB, a tremendous amount of moving is already going on; so much, in fact, that schools are struggling just to keep track of which students are coming in and out of their doors with any given year. The third reason why NCLB has failed to take hold in Worcester is that choice already exists.  There are, as there were in San Diego, a bunch of charter schools, a bunch of city-wide magnet schools that parents are participating in.  And there's a desegregation, a voluntary desegregation plan that's currently in place, and let me just say a couple of words about that desegregation plan. Choice under this program is highly constrained, and for three reasons.  One is that what Worcester has essentially done is broken up the district into four quadrants, and parents can switch schools--they can ask to switch schools, but only within their given quadrant. Second, because each of these residential zones has only one middle school and one high school, choice, to the extent that it exists, occurs strictly and solely among elementary schools. And, third, again parental requests are not necessarily honored.  While grievance procedures exist, the Parent Information Center ultimately can assign students when and where it pleases. So at first blush, what you might say is that, well, choice already exists so that all NCLB is doing is amplifying the array of choices that we're presenting to parents, and so all is to the good. But there are some deeper underlying tensions between the state and federal laws that are worth noting, and the reason is simply that choice is being offered in the service of two different objectives. NCLB is offering choice with the idea that what we're going to do is we're going to free students in failing schools.  The deisolation plan, by contrast, intends to improve the racial balance between student bodies.  So what happens when a white child attends a predominantly minority, albeit under-performing school?  What should the district do?  Should they serve the purposes of deisolation and keep that child there, or should they serve the purposes of the federal mandate and switch the schools, and therefore undermine the state law? And I think what's worth recognizing here is that when we think about the successes and failures of No Child Left Behind, we need to think about them given the backdrop of local regulations and rules and programs that are in place in each district and in each state.  It doesn't make sense to think about NCLB in isolation. The fourth reason why NCLB has failed to take hold concerns parental knowledge of and interest in school choice, and here I'm drawing from the survey that Rick alluded to earlier on and let me just say a few words about it. At first blush, parents in Worcester appear to know a fair amount about NCLB.  I had an over-sample of parents in Worcester, so I can isolate their responses.  Among those surveyed, 68 percent claimed to have heard about the Act, 50 percent claimed to know about the option of switching to an under-performing school, and 39 percent claimed to have heard about the availability of supplemental services. This strikes me as actually sort of encouraging.  The numbers are relatively high.  But because I know the names of the schools that the students are attending, I can identify whether or not parents, in fact, correctly know whether or not their school is under-performing, which is certainly the basic level of knowledge that parents are going to require in order to take advantage of the choice and supplemental services of No Child Left Behind.  They got to know whether or not they qualify, right? Overall, in Worcester, 49 percent got it.  They were able to correctly identify when asked did you--is your child's school under-performing, 49 percent of the time parents got it right, all right.  So you might say that's pretty good. Well, when you break it out by whether or not a child's--the parent's child is actually attending an under-performing school, huge disparities.  Parents with children in performing schools get the status of their child's school right 60, 65 percent of the time; that is, 60, 65 percent of the time, they have the knowledge that's required to know that they don't qualify for No Child Left Behind, right? Among parents with children in under-performing schools, they get it right in Worcester 7 percent of the time; that is, when you ask is your child's school under-performing, 93 percent of the time parents say I don't know, or they hazard a guess and they get it wrong, right? Clearly, there's a huge informational problem that we need to come to grips with when we think about conveying this information to parents who qualify for choice if we want to see these take-up rates increase. One thing interesting that's worth noting is that interest levels actually are markedly higher--interest levels in pursuing alternative educational options are markedly higher among parents with children in under-performing schools.  They do want out. Two things I want to say about that, though.  One is when asked about was there an alternative public school within your district that you'd rather send your child to, 40, 45 percent of the time when parents name a school they name another under-performing school in the district. Now, this isn't foolish on their part.  In the survey paper that I've got--you can take a look at it--they're actually picking schools that have higher test scores, more advantaged student bodies in a variety of ways, right?  But these are not options that are available to them under No Child Left Behind. When you ask about a wider array of school options--another public school within the district, a public school outside of the district, a charter school, or a private school; is there another alternative educational option that you'd rather have your child attend--far and away the single option that animates parents the most are private schools.  Fifty percent of the parents in Massachusetts as a whole claim, were cost not an obstacle, I'd rather have my child attend a private school.  Many of these schools that they have in mind are elite boarding schools, but that's what gets them excited, right?  But, again, this is not an option that's available to them under No Child Left Behind. Okay, the fifth reason why choice has not--No Child Left Behind has failed to take hold in Worcester, as best I can tell, simply has to do with the fact that the job of placing students in individual programs--I'm going to talk about supplemental services now--plainly falls upon districts. And given minimal state oversight, districts essentially hold captive clienteles.  They can wax eloquent about the benefits of their own programs directly and repeatedly, as they do in Worcester to parents, while giving short shrift to private providers who have few opportunities to interject. As the vice president of Kaplan K-12 Learning Centers told me, "The district is the owner of the relationship between the provider and the parent and I can't get in.  Private providers don't even know who qualifies for their services in Worcester, and companies, as such, must rely upon the district and its representatives to present their services to parents in as favorable a light as possible," which strikes me as akin to asking a BMW dealer to extol the benefits of buying a Volvo. Why aren't more parents jumping?  Five reasons I've [TAPE CHANGE]. [IN PROGRESS] have few incentives to aggressively promote NCLB choice and supplemental services provisions.  Two, Worcester public schools already face alarming student mobility rates, and hence have good cause to restrict additional movement, student transfers, when requested. Number three, the district oversees a state-mandated desegregation whose objectives can conflict with NCLB's.  Number four, parents know very little about the choice options that are available to them and they lack the most basic information required in order to exercise choice under NCLB.  And, number five, private providers of supplemental services have few means by which to communicate directly with would-be customers. Assuredly, some of these problems are going to work themselves out over time.  We all admit this is a brand new program and a lot of the kinks are being worked--or we're trying to work through.  As providers advertise their services and as media outlets continue to report on school performance rankings, information about and interest in NCLB's choice options may spread. Eventually, parents may even conceive, it seems to me, of new rights to exercise choice which may well accelerate the process of change.  But a basic challenge confronts the federal government and will continue to do so as long as demands for state oversight and accountability butt up against the realities of local school control. Districts, in short, have few incentives to fully implement and faithfully execute NCLB.  And absent wholesale changes in the federal government's involvement in public education, little can be done about it. Here is the deal: Where we have seen sort of progress, it seems to me--a number of these papers are saying where we've seen sort of inroads is precisely where change was already occurring, where states were already considering accountability measures and offering choice opportunities to parents.  And so it's not clear what the value-added is of NCLB, if you want to take a dire view.  All the federal government is doing is putting a stamp of approval on developments in local districts.  To get and sort of assess the independent influence of No Child Left Behind, what we ought to pay particular attention to is its ability to make inroads in districts like Worcester, where there is widespread opposition to these notions of accountability and choice. And we ought to sort of pay careful attention to whether or not additional inroads can be made.  Thus far, 18 months, 2 years into the program, there's little reason for hope, it seems to me, at least in Worcester. MR. HESS:  Thank you, William. MR. CASSERLY:  By the way, I should clarify that Worcester is not one of the Great City Schools. [Laughter.] MR. HESS:  Finally, a bit closer to home, Doug Reed, looking at our neighbor, Montgomery County, in Maryland. MR. REED:  For those of you who live in Washington, this may have some relevance, or near Washington. I'm looking at Montgomery County and I'm most interested in the interaction between the process of designating a particular school as a receiving school and the flows of students to those schools, because part of what seems to be coming out here is the district, as it develops its response, even if it's outright opposition or if it's a kind of grudging acceptance, there's still a whole series of decisions that have to be made about what are the political realities that it has to do deal with as it chooses receiving schools.  So I'm looking partly at two things, the process and criteria by which receiving schools are designated, and then whether that influences who goes, who makes the decision to exercise choice. First, I want to talk briefly about the context of school choice in Montgomery County, basic demographics.  Then we'll identify who the receiving schools are and look at the process of the criteria of selection, and then we'll try to see whether the travelers, as I call them, exhibit any particular characteristics in both year one and year two of choice in Montgomery County. I'll have a brief discussion about supplemental services.  That is just getting underway in Montgomery County and is quite incomplete at this point, and then conclude with some policy implications based on this. The characteristics of Montgomery County: It is a changing school district, it's a growing school district.  It's the 18th largest school district in the county [sic], about 140,000 students now.  It is a majority/minority district.  Only 45 percent of the students are white and the rest are African American, Asian, and Hispanic.  It is an increasingly diverse district. It has relatively high expenditures per pupil, upwards of $10,000 per pupil, and a relatively high tax base.  And it's also relatively high-scoring, high 60th percentile on norm-referenced tests in recent administrations. So, in sum, it's larger than most school districts, it's less white, it's more affluent, and it's higher-scoring, which gives the potential for choice to be a rather disruptive force, especially to the extent that those low-scoring schools, to the extent they exist in Montgomery County, are geographically concentrated. So if you've got disparities within the district, that's a very different thing that you've got uniform levels of under-performance like you might have in Baltimore City, for example.  So to the extent that you've got disparities, choice could really--if there's significant flows, could be quite a potentially disruptive force. As choice unfolds in Title I schools in Montgomery County, 10 schools were deemed as in need of improvement based basically at the passage of No Child Left Behind because of previous identification under Maryland tests, the MS--MSPAP [ph] test.  All of those are elementary schools; no junior highs or high schools are in the choice system. That means about roughly 6,200 schools were--I'm sorry--6,200 students were eligible in year one for school choice, and those are concentrated in two geographic clusters.  The light blue indicates the sending schools.  If you look to the lower right, that square is the District of Columbia.  Bethesda is just across the border.  This first cluster right over here is Silver Spring-Wheaton.  The next cluster farther up is Gaithersburg.  Wheaton has a significant Hispanic population, new immigrants.  Silver Spring has a longstanding African American community, and Gaithersburg has a mix of both. So where can the students go?  The district had criteria that it expressed for the selection of receiving schools based on test scores, school capacity, transportation time, and it excluded any school attendant zone whose boundaries were under change or being considered for a change. That criteria is a bit suspect.  It may be essentially a fudge factor.  But using those criteria and adding an additional criteria, poverty, for obvious reasons, we can predict which school attendant zones are eligible for consideration. And if we look at that map, the blue again are the receiving--I'm sorry--the blue are the sending school attendant zones.  The red are those that were actually designated as receiving school attendant zones.  The gray are those that meet the district's criteria if you're taking test scores as greater than the county mean, poverty levels as less than the county mean, and capacity--that is, the number of portables on a school campus--as less than the county mean. And so the selection, those that are bounded in red--we've got to come up with some reason to exclude those that are gray, right?  There's something going on here that not all of the gray ones were chosen, and it seems that there's some other additional criteria that's at work as we go forward.  This is Bethesda down here, and then farther up.  It should be noted that there's also a significant rural portion of Montgomery County that often is forgotten, but it is there. Okay, so can we get a more precise means of determining what the receiving schools are?  Can we sort of reverse-engineer some of this?  So I came up with some other criteria.  If you sort of relax the test score criteria by half a standard deviation, create some    ?    poverty criteria, keep the capacity criteria the same--I think that's a relatively fixed thing that districts can't fudge--and make it greater than--the county overcrowding can't be more than the county mean, and throw in race as a consideration--and I tried to select those in which the minority enrollment at the receiving schools is at least 33 percent.  How does that predict? Okay, one correction.  There's an error in this map.  This is not a receiving district, okay, or receiving school attendant zone.  That's just a coding error in the data.  I apologize. What we get here is a greater success at predicting receiving schools, okay, when we take race into account.  We have about the same error rate as a false prediction, those that are bordered in red but not shaded, so that the model doesn't predict that.  But we get a more precise fit between the criteria and the receiving school designation. Okay, and these are the pairings that wind up for the first year.  They expanded the choice options for the second year.  They only gave one choice for the first year, which actually is in violation of NCLB, but we'll give them the first year, okay?  The second year they complied and gave at least two choices for every year. What you see is that distance becomes the thing that's sacrificed, okay.  The longest commute is between Wheaton Woods and Belmont--the orange in the center, Wheaton Woods, and Belmont up in the north.  It's roughly about 10 or 11 miles apart through, as many of you who live in Montgomery County might know, relatively congested roads.  Broad Acres and Stonegate is relatively far apart, that pairing.  Some are relatively close.  So we did a mix of distance and close pairing. Okay, now who elects to enter into this send/receive system, given these school attendant zones?  These are the demographics of the travelers, as I call them, in year one and year two. Now, we have to also look at those who drop out of the system, okay, and so I call those the--well, those that stay in are called the continuers.  There's two reasons why someone may drop out.  If they begin in the fifth grade, they're not eligible for choice if they graduate from fifth grade because no junior high is included.  Or if they move out of the school attendant zone, that is a sending zone. So if you look at the county average and the sending school average of students who receive free or reduced-price lunch, they didn't give me that data for the first year.  I didn't know if they were embarrassed about it or it just wasn't available. For the second-year entrants into the system, it's roughly a third of the entrants meet that poverty criteria.  The sending school average is 62 percent, okay, so only about half of the sending school average. In terms of racial demographics, about 25 percent, on average, of the sending schools are African American.  The year-one entrants--only about 15 percent of those entering are African American. The one that really leaps out here, though, is the white schools--I'm sorry--the white students at the sending schools in year one.  Half of the participants in year one are white, even though they only comprise roughly 13.5 percent of students at the sending schools. Granted, these are very low numbers.  We only have 88 entrants into the system in year one--more than Worcester, probably roughly comparable to San Diego.  Sixty-six of those continued on into the second year, and we got a significantly greater infusion in year two that has a more reflective racial profile, okay; that is, it's more broadly distributed across racial categories than in year one. But if you look--and it's not here in this chart, but if you look at the percentages of students that are either Hispanic or African American and poor in year two, that, at best, is 20 percent of all the participants, okay.  So, arguably, the target group for No Child Left Behind are African American, Hispanic students in poverty at under-performing schools.  They are only roughly about 20 percent of the population pool of travelers in Montgomery County. Another thing to note is the significantly low participation rate for Hispanic students.  Only eight of the first-year entrants were Hispanics, even though they're half of the students at the sending schools.  That bumps up a bit to about 14 percent in year two, but they're still significantly underrepresented in the sending pool. Okay, now here we need to think about, well, what's influencing these flows?  Do the characteristics of the receiving schools influence those who are participating? And if we look at the--now, these are simple correlations because I've got relatively small ends here.  In year one, I only have ten schools.  It's hard to do a regression, so these are just simple correlations.  So be aware that there may be other things going on here. But in year one, the travelers really are paying attention to test scores.  That's a lure.  The higher the test score in the receiving school, the greater the numbers of the flows.  The percentage of black students at the receiving schools is a negative indicator; that is, to the extent that there are more black students at the receiving schools in year one, the less likely that students are to exercise choice to that school. Distance is not significant here.  In fact, the schools--that Wheaton Woods-Belmont pairing had the second highest participation rate, okay.  So people were very willing to travel relatively long distances. Year two is different, year two is different, and I think we should be aware of this.  Test scores are not as relevant, race is not relevant.  Percentage of students in poverty are relevant for both years.  If a receiving school has a higher percentage of students in poverty, there is a negative correlation with the numbers of students who go to that school. Okay, supplemental services.  There's not a whole lot known yet.  As is probably true in other places, Montgomery County is just getting out of the blocks on this one.  It's only available for the ten Title I schools.  The contracts were sent out in late November with a return date.  They asked them to have them back by mid-December to get the programs up and running by early January. There's six potential contractors.  Whether they've signed or not is uncertain, beginning most likely, like I said, in January of this year.  450 out of 1,200 students eligible have signed up.  So the take-up rate in supplemental services is greater.  And the district is applying to be its own supplier of supplemental services, as in San Diego County. Okay, policy implications here.  Student poverty limits entrants to choice, but in two ways.  Not only are those in poverty less likely to exercise choice, but to the extent that the receiving schools have students in poverty, students are less likely to go to those schools, okay.  So poverty limits the exercise of choice in two directions, both sending and receiving.  When you widen the number of options, school choice becomes more inclusive.  In the year first, we had a constrained set of options and it's an indication that it was less inclusive. I think there needs to be greater attention paid to Hispanic students here.  The participation rates in Montgomery County are low, and that needs to be further explained and accounted for. We also need to pay attention to the segregationist effects, potential segregationist effects of choice especially in the sending schools.  To the extent that white students are leaving and students in poverty are staying, that could intensify the problems there, and more attention to supplemental services. MR. HESS:  Okay, thanks so much, Doug.  Thanks to all the panelists for just dynamite stuff. Andrew and Emily will be coming around with the microphones.  Please catch my eye.  When you ask a question, please identify yourself by name and affiliation.  And as always, please make--let's try to keep it to questions.  Speeches to which a question mark is appended at the end will be disregarded at the moderator's discretion. Right here. MS.            :  Yes.  One of the papers yesterday described districts as-- MR. HESS:  Lynn? MS. OLSON:  Lynn Olson [ph], Education Week. One of the presenters yesterday described districts as bullies in this process because they may have a potential conflict of interest.  And, Mike, I wonder if you could address what you're seeing, and then if some of the other panelists, you know, could address that as well because it doesn't seem to be a clear picture. MR. CASSERLY:  Well, I wouldn't characterize the school districts and I wouldn't characterize my own school district as bullies.  There's obviously a difference of perspective and interpretation here. The conflict of interest question, I think, is an open one.  I don't consider the fact that school districts are sometimes their own providers as necessarily as a conflict of interest. And one also needs to keep in mind that even when many of these cities are their own providers, they're still often using outside providers on a contract basis to provide their own in-house services.  So I would take issue with the bully characterization. MR. HESS:  Anyone else? Okay, back here. MR. MARANTO:  Hi.  Bob Maranto, from Villanova University.  I loved all the papers.  I have two sort of questions.  One is if you could speculate on the degree to which the power of local districts may make it difficult for states in some cases to push them to implement the law. And the second one, cheating a little bit, if you can speculate on the degree to which over the long term the choice parts of NCLB are likely to lead to a little more segregation or a little more desegregation and what that might depend on. MR. HESS:  Anybody. MR. HOWELL:  I don't know the ethnicity of my one student who switched schools. [Laughter.] MR. HOWELL:  So I don't know if that improved matters or made matters worse. MR. CASSERLY:  I don't know about the segregation effect in the cities, but the demographic characteristics in the cities is really vastly different from your average school district.  Our average school district is 80-percent African American, Hispanic, and Asian American.  So, you know, I think the situation is much, much different than it is in a Montgomery County, for instance. I'm sorry.  Your question about the state relationship, Bob? MR. MARANTO:  It strikes me that if you've got a 3,000-student school district, the state could probably have the political power to come down on that district if they don't think they're measuring up.  If you've got a district like Montgomery County, does the state really want to tangle with them?  So I guess I'm wondering about that sort of political dynamic in all this. MR. CASSERLY:  I don't know.  It's probably different for all kinds of places, but I haven't yet seen a whole lot of state kind of active intervention.  I also haven't seen a whole lot of active state technical assistance at the local level.  As a matter of fact, the states have largely been invisible at the local level, other than providing data, sending down the AYP benchmarks and other regulations. So I haven't seen in either large or small districts yet, anyway, much in the way of intervention or assistance.  They've been large invisible. MR. BETTS:  I mean, in San Diego I don't think there's much potential for segregation in the short run simply because the busing patterns are piggybacked on the voluntary ethnic enrollment program.  So by design, a student moving from one school to another will make both schools more representative.  I think a counterpoint to your point about the large districts having more power is they also get more attention. MS. DANNENBERG:  I'd like to expand on what Julian just said.  We don't have the race or ethnicity of all of the students who are exercising choice in San Diego.  But in speaking to some of the receiving schools, their influx of students are not primarily white.  They are primarily minority students and they're coming into primarily white schools. MR. REED:  I just want to make one about the segregation.  I mean, I see it most in year one, and I think there may have been a group of folks that were primed to move and that may just be a one-time thing. MR. HESS:  Back here. MR. OLAF:  Good morning.  Mark Olaf [ph] with the National League of Cities.  I have a question for Anne and Julian about San Diego, especially supplemental services. We heard about San Diego Six to Six and all the different after-school providers there.  Yet, your data shows that not a whole lot of them signed up as supplemental ed service providers.  I was wondering if you had interviewed any of them to find out why. MR. BETTS:  We unfortunately didn't have time to do that.  We did hear from administrators about some of the issues.  So in the first year, their real problem was information.  The state did almost nothing to provide information on what the criteria were to quality as a state provider, and almost nothing to tell individual districts which of these 103 providers were willing to work in San Diego County.  They had to do a lot of work on their own and create surveys.  So there's an awful lot of work involved there. One of the challenges is there are a number of providers who apparently want to work on an off-site basis.  One was very close this year to signing a contract with the district, but it couldn't secure an off-site location to do this.  So that's one of the issues, I think. MR. HESS:  Okay, right here. MR. McGWYNN:  Patrick McGwynn [ph], Brown University.  Mr. Casserly mentioned that these schools are having to put aside a relatively large amount of money from their Title I funds for the choice and SES provisions, and that was a point that was made yesterday as well. I am curious, though, how that's impacting the programs and the services that are being provided to these students.  I mean, there's essentially a loss of funds here.  What's that doing to these schools? MR. CASSERLY:  It looked to us like about half of the cities were being required by their states to explicitly set aside 20 percent of their Title I allocation or an equivalent for choice and supplemental services.  The actual amount that they are budgeting is pretty darn close to that amount.  It doesn't mean that they're actually spending it yet because the demand level is fairly low so far. What is also pretty clear is that in the setting aside of that 20 percent or so, it's largely taking up the increase that they got in their Title I allocation over the last couple of years, which has run about 25 percent or so, on average. So, in general, what's happening financially is that the increase that you see in the federal appropriation is largely going to budget for the choice and supplemental services and the underlying program has not been substantially affected. MR. HESS:  Back here. MR.         :  Yes. MR. HESS:  Oh, actually, behind you.  I'm sorry.  We'll get to you. MS. STOLL:  Hi.  I'm Carol Stoll [ph] from the Council for Basic Education.  I'm wondering what kind of alignment is carried out between the standards and the district curriculum and the supplemental services, if any. MR. CASSERLY:  This is one area, as I said in my presentation, that's kind of one of the last frontiers on the supplemental service front.  It was pretty clear that the states are approving the providers on pretty loose alignment with the overall state standard; that is, if the state has a standard for reading and you are a supplemental service provider in reading, sometimes that's often close enough in terms of alignment. When you get down to the individual at the local level, it gets trickier.  So far, while there are discussions between the school districts and the supplemental services about aligning the tutorial services with the district curriculum, I know of only a handful of places where that alignment has actually been done in any detail. I suspect over the long run that this is going to be quite a struggle.  Some providers are probably going to want to do this more aggressively than others.  Some districts will insist on it more than others. My guess is that the success ultimately of the supplemental service provider program in its ability to improve student achievement will rest at least in part on the success of this alignment process, which has barely started. MR. HESS:  [INAUDIBLE.] MR.          :  I was intrigued by Professor Howell's description of how Worcester informs its parents of the status of their schools.  I'm wondering if this is affected one way or the other by the transparency or the user-friendliness of the state information, since this is essentially, you know, your school falls into a category that is determined by a state standard. And I know in California, for example, the academic performance index gives people who want to look at it lots and lots of different ways of understanding the performance of individual schools.  Does that have any impact at all on sort of elevating the level of understanding? MR.          :  Is the question primarily about Worcester or California, or both? MR.          :  Does it work in California and does it work, Mike, in any other places that you know about that have relatively user-friendly state accountability systems? MR.          :  I think in California the way API is set up makes a lot of sense, and a lot of people spend time on the Web looking at these things as published in all the local newspapers.  And you're right.  There are smart ways that the state is providing information.  They compare your school to every other school in the state and to other schools that demographically look similar, and I think that's valuable. Here's what one administrator said: "It took a number of years for people to understand the state accountability system.  In California, people are just now getting the idea.  NCLB adds a new layer on top of that.  Unintended consequences are most likely to crop up in public relations.  AYP is very hard to explain to any audience." And I should just interject here the district has a 21-page PowerPoint presentation on how to define AYP, and they find it very hard to get this across to folks. So going back to the quote, "I worry that some schools are about to be blind-sided.  Even though they are genuinely improving, we could be overly reactive.  Parents may flee schools based on very incomplete information, because a single number doesn't tell all.  Conversely, a pretty mediocre school could meet AYP even though all sub-groups just barely met targets." MR. CASSERLY:  Yes.  I suspect--I'll just give you a short answer--that the user-friendliness of the state data makes a big difference.  Obviously, if a lot of the data, however, are on state websites, we'll have an issue in many urban communities where parents don't necessarily have access to computers and won't necessarily think to check the state website.  They may be more likely to go to the local website. And a lot of the information is still brand new both for the district and for the individual parents, which is causing some of the confusion that you heard about.  And the fact that much of the data is being finalized fairly late--that is, after the beginning of the school year--means that there are some schools that are falling in and some schools that are falling out of the school improvement status, which is probably contributing to the inability of parents to tell whether their school is in school improvement or not. MR. HOWELL:  If I could add to that, accountability isn't new to the state of Massachusetts either.  They've got an up-and-running accountability system that actually is thought of quite favorably in Worcester.  I can give lots of really positive quotes by teachers and principals and Caradonio himself saying that we are all for accountability in testing. Two things are worth noting.  One is that lots of schools--not lots, but a number of schools that fail under NCLB actually look quite positive under the state accountability system, which generates a fair amount of confusion.  This is a point that I thought Jane Hannaway's presentation made quite clear yesterday. So you've got different signals being sent from the state and federal government, number one.  And, number two, the objections by administrators and principals in Worcester have to do again not with accountability so much as they do with the consequences of failing to meet AYP under NCLB, the idea that we're going to help kids by boosting additional school mobility rates and by transferring money out of our coffers and sending it off to private providers is the source of the objection by the school officials in Worcester. MR. HESS:  Right here. MR. ABRAMS:  Hi.  I'm Andy Abrams [ph] from the U.S. Department of Education.  Yesterday, there was a discussion about the disincentives for receiving schools to accept students from sending schools.  I was wondering if you know of any districts where you have schools in relatively close proximity, some of which are higher-scoring or high-achieving and affluent, and down the street you have low-achieving, less affluent schools that have actually managed to provide incentives for the higher-achieving schools to actually want to receive the students. I think, for example, of schools in Bethesda and schools in Silver Spring that are in very close proximity.  Schools in Bethesda are extremely affluent and high-performing.  Those in Silver Spring are not.  What is the incentive for a Bethesda principal to accept a student from Silver Spring? MR. REED:  I guess his job.  The district has--my sense is that they've tried pretty hard to be a good-faith complier.  I think they're operating within some constraints, political constraints, and they've been expanding the range of choice in sort of the down-county Bethesda area. Cold Spring Elementary, which is one of the top elementaries, if you look at test scores, first or second, was the one that received the greatest number of students from Rosemont.  And it's in the heart of Potomac, Maryland, which is one of the most affluent places in the country. So it's--there were closer schools that weren't designated as receiving schools.  That was largely because of capacity.  There are some very crowded schools in Montgomery County.  Garrett Park, I believe, was one that was adjacent to a number of schools in the Wheaton area, but they were not designated and they're just bursting at the seams. So I think they're trying to balance these sorts of things and I think they're making an effort to get principals to buy into it, but I think it's simply taking time. MR. HESS:  Checker? MR. FINN:  I'm curious whether any of you have found in any of these districts any evidence of third-party help to parents to get information or to change schools.  Community groups, activist groups, civil rights groups, church groups, the local newspaper--is anybody attempting besides the district itself to inform and facilitate the movement of kids from one school to another, or to get supplemental services for that matter? MR. CASSERLY:  Yes, we saw a fair amount of evidence, at least in the cities, where community groups are starting to be pretty active-- Boys and Girls Clubs, local BAYO and other organizations that are starting to do community newsletters, ads, flyers, phone calls, community forums and the like.  It's still pretty small-scale stuff, but in most of the major cities you're starting to see that emerging, and I suspect over time it will have an effect on choice participation rates. MR. HESS:  Anybody else  ?    . MR. BETTS:  In San Diego, I think the community-based organizations that Anne set up interviews with are keen to get involved in that, and one of their suggestions actually was to have funding to train them in the subtleties of the law. The biggest thing that I've seen recently is a Latino group, whose name I've forgotten, had a weekend fair to explain to the Latino community how they could take advantage of supplemental services and choice.  So there are--we're starting to see signs of third-party information providers. MR. HESS:  And is Worcester teaming with these, William? MR. HOWELL:  No.  There is a parent watchdog organization in Worcester, but thus far it hasn't actively participated in sort of a public information campaign about opportunities under NCLB.  But that's about it.  The teacher's union is quite strong in Worcester as well. MR. HESS:  Right here. MS. BROWN:  Hi.  Cindy Brown [ph], and I have a question mainly for Mike.  Nobody has mentioned the supplemental services provision in the law says that priority has to be given to low-income kids, whereas the transfer provision, you know, is for any kid in the school, as are services generally under Title I.  They're for disadvantaged once you're in the school, whether you're free and reduced-price lunch or not. So I wonder--obviously, with the low take-up on supplemental services, maybe this hasn't been an issue yet.  But do you see that eventually higher-income kids who may be low-performing in a school in need of improvement might be denied services, while a low-income kid gets it, or is this just a hypothetical issue that will never have much effect? MR. CASSERLY:  No, it's not hypothetical.  I don't think that what we're seeing in the cities runs parallel to the Montgomery County experience in this. I didn't mention this in the presentation, but we did ask in our survey for very specific information about how it is the school districts prioritize low-income, low-achieving kids for supplemental services.  And even the districts that do not have extensive supplemental service eligibility have already set up procedures by which they can rank-order students by free and reduced-lunch status, by school, and use either the state's performance scores--that is, the test scores from the state--or a combination of the state test scores, the local test scores, and/or grades to do this.  So most of the districts, at least in the cities, have a process set up already--it's very formal, often very kind of formula-driven--by which that prioritization is made. MR. HESS:  Okay, we have time for one more question.  Right here. MR. DUNBAR:  Chris Dunbar [ph], Michigan State University.  My question is for Mike because our findings in the paper that we wrote mirrored those that William had with respect to not much support for the school district. My question is might there be a difference in perception or endorsement from those who sit on your council and those actually charged with the implementation of zero tolerance on No Child Left Behind on the local level, because you see to think--or my understanding is that there is support from the members of your council and our findings suggest that there isn't maybe as much support. MR. CASSERLY:  Well, you're welcome to come to any of our meetings and ask that question.  I think you will generally find support.  There's also a fair amount of frustration in the field with the implementation of one provision or another.  But it's hard to ever have a conversation about anything in the cities and not hear grousing about something.  It's just kind of the nature of the beast, but don't mistake that for overall support for the legislation. By the way, we did not get a survey return from the Detroit Public Schools and everything that I know from having worked in Detroit over the last couple of years on their instructional program tells me that they are way behind, as is the entire state of Michigan, in the implementation of NCLB. MR. HESS:  Actually, we'll take one more. MS. WATTENBERG:  Ruth Wattenberg [ph], the American Federation of Teachers.  Let me start by saying I managed to miss all of the morning session.  So if I ask a question that's already been asked, please rule me out of order. But I want to just--getting beyond the general sense that comes through all these papers that the states and the districts are really just gearing up and they're not quite where they should be on the choice and the supplemental service provisions, I have two questions. The first is whether you can yet discern any patterns among parents that, given the choice, if they knew what was available and everything was up and running, would the parents of these kids in the low-performing schools rather send their kids somewhere else or would they rather get supplemental services at the school.  I wonder if that's clear, if people talked to enough parents to get a sense of that. The second question goes a little bit beyond the papers, but I thought maybe people might have picked this up, which is, again, if the districts and the states are a little slow on the uptake on the supplemental services and the choice, do you get the sense that there is nonetheless a big and perhaps productive scramble to meet AYP through some other means; that even though they're not doing these two things particularly fast that there's a lot of stuff going on in some other areas that, you know, is also useful? MS. DANNENBERG:  I'd like to answer the first question at least in part.  I think in San Diego it depends to a large degree on what grade level you're talking about in terms of parents. One of the things that we heard was that parents of younger children--and we saw that on the graphic that showed that the action is really happening for choice in the middle and high schools.  Parents of younger children really don't want their kids on buses going two hours to a different school, regardless, for a variety of reasons. And in terms of supplemental services, one of the things we heard which was a great suggestion is that if--somebody else here asked about six-to-six programs and I don't know if those are available nationwide, but in California they're essentially babysitting--but that if supplemental services could be piggybacked onto six-to-six programs that a lot of parents would take advantage of that. MR. CASSERLY:  Ruth, the answer to your second question, I think, is actually pretty easy in the cities.  And I think one of the things that people are trying to do on the AYP stuff is essentially raise student achievement.  I think that's the long-term strategy by which people are trying to address it. It's hard for me to tell from the work that we've done what parent motivation is relative to--choice relative to supplemental services.  It's very clear, however, that people participate in supplemental services in the city schools at a rate that's about ten times higher than the rate at which they participate in the choice programs.  And I think there's a whole range of reasons or possible reasons for that and it's probably not explained by any one factor. MR. BETTS:  If I could just add to that, I think the point you just made is exactly right.  The uptake rates are about one-tenth as high on choice as on supplement in San Diego.  But the primary reason for that is that by the design of the legislation and the problems with getting the test scores calculated quickly enough, parents typically are being asked to move their kids part way through the school year and that's really going to hold back the NCLB choice. When you look at the aggregate participation in all the existing choice programs in San Diego, one out of four students is in a non-local school.  So that tells me there's a very deep and strong interest in school choice in the district. MR. REED:  I want to add one thing about efforts in   ?   AYP.  Of the ten in Montgomery County, depending what happens in April of this year, four of those ten could be off the school choice list next year.  So they've made--they've already got the first year under their belt, so they may get the second year. MR. HESS:  William? MR. HOWELL:  I'd just make one pitch for the second paper I wrote for this was designed to try to get at some of the questions about levels of parental interest in alternative schooling options and how committed that interest is.  And the bottom line there is that there is actually a pretty high level of interest among public school parents in Massachusetts generally in pursuing alternative schooling options alternative to local neighborhood schools, and that that interest is concentrated among students who are attending under-performing schools. MR. HESS:  Okay, thank you.  I'd like to thank all the panelists. [Applause.] MR. HESS:  The way this will work is we're going to take a short break.  We'll reconvene with Checker moderating at about a quarter after and we'll hear from the discussants. [Recess.] MR. FINN:  Two minutes until blast-off.  Please get your coffee and get ready to resume. It's time to get going again.  Get your coffee and sit down, please.  Get your coffee and sit down, please.  Get your coffee and sit down, please.  We're going to start.  Thank you.  Places, coffee, attention. Gail? MR.           :  Thanks.  Thanks, everybody.  The trains have been running pretty much on time and if we keep them that way, again, we'll have time for some of your participation for about half an hour in this session. And then as Rick warned you last night, we're going to have a sort of a bring-your-lunch-to-your-place and take part in a closing discussion of conclusions, implications, and recommendations, open floor for that. In this session, as happened yesterday, we had essentially researchers presenting what they had found and then people from what I will loosely call the real world commenting on those findings.  We're following the same sequence today. The researchers presented what they found and four residents of the real world are now going to comment on their own experience and perspective, both on the papers and on the reality of the choice provisions of No Child Left Behind as these have unfolded. Again, there are bios in your packet and I'm not going to waste your time, other than to note very briefly that we're going to go in alphabetical order.  Keisha is actually involved with one of these third-party efforts that I was asking the last panel about; to note that John actually helps run the second largest school district in America; to note that my fellow Daytonian, Gail Littlejohn, chairs the school board in a small urban community that is affected by No Child Left Behind, and that Joe, who is a journalist who has observed both Milwaukee and now New York, is a close education observer of the nation's largest school system as we go forward. So with no further introductions and with a reminder that we have 15 minutes and we're going to enforce it, I'd like to ask Keisha to kick off, and push your button. MS. HEGAMIN:  Okay, thank you so much.  Good morning, everyone.  Again, my name is Keisha Hegamin and I am President of the Philadelphia Chapter of the Black Alliance for Educational Options, also known as BAYO. Our involvement with No Child Left Behind has to do with a federal grant that our organization received to inform parents of their rights under this law, which is something that I think is extremely important, given the depth of the information and the difficulty that even we have in understanding it and implementing it.  And parents are the ones who need that information more than anyone else. So my perspective is really going to be one from an advocate for parents and someone who has heard the issues, the challenges that parents face in their children's schools in dealing with No Child Left Behind.  So I know this is a policy discussion, but I just want to really focus on parents' needs and what this law looks like to parents in Philadelphia as it relates to some of the situations that were described in the papers that were read. Just for some background, Philadelphia has about 261 schools in our district.  We have all schools as Title I schools.  Every school in our district is a Title I school.  194 out of those schools this year are in need of improvement in various stages.  So that is just about 75 percent of our schools in our city are in need of improvement.  This is well over 100,000 students.  So I do acknowledge that it's daunting task for our school districts to properly implement this and make sure that every parent is informed. There end up being only about ten accepting schools in Philadelphia this year.  None of them are high schools, and spaces are very, very limited in those schools, sometimes maybe 20 spaces, sometimes a little more, a little less.  So this is what it looks like for parents there. My general perceptions of the law and its implementation and what I saw in the readings is that all of our cities seem to be having similar difficulties.  But I think that this law is very important in that it does provide choice on a large-scale basis that parents deserve and that parents need and that parents have not had in most situations. I also think it's important because it allows us to make--to allow parents to make informed decisions, and that doesn't mean just automatically go to another school because you have that choice.  It means really study and know, have the resources to help you know what the differences are in these schools and whether or not you should transfer. And I think that it's important, as one of the papers indicated, that this issue is coming to the forefront and people are talking about failing schools, test scores, and this sort of thing for the first time on a large-scale basis. My impression of the readings was that there seemed to be again the same issues that are facing a lot of our districts with regard to implementation.  There is a lot of times the reluctance of the school district to disseminate information to parents and really to comply with the law.  We see that on a district level in Philadelphia, as well as on a school level in Philadelphia, which I'll talk about. The information is not consistent; it's not consistent within the district, it's not consistent within the schools.  And for parents, that is extremely confusing to try to go to your school and find the information and be told I don't know what you're talking about, or you just have to wait a little longer, we don't have the paperwork, or go somewhere else.  And I think that that probably is consistent with other districts. The timing issue seems to be an issue that is a problem in all of the districts that were examined in the papers, and we have that same timing issue as regards to determining AYP, which schools are on a list, but also in terms of giving parents enough time to make the decisions that they need to make regarding school choice or supplemental services. One of the papers used the term "reluctant compliance" and I think that that's what we have seen in Philadelphia and what I know a lot of districts are seeing.  Reluctant compliance is what parents are getting when they go to their schools and to their districts to try and receive these services under No Child Left Behind. I think that there is a major issue, as was pointed out in a number of the papers, about the delivery of information.  How do you deliver this information in Philadelphia to 100,000 parents in our district? The delivery systems that are being used are not catching parents' eye, and I'll talk a little bit about what it is that--what the audience is that we are trying to get this information to.  It's not educators, and some of the language that's being used is very educationally-oriented, very high-level, and it's a turn-off for parents. These letters are very long.  These letters do not get to the point.  There's a lot of discussion in the beginning of the letters of how great the district is and all the things that the district is doing.  And that's what we've seen in Philadelphia and we've been a part of trying to revise some of that language and make suggestions to the district for how they can better reach out to parents. But I think that this is an indicator of a greater underlying problem of communication in our schools.  School districts are not being effective at communicating with parents and that's something that we as an organization try to help with, which I'll talk about. One of the things that came out in pretty much all of the papers was this question of why don't parents participate.  And although it's a bittersweet insight, it was good to see that we're not the only district that is saying parents don't want choice because nobody is using this law.  And I see that that is happening in other states.  So, you know, that is something that I think is extremely incorrect to assume. Parents do want choice, and we see that in Philadelphia and everywhere else by the large number of parents wanting to get their kids in charter schools.  We get requests all the time for scholarships for private schools.  Parents take advantage of our district's existing choice program, although it's very small.  Parents do want choice and I think it's insulting to parents to assume that a parent in an under-achieving school does not want to get out. So I really would like to see that attitude changed because it's just not true.  There are so many other reasons why parents are not taking advantage of this on a large-scale basis.  And I'll talk about a couple of those as I saw them in the papers and also in Philadelphia. First of all, as I said, this process and the information is confusing and frustrating.  When you look at who our audience is here, these are not the most sophisticated of parents.  These are not the highest educated, because the highest educated people are not often the lowest-income people. So you have parents who are already in a situation where they have a disadvantage as far as receiving the information, being able to process it.  Parents are very busy.  Parents come home.  They get this letter, they open it up, and the language is ridiculous. In Philadelphia, the letter--the font was tiny.  They must have used a 9- or a 10-point font.  It's single-spaced.  There's nothing that jumps out at you.  And, again, in the beginning it talks about how great the school district is and all the things that the school district is implementing.  So as a parent, you came home after a long day.  Maybe you're a single parent; maybe you've got to go ahead and fix the dinner.  You've got to go over the homework.  You've got to get your kids ready for school the next day. You get this letter and it just looks like the same thing that you probably get all the time from the district, another letter.  If it doesn't say your child needs to do this or if nothing jumps out at you, you're going to put it to the side, like I do with my mail, put it to the side and look at it maybe on the weekend when there's time.  And then all of a sudden, the three, four weeks that you have to make a decision are over, okay, and you haven't gotten a chance to really exercise this choice. In advertising and anything else, you know that--and marketing, you know that you have to look at your audience and you have to say what's going to jump out at your audience.  That goes for any type of advertising.  You don't hide something that you want people to see.  You don't hide it on the second page, you don't hide it amongst a whole bunch of other language. If you really want somebody to take advantage of the benefit you're offering, you put it in bold print and you put it up front.  You make it very clear.  You send out a flyer.  How many flyers do you get in the mail everyday, in color and what not, easy to understand?  This is a way of hiding this information, and parents know that; they know that, they see that.  So as a parent, you get this, you put it aside.  All of a sudden, the opportunity is over. On the other hand, you've got parents who get it.  They read it.  Maybe they have a great school that's giving a lot of information to parents.  And so they go to the school, they look at the option, and they decide this isn't for them, especially when it comes to school choice. We have to be comfortable with that and we have to allow parents that ability because part of choice is knowing that you can or cannot make the choice, depending on the information that you find.  There are a lot of reasons for parents maybe to not exercise this choice and that's their prerogative.  School choice is not going to be good for every child and every parent, and every parent has to make that own independent decision. There's the distance issue that was mentioned in many of the readings.  In Philadelphia, all of the schools, of course, that are poor schools are in one section of the city.  If you almost imagine like a diagonal shape, the poor schools are in the southwest section and the north section, and the schools that kids can transfer to are way out here in the northeast, sometimes 45 minutes away.  Some parents don't want that for their children. The small number of choices.  We just did an advertisement in a local newspaper, a big old ad.  It lists all of the nearly 200 schools and then the 10 little schools that you could transfer your children to.  Now, if you're a parent and you see that, you're going to think it's a joke.  I mean, you're going to say, wait a minute, is this really choice?  If you have a child in high school, there's no choices for you. And then it also says, well, you can apply to a charter school, another effect that was mentioned in the papers.  So now parents think that they're entitled to entrance into a charter school because the districts, although maybe they have not been supportive of charter schools in the past, all of a sudden charter schools are their best friend. So go to a charter school, and parents go to the charter school and the charter school says I can't admit you just because of No Child Left Behind; I have the charter school law to comply with and a lottery system.  So that's misleading to parents. Parents go to a receiving school and say I want to check out your school, I want to see what it's like, whether I should send my child here or not.  And the receiving schools say, I'm sorry, I'm not doing that, I'm not having it.  And it's just very confusing and very frustrating to parents, so I think we need to acknowledge that as much. And then there's the parents who just never get the information.  We have so many parents that respond to our outreach efforts that just never got the information.  Or maybe they got it and it's in their pile of things to read later, like my pile at home.  So that's an issue, too. A lot of parents go to the schools and again are told, I don't know what you're talking about, by the principals sometimes; I don't know what you're talking about.  And sometimes the principals really don't know what they're talking about because the information hasn't been given to each school. I think that in closing, especially for Philadelphia I think that there has been some improvement between the first year and the second year.  The district has given out a much--well, a bit of a better letter this year, and it's actually going to do information sessions for parents that are scheduled.  They're actually giving full information on the schools that kids can transfer to so that parents can make an informed decision. And they're giving them a full month this time, thankfully, which still I don't think is enough time, but it's a little bit more.  So I think that there has been some improvement, and I think one of the suggestions that came out of the papers was for community organizations like us to work with the school districts to help get this information out. There is no way that the district can get all this information out effectively to all of the parents.  So we need organizations like us and others to do the advertising maybe that the district won't do.  We've done door-to-door.  We've gone to supermarkets and stood there with a big sign that says "free tutoring."  And even then sometimes parents have a hard time trusting it because they're not used to having the choice, having the option. And they don't trust the district.  They're used to being rejected in one way or another from the schools, and so it's something new for them.  So I think that that's going to take time.  But, again, I think that there has been improvement and I'm looking forward to being part of making this implementation much smoother in Philadelphia. MR.          :  Thank you, Keisha.  That was terrific.  Just to repeat two numbers if I think I heard them right at the beginning, 100,000 eligible kids in Philadelphia? MS. HEGAMIN:  Yes, a little more than that. MR.          :  And 200 places in receiving schools? MS. HEGAMIN:  Probably, that's--well, yes, that's probably about right, yes. MR.          :  Okay. MS. HEGAMIN:  They haven't given out the official numbers this year, but it's apparently smaller than last year.  Last year, there were 1,200 spaces, but there were also 20 schools.  This year, there are only ten schools and they're going to have even more limited spaces because students have transferred into them from last year as well. MR.          :  Okay, thank you very much; interesting and concise. And we will now cross the country once again and go to Los Angeles. John? MR. LIECHTY:  Thank you.  Good morning.  First of all, let me just say it's a pleasure to be here.  Hopefully, I can share some insights. Let me just preface my comments by saying that No Child Left Behind is hard work in its implementation.  Just to give you an indication of that, in reality I'm only 25 years old and a year ago I had hair. [Laughter.] MR. LIECHTY:  I say that to you because obviously I've spent a long time as a public educator in Los Angeles, which is the second largest school district in this nation.  And sometimes my analogy to No Child Left Behind--and primarily the supplemental service piece is my responsibility within the administration of that within Los Angeles. But my analogy is trying to change the tire on a car that's moving about 60 miles an hour.  And so I want to preface my comments this morning by saying to you that I want to commend the authors of the papers.  I cannot applaud more this kind of effort.  When I read through those papers, I think it probably does capture the good, the bad, and the ugly. I think L.A. probably represents a little bit of all that.  I think we've got some good, we've got some bad and, yes, we've got some ugly.  We're still not there yet, but I want to put in context. No Child Left Behind was signed in January of 2002, with an implementation date of July 2002.  It's the most complex, complicated in many instances, piece of educational legislation in this country.  And I think the time line created some issues for almost all school districts, but especially school districts like Los Angeles. I mean, we were to have this implemented before we even knew what the state was doing, and I think you've heard that and I don't want to repeat all those kinds of information that was given to you both yesterday and today because I think the presenters have not only captured where we are in the implementation, but I think have cautioned all of us, I would hope, that we have to stay the course. And let me hit the first issue that I want to touch.  When I look around the nation--and I've called many, many school districts in trying to put this together to find out what they're doing--what I discover, as most reform efforts in education, a lot of it is based on one's attitude and belief system regarding the reform. And I must say to you it's got to start at the top.  If the superintendent and the board of education across this nation, are not, number one, understanding of what the legislation is and, number two, supportive of it, then it's going to be a struggle to make anything happen with it and it's all going to end up being piecemeal. The way the legislation rolled out, some of us were a little bit more informed than others, but the attitude thing has not changed.  In Los Angeles, just to give you a scope of that so you know what I'm talking about, Los Angeles this last year had 103 program improvement schools, supplemental services to be offered to approximately 186,000 eligible young people. Just the sheer numbers in itself--and I want to comment on Keisha--trying to inform and communicate, parents, yes, but schools that have program improvement status, so trying to develop an attitude and a belief system.  And we've approached this, all the issues that are out there that have been described, whether it's communication and possible timelines, conflict in policies, running into the busing program where you could impact a desegregation court order.  All of those things we've tackled with the glass half full as an opportunity for young people who are in need of additional academic assistance.  They should have that. But changing attitudes and getting people to believe that this is the right thing to do in the general public, let alone within a system, is not easy.  And I say that to you because as I look now, we're about 18 months into this.  We've just touched the tip of the iceberg. Let me hit some things for you.  Communication: Just communicating with schools in the various accountability systems, from the federal government to the state to our own local district, sometimes creates confusion.  If it doesn't create confusion, it creates divisiveness. People do not want to hear--and I'm talking to you as an educator now--low-performing schools, they don't want to hear one more thing that they're not doing a good job.  They know it, they live it, they breathe it.  It's not intentional thing, but it's a terrible thing for schools day after day to go and all they get is they're under-performing, they're not doing an adequate job. And so there's a tendency within large school districts--and I can only speak to the large ones--to want to avoid that bad news.  And so therefore when you're trying to communicate the kinds of things they want to do, when you add to that that there may be a disincentive--and let me hit on that, a disincentive for school districts to fully implement this program. There was a descriptor yesterday, and it came up again this morning, of school districts kind of being the bully because we have a captive audience.  I would suggest to many of the providers that work with the Los Angeles Unified School District we probably are the bully.  There are some disincentives for this school district, our school district to not implement this.  Let me give them to you, one of them. Our 20-percent set-aside, Title I, Part A, is in excess of $60 million for choice and supplemental services.  So you get a picture of a big school district.  Our district works this way.  It's 5 percent for choice, 5 percent for supplemental services, and the remaining 10 percent is either for choice and/or supplemental services. We already have a large choice program, as you've heard in some of these other districts like San Diego.  Therefore, the number of parents who are opting out for choice for No Child Left Behind are not the same kinds of numbers that are going to our magnets, our PWT program, which is Permit with Transportation; it's part of our integration program. The largest share of the dollars for this that we put aside is supplemental services.  The reason I'm saying to you the set-aside--why is that an impact on a school district?  Because those dollars cannot be touched until we determine what the demand is for choice and supplemental services. Well, on the one side, internally school districts will have a tendency to want you to push fast on the demand because once you determine the demand and if you have not used up the full allotment of the 20 percent, those dollars can then go to other Title I schools and Title I youngsters within the system. So if you start with July 1st of $60 million and you're not going to determine what the demand is until February or March, you may not be able to spend the money by the time you need to spend the money, and therefore it's not being used for kids who actually need it. When I say that to you, that's why the leadership and the attitude becomes important, because otherwise there's systems within a system that works against the full implementation of this law.  They want you to quickly determine what the demand is without giving parents adequate notice and adequate time to make a decision. Why?  Not because they're bad people or because they don't want the good thing for kids.  Because as soon as that demand is determined, we can determine through the budget process what the overall impact on this reserve is.  I just think as we move down the road with the federal government and everybody, the state, we need to recognize that.  I'm going to give you the example. Last year, because of the lateness of information and the lateness of getting started, most of the states--most of the districts had to seek waivers in order to carry over Title I dollars.  From a district point of view, those dollars are provided for the kids that are in the system this year, not next year. So when you're not using your resources wisely this year for those kids, to carry it over it means some kids who are going to leave the system, et cetera, will not have access to it.  I give you that just as a snapshot that there are other players within the system and outside the system that need to understand No Child Left Behind, not just the people responsible for implementation. So this whole issue of communication inside of a system, not only at the district level, highest levels, board of education level, but right on down to the school administration site so that, in fact, the schools understand what No Child Left Behind is basically supplemental services, and especially choice. Number two, our providers that are selected by the state.  We've taken kind of a different approach in L.A.  As San Diego pointed out to you, there's 103 state providers.  Los Angeles Unified is also a provider.  So there's 102 other providers besides the branch that I am also responsible for, a branch called Beyond the Bell. What we attempted to do was to reach out to the 102 providers that were selected at that time.  But, understand, our state is approving providers every month, so it's not a static list.  Yet, if you've already started your program, ladies and gentlemen, you can't have providers coming up after the parents have already made the choice. And so there becomes a rub that a provider will say to me, and rightfully so, well, John, you're keeping us out.  I'm not keeping you out.  We just want our program.  Next year, if you're part of the list and you want to work with L.A., right on.  Again, communication as to how not only districts are implementing it, but how states are implementing it. The second part of that supplemental service piece which is important is that our district, even though we have a number of outside providers, 25 of them who work with young people, whose parents selected them--and almost a hundred percent, they do it at their own facilities or at home or across the Internet.  We make our schools available to them, but only if they lease them.  It's a rub. The question came up, I think, from Edison, our good colleagues from Edison, about the use of school facilities.  We did not put this together without plenty of legal.  I'm the most popular guy on the block in Los Angeles, and I'm saying that to you very openly and very honestly, because I have the money. And so everybody in the world has a program and they want to use the schools and they want to get in to captive the parents and they want to do these kinds of things.  We do not let providers use our schools unless they're willing to lease them, and there are some legal reasons for that because if we let one provider do it, then don't we have to open it up to all providers?  So part of the issue that we have in implementing this is obviously the legal aspects of it. All of our providers, we use a contract.  Our providers--knock on wood--are just outstanding folks.  I think that was captured.  We're very similar to San Diego.  The district has the responsibility, the way the legislation is written, to not only inform parents, not only to provide them with a list of state-approved providers, but we have the obligation to work with parents if they need help in choosing a provider. That's a conflict right away because if I do anything that guides something to--it can be viewed as, well, you're guiding them away from us and you're guiding them to you.  And let me get that straight.  Many school districts are providers.  I'm not going to sit here and tell you we don't have an advantage.  We do.  Why?  We have the schools, we have the kids.  Teachers, parents know the system and sometimes they're more comfortable.  I didn't say we're better; more comfortable. And so there is an advantage, and so we are viewed as, and rightly so, maybe a bully or a conflict of interest.  I think as we move down the road with this legislation both at the state level and obviously at our district level, we'll work through some of these bumps, especially some of them that I've heard the last day, because there are some real issues out there that we need to come to resolve. But let me conclude my comments by really summarizing the papers.  I'm perennially an optimist.  I think we're a year into this and my belief system is that this is not only the right thing to do.  It's targeted for the right kids in our system, the kids of poverty.  This is doable. We need to take what we have here with No Child Left Behind and we need to integrate it into our whole system in the Los Angeles Unified School District, not just for the program improvement kids.  But where we're running up against it right now--we have tons of parents who want help for their children, but are not in a program improvement school. And here's the ironic thing with parents: We even have parents who are coming to us now to ask to move their youngsters to program improvement schools.  Now, folks, that's why we need to be able in our district to be able to take our other programs, kind of fold them in and integrate this so that, in fact, any youngster in any school who needs additional support or help academically has an access to receive that at their home school. Obviously, we're not there yet.  My hope is that we'll be there somewhere in the next five years.  But No Child Left Behind for us in Los Angeles has given us a focus that fits into the other reforms that have been taking place.  It gives us an impetus.  It is on people's plate.  They may not know everything about it, but it certainly has captured their interest. And I would again only commend the authors of the four papers that I read in great detail because I think they capture everything from I don't want to hear about this, I don't want to do it, this is the worst thing that I've heard of, to, you know what, this is the right thing to do and we're going to make it work. I would only recommend the following, that we figure out a communications system that's more inclusive, that we get out of our turfdom, public schools versus charter schools, versus outside providers, et cetera, to work collectively together to bring services to young people. And, number three, we work with the federal government to align the accountability systems so it's user-friendly, it's understandable not just for parents, but for us--your AYP, your API, your UPI, your IPU, your NEPU [ph].  I mean, we got more acronyms than Carter's got pills. I sent out a brochure, ladies and gentlemen, that's 27 pages in length for compliance issues.  So it's easy to comply with the law.  It's much more difficult to implement it. Thank you. MR.          :  Thank you, John.  I'd appreciate two numbers from you, if you could, also.  Approximately how many kids in LAUSD are currently using supplemental services in the NCLB sense of the term?  And of those, approximately how many are getting their services from the district? MR. LIECHTY:  We have approximately to date, as I sit here, about 20,000.  Of that 20,000, about 12,000 are from the district, 8,000 are from outsider providers. MR.          :  Okay, thank you very much.  That was terrific and very interesting. And now to the heartland and Gail Littlejohn. MS. LITTLEJOHN:  Hi, good morning.  How are all of you? Checker didn't tell you that he asked us to be succinct and provocative, I think was the word in the letter. MR. FINN:  But of course. MS. LITTLEJOHN:  So I'm going to be a little provocative, but first I want to tell you about Dayton.  I've lived there for over 30 years and there are some interesting things about Dayton, Ohio. The population is about 165,000.  Close to 80 percent of the people who live in the city don't have children attending public schools, which is interesting for me.  We have about a little less than 19,000 students.  We've lost almost 6,000 students in the last 5 years to the charter school movement. And if you talk to most people in the city of Dayton, they would tell you we have a disproportionate share of charter schools.  With the exception of about two, all of them are under-performing as well.  So we've got a majority of our schools, our public schools, that are under-performing and our charter schools that are under-performing.  So we didn't have a lot of opportunities for receiving schools. And our superintendent approached, I think, just about every one of the surrounding suburban school districts, which are predominantly white, and asked them if they would take our students and the answer was no.  So we have not been able to find other options for our students, and so we have turned a hundred percent of our efforts inward to move as quickly as we can to move each one of our schools up to acceptable academic performance.  So that gives you a pretty good profile of Dayton, Ohio. Despite what might appear to be our dire straits, there is a tremendous amount of hope that is growing in that community.  Last year, we had a school building levy on the agenda, on the ballot for the November election.  And we weren't sure it was going to pass, but we got a 64-percent passage rate to build 34 new schools, which is a complete overhaul of our facility structure in the community of Dayton.  And I think it's in part because they see, at least initially, a pretty good level of aggression in trying to tackle some of these academic reform issues. That's Dayton, but I want to respond to the four papers and what I see as early indicators.  It was actually interesting reading for me.  We are, I think, still members of the Council of Great City Schools.  Our student body has dropped somewhat, but we're still members; we're still active members.  We joined with a lot of energy in supporting all the objectives of the No Child Left Behind legislation.  So our support of that is unwavering. As the legislation and the regulations now stand, though, there's a lot of work and a lot of challenges, I think, that lie ahead for all of us.  First are the bureaucratic considerations.  On Capitol Hill, they were certainly expected, and one of the things that concerns me the most--I think all of us who have learned a little about reform, we know that the continuity of leadership is very important. Whether it's keeping your superintendent on board for more than two years or keeping your school board members on board for two years, having a program that has some continuity and some consistency is really important.  So the first thing I worry about is, you know, the change of folks at the top who will scrap the major parts of this legislation and decide they want something else, because I don't think we can afford to do that. And I think our energy has to be about tweaking this legislation and making it work rather than having the energy to throw it out with the bath water and starting all over again in two years or four years, or whatever it might be. I'm not sure, though, that anyone really anticipated the politics on the local level, the politics that are at play.  There's nothing that tells me that they were sufficiently planned for in any stretch.  Reluctant compliance, I think, is the best descriptor of what's going on in some of these communities, and I find that very disturbing. Technical violations such as allowing parents only one choice option--I don't know if that's deliberate or unconscious, but they call it a technical violation, but it's nonetheless a violation.  And I don't see any evidence that there was a reason why parents were only offered the once choice. Allowing some schools within the district, sometimes the highest-performing districts, to opt out, I don't think was ever intended.  And allowing some schools to do that either because they don't want poor students or they don't want students of color, I think, is intolerable. That leads us to what I consider major flaws in the process of implementing this legislation, as well as serious loopholes.  The testing process does not adjust for inherent mobility of poor students.  We've got a large percentage of--I think about 80, 85 percent of our students in Dayton are on--we call it free and reduced lunch program.  High rates of mobility; it's not unusual for some of our students to move two or three times within the same school year.  But that is not adjusted for and I think it has an unfair impact on urban districts.  Communities also use this as an excuse not to participate because they rationalize it will further increase the mobility rate, and I think there's certainly flaws in that logic. I also believe that states have been given so much autonomy in some cases that the stated objective for increasing academic performance will not be met.  They can define the subgroups that are exempted from yearly progress.  They can establish the difficulty of the test.  They can establish what is the appropriate level of proficiency and the rules for test exclusion. The most drastic difference was in the levels of proficiency for the high school literacy in California versus Nevada.  For California, it's 11 percent, and for Nevada it's 91 percent.  I don't know how we raise the standards of academic performance when we have that kind of disparity from one state to the other.  That kind of logic--it just defies logic for me that we can have those kind of disparate goals and the objectives of this legislation will still be met. And then the parties charged with doing most of the communication regarding this legislation, it's choice and intervention provisions, to me is a classic example of the fox guarding the hen house for some districts. Two years later, the parents of children most needy of these services know the least about its objectives and they know the least about its services.  The low percentage of Hispanic and African American families that participate to date, I think, should be closely examined. Choice is not present if customers are not even aware that they have a choice.  It becomes diminished when local politics manipulate the options to the advantage of some customers and to the disadvantage of others. If these four districts represented in these papers are the norm, you know, we have approximately 2 percent of students who are in need of these services participating at the end of two years.  And at that rate, I don't know how we get to where we're supposed to go.  The accelerated participation is really no more encouraging; it's slow acceleration. There's one positive thing that came out of O.J. Simpson trial.  A lot of marketers are using a concept called the O.J. effect, and the term is used to describe how different ethnic groups can see and hear the same information and draw very different conclusions.  And I think that is very telling.  And they draw those conclusions based on their different life experiences. And so, you know, my question is are the customers that this legislation was intended for--based on the messages that we're sending, are they really understanding the benefits?  Are they really understanding that there are choice options? Are there misunderstandings and suspicions based on suspicions or distrust of promises, because if you look at the cultures for Hispanics and African Americans, they have oftentimes been lied to; things have been misrepresented.  And so is there a trust issue here that's keep them from exercising their choice options? Or are there too many communities who have been very crafty or are engaging in schemes to limit what potential customers know?  Some of the things that I've read, the kinds of letters that are sent, the kind of communication, how little time parents are given to respond, the number of hoops they have to go through to even get access to these services, tells me that something is not working the way it was intended. Another flaw in the regulations in the lack of protection for the receiving schools.  I don't understand what the incentives are to accept low-performing students, other than a few more dollars, which, if you're in states like Ohio, you ask to receive the dollars the year after you take those students in. [TAPE CHANGE.] It may be the districts having to expand student ratios in the classroom, putting trailers on playgrounds or in parking lots, scrambling to hire foreign language teachers, speech therapists, and other people with special skills that are in short supply, not just in Ohio but all across the country. And so what happens is that I may risk--if I'm a principal, I may risk actually losing my wonderful proficiency rating because I have, in fact, been a good corporate citizen or a good public school citizen and taken in low-performing students. So I contend that there are several missing pieces to the No Child Left Behind regulations.  Some are major loopholes that, if allowed to persist, I think can potentially derail this wonderful piece of legislation. Number one, I don't see a sense of urgency.  I don't see it on the federal level and I certainly don't see it on the state level and I don't see it on the local level.  What I see, as a parent, grandparent, and also president of the school board and very active in my community, is I see students lost everyday and falling through the cracks.  I see that everyday that we're losing generation after generation.  So I want to see a higher sense of urgency. Number two, I think that the implementation of this legislation, these regulations, need to be aggressively audited by an independent body.  I don't think that at least these early indicators tell us that we can allow the fox to guard the hen house.  And so some kind of an auditing process, I think, needs to be put in place. I would go so far as to say we need hotlines for complaints.  If parents don't feel like they're being informed, if they feel like they've been given too many loops to go through, then let's put some hotlines in place.  Penalties for non-compliance need to have some teeth, and teeth that bite, when local school boards aren't doing what they're supposed to be doing. The number of receiving schools must be expanded, especially for communities like Dayton.  Our district, for example, according to state law, we're required to bus charter school students, and yet those charter schools do not have to be part of the receiving school mix. And then what about doing something really revolutionary and redefining school districts, borders, so that we actually increase the number of options for students who need receiving schools? I also challenge higher education and the effectiveness of both their teacher prep programs and their administrative programs.  I can tell you that we've got four universities around the Dayton area who supply our teachers and our administrators, and we get too many who lack the skills, they lack the commitment, or lack the sense of the needs of our students.  And so there's a real difference in terms of cultural appreciation. I would also call for an independent dialogue with the intended customers, parents and students.  The issue behind this lack of engagement, I think, needs to be identified.  Have they not heard the message?  Was the message about No Child Left Behind choices and services not clear to them? And then we must also find out what the authors missed in terms of putting this legislation in place?  Have they designed a program well-intentioned that does not meet some of the important needs of the African American community?  Is this sense of neighborhood so important to them that they don't want to move?  Is having all of their children in the same school--is that so important that they don't exercise their choice? Checker Finn has done several surveys with charter schools in the Dayton community and interestingly enough it's hard to read the surveys and see that the academic performance of the schools is the number one criteria.  They talk about the safety of their children, they talk about how they're treated when they go into the schools.  So are there other things more important for the parents that we didn't appreciate and understand when we put this legislation in place? Does the O.J. effect lend too much suspicion to this whole legislation, and does it come across as too good to be true, because if you think that you can choose any other high-performing school to send your child to and you've got access to all these services, intervention services you've never had access to before, it could come across as being too good to be true. And then to what extent was the customer consulted?  I don't know how this legislation was put together, but did anybody go out and ask some of the parents of poor Hispanic and African American students what they wanted to see in this legislation? And then the final missing piece may be the most important, and I would contend that this legislation does have the ability to improve education to students, but that we need to also look at other issues.  And I think one of the real issues here is one of poverty.  The state of being in poverty is not a good thing.  It limits your options, it limits your choices, and sometimes it also limits your decisions. We know that children in poverty often start two to three years behind other students.  They miss more days of school because of sickness, they change schools more often because of mobility, and they have more behavior problems.  And in short, they don't necessarily make the best students that everybody wants to go out and recruit. But I also see underlying some of this attitudes about race that are never healthy, but they're always on the fringes of social policy.  You see it in the pairing of schools, who gets informed, who gets the first choice at schools. But on a more in-depth level in a community, you know, these problems won't be resolved until we start dealing with things like red-lining and where people can live in communities.  They won't be resolved until we have people who are able to compete effectively for jobs, effectively for education programs.  So I see NCLB as a band-aid on a more long-term, deeper problem in this community. I'm especially concerned by the projections of the U.S. Census Bureau.  By the year 2050, we'll have--the majority population will be comprised of people of color, and if they continue to live in the lower levels of poverty, I don't know that we're going to be able to run fast enough to keep up with the influx of poor students who are coming into our school system unprepared to even start kindergarten. MR.          :  Thanks very much, Gail.  One more narrow factual question about Dayton.  Outside providers of supplemental services in Dayton--are there some and how busy are they? MS. LITTLEJOHN:  They're not busy [INAUDIBLE]. MR.          :  [INAUDIBLE] Williams, heading all the way east to, what is it, zero degrees in New York today? MR. WILLIAMS:  Yes.  I saw on the Today Show there were not many people outside the window this morning. [Laughter.] MR. WILLIAMS:  I also want to thank the authors and all the participants here.  I've found this conference to be--I've found more thoughtful discussion about this topic in the last 24 hours than in the last 2 years in New York City.  So I thank you all for participating. I also read through these papers during the football games last week, like one of the presenters yesterday, and came up with about a dozen pages of notes that I consider brilliant and original observations, although I've found since sitting through all these panels that while they were brilliant, they are not original.  So I'm going to scrap those and just talk a little bit about what it's like as a reporter covering this in the nation's largest school system. I have been very tough on the New York City schools in their implementation of this program, and they believe that I have not been fair.  So I want to establish that at the get-go and tell you, since they're not here, some of the things that they say in their PowerPoint presentations when they describe their role in implementing this, and also throw out a caveat of my own about what you should consider when you hear what I'm saying about New York about the timing of all this. The basic thrust of the PowerPoint presentation that they do when they're getting accused of dropping the ball on this is that they've done a better job than any other city in dealing with a difficult law.  And I think especially after this morning's presentation, I would say that we've done better than Worcester. [Laughter.] MR.          :  You've got more than one student? MR. WILLIAMS:  We do have more than one student, but I have to tell you that reading about San Diego and what I've been able to read about Los Angeles, there just seems to be a much more genuine interest in making it work in those two cities. The letter, I think, that Los Angeles sent to parents about their SES, where they even referred to it as "beyond the bell," which I thought was brilliant--the SES, I agree, is a goofy term to use when you're dealing with parents.  I got the sense in reading the letter that you send to your parents that you really want to make this work.  I think the word "opportunity" was used in the letter, which we haven't seen anything like that in New York City. And in San Diego, to hear that 86 percent of their--I think it was 86 percent of their kids that qualified for SES were signed for it--we're at 19 percent, and I got the sense in San Diego that there was a real effort to reach out to these parents and include them. So I take recommendations that people in Los Angeles and San Diego are making about how this law should be changed or implemented differently--I put a lot more stock in what they're saying than anything that New York City could offer right now because we haven't really lifted a finger at this point. The one caveat that I will give is that when this law was passed in 2002, in New York City we were in the process of giving control of our 1,200 schools to our mayor and it was a long, drawn-out process that involved all sorts of things that had nothing to do with education and involved a pay increase for teachers and a teacher contract.  It all sort of rolled together in a messy form of legislation in Albany. So, you know, by the spring of 2002, summer of 2002, you know, the mayor is just getting control of the schools.  July 1st, actually, he got control of the schools.  He hired his school chancellor about four weeks later.  So we begin the school year in September of 2002 with the state releasing its list of schools in need of improvement. And by the way, whenever I talk about this, I have to say don't ask me why we always call them failing schools in our stories because if I file a story that says schools in need of improvement, I'm going to get fired.  I can't use phrases like that.  So this term "failing schools" has been around, and I know it's not fair and I understand the concerns about that, but that's where it's at. So New York City was starting with this.  In the first round, we have 331 schools of our 1,200 schools that are on this list.  And that's sort of the point that it got on everybody's radar screen.  The list was announced on a Tuesday.  School began on Wednesday and we got a new chancellor.  The mayor is learning his way around. So I think some of the problems that we saw in the first year we could describe as bumps.  But some of the ongoing problems with implementation--to call them bumps, I think, is a disservice to the parents, who are really screwed right now because nobody has really been lifting a finger to help them. So that said, and before you just dismiss me as a cranky journalist, I just want to show you something by show-and-tell here.  And this is something that was in Mike Casserly's paper and something that was discussed here as well, the letters that have been sent to parents.  And we wonder why more parents haven't signed up for things in New York City. This is one of the ones in Mike's paper that is a two-page letter.  This was sent last spring.  I don't know if anybody can see it.  If you see the big white space at the top, that's my writing that's sort of filling in the white space. The word "transfer" doesn't appear until the second page; it's the 14th paragraph.  But if you look closely, in order to get it onto the second page, you had to hit the carriage return about 15 times before you started the letter.  So it was pretty obvious that nobody wanted parents to see the word "transfer." And just like in Philadelphia, the entire preceding 13 paragraphs, aside from legalese and education jargon, is about all the wonderful things that the city is doing to improve the schools, which is great to know.  It's important to know.  I don't know how Joel Klein [ph], as the chancellor, could even know what's happening at all of these schools at this point, but this is what we're dealing with. The nice part about the white space on the top of the letter is it gives you room to write other notes about other shortcomings with letters like this.  This was sent out in March.  Parents had nine days to respond from the day that they got it.  Five of those days were spring break, so that's one problem. But regarding those deadlines, at the end of the letter there's the card that you need to send in, and this is another problem.  You need to apply for a transfer.  This isn't even for the transfer; this is to apply to get more information about a transfer.  They have it in English and Spanish. For the Spanish version of it, the deadline is March 31.  In English, the deadline was March 21.  And in the letter it says--I'm going to just read it for a second--"Responses postmarked after March 21 will not be processed.  If you do not return the form by that deadline, your child will automatically remain in his or her current school." One other thing about this letter.  The return address said Virginia on the letters that--there was a mass mailing sent out by a company from Virginia. The day that I got a hold of this, I called my list of parents that I knew wanted transfers, and I'll explain how I knew they wanted transfers momentarily.  And every single one of them said, yes, I got something in the mail today, but I didn't look at it.  I just--you know, I saw the Virginia thing which didn't make--that's how I found about this.  I got a copy of the letter.  I hadn't seen the actual envelope, and from now on I'm going to ask to see the envelopes that they send out with this stuff. This sort of illustrates the kind of communication that we've seen between the school system and the parents.  And while there's been some improvement in that in the time since last spring, this is sort of what has caused a level of suspicion among parents that the system doesn't have their best needs at heart. We've had parents file lawsuits in New York City because they wanted to get tutoring and wanted to get transfers.  The suit was thrown out because the judge said under the No Child Left Behind law it's not the courts that can enforce this; it's the United States Department of Education.  This has been another problem. Rod Paige has spent a lot of time in New York City and this has always been a question that he has asked by reporters.  And I have to stress here I don't know what Rod Paige says privately to Joel Klein or Mayor Bloomberg, but publicly he has not said anything negative at all about their implementation of this. And it's sort of--you know, from a journalistic sense, the story kind of dies there when you're pointing out all of these problems and the one guy who has the bully pulpit to do something about it, Rod Paige, has been very, very quiet.  It has almost been a concerted effort, it looks like, to play nice with New York City.  I don't know if it's because it's a Republican mayor or what, but it's been an interesting dynamic.  And a lot of parents who were very gung-ho about this idea have become very bitter. Let me explain how I ended up meeting so many of these parents.  On that first week of September of 2002 when the list came out, we ran a list in our paper of all 331 schools, and the next day I probably spent about 12 hours answering phone calls from parents who were just learning for the first time that their child's school was on this list. And usually by the time they called me, they had made a call to their district office and were told that if you--one woman was told that if you really wanted a transfer, you had to have done the paperwork last spring, last month.  This was September when she was learning about this. This woman tried over the course of maybe four or five months, did not end up getting a transfer to her child until a city councilwoman named Eva Moskowitz, who chairs our council education committee, personally went over to the school administration building and said I'm not leaving until we have a transfer for this woman's child.  And I kept in touch with her, and in her case she's thrilled with the transfer that she has had. One of the things that has been surprising--I think maybe this is because I spent so much time in Milwaukee, where you're sort of conditioned to think of choice as a reality--is that from the get-go I assumed that the transfers were going to be the story in New York City.  But it's the tutoring, we have found, when parents find about it--to the question you asked before, when parents find out about the tutoring, they're very enthusiastic about it.  The problem has been getting the information to those parents. In Mike Casserly's paper--and this is part of New York City's PowerPoint, as well, when they talk about what they've done.  They posted first-class mail in ten languages.  They placed automated telephone calls in ten languages, which I give them credit.  I think that has been a very good start, but sending the flyers home in the backpack has been a very big problem. This fall, we had provider fairs for the private SES providers.  The principals were given these flyers to send home with their kids on a Friday and the fair was Friday night and Saturday morning.  I found out from a community-based organization that sent e-mails out to people.  I went to one of these fairs on Saturday morning and I was the only person that showed up for it. The supplemental providers that were there were extremely upset with the implementation.  And the one good thing, I guess, from their perspective that came from this is that the Department of Education was so embarrassed--the city Department of Education was so embarrassed by that that they sort of backed off on trying to steer kids to their own program. I think our first year, the Department of Education-sponsored SES programs had a lot more kids in them than this year.  There's been more of an effort, I think, out of embarrassment, to help some of the providers reach out to families. I think the one that--somebody mentioned yesterday Platform Learning is just starting up now.  I went to a luncheon that they held where they invited the parent coordinators at all the schools, these newly-hired positions in the New York City schools, to come and learn about their program.  They fed them, they gave them lunch.  You know, they went by the golden rule.  If you want these people interested, you got to give them food. The parent coordinators were telling me at this thing that they learned more about this from this group Platform Learning than they had learned from anybody at the Department of Education about any of this stuff. The messages that have been sent from the U.S. Education Department, our mayor, and our chancellor have been kind of disappointing to parents who are interested in this.  Early September of 2002, one of Secretary Paige's visits, this was a hot topic.  A lot of TV stations were doing the same thing I was doing, talking to parents who wanted to transfers and couldn't get them. And the mayor was asked, you know, what he was going to do about it and he says, well, within five years I'm going to make these schools better, so the transfers are not going to be an issue.  You know, I think that's what everybody is looking for him to do.  But for the parents thinking about waiting five years for something to happen, when they wanted a transfer now, it didn't go over all that great. Secretary Paige at that same event dismissed it saying, you know, it's a new law, it's going to be hard for everybody, everybody is trying.  And as I say, as a reporter, when you've got people like that talking so softly about an issue like this, it kills your stories because then it becomes the newspaper railing against this.  And I think our newspaper is wondering how much do we want to get involved, since the United States Education Department doesn't seem to care. One minute, all right.  Just some general observations about the transfers.  I think there's about 8,000 transfers this year.  There were a lot of references yesterday to desegregation.  I have noticed the phrase "NCLB kids" popping up a lot.  It reminded me a lot--I was a public school student in the '70s when the term "busing kids," "the bus kids," you know, "time to line up, bus kids"--I think it's been--you talk about the good, the bad, and the ugly.  That's been one of the ugly parts in this. There tends to be a lot of white middle-class parents who themselves were able to play the system to get transfers for their kids who seem to be against the idea of No Child Left Behind transfer kids coming into their school, along with their low test scores and in some cases along with the color of their skin. So that's it.  I wish I could be more encouraging.  I think it's getting there in New York, but I think that New York is a city where you have to kind of get beaten into submission to public officials.  And we're working on it, but we'll see.  But thank you for your time. MR.          :  Thanks, Joe.  Let me just ask you if you've got a couple more numbers near your fingertips.  You said 300-and-some schools are on the needs improvement list? MR. WILLIAMS:  Yes. MR.          :  And approximately how many-- MR. WILLIAMS:  This year, there's more than that.  That was the first year. MR.          :  Okay. MR. WILLIAMS:  It's up to like over 400 now. MR.          :  And about how many of those would be in the second-year status that triggers the public school provisions? MR. WILLIAMS:  That's their first, isn't it.  The first year, all of them-- MR.          :  That's right, okay.  All of those? MR. WILLIAMS:  Right. MR.          :  And does New York City designate receiving schools? MR. WILLIAMS:  They have, and actually I should give some credit for this, actually.  I don't have the full list of where the 8,000 transfers went, but after the New York Times in some of their columns addressed over-crowding at some, I looked at the schools specifically that the Times mentions. They all had schools, at least on paper, that had room in them last year and took in fewer No Child Left Behind transfers than the space that they had available.  So it looks at least on first glance that they actually did do a very thoughtful job in trying to, you know, not heap all of the kids onto a small number of schools, but trying to match individual kids up with the openings that were close to where they lived.  I should give them credit for that.  I think they did do a good job. MR.          :  So you're saying if somebody penetrated the system and actually applied for a transfer, there were options for them? MR. WILLIAMS:  Yes.  The problem, as I see it, has been the communication.  The kids who have applied have generally--there have been cases where some said that the eight schools that there were given had four schools that were viable, two that were across the city that they would never do, and then two others that, in reality, were worse schools, but didn't appear on the schools in need of improvement list. MR.          :  And any sense of whether that applies to high schools as well as elementary and middle schools? MR. WILLIAMS:  It's been much more of an issue on elementary and middle schools. MR.          :  Okay.  Well, thank you all.  These were all extremely interesting. I guess the big question on my mind, and then we'll take some questions from the floor, is given the mix of experiences that we obviously have here experienced and read about, are we dealing here with start-up problems and bumps or are we dealing with an inherently goofy set of assumptions about who should be responsible for what? Will the day arrive when it's reasonable to expect McDonald's to refer people to Burger King?  It strikes me that that's the basic assumption in the public school choice provisions, and indeed in the supplemental services provisions of No Child Left Behind, is that you can expect McDonald's to refer its customers to Burger King if the federal government says they should. And yet in some districts this sounds like it's working relatively smoothly and in others it sounds like it's very painful or not happening at all.  Are we dealing with start-up issues or are we dealing with an inherent problem?  Anybody.  John, then Keisha. MR. LIECHTY:  Yes, let me take a run at it.  I think on first glance, I would come to that conclusion.  Is any school district really going to refer their student population to some groups that are outside the educational issue? Let me put it in terms that I say to folks and let me use Los Angeles specifically.  There's more than enough work to go around, believe me.   We have so many youngsters in need of assistance that having as many partners, whether we're paying it in Title I or whoever is paying it. Again, I come back to the attitude thing.  I don't know why there's this belief system that we can't seem to come together.  And I'm going to use, because I'm old and I'm cranky-- MR.          :  [INAUDIBLE.] MR. LIECHTY:  No, no, but I'm just saying that right now in education, public education, there's this move that has been afoot the last few years that outside of education somehow or other you're magical, versus inside of education.  If you're part of the system, you can't be good.  If you're from the outside, you're automatically good. And I don't know that there's all the data there, and let me give you an example.  When I tried to get the background as to what the criteria was for approved providers from the state, you can't get it.  Even on us, you can't get what they use to say what's a provider.  I'm just sharing that with you as an audience. We have providers in Los Angeles who never see a youngster.  They mail things home back and forth.  It doesn't make any sense to me, but some parents pick them.  So I think there is the ability that we can refer as a district to outside providers to provide that hope.  I just would hope that we're all following the same assessments, the same guidelines, the same evaluation to determine the impact. MS. HEGAMIN:  I think that some of the problems are just start-up issues.  Some of them are--lessons learned, how do you do this better next time?  But I also think that there is also indicators of bigger problems that are underlying--communications between districts and schools, districts and students, schools and students and parents.  That is the underlying issue that I think makes this so difficult.  And it has been there before, but it's really coming out now because of the nature of this law. And I do think that we can and should expect McDonald's to promote Burger King because we're not talking about burgers and chicken here.  We're talking about kids, of course, and there's so much more at stake.  And I think that we have to do that.  I think that this isn't going to work until districts stop looking at this as a competition between themselves and the providers. And I've heard our CEO use that word; this is competition.  If students aren't going to you, it's because your services aren't better.  And we know that's not true because you've had the kids for years and herein lies the problem.  So there are benefits to having other providers come in.  I think districts should acknowledge that.  They can make money. I think there's nothing wrong with charging providers that come into the school, and we've seen benefits when that happens.  Platform Learning in Philadelphia had 1,000 of the 3,000 kids that got tutoring last year.  They partnered with the school, not the district, but the school principals to come in and provide the services to the kids. I got to go and see these kids, and the parents crying, kids crying, because all of a sudden kids can read.  All of a sudden, they like reading.  All of a sudden,   ?    kids are loving their books that they--and, you know, it's the first time.  So I think that we should expect that because there's benefits to it. MS. LITTLEJOHN:  Can I-- MR.          :  Go ahead, Gail. MS. LITTLEJOHN:  I'd like to weigh in on this one.  I think I see it, I guess, as two issues.  There are clearly some infrastructure issues.  As I mentioned in my remarks, if we take in a hundred new students this year, we don't get the money from the state until next year.  So that's not an incentive for you to rush out and recruit new students. But there really is a shortage of skilled labor to help with our low-performing students.  We have students with individualized performance plans and we can't get speech therapists to work with them.  We don't have enough counselors, we don't have enough social workers.  I mean, there's just a shortage.  And if we had them, we probably couldn't afford them, so just real infrastructure issues. And then the second one, I see it as it has almost been structured as a lose/lose proposition.  If you let your students go, you lose money and you've got to lay off teachers and you've got to downsize schools, and nobody likes to do that because it takes energy.  So you lose if you let students go. And then you also face losing if you take low-performing students in.  I mean, you lose the very thing that you worked so hard to achieve, and that's your performance status.  So I don't see that there are enough incentives built into the process to get people to want to do the right things. MR.          :  Joe? MR. WILLIAMS:  I think with the tutoring it'll be interested to see if there's good-quality tutoring that's being provided.  I think it's just inevitable that school principals themselves will start to see it as a way to get their schools off these lists.  And it won't matter to them who's providing it at that point if it gets there. And I have heard principals starting to talk more creatively about the kind of partnerships that they could get in, not wanting for the districts to deal with it, but the principal and PTA members bringing in groups that can use their building and sort of make it work. I don't think they care who is getting paid to do it. MR.          :  You're saying they will see it as in their interest to get their kids up to speed, no matter who is doing the tutoring? MR. WILLIAMS:  Right, yes.  If they can get their kids to improve through this, I don't think it's going to matter to them who is doing it.  It's going to get them off the list and their lives are going to be a lot easier. MR.          :  Do you know, is New York City letting these outside organizations work in their school buildings? MR. WILLIAMS:  Yes, in many, many more this year than last year, because it was an obstacle. MR.          :  Are they having to pay for it? MR. WILLIAMS:  That's a good question.  I don't know that, I don't know that. MR.          :  Okay, if our mike-wielders would wield, you've got an interested audience.  Why don't you start with Jane, and then next back there, next to the pillar?  Raise your hand so that Andrew can see you. Go ahead. MS.          :  Having been on a local school board and also now chairman of the House Education Committee in Missouri, and having gone through an equal access lawsuit all the way to the Supreme Court, if you are having to decide who you charge, then on a level playing field are you charging your own school district providers the same amount you're charging the private providers, if we're talking about equal access? MR.          :  This is for John, I assume. MS.          :  Yes. MR. LIECHTY:  Yes.  We get the same dollar amount that the providers would get based on the same student attendance.  So if-- MS.          :  No.  Do you-- MR.          :  That's not the question. MS.          :  Do you charge rent to your own people just like you do to outside providers so that there's an equal access issue to your buildings for your tutoring people, or equal opportunity?  In other words, it seems unfair to me that you charge the private people, but you don't charge yourself. MR. LIECHTY:  No, we do not. MR.          :  Okay, go ahead. MS. WALCOTT:  I'm Katherine Jovasic Walcott [ph] from West Ed [ph], and I have a question about supplemental education service providers.  And I think I have this right, but this fall districts will start getting identified for school improvement under No Child Left Behind, and my understanding at least in California has been if a districts get identified for improvement, it can no longer provide services. So my question is what is your advice to districts as they gear up for next year, assuming that especially large districts have a pretty good chance of being on that list program improvement, for preparing to build capacity and provide these kind of services when they no longer do that? MR.          :  You're raising a whole new wrinkle that I don't think has been on the table at this conference yet.  You're saying that this coming fall will be the first year in which whole districts are listed as needing improvements, and that it's your understanding that those districts will not be eligible to be supplemental service providers once they're on that list. MR. LIECHTY:  It's not the first year.  We have districts right now in California--Michael knows--that cannot offer--as a district, cannot be a supplemental service provider-- MR.          :  You've already got aid. MR. LIECHTY:  --because they've already been identified as a district program improvement district.  My recommendation--and L.A. obviously we're doing that.  This is why I want to use the incentive that Joe just mentioned. The game plan is to get as many youngsters as possible the academic help they can, with the goal that, number one, that's going to help young people.  But number two, that's going to help those schools not be program improvement schools, and principals are going to buy into that. Number two is that obviously our hope--and it's why we have partnerships, both independent supplemental service providers, as well as supplemental service providers that we contract with, okay, who are in alignment with our district goals in reading, language, and mathematics, so that if we reach that point we may be--these are the providers that, in fact, would be available to parents to choose from. So I would recommend to any district that has large numbers of program improvement schools--if you look at that trajectory, you better have a backup plan if, in fact, you become a program improvement district. MR.          :  Are you affirming, Mike, one of Mike Casserly's points, which is that the private providers that are, as it were, competing with you are also the same private providers that the district itself outsources to? MR. LIECHTY:  In some instances, yes. MR.          :  Okay.  Go ahead. MS. LOUCKES:  Hazel Louckes [ph].  I have a question about you talked, John, about the Title I dollars, the 20-percent set-aside.  We heard yesterday that that's about half the schools that do the 20-percent set-aside. We heard from the Department of Ed that a lot of school districts didn't even spend their money that they needed.  Do you think there is enough money eventually--I understood your problem that you don't know until March whether you can use the money and then you have to ask for some kind of a waiver or something.  To your knowledge, were the waivers given?  Is it indeed that there was more money that they needed or is it just the complications with law? I'd ask any of you to respond to that. MR. LIECHTY:  I'm not going to sit here with the TV on or whatever.  Money is always an issue.  My guess is--and again I'm optimistic, so I can just say that shortly, within a year, 18 months, if I go down the road, we're going to be oversubscribed as far as the number of youngsters who actually want service based on the amount of dollars that are available out of that reserve. MR.          :  So the 20 percent may not be enough at that point? MR. LIECHTY:  It may not be enough, may not be enough.  And so therefore then districts are going to have to do one of two things, either find additional Title I or other funds of money, and I  think that's what Nina was talking about yesterday.  Or, remember, eligible youngsters in these program improvement schools--it's based on the economic status of that youngster. If you're oversubscribed, then the priority is low achievement, so that you may be caught in a district, if you're looking down the road, that you've got 40,000 youngsters who want the service, but the dollars available--you're only going to be able to service 20,000 of them.  You're going to have to prioritize those youngsters, which can create another whole set of issues as you move down the road. MR.          :  But there's a prohibition against using the other 80 percent of the Title I dollars for this kind of activity, right? MR. LIECHTY:  No, there's no prohibition. MR.          :  Okay.  Yes, behind you.  Why don't you let Rick ask the question before he gives us lunch? MR. HESS:  There we go.  Rick Hess, AEI.  It seems to me one of the things we haven't spoken about much is that NCLB, of course, is premised on the notion of school accountability.  We're going to hold schools and districts accountable. But supplemental services, particularly the evidence-based element of it, is really about the sup service providers doing a decent job at a much more narrowly prescribed test, which, to raise Mike's point of alignment, may or may not be exactly what the district or the school as a whole needs the students to learn in order to meet AYP. Is this then a challenge in Dayton or New York or Philadelphia or L.A.?  And going forward, are there any concerns about this or does this seem to be a hypothetical that's not a real concern? MR.          :  You're saying, in effect, the school is accountable for whether the kids learn A, but the sup service provider might be teaching B and be judged on its effectiveness in teaching B. MS. GAIL LITTLEJOHN:  Well, I'll respond for Dayton.  We're certainly working on the alignment.  We have--I think the primary concern that we have is I think it's always easier to work with, you know, a larger agency like Sylvan, so you have one communication. So far, we have, you know, individual contractors for the most part coming forward and, you know, offering tutoring services.  And so that's a much more difficult objective when you've got, you know, dozens of people that you have to work with and try to get aligned versus, you know, one organization you can have that kind of an agreement with. MR.          :  Is Dayton already using private providers for any of its Title I activity? MS. LITTLEJOHN:  Yes, yes. MR.          :  Are you still working with the same ones, basically? MS. LITTLEJOHN:  Some new ones. MR.          :  Some new ones. MS. LITTLEJOHN:  Some of the same ones and some new ones. MR.          :  John? MR. LIECHTY:  I would echo what Gail said.  Obviously, folks, if you have one or two providers, much easier to coordinate, coming up with an assessment.  The analogy I use is every time I do a self-evaluation, very rarely do I have any areas that I need improvement. And obviously it's kind of like being out of the classroom.  The longer I'm out of the classroom, the better the teacher I was.  So in my own mind, I'm a legend.  But it also is true of providers of tutorial services both in the district and outside the district.  I've never had a provider come in and tell me, you know, John, you're right, we met the target totally.  Look at this, and we got no scores, we got full dosage. What we need to come to grips with is not the alignment, but the assessment we're going to use to measure the impact of these services.  If everybody is using their own assessment, they may come up with great data, but the bottom line it's hard to reach any conclusion.  And the more providers you have, the more difficult. And then the final piece of that: If you don't have multiple providers, then are you really providing parents a choice?  So you got to measure, balance the choice with the number of providers. MR.          :  Keisha, how many providers are there in Philadelphia operating now as far as you know? MS. HEGAMIN:  There's about 20 in Philadelphia, and there's much more, obviously, in the state.  But in order to contract with the district, there has to be that consistency with the curriculum.  So maybe not every provider is going to provide math services, or maybe not every one is going to provide reading services.  But the district outlines that for parents this year, very clearly, who it is that's providing what type of services, but they-- MR.          :  Does the district check for alignment before it contracts with the outside providers? MS. HEGAMIN:  Yes. MR.          :  They do? MS. HEGAMIN:  Yes. MR.          :  And so when you and your colleagues go around talking to parents, do you have a sort of menu of 20 possible providers of sup services? MS. HEGAMIN:  Yes, exactly, and we've done things like provider fairs so that parents can meet the providers.  I think that that needs to be an informed choice, just like school choice needs to be an informed choice. MR.          :  Do they have more than 12 hours' notice about the provider-- MS. HEGAMIN:  Yes, they do, a little bit more, not much more, but yes. MR.          :  Okay.  Andy? MR.          :  The question I had-- MR.          :  Introduce yourself. MR. SMEREK:  Andy Smerek [ph].  I work for the Charter School Leadership Council.  It seems to me that the way that you're going to actually prove--well, it's important that data collection, is what I'm concerned about.  The way that you short of shame New York City or Worcester into doing a better job of this is a couple of years from now, actually have the data to prove that in the cities, in the districts, in the schools that are actually making use of supplemental services, their students are doing better, or making use of school choice, their students are doing better. And this is something I probably should have asked the researchers about, but we're going to need this data; you know, run the regressions, show that districts that are doing the right things are getting better results.  But you need the data for that. So is there any comprehensive way, for example, in Los Angeles for you guys who are doing a very good job of tracking these students and how they're doing on reading tests, math tests, and so on and so forth? And then also maybe the answer is, no, no one--this isn't required in No Child Left Behind and no one has the money to do this, and is this the kind of thing that NCLB maybe envisioned the Department of Education giving a grant to collect this data? MR.          :  Surely, the school and district data will exist as part of the normal NCLB data-gathering system, won't it? MS. LITTLEJOHN:  Yes, but I think the issue that he's getting to is are we going to be able to track it student by student. MR.          :  Okay. MS. LITTLEJOHN:  Yes. MR.          :  [INAUDIBLE.] MS. LITTLEJOHN:  And in Dayton, we're moving to that.  We just developed our capability to evaluate school by school and classroom by classroom.  We are not yet at the child level, but we're going to be there next year, hopefully.  And what's the mobility?  I mean, you've got to have it.  Otherwise, with the rate of mobility in the urban-- MR. LIECHTY:  Los Angeles is also looking at it.  Our board of education obviously is very interested in, number one, the youngsters who signed up.  And again I think all of us who have been there at the implementation--just because you have these youngsters sign up doesn't mean that youngsters are getting the dosage that they're entitled to. This is hard work, whether you're an outside provider or an inside provider, because many of these youngsters Monday through Friday, during eight to three, are struggling to meet standards.  They're not oftentimes beating down the doors to come either after school or on the weekends. So you've got youngsters who signed up for a service and all of a sudden you look at the profile, and let's say it's a 50-hour dosage.  They only attended ten hours, and so not only must you identify the youngsters and track them, but then you got to start to benchmark at what point do you start seeing some positive results in achievement so you can start making more informed decisions. We're just getting ready for an RFP for evaluation to take a look at that.  I hope I'll be able to keep you informed.  I don't know how that's going to turn out. MR.          :  So we're probably down to our last couple of questions. MR. MOSS:  Jason Moss [ph] from Scholastic.  Among parents who are enrolling their children in the supplemental services, do you have a sense of what are sort of the key drivers for why they're choosing one provider over another? It seems to me that location would certainly be an obvious one in terms of convenience, but other than that, whether it's brand or its the efficacy behind what they're doing, any thoughts on that? MR.          :  Is Scholastic currently a provider? MR. MOSS:  No.  Scholastic sells materials to providers. MR.          :  Okay. MR. WILLIAMS:  I think location in New York City has been by far--if parents can get a provider that's going to do it either at their school or somewhere near to their school or someplace where they'll get the kids to their services, I think that's been the driving factor. You know, the biggest ones aside from New York City schools that are doing this are Kaplan and Sylvan and all.  And I think they're in the process still of determining which neighborhoods they want to set up shop in and where they're going to get the most parents coming in.  But this is still playing out. I think this is promising.  I think the more we figure out where we need to put these things and what parents are looking for, I think we are going to see more of that in New York City. MR.          :  [INAUDIBLE] to Philadelphia, what do they seem to be weighing as factors? MS. HEGAMIN:  Well, a lot of times in Philadelphia the   ?    kind of weed themselves out because, for example, like Platform might only go to sixth grade.   So a lot of times, depending on what grade the parents are--the children are in, that's going to be a determining factor. But we give parents a whole list of things that we think that they should check on when deciding so that they make an informed decision.  The time of the sessions is going to be important.  Some of them only offer Saturdays.  Some of them only offer it during the week after school.  And some parents have to have their kids in district- mandated after-school programs, so they don't want to overburden their kids during the week.  They might want it on a Saturday. We also--parents, of course, are going to look at the location, where it is that they can get the services.  The track record a lot of times--we had 1,000 kids out of 3,000 go to Platform.  It's a new company, but I think because they did so much outreach and they went to parents and, you know, gave them the applications and door-to-door--but parents are picking the big names, too.  They're picking the Sylvan and all that. Another factor that we tell parents to consider, and I think they do, is the cost of the services of that provider, because some providers charge a lot less than others for the services, which means you can get more hours of tutoring.  So for some parents, that's more important than who it is that's providing it. MR.          :  Anything to add on this one? MR. LIECHTY:  I think L.A. is a combination of all   ?     .  The first year, I think it was the alphabet, to be very candid with you, because it was new.  I'm just telling you I had many companies come to me wanting to change their name. MR.          :  You mean the triple-A company got-- MR. LIECHTY:  You know, we had a group, the first one in the brochure, because we had like 23--it was A-plus.  So A-plus as an outside provider in that particular last year got more parents than anybody else.  So it was automatically assumed by everybody it was alphabet, so they wanted to change their name to make sure it was in the As. I think the big-name companies are making real inroads, at least in Los Angeles, and parents are--through parent organizations are thinking more carefully. MR.          :  Are these big companies--we're going to have to close in a second, I'm afraid.  Sorry, Mike. Are the big-name companies advertising themselves?  I mean, are the school boards around Los Angeles saying-- MR. LIECHTY:  Yes.  We seem them on buses.  They've done a great job for the most part.  As a matter of fact, some of our partners that work with us--I know Sylvan, for example, even outreached to parents and put out flyers and did some things to try to get the message out because of this controlling factor, the big bully LAUSD. Remember, folks, as part of a district and as part of an implementation strategy, our schools are hit frequently, like everyday 23 times, about having a captive audience.  And people want inside that school for kids, for parents, or for teachers.  And so we control it to quite an extent because otherwise we're going to be disrupting the school day, the school week, and in the evening, et cetera, et cetera.  So many of our provider partners out there have reached out to the community and done some pretty good things. [END OF TAPE 2.  TAPE 3 IS BLANK.]</body></html>