<html><body>With the Best of Intentions: Lessons Learned in K-12 Education Philanthropy April 25, 2005 Unedited transcript prepared from a tape recording. <TABLE width="100%" border=0> <TBODY> <TR> <TD vAlign=top align=left width="25%">8:15 a.m.</TD> <TD vAlign=top align=left width="75%" colSpan=2> Registration and Breakfast</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">8:45</TD> <TD vAlign=top align=left width="25%">Welcome:</TD> <TD vAlign=top align=left width="50%"> Frederick M. Hess, AEI</TD></TR> <TR> <TD vAlign=top align=left width="25%">8:55</TD> <TD vAlign=top align=left width="75%" colSpan=2>Panel I: Understanding K-12 Philanthropy</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%">Presenters:</TD> <TD vAlign=top align=left width="50%">Richard Colvin, Hechinger Institute at Columbia University</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Tom Loveless, Brookings Institution</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Donald McAdams, Center for Reform of School Systems</TD></TR> <TR> <TD vAlign=top align=left width="25%">10:00</TD> <TD vAlign=top align=left width="25%">Break</TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">10:10</TD> <TD vAlign=top align=left width="25%">Discussants:</TD> <TD vAlign=top align=left width="50%">Michael Feinberg, Knowledge Is Power Program (KIPP)</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Susan Fuhrman, University of Pennsylvania</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Lawrence Patrick, Black Alliance for Educational Options</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">William Porter, Grantmakers for Education</TD></TR> <TR> <TD vAlign=top align=left width="25%">11:10</TD> <TD vAlign=top align=left width="25%">Break</TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">11:25</TD> <TD vAlign=top align=left width="75%" colSpan=2>Panel II: Assessing Major Strands of Giving</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%">Presenters:</TD> <TD vAlign=top align=left width="50%">Jane Hannaway, Urban Institute</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Bryan Hassel, Public Impact</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Jay Greene, Manhattan Institute</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Andrew Rotherham, Progressive Policy Institute</TD></TR> <TR> <TD vAlign=top align=left width="25%">12:45 p.m.</TD> <TD vAlign=top align=left width="25%">Luncheon</TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">1:30</TD> <TD vAlign=top align=left width="25%">Discussants:</TD> <TD vAlign=top align=left width="50%">Arlene Ackerman, San Francisco Unified School District</TD></TR> <TR> <TD vAlign=top align=left width="25%"></TD> <TD vAlign=top align=left width="25%"></TD> <TD vAlign=top align=left width="50%">James A. Kelly, National Board for Professional Teaching Standards </TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Bruno Manno, Annie E. Casey Foundation</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Kim Smith, New Schools Venture Fund</TD></TR> <TR> <TD vAlign=top align=left width="25%">2:30</TD> <TD vAlign=top align=left width="25%">Break</TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">2:40</TD> <TD vAlign=top align=left width="75%" colSpan=2>Panel III: Lessons Learned</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%">Presenters:</TD> <TD vAlign=top align=left width="50%">Peter Frumkin, Harvard University</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Steven Heyneman, Vanderbilt University</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Dan Katzir, Broad Foundation</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Leslie Lenkowsky, Indiana University</TD></TR> <TR> <TD vAlign=top align=left width="25%">4:00</TD> <TD vAlign=top align=left width="25%">Break</TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">4:15 </TD> <TD vAlign=top align=left width="25%">Discussants:</TD> <TD vAlign=top align=left width="50%">Alan Bersin, San Diego City Schools</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Howard Fuller, Marquette University</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Wendy Kopp, Teach for America </TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Warren Simmons, Annenberg Institute for School Reform</TD></TR> <TR> <TD vAlign=top align=left width="25%">5:15</TD> <TD vAlign=top align=left width="25%">Break</TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">5:25</TD> <TD vAlign=top align=left width="75%" colSpan=2>Conclusion: What s Next?</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%">Discussants:</TD> <TD vAlign=top align=left width="50%">Chester E. Finn Jr., Thomas B. Fordham Foundation</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Stefanie Sanford, Gates Foundation</TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%">Dan Fallon, Carnegie Foundation</TD></TR> <TR> <TD vAlign=top align=left width="25%">6:15</TD> <TD vAlign=top align=left width="25%">Reception</TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="25%"> </TD> <TD vAlign=top align=left width="50%"> </TD></TR> <TR> <TD vAlign=top align=left width="25%">7:00</TD> <TD vAlign=top align=left width="75%" colSpan=2> Adjournment</TD></TR></TBODY></TABLE> Proceedings: MR. HESS:  I used to be able to silence a high school class and I still can't silence a D.C. crowd, so I don't know what's going on.  Those of you who know me know I'm a time Nazi, so we're going to start on the moment and we're going to run to the moment all day because I know all of you are busy people and we've already got cellphones going and the whole mess.  I'd actually ask folks, please go ahead and put your cellphones on vibrate or turn them off.  You know, it makes it tough to have a conversation when we're constantly working our way through the ringing. I'm delighted that all of you could join us today for this conference, With The Best Of Intentions, Lessons Learned in K-12 Education Philanthropy. For those of you who don't know me, I'm Rick Hess, director of education policy studies at AEI.  For those of you who do know me, you know that there's not a whole lot of reason for me to run an event like this, if we're just going to be nice and talk pablum and not do a whole lot of anything. So my fondest hope for the day is that both as the presenters are presenting the results of the research they've been working on, as we discuss and engage with them, and as you contribute your questions and your comments and suggestions, that all of us engage in this in the spirit of serious intellectual conversation rather than in some form of kumbaya, or isn't philanthropy swell? because, quite frankly, there's plenty of that.  It's not nothing that anybody needs me to do. Let me talk briefly.  I'm going to take about eight or ten minutes, just kind of talk about what we're doing here today and how the day's going to work, and then we're going to roll right into the first panel. Tom Loveless is here, despite the glorious Metro system leaving him on a bus from 495 when it broke down, and Tom, being the intrepid soul he is, still made it here.  So God bless him.  All right.  Why are we holding this conference today? Well, as most of you know in this room, education philanthropy has always been with us but it's historically been more focused on local and private giving, to local schools, to community building endeavors, to favorite private institutions. There have always been notable exceptions but that has been the norm.  In the last 10-20 years, philanthropists, for a variety of reasons, partly growing out of the frustrations that had voiced "a nation at risk," partly because of the entrance of new givers into the sector, partly because of the nationalization of policy that played a much more instrumental role. Many of you in this room have experienced this personally.  I'm looking at Wendy, you know, I'm looking at Jim Kelly of the National Board.  Many of you in this room are well aware of how significant the role education philanthropy has played in driving some of the most interesting and significant reform efforts of the last, say, 10 or 15 years. Well, what's the purpose of this particular conference, then?  Why are we holding it now?  Well, I'm one who has suggested that this increase in public role of philanthropy is both appropriate and constructive and necessary.  I don't see any problem, and I'm a huge fan of philanthropists getting into this arena and starting to impact entrepreneurial efforts in major public policy.  I think that's great.  However, I think we've been too slow to note that there is a reciprocal change that takes place when philanthropy shifts from the private and the small scale into the public, and that shift is that you're no longer just talking about purely private function. What you're talking about is an effort to shift the way that public policy looks, that public policy addresses the needs of children, and the way, the public person "spin" [?].  This is part of a public conversation, and when you move into a public conversation, in my world, you need to embrace transparency and you need to accept scrutiny, often kind of tough-minded scrutiny of your aims, your activities and you methods. I think this is a win-win, even though it feels like a lose-lose.  I think it's a win-win because it helps policy makers and citizens and, you know, practitioners and researchers make good decisions and make good choices.  I think it actually benefits philanthropists because it gives then honest, critical feedback on what makes sense and when their money is being well-spent. Now when I say it's played a central role in reform in recent years, I think most of you in this room I don't need to prove this to, but let me just, you know, a couple of highlights. You think about just efforts in terms of urban reform, in terms of choice-based reform, in terms of reform to the teacher labor market, in terms of high schools, obviously.  There's been significant activity. The most notable kind of best illustration of this of course is from the spring when, you know, as you all know Bill Gates spoke to the National Governors, really crystallizing, in many ways, the national agenda that the Gates Foundation has been significant in helping to advance and promote, and as I've said, I think this is all to the good. What's interesting is not only is this engagement in some kind of cause-and-effect relationship with broader reform currents, but the folks in policy positions routinely make really strong claims about some of what goes on.  This is my personal favorite.  If you look back over the last decade, the Texas Education Commissioner, in 03, termed one philanthropic effort in Texas probably the most significant education reform ever undertaken in the state of Texas. Now anybody who's been following education reform for, I don't know, the last 20 years, that's just a hell of a statement.  But I think it's actually not atypical.  If you read the newspaper coverage, if you look at the way so many philanthropic efforts are discussed, both in the glossy brochure and in the analyses, you actually don't find that to be atypical.  For instance, one thing some of our research assistants and I did as part of a conversation with Philanthropy magazine over the last several months, was we looked at major media coverage of five of the leading kind of new philanthropies or most significant philanthropies of the last ten years.  We looked at--this is coverage of Walton, Gates, Broad, Milken and Carnegie, and as you expect, not surprisingly, about half the coverage was factual or neutral, which seems appropriate. When you look at the other half, though--and these aren't op-ed, these are news stories--look at the other half, 93 percent of these stories were positive, were excited, happy stories, and five of the seventy others, about 7 percent, were critical.  When you have this kind of environment it can become very difficult to talk in kind of tough-minded terms about what's going on. Here's a couple of the sample headlines.  These should look familiar to you.  We see these every day.  You know, you see verbs like "plugged in," "devotion"--devotion's not even a verb.  Beautiful!  "Award."  You know what I'm talking about!  I should never start these things before nine.  I'm not even that good after nine. And the fact is of course if the press is that nice, these famed junkyard dogs who were kind of insulated from, you know, the larger role of philanthropic giving, it's probably not a surprise that the folks actually in the world of philanthropic giving, the practitioners, the scholars, the reformers, themselves tend to be quite hesitant about saying anything too mean-spirited or too tough, when it comes to talking about various reform initiatives. I want to suggest that there's at least three kind of important reasons that we need to think about.  One is the obvious.  You know, we love these guys.  You know, look around the room at the donors who are here, and you're like "You are beautiful human beings."  You know, either you personally, or you're working for people who went out, made a whole slew of money, could be buying extra yachts, and instead of giving it away, largely to help underserved children have better--that's fantastic.  How could anybody be so mean to say something critical?  Only a guy like me actually would.  So that's actually entirely understandable. Second of course is that academics, activists, reformers, even policy makers, as we see in kind of the NGA Achieve initiative, know that philanthropic support is often the ticket to doing big, great, exciting things.  The last thing you want to do when you're putting together a grant proposal asking somebody for $3 million to do the study of your life, or to open a whole bunch of new programs in your district, is have to explain why, in that week, three months ago, you were calling them "no good sons a guns." So there's a real hesitance to pull the trigger and say tough-minded things.  And then third of course, even the most insulated scholars, even those of us in our little ivory towers in academia or in think-tanks, who, you know, in theory, are buffer to some extent, if we want to be out there, if we want to be doing research, if we want to be part of reform initiative, you need collaborators.  You need colleagues to work with. And quite frankly, folks in school districts, folks who are doing research, tend to be a little skittish of working with people who are known to be caustic when it comes to some of these prominent initiatives. Even if the foundations don't give them any reason to be.  They're just risk-averse.  They say, hey, there's a lot of evaluators, a lot of partners.  Why would I want to get in bed with somebody who's a known rabble rouser?  So what we wind up with is relatively little straight talk about philanthropy. What we do get a lot of is self-appraisal and self-evaluation.  And that's great.  It's appropriate, it's helpful, it's constructive.  Of course it's worth noting that most of these evaluations, except in rare cases, are done by colleagues or friends or institutions of long-time relationships with the folks who are hiring them to the evaluation, and these evaluations tend to come off a little soft or a little gentle or a little mealy-mouthed.  But that's okay, cause they always look good in the glossies at the end of the year. Now even though the self-appraisal is useful and helpful, it's not what I'm talking about.  What I'm talking about the fact is that these self-appraisals tend to take place in private.  You often include a couple skeptics.  I sometimes show up in these rooms just to be the token "bad guy." But they primarily include allies and grantees and people who share certain presumptions about how schools work and how reform should work.  Even kind of outspoken participants, even people who are known to be, you know, a little caustic, in these formats they often, I suggest, "pull their punches a little bit." They don't want to give offense to their hosts.  They're trying to frame everything in a way that's constructive and collegial.  And finally, you know, the fact is that the public forum, for all of the vitriol it brings out, is a great place to really "get out there," competing assumptions about strategies, about how the world works, and really about the validity of results from various analyses. But it's funny--even though some of these prominent initiatives, like, say, Teach For America, will sometimes be the subject of hard-hitting public back and forth, very rarely are the foundations themselves, or the grant strategies the subject of a hard-hitting back and forth. And I think we're all the poorer for it.  So what I want to suggest, the purpose of today's conference is that we need public analysis of the role of philanthropy in education.  Self-assessment is sufficient for purely private efforts, I would argue, but it fails, it falls short when we're talking about efforts that are geared explicitly to influence public policy and the public purse. Moreover, the fact is, I would suggest, that too often those best-suited to offer thoughtful, critical commentary on public giving, on philanthropic giving intended to affect these things, are those most hesitant.  The people least likely to talk about the Broad efforts or the Annenberg efforts, I would argue, are the experts in urban reform, because they're all engaged in them. The folks least likely to talk critically about what the Walton Foundation is doing in charter schooling are many of the nations leading experts in charter schools, because they're partners and colleagues with Walton and its important efforts.  It seems to me that the notion of being a critical friend is difficult in these circumstances and that what we need to do is create a public forum and create and environment where it's safer for people to play that role of critical friend or hostile critic, for that matter. So the purpose of this research and this volume that we're doing today is really to try to start to, step by step, expand the space for honest conversation. The panels today, first, a presentation by authors, then a presentation by a panel of discussants, is going to proceed in three sets.  We're going to start with the first panel, with Tom and Don and Richard, and I'll introduce them in just a second, is going to look at the landscape of giving, how much--excuse me.  What are givers thinking?  How does the new philanthropy work and what's going in urban districts? The next panel is going to look at approaches to reform, particularly how much money is given, why is it being given and how are these strategies being implemented? The third set of panels in the day is going to look at what we've learned so far, trying to draw on history, on international giving, on the experiences learned at one of these new philanthropies, and Peter Frumkin of Harvard who's been looking at these issues for a whole longer than I've been thinking about them, is going to offer some thoughts on how funders, in particular, and how applicants can think more constructively about the process. And finally, what we're going to do is we're going to ask a couple folks who have been thinking about these issues for a long time, and really, at the heart of many of these conversations, to try to summarize and reflect a little bit on what the research and conversation has meant for philanthropists, policy makers, practitioners, researchers, the whole gamut. We've got hard copies of all the papers, which you should have been able to obtain when you came in.  If you have not, or if you have colleagues who you think might be interested in some of this, all of the papers, as of today, will be available at this Web site, <A target=_blank href="http://www.aei.org/" target=_blank>www.aei.org</A>, event 959, and they'll be up there for a few months. This fall, the revised papers, along with some meanderings from myself in the form of an interim conclusion, will be published by Harvard Education Press in October, and that's going to be titled "The Best Of Intentions," and I'm sure it'll make a lovely Christmas gift for those who no longer read. With that, let me go ahead and roll directly into the first panel, and get us started with some thoughts on the landscape of education reform today. The participants in the first panel.  We're going to go in the order of Richard Colvin first.  Richard is director of the Hechinger Institute on Education and the Media at Columbia University. Next to Richard is Don McAdams, president of the Center for Reform of School Systems, and next to Don is Tom Loveless, director of the Brown Center on Education Policy at the Brookings Institution. Each of our speakers is going to speak for about 12 to 15 minutes.  I will ask you not to ask any questions during their presentation.  After they present, we are going to have about 20 to 25 minutes for Q&A.  As I'll remind you, after they speak I will ask that you keep questions pithy and actually in the form of questions.  I will ask that you wait for the microphone to come around and I will reserve the right, both with the speakers, and with yourselves, that if anybody gets into a meandering similar to the ones that I am so known for, because I'm the guy standing up here, I will off you off, out of respect for everybody's time. With that, Richard, would you please get us started. MR. COLVIN:  Good morning.  It's interesting to be up here because I see many familiar faces, people who have come to seminars that I've organized, and you've been up here, so it's a little daunting to speak to you about these things.  My assignment from Rick was to take a journalistic approach to the landscape of K-12 education philanthropy, kind of look at who are the important players, what are they up to, how do they decide what to work on, what their theory of action is, you know, how they evaluate the effectiveness of their efforts, and, you know, how they are responding to these enormous challenges of improving academic achievement for American students. As this reporting went on, I had additional conversations with Rick, and how this paper sort of fit in with the rest of them, and some additional kinds of issues emerged, and one of them, Rick has talked about some here in his opening remarks.  He also wanted me to think about this issue of accountability, both in some of the ways he was thinking about it and also the role of the press, and given that I spent about 25 years writing about writing, and much of that time writing about education in newspapers, I had some thoughts about that. And Rick is right.  I mean, what we're talking about is a big shift in philanthropy to where philanthropy is to go beyond having an impact on individual students or schools or districts, have to engage public dollars, so therefore it's very important for the public and the press to hold philanthropists accountable. So as any of you who have been involved in any projects that Hess has anything to do with, it was not a small or insignificant task.  I learned a lot in the course of my reporting.  It's a little daunting to share what I learned with an audience such as this, that knows so much more about it than I do, but I'm just going to try, in my remarks this morning, to just sort of point out some things that I as a journalist saw, that seemed interesting to me in that broad landscape. Let me also just say, in the interests of full disclosure, I'm in somewhat of an odd position in a way.  I'm one of the people that Rick, now that I'm at teachers college, that Rick mentioned, you know, all of the funding for the Hechinger Institute on Education and the Media comes from foundations, and the foundations generously underwrite our work with journalists to help them do a better job of covering education. Some of the foundations that I write about in this chapter and some of those who are represented in this room are generous funders of ours. They include, and I want to get those on the table, and--[laughter.]  They include the Broad Foundation, the Wallace Foundation, the Pew Charitable Trusts, Carnegie Corporation and the Hewlett Foundation.  I've tired to be as even-handed as I could in thinking about the impact of these entities but you'll have to be the judge. So how much is involved?  What are we talking about?  Jay Greene is going to talk on the second panel, in some detail, about what his estimates are.  I basically accept his ball park figure of somewhere between 1.2 and $1.5 billion. I think it's probably higher than that.  His data is 2002.  I think there's been significant spending since then. I think a lot of the money that goes towards higher ed actually ends up benefiting the K-12 sector and it's difficult to figure out what goes where. I think there's a lot of corporate giving.  Just one example.  In 2002, IBM gave $75 million to various K-12 efforts.  That doesn't show up in the foundation center tally of philanthropy, so there's significant amounts of corporate giving that's out there. Now not all of that of course, probably not even half of it, goes towards the changing, in any fundamental ways, what goes on with students and teachers and schools.  A lot of it is, you know, scholarships, capital spending, these kinds of traditional philanthropic efforts. So in terms of the volume of money that's going about to effect, that's going into school reform, it's difficult to know.  It might be what? half a billion dollars, maybe 750 million.  It's hard to know.  But it's significantly smaller than the total spending on K-12 education from a philanthropic perspective. This money is likely to increase.  We know that it's been written about a lot.  There's a huge amount of wealth that is going to be transferred generationally in the next two decades. If traditional patterns hold true, about a quarter of this will go into education generally.  The bulk of that--not a quarter of the 1.7 trillion.  A quarter of that money that goes to philanthropy will go to education generally.  Some portion of that will go to K-12, probably significantly less than the amount that will go to higher education. I was struck, in talking to philanthropists across the country, about this real sense of unease about where American schools are headed.  Certainly since 1983, and the Nation At Risk report, this has been a very common theme.  But it seems like the rhetoric that I was hearing from philanthropists and program officers, it's of a different character.  It's a different order of concern. I just want to quote a couple statements to you.  Last fall, when he was announcing the 2004 Broad Prize in urban education, Eli Broad has this rhetoric: Public education is, in many ways, in a crisis we can no longer ignore, we are in danger of becoming a second-class nation, it's up to everyone, he said, to take responsibility and get involved. Rick referred to Bill Gates' remarks earlier this year.  The rhetoric there, too, was really quite striking in terms of the seriousness with which he took this role of improving high schools.  Our high schools are obsolete, I don't mean that they're just broken, flawed or underfunded, although I couldn't argue with those descriptions. What I mean is that even when they work exactly as designed, our high schools cannot teach our kids what they need to know, and finally, he said, you know--and this comes from Bill Gates so he kind a knows about this area, I think. This is an economic disaster, one that is ruing children's lives and is offensive to our values. And there is also this sense in that rhetoric, that this is a global issue, and he, Gates himself said, you know, that the high schools he sees in other countries are doing a far better job.  There's a sense of the globalization of the economy, upping the stakes for American schools. I must say, though, that that sentiment of how bad things are, or not really how bad things are, but more how inadequate the schools are for the economic and demographic challenges that lie ahead of us.  That kind of rhetoric was not universal in my reporting. There were quite a few people who, you know, worked for large foundations, who said kind of, you know, the schools are doing a pretty good job for most kids.  It's just some kids that are not being well-served.  It's not the whole system, it's not like we have to change and redesign the whole system.  We need to kind a tweak it a little bit, redirect resources, make things more equitable, fix things in the poor urban districts. But, otherwise, you know, American education's on pretty good footing. So there was this divide in talking to philanthropists across the country.  I also want to point out that this boldness of the rhetoric that we hear from two of our most prominent philanthropists in the area of education is really quite unusual for philanthropy. You know, I think in the past there's been a reluctance for donors to put themselves forward and they've tended to promote the organizations, districts, the people who get their money. It seems to me there's both a potential power in that very public rhetoric but there's also a potential risk.  The power is of course celebrity and money can turn heads, as Rick suggested, and get people to pay attention to important issues. The downside is that if you say I'm going to fix high schools in America and you fail, well, you know, that spotlight can be pretty harsh.  So there's a up side and down side. I think we all kind of know the foundations that have been traditionally involved in this area.  I was struck by the fact that a number of them have stepped away from K-12 education, some of them to go into other areas, ancillary to education, notably Packard and Pew are now funding early education, before kindergarten. The Clark Foundation has decided, rather than middle schools being their agenda, they are going to fund a few providers of after-school services and try to help them develop a capacity to provide terrific direct services to kids. So there's a shift going on and as the--oh, also, it was notable to me--three minutes left?  Oh, man!  All right.  So we're going to fast-forward here. So there's a lot of new givers.  I've mentioned some of them.  Some of the ones that I was not aware of.  The Pisces Foundation, the founder of GAP is a big player in this area, spending more money than Carnegie or Ford or Rockefeller or some of these other names. The Dell family has a large endowment.  They're supporting Avid out of San Diego.  Rick mentioned the Walton Foundation.  There's a number of others.  So there are new players. Rather than go into so much what they're doing, I've suggested some of it, leadership management and governance are I think a new area that foundations are going into, notably by the Broad Foundation, the Wallace Foundation, into leadership.  The Stupski Foundation, also a big player.  If they were a charitable organization rather than an operating foundation, they would be among the twenty largest. The whole area of school choice and vouchers, I don't think that would exist without the work of donors.  And interestingly, the Walton Foundation doesn't work only on choice and charter issues.  They're also looking for districts that they think are responding well to competition, to try to support them, and Buddy Philpot, the executive director, said they haven't found as many districts that fit the description as they would like, which I thought was an interesting comment. Let me finally say that, just to speak briefly about what Rick had to say about the press.  I think he's right.  I think it is partly, as he suggested, we like to give praise to those who give their money away.  I think it's also surprisingly, even tough Rick says we're all "junkyard dogs."  We, the press, like any of us, sort of cowers in the face of tremendous wealth and power, and I think we're unwilling to challenge, in some sense, speaking truth to power. I think that's nice rhetoric but I don't think we do that all that well, and, in addition, I don't think that the press really knows very much--and I guess that comes back to what I do. They don't know enough to challenge the substance of what's going on, to challenge whether these theories of action really make sense, to challenge the education aspects of it, and so they don't.  I think that's fair to say.  We can about more of that in the final. Let me just share with you one anecdote which may, in some sense, be hopeful, and hopefully I won't be yanked before--it's very short. Alan Bersin, sitting right over here, told me recently, that when he came into office, he sat down in a room with a group of foundations that were giving the district about $1.7 million, and the funders basically said to him, you know, you have to start getting along with the union because we're going to take our money away unless you start getting along with the union and make nicey-nice. And what Alan said to me is that was important to the funders in that room was the process.  He said the foundations wanted us to be collaborative.  The children's agenda, Alan said, took a backseat. Fast-forward to 2001, a different group of foundations, different approach, one that's more focused on outcomes for students, had gotten involved.  They were giving about $40 million.  Hewlett, Gates, Broad, Walton, Carnegie, Atlantic philanthropies were all involved, and their reporting requirements were less onerous, they were more agenda-driven and the agenda was the children's agenda. it's been hard for educators, as Tom Paysant [ph] observed to me, to turn down donors' money, and so school districts get pulled in lots and lots of different directions. So there's a mutual responsibility of the role of foundations and districts and recipients to be disciplined in how they ask for money and how they give money, and they have to be focused on what really should be the bottom line, which is outcomes for kids. MR. HESS:  Thanks, Richard.  You know, I never envy Richard the job of this, because, you know, these long kind of takes, these thoughtful kind of takes on long issues don't lend themselves particularly well to this format. If you haven't had a chance to read Richard's piece, and I know most of you haven't, I really urge you.  It's worth the time.  I mean, I think the depth of the reporting and the discussion, it's hard to convey in this format and hopefully we'll get to a bit of it in the Q&A. But it's the kind of piece that I think is a really important read but difficult to really summarize in 15 minutes.  Don, please. MR. McADAMS:  Well, first of all, I want to say that I do really love philanthropists, especially those that see the value of the work we do at the Center for Reform of School Systems and our biggest funder and partner, the Broad Foundation, is well-represented today.  So, good morning Dan Katzir. Fortunately, this paper does not require us to say anything about the Broad Foundation. A second point I want to make is that you'll notice that this paper was written by Lynn Jenkins and Don McAdams, and the reason her name is first is not just because that's alphabetical.  She is really the primary author of this paper and I had a bit of a discussion with Rick about why couldn't we both sit up here, and he said there was not room at the table.  But, really, there is another chair there, Rick. Notwithstanding that, we sort of flipped a coin and I lost, so I have to make the presentation this morning, but when the Q&A comes, Lynn Jenkins there, raise your hand, Lynn, is going to help with a lot of the answers. We were asked by Rick to survey three districts, or four--I don't know that he gave us specific directions--but three seemed about right--to actually examine philanthropy in those districts. You know, who gave money, how much, what for?  Did it make any difference?  What did the philanthropists think about it?  What did the school district think about it?  Did it have any impact on the direction of the district? Was it just sort of sucked up and left no trace, or did it actually have some impact on moving the district in one direction or the other? Did it really help the district execute a clear theory of action, or was it really irrelevant to the district's theory of action, or maybe the district didn't even have a theory of action. So with this kind of overall guidance in mind, we selected three districts, and we selected Charlotte/Mecklinberg, Houston and San Diego.  The reason we selected these is they're all large.  These are big districts.  They are all districts that have received a lot of national visibility because of the reform work in those districts. Houston won the first Broad Prize,  Charlotte/Mecklinberg has been a finalist for the Broad Prize, the work of Alan Bersin in San Diego has been well-known and documented quite a bit, and the improvement in achievement, particularly at the elementary school, has been significant. So we thought these are big districts that would be worth looking at.  Also, they are on the East Coast, sort of the center of the country, and on the West, and also these districts have different theories of action for change. So it gave us an opportunity to look and see, you know, how philanthropic support played into those theories of action for change. Our methodology was fairly simple.  This is not an advanced piece of social science research.  We opened the door with the superintendents, got their permission to ask a lot of questions.  We designed an initial survey.  We followed up the survey with some fairly intensive telephone interviews and then also got ahold of all the documents we could from these districts about these grants.  There are some fairly interesting findings, I think.  One of the first is, is that if you try to find this information out about a district, it's hard to find out. Because most districts, in fact none of the three that we talked to, could readily tell you who's in charge, where the money came from, where it went, how it was evaluated, and so forth. There was sort of a, you had to sort of dig and poke around.  I think San Diego was the most clear on that.  I think maybe Charlotte/Mecklinberg was maybe the least.  Trying to just find out who knows about philanthropy in your district.  Well, some of it is known by this, you know, department, and some of it is known by this department. Also, if you try to find out exactly how much money was actually given, that's almost impossible to find out because the money comes from a lot of different sources and a lot of the gifts are small, and really are not focused on anything particular at all.  They're gifts to schools, you know, PTL fund-raisers and whatever. So what we tried to do was focus on the significant grants, on the biggest grants that we thought were the most important to the district, the most important to the, you know, major donors, and there was some specific point around this. So that's one finding.  It's sort of hard to actually find out what's going on. Another finding is that the school districts, only in the case of San Diego did the school districts actually have a strategic intent. In the case of the other two districts, it was really the grantors, rather than the recipients, that were interested in the grant.  That is, people came to Charlotte/Mecklinberg, the Ford Foundation, with an idea that some money would be useful for the district, particularly building collaborative community involvement in the district. This was the biggest grant that we examined here.  By the way, the total money to CMS is much, much smaller, measuring in the, you know, 8, 9, $10 million at most, whereas in Houston, if we would look at all of it, it would probably be over a 100 million, and in San Diego, forty plus.  So quite a differential in the amount of money. But in Charlotte/Mecklinberg, it was really people coming to the district with money, suggesting, wouldn't you like some money to do this or to do that.  In Charlotte/Mecklinberg, there have been a lot of problems around--this is a county district as well as a city district. It's sort of a, you know, sort of a donut with a suburban ring around it, and they have a very longstanding desegregation battle that has divided the community and is still, as we speak, dividing the community. So a grant for a collaborative working with the community sounded like a good idea.  A lot of partners, major players, both statewide and locally in Charlotte. This process actually sort a fell apart.  The partners were not being consulted, the district was actually taking it over and managing it directly and after a fairly short time the whole grant was focused around teacher retention. So I don't think--although teacher retention could be seen as one of the important points that the district was driving toward, you'd have a hard time saying that this grant really achieved its purpose or was really part of any strategic intent in the district. The second major source of funding we looked at was really a local education fund that was not started by the district, that was started by local people who first--this is the Charlotte Advocates For Education. Their idea was that they would become sort of a conduit to raise large amounts of local money and then distribute it in particular amounts at strategic points within the district. But it turned out they discovered that they had virtually no impact on the district, and rather than do that, they shifted their purpose to really become more of an advocate around public policy and, you know, trying to shape district policy rather than to fund district activities. The third focus in Charlotte/Mecklinberg was the establishment of the district foundation.  Again the district continuously pointing out how it needed more money.  Some of the business leaders said, well, why don't you, you know, form a foundation and see if you can raise money to supplement what the taxpayers are providing. So they do have a fairly active foundation in Charlotte now, which is quite effective in soliciting money, and lots of donors, moderate to small amounts, and here the plan is simply to match the interests of donors with the needs of specific schools. So I think you should have to conclude that in Charlotte/Mecklinberg, philanthropy, although it's been very helpful, and people have been glad to receive the money, and it may have done some good, there's not evidence that it's had any big impact on the district, one way or the other. The theory of action for change, by the way, in Charlotte/Mecklinberg, has really been an implicit managed instruction theory of action.  It was not during this period--this is Eric Smith and Jim Pughsley [ph].  It was not, and I don't think to this day, has yet been really clearly articulated for the public, where there's clear focus on what it is. But it is clearly a managed instruction theory of action and they have pursued this quite effectively. By the way, one of the other reasons we selected these three districts is because of comparative stability.  In CMS we have two superintendents back to back, you know, going down the same path.  So we have quite a long period of time where there is stability at the top. In the case of Houston, also, there has been stability.  Paige had a long tenure as a superintendent by the standards of urban districts, and his successor, Kay Stripling, again was following in the same path.  So we have another opportunity to look at a fairly stable situation. In Houston, we're talking about large amounts of money.  The Annenberg Challenge, which became known as the Houston A Plus Challenge, actually counting all monies raised, is around $60 million.  Project Grad which started in Houston--most of you are probably familiar with Jim Kettelson and his work.  I think it's now in seven feeder patterns in Houston and it's also expanded to some other cities.  These are very large amounts of money. In the case of Project Grad, Houston was expected to contribute funds toward it and ended up contributing, I think currently, about 19 percent of the total budget for Project Grad.  Project Grad does in fact cost a bit more.  They're very conscious of being cost-effective, I think, and the per pupil cost per year, it's about 5 to 7 percent above what the district would ordinarily be spending on those children. And I'm not going to go into the actual work of these two big interventions.  I think most of you are fairly familiar with this. The Houston A Plus Challenge really revolved around professional development.  That was the primary focus there.  Project Grad is bringing together a focus on reading and math instruction, along with classroom management and social services with, you know, scholarships for the young people who finish high school and go on to college.  These programs are huge. Both of these were not strategically designed by the district and then let's go get money for it.  These are people that came to the district and said we could do this for you, would you like us to do it? and of course the district said yes. If you try to evaluate the effectiveness of these programs, that is a different issue and we didn't try to get into that.  There's been a huge amount of money and research and everything else around these evaluations, and we're not prepared to go in here and redo all that evaluation. We have looked at what the evaluators have said.  You can believe them or disbelieve them; there's a lot of discussion around that.  One of the things we noticed interesting about the evaluation of the Houston A Plus Challenge is that a very expensive evaluation, a very thorough, comprehensive, a lot of evaluators led by a professor at the University of Texas, Pedro Reyes--if you look at the report, the verbiage is glowing. The pictures, the language, the quotes from the participants.  But if you look at the numbers in the actual report, they don't indicate that the schools participating in the Houston A Plus challenge have improved achievement any faster than the other schools in the district. In fact, in some cases, the achievement has not improved as fast as it has in other schools in the district. And of course it would be almost impossible to separate out the impact of the Houston A Plus Challenge work with the other things that HISD was doing for those same schools, because they were still very much integrally a part of the district. So notwithstanding the very glowing rhetoric, the point Rick made earlier about the people who do these evaluations--I only have three minutes left and I've got to rush.  It seems like a long time until you're doing this yourself and then it doesn't seem like it's nearly enough. With Project Grad, there's been a book written on Project Grad.  There has been demonstrable improvements in the districts where Project Grad has been involved.  On the other hand, the Houston Livestock and Rodeo, which has a big intervention in reading, which is one of the ones we did not discuss in this paper, point out that their improvements are even greater than those in the reading area that are done by Project Grad, because Project Grad uses a reading program, Success For All, that the Houston Livestock and Rodeo people don't think is as effective as the reading program they use. But the interesting point is, and I think it's Rick's point, earlier on, is that when it comes to evaluating these programs, the district certainly has no incentive to say that these programs have not been as effective as they could have been.  The third party, that is funding these, the Annenberg Foundation, the Brown Foundation, the Houston Endowment, and others, they certainly have no incentive to point out that the program might not be as effective as they hope, and certainly, the people at the Project Grad office of the Houston A Plus Office, like all good nonprofits, are very proud of what they've achieved and want everyone to notice their achievements. So there really hasn't been a critical eye on these programs.  But we're not able to evaluate and we didn't try to do that.  But we did look at the people who had evaluated them. By the way, Houston also has a foundation and there's huge amounts of money, now, that come in from a wide variety of sources but they do not have strategic intent. The interesting point about Houston is that because Houston's theory of action is really more a balance of performance with empowerment, which allows a lot of variety in the district and how things go, these grants were more favorably received and became more deeply embedded in the district.  And I want to come back to that in the 30 seconds at the end. San Diego is different.  In San Diego, there is clear strategic intent.  In San Diego, the superintendent, the district, they go out and they seek money to carry out a specific objective.  The blueprint.  The $40 million.  And the testimony of the participants is that without that money, it would have been very difficult to do this. This was money at the margin.  The budget for San Diego is huge but this money--and there was a lot of shifting of money internally in San Diego to focus more money on their--and by the way, in San Diego, it was a much more explicit managed instruction theory of action, and the strategic intent of the philanthropy was very clearly focused, and the results are very clear. In other words, the money was used for exactly what they said they were going to use it for, and in fact they can measure the results in the performance in the elementary schools where the money was spent, and Rick knows all about the investigation of San Diego, because wearing another hat, he has led a group of researchers who have examined San Diego. The downside in San Diego, and I think I can get this in--the downside in San Diego is the stability issue.  When we started this paper, we had the hope that Alan Bersin would be there, superintendent for several more years, and that the board would solidify around the blueprint and that the next superintendent in San Diego would be moving down this same path. I don't know at this point that anyone can predict, for sure, what will happen in San Diego.  And this maybe leads to some quick conclusions that I can make, and one is that we believe that philanthropy to districts is going to be far more effective if it supports a strategic intent. If the district does not have a strategic intent, it's pretty high risk to give them money.  We also believe that the money should be given at the margin.  That is, it should provide the extra tipping point to enable the district to do what it is already committed to do and wants to do and is putting some of its own money into. We also think that stable leadership is important, and this is a real problem because local politics always trump philanthropy, and local politics is huge.  So although you may try to pick a district where there is appearance of stability, you can't be sure that the stability will continue. And then the last point, since I'm out of time, is the importance of broad and deep local buy-in.  Because if you're going to have district stability, you're going to have to have that broad and deep buy-in. And, in Houston, there was much broader and deeper buy-in to the district's theory of action and consequently, changing superintendents have not affected this flow of cash and the value it's bringing. In the case of San Diego, where there was strong strategic intent, the buy-in was not broad enough or deep enough to guarantee that this will be continued.  So we would leave these as three points for philanthropists to consider.  Strategic intent, stable leadership, and broad support of the initiative. MR. HESS:  Thanks, Don.  And Tom. MR. LOVELESS:  Don't start the clock yet. [Start tape side 1B.] MR. HESS:  [in progress] we asked Tom to do a survey of education grants officers at, you know, a large number of foundations in K-12 education, and as far as I've been able to tell, it hadn't been done before, so Tom pretty much had to build this thing from scratch. MR. LOVELESS:  Thank you, Rick, and I'm delighted to be here.  I know everyone says that but at 7:45 this morning, I called Rick on my cellphone to let him know that the bus I was on had broken down, and I was commuting in from Reston. And then I called Rick at 8:15 to let him know that the bus still was broken down.  So after dawning some bib overalls and crawling under the bus, I managed to get here.  So I'm very happy to be here. The title of my paper is how program officers at education philanthropies view education.  It is inspired by a 1997 study of education professors done by the Public Agenda Foundation in New York City, and in that particular survey, what Public Agenda found was, and I quote from the study: "Professors of education have a distinct, perhaps even singular prescription for what good teachers should do, one that differs markedly from that of most parents and taxpayers." The survey generally found that education professors are outside the mainstream, and by the mainstream, I don't mean that as a pejorative term.  I mean that as reflecting the majority views of the general public, K-12 teachers, even students in some cases, and certainly parents. The study concluded with the following.  Quote.  "While the public's priorities are discipline, basic skills and good behavior in the classroom, teachers of teachers severely downplay such goals." So what I decided to do was give the same survey to program officers at education foundations, and in a nutshell, what I found was that they too are far outside the mainstream, on some issues even farther outside the mainstream than education professors. Basically in terms of methods, I'll just tell you briefly, we surveyed 240 education program officers at philanthropies.  These are K-12 giving philanthropies.  We received a response rate of about 53 percent, 128 program officers, and now I'll show you some of the data. First of all, we asked some background questions.  How many years at the foundation?  Education degrees.  You see about a fifth have an education degree.  40 percent of the officers have no teaching or administrative experience.  26.9 percent have served as administrators, and over half, 56 percent, have some kind of teaching experience, with most of that being in public schools. You'll notice the public school and private school, that sum's over a hundred because it's not discrete.  But 80 percent of the program officers who have teaching experience taught in public schools and they taught for an average of six years.  So there's quite a bit of experience there. Now we asked a series of questions in this first cluster about teachers.  What are important characteristics of teachers?  And in the paper, by the way, you'll find many more questions.  I've truncated this for the purpose of presentation.  You may have some trouble reading it. The point that was highlighted in the public agenda report was, if you take a look at the second row, this is the percentage responding that these are absolutely essential characteristics.  In the second row, teachers committed to teaching kid to be active learners who know how to learn.  This is endorsed by both program officers but if you compare that to teachers who are deeply knowledgeable about the content, the specific subjects they will be teaching, it's quite a bit less support for that as being absolutely essential.  47 percent on the part of program officers.  That's statistically significantly different from the 57 percent of education professors who think that's absolutely essential. Take a look at the bottom four characteristics of teachers, and I think this is very important.  Teachers trained in pragmatic issues of running a classroom, such as managing time and preparing lesson plans, only 27 percent of the program officers felt that was absolutely essential.  41 percent of education professors thought so. The bottom three all have to do with enforcing rules, and this is a pattern that we see throughout this survey.  Teachers who maintain discipline and order in the classroom.  I'm an old sixth grade teacher, so I would have ranked this quite a bit higher, I have to tell you.  18 percent of the program officers felt this was absolutely essential characteristic.  37 percent, twice as many of education professors.  Teachers who expect students to be neat, on time and polite, you can see support is dropping off in both groups.  Only 14 percent. And teachers who stress correct spelling, grammar and punctuation, that was only seen as absolutely essential by 12 percent. In the next table, again I'd like to focus just at the bottom end of the table.  You can see in these bottom five--let's call it six different classroom activities.  These are sort of mainstays of progressive education.  These are thing that come under criticism by progressives over the last 100 years. Take a look.  Should kids be given--this is the percentage that responded that more of this would be a good thing.  Should kids be given more homework assignments?  Only 21 percent of the program officers felt that they should be given more homework. 41 percent.  The education professors are tougher on homework.  Penalties for students who break the rules.  Only 19 percent of the program officers.  37 percent, again about twice as many of education professors.  And in a minute I'm going to show you what the general thinks about these things. The title of the public agenda report was Different Drummers.  The program officers at the philanthropies appeared to be even more different than the different drummers, at least on issues of discipline and basic skills.  Those are the two main differences. Memorization, endorsed by only 11 percent of the program officers.  Prizes to reward good behavior in the classroom.  This is Alfie Cohn's [ph] big problem, he has a big problem with that.  Only 11 percent.  And then multiple choice exams, not popular at all. Perceptions of the nation's schools.  I'm actually going to pass over this in the interest of time and go right to table five, school reform, and all of these I think are quite illuminating. These are ideas for changing the way public schools teach.  We gave the respondents a one to five Likert scale, so five means they certainly endorse it.  One means they reject the idea.  Not allowing kids to graduate from high school unless they clearly demonstrate they can write and speak English well.  Program officers endorse that, 62 percent.  But education professors even more so, and the public and K-12 teachers are nearly unanimous.  I mean, they're over 80 percent.  They overwhelmingly endorse the idea of not allowing kids to graduate who can't speak and write English well. Emphasizing such work habits as being on time, dependable and disciplined, 49 percent of program officers endorse that.  But look at the numbers here for the general public.  K-12 teachers, 93 percent.  The general public, 88 percent.  Raising the standards of promotion.  That's very much like this.  So social promotion. A huge gap here, only 34 percent of program officers finding social promotion to be a problem. And then the bottom two.  Both again deal with discipline.  Permanently removing from school grounds kids who are caught with drugs or with weapons.  That's endorsed by only 32 percent of program officers.  K-12 teachers, 84 percent.  66 percent of education professors.  76 percent of the general public.  That's a very popular idea. If you look at the bottom row, taking persistent troublemakers out of classes so that teachers can concentrate on the kids who want to learn, only 30 percent of program officers rated that highest.  Two-thirds of education professors and overwhelming support, again, from K-12 teachers and the general public. On table six we asked a series of questions about the program officers own community schools, because sometimes you get different answers when you ask about one's own community schools versus sort of schools in the abstract, and we do see some differences here that emerge between program officers and education professors. They look different on questions that I would call the big arguments that have been in the newspaper over the last 30 years, let's say, where public schools were attacked and people defend the public schools.  You often hear defenses such as-- this is not the slide that I'm talking about, so let me stop right there and change gears here. That was the slide that I skipped, actually, that I was going to tell you about.  But let's take a look at this one. Again, on the discipline issue, there are too much drugs and violence in the schools.  61 percent think this is a very serious problem.  61 percent of program officers.  Among the general public it's higher.  Schools don't teach kids good work habits, such as being on time to class and completing assignments, again, sort of a basic skills issue.  Only 41 percent of program officers see that as a serious problem.  It's seen as a serious problem by the general public, by 52 percent. And these bottom two are fascinating, they deal, especially if you compare them--Public Agenda served three different groups of parents--white parents, black parents, and self-identified traditional Christian parents. Look at the question on schools fail to teach religious values.  If you see that as a serious problem or not.  Only 6 percent of program officers think that's a problem.  Among traditional Christian parents, not surprisingly, 70 percent see as a problem. But even in the general public, almost half, 47 percent, think it's a serious problem. And then on the bottom row, schools are too graphic and explicit when teaching sex education, only 5 percent of program officers see that as a serious problem.  24 percent of the general public, a minority, but about a fourth. Now let me make one point.  Program officers are not monolithic in their responses.  They're not homogenous on these different views.  What we did is we took a look at the three big clusters where there appeared to be out of the mainstream, that being discipline, basic skills and  work habits and student accountability, and we took the questions out, pulled them out, and just added up.  These are all on Likert scales, what are known as Likert scales. So we basically added up the responses and we took a look at program officers who had teaching experience, and their responses, and compared them to those who had no teaching experiences. And this is really quite interesting.  The program officers with teaching experience are closer to the mainstream public views.  Now they don't get philanthropies all the way into the mainstream but they do close the gap, five to eight points. So we looked at discipline, work habits and student accountability.  Let me show you the data on discipline and work habits. Now you'll recall these questions, we've gone over or we went over most of them.  One very single one, the teachers with teaching experience are more likely to endorse stronger discipline than the program officers who had no teaching experience. So, for instance, on permanently removing from school grounds kids who are caught with drugs or with weapons, you recall that, the mean response of those with teaching experience was 3.23.  They were more likely to endorse that than those without teaching experience at all. Taking persistent troublemakers--and you can see the P values here are huge.  These are really very large differences in terms of any statistical test. Taking persistent troublemakers out of class, that teachers can concentrate on the kids who want to learn.  The program officers with teaching experience were more likely to endorse that compared to those without any teaching experience. There was a question on too many teachers are more interested in being popular than requiring respect and discipline.  Again more likely to endorse that. Teachers who maintain discipline and order in the classroom.  Recall, that was in that first table, about is this an absolutely essentially characteristic of good teachers?  Those with teaching experience believe it is.  Those without teaching experience are less convinced of that. Finally, let's look at basic skills and work habits, again comparing those with teaching experience and those without.  We see significant differences here in the top two rows.  They don't meet statistical tests after that on these bottom ones.  However, on every single question the trend all goes in the same direction.  More towards the mainstream for program officers who had teaching experience.  More towards the mainstream, closing that gap between the public, K-12 teachers, parents, and philanthropic program officers. And you can see on emphasizing such work habits as being on time, dependable and disciplined, the mean response was 3.85 for those with teaching experience, only 3.25 with no experience.  That really is a very large difference because we have some pretty robust samples here. There's not enough emphasis on the basics such as reading, writing and math.  2.57 was the main response for those with teaching experience.  No experience, 2.11. And the point on the basic skills and the emphasis on basics.  Basics almost seem to be a codeword.  If you asked about high standards, the program officers endorsed those positions.  But when you asked about basics, that seemed to be the key word that they didn't particularly like. So I have basically a very short summary and recommendations.  The summary is the program officers diverge from mainstream opinion on three issues, I think the three critical issues--discipline, basic skills, and by basic skills I also mean work habits, and student accountability. And then my recommendations for foundations seeking compatibility with mainstream opinion--and by the way, there are two things that could happen here in terms of what philanthropies could do with this data. They could take the position, number one, we know we're out of the mainstream, we want to reform the public view of education.  We want to reform how parents view education.  We want to reform how K through 12 teachers view education. Well, that could be but then that's really an extraordinary task, then, that philanthropies are going to take on to change all of the major stakeholders in education views of education at the same time that they try to change schools.  That's going to be a rather daunting endeavor. But if they want to get back to the mainstream and they want to promote reforms that are more likely to grab hold in schools because they have parents and kids.  Even kids by the way.  Public Agenda did a survey of high school students.  They come out very much like parents and the general public on these issues. They, for instance, endorse getting rid of troublemakers in classrooms because they disrupt their learning environment.  But I have three recommendations.  One is you could hire program officers with teaching experience.  Now this won't close the gap completely but it will get some pragmatism and it will get philanthropies back towards mainstream public views on this. Number two is to hire a staff, make sure in philanthropies that have staffs with balanced points of view.  Make sure you have traditionalists and nonprogressives on your staff, that you have people who believe in discipline and basic skills. This will allow perhaps some fruitful discussions to take place within the philanthropy. And then third, appoint an outside review committee to evaluate your grant portfolio.  Make sure that you've got parents, K-12 teachers, taxpayers, and all kinds of people who annually sit down and take a look at your portfolio and can give you feedback.  You may not want to hear the feedback.  A.D. Hersh [ph] calls this a worldview, when he's talking about education professors.  You may no want to hear the feedback but I think, in the end, it's going to benefit both you and the future of school reform.  Thank you. MR. HESS:  Thanks, Tom.  With that, we're going to open it up for questions.  We have about 15 to 20 minutes to take questions.  Morgan Gotley [ph], my assistant here, will be coming with the microphone.  Please catch Morgan's eye.  I'd ask that you please identify yourself by name and affiliation, just since we don't all know each other here, and then again, I will please encourage you to ask questions, or, you know, preferably ask real questions.  Please don't indulge in that famed D.C. habit of a speech followed by a question mark. Do we have any questions?  Yes, sir? QUESTION:  Hi.  My question sort of is which side are you on, or actually, which side are the philanthropists on in regard to the education wars that are being fought across this country, often under the name of the reading wars or the math wars.  You know, to what extent are they facilitating the reform agenda and to what extent are they facilitating maybe the opposite. MR. HESS:  Okay; thank you. MR. LOVELESS:  I don't mind taking a stab at that.  For the most part, they're on the neoprogressive side, which in the--I edited a book, a couple years back, called The Great Curriculum Debate, and it's about the wars in both math and reading that have occurred over the last 15 years, whole language versus phonics, and in math, math reform or NCT and math reform versus more basic emphasis on arithmetic and other traditional mathematics. And for the most part, the philanthropies have supported, financially, the neoprogressive side. MR. McADAMS:  Well, just sticking to the evidence in the paper that Lynn and I wrote, the Houston A Plus, which is the single biggest amount of money in any of the three cities we looked at, is clearly progressive.  The professional development for the teachers in all of these schools definitely had a strong progressive bent. MR.      :  Could I add one real quick comment.  One debate you didn't mention is choice in competition and I was struck by the wide, wide range of foundations that are donating, in various ways, to supporting additional choice.  I really think that the charter school movement really wouldn't exist without a wide range of, you know, foundations that are identified as conservative but also foundations that seem to be identified as progressive as well. MR. HESS:  And Bryan's paper will delve exactly in the question. MR.     :  And Rick, if I can add one more thing.  Just a historical note.  I begin the paper talking about two very famous examples of philanthropy, the Ford efforts in the late '50s and through the '60s and early '70s, and Annenberg, and both those supported, generally in terms of curriculum and instruction, what are known as progressive, student-centered interdisciplinary, hands-on, manipulatives over the use of text.  Teachers as coaches as opposed to didactic, mimetic instructors.  These kinds of approaches are mainstays of progressivism, and for the most part, those two big philanthropic efforts support a progressive curriculum instruction. MR. HASSEL:  Don, you talk-- MR. HESS:  Bryan, identify yourself, please. MR. HASSEL:  Bryan Hassel from Public Impact.  Don, you mention the importance of strategic intent but also the importance of buy-in, and these two are often in tension with each other and I think the San Diego story is a good example of that. What do you think are the prospects for districts and philanthropies that are supporting them, both having strong strategic intent and being able to get a wide range of people to go along with that? Mr. McADAMS:  Well, that's a good question.  That is why it's so difficult.  In San Diego there's clear strategic intent and the philanthropy is aligned with that strategic intent, but perhaps for that very reason it is more difficult to get the broad community buy-in. In Houston, there is a clear strategic intent but the theory of action does in fact support a sort of performance-based empowerment model where there is a very sharp focus on results but there are multiple paths allowed to achieve those results, and for that reason it's easier to get a lot of buy-in and a broader base of support. So I think the trick is for philanthropists, is to find sort of the right balance there and I think it is going to be difficult to do, and in fact I think for philanthropists who are investing in school districts, which is maybe a little different from a lot of other places you can put money to support K-12 change. For philanthropists who are investing in school districts, I think this is a very difficult job to do and really think you're making a low-risk investment.  I think it's going to be a pretty high-risk investment. MR. HESS [?]:  Well, Tom, I mean, doesn't your stuff speak directly to this question of the likelihood of achieving buy-in, given some of the premises for the kind of inside the classroom door? MR. LOVELESS:  Right.  If by buy-in we're talking about parents, teachers and the general public, and those are three pretty important groups when it comes to implementing school reform, it would indicate that the--if you begin from a philosophical basis, that is, directly opposed to what those three groups believe in, you're in for some "rough sledding." I would think that these data that I just presented, what they trigger in my mind is at least the hypothesis, that some of the disappointment that many people have over philanthropies being marginalized throughout the 20th Century may be due to the fact, not that they're implementing ideas wrong but that they're trying to implement the wrong ideas. Not that they have poor or faulty theories of change, but that the change that they're endorsing is not something that other people support. QUESTION:  Mike Lieberman with the Education Policy Institute.  It seems to me your findings would only be expected, that is, that people that have to wrestle with the consequences of a policy are going to have a different view than those who don't have.  To me, that would explain why, for example, people with practical experience are going to have--my question is this. The suggestions you made at the end are really up to the foundations, whereas, is there anything that can be done, whether they like it or not, to make them more accountable? MR. LOVELESS [?]:  Well, I think they can do some things. QUESTION:  No; no.  Not their option.  Is there anything that anybody else can do to make foundations more accountable, whether they want that burden or not? MR. LOVELESS:  I don't know, Mike.  That's really outside the scope of both my paper and my own thinking. QUESTION:  I just had a quick methodological question for you, Tom, which is-- MR. HESS:  Kim, identify yourself, please. QUESTION:  --how did you decide--Hi.  I'm Kim Smith with New Schools Venture Fund.  I just had a quick methodological question for you, Tom, which is how did you select the program officers that you surveyed, because Dan and I were talking about the fact that, to the best of our knowledge, no one in either of our firms was questioned, for instance, which led me to wonder if perhaps the people who were selected may have had something to do with the sort of pretty big disparity there, because certainly we know that we're funding a number of things that would not fall under that category. So I'm just curious, how you thought about how representative the group was that you surveyed? MR. LOVELESS:  We surveyed all the philanthropies we could identify that had given any money, that had anything to do with K-12 education. QUESTION:  [inaudible]. MR. LOVELESS:  I'm quite certain that both of your philanthropies are on our list.  Now whether or not--we did three waves and very often surveys don't get to the people--it was a mail survey--for whom they're intended.  But we did-- QUESTION:  [inaudible]. MR. LOVELESS:  Yes.  U.S. Postal Service, which sometimes works as effectively as the Fairfax County bus line, I realize.  However, let me say this.  That even with 240 surveys, which is what we mailed and 120 something coming back, there's no way on Earth that I could have cherrypicked, I could have found out--here are the progressives, I want to make sure, you know, they get special delivery on the survey. So I think the survey is representative, but again, like any self-selected survey, in responses you have to be somewhat wary.  The one thing I think that makes self-selection less a problem here in terms of philosophy is that there are statements that are very conservative and there are statements that are very progressive.  There are usually counter-opposing statements on both sides of any given topic. So, in order, for instance, for the program officers to show that they really did not believe in strong discipline, they had to endorse it from both when it was negatively stated and positively stated. So I don't think there would have been self-selection on the basis of philosophy. MR. HESS:  Tom, would you make the point also about mail versus telephone survey, the public agenda methods that you-- MR. LOVELESS:  On one of the tables, and I've forgotten the number now, but if you read the paper, there is one table that troubled me a little bit because the education professors all seemed more emphatic, even when some of the statements seemed contradictory, than the program officers, and so I was still getting these large statistical differences between the two groups. There's a very small literature on the difference in responses between telephone surveys, which is what the education professors were given, and a mail survey, and that literature says that when you're in a focus group or you personally talk to a survey person, you're being surveyed, you're more likely to give extreme responses, either positive or negative, and so my responses may look more like the mushy middle in terms of the education program officers. MR.     :  Which suggests, if anything, that the bias is actually understating the strength of your findings. MR. LOVELESS:  Except for one table. QUESTION:  Giselle Hoff [ph], the Jac..[?] Foundation in San Francisco.  Tom, in your conclusion, you mention that one of the ways that you recommended that foundations could look at evaluating their portfolio.  I've heard a great deal among my colleagues about evaluating program by surveying grantees and sharing information. I'm not familiar with that particular scenario.  Could you elaborate on that.  I mean, there's so much getting away from donor intent. There's so much of a disparity between trustees and what staff people are doing.  How do you put something like that into effect? MR. LOVELESS:  Well, I don't think it has to be determinative.  I don't think you need to appoint a body that somehow acts like a Supreme Court.  But you might appoint a review committee.  What I was thinking of is a review committee that represents a cross-section of people who have, like what Mike Lieberman said, very pragmatic interests in education, that can give you feedback on--look, you've gone off a cliff here. I mean, we think it's great that you're trying to help a lot of kids but you really are way outside of what most people want.  And it would just simply give you that feedback. You could then go back to your board or your founders, if your founders are accessible, and explain to them, here's the feedback we're getting on our basic thrust of what we're doing.  So that's what I had in mind.  Now I don't know enough--you're smiling, so maybe that's not practical do that but that's the best I could come up with. QUESTION:  Debra England with the Corrette [ph] Foundation.  Tom, I hate to be critical, so this is a critical inquiry, though, and I'm wondering if you set up a bit of a false dichotomy, because you talked about your recommendation based on the division of the research that you put out, did they have teaching experience or not. But in my experience as program officer in charge of education, where I see a dramatic dividing line is between people who have nonprofit experience and people who come out of the for-profit sector. I come out of the for-profit sector, I used to be in management strategy consulting, and I think we have a very sensible portfolio. Rick, I hope you agree since we fund you, and we fund-- MR. HESS:  We're all a little bit pregnant in this room. QUESTION:  We fund a lot of people in this room.  But my concern about the way that you divided it out is that--I'm not sure that that's really the dividing line that makes a huge difference between how funding is approached and I wonder if you could comment on that. MR. LOVELESS:  I think that's a very good point and you may be exactly right.  We didn't anticipate any kind of difference in opinions between those coming out of for profit or nonprofit, so we didn't ask about that.  So I just don't have the background information on that. QUESTION:  Can I just follow up with a brief point on that which I think one of the large distinctions that I see out in the playing field between program officers in education versus donors and donor intent, is that Bill Gates, Michael Dow [?], the Hewletts, the Packards, the Correttes--all these people come out--Eli Broad--every one comes out of the for-profit sector, and I think if there were more program officers coming out of that arena, that we might have very different ideas happening in the funding area. MR. HESS:  Richard-- MR. LOVELESS:  Well, Annenberg and Ford both did, and again, the basic mission of those two projects was right down the line of what I presented here. Mr. COLVIN:  Yeah, I think that observation that that's a big factor that's new in the world of philanthropy is absolutely right.  That a lot of the folks that I talk to do come out of business and do feel like their work in this area is more along the line of investments as opposed to giving gifts, and so they act very differently in terms of monitoring the effectiveness of those investments and also have a different view in terms of the role of the donor to build the capacity of the organization that they're funding, both in terms of their knowledge about education issues but also their kind of management of the grant. MR.      :  Rick, do you mind if I ask Tom a question real quickly? MR. HESS:  Sure. MR.      :  It's actually not so much a question, but one of the efforts that I looked at pretty closely in the paper is--and this is a business initiative out of Jim Barksdale's 100 million dollars to a very small group of very low-performing schools in Mississippi, and they, you know, were going to work on reading issues, and their reading model was very much a research-based National Research Council kind of approach that really focuses on all the elements, I mean, not identifiably whole language or phonics, really quite a good research-based model. They found it impossible to get the universities to buy into that model and to use it in their preparation of teachers, and eventually pulled all their money out of the universities where they were funding a bunch of professors to try to push that model. Moreover, they found it impossible to--not impossible, but very, very difficult to change the beliefs of the teachers that they knew how to teach reading, that there is this outsider coming in, as far as they were concerned, a lot of them had been teaching for 20 years, they knew how to teach reading, it was whole group, you read, you know, you read to kids, they learn, they don't.  It was really all about poverty.  You know, it really wasn't about their instruction. So I'm just wondering, you know, if the belief system is there, one way or another, how do you get past these sort of institutional, this institutional resistance, either universities or schools, to any change in terms of these pedagogues in terms of teaching? MR. LOVELESS:  Yeah.  I don't know.  [Laughter.] MR. HESS:  We have time for one more briefing question.  Right here.  It was a hellava question. QUESTION:  Yeah, and a hellava answer. I'm Andy Calkins [ph] from Mass Inside Education in Boston.  This is a question, quickly, for Tom and Don.  You've done really interesting pieces of work here, that leave me hungry for a third that combines the two. Tom, your research suggests that education program officers may be in a certain camp on certain issues.  Don, you've created a methodology for going in and using somewhat self-reported data, but also getting beyond that, to look at the numbers in terms of the effectiveness of some of the philanthropic investments. So that leaves me thinking, all right, could you guys construct a project that would combine the two and look at the relative effectiveness and impact on student achievement, using numbers and other means, and dividing it up by the spectrum of belief systems on the part of the foundations, and if you think there is is there a foundation in the room that would want to support that?  Or not? MR. LOVELESS:  As long as there's money for it; sure. MR. McADAMS:  I'd love to work with Tom on anything. MR. LOVELESS:  That's right. MR. HESS:  Let me ask you--we'll come back to this conversation in the discussant panel in about ten minutes, but before we break, let me also make the point, let's keep in mind, is that I really appreciate your questions, I think it's fantastic, and I really us to press these guys, and that's going to make for a much more useful and stronger volume.  At the same time--so please, just keep getting meaner and tougher.  I love it. At the same time, in assessing these efforts, thinking about the contribution, I think it's really important to keep in mind that each of the 11 authors really is pretty much breaking ground and doing kind of the background literature review and whatnot for putting together the book proposal and the research.  It was just astonishing. Outside of somebody like Ella Magaman [ph], the amount of serious attention to thinking about any of these issues in any systematic way has just not been there. So I think what this really suggests is that each of these studies really needs to be understood as kind of a first step and that hopefully there's funders out there, and there's hungry graduate students out there who are going to go and do all this stuff better and embarrass all of us up here today because all we're doing is kind of starting to get the pieces in place to think about this stuff more thoughtfully. With that, let me--Don. MR. McADAMS:  Yeah, because just building on this paper, if somebody wanted to give Lynn, with my support, a couple--you know, 18 months, and, you know, the ability to write a 200-page study, and to dig deeply in ten districts, and then actually go there and really look carefully in the full range of documents, the newspaper clippings, everything--you could take these sort of preliminary conclusions and you could drive them very deeply. MR. HESS:  [inaudible]. QUESTION:  [inaudible]. MR. LOVELESS:  Well, it's not, it's not.  But the main point here is that all--for the next two days you're going to be hearing a lot about strategic thinking and theories of change and implementation.  It's not always implementation that's the problem.  Sometimes it's the idea that's the problem.  And so the point that I'm trying to make is the philosophical basis, the philosophical premises that anyone has when they try to change schools, those are important, they need to be looked at, they need to be scrutinized, and that's what I'm trying to do. MR. HESS:  Okay.  With that, let's take a break.  We're going to reconvene at quarter after with the next panel. [Break.] MR. HESS:  Okay.  I'm going to go ahead and get started.  I know this thing was scheduled tight.  We tried to build in some time between sessions.  It was just one of those things.  We knew a lot of people would be coming in, both locally and from around the country.  There's always a trade-off if you try to set these things up, slowly, over two days, or kind of fully over one.  For various reasons, our sense was that fully over one was going to be most advantageous for everybody involved. So, you know, I'd just ask you to bear with me and we'll just try to keep this thing on schedule, and hopefully we've built in enough time. For instance, we don't have a lunch speaker because I hate the idea of lunch speakers.  So that's going to give you all 45 minutes to kind of have some of these conversations and catch up. With that, our second panel of the day is going to share some thoughts and reflections on the landscape issues touched upon by Tom and Don and Richard.  You know, one of the pleasures for me putting something together like this is that it's hard for me to think of four folks who are better suited to touch on these issues, except for maybe some of the other folks in the room or in the later panels. So we always have to make these trade-offs.  But I'm just thrilled at the folks who have agreed to be a part of this today. If you don't know them, closest to me--we're just going to go in this order right down the table--is William Porter, executive director of Grantmakers For Education.  Next to Bill is Lawrence Patrick, the president and CEO of the Black Alliance for Educational Options.  Susan Fuhrman, University of Pennsylvania.  For everybody, there are full bios in the package that you received when you came in, if you're not familiar with any of our participants today. And finally we have Michael Feinberg, co-founder of the Knowledge Is Power Program, KIPP, and superintendent of KIPP, Inc. It was also mentioned to me that I managed, because we forgot to put the mirrors up today, that I coudln't see the folks with hands raised back here.  We'll try to make sure to take that into account, and so not only do you guys have to sit out of the line of sight but at least we won't be rude to boot. With that, Bill, why don't you go ahead and get us started. MR. PORTER:  Great.  Thanks, Rick.  Just a point of context to begin.  As most of you probably know, Grantmakers for Education is a national network of over 200 foundations and corporate giving programs that are working to improve educational outcomes and achievements. Our members work from the early education area, working to universal prekindergarten through K-12 public schools, and on to getting more kids into colleges and successful in college. We calculate that our members probably are giving away somewhere in the range of $1.5 billion a year.  So my responses to the papers that were presented this morning will be speaking from that perspective. And I'll also beg your indulgence.  I'm actually going to read some remarks that I've been working on the last couple of days.  There is so much to reply to in these papers, I didn't want to lose anything. So I will try to vary the inflection of my voice so that it remains somewhat interesting because I know you're going to be listening to a lot of people today. I'm also going to take a page from Tim Smith's book and talk really, really fast because I have a lot to say. So, first, I just want to thank Rick, again, for convening this research project. Foundations don't face the same competitive pressures that nonprofits or private companies face.  They could stay in business regardless of what people think about their work and whether they do a good job or a bad job.  This independence is an asset when it comes to providing leadership or taking risks, but a liability when it comes to assessing results. I strongly agree with Rick that foundations need to better embrace and even seek out criticism of their philanthropic activities since they aren't likely to naturally get candid suggestions from their grantees, and I'm excited about the conversations he's convened today. I hope we have a high-minded, vigorous discussion about what we know about making education philanthropy more effective. It's very easy to find examples of where philanthropy has succeeded brilliantly.  It has fueled the revitalization of urban school systems in San Diego, Boston, Long Beach and Chattanooga, along with school leaders and local leaders in those cities.  It has incubated the idea of standards-based reforms and accountability, thanks to early grants to school districts, to comprehensive school designers, international organizations such as the New Standards Project. It has supported the rapid growth and diffusion of charter schools across the country.  It has created public education funds that in many urban areas provide a counterweight to the cronyism, political posturing, and lack of capacity in too many big city school systems. At the same time, it's also easy to find examples of where philanthropic dollars have made absolutely no difference, of where the impact didn't live up to the foundation's exhortations. And so with examples on both sides, I think the challenge is to avoid choosing sides and providing either a superficial chest thumping about how great philanthropy is, or a superficial screed about all the ways funder stumble, support projects some of us think are dumb, or fail in their efforts.  With at least one exception, I think the papers that we heard from already and that we're going to hear from later today do try to go deeper and move beyond generalities and innuendo. But reading through them, including the three that I was asked to examine most closely, that you've already heard about, I see four larger challenges that will continue to hamper our efforts to have robust conversations about effective education philanthropy, and I'd like to begin my remarks there. The first challenge I'll title Annenberg myopia. Judging by the discussions in all the three papers, if you read them, the Annenberg Challenge seems to continue to dominate and restrict outsiders' view of education philanthropy, as if it is the only meaningful lesson from education philanthropy in the last ten years. I'd say let's move on already. The second challenge is the paucity of meaningful research and even robust data about the effectiveness of our field, and Rick touched on that as he was wrapping up the last session. Perhaps we end up talking so much about the Annenberg Challenge because we have little other concrete information about how our field is doing overall, and because so few foundations are as willing to report on their failures as they are their successes. And so in reading the papers, both the ones we've heard from and the ones that are to be presented, I was struck by how little existing research and data the authors could rely on. We seem to be poking at tiny parts of a larger elephant. A third related challenge is a worry about whether we need to distinguish between national and local funders.  These papers pretty much rely on information gleaned from the 50 largest foundations which is the easiest added access.  But these funders often are national funders, meaning that they have little knowledge or commitment to a specific place. Does this cause them to achieve more or less than regional or local foundations who have chosen to concentrate their know-how, connections and resources in a single community over an extended period of time? Are national or local funders more effective?  The final, the thorniest challenge is finding ways of promoting and gauging effectiveness, and recognize the unique challenges endemic to all of philanthropy, and these include that foundations aren't the doers.  They're at least one step removed from the work, relying on third-party grantees to actually do the heavy lifting. Funders are also working in messy political environments where no one actor can impose a solution.  And also that we value philanthropy in part because it offers seed capital for experimentation and the ability to take risks that can't easily be taken with public dollars, unless it's not always expected to succeed. I think these are challenges that are endemic to philanthropy, that we just need to get smarter as we think through. And so what I want to say with these observations is I think this conference is a well-needed, first tough assessment of our field.  However, I think we also need to recognize its limitations and commit to strengthening and deepening the inquiry after today. So now let me turn to my assignment which is to reply to the papers you heard about. I think Richard Colvin's paper nicely reviews the dilemmas, interests, and motivations of some of the most prominent players in the field. I enjoyed reading it.  If I have a criticism, I think it's that he struggles too much to come up with a way to define the difference between the new prominent philanthropists who have recently entered the field and the old stalwarts. I think it may be a distinction not worth the trouble as there have always, and not just for the last five years, been a mix in how philanthropists approach their work, including those who are willing to fund long term versus those interested in short-term initiatives. Those viewing philanthropy itself as the change agent versus funding change agents drawn from the field.  Those working inside the public education versus pushing it from the outside. Those taking a hands-on approach versus those working from a distance.  Thus, rather than apply these labels of old versus new, I'd argue we need a topology that's finer-grained, that really captures some of these other dimensions, and then couple that with an assessment of which of these approaches lead to greater impact in which situations. A final observation about Richard's paper.  It might be even stronger if you review the full range of entry points philanthropy can use to try and improve public education, in addition to fixing school districts, which is what he mostly writes about. For example, policy change and public [inaudible] is an area where philanthropy has been especially successful, having helped influence dramatic changes in state and federal policies. Two more observations about Richard's comments this morning.  First, from my vantage point, looking at something as rough as the growing number of members in my organization, I observed that while some prominent national funders have shifted gears and moved their grantmaking away from formal K-12 public systems, as Richard mentioned, I actually see more funders entering the field than leaving it. The second point is that Richard doesn't do this in his paper, but there seemed to be a tendency this morning in the discussion to see education philanthropy as monolithic, and I think that's absolutely false. I think in fact one of the challenges in our field is that there is a vast diffusion of grantmaking interest from environmental education to school leadership to whatever, all governed by different values and different donor intent. All right.  Now for Don and Lynn's paper.  As I read it, the key lesson from this paper is the importance of local context and conditions, which can kill even the best-conceived philanthropic strategy. Don didn't say this specifically this morning but his paper largely concludes that private funding in the three districts was helpful in advancing reform. He quotes Christine Campbell, or they quote Christine Campbell as writing "Shrewdly invested, comparatively small amounts of foundation money can leverage major improvement." But they also conclude, and Don said it again this morning: Local politics trump philanthropy, which I would rephrase as context trumps philanthropy.  In a local school district, a foundation's investment for change only succeeds when it is shrewdly placed at a tipping point where enough factors are aligned to give it a chance to work. So overall I found their paper extremely useful. Turning to Tom's paper, let me say at the outset that I'm a big fan of his work and often refer others to his research and analysis, and will continue to do so.  Nonetheless, I disagree strongly with the methodology and conclusions in his papers.  It poses an intriguing hypothesis that education philanthropy may be hampered by the mindsets and opinions of frontline program officers, and then does very little to give us any data that would prove or disprove this hypothesis. I question his conclusion that too much of education philanthropy hasn't accomplished what it set out to do, that program officers are to blame, and that they are to blame because the majority of them are either out of touch or just plain wrong. There are several places where I would question Tom's interpretation of the data.  Rather than sort of nitpick those, I'll share those with Tom separately, and instead focus on three larger, deeper questions that I see with the paper. Kim raised this point but I question whether the survey sample is representative of foundation staff who work in education. We know nothing from the paper about the balance of his respondents from national foundations vesus corporate foundations versus East Coast foundations, for example. Tom consistently writes in his paper, This is what program officers think, but we know nothing about whether this is truly representative of program officers. For all we know, in fact this might explain something, 100 of Tom's 128 responses may have come from program officers at the Ford Foundation.  [Laughter.] I question the usefulness of applying a survey tool designed for education school professors to foundation employees.  While it may be useful to know what professors are teaching and what classroom management strategies teachers say they need, for example, I do not see this information as relevant to program officers doing their jobs.  Which strategies teachers could use to maintain order in their classroom has little relation to any sort of meaningful program or effort a foundation might initiative to help student achievement. And while I respect the work of Public Agenda, I do think this particular survey tilts toward questions that ask only about progressive viewpoints and does not probe what respondents also think about other issues on the spectrum, such as focusing on reading and literacy in the early grades, alternative certification, accountability systems in charter schools. Not probing these issues is a serious oversight since they include issues most foundations are actually working on. Isn't it possible to support both classroom level assessments, which is asked about in the survey, and standardized assessment for school accountability which is not asked in the asssessment?  We'll never know. Last point.  Iquestion the lack of correlation between Tom's findings, even if we take them at face value as accurate, and the role and influence of program officers, which varies dramatically at each foundation. There is no attempt made in the paper to link the supposedly progressive beliefs of program officers to the work of a foundation or the results of any grants made. The paper would be more persuasive if a causal link could be drawn between the beliefs of program officers, the grant requests that get funded or not by that foundation and the impact of the foundation's work in education. The paper also does not make any attempt to investigate whether the views of a program o in education are moderated at all by communities or boards of trustees, which I think they almost always are. Why do these points matter?  I point you to Denver, wehre the senior program officer at the Rose Community Foundation is indeed a former eduacator and probably one with strong progressive education credentials, given that he ran Denver's Expeditionary Learning School for a time. Based on Tom's research, we'd expect the program officer to be pushing grants out the door to fund multidisciplinary curriculum for students who have been kicked out of school. Instead, this program officer and the foundation have been key players in the groundbreaking philanthropic effort to get Denver to pay its teachers based on student performance. So my criticism here is that I think Tom's research doesn't help us understand how foundations choose to make investments in education.  It sheds a lot of heat but not much light. If being effective in education philanthropy was as--if only being effective in education philanthropy was as easy as hiring a program officer who wants to see more orderly classrooms. MR. HESS:  Thank you, Bill. Lawrence. MR. PATRICK:  Hi.  I want to spend a little bit of time--I don't have a prepared statement that I'm going to read really fast but I'm going to try to make a few--that was good, though; that was very good.  I have a few points that I want to make after reading these three papers. The first point is that I think it's pretty clear, in summary, that as a philanthropic community we're not satisfied with the rate of change, number one, but also not satisfied with the way changes have failed to take root and become permanent, especially in large urban districts. I think that's a significant--I mean, it's sort of an obvious thing but I think it is significant to talk about.  And the reason I think it's significant is something that's not mentioned in any of these three papers in a significant way, which is the impact, the politics that race play in grant making and in trying to turn around these behemoth school districts. One of the things that was interesting to me about the survey, the Public Agenda survey, was the very high percentage of respondents who believed that school are the most important democratic institution, and that really struck me because I think the reason we have that view or the reason that people have that perception is because our public education system has really had to bear the brunt of integration for our society. And so we have, I think, a very warped sense or warped view of the actual reality or the actual history of our public education system, and it's not pretty.  The reality is that the schools that exist in most big cities in America are essentially schools that were left behind as white people left the big cities to go to the suburbs. And while you may look at the statistics now and see, you know, lots of racial diversity, the bottom line is that you essentially are talking about trying to fix a system that was never designed to educate children in the system. And it was never a priority to educate those children.  That's a relatively new development.  It's not really fleshed out in these papers but I think it's significant.  I don't think it's fair to start from a premise, that we've been trying forever to educate low-income black children or low-income Latino children.  That's a relatively new concept. And we decided, as a society, that we would use the public education system as the way to integrate our society.  That became almost a larger priority than the education of those children. So even when we sort of opened up the public education system, we didn't open it up to try to teach black kids how to read and write.  We opened it up for the purposes of integrating our society; not the same goal. And so now it's not very surprising when we look, a few decades later, and realize that the system's not doing a very good job of educating black children.  I think that you have to address the historical context and the reality that we have not spent 50 years, although we would like to think we've spent--we haven't spent 50 years--placing a premium on the education of low-income black children.  That's a relatively new concept.  And so I think we have to approach our work with that context, number one, but number two, we have to deal with one of the primary tensions around creating this huge permanent change, which is the politics of race. You cannot escape the fact that in a lot of these big city districts--and when you look at Houston, when you look at really any big city, you look at the racial dynamics that exist, not only in the student population.  I mean, people talk a lot about D.C. and how there are not enough white students in D.C., when you get to high school, to be statistically relevant, which is totally out of sync with the actual population of D.C. But it's interesting that when you actually talk about fixing the school system, that you have to address the actual population of the school system and the difference between that and the surrounding community. You have to look at the politics of race and how they enter into the school board election, and in the selection of superintendents to lead the change.  You have to take into account some of the dynamics around creating change when you have essentially black people who are now in positions of power, I mean they have the formal authority, have the formal power, but are clearly in a system that's filled with black kids and it's not working, you have to deal with the dynamics around how people will respond to outsiders coming in with solutions. I think that you have to address that and philanthropists have to examine the politics of the situation before embarking on these massive efforts to turn around school districts.  Sometimes the politics of race can actually be the biggest factor, more so than having the right ideas or having the right implementation plan. A couple of other points that I wanted to make.  That when we talk about sort of the "new donors," I think the paper does a good job of laying out this sort of new class of donors, and I think one of the things that is missing from the description of the new donors is that part of the reason they have power and voice is because they're celebrities, because they're respected as business leaders. Bill Gates would be very powerful, and his words would resonate, and 45 governors would change their education agenda, even if he didn't have the largest private foundation, because he's viewed as a really smart guy.  He's viewed as one of the most powerful, I mean, he is one of the most powerful business leaders in the world. Our economy basically rests on his ability to keep his company going and doing a good job.  You can say the same thing about Wal-Mart.  I mean, when you have business leaders who are that visible, who have that much profile, I think their celebrity carries with it a certain amount of weight that can be very powerful, almost as powerful as giving money. I think that when we talk about the total amount of money given, it is, and it's a good point, that it's still not a significant amount in terms of the total amount spent on education and taxpayers will still have to bear the brunt of educating our children. Bill said roughly 1.5 billion annually.  I mean, the Detroit public schools budget is bigger than the.  You know, that's just one district.  So yes, it is working around the fringes, but I think one of the ways that it can be more effective, that isn't explored very much in the papers, is around advocacy. It talks a lot about creating great schools and a lot about trying to focus on teaching and learning and implementation and changing governance models.  But not a whole lot on advocacy. Imagine if the amount of money that was spent on creating schools was actually spent to advocate in those local communities for creating permanent change.  I mean, this would be a much more powerful way to think about our activities. The last thing I wanted to mention is that school choice I think is not really explored as a philanthropic tactic.  It's mentioned as a solution to try and improve student achievement but it's not really explored in any depth as a tactic for creating permanent change, which I believe it is.  I mean, in addition to the social justice aspect of school choice, in that it's the right thing to do. It also creates an impetus for permanent change by creating a permanent alteration of the power arrangements that protect the status quo. When you look at the history of giving in these big cities, and you say, well, why didn't it stick?  Why didn't it take root?  Why didn't it take hold?  Cause you had the same power arrangement.  You have the same people in power, the power's divvied up the same way, so the external force coming in to provide extra funding around the fringes to push a specific idea or a specific initiative, even though the initiative or idea is great, it's not going to be permanent because the power ranges are still the same. So it makes sense.  And what we're talking about is dramatic and radical reform.  We're not talking about gradual improvement.  Nobody in this room will be satisfied with gradual improvement.  We've got a gunshot victim.  We don't want to talk about, you know, nutrition or diet.  We have a urgent crisis.  And so radical, dramatic change is going to demand radical and dramatic action. What more radical and dramatic action than giving the person who knows the most about the customer the power to make decisions that directly impact the customer.  The parent knows more about their child than anybody else.  The superintendent doesn't know individual children.  Parents do; teachers do. And I think that what we have to explore is the idea that if you create a large-scale parental choice program, and if you advocate for that and you put money into protecting that, that you actually create the opportunity for there to be permanent change because you now have empowered parents by giving them control over the financial aid, by giving them the power to lead, by giving them the power to choose, and you essentially have upset the status quo and disrupted the status quo, in a way that it will not be possible to return to business as usual. Thanks. MR. HESS:  Thank you, Lawrence. Susan. MS. FUHRMAN:  Thank you.  I want to say thank you to everyone in the room who has ever given me money or ever will give me money, and to say that if I say anything slightly critical, it's not you.  [Laughter.] Actually, the context that I bring to this discussion of philanthropy and education stems from my work with Jim Kelly, my mentor, 30 years ago, when I was a mere child, and was helping him as a consultant at the Ford Foundation and his extraordinary leadership of the school finance reform movement. I want to spend a moment on it because I think Jim is too modest to talk about it himself.  I'm not sure anybody else will.  But I think it has such important lessons for all of us. Clearly, there was a goal to make school financing more equitable in this country, but what struck me, and I think, you know, many others, was the strategy that drove the goal, a very careful strategy, a well-planned one and exceedingly well-implemented one. The strategy was to establish a portfolio of grantees, not just one type of grantee in a number of cities, for example, but many different types of grantees.  Community advocates, legal experts, researchers who documented the problem and came up with alternative solutions. Policy maker associations and groups. There was training for doctoral students--thank you again, Jim--at a number of universities, and this was balanced, this portfolio, between local sites where there was high leverage, places like California where there are major suits, New Jersey where there are major suits, and national efforts that would inform all of us, and help those states who weren't get ready, and I think that we know, from looking back on this time, that that local-national balance is exceedingly important because we found school finance reforms in states, in fact, without court suits, but those that anticipated court suits because they were watching other states. These grantees were networked, they were brought together, and they were able to learn from one another.  I know that, intimately, because it was my job to find the conference venue, that we were usually in warm palces in the winter where everybody could come together and spend a few days learning from one another.  And Ford, at the time, was maybe the lead in this effort but certainly not the only funder involved, and Jim was able to bring in other private funders such as Carnegie, which was a major player, but also public funders.  The National Institute of Education was also a part of the effort. So this was a glorious endeavor with many aspects, and amazingly successful, and judging from the papers, which I didn't get much of a chance to read but which I skimmed, I don't think there was enough of that now, and that's where I want to move on. I think that the kind of opportunistic behavior that Rich described and that I think Don describes in Charlotte/Mecklinberg, on the part of districts and schools around philanthropy, or what some of us call the "Christmas tree effect" because there are so many different projects going on that it glows like a tree with a lot of bulbs on it, is aggravated by foundations, in many cases. I think funders carve out areas, separate areas from one another, to be distinct and to cause themselves attention.  There's not enough cooperation efforts to make the efforts aggregate. And you could have situations in which a district has different approaches at each level of education, for example, at elementary, middle and high school, because funders are driving those agendas, and, you know, certainly it's rare that an after-school program would coordinate with a program during the day or that early childhood efforts would be articulated with K-12. I mean, rare in any case because of the general issues of capacity at local district levels but aggravated often by philanthropists. I think that the evaluation that Don talked about in Houston as being an enormously expensive evaluation is rare, from what seen.  That very often foundations do not put enough money into evaluation, the kind of evaluation which would illustrate some of these problems. Evaluation is expensive, it's not the last, you know, 100,000 in a multimillion dollar grant, but it needs to be thought of separately and I thnk cost out separately, especially now when we're all realizing the importance of the what works question, and the question of Does it work? requiring an extensive multiyear design with a gold standard of random assignment being optimal.  That's expensive, not only because you have to fund the research, but expensive because you have to maintain the funding over the period of time when the effort has to be studied, and I do want to say that it may be that that kind of causal research about policy is only really possible in the context of foundation initiatives. Because, as we all know, publicly-funded initiatives are so subject to budget vagaries and politics, that it may be that the only initiatives that last a number of years to study whether they work are those funded by an outside party. And I also want to put in a plug--I think we need more of what was done at Ford, which is not just evaluation but other kinds of research.  Those of us in the research community watch Spencer's endowment daily because that's the place where education research is funded, and it's our wish that others would see that understanding the problem, defining alternative solutions, these are distinct from evaluation, these are research functions that we would hope that funders would see as important complements to program efforts. For one thing, if there were more research funded by foundations, we might be able to drag more education professors into the mainstream.  And I can say that cause I'm a dean. So I think that what I'm saying is that strategy is enormously important.  So it's not, Tom, just the goal.  Certainly what I witnessed at Ford was not a goal that was consonant with what the mainstream was thinking at the time. It was in fact revolutionary.  But the strategy was enormously important in making that revolution happen.  Thank you. MR. HESS:  Thank you, Susan.  Mike. MR. FEINBERG:  Thanks, Rick, for having me here.  It's great to be here.  I'm very grateful to our many philanthropy partners who are in this room, most of which are financial but there's also a reminder of the spiritual and inspirational type of philanthropy, every time I'm in a room and see Wendy Koff and Howard Fuller. You know, there's a lot of lessons learned from us KIPPsters on education philanthropy.  Back in '93, when we were two Teach America Corps members in our second year, when Dave Love [ph] and I stayed up all night, wrote the KIPP proposal, while "You too achtung baby" was playing on repeat play.  You know, we wrote the proposal.  The first thing we had to do was get it approved by the Houston school district, which that wasn't all that hard because we were used to breaking all the rules anyway. You know, but then of course we had to get funded, and I remember one of my early lessons learned is that I spent just weeks writing hundreds of grant proposals to corporations and foundations and all that.  You know, I raise about $10,000 that way.  Dave watched me do all that, then in 10 seconds picked up the phone, made two phone calls and raised $10,000 himself.  For some reason, we've been friends ever since. I'm going to talk very briefly because I want to make sure there's plenty of time for Q&A and I don't want to repeat all the good thoughts that were up here.  But just some lessons learned from what we've done. I think that there's two of our--you know, KIPP now have these five pillars which we're trying to implement around the country.  More time on task, choice of commitment, power to lead, high expectations, and focus on results. And two of them I think apply very much to the subject.  The first is focus on results, which we train school leaders and teachers to do but also certainly apply to the grantors and the grantees as well. You know, over a decade ago, we all learned the phrase, "It's the economy, stupid."  I think this is "It's results, stupid," I think applies very well to this one. It is all about results.  I think both grantors and grantees need to focus on results, they need to chase results, they need to search for results.  Grantors should be funding results and rewarding results and grantees should be accountable for results. If I haven't said results enough, I'll say it a few more times.  Results, results, results. And I think that's a little bit of a trust file [?].  There's too many people to get in the trap, thinking if only I got some funding I would be able to get the results.  I think that's reversed.  I think people need to go out there, get the results and put their trust in smart knowledgeable, well-meaning people, like several people in this room, that if they see the results and they see the promise in them, are going to fund them. The other pillar I think applies is power to lead.  You know, we believe that school leaders, when we talk about at the state level, the district level, the school level, should have power to lead their districts and their schools.  I mean, that they could choose who teaches in the building and also they can control their budgets and allocate their funds, which means that beyond just the ability of, from a grantor's standpoint, beyond the ability of who you're giving the grants to, and their skills and abilities, they have to be in an environment where any of those philanthropic dollars could be allocated for the best use, without going through some crazy bureaucratic formula, where, at the end of the day, who knows? maybe 20 cents on the dollar winds up doing the good that you all thought it was going to do. So certainly it has to be done within an environment of power to lead. I think the challenge in this is that I think there needs to be a balance between funding the current great results while allowing also for philanthropy and new ideas and entrepreneurial efforts as well, and that's a careful balance, because we don't want to continue trying to reinvent the wheel. Just continue to fund all the entrepreneurial ideas, and new ideas that are out there, will just keep us spinning the wheels and not moving forward.  At the same time, just funding all the sustaining efforts, by itself too, doesn't allow for fresh new ideas to come in, and eventually all of us reformers wind up becoming the protectionist establishment. So I think it's a very careful balance between those two of funding new ideas and getting great results, along with once those results are there, helping to sustain them as well. And lastly, I wish I was here earlier to hear more of that debate on the discussion of program officers but I got a chance to hear some of it and my thought there is it reminds me of the Gandhi sugar story that we teach our kids, which we teach the kids that a woman burst unto Gandhi and said, Gandhi, please help my stop eating, he eats too much sugar, help him stop eating sugar. And Gandhi said, sure, come back in a week.  So she goes okay.  She comes back in a week and says, all right, Gandhi, I'm back, can you please help my son stop eating sugar. He says sure.  He turns to the boy and goes: Son, stop eating sugar.  And the woman goes: That's it?  Why couldn't you have told me that a week ago? He said, Well, a week ago, I hadn't given up sugar yet. I don't know if that's really a Gandhi story or not but it sounds good and the kids like it. But, you know, I think that approach here as well.  I mean, do program officers, you know, do they need to be former teachers or educators with great results?  Not necessarily.  I think certainly that would help.  It seems to me it's a means to an end.  The end is to be empathetic and knowledgeable on what it is that we're trying to accomplish out there, and certainly, if someone had actually lived that life, that's the best way to make it happen.  But means to an end means there's probably some other avenues for that to happen as well.  You know, I think there's other ways, if I get that experience, but it's the empathy and knowledge about what it takes to get results and what results look like at the end of the day that really matter.  Thank you. MR. HESS:  Thank you, Mike.  Morgan.  Let's go ahead.  Please catch Morgan's eye, and we've got about 20 minutes for questions and comments. QUESTION:  Susan Traiman, Business Roundtable.  One of the things I think troubles the public is that many of the evaluators are perceived to be in the same bed as the funders and the projects, and that's why a few articles in the New York Times about Houston plant a seed of doubt, that all of the research that Don may have done, or that Broad may have done, those articles counterpart. I wonder if any of you could comment on that. MS. FUHRMAN:  I prefer it when the evaluation is bid or, you know, contracted separately from reform itself, and not built into the reform proposal.  I think it's a mistake to ask the recipient of the program grant to then go find an evaluator, even if that person is independent, it's painted, just from the get-go, and the perception is there, and it would be better if the foundation went outside, and had a stable of evaluators that it could use. MR. HESS:  Mike, you're the only guy up there who's actually a operating kind of recipient of philanthropic evaluations who gets evaluated on these kinds of issues.  Could you talk a bit about the tension from your perspective in terms of evaluators who are going to be credible and honest and arm's length, but at the same time, evaluators who are going to be likely to have, you know, be hired by Pisces, have a good working relationship with you all, be trustworthy in your eyes. MR. FEINBERG:  You know, because I'm a big believer in getting results, results, results and accountability, to me, there's room--it's not either/or.  Everyone should be coming in and evaluating, be accountable.  I believe in being very transparent, having an open door.  You know, none of our schools are what I call Thursday schools, that say please come and visit but you have to come on Thursdays. You know, the results should be out there, all the results should be out there.  From day one, Dave and I actually got in trouble for this, but we always showed the data on all our kids, including the ones that were exempted, not just the ones that took the test.  You know, I just think about all the data should be out there and to me, it's a blend.  If the people receiving the grant can have people come in, to make them better, and get an assessment, get evaluation, that's great, and if there's a need for outside independent people to come in as well to add credibility, fantastic as well. QUESTION:  Chester Finn, Thomas P. Fordham Foundation.  Bill seemed awfully defensive, I thought, in talking about Tom's paper.  I find it hard to believe you don't think that the core values of the staff members at the foundations have a very substantial influence on the kinds of projects that they support, and I'd like to push that by noting, at least implicit, I thought, in what Lawrence said, and what Susan said, and maybe what Mike said, is how few funders they can turn to that value the things they're interested in, be it school choice or education research. And there are 60,000 foundations in America, last I looked.  So if it's not the core values of the people in the foundations that shape what they're paying for, what is it? MR. FEINBERG:  Well, if I sounded defensive because I talk fast, I didn't mean to be.  I think what I tried to set up is that I think it's an interesting hypothesis.  I just don't think that research that Tom assembled helps us answer that or prove or disprove that hypothesis.  I think to your point, Chester, I worry that we sort of talk out of both sides of our mouth on this issue.  I think, on the one hand, we say we want foundations to be strategic, we want them to have a point of view, we want them to be leaders in their field, we want them to be the actors, the prime movers. And then, on the other hand, we say we want them to be responsive, there's no accountability, they should be completely responding to what people in schools and nonprofits want to do, and I think we need to get clear about which one of those we value more. I think some funders tend to value one side and some funders tend to value the other side.  But those are fundamentally sort of incompatible propositions. QUESTION:  Hi.  Erin McGoldrick [ph], Urban Education Partnership.  I was wondering if the panel could comment on--one of the things that I've noticed in working with districts is that in fact some districts actually do a pretty job of evaluating the effectiveness of programs, but I've never seen a case in which the program evaluation is connected up with the budgeting process. So you look at kind of the effectiveness of, you know, yes, this program was able to achieve student achievement but we don't know at what cost, and it strikes me for decision makers, for board members and superintendents who are trying to decide between two programs, knowing the cost of programs, in addition to their effectiveness is pretty important. I hadn't heard it talked about yet and I just wondered if the panel could comment on if the philanthropic community is also kind of grappling with that issue. MS. FUHRMAN:  I don't know the answer to whether the philanthropic community is grappling with it, although you would think so, because I think it's the job to put cost and benefits together.  But what I will say, and I said this before--the capacity at the district level is a major problem.  It's a major problem whether you want to turn around failing schools or it's a major problem if you want to evaluate programs or if you want to do the kind of cost-benefit analysis that would enable you to put together an excellent budget. Central offices in districts and state education agencies have suffered enormously over these last 20 years as more and more responsibilities are added, and we expect more and more.  I mean, who's supposed to turn around the schools that are in need of improvement?  Local districts and the states.  But the people who do that aren't there because the money has been pushed down, the money gets scarcer and it gets pushed down into the kids where, you know, it certainly belongs.  Maybe that's a plug for philanthropists to attend to that kind of capacity but it doesn't exist now in the public school system. QUESTION:  Ray MaGee [ph], SRI International.  Several of the panelists have talked about the importance of context and how context shapes a lot of what's happening in philanthropy, and even in fact Trump philanthropy, as Bill Porter mentioned, and of course Patrick, you talked about the politics of race, and that's a contextual issue as well.  Politics as well affects the evaluation in research as well.  Susan Fuhrman has talked about that. What can foundations do to address this issue of context and create a situation where their work, people can buy into their work, that is, they're not perceived as being outsiders necessarily, and they can be effective in really supporting districts in the work that they're trying to do through reform schools. MR.       :  [inaudible]. [off-mike,] MR. HESS:  Bill. MR. PORTER:  Well, I think that's, you know, one of the one million dollar questions that funders struggle with.  I think that we don't talk enough--it's very easy to talk about the failures like, you know, we often, many people point to the Annenberg Challenge, but we don't talk enough about, really, the obstacles that make funding in education, especially urban school systems, so difficult. I guess from my point of view, and you've heard some of the strands in the conversation this morning and I think you'll hear some more about it this afternoon when Alan Bersin and Arlene Ackerman, and others, you know, get their chance to sound off. But I think my list would include, you know, having a funder that's really articulated a clear rationale for what they're trying to do, and how they think what they're going to do with the resources they have is going to make any difference.  I think when the funder is clear about what it's trying to do, it's easier for the funder or the donor to find partners. I think they kind a go into a situation and say, What is it you want to do? you know, just kind a leads to this mushiness that often doesn't lead anywhere. So I think for funders to be clear about what it is they're trying to accomplish and how they think their very modest sort of investment is going to make any sort of a difference, and getting clear about that is important, just as Don and Lynn's paper talks about, the school system itself also having that clarity of thinking. And so I think you put those two things together and at least you can get people who agree on the central that needs to take place. MR. HESS:  Bill, you kind of raise a really interesting point, I think that comes out of what Don was talking about, is if, as you point out, we're kind of pushing on donors from both sides.  Look, you guys be coherent, make a difference, at the same time-- MR. PORTER:  Back off; yeah. MR. HESS:  Back off.  But if these guys have a theory of action, and they're not just kind of entities--there's people at this foundations who have signed off on a theory of action, and there's scholars out there who designed rationales for why this is going to work. And so then these folks go into districts that align with this larger theory of action and the superintendent puts their reputation on the line for it, and then we go out and find evaluators who these folks think are competent. I mean, I guess it's partly for [inaudible], and partly for Susan.  How do we get ourselves into a situation in which we're able to find kind of disinterested, thoughtful critics, who are kind of invited to the table, when everybody, even granted the test of intentions, has kind of this natural alignment of interest and commitment and belief that this stuff is working and is going to work, and that anybody who's a naysayer might just be somebody trying to cause a ruckus for ruckus's sake? MR. PORTER:  Well, I don't mean for this to be a copout answer but I think having meetings like this.  I mean, I think we've actually caught philanthropy at a very interesting time.  There's much more congressional scrutiny on the Hill right now.  There's a lot more discussion about Congress coming down and really changing the nonprofit sector in significant ways.  I think there's a lot more discussion in the field about accountability, about transparency.  I think funders as a whole, that the philanthropic community is a lot more open to this argument, and so I think now is the time to sort of press that case. MR. FEINBERG:  Again, we kind a talk about philanthropy as if it's some sort of a rational system.  You know, it's not, it's a product of the tax code, where the government decided, instead of taxing the money and using that money ourselves, we're going to instead let people sort of decide how they want to spend it for charitable purposes. So the good news is there's probably more money going into society's problems.  The bad news is that none of us, the politick, don't have any real say over where that money goes. And so I think we just need to think through that dilemma.  I mean, that's the inherent dilemma in philanthropy, and Peter Frumkin may talk some more about that later this afternoon. MS. FUHRMAN:  Well, I'm wondering if this is the right audience.  I'm not sure it's the program officers that we need to talk to.  Maybe the boards.  I'm on the board of two foundations and one is in house and one is a grantmaking foundation, and I'm, for sure, the only voice on that latter board that would recommend evaluation, or, you know, perhaps aggregating grants, say we're not giving $25,000 to, you know, 2000 people, so that we can make some sort of difference. You know, the other people on the board are established, long-time members, who are inclined to do things the way they've always been done, and even though we've had retreats and try to talk about that, it's hard to change, and I think that would be an important audience.  I don't know if the program officers agree. MR. PORTER:  I just want to add, I think this whole conversation about evaluation is actually a lot more complicated than we're making it sound.  I think that there's a lot of foundations that aren't investing a whole lot in evaluation, and that's because they're small and because they've made the decision that they want to push their dollars out the door to support sort of the charitable impact as they see it, instead of investing in evaluation or in staff to do evaluation. Now we can quibble with that decision but that was their decision to make. I think we also kind a lose--we talk about evaluation without really saying what is it we want to evaluate, and I would argue that different funders ought to be evaluating different things.  I think sort of the default maybe is that we're all sort of saying every single thing a funder funds should be applied to this gold standard of evaluation. I don't think that's the case.  I think we need to get a lot clearer about what we're talking about with evaluation. Are we talking about understanding the work that happened, what the grantees did, so that it might be able to be replicated? Are we trying to strengthen the implementation?  We see a good program but we want to help it improve or we want to help it replicate?  I mean, there's different reasons that a funder would want to go in and sort of evaluate and so I think we need to get some greater clarity about the question of what it is we need to know. MR. HESS:  Well, and Mike, I mean, when we talk about kind of evaluation, you know, you and David, and kind of the KIPP enterprise, now are at the point when, if you guys are talking to a community about going in and opening up a KIPP school, and there's a foundation out there that says look, we've got some seed money to help, isn't it quite difficult, at this point, for a school board or for a superintendent to then say, Look--we're opposed to bringing KIPP in?  Or is that an unfair characterization? MR. FEINBERG:  We still get our fair share of yes/buts, I mean, definitely out there.  It's starting, once again, you know, the flywheel is starting to turn in a positive direction, but there's still spoken, mostly unspoken cynicism, still, that's very alive and healthy, unfortunately, out there. MR. HESS:  And healthy in what sense? MR. FEINBERG: In that I think people still tend to be very cynical about one, What is possible? and two, when there's proof of, when the actual is proving the possible, it is legit or is it somehow contrived? [Start side 2B.] MR. PATRICK: [in progress] that even if you have a great school model like Pitt, that sometimes that's not the issue.  It's the politics that prevent KIPP from coming in to some cities.  I mean, I'm not going to put your business out but I know at least one place where you had the contract negotiated--everything--and it was the politics of the situation that kept it from going forward. So I mean, I think it's important to be clear that it's not just about quality and it's not just about having a good theory of action.  It's also about addressing a political reality, which is that in most of these cities you have people in power who are committed to protecting the status quo.  They're committed to protecting the status quo because they're advantaged under the status quo. Until we change that dynamic, any work we do, any gradual improvements we do around the edges is going to be just that--it's going to be around the edges.  But if you want to change the me, you've got to change that political reality, and I see, you know, school choice as one way to do that by essentially tipping the balance of power. MR. HESS:  Morgan. QUESTION:  Yes, my name is Elizabeth Carson and I'm a New York City parent, and the founder of New York City HOLD, which is a mathematics education advocacy organization, and I'm very excited about all the conversation I've heard today because it's extremely important, in particular the idea that a strategic plan and community buy-in are somehow at odds with each other.  I couldn't disagree more vehemently with,k and in fact I think one of the key ideas that philanthropy needs to address is the fact that to truly come up with strategic plans that are in fact going to work, the philanthropies must go into the communities and find out the values and standards of those primary education stakeholders and that would be parents, classroom educators, senior classroom educators and administrators, first and foremost, and I would add university subject experts to that mix, in particular, because of what I've learned through math education advocacy. These constituents must be at the table in terms of designing strategic plans, and I believe that if the philanthropies support the kind of research and development before the strategic plan is decided, ensuring that all of the stakeholders and the most knowledgeable and experienced people who are closest to the children are at the table in the design, I think you're going to find much less trouble in terms of community buy-in.  It will not eliminate-- MR. HESS:  Okay.  Is there a question, please? QUESTION:  Yes; yes.  The question is how can, or have already, any philanthropies formed the model that I am describing right now, because I know that in New York City most of the projects have not engaged the community and the stakeholders, that I think are so critically important in forwarding reforms. MR. HESS:  Okay; all right.  So are you all familiar with anywhere where this kind of collaborative effort that was just described has been [inaudible]? MR. PATRICK:  I know that the Skilman Foundation in Michigan is attempting to do that, and I think it flows partially--is anybody from Skilman here?  Okay.  I think it flows partially from their level of frustration with having spent $50 million in Detroit trying to improve things, and not seeing progress, and they are doing exactly that.  They're trying to go out and bring the key stakeholders in to really help them think about, and they've actually set up like a huge commission of parents and activists and teachers, just all the stakeholders, to try to figure out how do they create a collective vision for reforming education. MR. FEINBERG:  I mean, I'd add, for example, one--in fact New York City, there's a great effort in the Bronx, CC9, which has been largely funded by the philanthropic community, which is a community organizing effort that's brought together parent groups, the parents from low-income communities with the teachers union and the school district to really advocate for dramatic change in the schools in the Bronx. And so that's a case where these community organizers came together, organized parents, developed a vision for what they wanted to accomplish, and then sought philanthropic support for it.  I think you see another story in Chattanooga where foundations have largely supported the work of the public education fund there to help bring the community together and agree on a strategic plan. I actually think that most foundations, certainly effective foundations, actually do a whole lot of collaboration, they don't work in a vacuum, and that gets back to one of my questions about Tom Loveless's research, is that he kind of implies that because program officers have these points of view, that somehow that colors the whole work of the foundation. I think most foundations actually do go out and bend over backwards to try to talk to people and learn about their communities and learn about what the key issues are in the field.  Or at least the effective ones do. MR. HESS:  Okay.  With that, I'd like to thank the panelists, and we'll begin with the next panel, looking at approaches to giving in, I believe, 15 minutes, and we'll reconvene then. [Break.] MR. HESS:  I feel like one of those little kids in the pageant, you know, the kid who can't sing or anything, and just stand there and bangs things.  That's usually what I did, come to think of it.  As we're gearing up for the next panel, let me just remind you that after this one we're breaking for lunch.  Like I said, for me, too, it's great to see so many friends I don't get to see as much as I'd like in one room, in one place.  I know that a lot of this stuff, at least for me, is kind of interesting, and I want to chew on and talk about. But, you know, hopefully over lunch, and then later at the reception, and during breaks, you're getting enough time to kind of talk about these things. One point was made to me which is kind of interesting.  A couple points.  One, I always like it when I'm not the most cynical person in the room.  Of course any time Mike Castle is in the room, I'm not the most cynical guy in the room, which I love.  Mike, it was meant "with love!" But a couple folks who shall remain anonymous, have actually come up and asked, maybe proving kind of why this conference is useful, that it might be useful to solicit anonymous questions, if folks aren't comfortable kind of asking their questions in this forum, of which I have at least kind of one in front of me at this very moment, which I will ask on behalf of folks who themselves aren't as insulated by the vast right-wing conspiracy as myself. So given that, if you have questions that you may not feel comfortable, for whatever reason, asking, please feel free to just slip them up here and I'll be happy to ask them on your behalf. With that, let me go ahead and introduce our third panel. This is the research panel that's really looking at approaches to giving in K-12--oh.  One other thing.  It was pointed out to me that it might be nice if I actually indicated who was funding this endeavor, in the spirit of transparency.  And I will.  It is Dan Soros [ph] of the Soros Hospital--actually, I don't even know the name.  Nancy, what's the name?  Thank you.  Dan Soros, who funds so much of the work I do at AEI and permits me to tick off almost everybody, for which I am eternally grateful to him--personally funded this thing, and I think one of the interesting kind of comments on, you know, so much of these conversations, is--I think we've chatted with several funders about potentially funding this and they all expressed various degrees of reticence or kind of disinterest, that it didn't fit with their current funding priorities. So, you know, what it is.  Nancy, God bless you and Ed.  Anyway.  So that's it.  The entire thing is funded by Dan Soros, for what it's worth. With that, let me introduce this panel.  The first speaker is going to be Jay Greene.  Many of you are familiar with Jay's research on a number of topics.  I bent Jay's arm with the promise of a chance to go after new data, and Jay therefore undertook decoding of thousands of 990s because only Jay would, and Jay's a senior fellow at the Manhattan Institute Education Research Office. Speaking second will be Jane Hannaway, director of Education Policy Center at the Urban Institute. Speaking third will be Bryan Hassel, director of Public Impact, a North Carolina-based education policy and management consulting firm. And next to Bryan is Andy Rotherham, director of the Progressive Policy Institute's 21st Century Schools Project. Jay is going to give us the overview of giving to K-12 ed philanthropy and what this money goes towards.  Andy, Bryan and Jane are going to speak much more in depth about particular approaches to giving, touching on, particularly, some of the issues raised this morning. Jay. MR. GREENE:  Thanks all of you, for being here, and like Rick, I will try to aspire towards higher levels of cynicism and ticking people off.  Unfortunately, I think this paper may do that a little bit, but I do think that we need to have some new ways of thinking if we're going to make greater progress in philanthropy. And the basic conclusion of my paper is that trying to reform public education through the sheer force of private dollars is a little bit like pouring buckets into the ocean.  The problem is that there's just too little private money and too much public money, just like there's too little water in buckets and too much water in the sea. So if you have broad goals as philanthropists to reform the education system as a whole, rather than limited goals, trying to reform education for someone at some time in some place--if you have these broad goals, it's extremely hard to do it simply with the force of your private dollars. Now, first, let me just go through my efforts at trying to estimate how much private philanthropy there is going into public education. I should say that everything I've done here is filled with error, and it's filled with error because doing this kind of work is extremely hard.  It's not incredibly transparent.  The amount of information that foundations have to provide in government filings is limited, and so there's a lot of guesswork and data entry involved, and I almost certainly have made mistakes, but I don't think that the mistakes will alter any of the conclusions. So first, How much giving is there to public education?  Well, if we take a look at how much is given from the top 30 foundations, and that these top thirty are identified by the Foundation Center, they--I should also note, by the way, that my estimates for the giving from each of these top 30 foundations, differs, sometimes quite dramatically, from the amount that the Foundation Center reports as being given by these foundations. I don't entirely understand the difference, I have to confess.  I was offered a variety of possible explanations but I don't know which of these explanations or if something else is actually the thing that makes the difference. But if you take the giving from the largest 30 foundations, you see that there's about $649 million that was given in 2002 from the top 30 foundations.  The Foundation Center's overall estimates on those top 30 foundations is about the same as mine.  It's about 660 million.  So their overall picture, and my overall picture from the top 30 foundations is consistent. It's also the case that as you go down the list of the 30 foundations, the amount given drops dramatically as you move down.  So that there are very few foundations that are giving more than $20 million per year. And so even smaller foundations are giving much less, and in fact according to the Foundation Center, if you look at the top 1000 foundations, the total amount given is somewhere on the order of 1.3 billion.  However, that includes money given to private schools, and according to my estimate from the top 30 foundations, private school giving constitutes about 13 percent of the giving from those top thirty. So if we make the same assumption for all thousand foundations, something less than $1.2 billion is being given per year from the top one thousand foundations.  Smaller foundations and individuals are adding to this pile as well, but, again, the amounts are getting so small as we move down the list, that it's extremely unlikely that we have more than $1.5 billion going to public schools from private philanthropy, from the bake sale to the Gates Foundation. Now I've also tried to estimate it another way.  I administered a survey to the 100 largest school districts and asked them to report, in the most recent year, how much philanthropic money they had received, and I received responses from 41 of those 100 districts, that contained about 41 percent of the students in the 100 largest districts, and about 9 percent of all students nationwide. And the estimate from these districts was that they had received about $27 per pupil in the most recent year, and if we extrapolate that to all 48 million kids in public schools, we again come out with something like $1.5 billion. So different ways of getting an answer, looking at what I've collected from the top 30 foundations, looking at what the Foundation Center has collected, and looking at a survey of large school districts, we come up with about a billion and a half. How does this compare to what's spent by public schools?  Well, you have about 430 billion, or so, spent by public schools per year, at the same time, in 2002. They spent, on average, $8,922 per pupil, and so total giving from all private sources is about one-third of one percent of all spending by public schools.  So it's a very small amount. So if private giving is such a small amount of what public schools spend, how can they make a difference at all?  I don't think the situation is hopeless.  If the problem is that philanthropists only have buckets of water and there's the ocean for them to shape, what can they do? Well, they need to put down their buckets and start digging channels, building dikes, and building desalinization plants.  They have to shape the ocean, and shaping the ocean can be done by trying to use their private dollars to redirect how public dollars are spent in the future. The only hope for private dollars making a very broad impact is if they can change how public dollars are used in a leveraged way.  You have to leverage their private giving to redirect how public dollars are spent. Now everyone seriously involved in philanthropy says that they have a leverage strategy for giving; everyone says it.  Most don't do it.  Most don't do it because it's hard to do and because they may not fully appreciate what is required to alter future public spending. And so according to my calculations, if we look at the top 30 foundations, most of those dollars given by the top 30 foundations are given in low leverage ways, and in fact if you remove the inordinate influence of the Gates Foundation because it's so big, and look at the median foundation about the top thirty, you see that about fourth-fifths of the relatively small amount they have in the first place, about fourth-fifths are given in low leverage ways. Now what do I mean by "low leverage"?  Low leverage activities are ones that have a low likelihood of redirecting public spending in the future, and they have a low likelihood of redirecting public spending because they do not consider the barriers to public money being spent in the same way. Most things that philanthropists support could be purchased with public dollars.  Public schools could be buying, on their own, new technology in the classroom.  They could be buying, on their own, professional development for their staff.  They could be buying, on their own, a new curriculum. When private philanthropists give these things to schools, they are giving things to schools, ideas, programs, or thing that schools already have money to buy.  Now you might say the problem is the public schools have lots of money.  That's true.  But they also have lots of needs and their money's already spoken for.  They have none left over for these incredibly important things, and philanthropists can come in and give you key things that the public schools need but don't have any money left over for.  That's how they can be leveraged and make a difference. I'm afraid that I don't believe that that is actually leverage and the reason why it's not leverage is that there is a reason why public schools already have their money spoken for.  There's a reason why there's none left over for these critically important things.  If these are critically important things, if these are really a priority, why aren't they a priority for the public schools? And the reason why they're not a priority for the public schools is that public schools are governed democratically and in the democratic governance of schools, there are political realities that dictate how funds will be allocated. So if you give things to public schools without addressing why they wouldn't buy it on their own, you can only make a difference where you give it and for only as long as you give, because whenever you stop giving, or anywhere where you are not giving, those barriers still exist or will exist in the future and will prevent this good idea from taking root. Now please try to notice that I'm trying, in this paper at least, and in this conference at least, to be agnostic about what kind of reform is good.  I'm not saying here whether it's good to have big schools or small schools, or school choice or not school choice. The point is that whatever your reform strategy is, whatever your reform idea is, the best chance you have for getting it adopted broadly is to convince public dollars to support it, is to somehow redirect public dollars in a way that will support that thing. Even if you believe that the most important thing the public schools need is more money, your limited dollars are best spent advocating for more public spending on schools. You can make a much bigger difference with the taxpayers dollars than you can with your own dollars, even though you may sometimes feel, as philanthropists, that you have a lot of money, and it is a lot of money.  I mean, millions of dollars; that's a lot of money. But people forget how little that is compared to what is being spent by public schools.  And to make an analogy here, the amount that we spend on public schools is about the same as the GDP of Russia.  Okay.  The entire amount of goods and services produced in a year, by Russia, is about the amount that we spend per year on public schools. And so let's say we want to change Russia.  Would we say, oh, well, you know, the trouble is that--you know--well, if we have a billion dollars, a billion dollars feel like a lot.  If we have a billion dollars, can we change Russia with a billion dollars?  Probably not. What if we said, well, the problem in Russia isn't that they don't have good ideas.  It's that they don't have any money left over for the good ideas.  All the money's already spoken for.  If we just came in with the good ideas, then we'd be able to do it. Well, that would be ignoring all of the political and social barriers that exist to doing whatever you want Russia to do.  Russia has a big country with lots of people, lots of interests, and so they have an idea of what they want to do.  So how can you reform Russia? Well, you have to figure out some way of redirecting how Russia uses its own resources.  That would be your only hope.  And people engaged in philanthropy in Russia do it.  That's what they do.  When George Soros goes into Russia and teaches democracy classes, he's trying a leverage strategy. But when it comes to our own public schools, we don't have the same cynicism about the political barriers.  We don't recognize that schools are political organizations and we don't think about the political barriers to adopting reform. And we instead think that if we just give them good ideas, that they'll find them on their own, they'll replicate them on their own, and it'll solve itself, and it doesn't.  It's as if, to throw in one more metaphor, the reformers are sprinkling seeds, and they think that there's perfect soil, moisture and sunlight, and that reform will just sprout, if they just sprinkle the right seeds. And so the whole trick is to find the right seeds, and the problem with this way of thinking is that that garden would have to be the garden of Eden for this strategy to work because it ignores all of the limitations of human beings and all of their interests in political organizations, that would resist things, even if they were good, because what's good for the system might not be good for them individually.  So how can you overcome it? Well, you can have high leverage strategies, and high leverage strategies redirect future spending, either through research and advocacy, by trying to convince people that something else is good and that they ought to be using public dollars to support it or it involves structural changes in schools, that public dollars flow in a new way, like digging a channel. So small schools would be an example of a high leverage strategy, whether it's a good idea or not I'm not saying--but it would be a high leverage strategy because once you build a small school, it's now supported with the per pupil operating spending from public dollars, and you now have leveraged your idea, you've built the channel and water's going to flow through it. You've built the school, public dollars will support it.  Same with charter schools. And so I think that there are a number of clever high leverage strategies that philanthropists could be pursuing more. MR. HESS:  Okay.  Thank you so much, Jay, and Jay's paper really does two things.  One, it makes this kind of compelling argument, and I love the metaphor, but two, we've also got just a whole slew of data that nobody has ever really assembled and presented on.  The amount given, the amount given by the big 30 givers, and where their money is going, by category. So I'd really refer you to Jay's paper. Jane. MS. HANNAWAY:  Thank you.  I work in a soft money operation and not one to shoot myself in the foot.  I'm going to talk about some of the brilliant decisions that foundations have made in funding education reform.  I'm focusing my comments on philanthropy and labor market reform in education and I'm going to look specifically at the National Board for Professional Teaching Standards and Teach For America, and my collaborator in this paper is Kendra Bishop, who is also at the Urban Institute.  Kendra, where are you?  Back there in the corner. Okay.  Why did I choose to focus on labor market reforms and philanthropy?  Well, the first reason is that teachers are important, and we know teachers are important.  They're the most important in-school factor affecting student achievement. We even now know that having a good teacher for a number of years in a row can almost make up for the difference between kids from disadvantaged backgrounds and kids from advantaged backgrounds.  So it's very important input. We also know that labor market reforms are exceedingly difficult to pull off, and they're exceedingly difficult to pull off for--well, let me back up for a minute. We know teachers are very important.  We also know that we have a big problem with the teacher labor pool.  We know that there are many teachers out there who are non-certified.  We know that the "best and the brightest" are not going into teaching.  We know that the "best and the brightest" are leaving teaching. We know that we have to improve teacher quality in order to improve schools.  But labor market reforms are very difficult, and they're very difficult for a couple reasons.  One is something that Jay mentioned, is that institutions, large complex institutions of any type are unlikely to reform themselves.  It's just too difficult.  You can't change one piece.  It affects too many other pieces.  Individuals are very concerned about uncertainties, and if they think the results aren't adverse, there's always an aversion to uncertainty. So labor market reforms, in addition, are "bread and butter" issues, which also sets up people politically.  Labor market reforms also challenge people's professional identity, which also makes them difficult to pull off. So labor market reforms are unlikely to come from within the system.  Hence, it's a place where philanthropy can play a very important role.  Teachers are the most important input.  How can we insure that the system is set up in a way that the "best and the brightest" are attracted to the field? How can we set up the system in a way that the "best and the brightest" stay in the field? And that's what the National Board For Professional Teaching Standards and Teach For America are trying to do.  The National Board, one of its main objectives is to identify teachers in the field who are the best, identify the high-quality teachers, recognize them and reward them, and in doing that hope to retain them in the system. Teach For America is working at the other end of the system, trying to identify the "best and the brightest" and get them into teaching. So there's two reforms, both focused on changing the structure of the labor market, neither of which came from inside the system, both of which were stimulated outside the system, both of which have been able to sustain themselves and I think their story is an instructive one. National Board.  Let me go through that one first, and I do this with some intimidation.  Not only are the funders of these efforts here, but Jim Kelly is here, who's the first president of the National Board.  Joe Acquavera [ph], I think is here.  Wendy Kopp is here.  So there are people out in the audience that know a whole lot more than I do.  So if you have any real questions, I'm just going to throw them to them. National Board started after A Nation At Risk.  It was recognized by most people in the field, that teachers were important before a lot of the research came out, but it was pretty obvious that teachers are important. Carnegie pulled together a very large diverse influential group to investigate, just think about what could be done, and they came out with a report called A Nation Prepared: Teachers For A 21st Century. The results of this commission were somewhat radical.  I mean, when I went back and read the report, they came out with some fairly radical recommendations for reform that are familiar to us now. Setting high standards.  Differentiating the teaching staff in terms of quality.  I mean, those are sort of "fighting words" in some circles.  Promoting school autonomy.  Performance-based incentives for teachers and schools. I mean, these are radical reforms.  They were out there with A Nation Prepared. It's had all the critical actors at the table, they were part of it, but there was resistance.  There was political resistance.  There was tenuous support.  This is a comment made at the end of that report by Mary Futrell, who was president of the National Education Association at the time. "I have deep reservations about the report's conclusions and recommendations.  The report gives the impression that some teachers are not doing their job.  I'm concerned about the potential for abuse in the lead teacher concept. It suggests some teachers are more equal than others." So everybody wasn't jumping on the bandwagon.  Al Shanker had recommended the National Board two years prior, at a press conference at the National Press Club.  But there had to be a lot of influential cajoling to bring along the rest of the education establishment. But it happened.  While Mary Futrell had reservations, she was able to keep the National Education Association at the table.  They were all at the table in the beginning and all the players stayed at the table.  The other thing about the National Board is that it had an absolutely A team at the help.  Jim Kelly as president, and very importantly, Jim Hunt, the governor of North Carolina as the chairman of the board, who hung in there for 11 years.  These are all important lessons. There's been a lot of talk, I think so far, about how politics interferes with reform.  The National Board was an effort that recognized the politics, up front, and built it in very strategically into what they were doing. They also stuck to a narrow agenda.  While A Nation Prepared came out with a number of very grand objectives, they focused on one--teacher quality.  It was one thing that everyone would agree on.  They never touched issues of teacher salary within the board, for example.  So they kept the agenda narrow.  They kept everyone at the table.  Part of the reason they were able to do that, if you can see this, I hope, is that one of the deals that was struck in order to keep all the players there is that the teachers would have a majority on the board. It was a 63-member board.  Imagine running something with a 63-member board.  Very few people could do it.  A 63-member board and the majority had to be teachers, and this shows the distribution of who was on the board, the composition, and the players, again, were there.  But, again, majority teachers kind a kept them in the system.  It also affected the legitimacy of the product when it came out.  The teachers felt it was something that they had a big hand in creating. Financial backing.  Significant financial backing.  Carnegie, I think, deserves credit for a lot of the labor market reforms that have gone on in education in the United States over the years, starting with TIAA/CREF where most of us have our retirement invested, which led to labor mobility and higher education. But Carnegie was in there for 11 years with a million dollars a year in discretionary spending, again with the recognition, I am sure, that this was going to be a hard political battle, that this was going to be a long-term change if anything could be put into effect. So it was long term, significant amounts of funding, and again, with Masters at the helm, Kelly probably knew better than anyone at the time how foundations worked, how to raise money, where to go and had a lot of credibility, and there was tremendous political support there with Jim Hunt as the board chairman who could--lots of influence with other governors, lots of influence with the White House, especially under Clinton. So a lot of money was able to be raised.  National Board recognized, again--you know, these are winners and I'm focusing on and trying to see what lessons we can learn from the winners. They realized they had both a supply problem and a demand problem.  The supply problem was creating the certificates, creating credible certificates and multiple seals.  Now they had to create something to sell.  They also recognized, from the get-go, and this follows on Jay's comments--they also realized that they had a demand problem. Why would anyone want a certificate?  There had to be [inaudible] in order for teachers to want certificates.  And so from the beginning-- [Tape change.]  MS. HANNAWAY:  [In progress]--two governors were on it and both had heads of the teachers' unions were on this group that helped devise incentives, and what those incentives would be. When the federal money came in, and this is partly a Hunt-Clinton connection, although President Bush prior also had supported the board, a lot of money was really put into research.  As a matter of fact, it was restricted to research and the certificates started multiplying so that in the beginning the first two certificates only covered about 3 percent of the teacher labor market.  By 2003, there's a certificate that covers over 95 percent of the teachers. This was very important for the political side and the incentive side because governors were slow to jump in unless they could develop incentives that would apply to most teachers, not just a few teachers.  So they created the supply.  They also created the demand.  Governor Hunt was the first one to establish incentives in North Carolina.  There are now state incentives in all 50 states in the country of one sort or another that promote a national board.  There are also federal incentives. So there was a big effort put on the demand side.  It wasn't just creating summary form and putting it out there and hoping and praying that it was going to grow in fertile soil.  It was actually fertilizing the soil in basic, important and smart ways. The certificate numbers, the number of board-certified teachers had gone up dramatically.  It's still only about 1 percent of the teacher population, but it has gone up as they became available. An important part of this story, from the beginning I am told both the foundations and the people who were running the board thought that over time it had to support itself, that it would be supported for fees for service.  There was always talk about the certificates, the certification process would be paid for or on behalf of teachers and that's why they were working the demand side so hard.  You can see, in fact, they've pretty much pulled this off. In the early years, private funding, that's mainly large foundation funding.  When you look at the most recent year for which we have data, the majority of it is coming from fees for service.  It's now shifting gears.  It's got something up and running, something to sell.  It's now moving into other aspects, but let me move on to Teach for America because it's a contrasting story. It really started from the bottom.  It started with Wendy Kopp who's sitting here as a Princeton senior who came up with the idea there's a real problem out there, a very real problem.  There are hard to staff schools, school districts are having a hard time getting good teachers into them.  Meanwhile, she's surrounded by fellow classmates who would love to go into teaching who really want to go in for a while and pay back.  So she thought, why don't I just put these together? She ventured into this very naively.  She says herself I didn't have the experience to see that it couldn't be done and, hence, took off and tried to do it with neither a financial base nor political connections. But she had the supply and demand side set up.  Unlike the board who had to create it, she had the supply.  There is a large pool of very bright, interested college students who do want to teach in these hard to staff schools.  The supply is there.  The demand is there.  There are districts with hard to staff schools, so she didn't have to worry so much about the supply and demand.  That's what she saw what she was doing what she thought was going to be the simple part, and that was just putting these two pieces together. Not so simple, and this is again the reason why I think philanthropy has to be brought into any labor market reforms.  One minute powerful critics, they're out in the field trying to get this thing going.  TFA teachers hurt many schools and the children in them, good intentions that fail to produce good teaching for African-American and Latino children that looks like a thin veil for arrogance, condescension and continuing neglect. Most of us know Linda, a chaired professor at Stanford, former President of the American Education and Research Association who has different and legitimate views about how to improve the teacher labor force, but they run completely counter to what Teach for America is trying to do. This hit Teach for America at a very vulnerable point where they're going for their money, they're trying to recruit students.  Wendy can speak for herself, but I think this was the broadside criticism that was not expected because she was young and naive. Indicators of success, there is a certain degree of financial stability.  Certainly the supply has increased which goes to prove that there really is strong supply; 17,000 applicants in 2005.  They're getting about a 16 percent acceptance rate.  If you talk to college kids, it's really hard to get in.  It's highly selective.  Six districts are participating in 1990, right now there are 66.  It's increased tenfold. The funding picture I think is again a big one.  One of the things that TFA learned was that you can't rely on the big foundations, that you can get them going in the beginning, but for long-term support you have to be closer to your service delivery and I think this was a story both in TFA and the national board.  Both of these programs are in effect hitting a market test where the sources of support are coming much closer where the service is being provided. What can we learn?  Five quick things, and this is the last slide.  Number one, labor market reforms are complex.  They're likely to need pressure and support from outside the system, unlikely to come from inside the system on its own.  Changes are likely to be highly charged.  Hence, it's again important for philanthropy and important for them to hang in there through some of the hard times. Third, there's probably like to be an incubation period if these are real reform.  If they were simple reforms, they would have come from the outside.  If they're difficult reforms, then you need some incubation period.  Fourthly, you have to worry about the supply side and demand side.  I think people have been saying this using different terms so far here, but you really do have to worry about the demand side.  Last, if it really fits a need, it will very likely survive, and that is the real test.  I'm worry for going over. MR. HESS:  Thank you, Jane.  That's great.  Brian? MR. HASSEL:  Thanks, Rick, and thanks for having me.  I also wanted to mention my co-author Amy Way (ph) is also here, and we work together on this project about school choice because certainly school choice has been one area where there's been a great deal of ferment in public education over the last 15 years. There are a number of things that go under the school choice.  There are private school vouchers and scholarships that help kids attend private schools.  Charter schools which are relatively independent public schools that operate also as schools of choice.  Then a lot of activity within school districts to change the options that are available.  For example, a lot of recent activity around breaking up large high schools into smaller high schools that are then often schools of choice for the kids who attend them. What ties all of these together is really the idea of extending the range of options to kids and families that haven't really had them as much in the past.  There's always been lots of school choice.  What we've seen in the last 15 years is an effort to extend those options to kids that couldn't afford them overwise through publicly supported options that are tuition free or heavily subsidized so that they can be affordable. So the question that Amy and I looked at in this paper is what's been the role of philanthropy in the choice landscape having gone from a situation of 15 years ago where there were very little of these kinds of things existing to now a lot, although we'll talk at the end about how much is a lot. Richard I think in his opening remarks said if it weren't for philanthropy we wouldn't have any of this stuff, something along those lines.  He probably said it more eloquently than that.  But that's a pretty strong statement, and in fact, I think accords with the reality as we see it; philanthropy has really paid a central role in the development of all these activities. What we did in the paper was two things.  First, we took the analysis that Jay Greene mentioned where he looked at the top 30 foundations and categorized their grant making.  We took that data.  Thanks to you, Jay, for letting us use that.  Then we extended it to the next 20 on the list.  So we had data on the top 50 foundations from their IRS 990 forms, categorized how much of their giving went to activities that we saw as school choice related. Jay already gave the caveat about all the air in that data because we were just working with how the foundations describe their grants in their foundation tax forms, but we have that, and that's the first thing I'll talk about. Then second, we did some analysis of trends and patterns in grant making in this area, and particularly what I'm going to talk about in my remarks are some of the strategies that these foundations have used to advance these ideas and then talk a little bit at the end about what's the impact of all this. First the numbers.  These are the five categories that Jay had set up and we consolidated some of his categories, but these are the five categories that we thought related in some way to school choice:  grants to private schools, charter schools, vouchers, small and alternative public schools which tend to be more within school districts, and then, finally, research about school choice. These categories are the five that we looked at, and the first one is worth remarking a little bit about because private school grants on the one hand, of course, have something to do with school choice.  These are schools that people choose to go to. On the other hand, what I said at the beginning was we're mostly interested here in school choice that's designed to expand the range of options to kids that haven't ordinarily had them.  Some grants to private schools have that effect and that intention such as setting up scholarship programs to allow more folks to attend them.  But a lot of grants that we see to private schools don't really fall into that category. For instance, helping a private school build a football stadium or helping them build a new science laboratory, or just the general annual giving that a lot of philanthropists do to the private schools their kids go to or that they went to as children don't really fall into this category.  So that's a somewhat dubious inclusion here, but we're going to show that on all of our slides and you can judge what you think whether it belongs or not. What's remarkable here is that a lot of the top 50 foundations are giving some money especially to private school grants and to charter schools.  Few are giving to vouchers.  Few are giving to smaller alternative and public schools.  Then very few giving at all to research, but a broad base of support at least in the sense of the number of foundations giving to those top two. Looking at it in terms of dollars, of all the K-12 giving by these top 50 donors in 2002, about 63 percent, $500 million or so, went to nonchoice-related education activities; 14 percent to private school grants; then 23 percent to other choice grants. With this pie chart, you look at it and your thought is, wow, that's a lot of giving related to school choice.  Even if you take out the private schools, it's about a quarter of the money going into activities that have something to do with school choice, and that's certainly the truth. But it's misleading in one important way which is that there are two foundations, the Bill & Melinda Gates Foundation and the Walton Family Foundation, that really drive this pie chart because they alone give about 60 percent of the funding that we talk about in the choice-related grants, 60 percent.  If you just look at that purplish slice that's the 23 percent of other choice grants, so taking out the private schools, the Gates and Walton Foundations together make up about 87 percent of the grants in that pie slice.  So a lot of what this is showing is the heavy investment of those two foundations in this kind of activity. If you take out the Gates and Walton funding, you see the picture changes a lot.  Almost all the giving, 81 percent, is for private school grants.  The other activities, vouchers, charter schools and so on, much smaller slices of the pie.  In fact, if you look at just those slices, only about 5 percent of overall K-12 funding from those 48 foundations goes to these slices; 95 percent of their K-12 funding is going either to private school grants or to things that have nothing to do with school choice at all. So the bottom line of all this is that on the one hand there's a lot of giving that's going into school choice related activities.  On the other hand, it's not very broad-based.  There are a lot of foundations giving some, but if you look at the dollars, it's highly concentrated in just a couple of foundations. That has a lot of interesting implications.  One positive aspect of that from the point of view of the school choice funders is that it's pretty easy for school choice funders to work together.  There's not very many of them that are making large investments, and that can be helpful for funders that want to be strategic and collaborate and move things in a certain direction. That also raises questions, though, about the extent of power that these funders have within this movement when they can all really get in a room, and actually do get in a room sometimes, and talk about where things are going to go. I want to shift gears a little bit from the numbers and talk about what have these funders that have invested in this field done over the last 15 years. Let me apologize.  I'm about to show you a really crazy picture.  It's not crazy because I'm crazy, although I may be crazy, but it's crazy because this is actually what's been going on in the choice landscape.  What Jay mentioned in his talk about the buckets into the ocean, that sense has really motivated a lot of the funders that have gotten into choice-related philanthropy.  They've thought if we give our money to regular school district oriented reforms, we're pouring our buckets into the ocean.  What we're looking for is a place where we can have a real impact and school choice is that place because we can create new schools, we can create new institutions that are different from the start and meet kids' needs better. As a result of that orientation that a lot of choice funders have, what you see in the choice funding landscape is a lot of institution building, a lot of the creation of new kinds of organizations, and that's why this map is so crazy. I didn't coordinate with Jane, but I decided to go with the supply-and-demand framework here, too, because what's been going on in this area is an effort both to create the supply of new options for kids, but also to work on the demand side.  It's a little more complicated.  I'll come back to what I mean by the demand side in this case. But on the supply side, of course, the core of it is funding the creation of new schools, and there has been a great deal of activity in that regard both to fund single schools such as new charter schools, but also to fund the creation of networks of schools such as KIPP.  We heard from Mike Feinberg earlier and many other networks that are trying to run many, many schools across the country or across a state or across a local area. The Walton Foundation alone has provided grants to about 10 percent of the charter schools in America, and many local funders which numerous to list or even know about have made small grants to help charter schools get off the ground as one example. Also on the supply side, a lot of grant making to what we call support and advocacy organizations, organizations that aren't themselves schools, but exist to help schools in some way:  charter school resource centers, for example, charter school associations, organizations that help schools with their facilities problems or special education or any of a number of other needs that schools have especially when they're not part of a traditional school district.  So a lot of activity related to support and advocacy, all of that designed to build the supply of new options for kids. On the demand side, I'm going to talk about demand in two respects.  One is, of course, families.  There's a lot of evidence that there's plenty of demand for new options on the part of families especially in inner-city communities, so it's not so much a matter of stoking demand as it is supporting it in some way.  For example, by providing scholarships.  Organizations have been set up.  The largest and most notable is Children's Scholarship Fund which is currently providing scholarships to 23,000 children across the country with private funding.  So scholarship organizations exist at that level, but also in many smaller-scale levels around the country. Another area of investment here has been in information services, providing, for instance, web-based information through organizations like greatschools.net and Standard & Poor's School Evaluation Services now is called schoolmatters.com.  It could be dot-org, I guess, but dot-com.  Maybe S&P is going to go with dot-com.  So information services for parents. Another important area of investment on the demand side has been what we call grassroots organizing.  If you look at the Milwaukee story which is an important of the school choice landscape because it's got the largest publicly funded voucher program as well as charter schools, as well as within-district choices, one of the big parts of the story has been philanthropic support there for efforts to organize low-income parents to advocate for the creation of choice options and for the maintenance of them within the political arena over time.  You have the specter there of very conservative, straight-laced funders getting together with grassroots organizers who may ordinarily be demonstrating against them or their companies on other issues to work in this very interesting political coalition around grassroots organizing.  So that's one side of the demand side where a lot of institutions have been created. On the other side of demand is getting policy makers to demand more options.  This goes to Jay's overall point, I think, which is choice funders have seen from the beginning that they could create private scholarships, they could create new schools or fund the creation of new schools, but ultimately the success of their idea is going to hinge on whether they could get policy to shift to fund grand making or to fund tuition support for private schools or to fund the creation of charter schools and the maintenance of charter schools. So grassroots organizing, of course, fits in there, but there's lots of other institutional development under this heading, too; for example, legal defense funds.  Many of the laws, especially voucher programs that are designed to private tuition support have been challenged in court, charter school programs have as well, and through institutions like the Institute of Justice and through local defense funds, philanthropists have made a lot of contributions to fighting those legal battles; some very strategic support from think tanks and research organizations designed to support the idea of school choice. One of the seminal investments here was funding from the Bradley and Olin Foundations for the book that John Chubb and Terry Moe wrote called Politics, Markets and America's Schools, published in 1990 and thought to be one of the intellectual kickoffs of the current era of school choice.  And lots of other research activity in this area from some high-profile institutions like the Manhattan Institute, like Harvard's Program on Education Policy and Governance, and many other institutions as well. Some philanthropists have ventured into more direct political ventures.  Often this can't be done by the foundations, it can't be done by the straight nonprofits, but many of the individual philanthropists have given money to political action committees, have given money to candidates, and create organizations like a national one called All Children Matter, that are designed to directly intervene in the political process to get people elected to office who support these ideas, so much more overt political activity in this area. Then finally, those organizations I mentioned before also play an important advocacy function.  For example, most states that have charter schools have a charter school association.  One of their jobs is to go to the legislature and try to expand the law or keep it the way it is to make it a good place to do business as a charter school.  A crazy map, but simply resulting from a great deal of institution building with this sector largely philanthropically funded. What does this all add up to in terms of the impact that philanthropy has had in this area?  On the one hand, certainly it's a story of success at one level.  There has been a vast expansion in the kinds of choices that philanthropists set out to support 15 years ago.  So we have on the charter school front, for example, perhaps a million kids now attending charter schools, 3,400 of them or so nationwide.  Almost all the states and certainly all the large states have charter school laws on the books. On the voucher side, not as dramatic of an expansion success, but there are now four states and the District of Columbia that publicly funded scholarship programs.  There are several states that have various tax advantages if you make donations to an organization that gives scholarships.  So not the numbers that you have with charter schools, but certainly a lot of expansion on that side of the map, too.  Also a huge increase in the amount of choice that's going on within existing school districts. So on the one hand, a lot of expansion.  On the other hand, many concerns on the part of philanthropists:  Have we achieved the scale that we need to if we want to, A, serve the kids that we're trying to serve with these options?  And B, if we want to have a broader impact on the system which is one of the aims that many school choice funders express. Secondly, many concerns among many philanthropists about whether the quality of the options that are being created are really up to snuff and whether they're really furthering the aims that they have as schools. MR. HESS:  Thank you, Bryan.  Andy? MR. ROTHERHAM:  Thank you.  My project was to take a look at investments in public policy research and advocacy.  For the purpose of this I lumped them together because it's a very baseline analysis because I think as we have heard a lot, getting ahold of data and so forth here is very difficult, and Bryan and I experienced some frustrations on that front. I have to thank Rick.  This is obviously the most self-serving paper I've ever been asked to write in my life since I make my living analyzing policy and writing about policy and coming up with new policy ideas.  However, despite that and all of that notwithstanding, I still decided that we need more investment in research in policy. My presentation this morning is going to move relatively fast for two reasons.  One, not to run afoul of Rick's strictness on time, and there's plenty of detail in the paper and so forth.  Second, because a lot of these issues have been discussed.  For example, I have a discussion about Jim Kelly's efforts at Ford with school finance and so forth and we've been through that. Like everyone else, my funders are prescient, brilliant, wonderful, good-looking, especially the ones in this room.  What I did in the interests of disclosure, if you're interested, you can take a look in the back of my paper.  I listed everyone.  We are a soft-money organization.  I rely on philanthropic support.  I listed all the funders who have supported our work over the last several years, and I'll leave it to you decide whether or not I have Stockholm syndrome or something else. First of all, there are three primary reasons why grant makers can play a key role in research and policy.  First of all, quite simply, it's how we codify change.  We've heard again and again here that policies interferes and interacts with various reform strategies.  That's fairly basic.  I do believe schools change classroom by classroom, if you go to most state capitols in this country and turn around 360 degrees, you're going to see the headquarters of the state teachers' association.  They may well believe that change classroom by classroom as well.  They also clearly believe it happens in state legislatures and statehouses, and they have a presence there. This is the process by which we ultimately codify change to public policy with the political process.  Funders have to plan it if they're looking to make long-term change. Second, there are insufficient incentives for good research and development, good policy work and so forth in education.  For starters, this is not the pharmaceutical industry or the defense industry where there are big payoffs awaiting big breakthroughs.  With the exception of very few microareas of our field, there just aren't huge profit incentives. Second, there is not the relentless discipline to do R&D that you see in other fields.  For very good reasons, the public schools are not going to go out of business tomorrow depending on their R&D operations so you just don't see those incentives. Finally, there is insufficient federal, state and local investment.  It's almost negligible at the state and local level, and the federal investment is less than one-tenth of 1 percent of our overall education expenditures, so if you look at R&D across various sectors, it doesn't add up.  In fact, if you look across the federal government, the Department of Education actually spends less on R&D than a lot of other comparable agencies. Why does research and policy matter?  Again, it's how we codify change, as I said.  In their book on Tinkering Towards Utopia on change, Tyack and Cuban make a very famous characterization of the kind of changes that have endured.  They observed that those are changes that are add-ons, they are noncontroversial, that have influential and organized constituencies behind them, and that can be monitored through statute or regulation.  All four of those characteristics are obviously dependent on policy and public advocacy to one extent or another. Second is the politics of education.  This is a highly politicized field.  That's no great secret.  The policy process in education and, in fact, in most other fields is not a vigorous free-for-all between competing interests with the most favored ultimately winning out.  Instead, various actors in this debate enjoy different degrees of access to policy makers, different degrees of input on key decision-making organizations, organs, boards, state committees and so forth. So while outwardly it looks like we have this vigorous free-for-all and an open debate, in fact, you have some interests that are much more advantaged or disadvantaged in the debate.  For their part, foundations both historically and today are often supporting the very interests that are disadvantaged regardless of what you think about the issues specifically.  Some of the stuff that Bryan was talking about, these are interests that are disadvantaged in a way that the policy process has made today. Finally, educational reforms are difficult because they're general interest reforms, and that means they generally carry a broad benefit for the public at large, immediate benefits perhaps for affected students, but they carry costs for one constituency or another.  You can see this locally, for example, if you want to build a new school or introduce a new reading program that might require an increase or taxes or a bond issue, you'll see local tax activists that will become energized.  They'll go out and fight about that. In the case of a charter school you'll see people energized because it might cause a drain in students.  In the case of a mayoral takeover it will cause certain constituencies, certain folks to lose power. The good rule of thumb in politics is that organized interests bearing concentrated costs tend to win out over the general interests.  That's one reason, for example, we reform the tax code about every decade and we go in and we clean up all these special-interest provisions, and then a few years later it's all cluttered up again.  It's the same reason if you look at agriculture subsidies, many of them are wildly inefficient and almost comical in some cases. Education is no different.  Unfortunately, too often we don't look at it through the same analytic framework as these other fields and we think that our special interests in education are somehow more enlightened than what you see in other fields.  Politics is politics and it's the same thing here, and that's why you should not be surprised when there's an attack launched on Wendy and TFA or an attack launched on Mike Feinberg and KIPP.  What would actually be newsworthy is if these attacks weren't being launched.  This is how politics plays out in our country and foundations need to be involved in that if they want to see these changes to forward. I hope already I've convinced you in just a few minutes that you need to send more money to my colleagues and to me, but if not, again, thanks to Jay Greene for some of the help with the data here. The top 30 grant makers in K-12 education spent about $77 million in 2002, or about 12 percent of their grant making on research, policy, advocacy, all those things put together.  The top 10 grant makers, it went up to 15 percent, but only four in the top 10 were investing more than one dollar in five in these activities, and few funders in the survey that we did of the 51 largest foundations indicated that they saw substantial change in their giving patterns one way or the other over the next several years, although it's worth noting that in particular Bill Porter is here and Stephanie Saroki from the Philanthropy Roundtable, both indicated that among their members they are hearing more interest in looking at these issues, but the program officers themselves said in terms of immediate plans to change their patterns, they didn't see them. Of course, $77 million is obviously not an insignificant sum, but let's put it in perspective.  The California Teachers Association to fight against Arnold Schwarzenegger's initiatives there is raising $54 million.  If you look at the millions that were poured into the ballot referendum on charter schools in Washington State, if you look into all these different issues, $77 million at the end of the day in terms of what it takes to run even a small nonprofit on an annual basis in order to do this work, it's actually not a great deal of money. Why is there not more?  First of all, legal constraints.  Foundations operate under a variety of constraints intended to limit political activity, increase transparency and so forth.  That's a major issue.  Second, they can't lobby except in very carefully proscribed situations.  They cannot give funding explicitly for lobbying.  These are real constraints in how they operate, but a lot of the foundation staff we talked to felt that even beyond these explicit constraints, it's created an environment where foundations are very leery to do anything that's even perceived as potentially running afoul of the law, let alone whether or not it actually does either of the spirit or the letter, and they felt that these added constraints. Second, these politics.  Inside and outside the philanthropic community people said the politics are tough.  In fact, several funders indicated that one of the reasons both higher ed and pre-K are more attractive is because the politics are not nearly as difficult as you get in K-12. Paul Hill I think put it very succinctly in the paper.  He said, and a lot of foundations and program officers didn't disagree with this, that when they find that taking on the really hard issues is really difficult and not taking them on doesn't get you much, they tend to back away.  And in sort of a typically pithy comment, Checker Finn noted that most foundations, particularly corporate foundations, want to be on the front page being praised and not on the editorial page being lambasted, and that's a natural instinct. Finally, there's a lot of concern about where to give.  There was a lot of concern expressed among colleges of education, about think tanks, among other research organization, federal research organizations and so forth.  There was not a lot of confidence in either the rigor, relevance or ideology of a lot of work coming out of schools of education.  There was a lot of concern in the think tank world, all the work is just confirming preconceived notions and so forth, that it's essentially back-filling rather than moving forward.  And concern about all those issues with regard to federal programs. Then one other thing that I think is very interesting particularly in light of some back and forth we just saw this past week, concern that you could invest in the best study, the best policy development activity and so forth, and it can be swamped by one of lesser quality that's more slickly marketed.  There was real concern about that in terms of foundations from the point of view of trying to get good return on your dollars that this could be an area where you could do everything right and you could still get swamped by something else. In addition, there are two different theories of change at work.  Some foundations have a bottom-up orientation where they think you're putting the money at the bottom, you're causing change, and that's ultimately going to impact the policy process.  Conversely, though, several smaller foundations indicated that, first of all, they don't feel they have enough money to make that kind of change at the bottom, and that for their limited dollars they feel it's much more important to try to make those investments in policy. One program officer of a smaller foundation, it was interesting, put it in terms of her board and they think about it in terms of probability.  They said if you want zero risk, you can fund a program that affects 100 children, and that's terrific.  But if you can tolerate risk and think that there's even a 40 percent chance you can help 100,000 children through a policy-oriented grant, then that's a reasonable bet they feel for their foundation to make. But that runs afoul of what drives a lot of people into giving which is things that, first of all, are tangible and concrete.  You can go and see a school that's been adopted, a professional-development initiative at a small school.  Second, it's that risk tolerance. In closing, there are four models I think where you can see where foundations have made a real difference and four models that you can follow here.  The first is policy networks, then issue advocacy, agenda setting and building downstream capacity, and I'll talk about all four real quickly. Policy networks are simply formal or informal groups of key policy actors, academics, advocates and policy makers working together to address an issue.  Susan talked about probably what I would say is one of the best examples, the work that Jim Kelly undertook at Ford.  What you're talking about here is the same issue but different solutions.  What was very interesting about the Ford effort is, for example, they were simultaneously funding Jack Koons, the Berkeley law professor who is very liberal, probably almost a socialist.  He's a great guy and a good friend, a libertarian, but they were funding Koons and he was promoting vouchers.  They were also funding the American Association of School Administrators who obviously were not, with multiple solutions to the common problem of finance equity.  So again, the Ford Foundation and school finance I think is the best example. Today if you look around, Hewlett and the work they're doing out there in California is a real example of this.  Mike Smith has likened the work to a campaign, not in the sense that there's an end date and an election, but a campaign in terms of you're trying to change hearts and minds, you're trying to change how policy makers think. Second, issue advocacy which is the same thing:  formal and informal groups of policy actors and academics, but instead working together to address an issue, multiple groups and actors, but one solution issue.  I think the best example of this over the past 15 to 20 years is what's happened with Bradley and some of the other foundations on school vouchers, and we've talked about that. I actually dissent and I'm personally a skeptic on it, but in terms of just looking at it a neutral way, I'm a skeptic that this is an issue that hasn't gotten very far.  It's been an enormous policy change over the last 15 years.  When you look at what states are doing and the direction this is going, it's left to be cleaned up, but the debate is almost settled on that issue. What's noteworthy is Bradley was funding a variety of groups, from groups that shared their conservative political philosophy to a number of really liberal groups on the ground in Milwaukee and so forth to drive that.  They were funding people who were supported the issue regardless of where they were on other things. Then agenda setting, moving an issue from a marginal position to a mainstream one, flooding the zone around an issue.  The best example of this is obviously Gates on high schools.  That issue has gone from being a marginal issue on the nation's policy agenda to a very, very central issue.  Smaller foundations can obviously do that in localized communities.  Very few foundations can do something like that nationally. There are obviously up sides and down sides.  Gates has been criticized, What right do they have to set the nation's policy agenda like that?  In addition, the foundation's program officer was very interesting, he said they feel that sometimes when an issue becomes branded like that, it actually dissuades other giving.  So as opposed to what Jim Kelly was able to do at Ford where you saw Carnegie came in and the federal government came in, some people feel that this kind of branding actually dissuades people from wanting to come into an issue because it's seen both substantially and politically as taken. Then finally, building downstream capacity, supporting research, policy, entrepreneurs, analysts and so forth.  Spencer is definitely leading on this, and there's a table that looks at the grants they fund.  The question is, without getting into various kinds of research, is the work that's being funded right now actionable in the sense that policy makers are looking for tangible solutions, ideas, things that they can take to address problems?  And is the work that's being funded today, and I think this is an important discussion for the philanthropic community and the research community, relevant and policy makers can use, is it actionable, what's the utility of it, and I think there are some real open questions there in terms of what kind of capacity is being built particularly in terms of the problems that you look at with the mainstream foundations are trying to address. So quickly, research and policy matters.  Right now it's still a relatively low priority for foundations today despite how integral it is to their theories of change and what they're trying to do.  And despite the challenges, there are clear models of ways that you can make a difference.  This is certainly not an exercise in futility.  Thank you. MR. HESS:  Thank you, Andy.  We have about 15 or 20 minutes for Q and A. MR. REICH:  My name is Rob Reich (ph).  I'm from the political science department at Stanford. This is a question mainly for Jay, but I'd welcome a response from anyone.  The title of the conference is Philanthropy in K-12 Education, but we're heard almost entirely about foundation giving.  There's a hidden phenomenon beneath it all about the individual giving that sometimes in schools, typically in suburban schools, can amount to quite a good amount that is also philanthropic in nature. I'm wondering if, Jay, you've taken into account how you might have underestimated the amount of individual giving, nonfoundation giving, that goes to typically an LEF or a PTA or PTO that then funnels it to a school or school district.  I've been trying to collect some data just in California on this and have tallied 3,000-plus organizations, all 501(c)(3)s doing it. Using your metric to evaluate how effective this might be, buckets into the ocean when you cash this out on a per-pupil basis, there are 20-plus districts in California that are getting over $1,000 a year in private money per child.  You can point to some that are almost doubling the public dollars in the school. I'm wondering is that something that we want to celebrate and don't we want to take this into account at least, or is it something to worry about because it's going to exacerbate typically already existing funding inequalities between schools? MR. GREENE:  Let me just ask you a few questions. Twenty out of 300 in the one year you looked at--over a 6-year period, gave $1,000 on average over the 6 years?  Is that what you're saying? MR. REICH:  Per child, right. MR. GREENE:  Per year? MR. REICH:  Per year. MR. GREENE:  So in 6 percent of the 300 districts, and the 300 districts you selected, were they at random? MR. REICH:  No, it's in all California districts. MR. GREENE:  Of all California districts, and how much was being given in the other districts? MR. REICH:  Typically between zero and $20 per pupil. MR. GREENE:  What was the average in the entire set of 300? MR. REICH:  The median? MR. GREENE:  Median or average.  Both would be useful. MR. REICH:  Below 50. MR. GREENE:  Below 50.  So basically your numbers are consistent with my numbers, where when I surveyed big districts they report $27 per pupil, so it's about the same. MR. REICH:  Right. MR. GREENE:  From all sources.  There is a problem where people feel like they could make a bigger difference with their direct dollars, with private dollars, because they repeatedly fall into traps where they think that their dollars are more plentiful than they really are. One of them is to focus only on places where it's more plentiful and ignore average, I guess.  But other dangers are the philanthropic organizations are sometimes engaged in puffery, and part of how they do this is they announce multiyear grants at once, feels bigger, and then they also sometimes just simply overstate their activities. For example, The New York Times had a piece where they claimed that there was $9 billion in philanthropy.  As it turns out, there's a $9 billion item in the federal data for other revenue which does include philanthropy, but includes all other nongovernmental revenue including rent, tuition, lots of things. The New York Times went to the PTA which is a national organization and the PTA supplied them with information indicating that they had given a total of $10 billion, just the PTA.  I find it unbelievable.  It's not possible for the PTA to have done that.  But they wanted to convey to The New York Times that they were a big organization doing a lot of things. There are good intentions here.  Everyone involved in philanthropy has good intentions, they want to do good things, but you can fall into a trap and one of the traps is thinking that you have a lot more money available than you really do and the danger of that is that you forget that you need to redirect public dollars if you want to make a difference across the board. MR.          :  I'm not trying to dispute the overall picture of your findings but just ask if you're aiming to tell us is that it's buckets into the ocean, there are places in which it's not buckets into the ocean.  They may be far and few between and you can find places where it's not the foundation giving that's doing the work, it's the sum total of individual contributions where it's very, very large buckets. MR. GREENE:  I believe that there are places where private philanthropy is making a very big difference, and so you can have quite a big local impact. But that impact is limited in two important ways.  One, it's limited by place, that you're only making a difference where you're giving.  Second, and this is probably even more important, is that you're only making a difference as long as you're giving.  When you stop giving you're no longer making a difference unless, of course, you redirect how public dollars are being spent later. So, yes, in those districts there may be someone or a bunch of people who are stepping in and giving a whole lot to those places, but there are lots of other places where they're not having an effect, and even in those places the effects may stop after they stop giving which goes back to what we heard earlier this morning where people were saying, why don't some of these things have lasting impacts?  Why do they fade?  They fade because when you stop providing the private dollars, if you haven't changed the barriers to using public dollars for these activities, you'll revert to how things were before. MR. COMUS:  Jim Comus (ph) from the National Educational Knowledge Industry Association.  I was really pleased that, Andy, you focused on the investment in R&D in education.  I was going to ask you a further question that's related to what Jay has been talking about, and that is this use of high-leveraged funding to try to change the way that public dollars are being used.  Heaven forbid that research is not fully shaping policy right now in a way that could be having a more dramatic change in the overall environment. Andy, can you speak further to why you think there is such a dismal investment in R&D in education? MR. ROTHERHAM:  First of all, Jim, I'm not surprised you agree with me on that point. I think we could have a whole conference on that.  I think there are a lot of reasons.  There are a bunch of reasons why there's not.  I'm going to speak instead to why I think it's not penetrating. First of all, a lot of the work is not relevant to the main issues policy makers are facing today.  They don't have the luxury of examining these large issues in abstract and esoteric ways.  They need to know what to do now.  The appeal of Teach for America, for instance, is not because policy makers are not interested in Teach America because of some grand debate about the nature of the teaching profession, they need good teachers immediately in underserved communities where in some cases in the absence of TFA there are no teachers.  So they're making those decisions with a gun to their heads and a lot of the work that's produced in a lot of places simply isn't helping with those problems. Secondly, very simply, politics.  This is a hugely politicized field and people don't want to confront that head on.  As I said, we refuse to apply the same analytic framework; the NRA produces a study and no one things it must be fine because it's the NRA and they would never, ever--we submit it to great scrutiny.  But education studies, so-and-so is producing it, it must be okay, it must be on the up-and-up and so forth, and no one wants to apply the same lens that we all ourselves would apply unhesitatingly when we think about other issues to education.  Good intentions are important, they're not enough, and there's a huge political angle.  For funders, the punch line for today's conference is, you can't get anything done if you're not willing to engage with the reality that the politics are going to go on every day whether you play in them or not. MR. HESS:  Andy, on that front, I had a question submitted.  The question is, Are foundations themselves actually impervious to evidence?  Are they actually going to change their strategy?  What types of evidence are actually going to persuade foundations to alter or change funding strategies after they've issued press releases and issued years of annual reports and of board members have been told that this made a lot of sense?  What evaluations are actually going to cause foundations to do something differently because the numbers came up? MR. ROTHERHAM:  Speaking for myself, and this may be where you decide that I suffer from the Stockholm syndrome, the foundations we work with, I think they're remarkably transparent, they invite people in for real critical sessions to discuss these issues, and they do change; they look at where their bets and paying off.  I think a point Bill made is that this is a huge field and so I'm very leery of making a--philanthropy does this or philanthropy doesn't do that. I think a lot of the ones we work with are willing to take a look at the evidence, they are willing to change their minds.  They're in many cases large institutions, too, and so you have all the issues of how institutions change, and that can be slowly and I'm not nearly as much a cynic that it's not, but obviously that's a contention that's arguable. MR.          :  I'm a little more cynical, I think, not that no foundations will change based on evidence, but probably most of the change that you see in philanthropy over time is because there's always new funders entering and bringing new ideas.  New wealth is being created and then turned into philanthropy.  So that's another potential engine of change that doesn't require the change of person's mind, but requires new people to enter the field. MR. ROTHERHAM:  Let me also add this is also very subjective.  For instance, we can look at the school choice issue and you can look empirically at the research that gets done and whether or not vouchers had their effect on achievement.  But whether or not you think vouchers are good public policy, that's a--issue and so it's very easy to say they're not supporting vouchers, well, they're impervious to evidence.  Some of this stuff is he's impervious to evidence and he's not-- [End tape 3A, begin tape 3B.] MR. ROTHERHAM: [In progress] --everybody has views on these different issues. MS. LEE:  Beth Lee (ph) from the Institute for Learning.  I'm going to ask a question of Jay, although relying on some of the things that Andy said. Taking your argument about the need to leverage given how small the percentage of private dollars is with regard to public, and given Andy's point about the positive R&D money, I looked at your paper, and your largest bucket on low leverage is professional development.  I wanted you to speak a little bit about why you chose to label that as lower leverage and not higher leverage. I can understand if it's money to do the same kind of professional development, but if it is to change the way professional development is done, the content of professional development or the subject area, then might one argue that that's really high leverage? MR. GREENE:  Something is high leverage if it has a reasonable chance of redirecting future public spending, and the problem with professional development is that it's an activity that schools do engage in on their own with their own resources and can engage in on their own with their own resources.  So if you give them additional resources to do a different type of development than what they've done, what are your odds of changing how they spend money on development in the future? Not very good because, first, there were reasons why they weren't doing what you wanted them to do before.  Those reasons still exist.  They exist at that place and they exist at all other places.  So you don't have a very good chance of changing professional development anywhere else where you're not giving and, second, you don't have a very good chance of permanently changing professional development after you've stopped giving. Let's say you believe professional development is the key to educational reform.  I think that's a perfectly reasonable position.  But if that's what you believe, then I think the higher leverage, the higher impact way of getting at that, is by funding research and advocacy that will produce public spending on the kind of development that you want, and simply giving it to the schools has a much lower change of producing that effect. MR. HESS:  We have time for one more before lunch. MS. GILL:  Libby Gill (ph) from American Institutes for Research. In your work, and a question to all of you and maybe Jay, with the interest in really looking at significant leveraging for sustainable impact, I'm wondering if you saw any patterns of giving in relationship to the longevity of the investment, if you will? So if we look at grants that are provided for this purpose, are we looking at any expectations, 5 to 10 years, 3 to 7 years, so that there is some kind of consideration for the amount of time that is needed to look for the appropriate evidence base? MR. GREENE:  You're asking is something higher leverage if it's given for a longer period of time? MS. GILL:  I'm wondering if you saw any correlation in that area and if there are any patterns in all of the work that all of you have reviewed. MR. GREENE:  I don't have data on this over a long period of time.  It was pretty hard doing it for 1 year, so I don't know how things have changed. I do have some informal sense of change.  I think that some of the bigger foundations are beginning to realize that earlier lower leveraged tactics were not producing what they wanted.  I think Gates is a big example.  They were not at all involved in research in advocacy early on and they did a lot of giving straight to the needs which felt good, and they thought they had a lot of money, and they do, but they weren't getting what they wanted. They noticed that it wasn't happening in other places where they weren't giving.  That was frustrating.  They noticed that things didn't necessarily continue after they stopped giving.  So they've begun to think that they need to shift the mix of what they're doing more towards higher-leveraged giving.  In fact, by the time I collected the data for 2002, I think they had already shifted quite a bit towards higher leverage. MR.          :  I picked up--Libby, and I hear what you're saying--give more money to us, you should give it for much longer periods of time. Seriously, I hear from some program officers in terms of research and its impact on the policy process and policy development work, because it's so uncertain, we're still fighting school finance battles now 30 or 35 years later.  We're still fighting on the school choice debate 15 years later.  This time takes time, that for some funders they felt that that added to the tendency is to want to give to things you can touch, you can feel, you can go see, and this is really amorphous and it's uncertain.  Legislators can change, political control of statehouses can change, political control here can change, and that there's so much uncertainty that those time horizons that you really need to do this kind of work, some funders thought it was real difficult for them in terms of their organizations, their boards and so forth. MR. HESS:  With that, that's something that's a great note to end on for the morning session. Libby, I think again your point points out in terms of these papers that what we're really doing here is scratching the surface and raising a whole bunch of interesting questions that we need to then do more with. I'd like to thank the panelists for a great job. [Applause.] MR. HESS:  We're going to reconvene at 1:30 with the discussion panel on this. [Luncheon recess.] A F T E R N O O N   S E S S I O N [Tape change.] MR. HESS:  [In progress] --then we'll have a break.  Then we're going to have the Lessons Learned Panel and then discussants.  And then we're going to have the final panel with Dan Fallon, Stephanie Sanford and Checker Finn, seeing if they can pull any threads or summation out of everything that we're thrown up on the wall today. I want to thank you for being so cooperative and so great about this so far.  I know these things can be trying and I appreciate you being so patient and so helpful with it.  Again we're going to try to make sure we get questions from everybody.  Morgan is keeping an eye for folks back there. This session is going to run an hour.  There will be discussants who will be talking and are going to share some thoughts on previous papers on approaches to giving. To my immediate right, of course, is Jim Kelly, and Jim has already--from the audience about three times today.  Jim is currently an adviser to education organizations and government agencies.  He is former president and CEO of the National Board for Professional Teaching Standards. Next to Jim is Arlene Ackerman, Superintendent of the San Francisco Unified School District, and as local folks know, also a former Superintendent of D.C. Next to Arlene is Bruno Manno, Senior Program Associate for Education at the Annie E. Casey Foundation. To Bruno's right is Kim Smith, co-founder and CEO of the New Schools Venture Fund. What we're going to do is go in the order you see them seated before you.  So Jim, why don't you go ahead and get us started? MR. KELLY:  Thank you very much.  Is my time up right now? MR. HESS:  You have 30 seconds. MR. KELLY:  I want to commend AEI for having this conference.  I think the conversation is very constructive and timely and is constructively designed and implemented and I hope that it leads to a new intellectual, professional and political space in which we can all ventilate about this topic and keep working on it. Second, I appreciate the kind words that were said about me earlier this morning by Jane Hannaway and Susan Fuhrman. Because I'm old, I've had a long history with foundations from a college scholarship to getting grants later, to bring liberal arts graduates from outside education into a doctoral program at the Teachers College at Columbia, University, in educational leadership, to be for 11 years the program officer at the Ford Foundation, to being a substantial grantee of many wonderful foundations, many of them are here, to being married for 25 wonderful years to the President of the Community Foundation for Southeastern Michigan, and so I'm a grizzled veteran for whatever it is worth. But at the present time it is wonderful in my life that I do not have a job, and I do not have a grant from any foundation, and I like both of those very, very much. Next I would like to rise in defense of the young and naive.  It's one thing for older, wiser people who actually think they know something to take on the system and say we're going to do something that nobody has ever done.  I commend Wendy Kopp for doing it as a college senior.  When I was a college senior, my mind was a lot of things other than recruiting smarter people from liberal arts colleges to become school teachers in the United States.  With that introduction I will move on. Just a couple of things about the context of this which I hope will be borne in mind as we all move forward in this conversation.  While K-12 is mainly public as an enterprise, private giving and volunteerism is a defining characteristic of American life.  It's very important in higher education, not totally unrelated to public K-12 education, and it is very important in the wide open, some way wild market, of preschool education in the United States. Most giving in the United States is not given by organized philanthropy foundations or corporations.  It is given by private individuals who decide to give their money to some charitable cause.  I believe that the statistics are somewhere between 80 and 90 percent of all charitable giving in the United States is given by individuals, and a large percentage of that is going to the education sector broadly defined. It's good to start with K-12 education and foundation philanthropy, it's a very special and narrow case and I'm not sure that we want to stay with that assuming all this conversation can be kept going. Next I've tried to think of the seven or eight cases that really have mattered in American life that touch on education that foundations have been responsible for.  Dan Fallon didn't make me do this, but I have to say that Carnegie is involved in a large number of them.  They are in 5 seconds each, Carnegie Libraries, TIAA-CREF, already mentioned, Gunnar Myrdal's seminal book Changing America: An American Dilemma, addressing race relations in the United States just before World War II.  They didn't create a movement.  They didn't create a committee.  They didn't have it all evaluated by a bunch of scientists.  They had one wonderful, inventive, smart, wise man write a book that was so thick.  If you haven't read it and you want to understand America, wherever you come from, you have to read it.  The Flexner Report, Ford with its work with Paul Ylvisaker in Gray Areas in the late 1950s.  Carnegie again with Sesame Street, and I'll stop there. There are six points that I would like to make in my remaining moments.  The first of them is that there's a heavy focus in the tone of this conversation on urban school districts and a lot of talk about scale.  I agree it's hard to throw a few dollars out into a vast sea. On the other hand, most kids in America don't go to what we call urban school districts.  They go to suburban of rural school districts.  In fact, there are more poor kids in rural areas in America than in cities.  It may be that at least we should build into this conversation the possibility that philanthropic dollars and leverage can be more efficacious in establishing new models and trying them out in some smaller jurisdictions that have somewhat less difficult bureaucracies to work in than the multilayered splendors that we find in these descendants of Max Weber inside large school districts.  Arlene will tell me that I'm all wrong on that point.  That was my first point.  A lot of stuff goes on in urban and rural areas. Second, with regard to strategies, most foundation grants programs in K-12 education in my experience have started with the answer.  I certainly lived through a whole lot of answers up and down the hall at the Ford Foundation in the years that I was there.  We know what the problem is.  Basically, we know how to fix it and it is this very particular thing.  It's leadership or its technology or it's research or it's desegregation or it's choice.  This is the typical posture. Not starting with the answer is not typical, and one of the things that characterizes the Ford Foundation Program on Education Finance Reform in the 1970s, whatever you want to say about it, was that we were interested in promoting study of an issue, developing intellectual, political and human resources to work on that issue in a sustainable way, but not to come up with the answer in advance.  Never was there a Ford Foundation position on what the education finance systems of states should be. Third is localism.  I know this is un-American to come to Washington and talk about localism, but a large part of philanthropy in education is in fact local, and it will not be seen and observed and studied if all we do is look at the big national foundations.  I could say a lot more about that, but I don't, but local and state missions of foundations, it's not just community foundations.  My wife's foundation gives away $20 million a year for a population that's half the state of Michigan, but she represents 650 donors up to now.  She'll correct me because there are probably some more this morning. The point is it's very much woven into the fabric of that and you don't capture that with the big stuff.  So there is a huge amount of activity that is in the local and state areas.  It's not just community foundations.  There are private foundations of very large size that have decided to focus entirely or primarily on an area.  Think of Lilly in Indiana and in Indianapolis.  There are many examples, some no doubt represented in this room.  It's a different point of entry into this field through which to look at it. Fourth, an important aspect of grant making in education that has not yet been discussed but is I think very important and undervalued in the field is the way in which grants are used whether used consciously or in fact have the effect of creating, validating and enhancing careers of people.  I think that it's not possible to sustain a reform in American education, given all of the institutional inertia without having a gang of people who know what they're doing and why they're doing it and can work together.  I think it's a very important aspect of grant making that is a different kind of a dependent variable than the ones that are typically discussed.  That's my fourth point, people. Fifth is implementation.  All kinds of strategies and values and position papers, and all of our eyes glaze over at one more foundation position paper. Once you have a program and even if you have a clever strategy, a huge amount of what happens depends on your skill in implementing that idea through the real-world institutions of American education.  Dealing with the real world over time is not just incidental to whether a program actually succeeds or not.  Usually the initial design of it is about 10 percent of the work and the rest of it is just dealing with the real world as you move through the process. That means to have an impact in this field, longevity of commitment is obviously a factor and scale is a factor, but a lot of foundation programs these days are run by people who have not been responsible for trying to make the system change in some way before they went into a philanthropy.  That is naive if not young because it just doesn't change by wishing it.  The system is expert at swallowing small amounts of money as we have said earlier today and saluting that flag and going on with business more or less as it should because it is designed by state constitutions not to be penetratible by outside influences like nefarious influences from foundation people who have new ideas or whatever.  It's designed to be ostensibly a conservative institution. Fifth is new ideas.  I would preface this by saying I'm in favor of standards and movement on choice and more movement on policy research and so forth.  But in my personal opinion, this is the field that needs more big ideas being tried out and cultivated.  We need a pipeline for new big ideas. The big idea of having to have technology substitute for labor in this field is the way almost every institution in American life has become more productive.  I ask for you to think about that, the basic structure of schooling.  Other panelists have talked about this today.  If you look at it, there's a room about so big and there's a teacher and there's 30 kids and that's the game.  Tens of thousands, hundreds of thousands of good people have tried to make that system do better, and that's God's work.  So I'm in favor of new big ideas.  I'll mention just a couple and the quit. One is the idea, and again Arlene is going to throw me off this platform here, right now we pay former producers more money to work in support roles.  We pick them out because they were good producers, teachers, and we say to them, because you're a good teacher, we are going to pay you more money never to teach a child again.  A lot more money.  When you have a telephone or a bathroom break you can do a lot of things.  You can go to meetings like this at AEI. I was paid 35 to 40 percent more money for my second year of teaching to be called something called an assistant principal because I couldn't afford to--could we examine ways to build more investment into production of the mission?  Or how could we draw into the I-12 foundation field better pipelines for ideas that have transformed institutions in other sectors of American life?  Why does it have to be the largest, toughest silo outside maybe the Post Office that there is? So whatever the idea is, you take yours and I'll have mine and so forth, but I do think that we need new theories at the firm when it comes to the school, and I say that as a person who has always fought for and defended and advocated for public education and I say it in the sense of tough love for public education.  But we need new theories about how to create the institutions, and so long as we start with the box and the teacher and 30 kids and the building and all that stuff, we won't understand that to create a 21st century institution we have to start with the learner, the learner, l-e-a-r-n-e-r, not the student which is a role in a formal institution, start with the learner and then build a system that lets that person learn fast, a lot, and we don't even have theories of how to do that, but meanwhile we're trying to make 19th and 20th century institutional models do a lot more than everyone ever thought they could.  Thank you very much. MR. HESS:  Thank you, Jim.  Arlene? MS. ACKERMAN:  Thank you, and good afternoon.  I'm a teacher first and I like that--thing going on here.  So thank you.  Remember that.  I want that going on as I speak. It's great to be here for a lot of reasons.  It's great to be here because I'm not there.  I miss San Francisco, and I miss my job, but it is great to be here to listen and to be in a reflective mode as I hear people talk about things that actually impact the work that we're trying to do and that I'm trying to do. I guess I was surprised though that I was asked to be on a panel because there are obviously a few things that you didn't know, Rick.  I come from a family of eight Baptist ministers and everybody in my family is brevity challenged, so there is no way I can get a presentation out in 8 minutes.  It's just not going to happen, so we're going to fight probably in a little bit. And being a preacher's I'm a rebel and grew up in the 1960s, so I'm naturally going to not follow the rules.  So I want to speak a little bit about everything that I've heard as quickly as I can, not just the panel. I used to consider myself a progressive liberal educator and person in terms of my thinking until I moved to San Francisco.  I know I'm to the right of everybody in that city, so I'm questioning my own liberalism. The other reason that I was surprised to be asked, but happy to be here, is because over the last 5 years San Francisco has gone through this amazing movement from crisis to really an urban school district that I think people are beginning to take notice of because of our serious achievement gains.  That's unusual in a large urban diverse school district. We've done this with not a lot of philanthropic monies.  You want me to talk a lot about this?  Because I haven't gotten a lot of money from a lot of the donors. So I want to start by giving you just a little bit of a context of San Francisco, where we were 5 years ago, where we are now, and then talk a little bit about my reactions to what was said today both from the former panel and then some of the people this morning. I don't fight what is, Jim.  There are a lot of things that you say that I know are absolutely true.  If you really want to talk about how we keep our teachers in the classrooms, I actually have a plan for looking at the way we compensate teachers, throwing out the old compensation structure and paying them as we pay our doctors for areas of specialty so that they'll want to stay in the classroom.  So if you want to help me with that, we have a partnership going on after this meeting. Five years ago we actually were in a crisis, and I mean facilities, finance, all the operational, and we were in an academic crisis.  We put into place a strategic plan called Excellence for All and more than 3,000 from the community gave us input not only through community hearings, but in terms of surveys on the Net and any way that we could get them out to the community. We put in place 10 educational goals, three strategic priorities.  One is a focus on improving achievement for all of our students like raising the bar, but at the same time making sure we push the kids at the top. Maybe one of the reasons we haven't gotten a lot of donor support or donor dollars is because one of the first things I said to the staff in the schools is "say no" if it doesn't get you to your intended outcomes.  I think that's really important in working with school districts like San Francisco, we come in with a plan and I think it's important that the funders look at what we're trying to do.  Somebody talked about a strategic intent this morning, and that the support needs to come there. What I often see is the funders will have a plan and they say, here we want you to take this money, and because we're poor, school districts do it and schools do it.  But the focus was here are your outcomes and that was the second.  The third priority was accountability for all. So all of our schools used the data.  We came up with performance targets for them.  They set their academic plan on that.  Then they say no or yes depending on whether or not they could get to their intended goals. Then we looked at issues of equity throughout the system, inside of our system, and we put in place major initiatives, performance targets for schools, site-based budgeting, site-based governance, systems for supporting underperforming schools, small schools' reconstitution and we called the Dream Schools, and we got very little large donor support. We did get support from some major funders who are here, but for fear that I'll forget somebody, I don't want to mention them.  You know who you are and we thank you for that. Mainly some of our biggest givings have some from private donors once they had a chance to say I want to give to this.  It looks like it's working.  Because what people are looking for is what's working.  They get really frustrated to give money, and I know you know that, and then don't see the outcomes. Five years later we're now the top-performing large urban school district in California.  San Diego is second, right Allen?  I tell him he's like Avis, he's got to try a little harder.  This is 5 years from a district that was in crisis and I'm proud to say, and I know there are Broad people here, that we're a finalist now for a Broad Prize 5 years later without a lot of outside funding. So my question as I was sitting there was why?  How did we get missed?  What did we not do to engage the funders in the research over the last 5 years to study us or to support our plan?  And what could we have done differently?  How did we get missed? As I look at this and you're listening to a frustrated practitioner who wondered in a time of, actually for the last 3 years the state of California has been in a real serious fiscal budget or fiscal crisis, how were we not able to articulate this so that we could engage more people into this process?  And how did other districts get the money and they're failing? I can point right across the Bay Area, the Bay Bridge, and $25 million went to a school that's $100 million in debt and I'm saying so what is wrong with this picture?  I don't have the answers, but I'm asking that as a practitioner. As I think about how you might be able to help people like me or school districts like San Francisco, is remember that strategic intent really does matter and don't write us off because we have a plan.  If we have a plan, that's a good thing, and that results matter.  We've been getting good results now.  This is the fifth year.  So look at the results. You have an example of $25 million going into a district across the Bay and trying to figure out, so what was going on there?  What were people thinking about? Context does matter and so does race.  Some people said it this morning.  It really does matter.  I was glad that it was brought up because to not address it and not to put that in our plans as we begin to really address how we move all students is like putting our heads in the sand. And by-in matters.  Somebody said that this morning.  So listen to the people who are closest to the students and find out what is it that they want.  This morning there was a chart up on the screen where I was interested in the fact that African-American parents were out of synch with everybody, but the reforms that are in a large urban school district, and that's what African-American families want and Latino families want especially in large school districts are different from what everybody else thinks the reforms should be for their children.  So we have to listen to what they want and make sure that whatever we are doing, we are trying to use them and the knowledge they have of what is best for their children. Avoid the tendency, somebody talked about, to get on the bandwagon.  When I say just say no, I mean just say no.  We recently were told we're not going to get funding from a major donor and I was okay with it.  I said to my staff, we're not going to take something and we're not going to continue to take something if it's not getting us to what we want and what we need in terms of outcomes.  So I carry that as a badge of honor that they said no.  I said they actually should be giving us money trying to figure out what we're doing right as opposed to saying no because we won't change our strategic plan. Choose substance over fluff.  I don't know what to say about that.  Listen to critics and don't always see criticism as personal attacks.  I saw this morning the comment from Linda Darling Hammond (ph) about Teach for America.  I love Teach for America, but there are some things I talked to Wendy about.  I love Teach for America but I really would prefer not to put them in our underperforming schools.  Why?  Because those schools need stability.  They cannot take people coming in and out.  That's part of the reason that they're underperforming. So we were talking about how do we get them to stay longer.  A wonderful program and we use them in our schools. The nationally certified program, a wonderful program.  We give our teachers $5,000 for once they become certified.  It started with one 5 years ago and we now have over 100.  But there are problems with that program because all of the teachers that are getting certified are not great teachers.  They've jumped the hoops, they've gotten the certification, but they're not great teachers.  So what do we do about the gap between whatever is missing that is not taking place in the classroom as we see it. I want you to listen to those of us who are on the ground to use our criticism as an opportunity to make some changes in great programs. I'm going to close.  This is the hardest job I've ever had in my life and there are no easy answers for moving any district from crisis to promise to success.  It sometimes feels like moving a mountain given the various agenda, the views of multiple stakeholders about how to reform schools, given the political climate, and boy do I know that in San Francisco, the politics of race and the various cottage industries which is another whole day that we could spend that have come up and formed around, and they thrive on school districts that don't work so you got to be careful what you give your money to. But the good news is I believe that it's possible to move urban school districts to high performance and I believe in the public school system.  I'm going to ask this question, How many of you are graduates of public school systems?  That's why I believe in it.  I think it's about mobilizing our resources in a strategic intent that will get us to the kinds of results that we have seen in this room and across America.  Thank you. MR. HESS:  Thank you, Arlene.  Bruno? MR. MANNO:  Thanks, Rick.  I've organized my comments around four headings and I think I'll be able to get through all four of them. The first is around the topic of what I call a theory of leverage.  The second is on markets, what I could call here maybe a theory of action.  The third is on donors or what I would call a theory of results.  And the fourth is, What does this all mean for the aim of philanthropy? Let me start with the first one which is on money or theory of leverage.  I'm going to give you a short way of summarizing each of my points right off the bat and hope you'll listen to the rest. Point number one about this idea of money or a theory of leverage, and I don't want this to be understood as in the past and now, so it's not so much a comparison about in the past and now as it is about an approach some of which is visible in the past as well as today and vice versa. I think there is an approach that looks at money as a resource and grants as the tool, and I think there is an approach that looks at influence including various forms of advocacy as the resource with money being one tool to achieve that end.  Let me talk about a little bit, and I do think those are two very different approaches. Nearly every one of the papers presented, the four papers that we're talking about now, makes the point that philanthropic dollars are minuscule when compared to the amount of public dollars spent on education.  I agree.  It's ironic though, or perhaps a sign of the program organizers' mischievous hand that the papers in this section on assessing major strands of giving have ample examples of how philanthropic dollars made a significant difference in the world of schooling. Jay calls this type of giving "higher-leverage" giving.  It "can alter future public spending, A, by supporting research and advocacy efforts that inform policy debates, B, by creating new types of public schools or administrative structures through which public dollars will flow, or, C, by developing alternative professional associations and credentials that have the potential of alerting the political activities of educators or government regulators affecting who can become an educators."  That is a sentence, by the way, written by Jay Greene on page 8 of his paper. Jane and Kendra (ph) talk about how both the National Board and Teach for America are "structural reforms in American education."  Bryan and Amy describe the choice work they discuss as focusing "resources primarily outside of existing institutional structures rather than inside.  This outside focus develops new institutions, new schools, new support organizations and new political advocacy organizations." Finally, Andy's paper is on the topic of education grand making for research and policy and "our strategic and targeted investments can demonstrably change the national policy agenda."  He quotes Paul Hill to make the point that "foundations cannot be policy neutral in the education wars.  Philanthropic investments can either buttress the existing system or pressure it to change."  Those are all ways I think in which these four papers talk about so-called higher-leveraged giving. So in spite of minuscule dollars, philanthropy can have a major impact on the world of schooling, though all the papers show that the going is not easy. But in talking about and giving examples of the results of this so-called higher-leveraged giving, nobody outlines what I think can be called a theory of leverage.  There is discussion around the issue, and, frankly, I'm the only "funder" though Kim plays two roles, on this panel, and in fact, I'm the only discussant up until the last panel who's a funder.  So no one has really talked about what I would call a theory of leverage. I've learned a lot of this from Peter Frumkin, though in fact in his paper today he doesn't really talk about this.  A sound theory of leverage is an important part of any theory of action or logic model.  What do I mean by that?  A sound theory of leverage involves both grant-making tactics and programmatic tactics.  Grant-making tactics include things like whether you're going to provide project grants, general operating support, short-term grants, long-term grants, matching grants, or loans in program-related investments. On the other hand, programming tactics include things like whether you're going to have a geographic focus in a community, whether you're going to focus on a program area, or will you find commercial ventures with nonprofits.  Insufficient thought, I think, is often given I wouldn't even say by foundation people to both what I would call grant-making tactics and programming tactics to develop a sufficient theory of leverage, and none of these papers really discusses this in any detail.  Admittedly this is not very exciting stuff, but having a clear and well-defined theory of leverage is very important to making a difference.  Let me go on to my second point. I'll talk a little bit about what I call markets and a theory of action.  I don't want to really get hung up too much on talking about theories of action, but each of the papers does and a lot of the other papers beyond the prior panel did.  In particular, during the first session we had some discussion about various theories of actions in district. This is a foundation staff person saying here's my theory of action and how I go about doing due diligence on the potential of a grant.  This brings together a lot of the notions that are embedded in the theory of reinventing government.  I'll just quickly run through this. If one wants to create what I call performance value, and in schooling that means making sure that young people are prepared for adult success and well-being in the worlds of work, family and citizenship, you need to have some sort of sense I think in the investments that you make, that I try to make, about incentives to perform, autonomy to perform, the capacity to perform.  Incentives to perform are about giving people reasons to act. From the reinventing literature that's the customer and consequence strategy.  You have to have sufficiently thought through notions about what it means to have the autonomy to perform and the freedom to perform.  Within the context of this reinventing literature they talk about control strategies. Finally, you need some forethought about the capacity to perform or the ability to act.  Within the context of the reinventing literature they talk about the culture strategy and changing minds, hearts and habits.  All of that leads ultimately to what the reinventing literature calls the core strategy which was the mission that we're about. I would conclude that within all of the sorts of discussions that we've had today, there are various perspectives on each of those three issues.  I would bet that in many instances, and I'll be just as critical of funders as I will be of fundees, not very many people have thought through that very well.  But there clearly from my perspective is a kind of due diligence tool there that begins to give I think funders as well as fundees a much better sense about how to go about creating what I would call performance value. I'd like to conclude this section by giving my one line summary of what I've said.  Here I'll talk about markets a little bit.  I think there are those in philanthropy who say markets are part of the problem and philanthropy is there to correct it.  I think this perspective is a little different and talks about markets as part of the solution with philanthropy connecting to the market to create a kind of social market.  Those are two different approaches. Finally, a third set of comments about the theory of results.  Because it was a PDF file I was unable to have them adequately throw up on there an overview about this, but on this notion of a theory of action, as a funder there are three questions that guide me and my perspective on what results in grant making entail.  The three questions are:  Why are you doing it?  How well are you doing it?  And is anybody better off when it comes to impact influence and leverage. Impact means the direct effect that you have on kids and families.  Influence means are other people beginning to be persuaded by your approach that this is the way to go.  And leverage means the kinds of things that we've been talking about, whether it was Jay or the other people in their papers, bringing additional public and private dollars in support of what you're doing. If I had to talk about this in a shorthanded way, I might talk about donors being more and more focused on having and assessing impact, setting specific targets when it comes to grantees, funding strategies brought to them not only by nonprofits, but having their own funding strategies, and being explicit about them and inviting people to be involved in a process that leads to supporting those kinds of perspectives. I'll close on I think this perspective on money markets and donors has a ramification for what we think philanthropy is.  I'll raise a question.  Andy in his paper says philanthropy is in the change business.  I wonder if that's really an adequate way of talking about philanthropy is all about.  I might begin to move in the direction of defining it more along the lines of this notion of it's in the value business.  It's not say that change isn't a value, but I think as you begin to talk about it more as a value business or as something that creates performance value, you begin to get at issues that are not as explicit in talking about philanthropy as we've traditionally talked about it. MR. HESS:  Thank you, Bruno.  Kim? MS. SMITH:  As you might imagine, it's difficult in 8 minutes to talk about four papers so I'm going to depart from my normal pattern and actually follow the rules and talk about the papers in one minute. First I wanted to do a quick check of the audience.  Could I see a show of hands of how many people in the audience are actual government grant makers and people working for or have at one point worked for foundations?  How many people consider themselves practitioners, nonprofit folks, people who ask for grants?  How many people are researchers?  Is there a big group I've missed here? I think the biggest kind of overarching comment I would make does relate to Jay's paper in particular, but I think relates to all of them, as both a funder now and a grantee both now and in the past, the point that came up earlier around segmenting the funder's market I think is really crucial.  How I think about foundations and individual donors are collectively a philanthropic capital market and I think it's a mistake to think that there is a single conclusion for how the different players in that market should act because in fact, when you think about the kind of change or value we're trying to create in the world, it actually takes a pretty interesting blend of different players who are willing to take different kinds of risks and different kinds of actions. We talk internally talk about a continuum from one end which is very emotive, personal motivation which often ties to individual donors where they want to see something concrete, something local, something that relates to their lives at one of the spectrum, personal and emotive, and at the far other end of the spectrum, strategic and often can be but not always, national.  So these are two extremes both in terms of the way they think about their intent and the type of activity they pursue. They're not rational.  It's fair to say it's not a rational capital market, but I do think especially given how many hands went up of people who are seeking funds, it's important to acknowledge at the outset that one of the idiosyncracies of a philanthropic capital market is these folks don't have to give their money away particularly if they're creating a foundation.  Certainly if there's an endowment there's a different pressure. We have to respect that we need to find those people who are aligned with what we're trying to get done in the world while admittedly having conversations like this to get the whole capital market to perform better.  So I appreciate, Rick, both being invited and all the papers that were done in an effort toward constant learning. Some of the things I think I'd call up from the papers, and I'll do them alphabetically, in Jay's paper I thought there were a lot of great points and in many ways, Jay, the comments you made relate back to New Schools strategy ourselves and our explicit core beliefs that we call out around leverage because we're super tiny.  Our first fund was $20 million in a $400 billion industry.  It's very tiny. Leverage is good.  He makes the comment in the paper around sustainability being important, too, if you're able to fund something that can then survive over time, you can just think about it as then being able to do much more with the rest of your philanthropic dollars because that thing you helped start can continue on its own.  So leverage and sustainability are both important. We also clearly call out as one of our core beliefs that part of our job in venture philanthropy is to support this laboratory of entrepreneurs so that given what we learn from them we can collectively help steer public funding to have a higher social return on its investment.  So I think it's fair to say that that's an important role for philanthropy. Having said that, I do think that you can differentiate between direct impact and systemic impact.  I don't think it's fair to say, and I think Jay's overstates a bit, that everyone should be shooting for systemic impact because that presumes we actually know exactly what we want to do at the systems level, and I think you have to count on some of those direct experiments to actually figure out then how you want to translate that back into systemic impact. We look for a blend direct impact.  We think of it as an action tank.  So people who have strong hypotheses about what's going to work, then they have to do it and see if it works in a learning cycle and feed it back to policy makers. On Jane's paper, I think it was a great point, your clarification around how important the naivete can be in striking out to make change.  I also think one other clarifying point is Wendy has always said from the beginning that a big piece of Teach for America is to draw more leaders into the field.  I'm a pragmatist much as I appreciate sound methodology; pragmatically speaking you can look around and see a lot of Teach for America alums who are having a big impact at the leadership level, and I think it's important to count that as well as the immediate impact in the classroom. Another interesting point Jane makes is she points out the fact that when you are successful you will create second-order questions which people will then presume you are supposed to answer.  This is not I think fair to the people who have had an impact which led us to then be smart enough to look at the second-order problem much like if these teachers are great, how do we distribute them better?  Fair point.  Not Wendy's responsibility.  In my opinion, it's a systemic issue that we have to collectively address.  Let's not say that if you are successful you must then as an organization handle the second-order questions. She also raises a point about "survive on" which I thought was interesting, presuming that maybe--are better than donations, and I think we have to recognize there are different tradeoffs with different sources of revenue.  You have to get sources whichever type they are and they have different strengths. On Bryan's paper on choice, I do think I want to underline something he himself pointed out this morning which is that the funding base is incredibly concentrated and that gives an inordinate amount of power to the funders in that field which I think is a problem and I think we need to have more funders in that space to have a very healthy practitioner side because right now the funders have all of the power. On Andy's paper, I am a huge believer in the need to spend more money on R&D, to be strategic about it.  I do think we need to have a better connection between the experiments and then researching what works and translating it back into a policy impact.  I don't think foundations typically are investing it in quite that way, and I'm a big supporter of the increased investment Andy talks about. I thought his point about general benefit and specific costs was particularly important for those of us who are trying to look at actual change as opposed to just value because that issue I think is pretty core to how we think about our advocacy work. I would suggest that I think the amount of time and energy funders put into advocacy may be undercounted precisely because of the legal issues you pointed to.  For instance, in our case, a number of our donors were inspired by our work with grantees to go out and create a 501(c)(4) organization to do the advocacy work.  That work is heavily informed by the work we do as a funder, but we're not doing it because we're legally not allowed to do it.  So I think there are other ways to think about translating information into influence and later advocacy. His points I thought about policy networks and agenda setting and all those other pieces of how these pieces fit together were really important and worth taking seriously as funders. The last thing I'll say about all of them is the point about higher education being conflicted which I think is a huge piece of the puzzle we haven't really addressed much today.  I'm grateful for the folks who wrote these papers.  I'm also extremely frustrated that if this small a group of people writing these papers both on the foundation practices, but frankly more broadly on the practices in education that are really trying to create more systemic change or more change that really attempts to shift around the incentives and dynamics around existing turf.  So I do think it's crucial and I appreciated Sue Fuhrman's points, but we're leaving higher ed I think out of this equation a little bit because it's another important idea generation entity in our ecosystem, not just foundations, and we're counting on foundations to do more than their share because we're not counting on higher education to do its share.  Frankly, I think many of the people in the room have probably just given up on that. So I think we need to not give up on that and bring them back to the table so that collectively we can pursue I think better outcomes. MR. HESS:  Thank you, Kim.  We have time for questions and answers. MR.          :  I'm a retired-- [End of tape 4A, begin tape 4B.] MR.          :  [In progress] --have a very different list of what is useful mathematics from college professors in of math education and when you talk about doing more research and where other people have talked about doing evaluations and program directors--to what extent do you mean to make use of college professors of mathematics or science and history as opposed to college education professors? MR.          :  Many would laugh at my speaking on the subject of mathematics, but my one small answer is it's certainly not a field of any experience or concentration of mine, but I do serve on a major project NSF funds on mathematics education and in the leadership of that project which is co-run by two universities, professors of mathematics are integrally involved at all levels along with people who are involved in what would be called the math education field.  I have no idea how common that is at all, but the project that I see it's everybody sitting around the same table. MS.          :  During the discussions today I couldn't help thinking that a lot of the advice for philanthropists could also apply to government as a philanthropist.  Jim, you ticked off a few of the notable accomplishments that have made a difference in private philanthropy.  If you were looking at government other than the GI Bill and Land Grant Colleges, what other successes would you point to? MR. KELLY:  I think the Land Grant University idea which has been refreshed at different decades of our history is one.  I think the existence of vocational education and what old timers like me call special education are two other examples.  I don't know if other countries have the investment that has happened in the last 25 or 30 years in the education of children who have handicaps of various kinds.  While that may been stimulated by foundations in some way, it has very heavily been a governmental initiative. There are many others.  I think the most important one that has washed over the education field in the last 50 years that was stimulated in some sense by a foundation as an adversarial or hostile or outside organization or whatever is the Civil Rights Movement.  Clearly the government was deeply involved in that with major historic pieces of legislation.  That's just top-of-the-head stuff if you look back at it. On the other hand, if you look at the K-12 field, right now it's bedeviled by a combination of very strong federal pressure on one or two issues and a plethora of God knows how many dozens and dozens of other little dribble-down programs that talk today about what leverage you have with $400 billion a year with foundation money would apply also to many of the line items in the federal budget, and all of the same points today about leverage and sustainability and so forth are present there.  I don't think there's anything intrinsicly different about these sources. As long as I have the floor from Susan, I would like to say that another underdiscussed item today has been the role and the changing role of corporate philanthropy in the K-12 field.  I don't think there was much of a role until 30 years ago or 25 years ago, but in my experience, in that 25 years there has been an enormous influence of corporate leadership and corporate philanthropy far outstripping the number of dollars per se in the validation and political pressure and public image of corporate CEOs backing certain projects and certain movements.  To the extent that that will continue is an interesting question for another meeting, probably. MR.          :  I had another thing here, Jim, which is the National Assessment for Educational Progress, NAPE. MR. KELLY:  Absolutely. MR.          :  There again your caveat was there certainly has been private money in that, but for the most part particularly over the last 10 years since the Alexander--Report which was supported actually with private dollars reoriented the whole thing to state-based information and from there I think has gone the whole standards thing.  So I think that's another one I'd add. MR. CASTERLY:  Mike Casterly (ph) with the Council of Great City Schools.  I apologize if the question seems unusually provocative. Arlene, you mentioned that as an urban school district, and I agree with Jim, a lot of the conversation today has been about urban school districts, you've made a considerable amount of progress that was often without the support of foundation dollars, at least big foundation dollars, that you often saw going to other places that you wondered about.  Then in part of your presentation you asked the question, I wonder what I'm doing wrong here not to get or to attract more foundation dollars. I wonder if it has occurred to you that you've gotten the progress that you've gotten because you didn't get a lot of foundation dollars and it didn't take you off your strategy or your game? MS. ACKERMAN:  That's an interesting question.  On some level these monies come with some strings in order to urban public schools.  I have to talk about urban public schools because that's all I know.  I think that probably did play a part in it because we could stay very focused. Then on the other hand, I wonder what could we have done with $25 million, again, strategically focused on a plan that was working.  That's the other question for me. MR.          :  Jim, when the National Board idea was first proposed, my understanding is it called for control by the subject organizations like the American Mathematical Association and the American History Association, and it came out to be controlled by the unions.  I wonder if you can shed any light on how that happened. MR. KELLY:  Neither proposition is true.  That is my answer, neither of those two propositions is true. MR. HESS:  Kim, one of the things you alluded to here, and all of you touched upon how this stuff obviously doesn't extend nearly as far as we'd like it to in terms of looking at the pre-K questions, at the higher-ed questions, and the roles these different institutions play.  How much of this is a product of the way that we think about investment strategies and the way that foundations are constructed internally?  Is it to a degree partly the institutions, the ways foundations have traditionally existed?  Or is it that foundations are simply dealing with a larger environment?  Is it something else? MS.          :  Part of what I was thinking about, when we try to create visual aids for the business leaders we work with to explain all the different streams of change it takes to get to the final outcome we're all looking for, it's impossible to do it on one visual because it's so complex.  You all know as practitioners from all these different seats that it's a really complex system. People's heads start to break and they have to simplify it down to one piece they're working on and I think that's a natural inclination.  The problem is that's not good enough and then you fall into organizational silos and field silos which foundations I don't think do anything to alleviate, they play right into it, unfortunately. So it's both contextually people try to find a less-complex way to take action where they think they could measure impact, that's a problem.  And foundations I think I have to say that I have been disappointed in the ability of foundations to build in any flexibility to take action on things that are quite important opportunities but don't fit into preexisting silos on their funding rules side.  I think that's a simple fix foundations could do by setting aside some funds for really creative opportunities that don't fit into their current world view. I think that's a fair point.  I don't think they're alone, they're just repeating the silos that happen with researchers and practitioners and everybody else as well. MR. HESS:  It seems to me--a leveraged conversation today and I think I'm probably as guilty of this as anybody and being a big believer in incentives, a lot more conversation about higher leveraged giving almost seems to have a certain smugness sometimes, that we're been doing this for a century and now we finally learned that we need to focus on things which are going to change systems, Jay's metaphors, and if we do this stuff that we have now figured out how to tackle a lot of these institutional problems. I wonder a lot of times at the kind of issues that Tom raised this morning in his analysis, I think Bill raised reasonable caution.  But too often the conversation has shifted to this kind of business school lingo of leverage and organizational structures.  Sometimes I wonder if we're not all making this too easy on ourselves, me included. MS.          :  Do you want me to answer the too-easy part or the rest of it? MR. HESS:  Either or both. MS.          :  I think you can separate the two.  There is an obvious question if you talk about high-leverage structural changes that you presuppose we have some pretty good hypotheses for what those changes are, and I think that's a fair question that we're not necessarily sure on all fronts.  But I think there are some places where we know the answer and we just don't have the political will to get it done.  So there are two different types of problems. On the are we being too easy on ourselves or is this a rhetorical shift in the winds towards businessspeak?  I think there's some of that.  I think part of it has to do with most of the new donors and new foundations that have the flexibility to decide where they're going to focus because they have live donors and are by and large very engaged business people who look at the scale of change they're created in the world with a small amount of capital relative to the change that resulted, like Federal Express and the U.S. Postal Service, and they say, Why can't we do that in this place? It's very pragmatic to me.  If you have a limited number of philanthropic dollars and you want to have the biggest impact possible, you can't get away from having the leverage conversation.  It's fair to say maybe it could be less jargon-filled, but you still have to figure out that equation. MR.          :  Could I add a footnote? MR. HESS:  Yes. MR.          :  I think part of the answer to this goes back to actually what Kim said earlier about the segmentation issue and to think about this more in terms of a continuum rather than either/or.  On hand of the spectrum if you're talking about donors, maybe there is a focus on doing good in a more conventional or traditional sense of the word. On the other end of the spectrum there are those donors who, whether they're the type that you describe coming out of the business community, the Bill Gates or the Dells of the world or even those who are doing more and more when it comes from donor advice funds who may be more strategic and who may think about impact in a certain way. It's not either/or.  I think if anything, the lesson out of all this is that giving this this continuum.  We could argue about whether one is better or one is worse, but I'm not sure that's a very useful conversation to have. It almost goes back, I'll just sneak in my other point that I ended on, about think about philanthropy or defining its aim in terms of change which I think pushes you in a lot of less than useful conversations about what is change or what isn't change or what will change the system or what will bolster it.  I can have those conversations as well as anyone, but perhaps that's not just the right metaphor. What might be a more accurate metaphor, I suggest that maybe it's the notion of value and how people interpret what is valuable, and whether it's the thing on one end of the continuum which may be considered more conventional, a charity, or something on the other end of the continuum which might be more systemic or something like that. MR. HESS:  There's time for one more question. MR. GERGEN:  Chris Gergen (ph) from K-12.  I wanted to pick up on the question of leverage, and Kim, picking up on a point you made in terms of being an efficient or potentially irrational market.  One analogy is in Boston I think I read recently that there are 600 current giving sources for youth organizations in the Boston area alone; incredibly inefficient in the way that we're distributing our philanthropic dollars.  In that respect, zero leverage. I'm wondering how do we make this a more efficient market in terms of coordination, cooperation, accountability and look to ways that you can actually start replicating on venture philanthropy side some of the models we're seeing in the venture capital markets from the early stage to the mezzanine level to ultimately an exit, and where does government fit into all of that, in one minute or less. MS. SMITH:  I think it's totally fair to say the philanthropic capital market is neither efficient nor rational, but it is what it is.  So I think certainly as an intermediary who raises funds and aggregates different sources, we're trying to make it a little more efficient for the specific entrepreneurs that we fund so that particularly the early--where we work they don't spend 90 percent of their time as executives raising money.  We can try to concentrate it and help them get up on their sea legs and then they can go direct to the foundations.  So that's our particular answer to that issue. Having said that, I think we all have to acknowledge that structured as it is right now, there are huge ego issues involved in the philanthropic market and that many people want the credit for what they're doing because it's all they think they get, really, because someone else is doing the activity, someone else is working with the clients.  So you have to figure out a way to balance that out. I think that was the intention behind creating the United Way.  So we've seen this cycle before, and to some degree it works, but then it created its own bureaucracy and ended up not being efficient either.  So I think it's a fair goal, and my hypothesis is that continuing these types of conversations to get people to self-identify where they or their organization are on that continuum so that at least we have clarity.  The worst thing about the inefficiency and the irrationality is if you as a nonprofit manager waste your time for a year talking to someone who in fact is at a different on the continuum, that's heartbreaking because you had a lot of other things you needed to do to get things done. My goal would be clarity as a first step, and if some time later we can get to efficiency, I'm all for that, too. MR. HESS:  Jim? MR. KELLY:  My comment is that I think that we should appreciate the genius of the American system involving millions and millions of individuals who are not coordinated and don't want to be coordinated because we have such a deep cultural skepticism about the power of government, we do not want government to control everything.  We want to have individual initiatives cutting across every level of this whole system.  That is not a statement against public education.  All you have to do is go look at France and you'll see what's missing.  What's missing is millions of people giving and volunteering to make some system, even if it's primarily a public system, function better.  And I like the chaos.  I'll take chaos. MR. HESS:  I think chaos is a good note to wrap the session on.  I'd like to thank the panelists. [Applause.] MR. HESS:  We're running about 10 minutes late.  We'll be picking back up at 10 to the hour.  MR. KATZIE:  [In progress]--lot of philanthropies that focus just on the school level, or the classroom level, or the state work.  Our focus is primarily around district work.  We don't avoid the district when we make our funding, we don't go around them and fund an intermediary; we fund directly, by and large, to districts or to national organizations that then provide services to districts. And because we work in large urban systems, we want to help try to transform what are pretty dysfunctional bureaucracies right now into much more higher performing public enterprises, and for us that means improved student achievement, eliminating gaps between income in ethnic groups, and then sharing those exemplars or best-practiced examples with others in the field, whether they be urban districts or suburban or rural districts. So this is a little bit kind of like, I guess, our theory of action or theory of change slide.  Not quite, but it's kind of an overview of our strategy as it's come to be over the first five years.  We focus our efforts where the need is greatest, which for us is in large urban districts across the country. One out of every five kids in poverty go to school in a large urban district--I'm sorry, one out of every--40 percent of kids of poverty go to school in large urban districts, and we focus where we believe our philanthropic dollars can have leverage for greatest impact, which is on leaders.  So we fund school boards, superintendents, principals, labor leaders, the leaders at the top of the district chain.  We invest in first those accomplished and/or future leaders, mostly through programs that focus on recruitment, selection, training and placement.  We then support those leaders in building institutional capacity for success by investing in what we call systems or tools, or other enablers for the system to succeed, and we've just begun to work in funding what we call the external environment so there our investments can flourish.  And this is mostly through our policy work, but also in a more limited way in research dissemination showcasing success and competition we put into this area as well:  mostly funding, charter management organization. And we measure--and I'll talk about this at the end of the presentation--what matters most to us, which is improving achievement, closing achievement gaps, and then giving some existence proofs that these public systems can have operational excellence.  And we believe that the alignment of the governance management, labor relations, and competition investments that we make will actually lead to dramatic performance in K-12 systems and great results for the students in those systems. All right, how do we fund?  So as I said, these are the targets for our funding:  school board members, policy leaders, superintendents, et cetera.  We also have significant investments in data, showcasing success, and some new and growing investments in the policy area. We have five types of investments, and I'll start from the bottom up.  We do what we call "gift grants."  That's where we write--it's kind of the contractual or auditing relationship.  We write a check to a good cause, it's for a charitable purpose, it's within our mission but we write that check, and at the end of the year we get a report.  That's very different than what we categorize as an investment grant, which would be the rest of these categories. We have a small local venture fund for Los Angeles because that's where we live.  Sometimes those grants are on mission and sometimes they're off.  It's the reason why we created a separate category was so that the Broads [ph] could combine some of their philanthropic endeavors in their local community without us getting, let's say, 500 applications for after-school programs from places like Toledo or New York or Boston. The majority of our work is done in districts.  We also fund national organizations like Teach For America, New Leaders For New Schools, and TURN, which is a national union organization which works in multiple sites across the country.  And perhaps somewhat unique to our foundation, so I'll talk about it for a minute, are what we call our "branded flagship initiatives," and the paper walks through a number of those. The five in this area, one you heard of a little bit this morning when Don McAdams talked, the Broad Institute for School Boards, this is actually modeled off the Harvard University induction programs for new members of Congress, new mayors, new college presidents.  There was no national effort to induct urban school board members, and so we created an induction program and working very closely with Don in that effort and actually expanding that effort just beyond the induction component of that school board training. Probably what we're best known for, the next two:  One is the Broad Center, which has two programs, the Superintendents Academy and a residency.  The Superintendents Academy Arlene, and Alan, and Checker [ph], and Mike Kasterly [ph] have served either as presenters or faculty for that group.  But this is where we recruit both noneducators, so people from government, the military, higher education, corporate world, nonprofit, and educators become the next generation of urban school superintendents.  And we seek excellent talent from across all of those sectors because we believe K-12 education needs talent from all those sectors. We now have superintendents who have gone through that training placed and working in Houston, Charleston, Fort Wayne, Minneapolis, Montgomery, Alabama, Fort Worth, Cobb County which is right outside of central Atlanta, Oakland, and Christiana, Delaware, which is the largest district in Delaware. The Broad residency is a more junior executive program where we recruit MBAs and JDs and Masters in Public Policy to work for two years in the central office, and the Broad prize for urban education, which has been mentioned a number of times here is our effort to really honor and showcase success in large urban systems across the country. Schoolmatters.com is our partnership with the Gates Foundation, Standard & Poores, and the Council of Chief State School Officers to really create a movement, national movement to make data on student achievement and financial performance as transparent, ubiquitous and actionable for parents, educators, and policy leaders as we can, because data has been pretty bereft in K-12, and we want to be part of the movement to change that. One more thing on our approach.  A lot of people have talked about venture philanthropy.  This is kind of how we define our engaged philanthropic approach:  We look for entrepreneurs, we look for innovations, and we, as Mike Feinberg said, are all about results.  Eli doesn't care if it's a nice program if it's not producing results.  We seek out talent and ideas both, support venture with both human capital as well as dollars.  Eli is very clear that every project or program that we provide that's not a gift grant will have an ROI or it will not be continued to be funded. And we are also free--we talk about this in the paper--to fund organizations both in the seed stage as well as the scale-up stage.  And one of the things that we learned when we talked to other foundations when we first started is that many of them had five-year plans or they wouldn't continue with an organization after three years of funding, and we don't have any of those restrictions on us at all. Okay, lessons learned.  There are six in the paper and so, like Peter, I encourage you to read it.  [Laughter.] Okay, so I'm going to go--I'm going to go through--I'm going to go through a few of them quickly.  One is--and this kind of goes with the whole theory of change and unit of change--your belief about how the world works really should drive your grant-making, and so for us we are very clear that districts are our unit of change.  So we put a lot of money into San Diego and New York and Boston.  They all have very different or somewhat similar--some of them have very similar elements of their theory of action; some areas are very different in terms of pace and approach to their reform.  But they're all very clear about what it is they're going after and pretty clear about how it is that they're going to get there.  And so we fund to help provide support to that district theory of action and, hopefully, to help them improve the results from that. We actually believe that failure is an option.  I mean this is like any venture portfolio:  We're going to have some dogs, and we're going to have some stars.  And every year we take a look, and sometimes more than once a year about what we're going to call from the portfolio, what we're going to do, what we're going to put on pause and try to see if we can work to fix, what we're going to maintain and what we're going to scale up and provide either more human or financial resources to. Happy to talk about any of this in more detail during the Q and A. Lesson 3 is to be willing to face inherent tensions in the grant-making strategies.  What you'll see across the three of these is that we actually work to help improve the current system, but we also invest in blowing the system up or in strategies that might blow the system up.  And so there are a lot of things we do. We invest in side districts with strong district leaders at the same time we're investing in charter management organizations that are outside the system.  We see both as important; we're not sure that the systems of the future are going to be our current districts, but while they are in existence we want to do everything in our power to help them improve.  We also believe that charters will have a place, in fact a very central role, 20 years from now even, and we want to help grow that kind of new version of a system of schools. The same thing with us focusing on recruiting educators into the academy as well as nontraditional folk into the academy.  A lot of people say to us, "You're the guys who bring in the military or the corporate people into the superintendency."  Well, yes, that's true, we're one of the very few if not the only organization out there trying to tap that talent.  But fully 50 percent of every academy class that we bring and a huge amount of our recruitment effort and dollars every year are to find the very best educators who can step up and become the next generation of chief executives in large urban systems. We actually, recently, in about four weekends ago had a strategy retreat at Eli's house, and we listed all of these tensions in our portfolio.  And, basically, the message back to us --which is a good thing, I guess--was:  Don't worry about the tension and the feedback you might get for playing both--making bets on both sides of the house.  Go for it and keep doing it.  We'll see whether we're able to ride that out or not as a new philanthropy, but right now  we're going to continue to do that. And, finally, how do you make program evaluation meaningful?  The paper goes into this in much more depth.  We've had a, like most foundations, a fairly checkered history with evaluators.  We actually now, honestly, do a lot of our student achievement data and other evaluation in-house.  I can talk more about why we've made that switch to some degree and why we no longer work as much with external evaluators. But I just wanted to share with you one of our--kind of one of our data dashboards, since there's been a lot of talk about what people actually looked at in terms of impact and results.  And this is one of the ways we looked at our training programs.  And for those of you that can't see it in the back, I'll talk about it. This is for any program where we are recruiting, training, and placing folks like principals or superintendents.  This one happens to be an evaluation matrix for principal training, and it says elementary literacy.  You could imagine it for elementary math or middle school math or any of the categories that you want to look at and what we look at as overall improvement.  So what have been the gains of that principal placed in that school over time and compared to their predecessor?  So if I'm a new principal trained by Alan's leadership academy, and I'm in a school for two years or three years, how is my growth compared annually against the growth prior to when I was in that school? Second, we look at reduction in income gap; third, reduction in ethnic gap; fourth, movement from the bottom quartile.  All the other numbers that we look at in this chart are based on proficiency rates, but we also know that making a jump for those students from the very lowest quartile or quintile, depending on what state you're in, is as important to show movement in the direction of proficiencies, so we take a look at that as well.  And then the fifth, where there were other metrics like safety, attendance, graduation rates, et cetera. Across the top, again we look at versus that school or, in case of a superintendent that district, against themselves, against other newly placed principals that are not happened to be trained by us but happen to be in that district or state.  First is schools with similar demographics, i.e., around the same percentages of kids in poverty, English language learner, and nonwhite, and, finally, versus the district as a whole or, in the superintendent's case, against the state. So I'll stop there. MR.        :  Okay.  Thank you, Dan.  Les? MR. LENKOWSKY:  Thanks very much.  I want to start with two apologies:  First of all, I do not have any power point.  I regard it as a matter of professional pride never to do a presentation with power points, so those of you who'd like to see the third in a row will be disappointed. Secondly, this is the last week of classes at Indiana University, and every teacher knows that that means, so I'm afraid I got here just before lunch and will leave just before the reception in order to write three final exams and then endure the agony of actually grading them. To reflect on what philanthropy's accomplished or not accomplished, I'm always tempted to go back to an estimate Andrew Carnegie gave in one of the two essays that comprise the "Gospel of Wealth."  Writing in the 1880s he said that 950 out of every $1000 given to charity was, in his opinion, wasted.  Now, the interesting question, of course, 125 years later if we were to tote that up and apply it to educational philanthropy, whether the number would be higher or lower. Jim Kelly referred, interestingly, to one of the--the successful project, the Carnegie Corporation's funding of Gunnar Myrdahl's family book "An American Dilemma."  As those who've read Ellen Lagemann recounting of this know, the Carnegie Corporation thought this was a great failure as a project.  It had actually given money to Myrdahl to evaluate the work it and other foundations had been doing in southern education.  And Myrdahl, being a newly-minted Ph.D. from the University of Chicago wasn't going to have such a--be constrained by such a narrow assignment.  So he took it upon himself to write a great treatise on race relations in the United States. The Carnegie files are full of letters from the Chairman of the Board, Elia Root [ph] to the president, a very interesting man names Charles Dollard saying, "Well, where's our evaluation?  What's keeping this guy?  We can't, you know, look at our grant until we get this evaluation."  It was several years in the making.  By the time he delivers a very long report it is not only--it is not an evaluation, but we're in the middle of World War II, and the Carnegie Corporation has other things to concern it. So this is what I call one of--I'm compiling a list which is now up to five of philanthropic personality disorders.  This one I referred to as "philanthropic myopia," the difficulty of determining whether something you've funded is successful or not.  For those who are interested, some of the others have come up today in other fashions. "Philanthropic anemia," where money is not sufficient, or you think it's not sufficient to accomplish your goals. "Philanthropic mania," which is probably the chief one in which grantors spread themselves too thin, hopping around from project to project. "Philanthropic obsessiveness."  This is where donors want to be so tightly controlling that grantees have little leeway to try. And then the last of the five so far I call "philanthropic hubris" where donors are quite confident:  They know what works and are unwilling to hear any dissent from that. I'm only up to five now.  Some of you may have others you--I'd certainly be glad to collect them, and one of these days we'll issue a manual which the Senate Finance Committee will enact into regulation.  [Laughter.] However, looking backwards throughout history--and that's what I've tried to do in my paper--there's actually a more interesting story than a legacy of failing projects.  In fact, to the contrary.  Philanthropists have had a sizable impact on American education, I contend; indeed, sometimes the things that were regarded at the time as great solutions proposed and supported by philanthropists have now we regard as major problems that need to be overcome.  And I think there are some lessons that we can learn--and that's what Rick asked me to address--for the future both from experience of educational philanthropy over history as well as some very quick observations I'm going to make about philanthropy and other fields. We have an interesting contrast in scale on this panel.  To my left, as you'll hear, Professor Heyneman is going to be operating in a planetary focus.  He's out there in orbit somewhere.  My good friend Peter Frumkin was somewhere between 35,000 and 50,000 feet.  Mr. Katzir was right on the ground with a grant mayfair.  And I'm just going to do a broad brush through 400 years of history.  [Laughter.] My criteria, looking at various things that went through mostly secondary sources--I don't want you to think that I've really burrowed into the documentation here--was to look for innovations in education that were largely instigated by philanthropic support of one kind or another and were successfully widespread or brought to scale in the currently fashionable language and had some serious importance for education.  I kind of grouped them by five sets of objectives that philanthropists have had over these 400 years or so. The first is what I call "creating better schools."  This is the earliest and most continuous of the objectives one finds in philanthropy, and it aims to serve both ensuring an educated citizenry but also to assist in the moral development--and I'm using that very broadly--of American young people. A variety of key accomplishments--and I, too, would echo what has been said before, and if you want the details, read the paper--southern education, religious schools including non-Catholic ones.  We sometimes--we talk about philanthropy in education--forget that we've had an entire--well, several sets of school systems created largely by philanthropic support in the United States and still, to a great extent, maintained by it. Laboratory schools.  Of course, the famous one which was the laboratory school of John Dewey, but lots of others, that have been designed and supported by philanthropists with a conscious intention of trying to demonstrate new methods for improving education, and many have been successful at that effort.  Ted Sizer's coalition, some of the charter schools might fit in this area, too. A second set kind of objective, creating better teachers.  Again, this was one of the earliest objectives of philanthropists in the United States though it hasn't continued to the same extent down to the present.  The whole idea of normal schools--ed schools we call them today--grew out of philanthropic impulse of one Stephen van Renssalaer in 1826 in Troy, New York, who thought that teachers not only needed to have the tools to perform as teachers but also ought to have the right moral and spiritual values.  But we've kept the tools part--it's not so obvious.  At least our ed school doesn't offer much in the way of moral and spiritual instruction, though perhaps it's taken for granted. Professional standards also go way back.  The American Institute of Instruction was founded in 1830 on the premise that teaching was or should be a science and, therefore, scientific methods needed to be developed and employed in teaching school teachers. Teacher education and bringing teacher education into a university context was another preoccupation of philanthropists including such major ones as Carnegie in the General Education Board.  It also had its share of critics, not only Arthur Bester and the group that we today associate mostly with the Council for Basic Education, but also foundations that might have a more progressive sense these days like Ford. Today, of course, much teacher education has been institutionalized, largely run by government, paid for by students, although there are some notable examples of continued philanthropic support for the creation of better schools such as alternative certification programs, Teach For America and the like. Third, creating better curricula.  This became a preferred strategy of philanthropists as government funding came to dominate the support of education.  This was a--we didn't--they didn't call it then, but in today's language we'd call it a high leverage strategy.  You can't build schools or train teachers, at least effect what they're going to be teaching.  And again, all sorts of examples here, the development of kindergarten. One which I would include under this heading is standardized testing.  The advent of standardized testing was largely a creation of philanthropists really responding to the new waves of immigrants.  The whole idea was that you would track students and match students with the appropriate curricula.  Versions of this were things like the Advance Placement Program and Merit Scholarship programs coming into later in the 20th century, meant to affect curricula and with some success. Perhaps the most successful, though, was the idea of the comprehensive high school which grew out of workbook [?] funded by the Carnegie Corporation written by Jane Bryant Conant and, of course, set a standard which today many of us are somewhat skeptical of for how high schools should be developed. Others you're familiar with, the new math, the new physics, the new whatever, new technologies, such as TV, "Sesame Street," and--dare I say it?--"Postcards from Buster," and, of course, this, too, continues in a variety of ways through things like core knowledge, conflict resolution programs and so on. Fourth general objective: Creating better communities.  Here, too, we find the mixture of education with moral concerns:  Religious schools, of course, are a good example of this that the premise is that you really can't have a good education unless you had a goods community, and schools and communities are so intertwined that one way of fixing the problems of education would be to work on the communities. So this led, among other things, to investments in social conditions by foundations such as Russell Sage, visiting teachers, obviously amenities such as libraries.  The Parent Teachers Association grew out of this idea that you had to really bring the school into the community or at least members of the community into the school. "Parents" magazine was originally created by foundations as part of this, and, of course, the whole list of things related to race and racial issues within communities reflect the notion that the way to improve education might be to do something at the community level rather than directly in the schools. Finally, the last general objection, and then I'll offer a few lessons:  Creating better children.  If you can't create better schools, you can't create better curricula, you can't create better communities, why not let's go change the kids a little bit and get better kids?  And this takes a variety of forms. Scholarships I would classify under this category.  After all, what is a scholarship or today a voucher than a way to make it possible for kids without the means to attend certain schools to do so? A variety of child development efforts:  Head Start, the Comer Programs, what has sometimes been called child-saving programs ranging from the orphan trains that in the middle 19th century took young, mostly Irish orphans into the mostly Protestant communities of the Midwest as a way of saving them, to more recent things such as the Boy Scouts, Girl Scouts, and then, interesting to me as well, are the whole service learning and volunteering efforts going on in schools which are trying to make students more civicly engaged and are justified not only because civic engagement is good in itself but, allegedly, because being engaged leads to better academic performance. Of course, there is a real problem with creating better children.  It's called parents, an obstacle that has to be coped with.  Anyway, we're out of time so I'm--just give quick lessons. First of all, as you look back through this, the panorama of history and look broadly in education, three things seem clear:  One is the importance of sharp focus and, indeed, restraining ambitions.  The most successful projects have been the most limited ones, and this, at least initially, limited.  They've had big subsequent effects, but their initial impulse is quite limited.  And this is true--I'm going to skip some of the other examples from elsewhere in philanthropy, but there are plenty of examples there, too. Secondly, it's also important to go against the grain.  My predecessor as the head of the Hudson Institute, Herman Kahn, once said that when something becomes the conventional wisdom, you could almost be sure that it is wrong.  And some of the more interesting innovations in educational philanthropy have been things that went somewhat against the conventional wisdom of the time.  They went against the grain, the grain in this case being what was going on in education. An interesting example as well is they tended to reward excellence often, at least the earlier ones, rather than other kinds of criteria. And, thirdly, successful reforms embody more than new pedagogies or techniques and are not particularly dependent on the money behind them.  What one finds is a mixture of things designed to improve education, but also a compelling moral and social vision about American society itself and the role of schools within it.  And I want to emphasize "compelling." This leads me to the question I would leave you with: Is the problem of educational philanthropy really a resource problem?  A strategy problem?  A tactic problem?  Or is it the case that, overall, educational philanthropy does not have a particularly compelling moral or social vision. Thank you. MR.       :  Thank you, Les.  And Steven? As Les mentioned, Steve's charge was to look around the world and bring something back from it. MR. HEYNEMAN:  Well, I don't know if I'm correct in this, Rick, but as I read the program, we have two more panels of those who will comment, but I am your official last paper presenter  This is--[applause]--and I think that deserves--just to loosen things up. MR.       :  Thank you, Steve will be the last one. MR. HEYNEMAN:  And I can imagine what it was like when you and your colleagues were asking themselves about all these important and probing questions about philanthropy and the degree to which it has changed education for school children in K through 12.  And somebody says, "Oh, wait a minute.  What about school children outside the United States?" And someone says, "Oh, we forgot about that."  And then they turn to--I don't know--Janet Hansen [ph], and Janet says, "No, not me, let's get Heyneman to do it."  So here I am.  And I'm glad you asked the question about school children outside of the United States because if you were to add up all the children in the world who go to school, 98 percent of them go to school outside of the United States, and two percent of them go to school in the United States, and so by asking the question what about the other 98 percent, it behooves you as social scientists to take a genuine interest in that.  So here I am, and I'm going to try and answer that kind of question. Well, outside the United States the term "foundation" is not defined in the same way as "foundation" is defined within the United States, and so it raises a considerable level of--let's see if we can--the other one--so definitions of foundations differ.  The foundation may have--outside of the United States the foundation may apply to membership associations, corporations, government subsidized enterprises, and may imply either public or private ownership.  Foundations outside the United States may also be associated with functions that are not necessarily charitable such as political lobbying, research, fund-raising for private and public purposes. Legislation enabling individuals and corporations to reduce their tax burdens through charitable giving is not as common outside the United States or as generous.  Moreover, foreign foundations are often taxed on incomes received.  Organizations with structures and functions similar to private foundations in the United States do exist abroad, but other nations also allow the government to establish what are called public foundations. Germany, for instance, foundations are founded by political parties, and they receive public tax revenues.  These include the Konrad Adenauer Foundation, the Hans Sedelrosa [ph] Luxembourg Foundation, among others. In Switzerland, a foundation can be a fund in which families invest their corporate pensions for their own benefit.  A British foundation, drawing on commercial philanthropy may be illegal in France, and a commercial profit-making foundation operating quite happily in Norway could be illegal in Britain. That's the first challenge that you've given me in analyzing the goals and extent of educational philanthropy abroad is to decide which organizations can be considered philanthropic and whether these are known as foundations or by some other term.  So I'm going to use these ingredients in the statistics which follow and the generalizations which follow as my definition of what a foundation is:  Nongovernmental, nonprofit--so far okay?  Possesses a principal of its own, managed by its own directors and/or trustees, and one which promotes social and charitable, or religious and educational, or other activities which serve the public good. Now, there are about 357,000 foundations in Western Europe alone, but of those, if you apply these criteria to this larger number, about 84 percent of them--84,000--sorry--would meet these criteria.  And 75 percent of those are located in just a few countries:  Sweden, Denmark, Britain, Germany, and Switzerland.  So by a private foundation applied in these criteria, they're a smaller number than use the term "foundation," and they're located in a small number of countries. Now, I want to come to the question raised by at least one, or several of my colleagues, as to the role of religion and religious organizations as private philanthropy.  In both Europe as well as North America, religious organizations remain a common conduit for education and other activities and are financed by both public and private sources. Maybe many of you know this but I'll say it anyway:  The United States is the only industrial democracy in which public schools are not owned and managed by religious organizations.  Every other democracy finances public education through private and religious schools.  Public schools managed by religious organizations are common throughout Latin America, Africa, South Asia, and East Asia.  For the most part those are affiliated with Christian churches.  However, in the Middle East and North Africa and part of the former Soviet Union, these schools are affiliated with Moslem mosques.  Religiously affiliated public schools are so common that Americans would be wise to remember that their definition of public education, meaning schooling that is financed and owned and operated by the state, is an exception to the norm. The more common definition of public education is schooling which the states help to finance but does not monopolistically own or manage.  Wherever school systems re managed by religious organizations, it is common for parents and community leaders to organize voluntary donations to support their educational programs.  This is true for both domestic religious school systems in countries such as Canada and Australia, and for religious organizations which help finance school systems in low income countries. Catholics often provide assistance through caritas; Protestants do likewise, through Christian Aid and World Vision.  These organizations are among the largest private providers of educational assistance in the world. Now, among Muslims, the zachot [ph], which is a charitable tax, is assumed to be about two and a half percent of individual income and has financed hospitals, schools, public water supplies, and other public works for centuries.  In addition, there are a system of religious foundations called waqfs, W-A-Q-F, which is the Koranic method of allocation--or reallocating--personal wealth and property.  Waqfs are responsible for thousands and thousands of charitable projects throughout the Muslims world.  In countries with a high percentage of Muslims, the waqfs are so common that governments sometimes dedicate a public ministry to try and oversee their activities.  This is true, for example, in both Pakistan and Egypt. In the case of Pakistan, for example, the Ministry of Waqf actually helps to manage charitable activities, further confusing our distinction between public and private functions.   So what are we going to say about the--how much is private philanthropy working outside the United States?  Well, if we just take US-based foundations--those are foundations which are organized and located, situated, with headquarters in the United States--they disbursed in the year 2000 about $337 million, but K through 12 activity was less than four percent.  Higher education was about 40 percent; graduate and professional education about 24 percent, so those two alone make up the majority. Somehow, of the US-based educational foundations, the K through 12 priority is a very small priority, and I'm going to make that case, also with other sources of philanthropy as well. I'm going to try and make the case just for the next minute or two that outside the United States, in order to ask the question about philanthropy, we have to divide it into two categories.  One category which you're familiar with is private philanthropy.  But I'm going to argue that there's a public philanthropy that is called "foreign aid."  And foreign aid comes under my definition of philanthropy because it is voluntary.  It voluntary on the part of a] taxpayers to help other countries often with the same or very similar humanitarian and charitable purposes that you yourselves have been talking about.  So I'm going to ask the question:  What about foreign aid as a part of our philanthropy? U.S. foreign aid is a very small portion of the U.S. federal budget.  As--here's the U.S. budget in 2004, and foreign aid was 0.9 percent.  So by comparison to many other activities of the federal government, foreign aid is minor. The amount of aid that is given by the United States is larger than any other single country in terms of dollars.  In the year 2002 it was over $13 billion, but U.S. aid is smaller than any other country in the world in terms of percentage of GNI, gross national income.  The United States was at the bottom of the list of major donors at 0.13 percent of GNI. U.S. private philanthropy is larger than any other country, but still, when you add the amount of U.S. private philanthropy, it does not change the ranking of the United States in terms of overall philanthropy.  Here's public, here's private.  Add it together total in 2002, about $16 billion which only is up to about 0.16 percent of gross national income. Now, if you look at public philanthropy across all OSED countries, it's about eight perc- --in education it's only eight percent of the total.  In other words, the United States is not the only country that is--talks a lot about giving to education, but in terms of percentages doesn't give a lot at all.  Here's the total.  Here's all the aid from all the industrial democracies in the world to the field of education in 2002.  It adds up to eight percent of all their foreign assistance. In here it is again, showing you in a more graphic form. Now, one last question about what--Rick asked me, well, what, exactly, goes on?  Are there some success stories?  And I want to just mention the Sorrells Foundation.  Now, I'm not an apologist for the Sorrells Foundation.  Someone asked me a moment ago:  Does he teach people to hate George Bush?  And my answer to that is not that I'm aware of. But the Sorrells Foundation is one of the most interesting and influential foundations.  By the way, the size of the Sorrells Foundation, if you were to ask the question:  Which is the largest foundation to help school children?  Not school children in the United States but the largest, I believe, would be the Gates Foundation.  The second largest is the Sorrells Foundation.  So the Sorrells Foundation is today the second largest in terms of new commitments per year.  It's finance by George Sorrells, who migrated to the United States after Hungary--and has had multiple programs all using this kind of leverage that you're talking about in terms of garnering more public funds in support of their programs that have been successful. I have about two minutes, so let me end just with a couple of lessons.  First the definition of foundation differs and, therefore, we have to be quite careful about that. Secondly, I think foreign aid ought to be considered a normal part of our understanding of what philanthropy is all about.  I think we should treat foreign aid as a part of philanthropy, and we should ask ourselves the same questions about its effectiveness as we are asking about private philanthropy. I would believe that overseas U.S. private philanthropy has been very successful with a lot of numerous stories like the ones you've told about leveraging public funds.  I believe that public philanthropy has been less successful with one exception, and that is the issue of the multilateral development organizations such as the World Bank and other regional development banks that always land on the basis of conditionalities.   And I would ask you to consider the fact that conditionalities add to leveraging.  And if you want to find people who have thought very carefully about how to leverage philanthropic money to generate a higher public returns, talk to people who have been working at the World Bank.  It's one reason why I have suggested that there be a World Bank for U.S. education.  I will not go into detail now, I have no time.  But you go to "Education Week" last December, there's an editorial on that, and if you go to the American Education Finance Association, there was a address, the keynote address this year was on that topic, and it's on their website. Thank you very much. MR.       :  Thank you, Steve.  We've got questions running about 15 minutes behind, which is giving me hives, so we're going to cut this a little short.  But we've talked about four or five questions, so pleases catch Morgan's eye.  If-- MS. CARSON:   Hello.  My name is Elizabeth Carson.  I'm a New York City parent and founder of a national organization focused on mathematics education.  I have a son in District 2.  I have associates in San Diego, and, of course, I'm very familiar with reforms in Manhattan, District 2, San Diego, and New York City now. And my interest is in, specifically, the Broad Foundation support for the reforms, both instructional, administrative, curricular, and restructuring, generally in all of these areas.  Much of what is said is true in terms of the integrity of the overall structure and push; however, I am aware that there has been extreme controversy around every one of the components of the reforms in Manhattan, District 2, San Diego, and New York City. So because of this controversy and because of an extreme dissatisfaction among the constituents in the community in all three of those areas, which is still raging, I would like to know how the Broad Foundation, given your commitment now and the $5-and-$6 billion that we can expect the Broad Family to be forwarding into K-12 reforms in the future, what could the Broad Foundation so, going forward, to avoid this degree of controversy and dissonance when there's so much opportunity in those three areas for real constructive reform, if it was handled in a way that would engage senior classroom teachers, university subjects experts, administrators, and parents? MR.       :  Okay.  Dan? MR. KATZIR:  To quote Adam Urbansky, real change is real hard.  And I honestly am not sure there's anything that can be done to prevent the push back on really bold, innovative reforms whether it's in New York City or San Diego or San Francisco or other places where real leadership and innovation and change is taking place. As a foundation, one thing that we are thinking about doing is some wrap-around communications money for reforms that are moving very quickly to help districts that typically are not very good at communicating either internally or externally to at least get the word out about what they're doing and how they're doing it. MS. COLEMAN [PH]:  Jane Coleman, an independent--[off mike] [END SIDE A, TAPE 5] [BEGIN SIDE B, TAPE 5] MS. COLEMAN [continuing]: --asking the question on philanthropic practice, and for me the standout essay was by Stephen Schroeder, a former president of Robert Wood Johnson Foundation. Lessons to be learned from outside education philanthropy and particularly the lessons of time frame.  If you look at the major investments of the Rockefeller Foundation and look at a particular issue, are there practices outside of education philanthropy that we should be looking at or thinking about again if we're looking at change and distance over a time frame, that's a nominator of time frame doesn't always get as much attention. MR. FRUMKIN [?]:  Well, I think that's a very nice point in the sense that the wiring diagram I just threw up there of the kind of--the causal linkages in a theory of change.  What's interesting about that picture is that the first--the first one, two, three boxes happen like that [snap.]  The money goes in, the activities get conducted, the units of service are delivered, and then sometimes the outcomes are short, medium, or long-term, but then the impact, the ultimate is kind of social--particularly if it's grounded in an aggressive theory of--of kind of policy change or something--something big, it often takes a lot longer. And so that's one of the difficulties of crafting a really sound theory of change, which is that you have to have--you have to have--you have to have the ability to see the time frame as not being absolutely linear across the model.  That is, you may be able to race through and produce some nice outputs and show that you've actually used the money sensibly to deliver some nice services and so forth, but if you really are about, you know, driving your philanthropy to a, you know, fundamental impact on deciding them and a noticeable change, you have to have a much longer time frame.  So there's kind of a disconnect in some ways in a lot of philanthropy between the kind of initial framing of the problem and the design of a theory of change and the ultimate time frame needed to really drive through the second--the second to last and the last box, which are often much further out.  And some--I think that's right. I think time-- having a good appreciation for time frames is absolutely critical to getting through this, getting through this process of crafting a sensible strategy. MR.       :  Let me just add, briefly, Robert Wood Johnson was almost uniquely specialized, that its entire life history it's worked on health care.  It did not have multiple program areas competing for board attention, staff time, et cetera.  It's developed a lot of expertise, so the president who wrote that article actually taught in the New Jersey Medical College while he was president of the foundation. I agree with what Peter said about time. I think the other thing, though, is scale, and they're related.  Without having looked all that closely at the Robert Wood Johnson grants, I would be willing to bet that their most successful grants involve fairly specific, focused interventions.  Their least successful one, which almost got them in a lot of hot water with LEL [ph] authorities was work they did in connection with the health care reforms that Mrs. Clinton was championing when she was first lady, and Robert Wood Johnson's efforts, which were mostly communications.  You may recall there were a series of town meetings and things like that to provoke the ire of opponents of that plan, a cautionary note for anyone who is interested in greater advocacy and political involvement by philanthropies. MR.       :  Could I just add one other example, because the other thing I think is important to realize is that some of the best examples are not successes; they're actually failed grants.  And I always like to signal for special kind of accommodation, a real fiasco funded by the Rockefeller Foundation years ago, but it was a productive fiasco in the sense that it had connected to it a phenomenal evaluation by Mathematical Policy Research and was their Single Mother Employment Program in five cities.  And, essentially, they found four out of the five cities the program had a negative impact, significant, statistically, seeing a negative impact on earnings.  And only one city was mildly positive.  It was--and had a control group.  It was a beautifully executed study. And I just think there's--in saying what are the miles--I think it's often useful to say if someone doesn't succeed and chose that the wiring diagram is fundamentally flawed in its typical laying out, that's a valuable thing.  If you can do it well, show that it's flawed, that those arrows don't lead up, that there's a big--that there's a big break in it, that's a powerful thing.  It requires investment in valuation. And so, you know, Schroeder's article is nice because it focuses on--on the evaluation piece, but evaluation can be very valuable for showing when that--when that causal system just can't work.  And I give the Rockefeller Foundation a lot of credit for what I think is one of the two or three best evaluations done, but it's essentially demonstrating a breakdown in the wiring system that most people think leads from a programmatic intervention to employment by single mothers. MS. TAGGERT [PH]:  Hi. My name is Sarah Taggert, and I'm an independent consultant from Portland, Oregon.  I'm interested in this issue of conditionality, and I have two questions.  The first is, how much time and money should foundations spend up front trying to understand the context that they're interested in funding an initiative in order to make a good match with the receiver, whether it's a city, a district, or an individual program?  And does that pay off?  Are you finding that it pays off? And then the second question is, what is the role of the new, what I would say "cottage industry" which are the intermediaries that operate between the funder and the recipient, especially in the case of the large urban districts where we have more and more instances of coaches moving in and helping urban districts to understand a reform process or helping them link up with other cities and districts that are doing similar things? MR.       :  I can probably best answer the first one, so I mean I think I would answer that on two levels, just from our experience.  We were given the luxury of the first nine or ten months of our existence to actually just run around the country and learn as much as we could about what had happened in K-12 philanthropy in terms of leadership at the school board, superintendent, et cetera, level. We brought people in to talk to us, to talk to Eli.  We had--we went all over the place to try to figure out what had been done, what evaluation had or hadn't been done, what we would need to create ourselves versus what we might look to invest in.  And that kind of due diligence at the front end has translated into now a whole process that we use before we go into a new district or before we kind of embark on a new venture whether we create that or whether we're funding something that someone else is creating. I guess what I would say is, we feel a lot smarter and better about the investment recommendation that we make to the Broads about that, but like any investment, we're still rolling the dice, and you can never find everything out.  And as I said earlier, what you believe is going to happen, strategically, because it looks nicely on paper because you have the right entrepreneur in place or whatever, is never what happens on the ground once the organization starts, moves to its first implementation wave, or tries to grow to scale.  So I think it's absolutely imperative, but it's still a roll of the dice. MR.       :  On the second question, it could be the case that intermediaries, particularly that variety we used to call--we still call--"fund- raisers," would not get in the door of any respect--self-respecting foundation.  You'd never want to see a fund-raiser; you want to see the person who's actually going to be doing the work, if you're the staff of a foundation. I think the role of intermediaries is increasing inversely relationship to the size of staffs of foundations and the growing interest on the part of donors becoming directly involved.  To the extent that you do not have people on staff who can do some of this due diligence work for you, you become more reliant on third parties, even though those third parties may be working for the eventual--the would-be recipient.  A lot will depend on how credible, obviously, the third party is. MR.       :  Well, you mentioned the word "conditionality," which prompts me to say a few words.  The--outside the United States, they may see a large amount of activity of private philanthropy in the United States as a failure of the public sector as opposed to a celebration of entrepreneurship.  And I think it's important for us to remember that there's not a universal consideration; that it's a good thing that there's so much private philanthropy. The question here that has come up again and again is that, well, you really have two different kinds:  You have supply side purposes and demand side purposes, and what people call high and low leverage.  To me it's the same thing as supply and demand side.  Much of the private philanthropy tends to be on the supply side.  Your Broad Foundation sounds as though it's getting into the incentives as well as the supply side. So you ask the question "conditionality."  Well, conditionality can come in two ways:  One is that you can say, why should philanthropists give the money away as opposed to loan it and have it paid back?  And there, once money is loaned, then the partnership becomes a real partnership; it doesn't--it's not just a donor and a recipient.  It changes the nature of the enterprise. The other is the whole question of private philanthropy writ large.  What accountability do private foundations have to say that they have accomplished what they say they have accomplished?  Where is the independent evaluation of their performance?  If you were to submit private foundations in the United States to performance standards, the same or similar kinds of performance standards that you ask of public school districts, you might find that some of them wouldn't measure up. Now, you might say, well, it's private money.  It's there--it's theirs to give away, it's theirs to dictate as to their priorities.  But every private foundation exists on a public recognition of being in a tax category which is a public subsidy, and the question is whether the public has a right to know how effective private foundations have been. MR.       :  Okay, we'll take one more brief question, right here, and then we're going to break.  So please have the other question and the responses be brief. MR.       :  Yes, okay.  We're just at an interesting moment in ed reform for reasons, Rick, you alluded to earlier.  We've had a couple of Titans in K-12 philanthropy leave the field with sort of washing their hands, Pugh & Clark.  At the same time, we have Broad, Gates, New School Venture Fund, new model foundations coming in.  This is really for Peter and Dan:  You've done a lot of study, particularly in setting up Broad, of the experience of these other foundations.  Is there something to be learned from the departure of Goliaths like that?  And, in specific, I guess what I'm wondering is whether the conditions of public education right now are best addressed by new model foundations of the kind that you're really trying to create? MR.       :  Well, I think there's an endless search in philanthropy for the kind of magic bullet.  Often it takes the form of a programmatic initiative, or even institution creation, the creation of new intermediaries and new giving institutions. I guess--I guess the turnover that you've just described in ed reform, I think it's characteristic of almost all areas of philanthropy in the sense that there is a kind of a limited--there's a limited zone during which time funders tend to kind of focus on things.  And then things change.  I mean look at, for example, Atlantic Philanthropies, a huge, huge foundation.  It worked very hard in a bunch--in a bunch of areas, particularly the support of the whole nonprofit infrastructure, just changed overnight its focus to a bunch of other areas. I think--I think you just have to kind of start to view this as part of an evolving ecology.  These foundations are coming, they're going, they're interacting with each other, and I'm not particularly troubled by it.  I think it's actually good for there to be periodic changeover. Now, but back to your point, if it changes too quick before you've had a chance to really execute any meaningful theory of change, it's very counterproductive.  But if, you know, if there's a natural turnover, it can be a chance for new ideas to come in, new models of change, new ideas for--for--for breaking through in this areas and to come through. So, you know, it really--it's really a function of how patient and how long these institutions are willing to wait before moving seems to be the critical question.  Have they done a fair job of actually trying to execute the strategy they tried to implement?  Or have they just moved on for that sake alone?  And I think that would be the question I would use to kind of judge whether or not this is a good development or a bad one. MR.       :  Okay, I think that's a good note to end the session on.  I'm going to actually shave this break by five minutes, just to get us a little closer.  So we'll reconvene at 4:25.  Thank the panelists. [Applause.] [BREAK] MR.       :  All right.  If we could take our seats.  Thanks so much.  Right now I think--right now we're running about 10 behind schedule.  That's not bad.  I think we should be able to end with that.  Still it makes me uncomfortable, but these things I'll live with. All right, the next panel is discussants, we're going to talk, to discuss a little bit of the lessons learned, you know.  In structuring these things, I always think it's a lot like classroom teaching, is that, you know, you try to set up a structure which is generally going to work, but you don't want to kind of preclude useful, thoughtful exchanges.  So, you know, these folks are not precluded by rule from making reference to conversations earlier in the day if they think it's appropriate or helpful. Most of these folks don't need any particular introduction, but because I need to do something, I'll go ahead and introduce them anyway.  We're going to speak in the order of, closest to me down to Alan, in order, it's close to me is Warren Simmons, Executive Director of the Annenberg Institute for School Reform at Brown University.   Next to Warren is Wendy Kopp, President and Founder of Teach for America. Next to Wendy is Howard Fuller, Professor of Education, and Founder of the Institute for the Transformation of Learning at Marquette University and the former Superintendent of the Milwaukee Public Schools. And next to Howard is Alan Bersin, the Superintendent of the san Diego City Schools. Warren, why don't you take us up? DR. SIMMONS:  Good late afternoon.  I'm a morning person, and so I'll do the best I can right now. I want to make a series of very quick comments.  First of all, full disclosure.  I have been a researcher.  I got my doctorate in 1979, fully intending to get tenure somewhere and do research for the rest of my life.  Then I became a federal funder at the National Institute of Education in the early 1980s.  I worked as a central office administrator out here in Prince George's County in Maryland.  I've been a private philanthropist at the Casey Foundation.  I've headed local reforms at a core organization in Philadelphia working for the Annenberg Challenge.  And I am now head of the Annenberg Institute, one of the organizations established out of the $500 million gift that Ambassador Annenberg gave to public education. So I've worked on this issue from a variety of different vantage points, and I want to acknowledge that I've found the conversation thus far not satisfied, and it's not satisfying to me because I think we have given insufficient attention to the two issues that are on the table. One is the issue of education philanthropy and the other is the issue of K-12 reform, and I think we've been mixing the conversation of the two without separating them adequately enough, because much of what we see happening and not happening in urban school districts is the product of the districts and the product of philanthropy.  But we haven't separated that out. So I want to separate that out a little bit and use the vantage point of a locality, so that you get an understanding of what it feels like and looks like.  So let me start with the papers.  Leslie's paper was very helpful because I think in the United States we don't take enough of historical context, nor do we take an international context.  So important lessons. Improvement isn't easy.  It's made even more difficult these days because of standards-based reform.  Prior to standards-based reform, if you remember when the Annenberg Challenge got off the ground in 1993, the national goal wasn't to improve all schools and all students; the goal was simply to improve some schools.  Then standards-based reform came along and said not only do you have to improve all schools, but here's the standard.  So we actually introduced a new standard that actually requires results at scale, which is a fairly new challenge for philanthropy.  And I'm not sure that we are very clear about that.  So not only is it not easy, it never has been, but it's certainly not easy when we're now asking for results at scale. I'd also second the point made by Lenkowsky that successful efforts are often not popular.  So I still don't understand the Loveless paper what said we should be more in the mainstream, because I think whether you're progressive or whether you're conservative, you are squarely not in the mainstream.  And I think one of the roles of foundations is to push communities and school districts to actually get out of the mainstream because if we stay in the mainstream, the mainstream has produced the results that we now have that we are particularly not satisfied with. The other point I want to make is that we focused on one aspect of school reform, I think largely the technical side of it, that one of the points that Lenkowsky made is that this is not just about knowledge and skills; it's about other factors as well.  And so education in this country, particularly in urban districts which I know best, are--it's a political enterprise, it's a technical enterprise, it's a cultural enterprise, it's an economic enterprise, and it's a moral enterprise. I remember talking to Thomas Cook, who did an evaluation of the Comer School Development Project in Prince George's County, and he proudly touted how he had randomly assigned the Comer Project to different elementary schools and middle schools in Prince George's County, and at the end of five years there were difference in the Comer Schools and the non-Comer Schools. And I proudly said to him, well, I was a district administrator, and it was my job to make sure that schools that weren't Comer got Comer.  And so this idea that as a leader of a moral enterprise, I would sit back quietly and allow him to randomly assign students and schools to a particular treatment and wait for three years to see what would happen.  So this is not a closed system, as we might imagine it; this is a--is an open system. So I think that in the era of standards-based reform, we have to be clearer, as Dan said, about our units of change.  And many of us are now focused on creating systems of effective schools whether we see the traditional district as the vehicle for that system or an alternative system.  There's an understanding that we need not just individually effective schools, or models of effectiveness, but a system of effective schools. I also think that we have to be much more clear about our measures of success.  And right now we tend to rely on standardized achievement tests as a singular measure of success, which I think is going to be problematic when we know that education systems are also responsible to social, emotional, spiritual, moral, and health outcomes that often precede the academic achievement, and if we're not measuring that, we have a problem with some of the leading indicators that we have to identify. So for me, then, I think one of the things that I've learned in looking at philanthropy is that we paid far more attention to what works than what we have to how it works.  If I look at the investments and evaluations, if you look at the evaluation--thank you, he said give minutes up--I'm doing fine, because I've only got two more things to say. The evaluations that I have seen, and let me talk about my time in Philadelphia, so this will give you that local context.  I was in Philadelphia when David Hornbeck was superintendent, and I was leader of the Philadelphia Education Fund.  And the interesting thing that's often not acknowledged about the Annenberg Challenge is that, actually, it was a collaborative of multiple foundations.  That is to say that in almost every site there were funders who put up matching funds, and they developed an initiative in concert with a local community's views about what should be done around six or seven general principles. So if you looked at Philadelphia, the Annenberg Challenge in Philadelphia was funded by the Annenberg Foundation but also, Pugh, William Penn, and several other funders.  So it was co-constructed.  But if you were in the district, what it felt like was that there were several funders there all under different banners: leadership, students at the center, libraries, small learning communities, technology, after school, and also each of those funders had several evaluators often running around in the same schools attributing the results to their particular intervention or treatment.  I think I left out New American Schools as another group that was in Philadelphia. So let me spend the rest of this time talking about the relationship between district reform and philanthropic reform, because I think they are parallel.  So we often chide districts for being, operating as silos.  There are various offices--curriculum, professional development, and research--don't communicate with each other, don't work with each other, don't align their work.  I would say my experience of funders in a particular city is they operate as silos.  They come with their particular bullets; they don't spend much time talking with each other, and yet they expect the district that they're working in to somehow unify and align their work.  And so they operate in many ways like an external central office. I would also say that there's another phenomenon in central offices that we acknowledge, and that is that they are not very evidence-based.  They don't operate as learning organizations.  Part of the reason why is that most of the work that they do is evaluated, but it's not documented.  So they know that something they did worked or didn't work, and they don't know why it worked or why it failed.  And again, I think funders contribute to that by not devoting enough of their resources to documenting what goes on and how capacity is built. So I think one of the unexplored and uninvested regions of reform in districts is something that I think Bruno referred to, is building the capacity to take a reform to scale; not simply what works but how do you build a local infrastructure that has some chance at sustaining the reform and improving it gradually over time, and leading to transformation. So I'd say that in my experience foundations need to begin to collaborate more to think about how they could collectively inform and help build this local infrastructure so that they can, eventually, depart and invest in other communities.  Like the central office, they need to do a much better job of communications. I think foundations need to invest in their own communications and in local communication strategies.  I think we need to shift in districts and in foundations from the emphasis on evaluation to the emphasis on documentation.  I think that because education reform is a political and cultural enterprise as well as a technical one that we need to highlight the issue of race and ethnicity in school reform.  Many of the reforms that I see opposed are not opposed so much on technical merits; they're opposed of the people who are associated with the reform don't look like that people who are having the reform imposed on them, if you will.  And there's been no effort to engage that particular community, be it African-American, Latino, or poor, or for that matter in San Diego an affluent white community, in understanding that the reform is going to do for them. And so, the other things I understand about schools and districts is that they are differentially dysfunctional and systematically so.  So we keep talking about them as if they are monolith when, in fact, they systematically work well for some constituents and systematically work poorly for others.  And the same is true for schools.  So when you talk about reform, which you're always talking about, is helping some group out while you're goring somebody else's--what?  What do you gore?  Ox.  Just checking.  Just checking if you're still awake.  And that requires this advocacy in community organizing. So when I look at the infrastructure that's needed to--and these are my closing comments--I think funders need to examine, study, and invest in communications, advocacy in organizing, action research that focuses on a documentation, how the development of knowledge management and tool dissemination systems, because as I was telling people in District 2, which is now one of the regions, they actually contract with the institutes of learning to teach them about things they develop themselves because their own district doesn't have the capacity to share knowledge and tools. And, finally, what we're currently doing supporting reforms of core [ph] organizations, but also at the local level, this cross-sector leadership development so that leaders of the African-American community, Latino community, unions, the faith community come together locally, as we do nationally, to begin to build an agenda, share some ownership of it so it can be sustained over time. I will now stop. MR.       :  Thank you, Warren.  Wendy? MS. KOPP:  Thank you, for putting together I think such a thoughtfully constructed day on such an important topic, and thank you, all of those.  I feel like Teach for America has got a little too much play today, but thank you for all the words of support from all the different sources. You know, I struggle to figure out how to sort of come out on this, but I guess I'll just start by throwing out--I mean my general take on the question of kind of, you know, evaluating philanthropy and what approach would be most effective in, you know, advancing K-12 public education I guess comes from my general vantage point and general perspective just from over the years working at every level of the system that, you know, that essentially there's good news and bad news in this effort to improve public education; that the good news is that there's absolutely nothing magical about getting from here to there.  There's no one big idea that we're missing, the bad news being that there's absolutely no way around just the incredibly hard work of building a successful institution, meaning the same incredibly hard work that you would find in any sector.  And, to me, just everything I've seen--and no doubt I'm coming in part from the vantage point of having learned the hard way how to even, you know, manage an organization like the one I run--but I feel like all those lessons that I've learned, they just resonate so loudly when you're engaging with--with school districts.  And for me, personally, I just feel like ultimately this all comes down to whether, as I think one of them just said:  We can build the capacity within the system, you know, whether we can get to the point where--in fact, at every level of the system--there is true talent in our teaching positions and our, you know, district positions and support positions and leadership; whether there are, you know, clear sort of--whether there's clear buy-in to outcomes and systems for accountability and for continuous improvement; whether they're systems that foster constant innovation in the search to get better and better outcomes as well as, you know, identification of best practices and spreading of those best practices comes down to strong cultures based on all the right things. All this is, I think, incredibly hard to achieve.  And even those who, you know, advocate for choice and vouchers and whatnot as a way to sort of put pressure on the system, I think as we even heard this morning if I got it right, that even then we still have the same questions: Because vouchers and choices are only as good as the options we've got.  And there's no way to create the options except for through engaging in the incredibly hard work of institution building. So as I step back and think, you know, that being my general construct, you know, about--about philanthropy in this context, you know, despite what I think Jay Greene showed us about sort of the, you know, level of philanthropy versus the size of the overall spending.  I mean I think we've all seen that philanthropy can play an enormously catalytic role in that.  And my, you know, feeling is that wherever we can focus those philanthropic dollars in a way that they catalyze change against that paradigm of how do we build effective institutions that we're going to be best served. I think--although, you know, I hope I'm not getting this wrong--that a lot of that is at the heart of what the Broad Foundation as per Dan's paper, you know, is trying to do.  I think it's what a lot of the other new and old groups out there from the new schools to the Pisces Foundation, they're trying to figure out I think Peter Frumkin's framework of thinking through sort of within that context, sort of, you know, I think the idea of fostering a real dialogue and evaluation around the three questions or dimensions that Peter threw out will be no doubt instructive to the field of philanthropy like, you know:  What do we think are the most powerful theories of change?  You know.  I'd love the question to be, you know:  What are our theories of change around how we get to the point where we have true strong institutions within public education?  You know, what are our theories of engagement?  What have we learned about effective versus ineffective engagement?  You know, what are our theories of scale?  And I believe we must feel a moral imperative to scale our efforts while not sacrificing quality. I just want to share a couple of other things.  I mean one of the most striking things I thought about the set of papers that we're supposed to be commenting on, at least from my vantage point, was just reading Les's kind of history, and just feeling like it's such an interesting check, for someone like me at least, who, you know, I'm focused so much in the short term and thinking, you know, feeling such a great sense of urgency.  And yet it sort of led me to step back and think, you know, ultimately--I mean I think that's important and we all need to measure results, and we need to be accountable in the short run--but at the same time I think that it's interesting to ask ourselves, what do we think 20, 25 years from now when people look back on--on this era?  What do people think, you know, the most influential philanthropic investments will have been?  And I do sometimes wonder whether we're almost too short-sighted, and whether we don't need to think more broadly. I mean I think to one of the points that Ken made earlier around sometimes the things that seem most systemic are not necessarily the initiatives that at the end of the day effect the greatest systemic change.  And I just--I think it's interesting to look both from the short-term vantage point, but also to step back and really think broadly about, you know, long-term impacts. And then, finally--and I hope I'm not misinterpreting Professor Heyneman's suggestion around the World Bank, and no doubt this was not put up against "let's eliminate all other philanthropy and create a world bank"--but I'll use that as a launch pad to say that I think it's incredibly--I think it's incredibly healthy and just, you know, think, heaven, I think that we have so many different approaches out there and entrepreneurs and business people and philanthropists coming from different perspectives in this.  And I think that it's all of those things that will, hopefully one day--I personally feel like there's a lot of momentum building in--in the K-12 education reform movement. And I'm going to use this as the chance to get into the Tom Loveless commentary.  At the same time, I think it's so important that--and I think Dan's paper spoke to this--that we really recognize there's no way we're going to succeed in this without--without the leadership coming from within the system.  And what I feel strongly is that there's actually no way to do this from the sidelines; that you can't really know unless you're in it.  I mean as someone who's never taught, I feel like I can say that to a room full of some people who also haven't taught.  But I do--it's actually not surprising to me--I mean I guess there's some debate as to whether, you know, whether those are the right questions.  But if we were to hypothesize that, we would make more grounded decisions, decisions that related more to parents and communities and what they want, if people in positions of influence had the experience of teaching and, hopefully, teaching successfully, which I think is a key distinction, but in low-income communities and in general I think, personally--I mean that's, of course, our theory of change--but, you know, I think there's every reason to believe that that's true.  I think it's fascinating, and I hope we'll further explore that and that the paper--which I haven't read I must say, but I did just grab it--you know, I think it'll be interesting to further explore. My last thing, if I have one more minute is to circle back around to some things that Richard Colvin said at the beginning and how this whole day started, because I think, you know, the topic of journalism and its impact, and, you know, one of the things Richard said was that, you know, maybe journalists don't feel like they know enough to be critical of philanthropy.  And that just struck me as so--a little bit strange, because I feel like if anything journalists think they know everything and they're such a force for tearing down the visionaries out there, and for tearing down--I read The New York Times and it kills me. I mean I'm revealing my perspective on Joe Klein's reforms, but, personally, I think more people with grounded, real success like experience being successful within the education reform movement shaping the way we think as a country as generally to be incredibly powerful. [BREAK] [END TAPE 5, BEGIN TAPE 6] MR. FULLER:  Good afternoon.  As everybody else did--not everybody but many people--sort of thanked the funders who fund them, and I want to do the same thing, but I also want to say that I assume that you fund things that I've been involved in because you believe in it, and the last thing you want me to be is mealy-mouthed and afraid.  So I'm going to say what I got to say and then see if I ever get any more funding.  [Laughter.] First things, I do want to thank Rick for pulling us together, but I also want to say to Rick that--and I hope you'll all just have to understand me that--that we also got to find some black people and some Latino people who can write papers in addition to just responding, because we can do that, too.  I think we ought to--also ought to look around this room and understand how un-American this room is in terms of the number of people of color who are in this room.  And we ought to all be trying to like figure out ways to make sure that this kind of thing doesn't continue to happen. And, I mean I have to say that because, as a person who's in the room, I think I have a particular responsibility to make the case that more of us need to be in these rooms if we're really serious about the conversation we're having today. Let me make a general point, and then I'll respond to three of the papers.  I read Steven's paper, and I was going to try to respond, but I realize it was beyond me and I wasn't feeling it, so I just--so I'm just going to talk about the three, and maybe after I respond to the three you all will say I probably shouldn't have responded to those either. But let me make a general point.  The general point is that the perspective of funders by different audiences at a school district, that I found as a superintendent is that the reaction of particularly unions and other kind of organized interests when certain funders came in their immediate reaction was to attack the funders because, you know, either you're coming in because you're trying to privatize the public education system, or to begin to attack the corporations that, in fact, created the foundations.  And so you hear all this talk about how they treat their workers and all of that.  And that's a whole element of this that I really didn't hear any discussion about, but I think has a real impact on what happens when you're out there in school districts trying to find funding and so forth. I thought also, you know, this morning there continues to be a lot of this what I call pejorative types of terminology, you know, the liberal/conservative/progressive.  And I'm always trying to figure out who appointed themselves to be able to make that decision as to who's progressive and who's not, and how you make those kind of determinations. And Bill Porter invited me to speak for the Grantmakers for Education thing several months ago, and I'm going to repeat something that I said there, because I think it's a part of a general point I want to make.  It is there are a lot of times what is also happening here is that individuals get labeled by virtue of who they get their funding from.  And what I thought was most interesting was I had a group of young people come in to meet with me at Marquette, and they were brought there by the Rockefeller Foundation.  And they proceeded to tell me how suspicious they were of what I was doing because I took money from the Bradley Foundation. And what I thought was very interesting is I asked them:  Do you really want to have a conversation about how the Rockefellers got their money, or--or--or do we--do we not want to have that conversation?  And I'm raising that because there is a lot of this stuff that goes on, and we allow to happen.  And I'm just--I'm just addressing it in a--in a straightforward way, because it's used as a way to demagogue people who are trying to engage in change.  And, to me, that's the reality that we face. Let me just respond to the three papers.  First of all to Peter's paper--and I found all of these papers--and I really mean it--very interesting and I was happy to read them--I was hoping, though, when Peter raised the question of theories of change, I was hoping that what he was going to do was to take it to a discussion of collaboration versus conflict as a basis for change because so many foundations and other funders are insisting on collaboration.  And I would argue that a lot of the requirements that people collaborate in school districts set people up for what I would call illusionary change, which is a lot of bells and whistles, but at the end of the day there is no change that can be sustained because, as Lawrence Patrick this morning said, there's no effort to change the power-relationships that exist within these districts.  And so therefore, in my view, the idea--and--but the idea that there's really going to be fundamental change in these districts is in my mind illusionary. And at a certain level I agree with Warren.  I saw a lot of this stuff as more technical discussions.  It's almost like we're trying to find the science of giving to bring about reform as opposed to the politics, the real stuff that goes on when you're really down in these districts trying to make change.  And it's less of technical questions, and it's much more the--the fundamental kind of political issues that you have to wrestle with every day. And the other thing I was wishing Peter would have done was he mentioned this open system/closed system, and in my view school districts are closed systems.  They're not open systems, but we go after them in terms of our way of reforming them as if they are open systems. Let me talk about Leslie's paper, which I really found intriguing, and there's just a couple point I wanted to make, is I thought he made some very chief points, but they're almost understated because they were the last sentence in--in--in his point of discussion.  The one where he says that philanthropy is often most useful when it supports innovations that would not normally get government backing.  And then he said also that pushing past notions of good policy and practice to issues where there's a social and moral vision I thought was real critical because, essentially, what I thought you were arguing was that we don't need to just be focusing on collaboration when collaboration is not taking place between people who have equal power, and that until such time as we really try to create a situation where there's more equity in the power relationships, the discussion about collaborations, to me, is not going to lead to really fundamental change. And what worries me is that too many foundations want to be safe.  It's almost like what Ted Colter [ph] recalls "fuzzy altruism."  And so my question is, how are we going to deal with this issue of investment that show results if the objective is social justice is social change and actually changing the power relationships, what are the evaluation instruments that we're going to use to determine if that, in fact, occurs? And then on Dan and Wendy's paper, I just want to talk about Lesson Number 3.  And, actually, Dan was a lot more forceful in his statements than in the paper, because--and so I was all set to--I mean when he said it I was like, oh, man, he should have put that in the paper.  But when he got to Lesson 3, I mean he said, "I want to ask the question, why would it be a bad thing for charters to be a threat to the existing system?  I don't see a reason why we got to argue that the reason why charters are a good thing is 'cause they're not a threat to the existing system." My argument is what makes them a good thing is they are, in fact, a threat to the traditional system.  And I thought this was an excellent opportunity for them to say in the paper kind of what Dan said in his verbal thing, because in the paper it's much more of a Rodney King analysis of the world, you know, "Can't we just all get along?"  And--and--so--yeah. But so again, to me what was missing was the development of the political apparatus that will fight to sustain the change beyond the superintendent's or the board's tenure, because I would argue unless foundations start paying more attention to that, all of these investments that you're making, even if you get some minimal exchange, when you lost Alan, you lose other people, that--that to sustain that change without fighting to develop a political base that's prepared to fight the people who--the status quo that are going to always be there trying to maintain their power--I think we're missing something. The next thing is what I found was really interesting was the list of things that Dan talked about was an indicator of--of a place that they saw as a promising investment environment, you know, like a strong capable superintendent, collaborative functioning board, supportive business, and philanthropy.  The one thing I thought was missing there was the organization of the people, because unless you're going to--I'm--I actually believe what we used to say in the '60s, that we need to empower the people.  And--and--and what I actually believe now is so many of us who used to talk about empowering the people and fighting the bureaucracy are now the bureaucracy, and we're no longer interested in empowering the people.  And because we're not, no longer interested in empowering the people, we're not--we're not investing in empowering the people who have the most at stake for this change that were talked about, and that's parents and their children. And--and--and so for me, we need to have a lot more discussion about that approach to change and whether or not people are going to have a lot more willingness to deal with conflict and--and--and deal with how messy all of that is and how difficult it is to, quote, "evaluate." MR.       :  Thank you, Howard.  Alan? MR. BERSIN:  Listening to Howard gives me hope that there is life after the superintendency--[laughter]--and, in fact, it is only by listening to Howard that I think perhaps I was too moderate in my--in my approach. I want to start by actually cautioning all of you at the same time that I thank you for subjecting yourself to the same kinds of self-examination that you counsel us to engage in so frequently, and we do it--we do it hardly at all.  I think today has been a start, but I agree with Warren.  I think that this is--if the discussion stopped here, it would not be good for the sector, neither public education nor philanthropy.  In fact, as I listen to Warren tick off all of his involvements, I was thinking with hat on two careers ago "guilty as charged" in terms of the embarrassment with which we're looking at all--we're starting to look at this and say maybe there's some conflicts of interest engaged in this endeavor.  Maybe, in fact, we have to ask some very hard questions about where we fit into the system in the same way that so many of us inside the system have conflicts about acknowledging difficulties, looking at power relations honestly, facing up to the monumental task that this transformation that we all seek on behalf of our children is going to involve. So I say thank you, because it's not an easy process.  It's not an easy process to look and to recognize that, in fact, maybe by being involved with the K-12 section you're catching some of our maladies, that, in fact, instead of forcing us to change and keeping the edge on, in fact, perhaps you're adopting some of our worst habits, those habits being what Jack Welch talks about as "superficial congeniality."  This is all about the adults getting along.  This is all about us being smooth and, as Howard says, not rocking the boat too much.  That reflects itself, manifests itself in evaluations that, frankly, are not tough enough on us and not tough enough on yourselves in terms of what works and what doesn't. I wouldn't waste much time on hand- wringing about what the lack of leverage.  I'm amazed at the amount of leverage that philanthropists and foundations have on the K-12 system.  After all, this is a $450 billion annual expenditure sector, $2 billion that's spent in philanthropy has an enormous influence on the policy and practice and procedures of our districts. Where else, I ask you, and could you possibly have a $715 million expenditure over seven years that Bill and Melinda Gates Foundation has made, which represents hardly 75 percent of an annual budget in San Diego, and suddenly the whole way in which the country's talking about high schools is changed? Now, we've engaged in that discussion because we agree with it as far as it goes.  But the point here is, in what other sector would that kind of expenditure compared with and relative to the annual expenditure could you have that kind of an influence?  So leverage is actually not the problem. The problem, I suggest, is accountability.  And with all due respect--and I care deeply about you because I, too, am a laborer in the vineyards.  And, actually, I'm more optimistic than some of my colleagues.  I think, when we look back 30 years from now, some of the people sitting in this room are going to get some credit for some progress that will be made.  That doesn't mean that we should be self-satisfied.  But, frankly, this sector flees from accountability every bit as much as the sector you purport to help. This is a--it seems built into it.  And I asked myself, how could it be that you have so much influence on the system?  One of those factors is that the knowledge base in the education sector is extraordinarily weak.  You couldn't have this impact in professions where there is a strong knowledge base, and people would say, "What are you talking about?" We have little self-confidence in our endeavor, and for that reason we have been for the last generation willing to take virtually any theory of action that comes along and someone's willing to invest in, because it fills a vacuum.  that's a dangerous situation. The second is the way in which public education is financed.  It's premium on soft dollars.  Because we have wrapped our budgetary allocations around political accommodations, and because so few are willing to take on the conflict of changing those allegations, in fact, there are very soft dollars have an enormous influence in the sector when in fact the number of dollars is not great. Look, I say this one again, being generally an optimist or I could not have spent these years, and also being grateful for those who have labored alongside of us.  But one of the issues that's being faced, I think by the foundations, that has never been reflected before--and Richard raised it early on today--we hear different rhetoric from Eli Broad and different rhetoric from Bill Gates.  I think for the first time our foundations are confronting the fact that we actually have to face up to the issues of race and class very directly if we're going to accelerate if we're going to accelerate the growths in the innercity.  And, by the way, I take most of the discussion today, most of what I know about, like Arlene, is urban systems, large national foundations.  And I limit this comment to that:  I don't think--it would be ironic to think that the titans, after all, as Professor Heyneman points out, we are very different in this country.  Foundations and where they come from are very different.  If it were not for the tax code, none of us would be sitting here today.  That's a very unique situation that creates a peculiar position because it would hardly be--it's what the Marxists would call an "inherent contradiction" to think that fundamental change is going to come from the Carnegies, the Fords, the Gates, the Broads, in terms of changing the conditions under which the system has operated. I think it's important that we--we take heed from a professor from Leslie's paper because, in fact, we've been through this before.  It's just a new cycle of--of opportunities, but also challenges.  The difference is that people are less patient than they were in the '30s, '40s, '50s, '60s, '70s and '80s, and maybe even the '90s.  The question is not are we making progress.  We are making incremental progress in the urban systems; the issue is, are we making it fast enough to satisfy the expectations both in communities we now largely serve of color and from lower socioeconomic areas?  And are we making it fast enough to satisfy the taxpaid--taxpaying, voting public?  The jury's still out on that issue. So, very quickly, what--what should the agenda look like if--as we go forward, what's the causal claims that we ought to be making about the investment of these--these funds?  I think I would break it down in terms of building up the knowledge base of the sector.  How is it possible?  The foundation community has been extraordinary in creating more transparency around student achievement.  But how is it possible that in 2005 we don't have generally accepted assessment principles that every bit of data can become the subject of a usually political fight about what it shows and what it means?  We need to actually address that issue and invest in it, and I think the foundations are the ones who have to do that. How is it possible that we don't have an existence proof of the student identification system or teacher identifier system that is capable of giving us longitudinal data with regularity and with accuracy?  How is it possible that we're still debating at this juncture what colleges of education should look like when, in fact, we all agree, even most professors of education, that radical transformation is in order?  We need to actually think that through and publish it. We need to invest in research strategies.  The problem is that, in fact, when you look at the knowledge base after some phonemic awareness for early literacy purposes, we don't know what to tell our teachers to do in terms of instructional strategies.  We have not, if we'd invested one-tenth of one percent in--in how to teach English language learners how to access academic language that we've spent on fixing broken arms, we'd be much further ahead.  We need to create a climate in this country in which those investments become safe. The second series of investments, I'm a believer with Howard and others in this room, competition.  As long as our sector fears competition, the shorter our life span will be.  There is no other sector that have survived; we have the last legal monopolies operating in America in our sector, and we simply have got to welcome and embrace competition.  And I think the work that's being done in the context of charter schools and charter management organizations absolutely essential, absolutely essential from the standpoint of a practitioner. And lastly, in terms of how to improve the system, I think--and I commend all of you for having theories of action as to how to change the system.  The fact is I think we've known how to improve urban education for the last generation.  We need excellent principals in our school; we need highly-trained and honored teachers; we need modern books and materials; we need authentic parent involvement in the schools, and we need extended learning opportunities for our kids who need more time to master the standards.  We've known that for almost a generation.  These are all problems of implementation, and they are problems that involve deep political issues.  And as long as we always have 800-pound gorillas sit in our meetings and never get mentioned whether they had the relationship to local politics, and I would say school boards and governance issues have got to be put on the table and looked at. The link between school board elections and labor unions have got to be looked at, not that the alternatives that have been suggested are any help.  Indeed, I got message today the Mayor of San Diego resigned as of July 15th, so no necessary continuity there. But the fact of the matter is, even if we don't have anything extensive to say about it, we've actually got to put the issues on the table and then go there.  And then go from there. MR.       :  Thank you, Al.  We have some time for some Q and A.  Please catch Morgan's eye. MR.       :  This question is for Alan, but again I welcome a response from anyone.  Alan, you mentioned in some of your earlier comments that you--you wondered whether it was built into the nature of foundations that they were unaccountable somehow with an inherent part of the sector.  And learning from--from Peter's work especially, I think that is one of the distinctive organizational features of the sector in the sense that public institutions, public officials are accountable to electorates for profit organizations accountable to clients, consumers, but foundations, especially those that operate from endowments, are essentially accountable to no one. And so since all of you at the--on the panel now have received money and were wondering about accountableing [ph] questions, I'm wondering if you could describe what kind of accountability regime you'd like to see foundations subject to. MR. BERSIN:  First of all, it's the willingness to look at the issue so that if this were the last of this kind of meeting, it would not be good for the sector.  But accountability starts with transparency.  It starts with transparency, and absent transparent--that is sort of a toting up of expenditures versus results, and doing it in some public way in which we were urged to publish our student data, I think that's the place I would start. MR.       :  Yeah.  I think foundations are accountable, they're just not publicly accountable.  That is to say in my experience, after six years you'll see a change in the directors and the program officers, and even the foundation president.  And oftentimes that change, because, internally, they've looked at the data, and they've decided that they need to shift directions.  The problem is that isn't publicly discussed.  The criteria that are used isn't publicly discussed, and it isn't communicated.  So there's a good deal of accountability, but it's not transparent, it's not public, and, therefore, our ability to learn from it is limited. MR.       :  I have a question for Howard.  At what point does the--the revolutionary become the bureaucrat?  At what point does the effort to sustain change over time become merely the status quo, and then how do you break the cycle then? MR. FULLER:  You know, one of the things, you know, if you study revolutions, some of the people who fight revolutions should never be a part of the governance.  But having said that, having tried it, I would say that our problem is--and I say this even to charter school people--we cannot get committed to charters; we have to get committed to the reason why we have charters, because if 10 years from now charters are no different than the traditional system, why would we fight for them?  And so for me to issue how you deal with that problem is an issue of whether or not--whether you're committed to purpose or whether or not you're committed to the entity that's supposed to achieve the purpose.  And I would argue the moment you become committed to the entity and not the purpose is at that point that you're no longer a fighter for change because, by definition, what you become is a supporter of the status quo.  And I think that that is very, very difficult to do, but if you truly believe in the people, I think it's the posture that you have to take. Now, having said that, what one of the problems you fight--and I don't know if Alan felt this--but when I went in as a superintendent, you know, according to "Ed Week," I was the first of the so-called nontraditional superintendents during this period.  My goal was not to get captured by the bureaucracy because, when you're outside of the bureaucracy, you can talk about the faceless people.  When you get inside, those are not faceless people.  Those are people with names, and they're good people, and they got kids, and they work hard, and, you know, you're looking at them in the face every day.  So what was real hard for me to say was, "I'm like all of you all, but the system that we're in is totally dysfunctional, and I refuse to get captured by the system, but I don't want you all to misinterpret this that I don't like you as individual people.  It's just that this thing that we're in is--is--is not working for large numbers of our kids."  And every day when I went in there, I had to keep that in mind because, otherwise, I felt like I was going to become an apologist for the system. MR. MATTHEWS:  I'm Doug Matthews from Teaching Point.  We're producers of teacher preparation materials for highly qualified teachers, and I was thrilled to hear Alan Bersin say that we need to develop a knowledge base for teaching, and that we've often thought that where would medicine, architecture, engineering be if we hadn't over generations developed a knowledge base for those careers and professions?  And so I really think we need to do the same thing for teaching.  That's, for us, a-- MR.       :  What's the question? MR. MATTHEWS:  Yeah, I'm getting to the question.  The preservation project is, essentially, collecting the codified career knowledge of teachers around the country, because we have the lowest quintile of teachers, or college graduates going into teaching now.  And if we can preserve this knowledge base, I'm interested in the form in which Alan sees us. MR. BERSIN:  Well, usually, I don't think throwing money at a problem helps, and I don't think the foundations in this room or indeed in the country have enough money to actually solve this.  The issue is creating the climate in which we can make the investments over the next generation on the scale that have got to be made to get a handle on some of the teaching issues.  It's a--it goes to the nature of the instructional core or--and I thank one of the participants today, actually--well, I used to define it along Elmore's terms as the interaction of teacher and student in the presence of content.  It's actually more complicated:  It's the interaction of teacher and learner in the presence of content.  And because that's an emotional cognitive process and product, it's a lot more difficult than fixing a broken arm.  But there's a lot more that we can say about this, and, frankly, we've got to change the nature of our research industry, the research agenda that we have now.  And I think a lot of this is because of the way in which it's funded by government entities as well as foundations is a very fragmented research agenda, so that we have 20,000 people that go to the conference in Montreal, and you ask, what, actually, is the result?  What is the nature of that research that's being produced by that quantum of attention?  And the answer is, precious little that helps our teachers. So I think it's the role of a catalyst philanthropist would be to start to create the conditions by illuminating the issues about how we got to where we are and what kinds of investments it's going to take to get that third teacher some very practical knowledge about how to teach the English language learner algebra. MR.       :  I want to answer that question as well, because I talked about this issue of the importance of a knowledge management tool development system.  When I was in the U.K. recently and asked a teacher in an urban school, show me the curriculum, because I typically do that, because I often find when I ask that question in the United States, people don't know what to show me, or they show me something they actually never use.  And it's clear that they never use it. But when I asked that question, I was surprised that they actually, the head teacher which is their notion of principal, turned to the computer, turned it on, showed me the national curriculum guide, and showed me the adaptations to it that local teachers had made, and talked about how they used the technology to further their work locally. I don't see systems like that in many urban schools, which means that we have this experience of reinventing tools and knowledge over and over again which waste a lot of time and effort.  So I think investing in web based knowledge management, tool development and sharing infrastructure locally, not simply nationally, would strengthen our ability not only to know what works, but also to collect the information that, more importantly, answers the question of how.  I think the limitation of a what works clearing house operating in a national level is that it doesn't help you do the adaptation that you have to do in San Diego versus Chattanooga versus New York City. MS. WALSH:  Kate Walsh of the National Council in Teach Quality.  I'd like to follow up a little bit what Arlene was talking about today in terms of how she was able or not able to attract funding from foundations and that others were getting that she didn't feel were nearly as competent or as visionary or whatever the terms you wanted to use, and get Mr. Fuller or Mr. Bersin's reaction to that. Did you find, as superintendents, that you were all that interested in pursuing funding from foundations, or did you find it really distracted you, the time it took, the investment it took into gaining those funds? MR. FULLER:  You know, we--we too, the same approach, I think, that Arlene talked about in that--at least my argument was that the money shouldn't define what we were doing, that we needed to define what we were doing, and then if we could find money that would support that, we should do it. A lot of my staff told me that I was crazy because the tendency before that was any dollars out there anywhere, you just go get it.  And as a result you have all this disparate stuff going on.  There's no connection, and I couldn't--I couldn't stand that.  So I think in the period that I was there, we probably did not get a lot of foundation funding.  I really didn't go after a lot of it, but the big grant I think we got was the National Science Foundation thing.  But that was a part of our overall strategy to try to reform curriculum. MS. WALSH:  Yes.  So-- MR. BERSIN:  This is one area where Arlene and I have different experience.  We could not have leveraged the reform without the help of Board, Yulet [ph], Gates, Atlantic, National Science Foundation.  That was after, as Richard recounted, the first meeting with our previous funders was not so successful, because they had a different theory of action than we did, and we parted company, although the NSF eventually came back. See, the way in which public education is financed, particularly in California, makes it very difficult if you don't have those catalytic dollars.  Also, under federal law we couldn't have made Title 1 work, you know, the supplement not supplant rules of the Department of Education.  We could not have made, gotten literacy coaches and math coaches into every school--every school, if we didn't have those dollars.  We could not have started a leadership development academy which has produced 25 percent of our school leaders, had we not had that jump start in that investment. So I think if you have a point of view, if you have a theory of action, and you can build a collaborative relationship along Peter's typology with a funder, I think it can be a very valuable and certainly worthwhile enterprise. MR. FULLER:  If I could just add one other thing, I think one of the things that I dealt with was that the kind of resources that are now available to support so-called nontraditional superintendents and nontraditional approaches in districts, it wasn't there.  I mean, and so the Broads of the world, I mean they weren't there then to step up to the plate to support some of this. A second point that I would make, and I don't know if Arlene would agree with it, but one of the concerns that I had inside the school district was the best way for schools to get additional money was to fail kids.  And then the people who actually were successful with kids got nothing, and I was always confused, you know, about that--not that those schools didn't need additional resources, but the message that you sent inside the district was deadly in terms of, what does it take to get additional resources?  And that may very well be what happened to Arlene. MR.       :  [Indistinguishable.] MR.       :  Yeah, I was going to say something because Arlene's prompting me to say something.  And also, I have to go to the issue of race and class again.  So one of the things that I observed in education reform is because the knowledge base is either weak or polarized, because I'm not sure what it is, but it's a combination of the two, weak and polarized. Foundations tend to rely on gurus.  There are certain people that foundations go to on a regular basis when they ask the question:  What should we do?  And District 2 is one district.  How did it get the prominence that it received?  Because a guru evaluated it, not that there weren't other districts in New York City that were doing similar kinds of work or there weren't other districts around the country doing similar kinds of work.  But because that one district was evaluated or, more accurately, documented and documented by a nationally recognized guru, that increased the chances tremendously.  I would say that, you know, Arlene does decent work, and she attached herself to a guru--I'm not a guru--a sort of quasi-guru, I think.  [Laughter] Quasi-guru, if you attached yourself to a guru, a guru and got your work documented, you'd be more likely to attract.  So I think that this, how we break the hegemony of gurus and broaden the knowledge base so that we can, on both sides, districts and philanthropy, become more evidence based, the more pluralistic in the approach, I think, is a challenge that we have to ask ourselves. MR.       :  Warren, we have time for one more question.  Actually, then we'll get the priest [ph] to-- Let's take two questions real quick, and then we'll deal with them together, and then, actually, it occurs to me what we could do, switch panels rather than take a 10-minute break.  We could just take a moment or two, folks who are planning to depart or what not could go out, and that way we could then just--now that the full break?  Okay, we'll take the full break.  Let's take these last two questions real quick.  No? Okay, so let's take--we'll take a moment or two, and we'll just switch panels.  I ask you to pretty much stay seated as you're departing.  Okay, last couple questions. MR. KAIR [PH]:  Kolim Kair [ph], Fight for Children in Washington, D.C.,  just two quick questions for you to comment on either one or both. Can we--can foundations reasonably expect significant outcomes or a manifestation of their desire of reforms without addressing the economic infrastructure challenges within poor communities and cities? And second, what role, if any, can the black church, which still remains the major source of philanthropy among African-Americans, play in the school reform movement? MR.       :  Okay.  I need you to keep responses short.  Caprice? MS.       :  A related question, I suppose, and I was looking forward as No Child Left Behind kicks in and schools are closed, reconstituted, or slapped on the wrist, what should the philanthropists do to support that process? MR.       :  Let me take a shot at Kolim's first question, which is sort of like the Richard Rothstein's view of the world, that you got to eliminate poverty before we can help kids.  I think that's an interesting view of the world, but it's not about to happen anytime soon.  And so my view is that irrespective of the economic conditions of the kids, we have to figure out a way to educate them.  And the fact that there are people who are doing it tells us that it can be done, so that for people who are not doing it, the reality is you're not doing it but you can't make the argument that it cannot be done. Now, you know, I understand--I'm not an idiot--I understand the impact of poverty on our children's life chances, but the moment we buy into the notion that unless we're able to fundamentally change that structure first before we can educate our kids, we might as well close up shop, because that's not about to happen in America.  And so irrespective of those difficulties, we have to figure out ways to educate kids.  But the fact of the matter is--and everybody's said it--there isn't a lot of magic to it. For example, if a kid is three or four grade levels behind, if you think that you can reach them and bring them up with a regular school day, that's not going to happen without more time, more energy, more resources put into what those kids need.  And so what I worry about, Kolim, is that people use this economic argument as a way to duck their responsibility to deal with the reality of where these kids are today, and the fact that it's an issue of political will as opposed to the lack of know-how. MR.       :  Any--[indistinguishable]--respond to that? MR.       :  I think the economic realities of life in urban cities does need to be addressed, and I think there are opportunities where foundations, if they worked collaboratively and in partnership with each other, could address them. Providence, for instance, has Carnegie on high school reform, and Gates on high school reform, and has Wallace on leadership, and Wallace on after school.  It has the Casey Foundation doing community building, family center community building, and I would venture to say that few of those foundations have conversations together about how they might unify their approach so that the city as a whole will build the infrastructure it needs to create change. And so when we--even when we do have opportunities, we don't use them well.  So I think in some instances, I think some funders should think about how they can take a place-based approach and distribute their resources and interests so that it actually builds as stronger community infrastructure than we have now. MS. KOPP:  I just wanted to use the first question and jumping off of Howard's answer to make one more point because it's sort of nagging at me here, just because there's so much talk about teacher quality and what not.  And I just think, you know, there are teachers who are highly successful with kids in low-income communities, as we all know, like teachers who are advancing their kids dramatically despite all the broader socioeconomic challenges, just as there are schools that do. And we've seen through looking at what it is that differentiates those teachers that there are just--there are immense lessons, and I'm not sure that there is a connection between what we've found those lessons to be and where the broad general thought right now is around teacher quality.  And I mean we're--we need to get our act together and communicate what we've learned.  But I think, too, that it's just an interesting question and just framework in general.  Like I just wish policymakers and philanthropists and journalists would look at what is working out there.  And that's where we find the answers and then figure out, okay, now how do we get from where we are to what is working? MR. SIMMONS:  Let me have a crack at  Caprice's question.  No Child Left Behind in the year four provisions offers a tremendous opportunity for districts and foundations to work together to restructure schools, and there are five different ways to restructure schools: everything from contracting out to a university to contracting out to a labor union, to chartering the school and complete restructuring. Our experience in San Diego has been very encouraging, and it was largely through good critical friendship from the Carnegie Foundation combined with capacity that the Annie Casey Foundation provided in our communications that we actually had teachers and parents in the most genuine, authentic emergency of Grassley democracy that I have seen actually come and vote to take four of the eight schools in year four into a charter status.  It was very hard work, and it involved extraordinary, ongoing project meetings.  But parent leaders and teacher leaders actually voted to take their school in concert with the University of San Diego at--University of California at San Diego in one case, and University of San Diego in another case into a charter school. The challenge to the foundation world, I believe, is to see that these schools don't fail.  And I would end in a more general point that the large problem in public education systems is the lack of capacity.  We just don't have the capacity to support our teachers properly, to support our students properly, so I would ask that foundations consider what we're calling in San Diego "accountable consulting," which is that you actually take responsibility for project management, actually help the District both get a job done and leave capacity behind in terms of district people. We actually have a good example with the American Institutes of Research of a consultant who has actually taken charge in that way and is accountable in a very meaningful way. MR.       :  Okay, with that, let's wrap this panel.  I'd like to thank you so much. [Applause.] [BREAK] MR.       :  We're going to go ahead and get started with the closing session, and after this we will all adjourn for a stiff drink, which we all will need by the end of it. This last session I--you know, Bruno had mentioned earlier we discussed some of the dearth of folks actually at foundations, partly by design, partly just the way these things work out, and I thought it would be particularly useful kind of at the end of the day to have this other conversation, this broader conversation, and then let some of the folks, some of the folks who I consider some of the leading thinkers kind of in the field of the foundation world, respond to their kind of perspectives of being on the front lines of this.   The three are Dan Fallon, Chair of the Education Division at the Carnegie Corporation; Stefanie Sanford, Senior Policy Officer at the Bill and Melinda Gates Foundation; and Chester Finn, President of the Thomas B. Fordham Foundation. We'll just go ahead and go Stefanie, and then Dan and Chester are going to speak for eight to ten minutes.  Then we're going to take about 20 minutes of kind of Q and A.  What I particularly hope your questions might focus on in this last session is the suggestion for me to take up and try to craft it for volume and work with the authors tomorrow, what are some thoughts that we should keep in mind,  and what are some takeaways that you would think most valuable? Stefanie. MS. SANFORD:  Thank you.  This has been a terrific day, and congratulations on being such hearty souls.  I really am impressed that this many people have stayed, and I also am acutely aware of the fact that I am all that stands between you and free liquor.  [Laughter.]  So I will keep my remarks--my remarks brief. It really--this has been a terrific day.  I spent the weekend with these papers and found that an excellent weekend of professional development to spend time thinking about these things.  There was a little blurb that alarmed us.  I think it flew around on either Thursday or Friday in The National Journal that said:  At last somebody was going smack the Gates Foundation.  Welcome to AEI.  And so I came in sort of, you know, steeled for what might be coming, because frankly, coming to the Gates Foundation means never being told that you have a bad idea. And I think that while that may be socially attractive, it is--it's not good for us, it's not good for the sector, and so I think these papers were and discussions were lively and on point, and I would encourage more of this.  I think it's good for all of us to be able to do that.  And so for that I thank you. As I read Jay Greene's paper, I had a vision that he must have been standing outside our conference room a couple of weeks ago in Seattle with one edge of his ear against a glass and the other against the door so right on was he about the sort of interplay or the balance between program and sort of research in advocacy, and what would be the greatest lever for us.  It's tempting to think both because of this, the genuine scale of the Gates Foundation, comparatively, as well as the hype around the scale that if you can't make a difference here, you can't make a difference anywhere.  And, frankly, it's a little bit of an existential crisis when some of our evaluations come back and they're not so good. I loved the Russia analogy.  I think it really helped put things--put things in perspective for us.  It's hard to imagine that investments of our size can only be rounding error, but at last we know that they are so we have to be very thoughtful about what it is that we do.  So I think that that paper--I'll circulate all of these to our staff--but that one in particular seemed right on point and right on timing for us. Also, thank you to Peter Frumkin for the work on theory of change.  We have been through a pretty rigorous theory of change process with the-Bridge Span Group [ph] on our own work internally, and it can only be compared, I suspect, to a colonoscopy.  [Laughter.]  We have decided that if we had to go through it, our best grantees should, and so we are funding that to build capacity for our organizations. MR.       :  Do you have funding for gastroenterology? MS. SANFORD:  We're expanding our strategies.  And mainly, we, as it was touched on by many of the authors today, we surfed the tension between trying to be bold and transformational and trying to be measurable in the short run.  And I think a number of you said to me very smart, which is if you want to be perfectly safe and you want to be able to show causal effects really, really quick, take on, put a lot of money in a little--in a little prescribed thing and--and see what happens.  And we have--we and our benefactors have larger aims than that. So I'm going to just make four quick points into what's next, which was the title of this session.  We are moving towards, in our program work the creation of proof points, and that is, are there great new schools, new structures, new relationship, new forms that--that--that really work better for all kids?  I think Checker said it really well when he introduced Tom at the philanthropy roundtable about a month or so ago.  We are relentlessly focused on our goals, but we have been willing to learn as our strategy has unfolded and as we have evaluated and have modified some of our grantmaking as a result. And the focus--the realization that even as big as our program investments are, we're not going to get to the math.  If we really are going to be serious about preparing all kids to graduate ready to go on to post-secondary without the need for remediation, we're not going to be able to do that without animating and moving public money.  And the way to do that, we think, is to take on the risk of innovation.  Is that something that governments really can't and shouldn't do?  Because if some of our investments don't work then we learn and move on that the political and operational, perhaps ethical problem in government, and it's really not well suited to do that.  So we see that as a unique role for our sector. We will increasingly focus on new schools and networks of new schools.  Our most disappointing evaluation findings were just how difficult it is to create a new culture and to create new forms and do what we call conversion schools in existing schools.  It doesn't mean that we're giving up on that, but it does mean that we believe in the value and the primacy of competition, that we think if you're going to be serious about bold innovation, you need to start--you need to start and build from the ground up. Secondly, we do a good bit of policy and advocacy work already, and we think of that in two ways:  Policy closest to the work, and that is the idea of what are the--what are the barriers to actually creating these schools, locally?  In the case of early college, high schools, for example, it's an idea that is way out ahead of any kind of research and proof points, and I know that it will come as a shock to you all to think that policymakers are actually not learning organizations and that they occasionally make decisions and appropriations based on a vivid anecdote, or a heck of an idea, or an interesting newspaper article rather than research-based interventions.  And early college is one of those ideas that's done that, and it has raised a host of policy issues that we have worked with in our area that are doing that to be able to get those barriers out of the way and then take that up to a state level to say.  And we just found out that it passed unanimously through the Texas Senate and is on it's way to the House, actually, to institutionalize those waivers so they won't have to be waivers any more on just the strength of a good idea and an interesting proof point. From that, then we also are working at policy on the state and national level.  I think that we--we're not fully prepared for the--somebody used the term "celebrity," but the value of Bill's voice, personally, on this issue.  We were stunned at the reach of that speech, both how it has sort of helped to frame the debate about high schools, but also how it has sort of reverberated and given new momentum.  We will be funding and working with states now on--on helping to build on that momentum and engage in state level and national policy agenda, then implement it at the state level and have put out an RFC on the street that's due back in June 1st to really try to capitalize to take. So that we're working from the ground up from proof points and policy that helps--that helps kick barriers out of the way, but then also working with--with national organizations, the federal government, and then state government to be able to advance a policy issue--a policy agenda that will help both leverage this momentum and bring--and bring the kind of change that we--that we're aiming at. And then, finally, through our own communications and advocacy, frankly, to bring more clarity and transparency to our work, we've-- we've--I guess there have been sort of a couple of misperceptions about the nature of our work and a couple of critiques that have turned up sort of publicly lately.  And one is that--that high school is too late to intervene.  We want to make it very, very clear that the reason that we are doing high schools is because no one was doing high schools, not because we think that everybody should stop doing everything else and start doing high schools. When Bill and Melinda started this philanthropy five or six years ago, they gathered a bunch of people and sent them out across the world looking for the world's greatest inequities.  We work on the American high schools for the same reason that we work on infectious diseases in developing worlds, because it was a place where other people weren't putting their funding.  So as we look at the new foundations that are coming on line, we hope that they got through a very similar process that said, where are people not funding?  Where is activity not happening?  And go there.  By no means do we ever say that because we do high schools everyone else must. Now--and then, finally, the idea that Gates says small schools are a silver bullet.  As an advocacy professional, I would love nothing more than to find a silver bullet.  I would love if it was one thing, and it was really simple that we found a killer ap [ph], and it was going to be--and it was easy to message it, and in a six-second sound bite we can wrap it up, and that would be the thing.  We have never said that.  We know it isn't so.  I think Reed Lyons, Rick, in his, they said high schools are inherently complex.  There isn't one single variable.  That's what makes this work so hard and so difficult to talk about, is that, ultimately, you have to be able to work on a number of different fonts, and it's just about impossible to isolate a single variable that's going to automatically improve all schools for everyone. So it's an inherently complex strategy.  We talk about the new 3 R's, rigor, relevance, and relationships, and three basic kinds of schools, academic, applied, and student-centered.  And we believe that--that all kinds should have access to a range of those options which then leads you into discussions and complexities around choice and around finance, and around accountability, and around data systems, and, like I said, given the complexity of both that program and advocacy challenge, I would love nothing more than to say that we had a silver bullet, but, alas, it isn't so. So that's--that's sort of the view from the Gates Foundation.  I really would like come again to Checker's point of view, which is that we are relentlessly focused on our goals ensuring that all kids graduate ready for college work and citizenship and are, frankly, through boldly persistent experimentation, aggressive evaluation that is quite public there on our website, trying to figure out the ways in which we can achieve that. MR.       :  Thanks, Stefanie.  Dan? MR. FALLON:  I want to thank you very much for being here.  We have to be here and you don't have to be.  And I think that it's testimony to how rich, and exciting, and interesting this conference has been. I've organized my--my comments around two questions that Rick asked at the beginning of the day:  What does it mean?  And where should we go? On what does it mean, I'll give you the punch line first, but then I'm going to leave it, so we can come back to it later.  The punch line is that we've reached a critical turning point in our society of which this conference is one manifestation.  So I'm going to expand on that a little bit. But first, I wanted to talk a little bit about my philanthropist.  I happen to be very fortunate to be working in a philanthropy where the founding philanthropist has been dead for a good long while, but-- [Laughter.] MS.       :  They're okay when they're living. MR. FAllON:  But Andrew Carnegie had an extraordinary impact on philanthropy, and one could make the argument that--that there was a turning point in the late 19th century where the idea of philanthropy really took hold, and Andrew Carnegie was certainly part of that if not leading a force    there.  His dates are 1836 to 1919.  He arrived in this country as a boy of eight years old around 1844, and he was dirt poor.  His father literally died of poverty; his mother took up cobbling to feed the family, and he never went to school in this country because he didn't have the money to do so.  He started working right away at age nine or ten.  By the time he was 25 he had already earned a huge amount of money.  And, oh, he had met all his financial goals.  He knew that he could retire and have income for the rest of his life. And he was worried at that time by the conflict of whether it was possible to be both rich and good.  And this was something that really--really bedeviled him for a very long time, and it's a conflict which he resolved in 1893 when he wrote this extraordinary essay, "The Gospel of Wealth," which if you read it, it's extremely charming, you know, read through 20th century eyes. He argues that--that in the capitalist system persons of administrative--natural administrative talent will simply acquire large amounts of money.  That's just the way the thing works.  And he said the single biggest mistake that a person of natural administrative ability could make would be to believe that the money belonged to him.  He says the money actually belongs to the people and is held in trust by the capitalist and must be returned to the people in means that the market cannot provide.  The capitalist earns the money through the cold--obeying the cold and cruel laws of the marketplace.  So you never pay a worker more than the worker is worth; you never sell a product for less than it cost you to make it, but once you have the money it's your obligation to return it to the people who produced that wealth in the first place.  And he ended this essay with the ringing phrase:  "He who dies rich dies disgraced."  And, of course, at that time he was the wealthiest person in the world and everyone knew that. He then set about to give away his wealth, which he did systematically, and by 1919 when he died the cupboard was bare; he had, in fact, met his goal. In that period of time, if you actually look at the etymology of the word "philanthropy" and take a look at what it meant at the time he was born, it was very different than what it meant much later.  Think about what the word "philanthropy" means in the novels of Jane Austin or Daniel Defoe.  It means love of one's compatriots, of other people.  If you look at the Oxford English dictionary from 1780, you'll see that there is a definition there of dolphins as the most philanthropic of fishes, for example. The London Philanthropic Society founded in 1800 had as its sole objective to reduce crime in the city.  The idea of philanthropy as being tied somehow to something different only emerged in the latter part of the 19th century.  In fact, the current Oxford English dictionary says that the first mention of it in that context occurred around 1878 with the notice that a grand philanthropist had given a large sum of his money to some cause. Andrew Carnegie drew a very bright line between charity and philanthropy.  He despised charity.  There was a quote earlier today about charity and what he felt about it.  He felt that charity was like giving a dollar to a homeless person.  He says if you give a dollar to a homeless person,  you're teaching them how to beg; instead, the purpose of philanthropy should be to build ladders upon which the aspiring classes can rise.  That's the purpose of philanthropy, so it's project-building, and virtually all of his philanthropy was built around the idea of contingencies, including the libraries.  Those libraries, he didn't just go into a community and build a library, he had a request for proposals.  He insisted on income streams, he insisted upon a whole variety of conditions from the community before he built the library. When he died, he left us with our mission which is the advancement and diffusion of knowledge and understanding to benefit the citizens of the United States.  It's a pretty broad challenge.  That goal, that's really all we have to do.  Then he went on to say no board of trustees--this was to be administered by a board of trustees--no board of trustees can bind a successive board.  So this is to be interpreted over time.  So from his perspective what he was doing was leaving his wealth behind and allowing it to be interpreted in perpetuity by wise people who would figure out how to do it.  That's the way the whole business was set up. Let me just take a little side tour here to Gunnar Myrdahl that's been mentioned a couple of times today.  This, to me, is a lesson in evaluation.  Had you evaluated Gunnar Myrdahl's work in 1943 when that book was published, you would probably have had a very negative evaluation.  It was not only worried about by foundation officers but it was widely panned in the press and elsewhere because he was washing dirty linen in public. Now, the history of the book is quite interesting as to why it was commissioned:  It was commissioned because the officers expected the president of Carnegie E-Corporation in the early 1930s had come to the conclusion that the single most important problem threatening the United States was the problem of race.  And that was at a time, remember, when there was Hitler and Mussolini, and a lot of other problems in the world, and they said, no, the one single most important problem in the United States is race, and it has to be addressed.  And they felt that they couldn't get an American to do it, and that's why they chose a Norwegian. So if you'd ask was that a successful grant, he would have answered no in 1943.  But by the time you got up to the 1950s you had a very different story.  That's the most heavily cited text in the decision of Brown v. Board of Education, for example. Now, we're here, I just have to mention one little thing about James Bryant.  Conant [ph] as well, because he's often sort of fingered as a person who argued for large high schools.  You've got to read the work.  I mean Conant--Conant was saying a comprehensive high school needs to be a high school that is capable of teaching science and math and English and history and foreign languages.  And you cannot do that if you have a graduating class that is less than 100 students. So you must have a minimum of 100 students in your graduating class in a high school in order to be able to do those things.  It is very carefully calculated around a curriculum question, and for him a large high school was a high school that had a graduating class of 100.  A large high school for him was about 500 students, five to six hundred students.  So to say that he was the father of somehow these huge megaschools that we have of thirty-five-or-four-thousand students, he would not have recognized that. Let me just move to the present.  I want you to note the big change from a few movers and shakers in the late 19th century with big ideas to a more corporate structure today, extensive consultation, peer review, internal vetting, final board approval.  I should also point out that Carnegie Corporation of New York, which was the largest foundation ever when it was first created, has long since been dwarfed by the big money, and so we are now on that list of 30--we are, literally, number 30--and our asset base is nowhere near the asset bases of the new money. Philanthropy is relatively new, and therefore the study of it as an enterprise is only now developing, and we've learned that today. Now to the punch line:  Out of this conference and the rich discussion here today are evidence that we, as a society, are at a major turning point.  Within the past generation we've undergone a fundamental shift in the underlying political economy, mass higher education which occurred in this country around 1970 when more than half the population reported as having more than 12 years of schooling, a decline in heavy industry and agriculture, the rise of technology--knowledge and information that is the linchpins of our economy--a large accumulation and concentration of wealth, and the emergence of significant corporate philanthropy, all of these are signs of this major shift in our underlying economy. When you look at other signs of a shift, think only of the Grassley committee looking at philanthropy.  This wouldn't have happened a generation or so ago, and then look at Sarbanes-Oxley.  These are all examples of everybody recognizing that the ground is shifting, and we have to--we have to think about things in a different way. This is good.  It's not a bad thing.  We can expect philanthropy to increase.  There are going to be lots of new players, and we want out of this to have a lively market.  This is going to be very difficult to coordinate.  I won't give you examples here, but I can tell you that foundations are really hard to get to work together because they have lots of reasons why they can't do that.   Let me get to--let me answer and stop here with a question of where should we go?  I've, what I've been trying to describe is where are we right now, where should we go?  And the answer--there are two parts to this--the first is, we should create a system that has high accountability.  And second, we should, to use Rick's phrase, expand the space for public conversation. First, under accountability, we need transparency.  Everybody's talked about that.  Accountability is partly transparency.  We can't--we can't, of course, insist that all grants be successful in the sense of their meeting their objectives.  Transparency means that we need to know what we have failed at and what we have succeeded at.  We have to have many more publications from foundations.  We need to have more regular reporting. Second, I think research--this is all under accountability in which there needs to be more research in general--we need to have more attention to outcomes, we need to learn more from what we do.  We should be benefitting from the lessons of science and from research because science is a self-correcting process.  The more you work on data and the more you evaluate what you're doing, the more you learn, the more you move and move forward. Board members who are responsible for--[indistinguishable]--should be more accessible.  They should bring the public concerns to the administration and the program staff.  On extending and expanding the space for public conversation, there should be voice in the public arena.  We want to stimulate development of a marketplace for ideas about philanthropy.  We need to have more op eds.  We do have a lively democracy, so there are lots of voices shouting out there, but that's exactly what we want.  We want to encourage various foundations to evaluate the agenda and outcomes of philanthropies.  We want to find opportunities to promote debate around strategies and targets. And I should mention here something that came up several times during the day, and it reinforces the point made earlier in the day by Bill Porter and by Jenkins and McAdams that investing shrewdly in policies and practices that are timely and near the tipping point is precisely what foundations ought to be looking for and trying to do if they want to make big changes.  And this meshes very nicely--that's what Bill Porter said --that meshes very nicely with Jenkins and McAdams who said you should use your funds to accelerate and realize implicit or emerging strategies that are already at work within the system.  And we heard that reinforced today from Arlene and from Alan. Thank you. MR.       :  Thank you, Dan.  Checker? MR. FINN:  Boy, talk about coming at the end of a long day, but usually AEI doesn't spring for anything more than a very modest line, so anybody that actually wants a stiff drink may have to go to the local corner saloon. Rick has done his usual exemplary job of bringing together a really interesting collection of people and papers, and he's also kept the trains running on time, and did anybody notice that Rick's been standing up all day at that podium?  I think he deserves a round of applause. [Applause.] The Fordham is an interesting hybrid.  We're both--we're a small foundation that's both a small giver and a sometimes seeker of other people's money including, not coincidentally, the Carnegie and Gates Foundation.  The giver, for example, we actually paid for that public agenda survey of ed school professors seven years ago that Tom Loveless replicated.  As a matter of fact, we thought it up and public agenda carried it out.  And it was one of the more interesting things we ever did. But our eyes exceed our stomach, as my grandmother used to say, and so in order to do all that we want to do, we have to seek other people to help us pay for things we think we could do but we don't have the money to do ourselves.  And we do this locally in Ohio and nationally in a variety of ways that some of you have from time to time observed. I've been sitting here today pretty nearly all day filling--filling many pages with many notes about what I might say, and I finally decided I can make seven fairly quick points that have struck me in the course of today. First, let's accept with--gratefully, the dissonance and diversity, and even, as I think Jim Kelly called it, chaos of the philanthropic world.  There is no more reason to expect uniformity and consensus in education philanthropy than in education politics, and nobody has yet found the silver bullet that we all yearn for, and until somebody finds the silver bullet we could keep trying a lot of different things in a lot of different ways, guided by a lot of different stars in a lot of different places and see what we can learn from it. And let's welcome the disagreements.  We, indeed, a lot of people have disagreement about the diagnosis of the ailment, much less agreeing about the--about the remedy.  This is not a bad thing. Secondly, let's not overstate the significance of ourselves, that is, of philanthropy in the K-12 education reform sector.  I--yes, there are 50,000 foundations, and, yes, there's as lot of activity in the philanthropic world outside of foundations; and, yes, there's a lot of money spent on education.  But as we've seen today, most of it goes to alma mater or to do-good projects in one's own community.  The amount of philanthropy that is interested in transformational work in K-12 education, I rather think that about 80 percent of the people who care deeply about that have been here today, and that if Rick had planned a little further in advance, he could have gotten 90 percent in the room. This is not a huge population.  And the amounts that's being spent and the number of people working at, yes, through the philanthropic sector are trivial compared to the behemoths that they are trying to move both in terms of money and in terms of numbers of people.  Bear in mind that public education in the United States employs approximately five million people in the K-12 system, approximately five million people. How many of us are there?  It's a good turnout, Rick, but I think you got a ways to go. And I think that we mustn't--mustn't overstate this.  I think we also need to recognize that most school systems are largely unaffected by transformational philanthropy at this point. Alan Bersin and Howard Fuller are exceptions.  If you go around to the nation's 15,000 school systems, or the nation's few hundred largest and most troubled systems and say, how many of them have been palpably affected by transformational education philanthropy, and I think you will find a lamentably small number applied.  So I don't want to overstate.  I think we should avoid overstating the significance of what we're up to here. Third point.  Historically, the role of philanthropy in education reform was to blaze trails for the government to follow, and there's still some of that going on.  But I've been really struck in the last five years or so by the extent to which government at every level is now making policy changes so rapidly on both the standards front and the choice front that to a very considerable extent, philanthropy is now playing catch-up to government.   It's piggybacking on governmental initiatives.  It's saying it's not working.  It's saying should we tweak this?  Should we redirect this?  It's saying could we--could we study it?  And it's no longer sort of breaking new trails; it's trying to catch up with what the politicians and the elected policymakers are doing for better and for worse, but they're doing it. Of course, a paradox.  On the one hand I think everybody is in agreement that a serious evaluation of philanthropic initiatives is a wonderful thing.  On the other hand, a senior foundation person from a respectable foundation called me up the other day and said, "Can you recommend any truly objective experts on school choice?"  The answer is no, actually.  Truly objective experts on school choice.  The answer is no, there aren't any.  And I think if I knew other fields better, I could probably make the same assertion about 10 other hot education reform topics today that, important as evaluation is, there is nobody left that's truly objective. And, therefore, we tend to escalate the evaluation paradigm and assume that as long as some huge enterprise like Mathematica or Rand does that it's bound to be objective.  Ha.  It doesn't work that way either. Fifth.  Partly because of what I just said, at the end of the day what everybody funds in education really has quite a lot to do with rolling the dice with their unproven theories of action, their unproven hopeful theories of action, with rival notions about how the world ought to be.  There is not very much certainty here, and there's not very much science here.  I wish there were  more.  And if you'd like to read more about the Fordham theory of action, check out our website where it's amply displayed in more ways than you probably want to read. Sixth.  If you're smart in this field, you learn from experience.  You learn from your own experience, and from out of other funders, and you amend your strategy as you go, and you try new things when something didn't work, or when it suggests that you do something different.  One of the things that I admire about both Carnegie and Gates, among others--and, yes, I'm only a little bit sucking up to them here--is that they've both demonstrated the capacity to learn from experience and change their strategies:  ad strategy, cancelled strategies, deemphasized, amends instead of stubborn persisting with the same old thing even after you know it isn't working the way you hoped it might work. And so one of the reasons it's still so complicated is that everything is always in flux.  People who are smart learn from experience, and then they do something different tomorrow than what they did yesterday.  So somebody that's busily surfing the web to try to understand what it was they did yesterday are going to the foundation center in the hopes that by studying their previous grants it'll be possible to figure out what they might want to fund in the future. And there's an awful lot of that in this world.  The foundation center happens to be located in my office building, and there is a parade of people in there to study the past grantmaking patterns of foundations.  The smart foundations aren't doing tomorrow what they were doing yesterday, and so studying what they were doing yesterday is not a reliable guide to what they might do tomorrow. And, finally, it's terrific to keep studying this.  And I actually think that this conference has broken some new ground, and I really hope with the asserted self-aware, even self-conscious study of education philanthropy will continue. There's a very different thing between studying it and regulating it, and there have been a couple of suggestions in the course of the day--one just as few minutes ago, I think by Dan Fallon, with his reference to the Grassley Senate Finance Committee, and an earlier suggestion from Mike Lieberman that maybe somebody could make foundations do something--I'm not sure people are aware of the extent to which this blooming, buzzing world of philanthropy is actually, currently under --under some threat by the machinations of staff members, both republicans and democrats, in both the Senate Finance Committee and the House Ways and Means Committee.  I was thinking maybe this thing is just too free and free-wheeling, and maybe the government ought to have more to say about what philanthropy does, and, incidentally, not just about donors, also about recipient organizations.   And if you haven't been focusing on goings on on the Hill with respect to the 501C-3 part of the tax code, you might want to look because there could be big changes ahead in the next few months, and my current impression is that most of the changes being contemplated would not do the world any good at all. Thank you very much. MR.       :  Thank you, Checker. [Applause.] We'll take questions for about 10 or 12 minutes.  Particularly, in addition to questions, it would be useful if you have particular thoughts or suggestions either for myself or for authors.  As we work through advice to volume [ph].  It would be very helpful. MR. RATNER:  Gary Ratner, Citizens for Effective Schools.  I would like to suggest that integrating four different points that have been made in a way of possible takeaways, starting with Lawrence Patrick, who observed the trying to fix urban public school systems.  What we're faced with is something that these were schools that were never intended to effectively educate black and Hispanic kids.  That was his starting point, and I think a very profound point that we often forget. Alan Bersin observed that educators have known for almost a generation what needs to be done, and then he talked about excellent principals, well-trained teachers, involved parents, and extended learning time. Third, he pointed out that the large problem in public education is the lack of capacity, which I take it by what he meant was the lack of capacity to support teachers and administrators and parents. And, finally, Lawrence Patrick said  foundations could be more effective if they spent more money on advocating for creating permanent change.  I think all points are very sound when you integrate them together.  What it tells me is, what I would suggest to you for a takeaway, is that the foundations might do well to focus on what we do know, which was just summarized here, and advocacy to figure out how to develop that capacity. MR.       :  Thank you.  Okay, actually, there's something that Dan and Stefanie--I want to ask you Checker and Fordham--I don't want to intimidate anybody because you guys aren't that big, so it's an issue for your office. But one of the things that I've found really--kind of just really interesting I guess is the right word--I'm writing a couple of pieces on philanthropy in the last eight or nine months, and in the course of particularly these 11 intrepid souls who have done all this research kind of generously, is it agreed or there seems to be a disconnect.  I know a fair number of fair number of folks in the foundations, and most of them are perfectly nice people.  I don't know hardly any instances of them ever actually blacklisting people or coming after people.  That just doesn't seem, except in very rare instances, to see that be part of the fabric. Yes, in the conversations in the other side, as folks are trying to write about this, as sources are talking about this, there's a phenomenal kind of cautiousness, a great skittishness both on the people who are going on the public record, the people who are interviewed by folks for "Education Week," the folks who were kind of going on the record talking about foundations in which there's a great fear that the foundations are keeping a blacklist. One, if you could kind of say anything about kind of, how do you guys see this?  How does it look from your perspective actually doing this work?  And, two, if this does seem to have some basis in truth, this kind of perception, what can we start to do?  How can we start to think about ways to convince people that you guys aren't going to send knee breakers after them if they say anything? MS.       :  There's not a blacklist.  Yeah, it's very tough, and I've thought about this, actually, as I've been sitting in the audience, how people really do tiptoe around and trying to figure out how it is that we create a space for this robust set of ideas and how, both in actuality and in perception, because I do think that this is something that this is the kind of thing in a meta way that we need--that we need to fund.  But if we're funding our criticizers, how legitimate does that look?  Right?  And so it may be painful and illegitimate-looking, and so then what do you get? So, no, I sympathize, and I'd like to think about it a little bit more.  But--but I think that--I mean as I read and shared with my colleagues these papers today, I think we--you know, we welcome a vigorous discussion of the ideas and evaluation of--we, obviously, fund our own evaluation--but how are we doing?  How are things working?  Are we thinking about this in the right way? And I mean in that sense we feel sort of respected by the--by the scholarly enterprise.  But I understand that it is--I mean that it is a very tricky thing for people to--people to serve [ph.] MR. FALLON:  Just to address a little bit the skittishness question.  Foundations, most of them are relatively small, and they usually are overworked.  If you're virtuous in a foundation, you want to keep low overhead so you can put as much money as you can into program.  The Carnegie Foundation, Carnegie Corporation of New York which has a very substantial reputation, has a total number of employees, about 60 persons, and the professional staff has maybe 25 people.  And we do a lot of heavy-duty reading and writing all of the time, so our eyelids are drooping often. So you get nervous about having to divert yourself from what you're doing to some political agenda that you're not sure of.  That's the first part. The second part is that foundations tend to be dominated by--by interests associated often with families, with a way of going about things.  It makes them relatively small, and as a result of that, they're protective of their--of their own enterprise, and they're worried about whether they --they feel actually vulnerable.  So they're worried about whether or not they can--they can defend themselves. Then also there are issues of confidentiality.  For example, we do regular evaluations, and I know other foundations do as well, of our own grantmaking.  And when I do a major project, for example, in our teacher education reform initiative, we sought to have an independent foundation due a national evaluation which we have no connection with, so that that would be looked at completely independently. Internally, when we did our most recent evaluation, we evaluated--we surveyed--we had a--a group of scholars, actually from a very distinguished university, do this work for us, and we surveyed a stratified random sample of our grantees over the past 12-year period, and we also surveyed a stratified random sample of people who had applied and had been turned down.  And we then compared those results.  We got permission from four other foundations to get similar data from them, and we compared our performance relative to the others.  We were quite gratified by that outcome, but the net result of that was that we had a lot of sensitive information that we had internal.  In fact, we found that generally people thought highly of the foundation whether or not they got a grant, which was good news for us, but there was also some other kinds of information that was not so straightforward. So it made it difficult to try to respond, formally, to an external request when you're not sure what the agenda is.  So that's the first thing. Then we asked, if you think you're on a blacklist, what can you do?  Well, my response to that is, you know, send in a proposal that has good evidence behind it, that is well documented. MR.       :  Actually, let's--that's the answer, so I'm less concerned about people who think they're on it, but more on the case of if--well, first, I guess, you know, is your sense that there is some skittishness out there?  Do you see that as an accurate characterization or not?  And if there is, what are the things we might start to ask ourselves, what are the kind of conversations we might start to have about ways two reduce skittishness, if you accept--if you think that's an accurate characterization? MR. FALLON:  I said in my comments that I think that we are in a new era, and I think that this conference today is part of that.  And I, personally--I happen to be a transparency freak myself--so, you know, I think more conversations.  The more conversation the better.  The more people write about what they do in foundations for the public, the better off we're all going to be.  And we at Carnegie do a lot of convening.  We do bring people in and have--you know, we shout a lot and carry on--and I think that that's something that foundations can generally do. MR.       :  Checker. MR. Finn:  Anybody who is or thinks they might become a supplicant is going to have second thoughts about insulting those whom they intend to become a supplicant to.  That's not just with respect to foundations.  I mean I know people who pull their punches with respect to Secretary Spellings or Governor Taft or whomever, because they think they might want to get a grant from the Department of Education or the State of Ohio.  And this is, I'm afraid for better or worse, normal human behavior even in a free society.  And the better--the better job the funders do of sort of dispelling those kinds of anxieties and the thing they do believe in freedom of speech the better the society will be. I'm--I'm less likely now in print to say, Stefanie Sanford's a stupid fool and more likely to say, gee, this Gates initiative over here might not be their very best idea. [Laughter.] MR.       :  You're always a diplomat, Checker. MR. FALLON:  Very diplomatic.  And ever since Fordham became a supplicant, I have a trustee, who many of you know, who's been the spring in my ear:  Be careful.  You're no longer going to have freedom of speech.  And I intend to do my best to prove her wrong. MR.       :  Morgan? MS. ENGLAND:  Hi. Debra England from the Koret Foundation.  I started early in the morning, I want to book-end.  The question that I asked earlier about Tom Loveless' work about was it legitimately a dichotomy between program officers who had educational backgrounds and those who didn't.  And I suggested that it might have been much more interesting divide to explore those who came from for profit backgrounds and those who came from nonprofit backgrounds. So I'm going to step out now and be sort of the fly in the ointment here coming from the for profit background prior to this, which is I want to ask a kind of pushing-deck-chairs-around-on-the-Titanic question.  I'm going to address this toward Checker, and I'd ask--but I'd like a comment from everyone if you have something to say about it. I'm a little bit surprised that as we've been down in the trenches all day long and had a lot of interesting discussions, but what I have not heard at all is the larger global context within which there is an economic imperative taking place which is directly--our educational system and its success or failure over it is directly tied to our national performance, as hundred of thousands and millions of well-educated Indians, well-educated Russians, well-educated Chinese are stepping out into the international arena.  And I highly recommend that everybody, if you haven't gotten a chance to read it, pick up Thomas Freidman's new book.  He addresses that. It is not probably things if you're reading the news and following things, it won't be things that you're not acquainted with, but he's bringing them together, and he does address the educational issue.  I'm floored that it hasn't been addressed today because there is no way that I can --I can understand how to strategically approach the funding decisions that we have to make in a foundation without putting ourselves not only in the microlocal context--and God bless Arlene Ackerman, she's going a great job in San Francisco --but also these large global issues. So my question is why haven't we heard of that today, and what do you have to say about it? MR.       :  Actually, I think that's been underlying everything that's been going on today is why are we all engaged in education reform in the first place, whether we're a foundation, or an action organization, or a public official?  I think everybody here has as their fundamental context the nation being at risk in a whole variety of ways, be it poor kids from the innercity or engineers being trained in India.  And I just sort of took for granted that people took for granted in this room today that we're serious about education reform for a good and sufficient reason, and now we're talking about how to do it. MR.       :  And, Peter, why don't you take us out, last question. MR. FRUMKIN:  I don't have a question. MR.       :  Even better. MR. FRUMKIN:  You've been very good, tough on people, but now we're at the end I'm going to allow myself a comment. MR.       :  Please do. MR. FRUMKIN:  Which is, as I hear your comments in regard to this question of how does the foundation field get meaningful information and feedback on its work so that it can prove its labors in this tough field of, take reform. One thing jumps to my mind, which I've been to a lot of meetings of foundations--council, grant makers in education, other ones. And they're okay, but I always leave feeling what are these people doing at these conferences in the sense that if I went to a conference and I were a grant maker, what I'd want to do is I'd want to subject my best thinking to tough, critical appraisal by other people.  I'd want to be--I wouldn't want to hand plaques.  I wouldn't want to show data about, you know, long-term performance.  I want to get to the quality of my thinking.  I want to get peer review of the quality of either my theory of change or something else. I just would put this as far as productive comment, which is one thing that the skill could do is to develop a much more serious approach to kind of peer review of not of the actual--of impact.  That's very tough.  I'm just talking about the front end.  How good are the fundamental ideas?  What do my colleagues think of the set of causal claims I'm making about what I'm doing in my work.  I just think there's a great opportunity, and perhaps for you, Rick, to maybe in the future trying to work on this question of how can we construct some meaningful peer and learning networks among grant makers, because there is this conflict that you've alluded to. But I think there's a huge opportunity in the foundation field to do this work.  It doesn't seem like it gets done at the conferences, and at the various typical meetings of donors, and I would just encourage foundations if you want to get more information, get out there; ask for tough appraisals of your bridge span report, of your theory of change on your web site checker.  Get some independent ideas.  Bring people together and listen. MR.          :    Any last thoughts? MR.          :    I agree completely.  I mean it is a little bit difficult for the dynamics that everybody has referenced today and that Checker talked about in terms of people not being open, because you've got the money, and they want the money.  But at the same time, there are ways to get around that.  I mean when I was designing the teacher education reform initiative, for example, I deliberately went out and created a panel of five persons who I knew had very diversity opinions, some of whom had disagreed with me over the years, and I had disagreed with them.  And I shared the successive drafts--I think we must have gone through seven or eight drafts--with them in that process, and actually the final thing, although it was written by me and I took responsibility for it, was greatly influenced by the opinions that I got from those persons, who were not in the foundation world, but I thought were keen intellects who had important and sensible things to say about teaching and learning. MR.          :    Okay.  With that, I'd like to before I thank the panelists, let me--first, I'd like to thank Dan Surrow [ph.] for making this--making all this research and making this gathering possible.  I'd like to thank my assistant, Maureen Goatley [ph.] for doing a phenomenal job of really-- [Applause.] MR.          :    You know, actually doing all the work involved in this thing.  I get to take credit like so much else in life.  The conference papers and various other materials that you're free to help yourselves to are out here in the anteroom, as well--actually, and I don't know, Chuck, if it's cheapness or if it's just that kind of careful kind of--the D.C. stewardship thing--but it is only wine, as Checker--of course, and you have to be Checker to the wine, because otherwise-- [Laughter.] MR.          :    Anyway, with that, I'd like to thank all of your for, you know, bearing this out.  I know it has been a really useful and interesting for me.  I hope you've all found it as helpful; and look forward to talking to you all in the next room.  I'd like to thank the panelists for-- [Applause.] MR.          :    Thanks so much. [END OF TAPED RECORDING.]</body></html>