<html><body>Leave No Principal Behind: What Is Being Taught in Principal Preparation Programs? May 25, 2005 Unedited transcript prepared from a tape recording. <TABLE cellSpacing=1 cellPadding=1 width="100%" border=0> <TBODY> <TR> <TD>9:45 a.m.</TD> <TD>Registration</TD> <TD> <DIV class=BodyText></DIV></TD></TR> <TR> <TD> <DIV class=BodyText></DIV></TD> <TD> <DIV class=BodyText></DIV></TD> <TD> <DIV class=BodyText></DIV></TD></TR> <TR> <TD> <DIV class=BodyText>10:00 </DIV></TD> <TD>Presentation:</TD> <TD> <DIV class=BodyText>Frederick M. Hess, AEI </DIV></TD></TR> <TR> <TD> <DIV class=BodyText></DIV></TD> <TD>Discussants:</TD> <TD>James Donnelly, 2004 National Principal of the Year</TD></TR> <TR> <TD> <DIV class=BodyText></DIV></TD> <TD> <DIV class=BodyText></DIV></TD> <TD>Sharon Robinson, AACTE</TD></TR> <TR> <TD> <DIV class=BodyText></DIV></TD> <TD> <DIV class=BodyText></DIV></TD> <TD>Jon Schnur, New Leaders for New Schools</TD></TR> <TR> <TD> <DIV class=BodyText></DIV></TD> <TD> <DIV class=BodyText>Moderator: </DIV></TD> <TD>Paul Peterson, Harvard University</TD></TR> <TR> <TD> <DIV class=BodyText></DIV></TD> <TD> <DIV class=BodyText></DIV></TD> <TD> <DIV class=BodyText></DIV></TD></TR> <TR> <TD>11:30</TD> <TD> <DIV class=BodyText> <DIV class=BodyText>Adjournment </DIV></DIV></TD> <TD> <DIV class=BodyText></DIV></TD></TR></TBODY></TABLE> Proceedings: MR. PETERSON:  The topic of principals in our schools is one that has not risen to the top of the policy agenda, but that does not mean it is not an extremely important topic. I think there is one thing that most research agrees, and that is that you cannot have a good school unless you have a good principal.  The principal is the key to the school.  Of course, you have to have excellent teachers, but those teachers have to be recruited and motivated and sustained over time, and you have to have involved parents.  But only a principal can bring those parents together, and only a principal can provide that motivation to the teaching staff. And you need resources from the central office or from wherever else you're going to get them, and only the principal is in the place to identify the resources that are needed and find a way of bringing them to the school.  So the principal is key.  I don't think there's any disagreement across the political spectrum on this particular point. The interesting thing is that we have laws that govern the recruitment of principals to our schools and these laws take the form in many states of saying that in order to become a principal you must take a set of courses that will prepare you for that job in a particular institution that's accredited for that purpose. So that really raises the question of why do we have that rule?  Why is it that in order to become a principal, after all you, you just find the best person in the building to be the principal or in a building nearby or down the street or wherever and bring them into the position.  So there is an idea behind it that in order to be an effective principal you have to have control of a specific body of knowledge, an esoteric body of knowledge. So the question then is, are principals receiving the kind of training in these institutions that prepare them more than the random person who might get recruited to that job otherwise?  That's the issue that we will be addressing today. We're very fortunate to have a group of well-informed, thoughtful individuals with us to discuss this topic. We have as our discussants, I'll finish up with our speaker, James Donnelly, and the full biographies of each of these individuals is in the handout so I won't go into it in detail, but I do want to mention that he was named the National High School Principal of the Year for the year 2004.  Congratulations, Jim, on that award. [Applause.] MR. PETERSON:  We have also with us Sharon Robinson, the president and CEO of the American Association of Colleges for Teacher Education and the former President of the Educational Testing Service's Educational Policy Leadership Institute and a person who is certainly able to speak with authority on this question. Finally, we have Jon Schnur, the co-founder and CEO of New Leaders for New Schools who is bringing an alternative way of identifying leaders in our educational system and can bring us a distinctive perspective given his experiences with that program. Before introducing our main speaker this morning, let me just mention that this is a venture that has been undertaken by AEI and the Program On Education Policy and Governance that I direct at Harvard, and one of the pleasures of my life has been working over the years with Rick Hess, our speaker today.  The report that has been issued by the Program On Education Policy and Governance as well as by AEI is available out in the hallway.  If you didn't pick it up on the way in, make sure you do pick it up on the way out.  There are some other materials out there, including a short reader-friendly version of the report that's available in Education Next, the journal for which I serve as the main editor, and we have an editorial staff of four individuals in addition to myself one of which is Rick Hess.  So we're constantly working together.  Here's the way we look on our cover, but this is not the current issue.  The current issue is out there, not the current issue, but the issue in which this article appears is out there in the halls.  I recommend that periodical to your close observation. Having said all of that, let me just mention that Rick Hess was a student at Harvard University in the Department of Government, but also got his MA degree in the Harvard School of Education.  So what Rick brings in his training is an unusual combination of perspectives that come out of the field of education and the field of the study of politics and government, and it's a very small set.  There must be a few others that fit that categorization, but few so exemplify how well that training can prepare you for thinking anew and afresh on the ideas about how we need to move forward in education as Rick has, and it's a great pleasure to introduce him today. [Applause.] MR. HESS:  Thank you, Paul, for that generous introduction.  I have known Paul for a long time now, upwards of 15 years, and there's nothing like somebody who is always gracious and kind and always kind of more so than you deserve.  It always puts a nice start on a day. What I'm going to share with you today is two papers that I co-authored with my long-term research associate Andrew Kelly who many of you know who will be starting his doctorate degree in political science at UC Berkeley in the fall.  He is at this moment jetting around South Africa.  We had thought about putting his little picture up there or doing a facsimile thereof, but this is AEI and we don't do that kind of stuff.  I just wanted to make sure that Andrew got his fair share of the credit or blame for the findings here. Typically, I play the role Paul is playing so it's actually nice to get up here and talk substantively today.  What I'm going to talk about are two studies.  If you only received one or the other, you might want to actually get both in case you have two short legs on your coffee table, both of which were issued by the Program On Education Policy and Governance at Harvard University a couple of weeks ago.  One has been accepted for publication in the Teacher's College Record.  The other is currently under review at Education Administration Quarterly.  As Paul mentioned, a brief version came out in the spring issue or I guess the summer issue now of Education Next. What I'm going to take is about 20 minutes to just run you very briefly through the research questions, the research strategy on what we found.  I'm not going to expound a whole lot on the implication.  Our three discussants are probably a whole lot better suited to talk about what it means than I am, so I'm going to actually just focus on what we did and what we found, and then we can open it up for the conversation. What we're looking at here is not a whole raft of issues related to the principalship or school leadership.  We're looking at a limited but I think pretty important subset of questions which is the issue of the quality of principal preparation.  Most of you in this room are going to be familiar with Art Levine's study from earlier in the spring which generated a fair bit of fuss.  It was a pretty strong indictment of the caliber of preparation taking place in the nation's schools of ed administration.  If you remember, Art said the majority of programs range from inadequate to appalling even at some of the nation's strongest universities. One of the things worth noting that Art by design did not try to get into the content of instruction in these programs.  That wasn't what they sought to do.  They looked at cohorts, they looked at university productivity and research, they looked at surveys.  You can't do everything, and they didn't try to get into what's taking place in the classrooms. For instance, while they reported that nearly 90 percent of principals said that their ed schools had not prepared them for the realities of the job, nonetheless, it's not entirely clear what it was that was or wasn't preparing them. On a related note, many of you are familiar with the 2003 Public Agenda survey on this question.  Public Agenda reported that 96 percent of principals, of course, said that experience and advice than their colleagues was more useful than their graduate preparation.  I don't mean to throw that up as -- that's probably true of a lot of us.  A lot of us in any line of work hopefully are going to learn a lot in the course of day-to-day work and from colleagues, but it is a red flag, there may be some things that are not getting taught or are not being taught as effectively as they need to be. What we actually look at explicitly here then, as I mentioned, is the question of content.  Like I said, the Levine study simply did not try to get into this set of questions, so no study is going to do everything.  Levine raised important questions, but they're not all the questions. And Joe Murphy who recently returned to Vanderbilt as most of you all know in fact did a pretty extensive review of what we know about principal preparation research just last year and he concluded, of course, that we know almost nothing about the content of preparation. The only previous study that really does what we've sought to do here was a 1992 study by -- and Gainor.  They looked at 36 syllabi from the 37 programs in the University Council of Education Administration.  A similar study, a 1987 survey by Norton Levon (ph), they didn't look at syllabi, but surveyed UCEA doctoral programs and asked them what kind of stuff do you cover, and they reported that more than 60 percent of content addressed managing personnel, school administration and technical knowledge of law and school finance. What we do here is we actually conduct two studies and I'm going to speak about them in turn.  The first asks the simple question, what are principals actually being taught in their preparation programs?  The second asks, what are they actually being asked to read in the course of their preparation?  The two studies are descriptive.  They're not causal.  Whatever we find, we make no claims that we're explaining why principals are more or less effective or why some schools are more or less effective. Moreover, we actually certainly have normative priors.  We know what we think principals should learn more of, but we can't prove that we're right.  Whatever your priors are, whatever you think people should need to learn, you can read these studies in different ways.  The point of these studies is not to necessarily prove in any sense that one or another perspective of what principals need to learn is right.  It's to get it out there, what they are learning and what we are teaching them so that we can then have an honest conversation about whether what they're learning is what they need to learn. With that as a disclaimer, let me also say these studies are preliminary.  Like I said, nobody has attempted a similar study in 13 years, we know very little about these questions, so we took our best shot at it, our best shot, first shots though I think are inherently limited, make no bones about it. We begin with the presumption that there are new demands on the 21st century principal, particularly four kinds of demands that principals have not traditionally been asked to juggle.  One is the new emphasis leading to school improvement in an accountable framework.  Second is actively ensuring teacher quality, issues like compensation.  Principals have more leeway than their traditionally had.  Teacher selection, teacher termination and so on. Three, they're being asked to utilize data and research in new and unprecedented ways.  Fourth, they're being asked to operate in a public choice environment, one, or they're being asked to deal much more seriously with parental outreach and parental interaction.  And two, they are also being asked to take on responsibilities like budgeting and outreach and public relations that have not traditionally been part of the principal's job description. What we actually did here was we tried to a nationwide study of what is being learned in principal preparation programs.  How did we pool the sample we studied?  In 2004 the National Center for Ed Statistics, IPEDS, that's integrated postsecondary education data system database, listed about 500 ed admin programs across the country.  You can get different counts on this.  It depends on how you count it, but it's a rough proxy. From this universe we drew a stratified national sample of programs.  Dave Steiner (ph) as many of you recall did a similar study a couple of years ago when he looked at primarily elite teacher preparation programs.  One of the concerns was that those may not be representative of other kinds of programs.  So we tried to look at three different buckets of programs.  One, we looked at the top 20 ed admin programs as ranked by U.S. News and World Report because whether or not one agrees with that exact ranking, these tend to be the most influential and prestigious. Second, we looked at the 20 programs that granted the most MEDs in ed admin in 2004 as reported by IPEDS.  And third, we then did a random draw of another 20 programs out of IPEDS. By the time we actually double-checked on this, a couple programs didn't offer what they were supposed to out of 56 programs we tried to collect syllabi from.  We acquired course syllabi from programs and faculty.  Your typical institution, your typical program, has 5 to 10 required courses, either specifically required or a menu.  We tried to get courses explicitly off of that menu.  We didn't take any other syllabi.  Only about 5 to 10 percent, it's close to 5 than to 10, of the syllabi we were targeting were on the Internet or on the Web, so we had to get these by contacting the programs.  We contacted each program at least eight times.  Programs typically sent us to individual faculty.  We made sure between program and faculty, we contacted each at least eight times, before we gave up. At the end of the day we were able to get at least four core syllabi from 31 programs.  We wound up with 243 total syllabi.  Thirty-three syllabi were not codable because they were nothing more than bullet points, they had week structure or what have you.  We wound up with 210 codable syllabi with a little north of 2,400 course weeks, and about 1,850 assigned readings.  That is what we analyzed and the stuff I'm about to present. We conducted from this, like I said, two studies, one, what was in the content of the syllabi.  Second, what was in the most commonly assigned texts. We broke down the syllabi, it's in the paper.  We break it into seven categories based on existing literature.  What you'll notice here is that the biggest categories are what we called technical knowledge.  This is school law and school finance.  That's between a quarter and a third.  I'm not going to break it down by the three categories.  All that is reported in the papers.  The upshot is that there is not much systematic variation between one category and another. You'll notice managing classroom instruction.  This is education leadership.  It was about 10 percent.  A number of folks are going to think that's lower than it should be.  Managing for results, issues like using accountability, school change, focusing on achievement are about 15 percent.  Managing personnel 15 percent.  Norms and values 12 percent.  Dealing with external constituencies 8 percent, and then so on. So that's the overall breakdown.  All the stuff is walked through in painful detail in the reports that are available for you. On the next slide is what we see is when we look at particularly this question of managing for results which Andrew and I, with our particular and relatively narrow perspective, think is the most important stuff, we said of the 2,400 course weeks, what percentage of weeks were focused on results, student achievement, accountability, school change, school improvement, and mentioned accountability?  The answer here is 2 percent. When we say what percentage of these weeks dealt with issues like school improvement, school performance and dealt in some fashion with data or research, you see it's 5 percent.  That 7 percent is actually an upward bound on the time spent on managing for result using either accountability or data and research.  Why?  Because these two categories are not mutually exclusive.  So somewhere less than 7 percent of all instruction is focused on improving school performance using either accountability as a management tool or using data and research. On my next slide I'm going to talk briefly about managing personnel.  I think what we're going to see as a recurring pattern is it's not so much that ed admin programs are not incorporating a discussion of data in some fashion and it's not that they're not talking about improving teachers.  However, there are at least two schools of thought in the world of management on how do you improve the faculty.  One is you do it through what I call soft mechanisms, through building culture, through coaching, through mentoring.  These are obviously important and appropriate. There is also a school of thought that says you also have to marry that softer stuff with harder stuff, removing low performers, focusing on measurable performance, hiring only the best and so on.  What we're going to see throughout, I think, is an emphasis on the softer stuff, using data, for instance, but not on the harder stuff, hard-edged accountability systems or removing low performers. So what do I mean?  Here we see supportive evaluation is 18 percent.  Supported evaluation is going into the classroom and then talking to the person afterwards.  Six percent is using data, for instance or using other systematic instruments to assess teacher performance.  Only 3 percent of managing personnel weeks deal with compensation, another 3 percent termination, and only 10 percent deals with selection of personnel.  The majority don't deal with any of this stuff. What do they deal with?  They deal with a variety of issues.  They deal with stuff like professional development in general, conflict management, motivation, other generic kinds of human resource topics that are not inappropriate, but that we suggest probably should not make up a majority of the instructional time devoted to these issues. We looked at external leadership.  I want to again suggest that here is another case where it's hard to tell whether this is a good or bad breakdown.  I want to suggest, though, that it's a breakdown probably more appropriate to 1975 than 2005.  The majority of attention is on politics and policy and general community relations, the kinds of issues that principals are being asked to tackle more directly, parental relations or small-business skills, together constitute only about one-sixth of all the attention that's being devoted to working with external constituencies. On the next slide is what we see, as I mentioned previously, one of the common indictments of schools of education, of course, is that they are ideological, that too much of their instruction focuses on what we call here norms and values.  In fact, we see that only slightly over 10 percent of all the instruction in these programs focuses on norms and values.  So nearly 90 percent of attention is simply not focused on anything that can be identifiably recognized as dealing with ideological content. On the other hand, what we did was we went ahead and we said that that would seem to suggest that ideological critique is significantly overblown in the case of education administration.  Within that 12 percent, though, we said let's take a look at how the stuff breaks down.  This stuff is obviously normative in judgment, how do you code?  You can read through the paper and see whether you like the coding metric or not. What we tried to do is what we call a mirror image coding strategy where we simply said left-leaning is the opposite of right-leaning.  So left-leaning is advocating concepts like social justice and multiculturalism, right-leaning is critiquing ideas like social justice or multiculturalism.  Left-leaning focuses on race-based discrimination, right-leaning is questioning as victimization, questioning the emphasis on discrimination.  Left-leaning is critical of testing choice based reform, right-leaning is supporting, and so on. When you look at it, one of the critiques of Dave Steiner's study was that in only looking at elite programs, he found left-leaning bias, that there was concern that this would not be replicated if we looked at other types of schools.  This is only within 12 percent of instruction, remember, so let's not get carried away here.  What you see is both at elite, large typical institutions, you actually see a pretty consistent breakdown.  There is a left-leaning bias ranging from about 70 percent at elite institutions to 54 percent at typical.  Essentially 0 percent of weeks are identifiably right-leaning, and between about 30 percent and 50 percent of weeks are neutral, they're mixed.  So there does seem to be some reason to suspect that in that 12 percent of weeks there is some degree of bias. Here is a couple of the course week titles that are the kinds of things we're talking about, for instance, race and ethnicity, white privilege, critical theory, transforming with a social justice agenda, and so on.  Again, this is entirely appropriate if folks think that is what principals need to learn, but let's at least be clear about whether there's a rationality on the normative stuff. The bigger concern for me as indicated by that 12 percent issue is not so much that there is an ideological bias, but the ideological narrowness of instruction.  What you find and is reported in several tables is that most of the prominent authors who were assigned tend to be authors who were specialists in the relatively narrow field of education administration.  There is relatively little research pulled in from the broader field of education governance and productivity or from management thinking more generally. Here's an illustration.  In 2003, Bloomsbury Publishing and -- Media did a survey of management professionals, management professors and management students.  They said who are the most influential living management thinkers?  They generated a list of the top 50, recognizable names to those of us in education.  People who we talk about and we see in Ed Week, Jim Collins, Clay Christanson, Warren Bennis, Peter Drucker (phonetic spellings).  Obviously, these aren't everybody principals should read, but these are probably a good component of the healthy kind of background in management thought. What do we see of the 50?  Just nine were assigned a total of 29 times out of 1,851 readings.  In other words, these guys showed up only 1.6 percent of the time.  This is them.  This is everything else.  Again, nobody is suggesting these should be the majority of what folks are reading, but from our perspective, this seems to be an underrepresentation. What I want to talk about real briefly now is the second study, the text study.  What we did was we analyzed 11 of the most frequently assigned texts.  The most commonly assigned text, interestingly, was only assigned eight times across 210 courses.  Seventeen texts were assigned at least four times.  We tossed out three law textbooks on -- inequalities because they don't really deal with ed admin.  Of the other 13, we studied the seven that showed up five or more times and four of the six others.  We divided the text into three categories.  I'm not going to talk about that.  You'll find that in the papers. It's useful to note that our coding did not rely upon indexes or tables of contents.  We read every page of every text and coded appropriately. So what did we find?  What we did is we did two things.  One, we coded based on simple word count.  How often do different concepts show up.  Then second, what was the context in which those ideas were discussed. What we see first off is overall just if we look at the aggregate word count, we see that some of the most common critiques, particularly the ideological critiques, are, again, unfounded.  Achievement does show up a lot.  Achievement is the most common term we found, showing up 45 times per 100 pages. Data shows up relatively frequently, 16 times per 100 pages.  But what you'll notice here with the green bars are what I tend to call kind of soft.  They embrace the notions of achievement and data, culture, values, resources, but they do so in relatively soft terms.  The red, as is appropriate, is the mean stuff, talking about compensation, efficiency, accountability rather than data, and termination.  You see these are much less prevalent. Let me just give you a couple of examples.  First, when we talk about accountability, accountability only shows up five times per 100 pages, 20 percent of the mentions are negative, 60 percent are neutral, 20 percent are positive.  Prescriptions on using accountability only show up a fraction of one time per 100 pages.  Out of the 3,800 pages or so we coded, I think there may have been four of five mentions of how one might use accountability to improve school performance. On the other hand, moving to the next slide, we see that data in these programs -- much more comfortable with.  Very little negative reference to data.  In fact, the majority of the discussion is positive, and prescriptions on use are about five or six times per 100 pages.  So again this notion that data is good, accountability we're less sure about and we're not going to offer a whole lot of guidance on how to use it. Let me offer a third discussion which is just, again, personnel management.  Again, while we see an extensive amount of discussion on building culture on soft cultural issues, teacher termination and dismissal combined, for instance, show up just three times per 100 pages.  Nearly all of that is due to the inclusion of single HR texts by Arbouri (ph).  In fact, none one of these 11 texts including Arbouri ever made one mention where it suggested that eliminating weak or inadequate performers could ever be healthy or positive for schools as a whole.  Again, that's not necessarily a bad point of view, it's certainly an appropriate point of view, but we're suggesting it shouldn't be the only point of view to which principals are being exposed. Let's look at one more slide, again, on ideology.  Again, I want to suggest that the simplistic ideological critiques tend to be off the mark.  The notion that these programs are all about multiculturalism or diversity is simply wrong.  These texts rarely mention multiculturalism, about once per 100 pages.  Diversity is only mentioned about three times per 100 pages.  And reasonable folks can certainly suggest that's actually too little, perhaps.  That wouldn't be me, but I can understand that. The bigger critique, for instance, my critique is going to be that the school culture focus is broad and, again, school culture as Jon is going to argue is an essential part of school leadership.  No argument there.  My concern is not that we are teaching school culture, but that we're teaching school culture but not teaching the fist that needs to be under that softer kind of more culturally oriented glove. Let me just do two slides by way of finishing up.  One, three conclusions I'd like to leave you with.  One, principals I'm suggesting are not being quick to use accountability or take responsibility for teacher quality.  Two, I'm suggesting there is little evidence of ideological bias, although within the narrow span of weeks that deal with these issues, I think there is a real concern that principals are not being given a balanced take on some questions particularly those relating to schools of pedagogical thought or to structural forms like accountability and choice. Third, and perhaps most significantly, the kinds of structural changes that the Teachers College report put in the table make a lot of sense, I'm completely comfortable with, but I think we also need to pay a lot more attention to what principals know and what they're being taught. This brings us to our last slide, three suggestions for future research.  First, I think we need to think a lot more seriously and systematically about what skills and content we want principals to have.  We're obviously making some traction on that, but I don't think nearly the pace we need to.  Second, we need to ask whether the reforms that are being talked about are actually going to alter the content of preparation or whether they're going to reshuffle the boxes but leave the stuff inside in tact.  I think it's too early to tell. Finally, a really interesting question is going to be as we hail new programs whether it's leadership academies in cities like San Diego and New York or whether it's New Leaders for New Schools, there's a real question of how does their content compare.  What are they doing differently, if anything, and let's make sure that we're not holding traditional programs up to one degree of scrutiny and then not holding up other alternatives to the same degree of scrutiny. With that, I'm going to turn you guys over to folks who are better situated than I to talk about what any of this means. MR. PETERSON:  Thank you, Rick.  I'm not going to go through the introductions of everybody again.  I just wanted to remind our panelists that in order to save time for our audience, we should try to keep each presentation to about 10 minutes, and we have some little flash cards that will help you.  I can help, too.  I promise to do that.  Jim, why don't we begin with you? MR. DONNELLY:  Why me? MR. PETERSON:  Your name is first on the list and so I do whatever I'm told. MR. DONNELLY:  Initially when I was first asked to review this, my recollection of my training as a principal kind of corresponds to the research that came out, so, therefore, when I was asked to review education research, I said, this might be like watching paint dry. [Laughter.] MR. DONNELLY:  But actually when I got into the study and began to look at the research and what came out of this, I began to say yes quite often.  From an experiential base what I have found and my reaction to this is that this is a significant piece of research, this is a significant piece of information that I think that we should begin to take a look at. From my personal perspective, I would say the reason why I believe that I was given the honor of representing the principals nationally a year or so again, I inherited a district and I was 33 years old and the youngest person on staff.  In New York State we have regent's exams.  Only 27 percent of the students were actually challenged to take the exams.  It was a high -- district, and I had the audacity to come in and actually begin to evaluate teachers that had not been evaluated in some cases in 20 years. I think it was my naivete as well as my inexperience.  I walked into classrooms of, for example, the mayor of the town who was also a teacher there and told him he was boring.  There is no one from the press here, right?  And there were other key people who were embedded in the community that also had ties to the Board of Education. I would say that my mentor guided me through the whole process of actually asking teachers to raise expectations for kids.  There was an issue of courage that had to come into play.  I'm not sure you can teach courage.  There is the issue of the influence of teacher's unions, the prominence of teacher's unions not only within the school but also on school boards as well.  My experience has been that near 50 percent of school boards have a relationship to the union, whether they have spouses who are teachers, whether they are former teachers, those play a role in terms of how evaluation and termination of faculty is viewed in schools. What this research really led me down to is to begin to think about is that why are people afraid to teach about termination?  Why are people afraid to confront this issue?  The reason why is that it is a difficult proposition.  It is hard to do legally as well as just in terms of time and energy that has to be devoted to it. At the same time, over the last 10 years when I hire people I've taken on, and it's a standard practice that I've utilized, I tell every new teacher that I hire that it's my job to support you 100 percent.  At the same time, it is my job to fire my mistakes.  That statement alone has gotten me more accolades and more trouble than I could go on for hours here describing. Talking to my peers along the same venue, I would say that there is a part of the system, and I'm not sure that the research is not a reflection of schools of education or whether this research is a reflection of the overall system that we're in.  Also, too, is that do graduate schools that teach preparation of principals have a relationship with teacher education programs, and would it be a conflict of interest for them to in fact teach the administrators how to effectively eliminate weak product that they might be producing, and that's another issue that's out there that needs to be investigated. One thing that I hear about is how teachers coming out of teacher training programs are not trained effectively and that we spend an inordinate amount of time helping and mentoring new teachers.  This should be done as a part of our job, and I'm not bemoaning that fact here, but the fact is, is there an interrelationship there that this report unearths? A couple of issues that were ignored in this was student discipline, for example.  I don't mean student punishment, but I mean just the practice of how do you create a disciplined environment not so much from a punishment perspective but from an environment of good discipline that encourages good practice for student learning and achievement.  There is a lot of interest in that area in the field, but yet I was surprised to see that there was almost no attention paid to it in the classroom. I'll be honest with you, where I was trained in principal leadership, I did have a single principal on the staff of the university that trained me.  As a matter of fact, there was not a single individual that had any experience in administration at all.  So taking a look at that in retrospect is an interesting point. Yesterday on the news there were two episodes that were highlighted in Virginia of sexual harassment of some students in schools and there was an arrest made.  The growing percentage of our time that is devoted especially in at-risk districts, high-need rural, high-need urban districts, where accountability measures are most effective or should be most effective, our time is being spent more and more on at-risk behaviors that students are involved in. In the last week I spent literally 80 percent of my time in dealing with police, dealing with students and parents, as opposed to maybe 10 percent of my time evaluating and observing teachers.  If you talk to principals that are in high-need environments, that is a growing tide.  This is another area that is neglected in the research and in schools. Let's face it, if we're going to raise expectations and raise aspirations for children in high-need environments, are our backgrounds sufficient to be able to calculate that? I come from an upper-middle-class background.  I would dare to say that 90 percent if not all of us here in this room come from an affluent to a middle-class background, but yet, but NCLB does, to steal a quote, attacking the bigotry of underachievement that has occurred in many schools, are we really equipped to begin to take a look at issues of generational poverty, for one example?  One text that I encourage all my teachers to read is Ruby Payne's piece on the impact of generational poverty.  I didn't see that piece of literature at all in any of the listings. Not to ramble on here, I think this is a great starting point.  This research got me thinking and really pushed me to rethink what has happened in my career, for good, bad and indifferent, but the one point I'll make is that the issues highlighted in this piece, the issues of accountability, teacher quality issues, data and PR, I think those were always used by effective principals.  I don't think this is new.  What I think is that the environment that we are in today politically is forcing us to do the right thing that we should have been doing already, and the question is, why wasn't that occurring and what are the mechanisms within the system that are preventing that. Frankly, my experience is that many superintendents have little or no experience with these issues, and yet they are the leaders of the principals, supposedly.  At the same time, they are the arbiters with boards of education.  Many of the boards have no or little experience with this.  So I think that that dynamic is going to be a good one to watch over the next number of years, so we'll go on from here.  Thank you. MR. PETERSON:  Sharon Robinson? MS. ROBINSON:  Good morning.  I am pleased to have a chance to join this panel and comment on these reports. I felt a bit hesitant about this assignment because, frankly, I didn't expect to be brought into the content of this issue in such an intriguing way.  So, Rick, let me say that your work in this area I find really useful because it helps us begin the conversation, I think as Jim has pointed out. I struggled with what would be the theme of my comments today, and the best I could do was is not missionary work nor is it drill practice.  I want to return to an effort to understand the nature of the job of leadership in schools and school districts today.  It is to help the whole enterprise focus on productivity, meaning students will achieve and they will do so through practices in the classrooms, in the schools and in the schools as part of the community that all focus on achievement and create an affirming, nurturing, demanding environment for students. While I am thinking of the leadership function of principals and superintendents, it is in the context of, first, an enterprise of education and also in the context of a community institution. That is the bias that I bring to the assignment, and as such, I think that it is useful to begin the query about how to develop the requisite knowledge and skills in the next generation of practitioners of the leadership function of schools based on the reality of the job.  You must first consult that reality and understand it, and you should also consult what is it that we're now.  I think, Rick, your work helps to inform the reality of what happens in the programs that prepare principals and superintendents, and your perspective, Jim, is invaluable. The course content approach to this was organized around the course week as a methodology that I will not claim to fully understand, but I did note that this measure of the content is a significant part of the program, but not all of it.  One would wonder how this content contributes to other activities of instruction and what is the process of understanding mediation, bringing in the reality, if you will, of practice which the content should inform.  I would want to know the interaction of the content and other instructional methodologies to lead to what objective, which perhaps we aren't real clear about right now in terms of what we're trying to help principals and superintendents know and be able to do.  We have a big chunk of the time spent in preparation that is not fully explained or brought into this study which I think offers an interesting opportunity for future work. The other thing that concerns me about this work is the need to get ideology out of it because this ain't missionary work.  I would ask the office to consider stepping it up a bit and getting to a level of conceptual understanding of the context, the nature of who is coming to school and what people need to know in order to serve them well when they look at issues like diversity, multiculturalism, critical race theory and so on, because this is not about politics.  This is about practice. Having said that, I want to agree that there does need to be more focus on accountability because we need principals and superintendents to understand that their job is to product learning in the students who come to school, not those they wish they had, but the ones who show up. If you would join me in focusing on accountability and leaving ideology at the door, because, frankly, I don't care what they believe, I care what they're able to do on behalf of serving learners.  In order to serve these learners, they're going to need to know a lot about what creates them as learners, and that has to do with the culture, languages and other aspects of living in this world that impact on the learner.  I suggest to you that this is a critical aspect of knowing how to provide leadership in the reality of today's schools. Finally, I'd ask that we think about this whole effort as a matter of work force development and consider very seriously all those issues of management as a work force development puzzle that we must put together with high integrity and, in fact, high productivity so that we can get the job done.  If we were to do that, I think we would quickly join the movement to create a much more high-integrity clinical experience for all of those who will serve in schools so that the content and the practice come together to create hard-edged, well-honed tools of practice that will in fact support student achievement. This might seem like yet another banner that we are carrying to create another aspect of reform in education, but I suggest to you that nothing is more urgently needed that the confidence that what we are producing in teacher education programs, principal programs, other programs that train school workers, we need to know that they're able to produce when they meet the reality of their work place.  That's the only way we can serve the students.  This is not about politics.  This is about productivity and I welcome your contribution to this conversation. MR. PETERSON:  Thank you, Sharon.  Our last presentation is by Jon Schnur. MR. SCHNUR:  Thank you.  Thanks for including me on this panel, and thanks for everybody coming today.  I know time is the scarcest resource for just about everybody here, and there is a lot of expertise and perspective both on this panel and in this room related to leadership and the broader context of education reform.  It's an honor to be included in this and I look forward to questions, answers and discussion to draw on the perspectives and expertise in this room. I'd like to do three things.  One is talk about why this report, this study and this issue is so absolutely critical at this point in our nation's history and the history of changing education.  Number two, I'd like to make a few overarching points or conclusions about the studies themselves.  Three, I won't go into a lot of depth because we don't have time, but go into some specifics in skill areas that I think are absolutely right to be focused on here and I think some things that are missing as well. First, why is this important?  I think that despite our differences over specifics, it seems that this city and this country is united behind the proposition that education reform is one of the most important activities we can do as a country and that it must focus on ensuring that kids from every background, all children from every race, every socioeconomic status, from every part of the country must achieve at high levels academically in order to give them a shot at the American dream as well as to ensure the competitive economy we need and that have citizens who are able to succeed in democracy.  That's a notable achieve, I think, that this country is united behind that proposition. In that context, we have over the last decade or more and very importantly over the last few years through No Child Left Behind focused in this country on standards, tests and accountability.  It should be tweaked.  I feel like there are very important ways, the ways that measures and tests are used, need to be more sophisticated over time, but the focus is absolutely right.  We actually have high standards in place or are getting in place, testing and accountability to focus on the urgent need for closing the achievement gap particularly for kids who are achieving at low levels.  That focus is absolutely right, and I think we have not a consensus around that, but a strong coalition behind that both in Washington and across the country.  You all know the data, but it-- [End of tape 1A, begin tape 1B.] MR. SCHNUR:  [In progress] --from low-income families who can't read at grade level at the fourth grade, about two-thirds of kids of color who can't read at grade level at fourth grade.  The Gates Foundation has done a study showing that one-third of our students graduate from high school ready for college, that's overall, and it's probably about half that for kids of color and for kids from low-income families. There are lots of other data that you know, but the fact is that our educational system was a system designed in another age for another purpose when we were okay getting a few kids to high levels of achievement and most kids at low levels.  Our economy has changed, our world has changed, and we as an educational system including school systems, universities, training programs, educators, haven't caught up to the new age where every child needs to get a least a high-quality high school education and at least some postsecondary education and beyond. Given that focus and the united or at least coalition of people around that, it's not surprising to me that we are beginning to focus as a country on the key lever of the principalship.  I'll quote from the study and I think it's absolutely right, "In the ear of bottom-line results, skill and knowledge of principals matter more than ever.  School leadership is key to school improvement."  I think that's absolutely right, the research I think supports it, that while there is nothing that correlates more to faster gains in achievement at the classroom level than a quality teacher, the research is also clear that there is a great correlation between the quality of a principal and gains in achievement across the whole school level. For most people, that's common sense, of course a good school has a good principal, but the urgency around that has been heightened because of this focus on high standards and accountability and a new principalship is really being demanded in this country which is very similar to some of the few principals who despite the system have been successful, but it's very different from the way that we as systems in lots of ways have cultivated principals. Why is this true?  I don't want to spend much time on this, but why is the principal so key to success?  It's almost such an obvious point that you could skip over this, but if you look at the research in organizational change in changing any organization whether in education or not, you don't make dramatic change in organizations without good leadership and management.  When you have something running pretty well, it's actually not quite as important to have great leadership, but you don't get change in any organization without good leadership and management.  So in an era of change, it's not surprising we're focused on the importance of good leadership. Second of all, if you look at the research, which there is good research, actually, on the key success factors of schools that are actually succeeding "despite the odds" at educating kids from all backgrounds at high levels and the things that the research shows I think very arguably are levers that the principal is most capable of pulling.  They don't do it alone, but they're most capable of pulling.  The research is clear. High expectations for every child.  Use of data and accountability to drive instructional improvement.  Driving improvement in teacher quality and instruction.  I would argue school culture and others.  The principal really is capable of driving those levers more than anyone else. The state of the principalship today overall is one of a principalship is a relic from the past focused on managing the status quo and operations, maybe keeping parents happy sometimes.  It hasn't yet caught up to this vision of driving change.  Even the purpose articulated in the study and I think shared on this panel that the principalship needs to have the primary purpose of driving high academic achievement for all kids and being accountable for that, I thought it was of note that this study referred to a poll that showed principals across the country are split evenly on that fact.  Half of the principals believe that is the purpose and half don't, and I actually am surprised to see it's that high given what the principalship has been in the past. That's because schools systems have cultivated, hired for and trained for an administrator of the status quo principalship, not a change or leadership oriented principalship, and universities have trained in general with some exceptions for that kind of principalship as well.  The good news to me is that the numbers and the demographics are on our side in changing this. There are some phenomenal current and former principals who are beacons who can lead the way, but we have principals who largely were not selected or trained for this job, and it is a challenge but an opportunity that in many cities across the country, which is our focus at New Leaders for New Schools, high-need urban schools, 30 to 50 percent in many of our cities of principals are eligible to retire right now. We have this challenge and opportunity to recruit and train the next generation of principals who will lead our schools for a long time to come because principals in general have been teachers and are older than teachers and are ever approaching retirement age and we have this opportunity to get it right.  That's the context of why I think this is so important, and overall, how do we transform the principalship to an agent of change focused on high achievement for all kids and closing the achievement gap, I think, is one of the central questions of education. Very briefly, a few points on the study itself.  One, I do think that implicit in my comments so far is that the study rightly focuses on the need for the transformation of the principalship.  Secondly, I would argue that principal recruitment and training on the front end, given the numbers and the demographics, is a necessary but not sufficient means to transform the principalship.  Some people will point out there are other issues, and I would, too.  There are very important other issues, how principals are recruited, selected, paid, evaluated, removed, job design, what kind of leadership teams they have.  There are a lot of issues in the system, union contracts.  But none of those, as important as they are, can be changed if you don't have a new kind of principal who is ready to lead in that context.  So I'd absolutely agree with the focus on as a necessary but not sufficient mechanism for change, a change in principal training. Another point, the study acknowledges and highlights a critical gap in the study and in our knowledge.  At one point in the study, and Rick, you echoed it, the study was not about explicitly saying what principals should know, although you used prior norms about what that should be, it was really descriptive about what principals teach, and I think to that end it's a very, very interesting study about what principals are getting taught in principal preparation programs. I think that it's not just the study that is missing a really clear research-based statement on what principals should know and be trained for, I'd argue, and I'm in this field, as a country we actually don't really know based on really good research and evidence what are specifically the characteristics and skills that principals need to drive dramatic change in schools.  We have a lot of good hypotheses, and the best examples are from great principals who you can see exemplifying many of the characteristics, but we actually haven't done really serious research yet on what those are.  So New Leaders for New Schools is just one example, I'd call us an action tank as opposed to a think tank, we're creating new leaders and principals across the country who we think can drive gains in student achievement in urban school systems around the country.  But even more important than our success in the next few years, and we haven't demonstrated that yet although we have some promising examples of schools making great progress, even more important than our success is with a very careful research and evaluation effort, looking carefully and documenting what characteristics do principals come in with on the front end through selection processes, what skills do they gain over time, how do the schools that they lead to in terms of driving gains in student achievement and school success. It's been a few years, and I hope that we will be part of a national effort on this, when you look at the schools that are making the most progress under specific principals, what are the characteristics that those people came in with and what are the skills they acquired over time that seem to correlate with faster or slower gains in academic achievement and school success. I think at that point, not through our effort but along with others we'll have a better research-based idea of what the knowledge really should be in principal preparation programs. I know I'm about out of time and I have one minute left, so I would just make another macro point on this which is that the analysis of textbooks and principal training programs I am not surprised reflects a serious gap in what needs to be taught partly because textbooks reflect the state of knowledge, and the state of knowledge isn't particularly good, and partly because textbooks and university syllabi are a limited tool.  Ninety-six percent of principals say their best experience, their best preparation came from real experience, and that's what I think new leaders and a number of others are doing in experimenting with this notion of a year-long resident principalship where people are getting as much learning from a leadership position on the job with opportunities to learn some of the theory and apply it as through course work which I think is very important and one of many models that could be used to apply that.  But I think there are limits of textbooks both because of the content in the field right now and because of the methodology.  And I would close on this point, because we are facing a challenge at this point that we are preparing new principals in some ways for an old principalship, and even if we train all of our principals for some of the skills they need, in many cases while that's necessary, it's not sufficient because the principalship currently doesn't allow them to exercise some of those skills so we're going to have to make parallel improvements in transformation of the principalship while we transform principal training. I'm happy to comment more on some of the specific skills areas in the questions and answers. MR. PETERSON:  Thank you all of you for your interesting comments and observations, and, Rick, for your research which really highlights the importance of this topic. I think it's now time for the audience to raise their questions, and please keep your preceding comments to the question very brief and make sure they end with a question mark and we can move quickly through all the ideas that are in the room here. MR. HOLLY:  Bill Holly (ph) from the University of Maryland and the American Association of School Administrators. I want to make three points that I think are kind of challenges if not questions.  I agree with Sharon that this work is useful.  As you well know, the coding problem here is an important one for all of us, and all of us who have done this work know no matter how hard you try, you still get it wrong.  So making the data accessible is important. I was interested to know that it was left-leaning to teach about how students learn and the ways of the American Psychological Association and the National Research Council think are right, not right, but centered.  I suspect a little ideology found its way into that coding process and you might want to respond to that as a failed question. Secondly, I think an important here is that content and pedagogy as strategies for learning are not separable.  I think Sharon was trying to make that point, but certainly Jon operates from the principle that how we learn as important as what we learn and the two are not different.  So that if we learn about collaboration in one way or another, we get a get a very different way of learning.  So somehow or another the next step has to get into that process which is the point I think you were trying to make. The third is what are we going to do about evaluating the outcome of programs like yours?  As we critique appropriately the existing programs, and this is not limited to this field, we jump to new strategies without worrying a whole lot about whether those work because we're so fed up with the old process, with the process we've been using. MR. PETERSON:  Rick, do you want to respond? MR. HESS:  Yes.  I think that's right.  Bill is exactly right, coding any of this stuff is messy and we were incredibly hesitant to do any of it except that for a couple of years in trying to talk about this we just kept wading into a morass about any time you stipulated anything relating to what was going on in ed admin programs, people just said, what's your evidence for that?  So I figured let's try to get something, no matter how sloppy and imperfect it is. Yes, I think all the coding decisions as one goes through, there are reasonable questions one can have about all kinds of stuff how do we rule in and out and we try to be concrete. Particularly on the norms and values stuff, you're absolutely right, different people are going to code the stuff with different break points in different places.  We just tried to be very clear about how we coded and what we did and then folks can decide whether or not they thought that we gave it a reasonable effort. In the terms of availability of data, of course, the one issue is that because of the way we had to collect the stuff because we're not an institutional organization in which we collect this routinely, therefore, we actually had to promise confidentiality and anonymity in the programs. For instance, regarding the AACTE, some of the members would contribute these kinds of things on an ongoing basis, I believe.  I think it would be wonderful if other folks replicated this and if they analyzed it differently or using different lever points or came to different findings, that's great.  I would welcome the conversation. MR. SCHNUR:  I also would like to comment on the evaluation question, if I may. I have two points.  One is -- called it an alternative program, although it is an alternative approach, our purpose at New Leaders is we are creating a large national core of principals in partnership with the school systems, but it's an attempt to inform not replace the system of preparation of higher education, and one of the things are very much trying to do is create a real exemplar, successful or not, a knowledge base about what works and what's needed in the training of principal. So on the evaluation side, briefly, in our evaluation, and we would love for more programs to join with us in this kind of evaluation, we're looking very carefully over time at gains in student achievement and improvement in other school success measures including graduation rates, college readiness, student attendance and other factors in schools led by New Leaders principals across the country for at least 3 years.  We'll have comparison schools where other first, second and third year principals with similar demographics. We'll look at those changes over time, and then also have very good data on not only how our people perform in our selection process, we select about 6 percent of candidates and we measure them on 10 criteria.  We collect data and measure how they do on those 10 criteria as well as how they progress in certain skills areas over time.  So in the end we'll have data on what characteristics people had coming in, what skills they learned or didn't learn to proficiency over time, and progress of schools, and at the end, regardless of whether we have, which I hope and expect we have, that most of our schools are making dramatic gains in student achievement, will at least be able to say on the high performers, mid performers and low performers, here are the trends in terms of what characteristics they had on the front end, what skills they developed, and we'll have an evaluation of our program effectiveness which I hope will inform the field. That's the kind of evaluation we're doing, and I would encourage any policy maker or funder to support and encourage. MR. PETERSON:  Thank you, Jon. MR. FEEBER:  My name is Larry Feeber (ph) and I'm a living example of much of what you've talked about today.  I was a school principal for 26 of my 34 years in education.  I was our state principal and our national distinguished principal.  I tell you that because when I retired I applied at a local college to teach school leadership and they told me I wasn't welcome to do that because I didn't have a doctorate.  Yet there was nobody on the faculty of this particular college who had ever been a principal of a school. One of the concerns that I have that I'd like you do address is that during the time that I was a principal working voluminous hours, as all principals do given the enormity of the challenges that we face, our professional journals in our state were offering these doctoral programs, get a doctorate during the weekends and in the summer with these cohort programs at a cost of $50- or $60,000 to those of us who are hard-working principals doing the day-to-day work that we are expected to do in our schools. So because I was never able or willing to give up the time from the children that I was serving to go to these cohort programs and spend $50,000, I've ended my career with an enormous amount of experience and yet am not able to go and teach school leadership. The bottom line is, what do we do to reconcile these departments of education to look to people who have had successful careers, exemplary careers, to go in and really work with those who are aspiring to be principals, one, getting the paradigm shift on that level. Then the other factor is, cautioning those of us who are principals to look very closely at these cohort programs, one with a price, and also with the ideas and the way they advertise, get your doctorate in 2 years on a very temporary basis.  I've had trouble reconciling this, so perhaps you can help me understand how we can change some of this. MR. PETERSON:  Thank you, and thank you for introducing yourself.  I hope all the people before they pose their question introduce themselves.  Rick, do you want to respond, or Sharon? MS. ROBINSON:  I'm glad you're here because I think that is such an important question.  It exemplifies the really big gulf we have between the academy and the world of practice, that we need to bridge that gulf with a system of clinical activities in a system of engagement, if you will, so that within the schools of education across the work force that they develop, they're doing so with the perspective of the reality of the work and they can provide the candidates a setting in which they might practice under good supervision, a system that would give them good feedback and allow the candidate to develop skills rather than have them go into a situation to sink or swim or go into some kind of isolated situation of student teaching or internship that is just so removed from the academy that there is no interaction.  I think that's one important development that we need to get on with like right now because it's really important to the enterprise. Next, I believe in not voluntary, but mandatory accreditation of all programs that provide training to the education work force so that we can bring to the table a system of accountability, the requirement of content and practices in the development of this work force that we know to the best of our knowledge at the moment to be necessary. This will change over time.  Just yesterday the National Academy of Education released the report on the required curriculum for instructors, teachers, and that's really an important development because that consensus will now inform that happens in accreditation.  But in education, accreditation is voluntary and the policy system seems to be encouraging that.  So that right now until we have a means of accountability, if you will, that is required, we've got loopholes all over the place. MR. PETERSON:  I think, Rick, you should comment on Sharon's suggestion that we need more accreditation. MR. HESS:  On Sharon's first point I agree completely.  One of the issues is whatever kind of regulatory and kind of accreditation regime you're going to have has a set of questions like what does good practice look like.  I endorse fully this notion that what you want to have is a much more thoughtful and nuanced model of how do you get people equipped for what their job is going to mean. All three of these folks who have actually dealt with this stuff on a very hands-on level are exactly right, that we don't in this paper look at the clinical model and we don't look at these questions of the interaction between the content that's being learned and the context in which folks are operating.  Bill raised the questions about the coding.  We couldn't even begin to get our heads around how we were going to code this clinical stuff at this first stage, but I think it's certainly a question that future research should look at. On accreditation, that's a question that I'm going to comment on differently than Sharon primarily because of the kind of role that ISLLC, the Interstate School Leaders Licensure Consortium, is playing, for instance, the kinds of models of accreditation that are being held.  In some ways, a voluntary accreditation with teeth which is based on demonstrated effectiveness of the kind of stuff that Jon is talking about, Sharon and I might be more in line on. The kind of way that accreditation has traditionally been used in education, my concern is that is has too often been used to shut out promising innovations and programs and not give them a chance to prove themselves on their merits. I would disagree with Sharon, I think, although we could talk about the particular points of disagreement, but I would suggest that accreditation to my mind is probably not the best way to go at this juncture. MR. PETERSON:  Are there any other questions? MS. DAVIS:  Jacqueline Davis with New Leaders for New Schools. I'm curious to know in all the review you did of the content if any of those schools were teaching about belief and the philosophy around what we think kids can do and the difference in and understand of innate intelligence versus developmental intelligence and how much that then informs what our principals believe and then how they practice and if there was any review of that and what schools are thinking on that front. MR. HESS:  All of this stuff that gets to beliefs or sense of purpose or mission of schooling comes under the norms and values weeks. Like Bill pointed out, one can look at the discrete pieces we coded and debate whether or not a given component of it is left- or right-leaning.  We did what we thought was the most reasonable job given the kinds of limitations. This is actually what I would suggest, but it's not just about politics, though, and I think Sharon would probably agree with this, Sharon's point about how we're in an accountable culture, Jim pointed out that in some senses, the best principals have always lived in this culture, but now what we're saying is everybody whether it's comfortable or not, guess what, you're stuck in this culture. Part of that is this notion of whether or not principals embrace that notion, whether they personally take responsibility for the quality of their faculty and for the learning of students.  What public agenda surveys have suggested is we're not there yet.  A third of our principals the NCES reported in 1999 are former gym teachers, 34 percent.  That doesn't necessarily mean that you've been attuned to notions of pedagogy and curriculum mastery.  Principals have traditionally not been selected, hired or promoted based necessarily on their ability to drive student achievement or ensure faculty quality. So in some sense I'm trying not to vilify these programs because in a real sense what they've built is what's appropriate to the world they're in.  They were asked to process people, take teachers who went to summer institutes, get them their credentials so that they could then be hired, and this was the way the game was played. We've decided that we don't want to play the game that way anymore.  It's unfair to vilify current principals for the world they grew up in, and it's unfair to vilify these programs.  It's not unfair to say if you want to be deemed a competent professional going forward, the rules have changed.  You need different skills, programs need to teach people different skills, and we need to redesign the job. So for me, the beliefs and norms stuff is not so much political as it's about what kinds of values are we promoting to principals as we ask them to take this on, and so it strikes me that programs which are explicitly skeptical, for instance, of accountability or testing and not ever explicitly supportive of this, are likely to put principals out there who are not only being given a balanced view of the tradeoffs, but are being encouraged to hold certain views on these kinds of things. MR. DONNELLY:  Also I think your question is very pertinent in the fact that, for example -- Marzano Pickering & Pollard, Classroom Instruction that Works in the classroom, the foundation of that piece of literature begins with the discussion over for the last 30 years there has not been a universal belief that the teacher makes a difference in the classroom.  There has been a universal belief that how kids come to school is how they come to school and our efforts in the classroom really don't make a big difference.  I think that that is the crux of the debate that's going on even within the testing issue because if you believe we make a difference in the classroom with children, then you have to believe that that should be able to be measured and there has to be some accountability along with that. I think that if you go back to look at significant teacher training literature that's out in the field and you look at the debate on why is there a debate about that issue at all, and that's why I made the point earlier about our own socioeconomic backgrounds as a community of people and a community of learners is that the people that were being compelled to educate and being compelled to raise the standard on, many of us, do we believe that they can learn? I think that that's the challenge that many educators are placed in the position of as a principal, and I'd welcome comments.  I'd like to know how you retained all your hair and why it's not gray.  I need some mentoring on that issue. [Laughter.] MR. DONNELLY:  That gets down to the crux of the debate that I think principals are involved in, and it's a tough one.  I think that from my perspective, the effective principals and the people that I respect come down on that issue, that, yes, we can make a difference in the lives of all children. MS. CUNNINGHAM:  My name is Sarah Cunningham.  I'm from AALE's Charter School Accreditation Program.  I'm not going to comment on accreditation, but my first question is, how much is this new role for principals driven by charter school leadership?  Charter school leaders are required to deal with public relations, accountability, data, teacher quality.  They have power in hiring and firing, and they have a relationship to governance that's much more intimate. Often they have a very close relationship to the core curriculum in the schools which leads to my second question.  As part of the charter school movement, too, I think there are many training programs that are not university programs, the Principals Training Program, Building an Excellent Schools Program in Boston, so there are these programs, surveying the charter school leadership model. My other question is, what is the role for liberal arts in this discussion?  I heard some of you talking about Aristotle and Euripides when I walked in, and many of our school leaders who have done very well in their schools, top charter schools in the nation, often in the states such as BASIS Tucson, the Black River Public School in Michigan, KIPP Houston, these leaders are liberal arts educated individuals and they're educated in reflection and judgment and text that maybe don't have direct pragmatic value but have led them to reflect on goodness and what is the good life and things that students are going to reflect on in college when they get to college. I'm wondering what role this possibly could play and perhaps what role the charter movement has played in creating a new image for principals. MR. PETERSON:  Rick, do you have a response? MR. HESS:  Yes, unless Jon wants to respond more directly.  I think the charter movement has absolutely been a part of this.  Certainly, the folks who are most directly dealing with what we call the 21st century principal leadership skills are absolutely folks in charter schools or in public systems which have moved the furthest in devolving authority to site-based -- this would be Edmunton, for instance, in Canada, would be another more traditional exemplar of this kind of school level leadership.  I think that's absolutely part and parcel of this. This actually speaks directly to Jon's point, that it's a question in synchronicity, changing what principals are actually able to do as well as what we're training them to do.  It doesn't do any good to train them to identify, select good people and remove ineffective ones if they find it incredibly difficult to do so once they're on the job.  So you can understand why you wouldn't bother to teach them that in the first place.  I think charters are certainly a part and in some sense a very visible manifestation of these kinds of broader changes. As to the second half of that first question, programs like the KIPP Academies' training are certainly really interesting.  Sometimes we get so caught up in our enthusiasm for lighthouse programs that we give them a pass, so the exact same time of scrutiny that Andrew and I have turned here on traditional programs I think needs to be turned on KIPP.  We don't want to be using different scorecards to evaluate different kinds of programs.  That's just not helpful. MR. SCHNUR:  On the charter side, I do think that charter schools with the latitude that leaders have, the need for engaging with boards and with all the management issues, depend in some ways at least if not more heavily on a revolution of the identification and preparation of leaders to lead schools effectively. By this fall we'll have 230 new leaders across the country, and about 20 percent of our new leaders are going into charter schools and leading charter schools, and we do see some overlap and some differences in the needs and leadership is even more crucial.  The charter movement will not be successful in helping kids reach high standards and driving broader systemic change unless we have high quality and you won't have high quality there without good leadership, so I think that's critical. One point I'd make on that is charters are just one example of why we actually don't know what the scorecards should be yet for principals, and I would emphasize that point.  Earlier when I was finishing up before I wasn't talking about new leaders, but policy makers and funders should be focused on helping -- preparing and training principals to ensure they've got good evaluation in place to look at results of those because we don't know what the scorecard could be yet. I think one of the most critical tasks in the next 2 or 3 years is to actually get good data from across the country about what the effectiveness not based on Rick has got a good scorecard, I think it's great in many respects, I think it's missing an important focus on school culture, I think it's missing a focus on inculcating this belief that adults are accountable for delivering on kids high potential academically and that all kids from all backgrounds can learn and another agency has a different view.  We all have different ideas of what the scorecard could be, and my most important point is I think in the next few years we need good evaluation and research on impact and achievement so we can actually come up with a scorecard that would allow us to have a smarter system of grading and potentially accrediting programs. The last point I'd make on the role of liberal arts in this is I think it's a very interesting question, and I do think it gets to the notion that leadership of schools and skills are more practical than theoretical in many ways and this is why at New Leaders we had half of our people who have been great principals and may not have doctorates, and I hope more programs go that way, and that the theoretical knowledge in many cases is more from a liberal arts background. This was not deliberate, but we happened to look recently at the percentage of new leaders who had majored in education as an undergraduate and the survey so far has shown that about 90 percent of the new leaders that we've selected all with teaching experience who we picked partly for their instructional expertise did not major in education, they majored in various aspects of liberal arts and then later they got that teaching experience including in many cases master's degrees in education or more experience on the job.  So broader knowledge that people need that could be built on over time with more practical educational expertise informed by theory is an important direction for us to explore. MR. PETERSON:  We have time for one more question, and that's the pay I'm going to get for my services today, the right to ask the last question.  The question I want to ask of Rick is how do business schools educate people who are going to go into management, and how do schools of education that train principals, how does that program differ from that that goes on in business schools? I don't know that there's any accreditation that goes on, that is to say, any firm out there can hire anybody, so they're in a very competitive market.  You've got to prove yourself in order to sustain your place in the hierarchy of business school, so they must be very attentive to what's needed out there and they must need to change their curriculum whenever it's not meeting the needs of the marketplace. What are the clues that educators can obtain by taking a close look at what's going on in business schools? MR. HESS:  Let me give my crack at it and let's let Sharon offer her thoughts in response to this. Actually, it's fascinating.  Just this week a report, one of the Thinkers 50, I can't remember exactly who it was, co-authored a study blasting the quality of the nation's business schools as inattentive to practice, as divorced from the realities of organizational leadership, and is obsessed with fancy research that would be regarded in business scholarly journals rather than with hiring folks or staffing with folks who were actually going to be effective mentors for people going into business. Beyond that, I think it's actually interesting, business schools shortly after World War II were a joke.  They were the sorry step-cousins of economics departments.  If you couldn't do anything else and you had a rich uncle, you wound up being a business school professor.  This transformed in that first generation postwar when by the late 1970s or early 1980s some of the elite business schools had really come into their own as prominent institutions. There are a couple of factors.  One, there are a lot fewer business schools than there are ed admin programs.  There are only about 150, give or take, I forget the exact count, of business schools in the country, maybe 170, which is about one-third or one-quarter of the number of ed admin programs.  We're simply asking them to graduate fewer people, for one thing, which makes quality control a lot easier. Second, of course, is that both the incentives in the pipeline into business schools I would argue make it easier for these folks to be selective about the folks who they're going to admit.  Given that we pretty much future ed administrators to folks who have been teachers with at least 3 years of service and that we then want these programs to allow people to do this while they continue their teaching careers, ed admin programs often wind up feeling that their hands are tied in the kinds of ways that were alluded to.  Business schools for the most part don't struggle with these same kinds of constraints. So one of the really interesting questions for me always, and it's the same question that comes up when I think about law schools, it's entirely unclear how much of the kind of general perceived quality of business schools or law schools is due to the fact that they're doing an effective job of adding value, are really preparing people to be better than they would have been in the absence of this preparation, versus the fact that they're operating in environments where they are recruiting very effectively and selectively from a relatively high-achieving pool. I would be hesitant as we work through these challenges just as I would be hesitant -- Art, as you know, in his Teachers College report pointed to the British induction model for principals.  I would be hesitant to that for some of these similar reasons. I think, quite frankly, the bottom line is that there are no examples across the board where we have any kind of research-based evidence that MPP programs or business school programs are necessarily training people who are much better the day they leave those institutions than they were the day they walked in. I think there is a lot of what I find persuasive suggestive and anecdotal evidence that particularly the nation's 25 best MBA programs are recruiting with the farsightedness and a focus and are attending to curricular design and instruction with a focus and are worrying about accountability in a way that we're not seeing at even the best ed admin programs, but we don't have evidence of that kind that we're now demanding that IES is demanding in other lines of work. My lessons I would take is I'd say we need to try to unpack the quality of student issue from the quality of instruction issue.  Then the biggest thing I think off the cuff that we see business schools doing is a recognition that there is not a recipe for leadership, that they're not using the same kind of texts that we're seeing in ed admin, they're not seeing the same focus on technical instruction.  What we're seeing is a lot more case study leadership based on the notion that actually leadership is about working with folks in a noisy environment where you need to focus on bottom-line results, but the way you get those results is never cut and dry, but is about harnessing an attention to quality of your personnel, harnessing an attention to organizational culture, harnessing an attention to bottom-line metrics, and then using those in thoughtful ways. Right now I think too often what ed admin programs are looking for when they try to bootstrap up is they're trying to look for cookie cutter recipes of what a good principal does, and I think that's not the path to improvement. MR. PETERSON:  I think we'd better end there.  Thank you all for coming today, and thank you to our panel for their comments. [Applause.]</body></html>