Jay P. Greene's Blog

Another video from ChoiceMedia.TV:

Peter Meyer at the Fordham Institute asked me to contribute a piece to his Board’s Eye View blog to address The BIG Questions on school governance.  Here is what I sent him:

Being against greater national control over education policy is not the same as being for local school districts.  I appreciate Peter Meyer for giving me the opportunity in this space to explain what I am for when it comes to school governance.

Fundamentally, I am for parental control over the education of their children, so I guess that I am for as little governance over education as we can manage.  In my ideal world, which I’ve tried to explain and justify at greater length in this book chapter, parents would be given as much money as is minimally necessary to fulfill their obligation to educate their children and would choose the location, manner, and content of that education.  Since education is just a subset of all of the activities in which parents engage to raise their children to be productive adults, we should defer to parents as much in how they educate their children as how they raise those children more generally.  As long as parents do not neglect or abuse their children, the government should have as little role in education as is possible.

But we don’t live in my ideal world and I have no expectation that we will.  All that I can hope for is that we will inch closer to my ideal rather than further away from it.  With that in mind, I favor governance arrangements that facilitate greater parental choice and control over education over those that would reduce parental choice and control.

So, I have no particular love for local school districts.  They just more closely approximate parental choice and control than does granting more power over education to the state or national governments.  It would be even better in my view to abolish school districts and have every school be like a charter school – a publicly regulated school of choice that would choose its own method and content of education and would have to attract willing families to generate the revenue to pay for it.  But I understand the idea of abolishing school districts and having every school operate as a charter school is only slightly less unrealistic than a virtually unregulated world of parental choice and control.

As unrealistic as making every school a charter school may be, we have been inching in that direction.  A little more than two decades ago we had no charter schools.  Today charter schools constitute nearly 5% of all public schools and educate about 3% of all students.  And the expansion of parental choice and control has been even greater when one considers the fully array of choices that have been introduced over the last two decades, including vouchers, tax credit funded scholarships, virtual schools, inter-district choice, magnet schools, etc…  My ideal world may be an unattainable fantasy, but my vision of gradual progress toward that ideal has been a fairly accurate description of the trends over the last few decades.

But there are some people, primarily edupundits located within the DC beltway, who have very different fantasies about ideal governance arrangements.  Rather than shifting arrangements directly toward greater parental choice and control, they dream about measures granting greater control to state and national authorities.  They rightly point out the defects of local school districts, but they wrongly see the solution in greater centralization of power rather than in the expansion of parental choice and control.

Their justifications for increasing the power of state and national authorities over education are more like empty political slogans than actual intellectual arguments based on principle.  For example, we’ll hear some say that a decentralized system of education cannot meet our needs in the 21^st century: “The system of schooling we have today is the legacy of the 19th century — and hopelessly outmoded in the 21st.”  Of course, representative democracy is also a legacy of the 18^th and 19^th centuries, but that doesn’t mean we need to dispense with it to meet the challenges of our brave new 21^st century world.  Saying that the 21^st century demands certain skills or governance arrangements is just sloganeering and manipulating people to submit to a proposal, not a real argument.

Some attempt to justify greater centralization in education by saying that our current system is too uncoordinated, contradictory, duplicative, and confusing.  We need the greater coherence, planning, and order that more centralized control can offer.  Do you notice how the central authorities in these proposals are always imagined to be highly competent and benevolent?  They never entertain the very real possibility that the central authority might be coherent, well-planned, and orderly in pursuing something awful.  Those attracted to central planning in education may want to consider how well economic central planning has turned out.

Some attempt to justify granting more power to state and national authorities by looking overseas and claiming that the highest achieving countries have more centralized governance arrangements.  Let’s ignore for a moment that these are not accurate descriptions of how many high-achieving countries have structured their governance – Canada and Australia, for example, are high achieving and have decentralized governance arrangements.  The more fundamental problem is that the “best practices” movement of imitating some of the practices of others who are successful fails to consider what actually caused others to be successful.  Just imitating some of what they do is like the Cargo Cults found in Pacific Islands following WW II, where locals believed that if they built imitations of planes, runways, and control towers, the cargo and plentiful goods that had arrived during the war would return.  They didn’t understand that imitating the trappings of an airport doesn’t cause cargo to arrive any more than imitating the trappings of other countries’ governance arrangements will cause high achievement.

Lastly, some advocates of centralization argue that you actually need to centralize certain things in order to facilitate better decentralized control over other things.  They describe this approach as “tight-loose,” where the central authority assumes greater control over determining and regulating the goals of education and local authorities are then given greater flexibility over the means for meeting those goals.  Of course, ends and means are not so easily separated.  Ends often dictate or at least constrain the selection of means.  In addition, in what fantasy world would the central authority carefully limit its role to setting and regulating ends once it is given authority over an issue?  At least I recognize that my fantasy of parental choice and control is unrealistic.

The folks at ChoiceMedia.TV have developed a new PBS series focused on education reform issues called “Reform School.”  You can see some clips of the pilot episode with yours truly here:

(Guest Post by Mike McShane and Gary Ritter)

That’s it, you heard it here first folks, packs of wild dogs have seized control of America’s major cities.

As crazy as that sounds, a study has been circulating the AERA-/Blogo-/twittersphere that’s states that urban Texas school districts have a black male graduation rate of over 80%.  We all know how much some folks here on the Jay P Greene Blog love Texas, but that is just a bit too hard to swallow.

Let’s back up a minute.  Over the past few days, the press (or actually, Diane Ravitch’s twitter page, and then the press, obediently) picked up a story about a “new” study.  OK, it actually isn’t “new” (it first came out in the Berkeley Review of Education in the Fall of 2011) nor is it really a “study”, but more on that later.   The purpose of this study was clear (to attack KIPP) but in the authors’ zeal, they ended up reporting something too good to be true.

Very quickly, using some rough data from schools in Texas, the authors claim to find that charter schools in Texas, and KIPP schools in particular, have higher attrition rates than comparable public schools, even though KIPP schools allegedly spend more money per pupil.

It appears that the authors, in their haste to smear KIPP schools and disprove the strawman idea that choice (as envisioned today) is a “panacea” (using a 20 year old quote) for all of the ills of the American education system, made some pretty shocking errors and omissions that call into question nearly all of their conclusions.

First of all, several of the alleged “findings” were not “found” in this “study”.  Rather, the authors fill their abstract and conclusions with rehashed claims from an earlier, widely discredited study (see this and this and this and this criticism of the flawed Gary Miron paper).

As for the errors in this paper, there are several.  We’ll just highlight a few of the most glaring:

1.  First, we derive the 80% graduation number from tables 7 and 8 (pg. 169), which report an annual dropout rate from black students of 3% for grades 6-12 in the “comparable urban districts” of Austin, Dallas, and Houston.  Before we dive into the glaring problems of tables 7 and 8, we must first draw attention to the author’s violation of the denominator law.  We don’t know, in the context of this report, what 3% even means.  That is, what is the numerator and what is the denominator that created that rate?  Is that a yearly figure?  Is that a cohort figure?  The authors are absolutely unclear.  Our best guess is that this is a yearly figure, which if compounded, would put the dropout rate for those districts at about 20% for that time period.  As a point of comparison, the dropout rate nationwide for Black males is 53%; if the authors are right, we should all move to the Lone Star State!

 2.  If that is too hard to believe, the tables also report that this 3% figure is lower than the 4% of black dropouts in the rest of the state.  So, if the Texas miracle didn’t do enough to impress you, you can find Texas to be probably the only state where suburban and rural areas have higher dropout rates than cities.

 3.  In addition to farcically large results, tables 7 and 8 (on pg. 169) also appear to have either basic arithmetic mistakes and/or are missing many of their observations when calculating their graduation rates.  The first two columns “Majority black” and “not majority black” should be comprehensive; that is, all of the observations should fall into one of those two categories.  The same is true with the third and fourth column “>100 (Black Students)” and “<100 (Black Students)”. Thus, both of the numbers in the N’s of these columns should sum to the same number.  However, they don’t. In table 7, the first two columns sum to 167 total charter schools, while the second two columns sum to 245 (incidentally the same number as the “All Charters” N).  The same holds true in Table 8, where the first two columns sum to 243, while the second two sum to 373 (again the same number as the “All Charters”).  So where did the other schools go?

Beyond these problems with the author’s primary analyses, this article eschews higher quality studies of the question at hand to focus on clearly flawed research on the topic.  Mathematica already looked into this question in rigorous studies that found positive impacts on achievement, and “did not find levels of attrition among these KIPP middle schools systematically higher (or lower) than those of other “ schools within their districts (they were also clear about the descriptions and sources of the numbers used in the analysis).

In short, any reasonable person who actually read the content of this “new study” would immediately see so many red flags as to take some serious pause before disseminating the findings unqualified to the universe of education news followers.  (We wonder how closely Ms. Ravitch reviewed the study?  She may well have tweeted first and asked questions later!)  Unfortunately, we live in a world populated by many, many, many unreasonable people.

On the bright side, good research continues to show that KIPP schools are effective for underserved students, most serious people disregard “new studies” that are neither “new” nor “studies”, and hard-working KIPP students, teachers, and school leaders keep going about their work each day.

By the way, if you want to see KIPP’s response to this study, it is here.

I don’t think Marc Tucker liked my review of his book in Education Next.  He responds in this post on the Ed Next blog.  And my reply to his response is also on the Ed Next blog.

Above I’ve posted a video of Tucker explaining his response.

I received this amazing press release from Tom Vander Ark about how computer grading of essays may be as accurate as human grading.  I’m not sure if this means that computer grading has really advanced or if human grading really stinks.  Besides, I don’t even know why the scientists invented the robots.

In any event, here is the release:

A direct comparison between human graders and software designed to score student essays achieved virtually identical levels of accuracy, with the software in some cases proving to be more reliable, a groundbreaking study has found.

The study, which was underwritten by the William and Flora Hewlett Foundation and conducted by experts in educational measurement and assessment, will be released here on Monday, April 16th, at the annual conference of the National Council on Measurement in Education. An advance copy of the study is available today at http://bit.ly/HJWwdP.

“The demonstration showed conclusively that automated essay scoring systems are fast, accurate, and cost effective,” said Tom Vander Ark, CEO of Open Education Solutions, which provides consulting serves related to digital learning, and co-director of the study.

That’s important because writing essays are one important way for students to learn critical reasoning, but teachers don’t assign them often enough because grading them is both expensive and time consuming. Automated scoring of essays holds the promise of lowering the cost and time of having students write so they can do it more often.

Education experts believe that critical reasoning and writing are part of a suite of skills that students need to be competitive in the 21st century. Others are working collaboratively, communicating effectively and learning how to learn, as well as mastering core academic content. The Hewlett Foundation calls this suite of skills Deeper Learning and is making grants to encourage its adoption at schools throughout the country.

“Better tests support better learning,” says Barbara Chow, Education Program Director at the Hewlett Foundation. “This demonstration of rapid and accurate automated essay scoring will encourage states to include more writing in their state assessments. And, the more we can use essays to assess what students have learned, the greater the likelihood they’ll master important academic content, critical thinking, and effective communication.”

For more than 20 years, companies that provide automated essay scoring software have claimed that their systems can perform as effectively, more affordably and faster than other available methods of essay scoring. The study was the first comprehensive multi-vendor trial to test those claims. The study challenged nine companies that constitute more than ninety-seven percent of the current market of commercial providers of automated essay scoring to compare capabilities. More than 16,000 essays were released from six participating state departments of education, with each set of essays varying in length, type, and grading protocols. The essays were already hand scored according to state standards. The challenge was for companies to approximate established scores by using software.

At a time when the U.S. Department of Education is funding states to design and develop new forms of high-stakes testing, the study introduces important data. Many states are limited to multiple-choice formats, because more sophisticated measures of academic performance cost too much to grade and take too long to process. Forty-five states are already actively overhauling testing standards, and many are considering the use of machine scoring systems.

The study grows from a contest call the Automated Student Assessment Prize, or ASAP, which the Hewlett Foundation is sponsoring to evaluate the current state of automated testing and to encourage further developments in the field.

In addition to looking at commercial vendors, the contest is offering $100,000 in cash prizes in a competition open to anyone to develop new automated essay scoring techniques. The open competition is underway now and scheduled to close on April 30th. The pool of $100,000 will be awarded the best performers. Details of the public competition are available atwww.kaggle.com/c/ASAP-AES . The open competition website includes an active leader board to document prize rules, regularly updated results, and discussion threads between competitors.

The goal of ASAP is to offer a series of impartial competitions in which a fair, open and transparent participation process will allow key participants in the world of education and testing to understand the value of automated student assessment technologies.

ASAP is being conducted with the support of the Partnership for Assessment of Readiness for College and Careers and Smarter Balanced Assessment Consortium, two multi-state consortia funded by the U.S. Department of Education to develop next-generation assessments. ASAP is aligned with the aspirations of the Common Core State Standards and seeks to accelerate assessment innovation to help more students graduate from college and to become career ready.

Jaison Morgan, CEO of The Common Pool, a consulting business that specializes in developing effective incentive models for solving problems, and co-director of the study, said the prize and studies will raise broader awareness of the current capabilities of automated scoring of essays.

“By offering a private demonstration of current capabilities, we can reveal to our state partners what is already commercially available,” Morgan said. “But, by complimenting it with a public competition, we will attract new participants to the field and investment from new players. We believe that the public competition will trigger major breakthroughs.”

ASAP is preparing to introduce a second study, in which private providers and public competitors will be challenged to reveal the capabilities of automated scoring systems for grading short-answer questions. The second study will be conducted this summer. There are another three ASAP studies in development.