(Guest Post by Matthew Ladner)
TIMS released 2011 results today in a variety of subjects. This time a handful of states were brave enough to volunteer for a pullout of their results. Here are the results on 4th grade reading:
Here are the pullouts:
You got it: Florida students notched the second highest score in the world. Even above (gasp!) Finland.
Late for a meeting. More later, but for now:
10 Comments | 
research reports | Tagged: Florida Reading Achievement, PIRLS | 
Permalink
Posted by matthewladner
When Brian Kisida, Jonathan Mills, and I released our study of administrative bloat in higher education through the Goldwater Institute, we thought it was bad that universities had increased their hiring of administrators (professional staff who are not faculty) at twice the rate of faculty.
I now realize that the perpetrators of waste in higher ed are mere amateurs. The administrative bloat pros can be found in K-12 education. According to a new report from the Friedman Foundation released today, student enrollment has increased 96% since 1950, but the growth in “administrators and other non-teaching staff [was] a staggering 702 percent.”
The report provides results state by state, highlighting the growth in staffing in recent years. Even in the few states where enrollment has declined, staffing levels have grown dramatically. Check it out.
2 Comments | research reports | Permalink
Posted by Jay P. Greene
(Guest post by Greg Forster)
Cato has new research out from Richard Buddin, examining where charter schools draw their students from. Adam Schaeffer offers a summary, emphasizing the dangers of charter schools: “On average, charter schools may marginally improve the public education system, but in the process they are wreaking havoc on private education.”
I agree with the basic premise: charters don’t fix the underlying injustice of government monopolizing education by providing “free” (i.e. free at the point of service, paid for by taxpayers) education, driving everyone else out of the education sector. As Jay and I have argued before, vouchers make the world safe for charters; that implies you can view charters as a response by the government to protect its monopoly against the disruptive threat of voucher legislation.
But what interests me more are the urban/suburban and elementary/secondary breakdowns of these data. It appears that charters are only substantially cutting into private schools in “highly urban” areas. In the suburbs, the charter school option is framed much more in terms of boutique specialty alternatives (schools for the arts, classical education, etc.) rather than “your school sucks, here’s one that works.” If you’d asked me, I would have guessed that would also cut heavily into the private school market – it would appeal to parents of high means who are looking for something out of the ordinary for their children, and that demographic would be most likely to already be in private schools. Yet the data show otherwise; apparently the families choosing boutique suburban charters weren’t much impressed with their private school options. And what’s up with this weird distribution on the elementary/secondary axis? Apparently public middle schools really stink in urban/suburban border areas.
Discuss!
12 Comments | charter schools, research reports | Tagged: Adam Schaeffer, Cato, Greg Forster, Richard Buddin | Permalink
Posted by Greg Forster
(Guest post by Greg Forster)
Jay’s proposed reforms to the way Gates handles science are relevant far beyond the Gates Foundation, and foundations generally. He’s helping us think about how to wrestle with a deeper problem.
Public policy arguments need an authority to which they can appeal. The percentage of the population that is both willing and able to absorb all the necessary information to make a responsible decision without relying on pretty sweeping appeals to authority is very small. And even for us wonks, you can’t reduce the role of authority to zero; life doesn’t work that way. (Economists call this “the information problem.”)
So it’s normal, natural and right for public policy arguments to make some appeals to authority. The problem is that increasingly, our culture has no widely recognized authorities other than science. When there are many potential loci of authority, there is less pressure to corrupt them. If the science doesn’t back your view, you can appeal to other sources of authority. Where there is only one authoritative platform, there’s no alternative but to seize it.
As I once wrote:
Say that you favor a given approach – in education, in politics, in culture – because it is best suited to the nature of the human person, or because it best embodies the principles and historic self-understanding of the American people, and you will struggle even to get a hearing. But if you say that “the science” supports your view, the world will fall at your feet.
Of course, this means powerful interest groups rush in to seize hold of “science,” to trumpet whatever suits their preferences, downplay its limitations, and delegitimize any contrary evidence. If they succeed – which they don’t always, but they do often enough – “the science” quickly ceasees to be science at all. That’s why “scientific” tyrannies like the Soviet Union had to put so many real scientists in jail – or in the ground.
We need other sources of wisdom and knowledge – and hence of authority, because those who are recognized as having wisdom and knowledge will be treated as sources of authority – besides science. As Jay has written:
Science has its limits. Science cannot adjudicate among the competing values that might attract us to one educational approach over another. Science usually tells us about outcomes for the typical or average student and cannot easily tell us about what is most effective for individual students with diverse needs. Science is slow and uncertain, while policy and practice decisions have to be made right now whether a consensus of scientific evidence exists or not. We should rely on science when we can but we also need to be humble about what science can and can’t address…
My fear is that the researchers, their foundation-backers, and most-importantly, the policymaker and educator consumers of the research are insensitive to these limitations of science. I fear that the project will identify the “right” way to teach and then it will be used to enforce that right way on everyone, even though it is highly likely that there are different “right” ways for different kids…
Science can be corrupted so that it simply becomes a shield disguising the policy preferences of those in authority. How many times have you heard a school official justify a particular policy by saying that it is supported by research when in fact no such research exists? This (mis)use of science is a way for authority figures to tell their critics, “shut up!”
To summarize the whole point, our group of school choice researchers put it well (false humility aside) in our Education Week op-ed earlier this year:
Finally, we fear that political pressure is leading people on both sides of the issue to demand things from “science” that science is not, by its nature, able to provide. The temptation of technocracy—the idea that scientists can provide authoritative answers to public questions—is dangerous to democracy and science itself. Public debates should be based on norms, logic, and evidence drawn from beyond just the scientific sphere.
What can we do about it? Beyond building in checks and balances to ensure that science isn’t being abused, we can make a deliberate effort to appeal to non-scientific sources of wisdom. There’s nothing unscientific about relying on “norms, logic, and evidence drawn from beyond just the scientific sphere.” In Pride and Prejudice, Caroline Bingley comments that it would be more rational if there were more conversation and less dancing at balls; her brother comments that this would indeed be “much more rational, I dare say, but much less like a ball.” It might be more scientific if our civic discourse appeals to nothing but science, but it’s much less like civic discourse.
For a good example of what I mean, check out Freedom and School Choice in American Education. When it came out, I commented on how it showed the diverse values that had led the authors to support school choice:
What’s particularly valuable about this book, I think, is how it gives expression to the very different paths by which people come to hold educational freedom as an aspiration, and then connects those aspirational paths to the practical issues that face the movement in the short term. Jay comes to educational freedom with an emphasis on accountability and control; against the Amy Gutmanns of the world who want to set up educational professionals as authority figures to whom parents must defer, Jay wants to put parents back in charge of education. Matt comes to educational freedom with an emphasis on alleviating unjustified inequalities; against the aristocrats and social Darwinists of the world who aren’t bothered by the existence of unjustified inequalities, Matt wants social systems to maximize the growth of opportunities for those least likely to have access to them. And I come to educational freedom with an emphasis on the historical process of expanding human capacities, especially as embodied in America’s entrepreneurial culture; agaisnt all forms of complacency, I want America to continue leading the world in inventing ever better ways of flourishing the full capacities of humanity. And each of the other contributors has his or her own aspirational path.
Individual liberty; the lifting up of the poor and the marginalized; the American experiment in enterprise culture. These are fine things worth fighting for, and they would remain so no matter what the science says.
6 Comments | research reports | Tagged: Greg Forster | Permalink
Posted by Greg Forster
Last fall Josh McGee and I developed the Global Report Card (GRC) for the George W. Bush Institute. The GRC is a tool that allows people to compare the level of academic achievement in virtually every school district in the United States to the average for their state, the country, and a comparison group of 25 industrialized countries.
Above is a new interview with President Bush in which he discusses the Global Report Card (it’s around minute 25).
The Global Report Card received a fair amount of coverage when it was released, but keep your eyes out for an updated and improved version this upcoming fall. The results of the GRC are consistent with other international comparisons, including a series of pieces by Eric Hanushek, Paul Peterson, and Ludger Woessmann (the most recent of which can be found here). But the GRC goes a step further by allowing comparisons to be made at the school district level. GRC 2.0 will also have some new features and comparisons that people might find useful.
3 Comments | research reports | Tagged: Global Report Card | Permalink
Posted by Jay P. Greene
(Guest post by Greg Forster)
Today, the Friedman Foundation is releasing a study I did with James Woodworth: The Greenfield School Revolution and School Choice. We know from previous research that vouchers (and equivalent programs like tax credits and ESAs) consistently deliver better academic performance, but the size of the impact is not revolutionary. Meanwhile, the whole world is watching as charter school operators (Carpe Diem, Rocketship, Yes Prep, etc.) reinvent the school from the ground up.
It’s ironic that these schools are charters, not voucher schools. A properly designed (i.e. universal) choice program would do a better job than charters of supporting these highly ambitious “greenfield” school models. But existing choice programs are not properly designed, so our impression was that they’re excluding these educational entrepreneurs, instead simply transferring students from one existing set of schools (public) to another (private).
We wanted to test our theory and make sure it was true, not just an accident of publicity or media bias, that the reinvention of the school wasn’t being supported by existing choice programs. We combed through twenty years’ worth of federal data (CCD and PSS) to see if we could find any evidence of disruption in the structure of the private school sector in places that had school choice programs.
We found that while existing school choice programs may be delivering moderately better academic outcomes, they aren’t disrupting the private school sector the way they need to be. In one or two places we found visible impacts, but nothing like a reinvention of schooling. The only impact of any considerable size is the dramatic change in racial composition in the private school population of Milwaukee.
In addition to the empirical findings, the study outlines 1) why radical “greenfield” school models are essential to drive the kind of education reform we need, and 2) why universal school choice would do a better job than charter schools of sustaining it.
Special thanks to Rick Hess, from whom we borrow the term “greenfield,” and Jay Greene for giving us their comments and insights as we developed this study!
3 Comments | charter schools, Education Savings Accounts, research reports, tax-credit scholarships, vouchers | Tagged: Friedman Foundation, greenfield, Greg Forster, James Woodworth | Permalink
Posted by Greg Forster
Check out this interview that Jason Riley of the Wall Street Journal did with me regarding my blog post on the delay and manipulation of Head Start research by the Obama Administration.
I still haven’t figured out how to embed from WSJ, but here is a screenshot:
5 Comments | research reports | Permalink
Posted by Jay P. Greene
The Department of Health and Human Resources is up to its old tricks of delaying research whose results are likely to undermine their darling program, Head Start. A group of five U.S. Senators sent a letter to HHS Secretary Kathleen Sebelius last week demanding an explanation for why the latest round of results of the congressionally-mandated study have not been released four years after data collection was complete and one year after the report was scheduled to be released.
In 2010 I told you about how the Department of Health and Human Services delayed the release of the previous round of disappointing research results about the lasting effects of Head Start. When the extremely high quality study, involving a random-assignment design on a representative sample of all Head Start programs nationwide, was finally released three years after the data collection was complete, it found that students randomly assigned to Head Start performed no better on cognitive measures by the end of kindergarten and first grade.
Despite these null results, HHS issued a statement that in typical Orwellian fashion declared the program a huge success. Assistant Secretary for Children and Families Carmen Nazario was quoted in the statement concluding that “Head Start has been changing lives for the better since its inception.” And Health and Human Services Secretary Kathleen Sebelius was quoted declaring that “research clearly shows that Head Start positively impacts the school readiness of low-income children”
If the government’s proclivity to delay the release of politically undesirable results and to manipulate — actually, completely distort — the findings is not enough to engender skepticism among reporters, researchers, and policymakers, I have no idea what will. But I continue to see reporters, researchers, and policymakers invoke government research as authoritative without the least bit of critical scrutiny.
This uncritical acceptance of government press releases as gospel by reporters is particularly disgraceful. I understand that reporters are miserably paid and stretched beyond their limit as staffs are reduced, but the heart of a reporter’s responsibility is to challenge the powerful. And there is no one more powerful than the government. They are so powerful that they can delay the release of research and declare that up is down when the results do come out.
11 Comments | politics, research reports | Tagged: Head Start, Health and Human Services | Permalink
Posted by Jay P. Greene
Pictured (L to R): Rick Hess, Jay Greene, Greg Forster, Mike Petrilli and Matt Ladner
(Guest post by Greg Forster)
Today, Education Week carries a joint editorial signed by nine scholars and analysists. We came together to agree that Mom and apple pie are good, Nazis and Commies are bad, and the empirical research supports the expansion of school choice:
Choice’s track record so far is promising and provides support for continuing expansion of school choice policies…Among voucher programs, random-assignment studies generally find modest improvements in reading or math scores, or both. Achievement gains are typically small in each year, but cumulative over time. Graduation rates have been studied less often, but the available evidence indicates a substantial positive impact. None of these studies has found a negative impact…Other research questions regarding voucher program participants have included student safety, parent satisfaction, racial integration, services for students with disabilities, and outcomes related to civic participation and values. Results from these studies are consistently positive…
In addition to effects on participating students, another major topic of research has been the impact of school choice on academic outcomes in the public school system…Among voucher programs, these studies consistently find that vouchers are associated with improved test scores in the affected public schools. The size of the effect in these studies varies from modest to large. No study has found a negative impact.
We have diverse viewpoints on many issues, but we share a common commitment to helping inform public decisions with such evidence as science is legitimately able to provide. We do not offer false certainty about a future none of us knows. But the early evidence is promising, and the grounds for concern have been shown to be largely baseless. The case for expanding our ongoing national experiment with school choice is strong.
This may well be the most important part:
The most important limitation on all of this evidence is that it only studies the programs we now have; it does not study the programs that we could have some day. Existing school choice programs are severely limited, providing educational options only to a targeted population of students, and those available options are highly constrained.
These limitations need to be taken seriously if policymakers wish to consider how these studies might inform their deliberations. The impact of current school choice programs does not exhaust the potential of school choice.
On the other hand, the goal of school choice should be not simply to move students from existing public schools into existing private schools, but to facilitate the emergence of new school entrants; i.e., entrepreneurs creating more effective solutions to educational challenges. This requires better-designed choice policies and the alignment of many other factors—such as human capital, private funding, and consumer-information sources—that extend beyond public policy. Public policy by itself will not fulfill the full potential of school choice.
Although I also feel particularly strongly about this:
Finally, we fear that political pressure is leading people on both sides of the issue to demand things from “science” that science is not, by its nature, able to provide. The temptation of technocracy—the idea that scientists can provide authoritative answers to public questions—is dangerous to democracy and science itself. Public debates should be based on norms, logic, and evidence drawn from beyond just the scientific sphere.
Signatories:
Kenneth Campbell is the president of the Black Alliance for Educational Options, in Washington.
Paul Diperna is the research director for the Friedman Foundation for Educational Choice, in Indianapolis.
Robert C. Enlow is the president and chief executive officer of the Friedman Foundation for Educational Choice.
Greg Forster is a senior fellow at the Friedman Foundation for Educational Choice.
Jay P. Greene is the department head and holder of the 21st-century endowed chair in education reform at the University of Arkansas, at Fayetteville, and a fellow in education policy at the George W. Bush Institute, in Dallas.
Frederick M. Hess is a resident scholar and the director of education policy studies at the American Enterprise Institute, in Washington, as well as a blogger for Education Week.
Matthew Ladner is a senior adviser for policy and research at the Foundation for Excellence in Education, in Tallahassee, Fla.
Michael J. Petrilli is the executive vice president of the Thomas B. Fordham Institute, in Washington.
Patrick J. Wolf is a professor and holder of the 21st-century endowed chair in school choice at the University of Arkansas, at Fayetteville.
Our color-coordinated mechanical lion battle chariots that join together into a giant robot are still under construction.
Defender of the empirical research universe!
19 Comments | charter schools, research reports, tax-credit scholarships, vouchers | Tagged: Education Week, Greg Forster, Jay Greene, Kevin Chavous, Matt Ladner, Mike Petrilli, Pat Wolf, Paul DiPerna, Rick Hess, Robert Enlow | Permalink
Posted by Greg Forster
Comic book geeks are familiar with Bizarro World, a place where everything is the opposite of what it is in the normal world. In Bizarro World, people would abandon a policy strongly supported by rigorous evidence while embracing an alternative policy for which the evidence showed little promise.
I was thinking about Bizarro World and then it struck me — Perhaps the Gates Foundation has somehow fallen into the Bizarro World. It’s just about the only thing that makes sense of their Bizarro choices with respect to education reform strategies.
The dominant education reform strategy of the Gates Foundation before 2006 was to break large high schools into smaller ones, often using school choice and charter schools. As a Business Week profile put it:
The foundation embraced what many social scientists had concluded was the prime solution: Instead of losing kids in large schools like Manual, the new thinking was to divide them into smaller programs with 200 to 600 students each. Doing so, numerous studies showed, would help prevent even hard-to-reach students from falling through the cracks. The foundation didn’t set out to design schools or run them. Its goal was to back some creative experiments and replicate them nationally.
But the Gates Foundation wasn’t patient enough to let the experiments produce results. Instead, they hired SRI and AIR to do a very weakly-designed non-experimental evaluation that produced disappointing results. Gates had also commissioned a rigorous random-assignment evaluation by MDRC, but it would take a few more years to see if students graduated and went on to college at higher rates if they were assigned by lottery to a smaller school.
Gates couldn’t wait. They were convinced that small schools were a flop, so they began to ditch the small school strategy and look for a new Big Idea. Tom Vander Ark, the education chief who had championed small schools, was out the door and replaced with Vicki Phillips, a superintendent whose claim to fame, such as it was, came from serving as Portland’s superintendent where she consolidated schools (not breaking them into smaller ones) and centralized control over curriculum and instruction. As one local observer put it:
In her time in the famously progressive, consensus-driven city, she closed six schools, merged nearly two dozen others through K-8 conversions, pushed to standardize the district’s curriculum, and championed new and controversial measures for testing the district’s 46,000 children-all mostly without stopping for long enough to adequately address the concerns her changes generated in the neighborhoods and schools where they played out. During her three years in Portland, Phillips’ name became synonymous with top-down management, corporate-style reforms, and a my-way-or-the-highway attitude.
Under Phillips and deputy education director, Harvard professor Tom Kane, the Gates Foundation has pursued a very different strategy: attempting to identify the best standards, curriculum, and pedagogy and then imposing those best practices through a national system of standards and testing.
And here is where we see that Gates must be the Bizarro Foundation. The previous strategy of backing small schools has now been vindicated by the rigorous random-assignment study Gates couldn’t wait for. According to the New York Times:
The latest findings show that 67.9 percent of the students who entered small high schools in 2005 and 2006 graduated four years later, compared with 59.3 percent of the students who were not admitted and instead went to larger schools. The higher graduation rate at small schools held across the board for all students, regardless of race, family income or scores on the state’s eighth-grade math and reading tests, according to the data.
This increase was almost entirely accounted for by a rise in Regents diplomas, which are considered more rigorous than a local diploma; 41.5 percent of the students at small schools received one, compared with 34.9 percent of students at other schools. There was little difference between the two groups in the percentage of students who earned a local diploma or the still more rigorous Advanced Regents diploma.
Small-school students also showed more evidence of college readiness, with 37.3 percent of the students earning a score of 75 or higher on the English Regents, compared with 29.7 percent of students at other schools. There was no significant difference, however, in scores on the math Regents.
Meanwhile, as part of their newly embraced top-down strategy, the Gates effort to identify the secret formula for effective teaching has failed to bear fruit. The Gates -operated Measuring Effective Teachers Project failed to identify any rubric of observing teachers or any components of those rubrics that were strongly predictive of gains in student learning. And the Gates-backed “research” supporting the federally-orchestrated Common Core push for national standards and testing has been strikingly lacking in scientific rigor and candor.
In short, the Gates Foundation has ditched what rigorous evidence shows worked and is pushing a new strategy completely unsupported by rigorous evidence. They must be in Bizarro World. Somebody please get me some blue kryptonite.
16 Comments | instructional reform, research reports | Tagged: gates foundation, Measuring Effective Teachers, Smaller Schools | Permalink
Posted by Jay P. Greene
You are currently browsing the archives for the research reports category.
Jay P. Greene's Blog 