The Professional Judgment Un-Dead

March 25, 2009

It’s time we drive a stake through the heart of “professional judgment” methodologies in education.  Unfortunately, the method has come back from the grave in the most recent Fordham report on regulating vouchers in which an expert panel was asked about the best regulatory framework for voucher programs.

The methodology was previously known for its use in school funding adequacy lawsuits.  In those cases a group of educators and experts was gathered to determine the amount of spending that is required to produce an adequate education.  Not surprisingly, their professional judgment was always that we need to spend billions and billions (use Carl Sagan voice) more than we spend now.  In the most famous use of the professional judgment method, an expert panel convinced the state courts to order the addition of $15 billion to the New York City school system — that’s an extra $15,000 per student.

And advocates for school construction have relied on professional judgment methodologies to argue that we need $127 billion in additional spending to get school facilities in adequate shape.  And who could forget the JPGB professional judgment study that determined that this blog needs a spaceship, pony, martinis, cigars, and junkets to Vegas to do an adequate job?

Of course, the main problem with the professional judgment method is that it more closely resembles a political rather than a scientific process.  Asking involved parties to recommend solutions may inspire haggling, coalition-building, and grandstanding, but it doesn’t produce truth.  If we really wanted to know the best regulatory framework, shouldn’t we empirically examine the relationship between regulation and outcomes that we desire? 

Rather than engage in the hard work of collecting or examining empirical evidence, it seems to be popular among beltway organizations to gather panels of experts and ask them what they think.  Even worse, the answers depend heavily on which experts are asked and what the questions are. 

For example, do high stakes pressure schools to sacrifice the learning of certain academic subjects to improve results in others with high stakes attached?  The Center for Education Policy employed a variant of the professional judgment method by surveying school district officials to ask them if this was happening.  They found that 62% of districts reported an increase in high-stakes subjects and 44% reported a decrease in other subjects, so CEP concluded that high-stakes was narrowing the curriculum.  But the GAO surveyed teachers and found that 90% reported that there had not been a change in time spent on the low stakes subject of art.  About 4% reported an increase in focus on art and 7% reported a decrease.  So the GAO, also employing the professional judgment method, gets a very different answer than CEP.  Obviously, which experts you ask and what you ask them make an enormous difference.

Besides, if we really wanted to know about whether high stakes narrow the curriculum, shouldn’t we try to measure the outcome directly rather than ask people what they think?  Marcus Winters and I did this by studying whether high stakes in Florida negatively impinged on achievement in the low-stakes subject of science.  We found no negative effect on science achievement from raising the stakes on math and reading.  Schools that were under pressure to improve math and reading results also improved their science results.

Even if you aren’t convinced by our study, it is clear that this is a better way to get at policy questions than by using the professional judgment method.  Stop organizing committees of selected “experts” and start analyzing actual outcomes.


Moe and Peterson on DC Vouchers

March 19, 2009

The Charter/Voucher Extended Dance Remix

March 16, 2009

My post last week on why supporters of charter schools don’t also support vouchers generated a large amount of discussion.  Here are some tidbits on the same topic:

First, Doug Tuthill, the former teacher union leader and now head of Florida’s choice advocacy group, Step Up For Students, emailed me to say:

In Florida we have adopted the same approach to charters and vouchers you advocated in your recent exchange with Andy Rotherham.  Given the idealized status of “public education” in our culture, getting into a public versus private school debate is a losing proposition.  Therefore we argue that  publicly-funded private schools are part of our public education system and the real issue is how best to regulate all publicly-funded education.   Our approach is aided by the reality that public education in Florida is expanding to incorporate a variety of publicly-funded private providers.  Many of the state’s best secondary magnet programs are run by private providers (my favorites are two aeronautic magnet programs run by Embry-Riddle University in Okaloosa County), the K-5 portion of our state online school, the Florida Virtual School, is run by a private provider, the overwhelming majority of our charter schools are run by private providers, and of course all the schools providing services through the McKay and Corporate Tax Credit Scholarships are private providers.  To arbitrarily label some of these public-private partnerships as “voucher” programs and therefore suggest they are bad is nonsensical.  The goodness or badness of any publicly-funded education program should be determined by its effectiveness and efficiency, not by how it is labeled.

 

Andy and President Obama, among others, are caught in the old “public-versus-private-school” paradigm, which is why they support “charters” but not “vouchers.”    But as you pointed out in your exchange with Andy, this is an illogical distinction unless one is stuck in the “public is good, private is bad” mindset and possesses a myopic view of what constitutes public and private. 

 

And here is an interesting item about how Orthodox Jewish groups have come out against secular charter schools as a substitute for a Jewish education.  The Ben Gamla Hebrew Charter School in Florida focuses on Hebrew language and Jewish history but has no religious instruction.  And it is drawing students away from Jewish private schools in part because the charter is free to students while the private schools have to charge tuition.  The huge expansion in charter schools may be posing an even greater competitive threat to established private schools than to traditional public schools.

 

Leaders of the Orthodox movement recognize this threat to their efforts to focus on religious instruction.  A spokesman for the Orthodox group Agudath Israel of America, Rabbi Avi Shafran, commented: “We are not pro-charter schools. They are not replacements for a yeshiva education, and anyone who can imagine they could be are fooling themselves… There is no end around the fact that Jewish education is a Jewish education and that can only happen in a Jewish institution with a Jewish curriculum. And because our nation does not permit religious instruction in public schools, there is no way to avoid the need that Jewish parents have for yeshivas.” 


Pay No Attention To My Legislative Agenda

March 11, 2009

President Obama gave a great speech yesterday in which he strongly endorsed charter schools and merit pay.  He also emphasized the need to remove ineffective teachers from classrooms and to expand access to pre-school.

The problem is that these words bear almost no resemblance to the education priorities contained in Obama’s legislative agenda.  This is really strange.  I’m accustomed to presidents exaggerating the attractiveness of their proposed policies.  But Obama is the first president that I can think of who pushes the attractiveness of policies that he is hardly pursuing in legislation while concealing the bulk of his actual efforts.

I’ve previously written about how the bulk of Obama’s increased education spending goes to status quo programs, such as Title I, special ed, Pell Grants, school construction, and generally holding localities harmless against losses in tax revenue.  Almost no money has been devoted to charter schools, merit pay, efforts to remove ineffective teachers, and even pre-school (which received only $4 billion of the $800 billion stimulus package, and most of that was for propping up status quo Head Start programs).  All of the great (and not so great) education policies that Obama talks about are almost completely absent in legislation that he has backed.  And he hardly says a peep about all of the education policies that he does throw money at. 

Obama just distracts us from his actual efforts with pretty words about things that he is hardly doing.  Of course, the most obvious thing he was distracting us from with his speech yesterday was the Senate vote to begin the execution of the DC voucher program.  He didn’t say a word about yesterday’s actions, knowing that all of the headlines would be about the reforms he did endorse (but has done almost nothing to actually enact).


But I Won’t Do That

March 10, 2009

There’s an awful Meatloaf song where he declares that he would do anything for love… but he won’t do that.  It’s isn’t entirely clear what Mr. Loaf (as the New York Times calls him) won’t do for love.  But it is clear that there is something he will not do even though he has just declared that he would do anything

This incoherent, cheesy awfulness reminds me of the argument that we should support charters but oppose vouchers.  I’m sure that’s what you were thinking as well.  It’s like they would do anything for choice… but they won’t do that.

If one supports the view that expanding choice and competition help students who choose and either helps or does no harm to traditional public schools, why limit that support to charter schools?  I know people say that at least charters are still public schools, but why exactly does that matter?  There is no magic pixie dust in the word “public” that makes things good or serve the public interest.  If we add the word “public” to vouchers so that we now call them “public vouchers” does that make them acceptable to pro-charter/anti-voucher folks?

I know that some suggest the important part of charters being “public” is that they can be regulated so as to assure public goals.  But whatever regulations are really necessary for public goals can be attached as a condition to vouchers as easily as to charters.  If we think teachers need to have certain credentials or students have to take certain tests, that can be (and has been) required of voucher-receiving private schools.  It’s not clear what about the publicness of charter schools, or even traditional public schools, make them better suited to serving the public good or being regulated for that purpose.

I suspect that some of the real rationale behind supporting charters but opposing vouchers is an unstated uneasiness with vouchers supporting students in religious schools.  But the U.S. Supreme Court has settled this issue as a matter of constitutional law.  And if the objection is one of desirable public policy, why is there near universal support for vouchers (Pell Grants) to attend BYU or Baylor but not St. Thomas Aquinas High School? 

This leads me to suspect that the real REAL reason for folks supporting charters but opposing vouchers is the political desire to appear moderate regardless of how incoherent and irrational it is.  Today President Obama is going to tout his support for charter schools.  And he’s going to tout his support for Pell Grants.  But he will not support vouchers.  Holding all of these positions make no logical sense, but they are thought to have some political appeal. 

I guess I understand why politicians take these incoherent positions, but why do people in academia, think tanks, and the blogoshpere do this?  Unlike politicians we don’t have to lie or make false distinctions for a living.  So I challenge anyone to explain exactly why, other than for the political advantage of triangulation, people should support charters but oppose vouchers.

My bet is that any argument will resemble the “look at the silly monkey” argument.  It’s even more powerful than the Chewbacca defense because it makes your head explode.


Andy’s Just Plain Wrong

March 5, 2009

Andy Rotherham is a great guy.  And he’s often right.  But I’m afraid that on vouchers he’s just plain wrong.

Andy responded to my post, which was a response to an earlier post he wrote on vouchers.  Let me just run through his arguments:

First, Andy wants to argue that vouchers have stalled politically.  I pointed out that there are now 24 voucher or tax-credit programs in 15 states serving more than 100,000 students.  And two new programs were adopted last year and a third significantly expanded. 

No fair, Andy cries, including tax-credit programs “creates a false sense of scale for intentional choice plans.”  What’s false about counting tax credit programs, like the one in Florida which functions as the largest voucher program in the country?  The program gives vouchers — excuse me — “scholarships” to students from organizations that are funded with dollar for dollar tax credit donations from corporations.  This is virtually identical in financing and effect as the state simply giving vouchers to students.  The only difference is that the tax credit is treated better by the courts (don’t ask why) because the money never enters the state treasury before going right back out the door as a voucher.

But let’s say we grant Andy his odd position that tax-credit programs don’t count.  We still have 13 voucher programs in 10 states serving about 50,000 students.  And the two new programs adopted last year were both voucher programs.  Wish as he might, Andy still can’t show that vouchers have stalled politically.

Second, Andy rightly says, “Reasonable people can review the cumulative literature about choice plans and disagree on how substantively significant or transformative these effects are (or could be at scale) and what that means for vouchers as a policy. ”  While reasonable folks could disagree about the magnitude of the effect of expanded school choice on public school performance, no reasonable person could disagree with the observation that the research literature supports at least some positive impact.  Given how hard it is to find any policy intervention that raises student achievement, consistently finding a positive impact from the systemic effect of vouchers should be treated as a big deal.  It isn’t to Andy. 

Third, Andy concedes that the more frightening prospect of vouchers helped spread charters, at least in the early stages of the charter movement.  But now that charters have reached critical mass, they may well do just fine without the viable threat of new and expanded voucher programs.  Folks who are really sincere about charters shouldn’t get so comfortable.  Just look at the unionization of the KIPP charter in NY or the constant effort to regulate charters to death in many states.  Dropping vouchers from your arsenal would be like confronting a resurgent Russia after dismantling all of your nuclear weapons.  You may think your conventional forces are up to the task, but ask the Poles how they would feel about it.

I’ve never understood why people would support charters but oppose vouchers.  The theory that expanded choice is good for the participating student and helps spur improvement in traditional public schools is required for both reforms.  Yes, charters are more easily subject to regulation than private schools receiving vouchers, but healthy charter programs require light regulation and states have not been shy about applying similar light regulation to voucher programs. 

The only reason I can imagine that folks would support charters but oppose vouchers is for political gain since the theory and evidence for both are essentially the same.  And I understand why politicians invent these false distinctions to prove their moderation and good sense by opposing the one they artificially dub as radical.  But we aren’t politicians.  We don’t have to lie or invent false distinctions to please constituencies.  Universities, think tanks, and the blogosphere should be refuges for reasoned inquiry and dispute, not rhetoric for political advantage.  As it says on the great seal — Veritas.

UPDATE — Andy’s a nice guy.  I tried to make my post as hyperbolic as possible and he responds kindly and reasonably.  Damn, he’s good.

UPDATE TO UPDATE — Just to be clear, I still think Andy is just plain wrong.  The fig leaf that Andy uses to be pro-charter while anti-voucher is the concern that vouchers sever “the connection between avenues of democratic input into schooling decisions and those decisions.   In other words, for some people the issue isn’t choice, rather it’s accountability in a broad sense.”  The reality is that there is as much opportunity for democrat input in the  design and operation of voucher programs as charter schools or traditional public schools for that matter.  The public can place whatever regulations it deems necessary on voucher schools as a condition of receiving those funds, just as it does with charter and traditional public schools.  Of course, all of these systems would operate best with minimal regulation.  If regulation were the answer to school effectiveness our public schools would already be fantastic.


New DC Voucher Study

January 13, 2009

My colleague Pat Wolf released today a new study of the DC voucher program based on focus group interviews of families.  It found high levels of parental satisfaction with the program, even among families that returned to the public system.  People appreciated having the choices and felt more involved in their children’s education.

Of course, these satisfaction outcomes don’t usually move the debate very much.  Opponents of voucher programs tend not to be persuaded by parental reports of satisfaction because they doubt the judgment of parents.  That’s why they are skeptical about choice.  And supporters of vouchers view satisfaction outcomes as important, but they are already inclined to trust parental assessments.

But the report provides plenty of contextual information that is useful and interesting even if it is not decisive.  A new test score analysis of the DC voucher program is expected sometime this Spring.


Update on Fiscal Impact of Milwaukee Vouchers

December 15, 2008

(Guest Post by Robert Costrell)

Does the funding formula for the Milwaukee Parental Choice Program (MPCP) adversely affect Milwaukee taxpayers, even as it benefits taxpayers statewide?  The answer I gave in my recent Education Next article is yes, based on data through the 2007-08 school year.  Since publication, some confusion has arisen as to whether this result still holds for the current school year, as reported in the news and opinion columns of the Milwaukee Journal Sentinel (“Fairness is in the Eye of the Beholder,” by Alan J. Borsuk and “Taxpayers, Parents on the Same Side,” by Patrick McIlheran) as well as an early version of Greg Forster’s post, since corrected here.

So let me update my Ed Next figures to the 2008-09 school year (hereafter FY09, for fiscal year).  The answer, in short, is still yes:  the adverse impact of the MPCP formulas on Milwaukee taxpayers continues unabated, even as the statewide benefits grow.  This is true, despite some modest efforts over the last two years by the Wisconsin legislature to address the problem.

First, let’s consider the size of the pie — the net savings available from the voucher program.  These savings derive from the fact that voucher expenses are $6,607 per child, while state and local revenues for MPS are set by the revenue limit at $9,462 per child.   These public savings are partially offset by the voucher expenditures on students who would have attended private schools anyway.  My best estimate is that these students comprise 10 percent of MPCP’s 19,500 students (see my full report for an explanation of this estimate, as well as my evaluation of different estimates).  Under this assumption, the net savings available to taxpayers totaled $37.2 million in FY09, up from $31.9 million in FY08 (the figure given in my Ed Next article).

The problem lies in the distribution of these benefits.  The savings accrue entirely to property taxpayers outside of Milwaukee and to Wisconsin state taxpayers:  Milwaukee property taxpayers do not share in these savings. 

The fiscal impact of MPCP on Milwaukee property taxes is driven by the fact that 45% of voucher expenditures are deducted from MPS aid, even though MPS receives no aid for these students.   As I explain in Ed Next, it certainly made sense to remove MPCP students from MPS enrollment counts when the system was reformed in 2000, but it made no sense to continue to deduct any of the voucher expenses from their remaining aid.  Milwaukee is allowed to raise its property taxes to recoup this deduction (the “choice levy”), and has to do so if it wants to maintain MPS’ per pupil revenues at the level specified by the revenue limit formula.   That is the essence of the “funding flaw.”

This is partially offset by 2 things.  First is that the removal of MPCP students from MPS enrollment counts saves the state aid money and some of that is passed on in statewide property tax relief; Milwaukee receives a small share of this.  The other offset, which began last year, is “high poverty aid,” an ad hoc appropriation to alleviate a portion of Milwaukee’s choice levy.

For FY09, these 3 pieces are $58.0 million (45% of voucher expenditures) minus $3.4 million (my estimate of Milwaukee’s share of statewide property tax relief) minus $9.9 million (“high poverty aid”) equals $44.7 million.  This is the adverse impact on Milwaukee property taxpayers of the voucher funding mechanism. 

It is worth emphasizing that this impact is on property taxpayers, not Milwaukee Public Schools.  Per pupil revenues available to MPS are unaffected by the voucher program, so long as Milwaukee fully utilizes the tax capacity granted to it under the MPCP formulas.   Milwaukee did utilize its tax capacity in FY09 (as it has done in all other recent years, with one exception noted below).

The picture is different for the rest of the state.  For FY09, I estimate the net benefit to property taxpayers outside of Milwaukee at $52.0 million, and the net benefit to state taxpayers at $30.0 million.   (The assumptions underlying these calculations and the basis for them are laid out in my Ed Next piece and my longer report for the School Choice Demonstration Project, along with details of the calculations.)

Taken all together, the net benefit to Wisconsin and Milwaukee taxpayers from the voucher program is $52.0 million (benefit to property taxpayers outside Milwaukee) plus $30.0 million (benefit to state taxpayers) minus $44.7 million (adverse effect on Milwaukee taxpayers) equals $37.2 million.   This is the net savings figure given above.

The pattern of winners and losers is depicted below, in the update of my Ed Next Figure 4.  The loss to Milwaukee property taxpayers is depicted by the blue bars in negative territory; the gains to other property taxpayers and state taxpayers are depicted by the maroon and tan bars in positive territory.

What was the impact of the “high poverty aid” program, enacted last year to alleviate the “funding flaw?”  As the diagram indicates, because of this additional aid, the adverse impact on Milwaukee property taxpayers for FY09 is no worse than in FY07, which is to say it did not grow as it would have without the aid. 

In addition, last year Milwaukee chose, for the first and only time in recent years, not to tax all the way up to the limit allowed by law.  There was $15.1 million of unused tax capacity.   Consequently, the diagram’s blue bar depicting the adverse impact on Milwaukee property taxpayers is shorter than it would otherwise have been for FY08.   This means that MPS received less than the per pupil revenue limit.   The figure attained was $8,978 instead of the revenue limit of $9,141.  The per pupil revenue still exceeded the FY07 figure, but did not increase as much as state law allowed.  In other words, this $15.1 million represents the shortfall for MPS, relative to the per pupil revenue limit.  This is depicted in the figure by the green bar for FY08, in negative territory. 

This year, Milwaukee has resumed its past practice of taxing up to the revenue limit, so the green bar disappears and the blue bar is no longer truncated:  there is no adverse impact on MPS, as the property taxpayers of Milwaukee make good on the full amount of the choice levy.

To summarize: 

(1)  Net savings from the Milwaukee voucher program continues to grow along with MPCP enrollments, and the widening gap between the voucher and the MPS revenue limit.   I estimate the net fiscal benefit at $37.2 million for FY09, up from $31.9 million for FY08.

(2) Milwaukee property taxpayers do not share in these benefits.  I estimate the adverse impact for FY09 to be $44.7 million.   The “high poverty aid” enacted in FY08 has kept the adverse impact from growing beyond its FY07 level, but has not materially reduced it either. 

The “funding flaw” persists.  As I stated in the conclusion of my Ed Next piece, “It remains to be seen whether, as the program grows, this flaw will undermine it or instead lead legislators to complete the reforms … so the benefits can be shared by all.

bob-3

 (Note 1:  the bars depicted for FY08 are revised from those published in Ed Next.  There I assumed Milwaukee taxed up to the revenue limit, as it had for preceding years.   This one-year departure from past practice came to my attention when the article was in press, too late to amend Figure 4.)

(Note 2: Alan Borsuk’s article, “Fairness is in the Eye of the Beholder,” includes a short summary of my Ed Next article, which states that I conclude “MPS is losing money […] on a per-pupil basis.”  My article actually states, “To avoid this result [MPS revenue loss on a per-pupil basis], MPS is still allowed to offset the [voucher] deduction by raising property taxes and it has chosen to do so.”  As the diagram above shows, the loss is for Milwaukee property taxpayers, not MPS, except for FY08, when Milwaukee chose not to offset the entire choice levy.)


Palin Backs Special Ed Vouchers

October 24, 2008

In a speech in Pittsburgh today, Governor Palin endorsed the idea of special ed vouchers saying, “In a McCain-Palin administration, we will put the educational choices for special needs children in the right hands — their parents’. Under reforms that I will lead as vice president, the parents and caretakers of children with physical or mental disabilities will be able to send that boy or girl to the school of their choice — public or private.

Under our reforms, federal funding for every special needs child will follow that child. Some states have begun to apply this principle already, as in Florida’s McKay Scholarship program. That program allows for choices and a quality of education that should be available to parents in every state, for every child with special needs.”


Did Milton Friedman Support School Choice Tax Credits?

October 15, 2008

(Guest post by Greg Forster)

Did Milton Friedman support school choice programs where the financing runs through the tax code rather than the treasury? He always made it clear that if he had a choice between them, he preferred vouchers (funded through the treasury) over either of the two alternatives forms of school choice that use the tax code (direct tax credits for families to offest thier tuition costs or scholarships distributed by charitable organizations and funded by donations that make the donor eligible for a tax credit). But that doesn’t mean he didn’t support the tax-code alternatives or didn’t consider them to be “true” school choice programs.

I bring this up because Robert Enlow of the Friedman Foundation has dug up a letter that Milton wrote in support of Florida’s tax-credit scholarship program. The letter was written on May 17, 2005 and was addressed to a Florida corporate leader who was considering whether or not to support the program.

Milton wrote:

I agree with you completely that the tax code should be used solely to raise revenue to fund necessary government spending and not to create social policy. Unfortunately, in schooling, the tax code is already being used to create social policy, by devoting tax funds to maintaining a socialist education system. If the state decides to subsidize the schooling of children, the straightforward way is to provide a voucher to each parent and let the parent choose the school that he believes is best for his or her children. Let the private market provide the schools. If the state wants to set up schools, let them charge tuition and compete with private schools on a level playing field.

Unfortunately, for reasons we are both well aware of, ranging from unions to school administrators and religious concerns, that ideal solution is not feasible.** Where it has been feasible to any significant extent, as in Milwaukee and the Florida Opportunity Scholarship Program, it has worked well. But again and again, as currently in Florida, an inferior tax credit program seems the only political option. Tax credits are an indirect, and I believe less efficient, way to do what vouchers do more directly. But they do promote the basic objective, of expanding parental choice and thereby introducing more competition into the educational industry. As a result, I have reluctantly supported tax credit programs in a number of states.

Let me just repeat, the tax system is being used for social purposes with both vouchers and tax credits.

**I think we may take it as given that he means “not currently feasible in Florida.” Any attmept to attribute to Milton Friedman the view that vouchers were “not feasible” in general would be absurd in light of his continuing active support for voucher efforts right up to the end of his life.

Milton was always completely open about his opinions, even to a fault. If he was thinking about whether to change his mind about something, but wasn’t yet sure whether to change his position, he would say so – and this would sometimes drive people to claim he had in fact changed his position.

We see that openness in this letter. He admits that his support for tax credits is only as a second-best option and even says the he supports them “reluctantly.” But if you knew Milton even a little bit – and I was only privileged to know him a little bit before he died – you know that he wouldn’t say he supported them at all unless he really supported them. The emphasis here is on the support, not the reluctance; the reluctance only comes in because (as the opening sentence makes clear) he’s addressing an audience that shares his concerns about tax credits.