The Rush to Judgment

June 30, 2009

David Figlio’s latest report on Florida’s Corporate Tax Credit (CTC) Scholarship program was released yesterday.  I can’t find the report online but Ron Matus of the St. Pete Times sent it to me and you can read about it in his article

(UPDATE:  Here’s a link to the study.)

I agree with almost everything said in the article.  I even agree with Mark Pudlow, the spokesperson for the teacher union when he said: “There is no quick fix for struggling students.” 

The problem is that the standard for success when it comes to school choice is that it has to produce a quick fix or critics deem it a failure and declare: “we really ought to reconsider why we’re doing it.”  No one demands that every other education policy produce huge gains in a single year or they should be “reconsidered.”  Yes, promoters of policies may make unrealistic promises to get them adopted, but the standard for success should be long-term progress, not promises made by politicians.

So let’s slow the rush to judgment and review what we really know about the CTC program.  First, Figlio finds that 92.7% of all CTC students in private schools provided a usable standardized test to the evaluation.  This shows widespread compliance with the legal requirements for those students to be tested to satisfy political concerns for accountability.

Second, Figlio finds that the CTC program has largely targeted students who are significantly more disadvantaged than students remaining in Florida’s public schools — even significantly more disadvantaged than public school students receiving subsidized or free lunch.  So, concerns that the program would cream off the best students appear unfounded.

Third, and most importantly, Figlio’s report does not make any claims about whether students benefited academically from participating in the CTC program.  He simply provides descriptive information on the academic achievement of CTC students as well as subsidized lunch students in Florida public schools.  But we know that CTC students are even more disadvantaged than those public school students and Figlio makes no attempt in this report to control fully for those disadvantages.

Figlio makes these points explicitly and repeatedly in the report: “it is important to recognize that they are not causal estimates of the effect of program participation on student outcomes. Causal comparisons require more complete modeling of the selection decisions into the scholarship program and fuller data from a baseline than is afforded using the 2006-07 school year test score collection. More compelling causal estimates of program participation will be possible following the collection of the 2008-09 school year’s test score data. The comparisons in this subsection should be interpreted as purely descriptive in nature.”

Unfortunately, most people never pay attention to these warnings and rush ahead as if descriptive information is causal.  Folks wrongly conclude that if CTC students make year to year test score gains that are about the same as subsidized lunch public school students, then they must not be benefiting from the program.  Nothing in Figlio’s report supports that conclusion. 

To know whether CTC students are benefiting we would have to know how they would be doing had they remained in public schools.  The best way to judge that is with a random-assignment study where students admitted to the CTC program by lottery are compared with students who lose the lottery and remain in public schools.  Unfortunately, that research design is not possible because there was no lottery.  The next best thing would be to use a research design that approximated random-assignment (like a regression discontinuity) or a rigorous quasi-experimental design that controlled for all observed differences between the two groups.  But Figlio didn’t do that in this report.  He just provided descriptive statistics while promising a more rigorous research design next year.

Of course, we might wonder why Figlio bothered reporting this descriptive information without a more rigorous analysis.  I suspect that he was required to produce a report each year by the legislature, so he complied even though he didn’t have the information he needed for a causal analysis. 

And the descriptive information is useful.  It suggests that choice was no miracle cure since the raw differences between CTC and public students in academic progress were not huge.  Again, miracle cure is the wrong standard for judging a program’s success. 

The CTC program may well have attracted students who had been on a downward trajectory before they switched to a private school.  And the CTC program may well helped those students level-off and may, over time, enable them to make significantly greater progress than they would have made had they remained in public schools.  This is what we’ve seen from rigorous evaluations of other choice programs, including the most recent evaluation of the DC voucher program.  But these things require careful research designs and time to show themselves.  Let’s give David Figlio more time to use a better research design so that we can actually say something about the academic effects of the CTC program.

(edited for typos)


Reaction To SC Decision on Special Ed

June 29, 2009

Reactions are beginning to pour in on the Forest Grove School District v. T.A. Supreme Court decision, which Greg and I wrote about last week.  Predictably and unfortunately, those reactions are informed by concerns for the financial burdens of the decision that are lacking in both facts and perspective. 

Even the dissenting opinion, written by Souter and joined by Scalia and Thomas, made a point of worrying about the costs:

The majority’s suggestion overlooks the terms of the IDEA process, the substantial procedures protecting a child’s substantive rights under the IDEA, and the significant costs of its rule. To start with the costs, special education can be immensely expensive, amounting to tens of billions of dollars annually and as much as 20% of public schools’ general operating budgets. See Brief for Council of the Great City Schools as Amicus Curiae 22–23. The more private placement there is, the higher the special education bill, a fact that lends urgency to the IDEA’s mandate of a collaborative process in which an IEP is “developed jointly by a school official qualified in special education, the child’s teacher, the parents or guardian, and, where appropriate,the child.”Burlington, supra, at 368.

Just how much private placement is there?  How much does it really cost?  How big is this relative to total enrollment and expenditure in public schools?

As of 2007 there were 67,729 disabled students in private school at public expense who were there at the initiative of their parents.  That is 1.1% of the 5,978,081 students in special education and 0.14% of the 49,610,000 students in public education.  Theses percentages were not significantly different before 1997 when Congress amended the special education law in a way that the dissent believed would constrain burdensome private placements.  And Marcus Winters and I estimated that the total financial cost of private placement is less than a billion dollars and amounts to less than one-quarter of one percent of total public school spending

I understand that a billion dollars is a lot of money, but in a public education system spending more than $500 billion it is almost rounding error.  Souter, Scalia, and Thomas violate the Denominator Law, where it is required that all claims of “big” problems have to be put in perspective by including a denominator to show how large the problem really is given the full context.  As officers of the Court they should know that ignorance of the law is no excuse!

It’s also strange that Scalia and Thomas would join in a dissent that is based at least partially on concerns for the financial implications of their decisions.  I thought Scalia and Thomas believed in finding the original intent of the law.  A court ruling based on (false) fears of financial burdens of the law sounds like policymaking from the bench.

However, Debra Saunders, in a column on the decision at the San Francisco Chronicle, seems confused about what constitutes policymaking from the bench.  She writes: “the court arguably engages in policy-making when it tells districts how they must spend valuable education dollars.”  There is a federal law, IDEA, that tells schools how they must spend their money (along with the money they receive from the feds).  It says that all disabled students are entitled to a free appropriate public education.  It isn’t policymaking from the bench to say that students unreasonably denied appropriate services shouldn’t have to wait 2-3 years for the Courts to order the schools to provide those services.  The Courts can’t also provide a time machine, so we have to have a mechanism that handles what happens to kids while their parents fight with schools in the courts. 

In Forest Grove the Supreme Court said that parents should be able to take the risk of placing their children in private school and seeking reimbursement.  If the courts agree that the schools unreasonably denied services, then they can get reimbursed for their costs while they were waiting.  If the public schools were reasonable, then the parents are out the money. 

I agree with Debra Saunders that the facts in this specific case make it hard to understand how a lower court found that the public schools behaved unreasonably.  But the law isn’t about one set of facts; it applies to all instances.  If we pretend that the lower courts find that public schools denied a special education classification unreasonably, then obviously students would be denied their rights under IDEA if they had to wait 2-3 years for those services.

Debra goes on to violate the Denominator Law, writing: “It’s one of those nice people things. The government has expanded the notion of disability to the point of absurdity. But nice people refuse to look at the impending drain on public school budgets, or how one child’s boarding school tuition can mean that much less funding for all the other students’ educational needs.”  This was especially frustrating because I pleaded with her to report claims of financial burdens in context.  Besides quoting me, she chose to ignore my point and repeat her claims of burden with no basis to support it.

My colleague, Walter Olson, also has a post on Forest Groveon his blog, Overlawyerd.com.  While I disagree with Wally on this issue, I am sympathetic with his concerns.  Specifically, he notes that private placement is a remedy much more available to wealthy families than poor ones.  And he doubts the justice of disabled students having federally protected rights to an appropriate education while no one else does. 

I agree that rich kids have better access to this remedy than poor kids. That’s why I favor vouchers for special education, both to democratize this remedy and to better control costs. Vouchers control costs because the voucher is worth no more than would have been spent in public schools or private school tuition, whichever is less. Special ed vouchers also discourage over-identification of disabilities because schools would risk losing students when they classify students as disabled.

And I also agree that it is unclear why non-disabled students should have to wait in schools that fail to serve them appropriately while disabled students are entitled to find an appropriate education. But the solution is not to strip disabled students of that right.  The solution is to extend it to all students. Give vouchers to all students worth the amount that would be spent on them in public school (the amount would vary based on the cost of educating different kinds of students). If any student is unable to find what his family believes is an appropriate education, give them the resources to find it somewhere else.


Pass the Popcorn: The Red Violin

June 26, 2009

Title Screen

(Guest post by Greg Forster)

I don’t get to see many movies anymore – at least, not many by my standards. And when you can only see maybe four to six movies a year in the theater, you’re going to end up seeing the obvious ones – Batman, Star Trek, whatever Pixar does this year, etc.

But there was a time when we used to see a lot of movies. And that meant we had the luxury of picking through the enormous pile of garbage that is the arthouse and finding the few movies that make the arthouse worth going to. So I’m going to start using our Pass the Popcorn feature to show off some of our “finds,” in the hope that they won’t fade too far into obscurity.

The Red Violin

The Red Violin stars Samuel L. Jackson as the most badass professional musical instrument appraiser you will ever see depicted on screen. I’m serious, don’t mess with this guy. At one point a clerk fails to deliver an important package to him promptly, and he tears into the clerk so hard I thought he was about to start reciting passages from Ezekiel.

Jackson & violins

“And you will know my name is the LORD when I lay my violin upon thee!”

OK, now that I’ve sold Matt, here’s what the rest of you need to know.

The movie tells the story of a violin that was created by a Renaissance Italian craftsman as his greatest masterwork. Over the course of the movie, the action shifts back and forth between three storylines.

Making the violin 2

In 17th century Italy, we see the craftsman’s initial aspiration – to create an instrument worthy of his love for his unborn child, due to arrive any day, to whom he intends to give the violin as a gift. But in a surprise twist at the end, the violin comes to have a different, but equally profound, significance for him.

Jackson peeking

In our own time, the violin is going up for auction. Everyone else thinks the violin is nothing special, but badass appraiser Samuel L. Jackson suspects otherwise – that it may be the long-lost “red violin” made by that famous Italian craftsman. Once the truth becomes known, everyone wants the violin – but nobody other than Jackson wants it for the right reason, leading to a surprise twist at the end in which Jackson triumphs over the greed and pride of his adversaries.

Gypsy

In between, we see what happened to the violin as it travelled around the world between its creation and its eventual rediscovery. Each vignette in this storyline illustrates the characteristic ways in which different civilizations have responded to the mystery of great art.

Kaspar dressed up

18th century Vienna is so obscessed with technical skill that art is reduced to mere performance – the ability to play very complex pieces very fast is valued above beauty. Following this path ultimately leads to the reduction of art into the novelty act of child prodigies – because the younger you are, the more amazing your skill is, and that’s all that counts. Form obliterates matter, and since form can’t exist without matter, it obliterates itself, ending in tragedy.

Victorian couple playing violin

In Victorian England, by contrast, “art” is put up on a pedastal and idolized. “Creativity” is fetishized to the point where mere novelty and thrill displace beauty, just as mere technique had displaced it a century earlier. Craftsmanship goes out the window in favor of irresponsible artistic self-indulgence. Matter obliterates form, and since matter can’t exist without form, it obliterates itself, ending in tragedy.

In China with violin

In 20th century China, during the cultural revolution, just to have an interest in art as such is a life-threatening proposition. The state holds that art, like everything else, exists only for political ends, so the desire to make art for the sake of beauty is an act of treason against the people. Far worse to make art that has western origins, given the regime’s awkward attempt to fuse its totalitarian ideology with crude appeals to nationalistic Chinese chauvanism. Yet in the end, the totalitarians prove as incapable of eradicating the desire for beauty as they are at eradicating any of the other fundamental desires of human nature.

Jackson & restorationist

What do we want beauty for? All answers other than “we want it because it’s beautiful” ultimately prove futile. The goodness of beauty, like the goodness of knowledge or the goodness of virtue, is categorical. Make it instrumental towards some other good and you destroy it.

What, then, should we do with beauty when we find it? To that, the movie has a definite answer, and I think it’s exactly the right one. But to find out what it is, you’ll have to see those two surprise twist endings I mentioned earlier – and I’m not spoiling them for you. Go rent it and find out.


GI: Hold the Phone on Higher Taxes

June 26, 2009

(Guest Post by Matthew Ladner)

Arizona has one of the worst budget deficit problems in the country (live by the property bubble…) written about here and here. New governor Jan Brewer has called for a “temporary” sales tax increase to prevent the sort of belt-tightening of the Arizona budget already having been done by most Arizona families.

We at the Goldwater Institute strongly disagree, and put out the following humorous video today to explain why:


GreenDot UFT Bargain

June 26, 2009

(Guest Post by Matthew Ladner)

In Shakespeare’s Richard III there is a scene where Richard makes an offer to Elizabeth, widow of the former King Edward. Allow me to marry your daughter and you will continue to live a life of royal luxury. If you refuse, you die.

Elizabeth asks Richard “Shall I be tempted by the Devil then?”

Richard replies “Yes, if the Devil tempt you to do good.

Needless to say, the Devil is not in the business of tempting anyone into doing good. People with far more expertise than I possess will be required to evaluate GreenDot’s own deal with the proverbial devil in the form of a collective bargaining agreement with the United Federation of Teachers. Needless to say, there is a potential for a win-win on both sides here, but as always, the devil is in the details. Here is one devilish detail that I certainly would have refused:

Article 10

Student-Teacher Ratio, Class Size and Teacher Load

 

The School shall maintain a school-wide staffing ratio of no more than twenty (20) students to one (1) full-time classroom teacher. Unless otherwise approved by the Calendar Committee and ratified by a majority of Bargaining Unit Members and the Board, an individual class may not exceed thirty (30) students. Moreover, there must be a total of no more than one hundred thirty (130) students in all of a teacher’s classes excluding advisory.

Translated: “There shall be an arbitrary and unsupportable cap on the total productivity of any individual teacher imposed on these schools. This cap will have the added benefit to the union of limiting the possibilities for differentiated pay, or attracting and keeping talented teachers in the classroom. Oh and by the way this will happily hamstring any hope of bringing Green Dot to a widespread scale.  Thanks for playing sucker– this mole has now been successfully whacked.  HURTCHA!!!!”

These standards may very well fit comfortably into GreenDot’s current practices. GreenDot’s current practices, however, are for a niche player and are not at scale. These terms of course could be renegotiated in the future, but good luck changing them once they are in place.

Teacher unions are rational actors. Their rational incentive is to maximize employment for dues paying members. If this happens to mean that we wind up throwing legions of all-to-often-ill-suited-and-ill-prepared-bottom-of-the barrell-students into the teaching profession at great expense and to castastrophic effect, them’s the breaks.  Things would be better if we spent (even more) money!

This agreement is almost certainly an improvement over what NYC’s rubber room contract, but that does not mean it will prove to be worthwhile.


Freedom from Responsibility Preview part Deux

June 25, 2009

 (Guest Post by Matthew Ladner)

In the end more than they wanted freedom, they wanted security. When the Athenians finally wanted not to give to society but for society to give to them, when the freedom they wished was freedom from responsibility, then Athens ceased to be free.

Edward Gibbon

Last week we had our first sneak peak at Freedom from Responsibility.

Today, more details about the results. The Goldwater Institute randomly drew 10 questions from the United States citzenship exam item bank. We hired a survey firm to interview a sample of both Arizona public and private school high school students.

The questions for neither the citizenship test nor our survey were multiple choice. When you are asked “Who was the first President?” you must answer “Washington” in order to receive credit. Applicants for citizenship must get six out of the ten questions correct to pass. A recent trial of a slightly reformatted exam found that 92.4% of citizenship applicants passed the test on the first try.

Charles N. Quigley, writing for the Progressive Policy Institute, explained the critical nature of civic knowledge:

From this nation’s earliest days, leaders such as Thomas Jefferson, James Madison, and John Adams recognized that even the well-designed institutions are not sufficient to maintain a free society. Ultimately, a vibrant democracy must rely on the knowledge, skill, and virtues of its citizens and their elected officials. Education that imparts that knowledge and skill and fosters those virtues is essential to the preservation andimprovement of American constitutional democracy and civic life.

Paul D. Houston, the executive director of the American Association of School Administrators, also put the issue in focus:

If you look back in history, you will find the core mission of public education in America was to create places of civic virtue for our children and for our society. As education undergoes the rigors of re-examination and the need for reinvention, it is crucial to remember that the key role of public schools is to preserve democracy and, that as battered as we might be, our mission is central to the future of this country.

Here are the 10 questions randomly selected, and their answers:

1.What is the supreme law of the land?Answer: The Constitution

2. What do we call the first 10 amendments to the Constitution?

Answer: the Bill of Rights

3. What are the two parts of the U.S. Congress?

Answer: Senate and House

4. How many Justices are on the Supreme Court?

Answer: Nine

5. Who wrote the Declaration of Independence?

Answer: Jefferson

6. What ocean is on the East Coast of the United States?

Answer: Atlantic

7. What are the two major political parties in the United States?

Answer: Democratic and Republican

8. We elect a U.S. Senator for how many years?Answer: Six

9 . Who was the first President of the United States?Answer: Washington

10. Who is in charge of the Executive Branch?Answer: The President

 
A majority of Arizona public high school students got only one of these questions correct, with 58% correctly identifying the Atlantic Ocean as being off the east coast of the United States, with 42% unable to do so. It was all downhill from there. 29.5% of students identified the Constitution as the supreme law of the land, 25% of students identified the Bill or Rights as the first 10 amendments to the Constitution (12% said they were called “The Constitution” and 16% “The Declaration of Independence.”)

Twenty three percent of Arizona public high schoolers identified the House and Senate as the chambers of Congress. Nine point four percent that the Supreme Court has nine justices. Only 25% of students correctly identified Thomas Jefferson as the author of the Declaration of Independence. An almost majority of 49.6 percent identified the two major political parties, only 14.5% answered that Senators are elected for six year terms. Finally, only 26.5% of students correctly identified George Washington was the first President. Other guesses included John F. Kennedy, Ronald Reagan, George Bush and Barack Obama.

Only 26% could identify the President as being in charge of the Executive Branch. All in all, only 3.5% of public school students passed the test by getting six or more items correct. That’s 40 students out of a sample of 1,134 district students.

There were no major differences in performance based on grade (Seniors did approximately as poorly as Freshmen) nor by ethnicity. Profound ignorance is quite equally distributed in large measure across students in the public school system.

Two obvious questions to ask: is it fair to give this test? In order to answer, I examined the Arizona state standards for 8th grade social studies, which all or nearly all of these students will have taken. These standards are included as an Appendix in the study. What they show is that students are supposed to have learned about John Locke, the Mayflower Compact, the Articles of Confederation, the Constitution, Checks and Balances, Seperation of Powers, etc. etc. etc.

Everything they ought to have needed, in other words, to have passed this test. If, that is, they had actually learned any of that material in practice, which they obviously did not.

Second, I gave the test to my own 1st and 2nd grade sons. They both got 3 answers correct. We’ll be working on that. In so doing, they outscored about 40% of the Arizona high school sample, and tied or exceeded about 60 percent.

Charter school kids performed far better but still terribly- with a passing rate about twice as high as the public school kids. Private school students passed at a rate four times higher, which ultimately is both much better and still pathetic.

I had a very difficult time writing a conclusion to this study. More on that for the next post, but you tell me: if you were an Arizona lawmaker what would you do about this?


The Supreme Court’s Special-Ed Vouchers

June 24, 2009

(Guest post by Greg Forster)

Following up on Jay’s observations on the Forest Grove School District v. T.A. decision – not being a lawyer, I’m not going to dwell on this point. But it seems to have passed under the radar and I want to bring it out into the open so that others, who may be better qualified than I, may give it the debate it deserves.

For years, whenever I’ve explained that if the IEP procedure (over which schools have predominant control) does not deliver to students their IDEA rights, the only recourse parents have is a lawsuit, people have looked at me like I was nuts. They just stare with this dumbfounded look on their faces. The system can’t really be that crazy, can it?

The Supreme Court seems to have the same dumbfounded look on its face:

Having mandated that participating States provide a FAPE [free and appropriate public eduction] for every student, Congress could not have intended to require parents to either accept an inadequate public-school education pending adjudication of their claim or bear the cost of a private education if the court ultimately determined that the private placement was proper.

Now, as a psychological observation (which it is formally presented as being) this is farcical. Why couldn’t that have been Congress’s intention? Why assume Congress had pure and unsullied motivations when it created IDEA, rather than allowing the possibility – just as a possibility – that Congress knew darn well what it was doing, and decided to screw over children with special needs in order to serve its real constituents?

But, of course, while this is formally a psychological observation, it is serving the function of legal reasoning. What the court is really saying is that it would be fundamentally unjust – it would be a disgusting inversion of the fundamental function of the law – for Congress to deliberately legislate a right and then create a process desigend in a way that effectively denies relief to people if they are denied that right. So the court is entitled to assume that this was not Congress’s intention.

This takes us very rapidly into deep philosophical waters. Should the court interpret the law on the assumption that Congress does not intend to use the law as a cover-up to screw innocent people?

This, it seems to me, is one of the problems that motivates Scalia’s distaste for framing legal interpretation in terms of “original intent” rather than what he calls “original public meaning.” And I’ve always thought, without being an expert in the field, that “original public meaning” was a much more plausible standard for legal interpretation.

Does that explain Scalia’s dissent in this case?

Discuss among yourselves.


Global Warming Evidence vs. Dogma in Australia

June 24, 2009

(Guest Post by Matthew Ladner)

Robert Tracinski and Tom Minchin have written an interesting article about the role of evidence in the global warming debate in Australia. By their account, the Aussie’s have looked over an economic cliff and decided not to jump.


Choice Victory in Special Ed SC Case

June 22, 2009
The Supreme Court ruled 6-3 today in Forest Grove School District v. T.A. that disabled students that the public schools unreasonably failed to identify as disabled don’t have to wait to seek placement in a private school and reimbursement for those costs. 
This ruling seems to give families with disabled children unilateral access to vouchers for private school if they can later prove that the public schools failed to provide adequate services or unreasonably failed to identify the disability.  The families assume the financial risk if they act unilaterally, but they can be fully reimbursed for their expenses if they are proven right.  The majority reasoned that delays were so long in adjudicating these disputes, that children would be denied their right to a free appropriate public education (FAPE) if they had to wait:
 
“Our decision rested in part on the fact that administrative and judicial reviewof a parent’s complaint often takes years. We concluded that, having mandated that participating States provide a FAPE for every student, Congress could not have intended to require parents to either accept an inadequate public-school education pending adjudication of their claim or bear the cost of a private education if the court ultimately determined that the private placement was proper under the Act.” (see p. 7 )
 
Now all we need to do is to grant to all children what we have given to disabled children.  Why should any child, disabled or not, be made to wait for an appropriate education?  Why can’t all parents seek a unilateral private placement and sue to be reimbursed if they can demonstrate that the public schools were failing to provide an appropriate education? 
 
Even better, why should we make parents prove to a court that the education in the public schools was not appropriate?  Why not let the parents be the judge of the appropriateness of the education being offered?
Updated:  I just noticed that Matt made a similar argument a while back.

Teacher Unions = The Tobacco Institute

June 22, 2009

I want to add a little to my post the other day about how the teacher unions lie and so should not be treated as credible players in policy discussions. 

The unions don’t have to lie.  The NEA didn’t have to falsely claim that the DC voucher program “yielded no evidence of positive impact on student achievement.”  They could have said something about the effects not being large or that there are other harms to vouchers that are greater than the benefits.  A pattern of lying fundamentally undermines the credibility of the teacher unions so that they will increasingly be shunned in policy discussions and lose in policy debates.

You may think that the unions are so powerful that they can just lie and get away with it, but you’d be wrong.  Remember the fate of the tobacco industry.  They created the Tobacco Institute, which produced “research” claiming to be unable to find links between smoking and cancer. 

The tobacco companies didn’t have to do this.  They could have just said that people should be free to choose whether they smoke or not regardless of health risks.  They didn’t have to lie about health effects, they could have just said that it was none of the public’s business whether people chose to smoke or not.

At the time it was conventional political wisdom that the Tobacco Institute could get away with lying because the tobacco lobby was so powerful and rich that they could do almost anything.  But eventually lying destroys one’s credibility in a way that no amount of money can restore.  And the teacher unions may suffer the same fate as the Tobacco Institute.  They may seem all-powerful right now, but over time it is hard to sustain dumb ideas, especially when lying.