Jay & Marcus in NR

October 2, 2009

NR cover (Jay & Marcus article)

(Guest post by Greg Forster)

In the new National Review, Jay and Marcus review the research on special education funding incentives, including the findings of their recent study on the impact of vouchers in Florida.

Financial incentives are particularly important in low-level disability categories like SLD, where a diagnosis is easily fudged. While you need pretty solid evidence to diagnose a child with a traumatic brain injury or other severe disabilities, schools have plenty of leeway on SLD. Some research suggests that public schools use low achievement alone to serve as an indicator of SLD. Studies dating back to the 1980s found that SLD students are indistinguishable from low-achieving regular-enrollment students, with one study estimating that over half the students identified as SLD in Colorado did not fit either federal or state definitions for SLD.

Digital subscribers go here; paper-only subscribers go here; non-subscribers go here.

Leave a Comment » | special ed, vouchers | Tagged: , , , , | Permalink
Posted by Greg Forster

Just About Everything is Endogenous

September 30, 2009

A common technique in analyses of education policies (and popularized in the book, Freakonomics) has suffered a setback recently.  The technique attempts to correct for endogeneity, which occurs when your dependent variable is causing one of your independent variables rather than simply the other way around.

It’s probably best to explain this with an example.  Let’s say you want to know how the number of police officers in a city affects the crime rate.  In this example the dependent variable is the crime rate and the independent variable is the number of police officers.  That is, you are trying to explain how the size of the police force causes crime rates to be high or low.

The trouble is that the causal arrow also goes in the other direction.  The crime rate affects the size of the police force because cities with a lot of crime may decide to hire a lot of police officers.  So, the number of police officers is endogenous to the crime rate.  

That endogeneity could produce some odd results if we didn’t do anything to correct it.  We might find that the number of police officers causes crime rates to be higher when it might really be the case that the size of the police force reduces crime but high crime rates cause larger police forces.

This kind of problem comes up quite often in econometric analyses in general and in particular in evaluations of education policies.  So, it was a great a thing that University of Chicago economist James Heckman developed a technique for unravelling these circular relationships and correcting for endogeneity bias.  Basically, the technique uses some exogenous variable to predict the independent variable without bias.

Again, it’s probably easiest to explain with an example.  If we can find something that predicts the number of police officers that has nothing to do with the crime rate, then we can come up with an unbiased estimated of the number of police officers.  We can then use that unbiased estimate of how many police officers there would be (independent of the crime rate) to predict the crime rate.  In theory the technique works great.  Heckman won the Nobel Prize in economics for developing it.

The tricky part is coming up with a truly exogenous instrument (something that predicts the independent variable but has no relationship with the dependent variable).  The only obviously exogenous instrument is chance itself.  An example of that kind of instrument can be found in analyses of the effect of using a voucher on the student achievement of students who actually attend a private school when the vouchers are awarded by lottery.  Those analyses use whether a student won the lottery or not to predict whether a student attended a private school and then used that unbiased estimate of whether a student attended a private school to predict the effect of private schooling on student achievement. 

Whether a student won the lottery is purely a matter of chance and so is completely unrelated to student achievement, but it is predictive of whether a student attends a private school.  It is a perfectly exogenous instrument.

The problem is that other than lotteries, it isn’t always clear that the instruments used are truly exogenous.  Even if we can’t think of how things may be related, they may well be.

A perfect example of this — and it is one that raises questions about how exogenous all instruments other than lotteries truly are — was recently described in the Wall Street Journal having to do with date of birth.  The date during the year when babies are born has long been thought to be essentially random and has been used as an exogenous instrument in a variety of important analyses, including a seminal paper in 1991 by Josh Angrsit and Alan Krueger on the effects of educational attainment on later life outcomes. 

Since states have compulsory education laws require that students stay in school until a certain age, babies born earlier in the year reach that age at a lower grade and can drop out having attained less education.  By comparing those born earlier in the year to those born later, which they believed should have nothing to do with later life outcomes, they were able to make claims about how staying in school longer affected income, etc…

But new work by Kasey Buckles and Daniel Hungerman at the University of Notre Dame suggests that the month and day of birth is not really exogenous to life outcomes.  As it turns out, babies born in January are more likely to be born to unwed, less educated, and low income mothers than babies born later in the year.  The difference is not huge, but it is significant.  And since this variable is not exogenous, perhaps some or all of the effect of attainment Angrist and Krueger observed is related to this relationship between date of birth and SES, not truly attributable to attainment.

And if birth order is not random when we all assumed it was, what other instruments in these analyses are also not truly exogenous but we just don’t know how yet?  It’s a potentially serious problem for these analyses.

6 Comments | methodology | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

Pass the Clicker — Filled with Glee

September 25, 2009

With the possible exception of Flashforward, which I haven’t had a chance to see yet but looks promising, Glee is the best new TV show of the season.  It’s a dark high school comedy about the struggles and triumphs of the glee club coach and its members. 

The best part about it is its unhindered departure from realism.  No glee club really sounds that good.  All of the characters are outrageous stereotypes.  No high school is filled with as much viciousness.  I especially love the coach of the “Cheerios” cheerleading team enforcing the Darwinian social hierarchy and Principal Figgins with his hand always on the calculator looking to save money by feeding the students prison food.

But in fully departing from realism the show probably better captures the reality of high school life than any of the sappy, gritty, “realistic” high school dramas, like Boston Public, Dangerous Minds, or Dead Poets Society.  Those are adult fantasies of what they would like high school to be — filled with heroic teachers battling the odds to save eager students . 

That’s not high school.  High school is often banal, outrageous, awkward, and pathetic.  Surrealism captures the experience so much better than realism.  The only other depiction of high school that I can think of that similarly captures the high school experience is the movie, Election.

Given that much of the attraction of the show is its outrageousness and novelty, I expect that the quality of the show will rapidly fade.  By season 2 we will have exhausted the one-dimensional characters and become jaded to the show’s novelty.  But enjoy the ride for now.

And yes, a show about a glee club is pretty “gay.”  The show tackles this issue head-on by featuring the tension between youthful anxiety about masculinity and youthful desire to express one’s creative self.  Just watch how the football team uses a dance to Beyonce’s All the Single Ladies to win the game:

And if you need more dancing to that song (and who doesn’t?), check out this video of All the Single Babies:

(edited to correct typo)

3 Comments | Pass the Clicker | Tagged: , , | Permalink
Posted by Jay P. Greene

More Charter Evidence

September 22, 2009

Diane Ravitch has declared that the Obama administration’s policy of expanding the number of charter schools has “no credible basis in research.”  This is just plain wrong.  And a new study coming out today from Stanford’s Caroline Hoxby demonstrates that she is even more wrong.

I’ve already noted that the highest quality studies — those that avoid bias from the self-selection of students into charter schools either with random-assignment or rigorous instrumental variable research designs — show significant academic benefits for students who attend charter schools instead of traditional public schools.  These studies examine the effect of charter schools in Massachusetts, Florida, Chicago, and New York City. 

And now add to that pile an updated study from Caroline Hoxby mentioned in today’s WSJ and NYT on New York City charter effects.  Students accepted by lottery into one of NYC’s charter schools in kindergarten and remained in a charter school through grade 8 closed the achievement gap with wealthy kids attending schools in Scarsdale entirely in math and two-thirds of the way in reading.

Critics are clinging to a study by Margaret Raymond at CREDO, which shows more mixed results.  While that study has the benefit of covering 15 states and DC, it can’t correct for the self-selection of students into charter schools like the highest quality studies linked above.  On average, students appear to be drawn to switching to charter schools because they are having trouble in their traditional public school.  Simply controlling for those students’ prior achievement and other observed demographic factors doesn’t quite correct for whatever negative factors may have caused students to switch to charters and that may continue to hinder their academic progress.  The CREDO study is as good as it can be given its approach, but I would have greater confidence in the consistent findings from several studies in different locations that do control for self-selection into charter schools.

8 Comments | charter schools | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

Over There (But Not Over Here)

September 21, 2009

Several years ago I was part of a delegation sent by the U.S. Department of Education to a conference in China on private education.  The U.S. Dept of Ed believed that encouraging the expansion of private education in China would help promote democracy.  Apparently, they thought private schools were good for democratic values over there, but not over here. 

I was reminded of that experience while reading a recent New York Times article about severe problems with education in South Africa.  The piece states:

Despite sharp increases in education spending since apartheid ended, South African children consistently score at or near rock bottom on international achievement tests, even measured against far poorer African countries. This bodes ill for South Africa’s ability to compete in a globalized economy, or to fill its yawning demand for skilled workers. And the wrenching achievement gap between black and white students persists.

Sound familiar?

And what does the NYT tell us is a central part of the problem:

The teachers’ union too often protected its members at the expense of pupils, critics say. “We have the highest level of teacher unionization in the world, but their focus is on rights, not responsibilities,” Mamphela Ramphele, former vice chancellor of the University of Cape Town, said in a recent speech.

I see.  Teacher unions over there = bad, while over here = good.  Sometimes you have to get people outside of their vested set of domestic interests to see how they really think the world works.

Leave a Comment » | teacher union | Tagged: , , , | Permalink
Posted by Jay P. Greene

The Echo Chamber of Public Input

September 17, 2009

The Fayetteville school board and district leaders fully supported a plan that was soundly rejected by the voters this week.  How did school officials so badly mis-read what voters wanted?  It’s especially puzzling how school officials could have seriously misjudged their constituents given the years of deliberations, countless hours of public meetings and charrettes, and even a commissioned opinion poll.

Unfortunately, these countless rituals of public input are exactly what misled school officials to support an unpopular plan.  They were misled because these rituals of public input are better indicators of the views of the self-selected, small minority of people with the most intense (and often the most extreme) preferences than they are indicators of what the electorate would want.  School officials mistook the opinions of this self-selected few as the voice of the people. 

School officials also hired consultants to lead these public conversations, but in doing so they were steering discussions in a pre-determined direction.  Bringing in education consultant Tony Wagner and requiring all school employees to read his book steered the plan toward a high school divided into small learning communities.  That idea didn’t come from the voters.  It came from certain school officials, was made the topic of discussion in schools and community events, and then was echoed back to school officials. 

Similarly, the design “charrettes” led by consultants from New Orleans were not truly open brain-storming sessions about a new high school.  If they were, how did several small break-out groups independently arrive at the same Trail of Tears design concept? 

There is nothing inherently wrong with holding public discussions on important decisions or with bringing in expert consultants to inform and direct those conversations.  The problem is in falsely believing that what results from those discussions is in fact the opinion of the community.  They are more like echo-chambers, repeating back the preferences that school officials had going into them.

But school officials saw the community discussions as a sign of general public support for their vision.  They even went so far as to describe the plan that was developed from these events as “The People’s Plan.”  And then when asked why voters should support the millage, the advocates and editorial writers told us that it was The People’s Plan and had come from us so we should support what the community had developed.

This People’s Plan campaign strategy almost felt like bullying.  If you weren’t among the tiny, minority of atypical people who could spend evening after evening in community discussions, you had lost your chance to have a say.  It was time for you to get in line and support what the involved people had already determined.

Perhaps for this reason opponents of the millage stayed generally quiet during the campaign.  Yes, there was a handful of active letter writers and a Facebook group with fewer than ten members, but there was no organized opposition, no “vote no” yard signs, and a string of elite (even if tepid)  community endorsements.  But in the privacy of the voting booth, people clearly felt free to open-up and clearly say no.  Once the result had been announced, opponents discovered that they weren’t so isolated, and Facebook pages began to light-up with people explaining their reasons for opposing the millage despite their commitment to education and their understanding of shortcomings of the existing facility. 

The solution is not to hold even more public input rituals to scale back the cost of the project but leave all other decisions in place.  Presumably, the $116 million price tag followed from all of the design and policy decisions that had preceded it.  If all of the design and policy goals could have been met for a lower cost, why wasn’t the initial millage for a lesser amount?

Instead, the solution is to stop the echo-chamber decision-making of meetings, charrettes, and consultants, and start with real leadership.  School officials should step-up and tell us what they think would be educationally desirable at a reasonable cost.  Of course, it is difficult for them to gauge what the community would consider a reasonable cost without public input, but the election result has given them better feedback than any town-hall discussion or charrette ever will.

Superintendent Vicki Thomas is particularly well-positioned to offer her vision of our educational future.  She bears no responsibility for the development of the failed millage plan and can start with a fresh slate.  We hired her to lead our schools and leadership is what we need.  She has enough information from voters and past public meetings to assess the community’s priorities.  Now she can give us a new plan and convince us that it is what she thinks is best, not what she thinks we told her to say.

3 Comments | Fayetteville, politics | Tagged: , , , | Permalink
Posted by Jay P. Greene

Have you “Experienced” The Riffs at Mid-Riffs?

September 15, 2009

Mid-Riffs, a blog started by a bunch of my friends, is off to a great start with several posts on the high school millage in Fayetteville, Arkansas.  Sometimes I agree with them and sometimes I don’t, but they are always fun to read.

The election is today, so be sure to check out their excellent information and analysis.  In particular, they have argued:

  • The high school is not falling apart. (In a 2006 statewide ranking of buildings needing repair, Fayetteville high school was ranked 988 out of 1,129 K-12 public school buildings, where 1 was most need of repair.)
  • There is no evidence buildings improve student outcomes.
  • The current facility has deficiencies, but they don’t necessitate complete demolition and reconstruction.
  • There is a case to be made for economic development, but any positive effects will be much diminished by the necessary tax increase.

    • But Mid-Riffs did make a case for why we might want to spend $116 million to tear-down the currently functional building for a brand new one — we like shiny new things.  We don’t need to buy diamond engagement rings, but people like to have them.  We don’t need a new building, but we might still want to have one.

    It’s not a very compelling argument, but it is no worse of a reason than your reason for buying that new Lexus.

    4 Comments | blogging | Tagged: , , , | Permalink
    Posted by Jay P. Greene

    Jay: Wake Up and Smell the Incentives

    September 14, 2009

    (Guest post by Greg Forster)

    Well, it seems to be op-ed day for friends of JPGB today. Below, Matt appreciates Robert Enlow as a man who has “the whole package” – and delivers it in today’s Indy Star. Meanwhile, over on NRO, Jay has a column on the perverse incentives that artificially drive up special ed diagnoses:

    Schools have discovered that they can get extra funding from state and federal ‎governments for small-group instruction to help lagging students catch up if they say that ‎the students are struggling because of a processing problem in their brains. School officials who admit that the students are lagging because of poor previous instruction or a difficult ‎home life, by contrast, are left to pay the costs of small-group instruction entirely out of ‎their own budget.

    If you’ve been reading JPGB, that part is all old hat to you by now. If not, this NRO piece is a good (though very brief) introduction to the topic.

    The NRO piece does make one point I hadn’t thought of before:

    In New Jersey, for example, 18 percent of all students are ‎classified as disabled, but in California the rate is only 10.5 percent. There is no medical ‎reason why students in New Jersey should be 71 percent more likely to be placed into ‎special education than students in California.

    Indeed.

    Leave a Comment » | special ed | Tagged: , , , | Permalink
    Posted by Greg Forster

    Mostly Harmless

    September 8, 2009

    Many electrons have already been spilled on Obama’s speech today to the nation’s school children.  When news first broke of the planned speech, alarms were raised by Michelle Malkin, Glenn Beck, and Neal McCluskey (among many others). 

    This was followed by a counter-backlash from the left as well as folks on the right, including the Wall Street Journal and Tunku Varadarajan at Forbes, who said that the initial reaction was “overwrought” and “demented” (respectively if not respectfully).

    The counter-backlash is correct that the speech is basically harmless.  Telling kids to stay in school, say no to drugs, and the like is the sort of thing that Nancy Reagan used to say (and people used to mock not because it was indoctrinating but because it was likely ineffective.)

    It’s worth stepping back from this kerfuffle to wonder why the president making a speech to the nation’s school children while they are in school is such a big deal.  The counter-backlash wants to suggest that the original backlash against the speech was motivated by crazy, conspiratorial thinking.  Presidents talk to the country all the time, they note.  And if the problem is supposed to be in the lesson plan proposed by the U.S. Department of Education, teachers can use or ignore these suggestions as they wish, just like they can regularly choose lesson plans.

    But that is at the heart of the backlash and is not entirely crazy.  Parents sense a lack of control over what their children are taught in school.  This is as true of every day’s social studies lesson as it is of Obama’s speech.  Most of those lessons, just like the president’s speech, are likely to be unobjectionable to most parents. 

    But on a fairly regular basis schools teach (or fail to teach) some things that are contrary to the values that parents would like conveyed to their children.  To those of us who see education as an extension of child-rearing, compulsory education privileging government-operated schools is an intrusion of the government on this parental responsibility.  To others, the intervention of the government is a positive good, protecting children from potentially dangerous values of the their parents and assuring allegiance to a common set of ideals necessary for our society to function.  As an empirical matter, government-operated schools are actually less effective at conveying that common set of ideas than are schools selected by parents.  

    Amy Gutmann, in the widely read book, Democratic Education, argues that this is not really an empirical question.  The principle is that there should be some democratic input into what is taught to children, not just parental control.   But in a chapter in the book, Learning from School Choice, I dissect Gutmann’s book to show that her scheme isn’t democratic at all.  She believes that local democracies should control schools as long as they avoid discriminating and repressing.  The problem is that almost everything of importance that they do could be portrayed as discriminating or repressing.  So who, under her scheme, resolves these disputes about what is permissible for local democracies to control in schools?  Unelected judges and unelected teaching professionals.  Gutmann’s proposal is really to substitute the dictatorship of an elite for the dictatorship of parents.  As I’ve argued before, I prefer to trust even poorly educated parents to make decisions in the best interests of their own children than well-trained but differently motivated bureaucrats.

    So, beneath the over-reactions and counter-over-reactions on Obama’s speech today is a real issue — Who should have primary responsibility for raising (educating) children?

    3 Comments | civic values | Tagged: , , , , , , , | Permalink
    Posted by Jay P. Greene

    Universal Voucher Benefits

    September 3, 2009

    Economist Maria Marta Ferreyra of Carnegie Mellon University has a new article coming out in the American Economic Review that models what would happen in a multi-district urban area if there were universal vouchers.  She finds that universal vouchers would generally improve income and racial integration and improve educational outcomes.

    She explains it all in this excellent video:

    UPDATED to correct typo

    6 Comments | civic values, competitive effects, vouchers | Tagged: , , | Permalink
    Posted by Jay P. Greene

    « Previous Entries
    Next Entries »