Just About Everything is Endogenous

September 30, 2009

A common technique in analyses of education policies (and popularized in the book, Freakonomics) has suffered a setback recently.  The technique attempts to correct for endogeneity, which occurs when your dependent variable is causing one of your independent variables rather than simply the other way around.

It’s probably best to explain this with an example.  Let’s say you want to know how the number of police officers in a city affects the crime rate.  In this example the dependent variable is the crime rate and the independent variable is the number of police officers.  That is, you are trying to explain how the size of the police force causes crime rates to be high or low.

The trouble is that the causal arrow also goes in the other direction.  The crime rate affects the size of the police force because cities with a lot of crime may decide to hire a lot of police officers.  So, the number of police officers is endogenous to the crime rate.  

That endogeneity could produce some odd results if we didn’t do anything to correct it.  We might find that the number of police officers causes crime rates to be higher when it might really be the case that the size of the police force reduces crime but high crime rates cause larger police forces.

This kind of problem comes up quite often in econometric analyses in general and in particular in evaluations of education policies.  So, it was a great a thing that University of Chicago economist James Heckman developed a technique for unravelling these circular relationships and correcting for endogeneity bias.  Basically, the technique uses some exogenous variable to predict the independent variable without bias.

Again, it’s probably easiest to explain with an example.  If we can find something that predicts the number of police officers that has nothing to do with the crime rate, then we can come up with an unbiased estimated of the number of police officers.  We can then use that unbiased estimate of how many police officers there would be (independent of the crime rate) to predict the crime rate.  In theory the technique works great.  Heckman won the Nobel Prize in economics for developing it.

The tricky part is coming up with a truly exogenous instrument (something that predicts the independent variable but has no relationship with the dependent variable).  The only obviously exogenous instrument is chance itself.  An example of that kind of instrument can be found in analyses of the effect of using a voucher on the student achievement of students who actually attend a private school when the vouchers are awarded by lottery.  Those analyses use whether a student won the lottery or not to predict whether a student attended a private school and then used that unbiased estimate of whether a student attended a private school to predict the effect of private schooling on student achievement. 

Whether a student won the lottery is purely a matter of chance and so is completely unrelated to student achievement, but it is predictive of whether a student attends a private school.  It is a perfectly exogenous instrument.

The problem is that other than lotteries, it isn’t always clear that the instruments used are truly exogenous.  Even if we can’t think of how things may be related, they may well be.

A perfect example of this — and it is one that raises questions about how exogenous all instruments other than lotteries truly are — was recently described in the Wall Street Journal having to do with date of birth.  The date during the year when babies are born has long been thought to be essentially random and has been used as an exogenous instrument in a variety of important analyses, including a seminal paper in 1991 by Josh Angrsit and Alan Krueger on the effects of educational attainment on later life outcomes. 

Since states have compulsory education laws require that students stay in school until a certain age, babies born earlier in the year reach that age at a lower grade and can drop out having attained less education.  By comparing those born earlier in the year to those born later, which they believed should have nothing to do with later life outcomes, they were able to make claims about how staying in school longer affected income, etc…

But new work by Kasey Buckles and Daniel Hungerman at the University of Notre Dame suggests that the month and day of birth is not really exogenous to life outcomes.  As it turns out, babies born in January are more likely to be born to unwed, less educated, and low income mothers than babies born later in the year.  The difference is not huge, but it is significant.  And since this variable is not exogenous, perhaps some or all of the effect of attainment Angrist and Krueger observed is related to this relationship between date of birth and SES, not truly attributable to attainment.

And if birth order is not random when we all assumed it was, what other instruments in these analyses are also not truly exogenous but we just don’t know how yet?  It’s a potentially serious problem for these analyses.

6 Comments | methodology | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

Market Ideas at Work Around the World

September 29, 2009

(Guest Post by Jonathan Butcher)

It is a beautiful thing when improvements in how we live can be explained by economic theories rooted in free market principles.  When someone halfway around the world sees their way of life improve and this is featured in the media, dust off your favorite book by a free market thinker and look for the theory that explains it.

This week’s Economist provides an opportunity to do just that.  In a feature section on the telecom industry and emerging markets, this excerpt on cell phones in an article entitled “Eureka Moments” caught my attention:

“How did a device that just a few years ago was regarded as a yuppie plaything become, in the words of Jeffrey Sachs, a development guru at Columbia University’s Earth Institute, ‘the single most transformative tool for development’?  A number of things came together to make mobile phones more accessible to poorer people and trigger the rapid growth of the past few years.  The spread of mobile phones in the developed world, together with the emergence of two main technology standards, led to economies of scale…” (emphasis mine).

The casual reader may miss the significant principle at work here: the poor, even on the other side of the planet, benefit from developments in wealthier nations.  This idea is at least as old as 1960, as Friedrich A. Hayek, beautifully elaborates in The Constitution of Liberty:

“There can be little doubt that the prospect of the poorer, ‘underdeveloped’ countries reaching the present level of the West is very much better than it would have been, had the West not pulled so far ahead.  Furthermore, it is better than it would have been, had some world authority, in the course of the rise of modern civilization, seen to it that no part pulled so far ahead of the rest and made sure at each step that the material benefits were distributed evenly throughout the world…

“The over-all speed of advance will be increased by those who move fastest.  Even if many fall behind at first, the cumulative effect of the preparation of the path will, before long, sufficiently facilitate their advance that they will be able to keep their place in the march.”

So, if we want to help the poor at home or abroad, powerbrokers should do everything they can to foster the success of the successful.  Instead of redistributing wealth through higher taxes on the rich, policymakers should make policy that helps entrepreneurs succeed.  For it is the knowledge they create, use, and pass on with their enterprises that quickens the pace of progress, pulling everyone along at a faster rate as the new technology spreads.

It is not just tax policy or legislation pertaining to businesses to which this idea applies; other social programs can be improved in the same way, and education is no exception.  Charter schools are an excellent example of a public policy that promotes individual liberty and entrepreneurship—resulting in the creation of new ideas that can then be used widely.

Everywhere charters have spread, the new ideas on leadership and teaching, for example, that they carry with them have been copied.  Even those opposed to charter schools have decided to combat them using the charter concept.  For example, Pilot Schools were created in Boston by existing school leaders in response to charter schools, using concepts central to the charter movement (more freedom over administrative decision making, specialized mission statements, etc.).  The result is that parents have even more options than before—more schools to chose from and more freedom.

Likewise, President Obama’s recent call for a longer school day and year is nothing new; the much-heralded KIPP Academies, also charter schools, have been operating with this policy for many years.  Again, new ideas that survive once implemented, created in the realm of entrepreneurship, are difficult to ignore even at the highest level.

The Economist’s piece goes on to explain how cell phones help the poor in rural areas around the globe, by “generally mak[ing] it easier to do business.”  In fact, a recent study found that “adding an extra ten mobile phones per 100 people in a typical developing country boosts growth in GDP per person by 0.8 percentage points.”  Would that we could see such an improvement in student achievement from policies promoting educational entrepreneurship.  Here’s hoping policymakers let the successful succeed, in education and elsewhere.

1 Comment | economics | Permalink
Posted by Jay P. Greene

Pass the Clicker — Filled with Glee

September 25, 2009

With the possible exception of Flashforward, which I haven’t had a chance to see yet but looks promising, Glee is the best new TV show of the season.  It’s a dark high school comedy about the struggles and triumphs of the glee club coach and its members. 

The best part about it is its unhindered departure from realism.  No glee club really sounds that good.  All of the characters are outrageous stereotypes.  No high school is filled with as much viciousness.  I especially love the coach of the “Cheerios” cheerleading team enforcing the Darwinian social hierarchy and Principal Figgins with his hand always on the calculator looking to save money by feeding the students prison food.

But in fully departing from realism the show probably better captures the reality of high school life than any of the sappy, gritty, “realistic” high school dramas, like Boston Public, Dangerous Minds, or Dead Poets Society.  Those are adult fantasies of what they would like high school to be — filled with heroic teachers battling the odds to save eager students . 

That’s not high school.  High school is often banal, outrageous, awkward, and pathetic.  Surrealism captures the experience so much better than realism.  The only other depiction of high school that I can think of that similarly captures the high school experience is the movie, Election.

Given that much of the attraction of the show is its outrageousness and novelty, I expect that the quality of the show will rapidly fade.  By season 2 we will have exhausted the one-dimensional characters and become jaded to the show’s novelty.  But enjoy the ride for now.

And yes, a show about a glee club is pretty “gay.”  The show tackles this issue head-on by featuring the tension between youthful anxiety about masculinity and youthful desire to express one’s creative self.  Just watch how the football team uses a dance to Beyonce’s All the Single Ladies to win the game:

And if you need more dancing to that song (and who doesn’t?), check out this video of All the Single Babies:

(edited to correct typo)

3 Comments | Pass the Clicker | Tagged: , , | Permalink
Posted by Jay P. Greene

More Charter Evidence

September 22, 2009

Diane Ravitch has declared that the Obama administration’s policy of expanding the number of charter schools has “no credible basis in research.”  This is just plain wrong.  And a new study coming out today from Stanford’s Caroline Hoxby demonstrates that she is even more wrong.

I’ve already noted that the highest quality studies — those that avoid bias from the self-selection of students into charter schools either with random-assignment or rigorous instrumental variable research designs — show significant academic benefits for students who attend charter schools instead of traditional public schools.  These studies examine the effect of charter schools in Massachusetts, Florida, Chicago, and New York City. 

And now add to that pile an updated study from Caroline Hoxby mentioned in today’s WSJ and NYT on New York City charter effects.  Students accepted by lottery into one of NYC’s charter schools in kindergarten and remained in a charter school through grade 8 closed the achievement gap with wealthy kids attending schools in Scarsdale entirely in math and two-thirds of the way in reading.

Critics are clinging to a study by Margaret Raymond at CREDO, which shows more mixed results.  While that study has the benefit of covering 15 states and DC, it can’t correct for the self-selection of students into charter schools like the highest quality studies linked above.  On average, students appear to be drawn to switching to charter schools because they are having trouble in their traditional public school.  Simply controlling for those students’ prior achievement and other observed demographic factors doesn’t quite correct for whatever negative factors may have caused students to switch to charters and that may continue to hinder their academic progress.  The CREDO study is as good as it can be given its approach, but I would have greater confidence in the consistent findings from several studies in different locations that do control for self-selection into charter schools.

8 Comments | charter schools | Tagged: , , , , , | Permalink
Posted by Jay P. Greene

Over There (But Not Over Here)

September 21, 2009

Several years ago I was part of a delegation sent by the U.S. Department of Education to a conference in China on private education.  The U.S. Dept of Ed believed that encouraging the expansion of private education in China would help promote democracy.  Apparently, they thought private schools were good for democratic values over there, but not over here. 

I was reminded of that experience while reading a recent New York Times article about severe problems with education in South Africa.  The piece states:

Despite sharp increases in education spending since apartheid ended, South African children consistently score at or near rock bottom on international achievement tests, even measured against far poorer African countries. This bodes ill for South Africa’s ability to compete in a globalized economy, or to fill its yawning demand for skilled workers. And the wrenching achievement gap between black and white students persists.

Sound familiar?

And what does the NYT tell us is a central part of the problem:

The teachers’ union too often protected its members at the expense of pupils, critics say. “We have the highest level of teacher unionization in the world, but their focus is on rights, not responsibilities,” Mamphela Ramphele, former vice chancellor of the University of Cape Town, said in a recent speech.

I see.  Teacher unions over there = bad, while over here = good.  Sometimes you have to get people outside of their vested set of domestic interests to see how they really think the world works.

Leave a Comment » | teacher union | Tagged: , , , | Permalink
Posted by Jay P. Greene

The Echo Chamber of Public Input

September 17, 2009

The Fayetteville school board and district leaders fully supported a plan that was soundly rejected by the voters this week.  How did school officials so badly mis-read what voters wanted?  It’s especially puzzling how school officials could have seriously misjudged their constituents given the years of deliberations, countless hours of public meetings and charrettes, and even a commissioned opinion poll.

Unfortunately, these countless rituals of public input are exactly what misled school officials to support an unpopular plan.  They were misled because these rituals of public input are better indicators of the views of the self-selected, small minority of people with the most intense (and often the most extreme) preferences than they are indicators of what the electorate would want.  School officials mistook the opinions of this self-selected few as the voice of the people. 

School officials also hired consultants to lead these public conversations, but in doing so they were steering discussions in a pre-determined direction.  Bringing in education consultant Tony Wagner and requiring all school employees to read his book steered the plan toward a high school divided into small learning communities.  That idea didn’t come from the voters.  It came from certain school officials, was made the topic of discussion in schools and community events, and then was echoed back to school officials. 

Similarly, the design “charrettes” led by consultants from New Orleans were not truly open brain-storming sessions about a new high school.  If they were, how did several small break-out groups independently arrive at the same Trail of Tears design concept? 

There is nothing inherently wrong with holding public discussions on important decisions or with bringing in expert consultants to inform and direct those conversations.  The problem is in falsely believing that what results from those discussions is in fact the opinion of the community.  They are more like echo-chambers, repeating back the preferences that school officials had going into them.

But school officials saw the community discussions as a sign of general public support for their vision.  They even went so far as to describe the plan that was developed from these events as “The People’s Plan.”  And then when asked why voters should support the millage, the advocates and editorial writers told us that it was The People’s Plan and had come from us so we should support what the community had developed.

This People’s Plan campaign strategy almost felt like bullying.  If you weren’t among the tiny, minority of atypical people who could spend evening after evening in community discussions, you had lost your chance to have a say.  It was time for you to get in line and support what the involved people had already determined.

Perhaps for this reason opponents of the millage stayed generally quiet during the campaign.  Yes, there was a handful of active letter writers and a Facebook group with fewer than ten members, but there was no organized opposition, no “vote no” yard signs, and a string of elite (even if tepid)  community endorsements.  But in the privacy of the voting booth, people clearly felt free to open-up and clearly say no.  Once the result had been announced, opponents discovered that they weren’t so isolated, and Facebook pages began to light-up with people explaining their reasons for opposing the millage despite their commitment to education and their understanding of shortcomings of the existing facility. 

The solution is not to hold even more public input rituals to scale back the cost of the project but leave all other decisions in place.  Presumably, the $116 million price tag followed from all of the design and policy decisions that had preceded it.  If all of the design and policy goals could have been met for a lower cost, why wasn’t the initial millage for a lesser amount?

Instead, the solution is to stop the echo-chamber decision-making of meetings, charrettes, and consultants, and start with real leadership.  School officials should step-up and tell us what they think would be educationally desirable at a reasonable cost.  Of course, it is difficult for them to gauge what the community would consider a reasonable cost without public input, but the election result has given them better feedback than any town-hall discussion or charrette ever will.

Superintendent Vicki Thomas is particularly well-positioned to offer her vision of our educational future.  She bears no responsibility for the development of the failed millage plan and can start with a fresh slate.  We hired her to lead our schools and leadership is what we need.  She has enough information from voters and past public meetings to assess the community’s priorities.  Now she can give us a new plan and convince us that it is what she thinks is best, not what she thinks we told her to say.

3 Comments | Fayetteville, politics | Tagged: , , , | Permalink
Posted by Jay P. Greene

Millage Defeated

September 16, 2009

For those readers of JPGB who care (and I mean to include both of you), the Fayetteville millage was defeated by a margin of 59% to 41%.  You can get the latest news and analysis at Mid-Riffs.

6 Comments | Fayetteville, politics | Permalink
Posted by Jay P. Greene

Have you “Experienced” The Riffs at Mid-Riffs?

September 15, 2009

Mid-Riffs, a blog started by a bunch of my friends, is off to a great start with several posts on the high school millage in Fayetteville, Arkansas.  Sometimes I agree with them and sometimes I don’t, but they are always fun to read.

The election is today, so be sure to check out their excellent information and analysis.  In particular, they have argued:

  • The high school is not falling apart. (In a 2006 statewide ranking of buildings needing repair, Fayetteville high school was ranked 988 out of 1,129 K-12 public school buildings, where 1 was most need of repair.)
  • There is no evidence buildings improve student outcomes.
  • The current facility has deficiencies, but they don’t necessitate complete demolition and reconstruction.
  • There is a case to be made for economic development, but any positive effects will be much diminished by the necessary tax increase.

    • But Mid-Riffs did make a case for why we might want to spend $116 million to tear-down the currently functional building for a brand new one — we like shiny new things.  We don’t need to buy diamond engagement rings, but people like to have them.  We don’t need a new building, but we might still want to have one.

    It’s not a very compelling argument, but it is no worse of a reason than your reason for buying that new Lexus.

    4 Comments | blogging | Tagged: , , , | Permalink
    Posted by Jay P. Greene

    More 21st Century Skills

    September 14, 2009

    I’ve written before about Tony Wagner and the 21st Century Skills movement, here, here, here, and here.  My colleague Sandra Stotsky has an excellent review in the current issue of The Weekly Standard of Tony Wagner’s book, The Global Achievement Gap.

    Here’s the money quote:

    It is disingenuous to imply that the development of analytical thinking and effective oral and written communication (goals of the lyceum in ancient Greece) are new to the 21st century. American education schools and their satellite networks of professional development providers heavily promoted such “21st-century skills” as critical thinking, problem solving, and small group work throughout the 20th century.
     
    If our teaching corps hasn’t yet been able to figure out how to translate these buzzwords into effective classroom lessons, what does this tell us about the teaching skills of our very expensive standing army of teacher-educators, either to prepare teachers properly in the first place or to get them up to snuff after they’ve failed in the field?…
     
    Evidence-free rhetoric in support of reducing academic content in the schools, diluting academic standards for K-12, and eliminating large-scale academic testing, has found a receptive audience across the country among those who don’t want any form of real accountability. Unfortunately, the valuable skills misidentified as 21st-century skills cannot be taught and assessed without a strong emphasis on academic substance, standards, and objective assessments–as academic researchers know.
     
    Wagner is the latest in a long line of educational pied pipers leading an uncritical and growing mass of school administrators and teachers into a curricular wilderness. And this latest book is just the current manifestation of the goal driving most of our education schools and professional development providers–how to reduce the academic content of the curriculum while claiming to enhance it–this time in the name of closing the “gap,” or providing worker bees for this century’s employers.

    1 Comment | instructional reform | Tagged: , , | Permalink
    Posted by Jay P. Greene

    Facebook in Reality

    September 13, 2009

    (ht: a Facebook friend)

    1 Comment | blogging | Permalink
    Posted by Jay P. Greene

    « Previous Entries
    Next Entries »