Review the Charter Research, Don’t Pick the Outlier

November 2, 2011

Julian Betts and Emily Tang at the University of California at San Diego have a new systematic review of the research on charter schools.  They look at more than 30 studies that meet minimal criteria for research quality.  They find that charters have statistically significant positive effects on math and reading achievement in elementary grades and on math in middle school.  There are no significant effects for reading in middle school or for high school student achievement.  The size of the effects are modest, ranging between 2% and 6% of a standard deviation.  (See Table 2)

It’s important to step back and review an entire literature, rather than focus on a single study.  It is sensible to focus on higher quality research, since results are highly sensitive to research design.  But it is completely inappropriate and misleading to pick a single study while ignoring all others of equal or higher quality simply because that one study produces the result you like.

Of course, highlighting the one study she likes is Diane Ravtich’s stock in trade.  All we hear from Ravitch and her Army of Angry Teachers is about the CREDO study.  That’s one study — and not a high quality one.  And even then Ravitch distorts what CREDO finds.

But Betts and Tang’s review includes CREDO and dozens of other studies.  When we look at the full set of research, we see some significant and positive results.  And in Table 4, Betts and Tang show us that if you exclude the CREDO study, the positive effects for charters get stronger so that charters significantly improve math achievement across all grades.

Of course, you shouldn’t exclude that one study, but it is informative that the one study that Ravitch and her Army of Angry Teachers hold up as proof of their view that charters don’t work is clearly an outlier from the full set of research.  And if we focus on the highest quality random-assignment studies of charter schools, the positive results are even stronger.

I wonder if Diane does this in her historical research.  Does she pick the one quotation or document that supports her argument while misleading readers about the entire set of information?  It’s harder to catch Ravitch in this sort of deceptive scholarship in historical work, but in quantitative empirical research, it is the essence of what she does.

(edited for typos)

5 Comments | charter schools | Tagged: , | Permalink
Posted by Jay P. Greene

Save a Pretzel for the Gas Jets

November 1, 2011

There’s a hilarious series of videos of “bad lip reading” of presidential candidate’s speeches.  Check out these to get you started.  And for your information, I am in 100% agreement with Perry that we should “save a pretzel for the gas jets.”

Leave a Comment » | random pop culture | Tagged: | Permalink
Posted by Jay P. Greene

Winner of the 2011 Al Copeland Humanitarian Award: Earle Haas

October 31, 2011

We had several especially worthy nominees for this year’s Al Copeland Humanitarian Award.  Greg nominated Charles Montesquieu, I nominated David Einhorn, and Matt nominated Steve Wynn.  But Anna Jacob nominated this year’s winner: Earle Haas, the inventor of the modern tampon.

Montesquieu’s nomination was especially important in light of the clear shift away from rule by law to rule by regulators.  No one knows what the health care bill actually required; most of the important parts were left to be decided later by regulators.  The new financial regulation is similarly just a template for regulation to be determined later.  This is a very corrosive and dangerous development in a representative democracy and Greg was right to warn us about this.

But it is not quite worthy of “The Al.”  As Greg notes, we are letting Montesquieu’s ideas about the rule of law slip away from us.  If someone were able to reverse that alarming trend, that person might be worthy of “The Al.” There is no shortage of good political philosophers and most have received plenty of recognition.  The tricky and courageous part is to properly implement those ideas.

My nomination of David Einhorn was, I thought, a useful antidote to the economic illiteracy of the Occupy Wall Street movement as well as the petty financial regulators who impede short-selling to appease mob anger and (they claim) promote stability.  But I’m not sure Einhorn’s contributions have lacked proper recognition.  Financial news regularly repeats whatever he says and gives him plenty of credit for correctly identifying past corporate frauds and excesses.

Steve Wynn probably most closely resembles Al Copeland in his personal biography.  They are both business people whose products are adored by some while denounced by others.  They both nearly drove their businesses into the ground.  They both have reputations for being personally difficult.  But despite all of these and other failings, they have both done much more for humanity than most people who win humanitarian awards.

But the “Al” is not given for personal similarity to Al Copeland.  Wynn has built some very nice hotels, turned around an increasingly seedy Las Vegas, and launched a fast-growing subsidiary in Macau.  But frankly my life (and most other people’s lives) would be just fine without these things.  I like the frivolous nature of Wynn’s gambling hotels, just like Copeland’s spicy chicken, but the “Al” is not mostly about being frivilous.

Earle Haas’ tampon, on the other hand, has made an enormous difference in improving women’s lives and making our society better.  Women often felt confined to their house a few days every month.  This limited their ability to work, travel, and engage in public life almost as much as restrictions on women in Saudi Arabia — at least for those few days.  Just look at the ad copied above.  The tampon allowed women to more regularly work during World War II to increase productivity.  The tampon helped defeat the Nazis!

But the tampon also helps illustrate where advancements for women really tend to come from.  Technological innovation, like the tampon, helped liberate women and that innovation comes from a capitalist system.  Earle Haas invented the tampon, at least in part, to make money.  Tampax Corporation brought the product to a mass market primarily to make money.  And women were successfully educated about the benefits of tampons through advertising.  Contrary to the loosely Marxist notion that advertising artificially creates desires for unnecessary products, just look at how essential advertising of tampons was in overcoming irrational opposition and ignorance of its benefits for women and society.

Activists, politicians, and other miscreants who regularly receive credit for advancing the interests of women have often done no such thing.  In fact, quite often they have undermined the cause.  Demanding that women be paid the same even if they are more likely to be absent from work just discourages people from hiring women.  But inventing new technologies that reduce the need for women to be more absent from work allows women to be paid the same without harming their employers.  If speeches and laws created reality we would be right to recognize the activists and politicians.  But Earle Haas was able to create a new reality that no speech or law could do on its own.

If the tampon was good enough to defeat the Nazis and undermine Saudi-like restrictions on women, its inventor is good enough for the “Al.”

(Updated for missing paragraph)

2 Comments | Al Copeland Humanitarian Award | Tagged: , | Permalink
Posted by Jay P. Greene

Nomination for the Al Copeland Humanitarian Award: Earle Haas

October 27, 2011

Greg has nominated Charles Montesquieu and I have nominated David Einhorn for this year’s Al Copeland Humanitarian Award.  Here is Anna Jacob’s nomination of Earle Haas, the inventor of the modern tampon:

An overlooked everyday convenience, the mighty tampon is a transformative technology, offering comfort and convenience to consumers. Women had been improvising for thousands of years prior to this revolutionary invention- in Ancient Egypt, using softened papyrus; in Rome, using wool; in Japan, paper; in Indonesia, vegetable fibers, and in Equatorial Africa, rolls of grass. Made of compressed cotton, the tampon as we know it was patented in 1933 by Dr. Earle Cleveland Hass of Denver, Colorado whose “catamenial device” included such modern conveniences as a removal string and insertion applicator. Dr Hass sold the patent and trademark to Gertrude Tendrich who went on to found the Tampax company three years later. The Tampax sales team realized that advertising alone would not ensure the success of their product and established an education department to dispel myths and misconceptions. Mabel Matthews, Tampax’s first full-time educational consultant, travelled to women’s colleges throughout the U.S. discussing the safety and effectiveness of her product.

At the beginning of World War II, the tampon industry faced a setback that could’ve killed this invention before it ever reached mass production. Production of cotton bandages for the troops exploded and Tampax had to assert its product as an essential health product if it was to secure the raw materials of cotton and paper that were need for its production. Company executives travelled to Washington DC numerous times to plead their case, basing their strategy on the lesser size of their product in comparison to its alternative, the sanitary napkin or “maxi-pad.” Tampax succeeded in ensuring access to these materials by asserting that both production and transportation of Tampax actually opened up shipping space and raw materials for the war effort.

Dr Haas’ invention has been transformational for women both personally and professionally, allowing them to travel, work, and swim, unhindered by a predictable restriction three to five days out of every 28. Thank you Dr Haas!

5 Comments | Al Copeland Humanitarian Award | Tagged: | Permalink
Posted by Jay P. Greene

Gates Responds

October 26, 2011

Steve Cantrell, a senior researcher at Gates, sent me an email last night in response to my post from yesterday asking for the MET results to be released.  He said that I was right in suggesting that large, complicated projects sometimes take longer than originally planned.  He said that final scores for coding the videos had just been delivered to the research team and that the full results for the 2009-10 year were now scheduled to be released January 5, 2012.  It’s unclear whether that report will also contain information for the 2010-11 year as well.  The MET web site will be changed to reflect this new schedule.  (Update: According to another email from Steve Cantrell, the January release will only have the full 09-10 results.  The final results including 10-11 and are scheduled for release in early summer of 2012 .)

Steve also clarified information on the cost of the project.  Last year I repeated the New York Times and LA Times description of the project costing $45 million.  More recently I’ve repeated the Wall Street Journal description of the project cost as $335 million.  Steve resolved the confusion by saying that the MET study costs about $50 million and the $335 million figure includes grants to the partner districts.

Let me be clear that I think Gates has a lot of good and smart people working on the MET project.  My concern is not that these are bad people.  My concern is that Gates has a flawed strategy based on centrally identifying what educators should do and then building a system of standards, curriculum, and assessments to impose those practices on the education system.  I don’t think this kind of centralized approach can work and I fear that it creates enormous pressure on good and smart researchers to toe the centralized line — even if it becomes obvious that it is not working.  Everyone at Gates can see what happened to the folks who pushed small schools when the Foundation decided that approach was not working.

And unlike Diane Ravitch, Valerie Strauss, and the Army of Angry Teachers, I am not criticizing the Gates Foundation because I think Bill Gates is in the “billionaire boys club” and therefore somehow disqualified from using his wealth to try to improve education.  I am critical of recent Gates Foundation efforts because I believe Gates can and should try to improve education by adopting a more fruitful strategy.

(corrected typos)

11 Comments | instructional reform, research reports | Tagged: , | Permalink
Posted by Jay P. Greene

Gates Foundation — Release the MET Results

October 25, 2011

A sketch of the $500 million new Gates Foundation headquarters

Bill and Melinda Gates mentioned again in the Wall Street Journal the Measuring Effective Teachers (MET) project that their foundation is orchestrating.  Bill and Melinda may want to check on the status of the MET research they’ve been touting since full results were promised in the spring of 2011 and have yet to be released.

Just to review… In an earlier interview with the Journal, MET was described as follows:

the Gates Foundation’s five-year, $335-million project examines whether aspects of effective teaching, classroom management, clear objectives, diagnosing and correcting common student errors can be systematically measured. The effort involves collecting and studying videos of more than 13,000 lessons taught by 3,000 elementary school teachers in seven urban school districts.

The motivation, re-iterated in the new piece by Bill and Melinda Gates is to identify  what “works” in classroom teaching to develop systems that train and encourage other teachers to imitate those practices:

It may surprise you—it was certainly surprising to us—but the field of education doesn’t know very much at all about effective teaching. We have all known terrific teachers. You watch them at work for 10 minutes and you can tell how thoroughly they’ve mastered the craft. But nobody has been able to identify what, precisely, makes them so outstanding….

The intermediate goal of MET is to discover what we are able to measure that is predictive of student success. The end goal is to have a better sense of what makes teaching work so that school districts can start to hire, train and promote based on meaningful standards.

As I’ve argued before, using research to identify “best practices” in teaching only makes sense if the same teaching approaches would be desirable for the vast majority of teachers and students, regardless of the context.  And as I’ve also  suggested before, I don’t believe this effort is likely to yield much in education.  Effective teaching is like effective parenting — it is highly dependent on the circumstances.  Yes, there are some parenting (and teaching) techniques that are generally effective for almost everyone, but those are mostly known and already in use.

This doesn’t mean we are completely unable to measure effective teaching (or parenting).  It just means that we have to judge it by the results and cannot easily make universal statements about the right methods for producing those results.  To make a sports analogy, there is no single “best practice” for hitters in baseball.  There are a variety of stances and swings.  The best way to judge an effective hitter is by the results, not by the stance or swing.  And if we tried to make all hitters stand and swing in the same way, we’d make a lot of them worse hitters.

It is because of this heterogeneity in effective teaching practices that I think the MET project is doomed to disappoint.  And according to inside sources, I’ve heard that results are being delayed because they are failing to produce much of anything.

According to the MET web site, the full results for the 1st year should have been released in the spring:

 In spring 2011, the project will release full results from the first year of the study, including predictors of teaching effectiveness and correlation with value-added assessments.

It is almost November and we have not seen these results.  I understand that in very large and complicated projects, like MET, things can take much longer than originally planned.  If so, it would be nice to hear that explanation.  It would be even nicer if the Gates Foundation released results if they have them, even if those results were not what they had hoped they would find.

Some inquisitive reporters should start asking Gates officials and members of the research team about the status of the MET results.  Reporters should go beyond talking to the media flacks at Gates HQ and actually talk to individual members of the team confidentially.  If they do that, they may confirm what I have been hearing: MET results have been delayed because they aren’t panning out.

(UPDATE:  Gates responds.

8 Comments | instructional reform | Tagged: , | Permalink
Posted by Jay P. Greene

What’s Going Right in Waconda?

October 24, 2011

According to the Global Report Card that Josh McGee and I developed, tiny Waconda, Kansas is one of the top-performing school districts in the United States.  Other than being the home to what residents claim is the world’s largest ball of twine (pictured above), one might not think that there was anything exceptional about this rural, farm community in north central Kansas.

But in 2007 the average student in Waconda performed better than 91% of students in our 25 country comparison group in math achievement.  If we relocated Waconda to Finland, the average student in Waconda would outperform 88% of the students in Finland in math.

A reporter for Yahoo News was curious about what they were doing right in Waconda.  Here is what she found:

So why are Waconda kids–65 percent of whom live in poverty–doing so well? And can other schools follow their lead?

The Waconda district comprises four small towns–Cawker City, Downs, Glen Elder and Tipton–and seven schools spread over 411 square miles. Most people in the area work in agriculture or in manufacturing.

The district’s superintendent of seven years, Jeff Travis, told Yahoo News that after years of high test scores, the community expects its students to excel. Most years, he added, no one drops out of high school. The district won 14 state Governor Achievement Awards and one national “Blue Ribbon Award School” over the past four years.

“It’s a tradition now, and they expect themselves to do well,” Travis said. “Like a ball team that continues to win because of a tradition, we have an academic tradition.”

Still, the community doesn’t quite seem to get how exceptional they are. “Everybody’s pretty happy [but] nobody understands how big a deal it is,” he said.

Travis says the students’ high level of achievement is even more extraordinary given that 65 percent of them qualify for free or reduced federal lunches, an indication that they live in poverty. High poverty schools are often dogged by low test scores and high dropout rates. Many educational observers indeed blame the nation’s sky-high child poverty level for the country’s comparatively low performance in math.

One theory Travis has is that Waconda school kids have no sense that they’re materially deprived. “North Central Kansas is rural, and urban poverty is kind of different [from] rural poverty,” he said. “A lot of our people don’t even understand that they’re living in poverty.” According to state data, most of the students are white, and no kids need English language learning classes.

About 10 percent of the students in the school district are foster kids, Travis says. “We just [have] a lot of adults that care about kids, so it’s been a popular thing for parents to take in foster children.”

But Josh adds a useful note of caution at the end of the reporter’s piece:

One of the Global Report Card’s authors, Josh McGee, says the small size of Waconda schools may have skewed the results slightly, since randomness has a greater impact on a smaller sample size. Most of the best-performing school districts in his ranking were small, and many of them were also made up of charter schools. You can read more about his methodology here.

We may not be able to generalize much from the success in Waconda, but it is a fun and impressive story.

Meanwhile, coverage of the Global Report Card continues to stream in.  For an updated list of media for the Global Report Card (with links), click here.

Leave a Comment » | school accountability | Tagged: , | Permalink
Posted by Jay P. Greene

Josh McGee on the Global Report Card

October 18, 2011

Check out Josh discussing the Global Report Card at this Fordham Institution event on “The Other Achievement Gap” focusing on the disappointing performance of some of our “best” students and school districts.

Josh starts at around 12:30 in the video.

Also check out the updated list of coverage on the Global Report Card.

Leave a Comment » | school accountability | Tagged: | Permalink
Posted by Jay P. Greene

Excellence in Failure

October 18, 2011

It’s long been a goal of mine to be so awful at a job that they have to pay me to leave.  Unfortunately, excellence in failure is something that has escaped me — even though I normally seek excellence in all things.

But you’ll be happy to know that football coaches, superintendents, CEOs, and even large corporations have all too often perfected the art of sucking so bad that people pay them for that failure.

These large severance packages are generally a problem when the people offering the package are doing so with OPM — Other People’s Money.  Athletic directors, school boards, boards of directors, and the government find it so much easier to be generous when the money they are offering to their failed coaches, superintendents, CEOs, and large corporations is not their own money.

The Occupy Wall Street folks have (rightly) highlighted the sweet deals offered to failed CEOs and corporations from tax funds, but they tend not to mention the frequency with which superintendents are given large severance packages with OPM.  Yes, the average size of the superintendent packages tends to be much smaller, but there are so many more of them that it adds up to real money.

A Chicago Tribune analysis last summer found:

The newspaper’s review of more than 100 superintendent contracts, financial records and severance agreements over a decade revealed that boards have handed out six-figure separation checks; district-paid health care; cash or retirement credit for hundreds of sick days; and, in one case, a Mercedes — all to be rid of superintendents….

“Boards have not been held responsible because they do not care about taxpayers, period. … They do not care about how much money they spend,” said Kenneth Williams, board president in Thornton Township High School District 205, which recently approved a $350,000-plus severance package for J. Kamala Buckner, a veteran superintendent. Williams tried unsuccessfully to rescind the package in May….

Using available state data, the Tribune tracked the number of superintendents in Illinois school districts from 2000 to 2010, finding nearly half had gone through three or more superintendents.  That turnover not only fuels buyout deals but can take a toll on issues ranging from policy to student achievement….

Changeover also means some superintendents get multiple severance payouts.

Rosemary Hendricks got a $132,000 settlement in 2009 after filing a lawsuit against Hoover-Schrum District 157, where she had served as superintendent about a year. Earlier, she had gotten a $75,000 settlement in Bellwood District 88, where she had a short stint as superintendent, records show. She is back as Bellwood superintendent.

Of course, sometimes these generous severance packages are the fault of boards, not the departing executive.  Boards sometimes choose to get rid of someone on a whim or simply because the majority composition of the board changes.  These reckless changes by boards to get rid of someone under contract or who could justly sue for wrongful termination, force boards to fork over large sums of OPM to avoid litigation.

My point is the irresponsibility that OPM encourages.  OPM encourages organizations to write contracts that are excessively long and generous and then require large severance packages to get out of them.  OPM encourages changing leadership without cause and at great cost.  OPM makes it so much easier to justify the bailout to avoid “systemic risk” or to promote “stability.”  And of course OPM facilitates coaches, superintendents, CEOs, and corporations to make unreasonable demands, take unreasonable risks, and fail catastrophically.

OPM encourages excellence in failure.

4 Comments | economics, teacher pay | Permalink
Posted by Jay P. Greene

Nominated for the Al Copeland Humanitarian Award: David Einhorn

October 11, 2011

The Al Copeland Humanitarian Award is meant to honor unheralded contributions to improving the human condition.  Unlike most humanitarian awards, which are given to hack politics, community activists, and others whose contribution to humanity may often be more negative than positive, “The Al” tends to highlight entrepreneurs and  inventors.  The not-so-implicit argument of “The Al” is that people can often do more to improve the human condition by creating and building something from which they profit than by giving away their profits or (in the case of politicians and activists) trying to take away someone else’s profits in order to give them to others.

With the recent adulation of Steve Jobs, the role of the entrepreneur as humanitarian has received a lot more recognition that it normally does.  People acknowledged the extent to which Steve Jobs improved the human condition by creating and promoting products that people enjoy.  The fact that he accumulated an enormous amount of wealth for himself, his family, his employees, and his shareholders does not spoil the contribution that he made.  And it was not necessary for Steve Jobs to give away that wealth to do good in the world.  Steve Jobs was probably better at improving the human condition by creating wealth than he could have been by giving it away.

Given this recent recognition of the importance of people who create and build things, I would like to use this year’s “Al” to highlight the potential humanitarian benefits of those who destroy things.  In particular, I would like to highlight the contribution to improving the human condition made by short-sellers.

Short-sellers are people who sell something that they do not own in the hopes of profiting by a decline in the price of that thing.  They are betting that prices, usually of particular stocks or bonds, will go down.

What good comes from people who profit when stocks or bonds lose value?  Basically, short-sellers do as much good as people who profit from a rise in prices, or those who go “long.”  They help insure rational pricing of stocks and bonds so that capital is properly allocated to the places where it can get the highest long-term return (and do the most good for humanity).  If prices are irrationally low for something, then an insufficient amount of new capital will be allocated to that thing.  Conversely, if prices are too high, then too much capital will be allocated toward that thing, which will waste those resources and deprive it to other endeavors that would be more beneficial.

It’s relatively common to laud people who profit from stocks and bonds going up in price.  Warren Buffett has received plenty of positive attention.  But remember that Buffett’s enormous profits were made by others selling to him at lower prices and missing out on the increase in value.  People tend to revile short-sellers for doing the exact same thing — just in the opposite direction.

The short-seller I would like to nominate for this year’s Al Copeland Humanitarian Award is David Einhorn.  Einhorn is most famous for having taken large short positions on Allied Capital and Lehman Brothers.  According to Wikipedia:

In May 2002, at the Ira W. Sohn Investment Research Conference, Einhorn made a speech about a mid-cap financial company called Allied Capital, and recommended shorting it. The stock opened down 20 percent the next day.

Einhorn claimed that Allied was involved in lending practices that actively defrauded the Small Business Administration in the US, and therefore, indirectly, the US taxpayer. Allied said that Einhorn was engaged in market manipulation. To prove this, Allied fraudulently accessed his phone records using pretexting.

The U.S. Securities and Exchange Commission (SEC) investigated Einhorn for market manipulation. Eliot Spitzer announced that he intended to start another investigation. In June 2007, the S.E.C. found that Allied broke securities laws relating to the accounting and valuation of illiquid securities it held.[7]

Einhorn has published a book, Fooling Some of the People All of the Time[8] regarding his battle. FoolingSomePeople.com, the website to promote the book, has information on his angle of the dispute, including a press release from Allied in February 2007 admitting that an agent of the company obtained Einhorn’s phone records fraudulently.

His battle against Allied Capital lasted six years in all. Einhorn would come to view Allied as a microcosm of market trends. “What we’ve seen a year later is that Allied was the tip of an iceberg; that this kind of questionable ethic, philosophy and business practice was far more widespread than I recognized at the time,” he said. “Our country, our economy, is paying a huge price for that.”[9]

In late 2008, the Office of the Inspector General of the Securities Exchange Commission announced that it was investigating some of the charges that Einhorn has made about the SEC’s mishandling of this matter, including the possibility that “a former SEC attorney may have taken confidential investigative materials with him when he left the Commission and provided those materials to a company he went to work for as a lobbyist.”

And Wikipedia describes Einhorn’s short position on Lehman Brothers:

Starting July 2007, Einhorn became a short seller in Lehman Brothers stock. He believed that Lehman was under-capitalised, and had massive exposures to CDOs that were not written down properly.[10] He also claimed that they used dubious accounting practices in their financial filings.[2] …

Lehman went bankrupt in September 2008.

Yes, Einhorn profited by the collapse of Lehman Brothers.  And yes, that collapse had a “contagion” effect that was part of the economic recession of 2008-09 from which we have yet to recover.  But that collapse had to occur.  Real estate prices had reached irrational levels, lending practices were grossly irresponsible, and all of this was mis-allocating capital in very harmful ways.  Building more and more houses in the Nevada desert was a waste of resources that should have been going to more productive ventures.

Einhorn should be lauded, not condemned, for his role in bursting that bubble.  And Einhorn can’t be blamed for the way in which our government has bungled the aftermath of that bubble bursting.  Eventually housing prices had to come way down, banks had to recognize their bad loans and recapitalize, and the construction industry had to grind to a halt for a time.  Short sellers, like Einhorn, helped us recognize these realities sooner.  If the bubble had continued and grown larger, the pop would have been even worse.

The fact that Einhorn is also a competitive poker player, having finished 18th in the World Series of Poker, and the fact that he has given away his poker winnings and half his profits on shorting Allied Capital (in addition to a variety of other high-profile charitable efforts), is only icing on the cake.  Einhorn’s contribution to improving the human condition comes from his deriving a profit from exposing fraud and excess in companies.  That helped pop financial and real estate bubbles before they grew much worse.  By destroying waste and fraud, Einhorn clears the path for others to build.

2 Comments | Al Copeland Humanitarian Award | Tagged: | Permalink
Posted by Jay P. Greene

« Previous Entries
Next Entries »