Los Estados que no desea ser reencarnado en si viene como un niño pobre de los hispanos.

November 4, 2011

(Guest Post by Matthew Ladner)

So how did my English to Spanish translator website do? I studied French while a student, which has come in handy about three times in my life, and may never do so again.

But I digress. The chart here ranks states by the percentage of low-income Hispanic students scoring “Below Basic” on the 4th grade NAEP reading exam in 2011.

Like any of these reincarnation charts, there are any number of factors to bear in mind. Some states have more ELL students than others, generational effects are important, and Hispanics are far from monolithic.

Nevertheless, isn’t it interesting that Oregon yet again makes an appearance in the hall of shame. Last time I visited, Oregon was way up in the Pacific Northwest and far from the southern border.

Now that the mandatory Oregon mocking is complete, let’s talk serious business: California is a disaster. The sheer size and low scores of the California Hispanic population ought to be a national concern. While it is fun to poke at Oregon for a being even worse than California, California’s Hispanic population is a sea to Oregon’s pond.

Matters are far better in Texas, the home of the second vast Hispanic population in America, but still very much in need of improvement.

California and Texas educate more than half the nation’s Hispanics, almost 5.5 million students. We need California to wake up, and for Texas to step up.

Leave a Comment » | research reports | Tagged: , , , | Permalink
Posted by matthewladner

The New No Excuses: Where Not to be Reincarnated a Rich White Kid

November 3, 2011

(Guest Post by Matthew Ladner)

So the plot thickens, as many JPGB readers (including this author) was born as an American White kid who was not eligible for a Free or Reduced Lunch. In the Great Reincarnation to Come, maybe that is always how it works out!

Or maybe not.

In any case, you ought not to feel overly reassured. Assuming again that you want to learn to read, the above chart shows achievement levels from the 2011 NAEP for non-FRL eligible White students.

Before proceeding to dwell on West Virginia and others, I should note that DC has finally come in first place in something! If you are an ultra-wealthy White student going to one of the highly exclusive public schools in Georgetown, your reading ability rocks. Congratulations to the portion of the DC school into which few poor kids ever step foot much less attend.

Something has been going wrong in West Virginia, as their NAEP scores have been declining. Alaska is a different sort of place that obviously needs to get their act together on K-12. Tennessee can’t be happy to see themselves near the top of this list, and Nevada needs to let go of the idea that you don’t need to be well-educated to deal blackjack.

And then, there’s Oregon. Someone please explain to me why 21% of middle and upper income Anglos in Oregon should be illiterate.

 

 

7 Comments | research reports | Tagged: | Permalink
Posted by matthewladner

Gates Responds

October 26, 2011

Steve Cantrell, a senior researcher at Gates, sent me an email last night in response to my post from yesterday asking for the MET results to be released.  He said that I was right in suggesting that large, complicated projects sometimes take longer than originally planned.  He said that final scores for coding the videos had just been delivered to the research team and that the full results for the 2009-10 year were now scheduled to be released January 5, 2012.  It’s unclear whether that report will also contain information for the 2010-11 year as well.  The MET web site will be changed to reflect this new schedule.  (Update: According to another email from Steve Cantrell, the January release will only have the full 09-10 results.  The final results including 10-11 and are scheduled for release in early summer of 2012 .)

Steve also clarified information on the cost of the project.  Last year I repeated the New York Times and LA Times description of the project costing $45 million.  More recently I’ve repeated the Wall Street Journal description of the project cost as $335 million.  Steve resolved the confusion by saying that the MET study costs about $50 million and the $335 million figure includes grants to the partner districts.

Let me be clear that I think Gates has a lot of good and smart people working on the MET project.  My concern is not that these are bad people.  My concern is that Gates has a flawed strategy based on centrally identifying what educators should do and then building a system of standards, curriculum, and assessments to impose those practices on the education system.  I don’t think this kind of centralized approach can work and I fear that it creates enormous pressure on good and smart researchers to toe the centralized line — even if it becomes obvious that it is not working.  Everyone at Gates can see what happened to the folks who pushed small schools when the Foundation decided that approach was not working.

And unlike Diane Ravitch, Valerie Strauss, and the Army of Angry Teachers, I am not criticizing the Gates Foundation because I think Bill Gates is in the “billionaire boys club” and therefore somehow disqualified from using his wealth to try to improve education.  I am critical of recent Gates Foundation efforts because I believe Gates can and should try to improve education by adopting a more fruitful strategy.

(corrected typos)

11 Comments | instructional reform, research reports | Tagged: , | Permalink
Posted by Jay P. Greene

Buckle Up…2011 NAEP release on Nov. 1st

October 18, 2011

(Guest Post by Matthew Ladner)

NAEP is releasing 4th and 8th Grade Reading and Math results for 2011 on November 1st.

I’ll comb through the data and post the results here.

2 Comments | research reports | Tagged: | Permalink
Posted by matthewladner

J.K. Rowling: The Jeb Bush of the NEPC Florida Fantasy

September 13, 2011

(Guest Post by Matthew Ladner)

The Arizona School Boards Association had their annual law conference last week, and had William Mathis from the Think Tank Review Project present on “Are Things as Sunny as They Seem in Florida?”

I went first, and presented charts like this, showing the vast improvement in Florida’s 3rd grade reading scores:

I have repeatedly asked the Think Tank Review Project people to explain why Florida’s 4th Grade NAEP scores continued to rise in 2007 and 2009 even as 3rd grade retention fell substantially. Or for that matter, why their 3rd grade scores have improved so strongly. Dr. Mathis made no attempt to address the issue.

I also presented charts like these:

Now, call me crazy, but when you are the state called “Arizona” in above chart, you might want to make a careful study of what the other state did to get their English Language Learners to read. This phenomenon  of course is not limited to ELL. Another chart I used showed the combined learning gains on all four NAEP tests for children with disabilities for the entire period we have data from all 50 states (2003-2009).

Just in case you are squinting that’s Florida in red with a gain of 69 points and Arizona in green with a decline of two points.

Dr. Mathis proceeded with his presentation unperturbed. He complained about the 3rd grade retention policy without any effort to explain why Florida’s 3rd grade scores had so profoundly improved, and why Florida’s 4th grade NAEP scores continue to increase even as retention rates have significantly declined.

To give Dr. Mathis’ presentation the fairest possible reading, I would say that he was trying to make the following points: that correlation is not causation, and that to use the terminology of Campbell and Stanley, I had not “controlled for history.” That is to say, there could be other possible explanations for Florida’s gains other than the reforms.

Now it is of course the case that correlation can lead us very much astray, and it is the case that “history” has a nasty habit of bedevilling our theories of causality. As I have noted in the past, however, the Florida reforms unfolded in the real world, rather than in a random assignment study. A great many things unfolded all at once. This is called “life” and there is nothing to be done about this but to gather as much data as possible to draw the best informed decisions we can.

Both Chatteriji and Mathis ignored the Education Next piece in which Dan Lips and I examined other possible explanations for Florida’s gains. Huge spending increases (nope), decline in the percentage of low-income or minority students (nope-increases in both), preschool voucher program (nope- students too young to have aged into the NAEP sample) and class size amendment (nope- implemented very slowly, gains already well under way, formal evaluations negative) and retention law (scores continued to rise even as retention fell). This sort of information might be unhelpful if you are simply trying to get the idea in that something other than a set of hated reforms drove the gains.

Mathis however posited other types of “history” and noted other ways that the world had changed after 1998. On his list of other parts of uncontrolled ”history” with regards to Florida’s gains were Harry Potter books (kids reading more fiction) and the more widespread availability of personal computers at home.

Sadly, the format of the panel did not provide time for rebuttal. We had two other people on with us, and took questions from the audience. Had there been such time, however, I would have noted that while Arizona may seem backwards to outsiders (Dr. Mathis lives in Colorado) that we do in fact have Harry Potter books and even personal computers in our humble little patch of cactus. In fact, I am rather confident that Harry Potter books and personal computers became increasingly pervasive in all 50 states.

You never know, Harry Potter books could have powerful educational properties that only manifest themselves on massive peninsulas with high rates of humidity and large concentrations of alligators. The children of Arizona, landlocked in an arid climate, and with not much more in the large lizard department than the occasional Gila monster, may have been left behind. I can’t prove that this isn’t the cause after all.

Nevertheless I’m going to stick with my theory that Governor Bush’s success in implementing a varied and comprehensive set of K-12 reforms in 1999 served as the driver for the large increases in academic attainment seen in Florida’s NAEP scores since 1998. Dr. Mathis and his compatriots can continue to play their stategic nihlism game if they wish, ignoring the problems with their arguments and the studies most on point for the subject at hand (like the regression discontinuity studies of Florida’s retention policy).

Until they put forward a plausible explanation for Florida’s gains, I cannot for the life of me find any reason to take them seriously.

7 Comments | research reports | Tagged: , , , | Permalink
Posted by matthewladner

Florida Tax Credit Analysis find Participant Gains

August 31, 2011

(Guest Post by Matthew Ladner)

A careful analysis of test score gains by David Figlio of Northwestern University has found a modest but statistically significant gains for Florida tax credit students. The data in this study are messy, and Dr. Figlio admirably goes about sorting through the various issues in an even-handed fashion.

Figlio employs a regression discontinuity design to analyze the data, and his finding of a small but statistically significant academic gain fits quite comfortably with the larger random assignment literature, which find small year to year gains which accumulate over time.

One of the under-appreciated features of the random assignment literature: the studies usually fall apart after three or four years due to attrition in the control group. Our window into the academic benefits of choice is therefore limited. Figlio’s employment of a different analytical technique provides confirms previous findings, and may (?) open the door to longer term assessment. The challenges with the data described in this paper, however, suggest that it may not be easy.

Money quote from the study, with a definite echo of previous random assignment studies:

These differences, while not large in magnitude, are larger and more statistically significant than in the past year’s results, suggesting that successive cohorts of participating students may be gaining ground over time.

Good discussion of the results over at RedefinED, including a discussion of the baseline results (tax credit students are poorer and less Anglo). Emerson also puts this study in the context (Figlio also found positive public school effects associated with the Step Up for Students program).

So, the Step Up for Students program has now been found to help improve public school results, help improve participant academic gains, generates high levels of parental satisfaction. Sounds like a rock solid justification for expansion to me.

 

7 Comments | research reports, tax-credit scholarships | Tagged: , , , , | Permalink
Posted by matthewladner

Carr Makes It 19-0

August 17, 2011

This finding’s been replicated more often than Picard’s Earl Grey.

(Guest post by Greg Forster)

Still clearing the backlog: I haven’t had a chance yet to tout this new empirical study of Ohio’s EdChoice voucher program, by my old colleague Matt Carr, finding that – guess what, you’ll never believe this – vouchers improve outcomes at public schools!

Building on a large body of previous studies, this makes it nineteen (19) high-quality empirical studies finding school choice improves public schools and zero (0) studies finding it harms public schools.

Interestingly, Carr finds the positive impact is concentrated among the highest and lowest performing students. Since EdChoice is a failing schools voucher, you might expect schools to respond by improving service to those “bubble” students who are near the state proficiency cutoff. However, Carr finds the opposite.

Matt hypothesizes – plausibly enough – that schools are responding by improving services to the students who are most likely to use the voucher to leave. Low-performing students have the most obvious motivation to seek better services, while high-performing students are the most likely to have actively involved parents.

I do have one quibble with the study. Matt writes that his study “provides an analysis of a voucher program that has not yet been rigorously studied for its competitive effects on traditional public schools.”

Oh, really?

2 Comments | research reports, vouchers | Tagged: , , , | Permalink
Posted by Greg Forster

Podcast on Power

August 16, 2011

(Guest post by Greg Forster)

Still catching up on a travel backlog – in case you missed it, here’s my latest pod-type casting module over the inter-net system of tubes via Heartland. Main topic: the delicate balance between truth and power, and how the late unpleasantness shows the dangers to which some education reformers are already succumbing as they displace the minions of the blob as gatekeepers to the center of the conversation. Our old friend Jack Jennings, aka the human torch, also makes an appearance!

9 Comments | instructional reform, politics, research reports, tax-credit scholarships, vouchers | Tagged: , | Permalink
Posted by Greg Forster

NEPC’s attempt at strategic nihilism

August 4, 2011

(Guest Post by Matthew Ladner)

In the American film classic Animal House there is a scene where students smoke marijuana with their English professor, played by Donald Sutherland, and speculate that it could be the case that the molecules in your fingernails each contain a microscopic universe.

You can’t prove that there aren’t microscopic universes in your fingernails, after all, so they might be in there!

A nice post from Mike Petrilli on the Florida NAEP score gains prompted a response from Kevin Welner from NEPC that shows that the spirit of Sutherland’s Professor Dave Jennings is alive and well at the University of Colorado.

Again there is no attempt to address any of the gaping holes in retention theory. These holes include the fact that Florida’s 4th grade reading scores had improved substantially before the retention policy went into effect, and that they have continued to rise even as retention has fallen off substantially, and that they have fallen off substantially because of a very large improvement in 3rd grade scores.

Welner attempts to tiptoe around this by noting that our EdNext article addressing these points were addressed to a previous Walter Haney paper on the subject rather than the NEPC stuff, which is a distinction without much of a difference. The Chatterji paper contains a carbon copy of the Haney analysis. Amazingly, Chatterji dinged Burke and I for not doing a literature review (not the norm in our tribe) and then cites neither the Education Next paper nor Haney’s analysis. At best, she employed a double standard and at worst, she owes Professor Haney an apology.

Welner’s broader project is to attempt to use the causation problem as a shield. We don’t know, after all, exactly what caused Florida’s remarkable learning gains. Florida’s reformers had to implement their reforms in the real world rather than in a petri dish or in an Intention to Treat Random Assignment study. Welner believes that this allows him the opportunity for strategic nihilism:

The truth might be: (a) there are not actual improvements (the current study is too weak to say whether or not there are), (b) there are improvements, and they’re caused by a combination of all these things, (c) there are improvements, and they’re caused by something none of us pointed to (perhaps the green shirts??), or (d) there are improvements, and they’re caused by some of the things we’re pointing to BUT some of the other things we’re pointing to are actually harming students (just not enough harm to overcome the benefits of the other things).

In other words, when it comes to understanding the FL package of reforms, we are flying blind.

Welner is flying blind all right, but it is by choice. Let’s take each of these little gems on one at a time:

A. The NAEP results show very substantial improvements, as do other indicators.

B. I have always held that the exact cause for the improvement is impossible to know, because Florida’s reformers enacted multiple reforms simultaneously. The logical response to this is not to do none of the Florida reforms, but to do all of them.

C. Florida lurked near the bottom on NAEP for many years, enacted reforms in 1999, and then enjoyed sustained gains over time. While it could be the case that some mysterious X-factor caused the improvement, I’ve yet to hear a plausible theory regarding this. Dan Lips and I addressed multiple possibilities in the Education Next article, including demographic change, spending, etc, and found no evidence to support them.

D. This could be the case, but I haven’t seen a single scrap of evidence to suggest that it is actually the case- return to B above.

Welner is of course correct that there is a correlation and causation problem to consider. As a practical matter, there is nothing else to do but to carefully examine the evidence and history and draw the best conclusions that we can. Dan Lips and I did this in the Education Next article. Florida’s reforms coincided with the student population becoming poorer and less Anglo. State lawmakers increased funding per pupil, but it wasn’t by much and is still below the national average. NEPC complains about a lack of mention of the preschool voucher program when those kids have yet to age into the 4th grade NAEP sample. The class size amendment was implemented very slowly, long after Florida’s scores had begun to rise.

If Dr. Welner would like to provide a plausible explanation for why Florida’s NAEP scores increased so much after 1998, I’d be very interested to read it.

If he prefers to attempt to continue to play games, NEPC’s credibility will go on double secret probation.

 

 

 

 

4 Comments | research reports | Tagged: , , | Permalink
Posted by matthewladner

Speaking of Intellectual Corruption!

July 27, 2011

(Guest post by Greg Forster)

We now interrupt this serialized political philosophy dissertation on how power corrupts knowledge systems to bring you something completely unrelated – the latest nonsense from the increasingly irrelevant, lifelong Democratic party hack Jack Jennings!

Talk about weasel words. Studies clearly and conclusively finding modest positive impacts from vouchers magically become studies finding that voucher outcomes are “similar” to outcomes at public schools, which then magically become studies provding no “clear” evidence that vouchers have any impact. Never mind that most studies of vouchers have been conducted at Harvard, Princeton, Cornell, Stanford, the Fed, etc. – they’ve been conducted by “pro-voucher” researchers, which is technically true if you define any researcher who does a study finding positive impacts from vouchers as a pro-voucher researcher. (By the same method, you can prove that all studies finding cigarettes cause cancer were conducted by “anti-smoking researchers.”) Extremely minute scrutiny of studies by think tanks have found no scientific flaws in their work, which proves that even more minute scrutiny is obviously needed. And for some reason, a shift in the political winds away from supporting smaller voucher programs toward supporting larger voucher programs is evidence that vouchers are failing! Right, sure they are.

For the record, the empirical evidence supporting school choice is overwhelming – stronger than the evidence supporting any other reform policy. Impact sizes are sometimes large but usually modest, not surprising given that existing programs are tiny, underfunded and overregulated. But the evidence is consistent that school choice produces benefits even under these disadvantageous conditions.

Oh, and more good news: everyone’s onto Jennings’ game, so no one cares about his nonsense. Mickey Kaus:

At this point if Jack Jennings doused himself in gasoline and set himself ablaze in front of the NEA, would anyone notice?

I have Jennings’ professional reptuation hanging on the wall in my trophy room, right next to my Jay Mathews meal ticket.

4 Comments | research reports, tax-credit scholarships, vouchers | Tagged: , | Permalink
Posted by Greg Forster

« Previous Entries
Next Entries »
Follow

Get every new post delivered to your Inbox.

Join 940 other followers