Sneak Preview: Report Card on American Education

August 30, 2010

(Guest Post by Matthew Ladner)

Later this week the American Legislative Exchange Council will release Report Card on American Education: Ranking State K-12 Performance, Progress and Reform written by yours truly, Dan Lips and Andy LeFevre.

As suggested by the title, we grade each state by their academic performance, their academic gains and their K-12 reform policies. On the later, we use a “poll of polls” technique and average the grades assigned for particular policy areas on academic standards, teacher quality, charter school laws, private choice, digital education etc.

Sneak peak: a B+ was the highest grade.

On the performance and progress, we utilize NAEP with an eye to maximizing comparability  between states. After all, no one can be shocked that Connecticut has higher NAEP scores than Mississippi, given the huge disparities in income between the two states.

We therefore judge each state based on the scores of free and reduced lunch eligible general education students on all four main NAEP exams: 4th grade reading and math, 8th grade reading and math. We use the period for which all 50 states and the District of Columbia have participated in NAEP (2003-2009). Using free or reduced lunch eligibility keeps the income range of students under a known limit, whereas non-free and reduced lunch kids can vary in income from still relatively hardscrabble to billionaires.

We made no effort to control for race or ethnicity despite the well-known existence of racial achievement gaps. This is because we believe that such gaps can in fact be closed. We believe that the gaps exist due to policy and cultural factors, all of which can be changed. Schools in particular are in the business (or should be) of promoting a strong academic culture focused on learning-aka controlling the culture of the school.

You’ve never heard of a racial combat effectiveness gap in the United States Marines Corps because it doesn’t exist. The fact that the Marines are a well-led organization with a strong culture has a great deal to do with that, as does the fact that every Marine is a part of the Corps by choice.

In any case, we do not claim that our NAEP rankings provide perfect comparability  just enormously better comparability  than looking at raw NAEP scores.

So you are dying to know whether your state rocked or sucked wind in the rankings. Calm down- pace yourself!

All will be revealed later in the week.


Administrative Bloat — Updated Coverage

August 18, 2010

Here’s the updated list of coverage of the report on administrative bloat in American universities that I wrote with Brian Kisida and Jonathan Mills for the Goldwater Institute:

Op-eds

Atlanta Journal Constitution

Austin American Statesman

Baltimore Sun

Indianapolis Star

AOLNews

News

Dallas Morning News

Indianapolis Star

USA Today

Chronicle of Higher Education

Arizona Republic

Arizona Daily Star

Sunshine News

Phoenix Business Journal

AZ Daily Sun

Arizona Capitol Times

Inside Higher Ed

Modesto Bee

Time

Kiplinger’s

Fort Wayne Journal Gazette

Baylor Lariat

Columnists / Editorials

The Economist

Forbes

Arizona Republic

Dallas Morning News

Milwaukee Journal Sentinel

Denver Post

Dallas Morning News

Selected Blogs

Instavision

National Association of Scholars

Instapundit

Cato@Liberty

Phi Beta Cons

Reason Foundation — Nick Gillespie

Reason Foundation — Lisa Snell

George W. Bush Institute

Pelican Institute

MacIver Institute

Nevada Policy Research Institute

The Five Year Party

Carolina Journal

American Council of Trustees and Alumni

The Volokh Conspiracy

Minding the Campus

And here is our rebuttal to ASU’s statement attacking the report.  The rebuttal works for most of the issues raised by other universities as well:

Our Rebuttal


Administrative Bloat is Here

August 17, 2010

Here’s the updated list of coverage of the report on administrative bloat in American universities that I wrote with Brian Kisida and Jonathan Mills for the Goldwater Institute:

Op-eds

Atlanta Journal Constitution

Austin American Statesman

Baltimore Sun

Indianapolis Star

AOLNews

News

Dallas Morning News

Indianapolis Star

USA Today

Chronicle of Higher Education

Arizona Republic

Arizona Daily Star

Sunshine News

Phoenix Business Journal

AZ Daily Sun

Arizona Capitol Times

Inside Higher Ed

Modesto Bee

Time

Kiplinger’s

Fort Wayne Journal Gazette

Baylor Lariat

Columnists / Editorials

The Economist

Forbes

Arizona Republic

Dallas Morning News

Milwaukee Journal Sentinel

Denver Post

Dallas Morning News

Selected Blogs

Instavision

National Association of Scholars

Instapundit

Cato@Liberty

Phi Beta Cons

Reason Foundation — Nick Gillespie

Reason Foundation — Lisa Snell

George W. Bush Institute

Pelican Institute

MacIver Institute

Nevada Policy Research Institute

The Five Year Party

Carolina Journal

American Council of Trustees and Alumni

The Volokh Conspiracy

Minding the Campus

And here is our rebuttal to ASU’s statement attacking the report.  The rebuttal works for most of the issues raised by other universities as well:

Our Rebuttal

[UPDATE — This page has been revised to mirror the updated coverage post, which is here .]


More On DC Vouchers

June 28, 2010

Over at the EdNext blog, Paul Peterson has a very thoughtful piece about the good news found in the recently released evaluation of the DC voucher program.  You can read Peterson’s article here.  Here is an excerpt, where Peterson explains how it is possible that the Program had such a strong effect on students’ ability to graduate, yet still showed no “conclusive” achievement effects.

“But how were such high graduation rates achieved, when voucher students learned no more than the other students?  The answer to that riddle is that the study shows exactly the opposite: Those who went to private school scored 4.75 points higher on the reading test, an effect size of 0.13 standard deviations.

Admittedly, that is not as big an effect as is the voucher impact on graduation rates, and it is only fair to point out that statistician purists insist that any finding, before it can be declared undeniably true, must have only 5 chances in 100 of being wrong. The chances that the reading impact is in fact phony are greater than 5—in fact they are 6 in 100–and so it must be declared—by the statistician purists who supervise reports by government agencies—that “there is no conclusive evidence that [the vouchers] affected student achievement (p. xv).”

But notice the wording—there is “no conclusive evidence.” That is quite  different language from saying there is “no evidence” that vouchers raised achievement.  Indeed, if you invested $1,000 every time you had 94 chances in 100 of picking the right stock—and only 6 chances of getting it wrong–as is the case here, then, with modern technology, you could become richer than Bill Gates by sundown.”

This is an especially interesting perspective when you consider that Congress is insisting that DC vouchers be killed, and that students should be returned to the DC public schools that have only 6 in 100 chances of doing better in terms of reading achievement.

And, in case you missed them, you can read the editorial that appeared in the Washington Post about the study here, Mike DeBonis of the Washington Post had an article about the study which can be found here, and you can see Ed Week’s coverage here.   None of these observers, however, provided the insight that Peterson’s analysis did.

(Guest post by Brian Kisida)


Vouchers and the Rising Tide

April 7, 2010

“A rising tide lifts all boats.”

(Guest post by Greg Forster)

I haven’t had a chance to read the details yet, but from the executive summary of the new results released today by the School Choice Demonstration Project, it looks like vouchers have done a good job of improving education for all students in the city of Milwaukee.

What? That’s not the way you heard it?

Of course not. Because the new result, taken in isolation from other information, simply says that after two years, the voucher students are making learning improvements about the same as public school students. The scores for the voucher students are higher, but the difference is not statistically certain.

However, let’s plug that into the larger universe of information. We know – from the very same research project – that vouchers are improving education in Milwaukee public schools. The positive incentives of competition and the improved matching of student needs to school strengths are causing public schools to deliver a better education.

So if the voucher students and the public school students are doing about the same, and vouchers are improving results for public school students, it follows that vouchers are improving results for everybody.

That, of course, is the consistent finding of a large body of research. The overwhelming research consensus is that vouchers improve public schools.

Also, let’s not forget that in several previous longitudinal studies, the results from the first one or two years were similar – the voucher students ahead, but the difference not statistically certain – and in those cases, in later years the difference always became statistically certain. It just took the accumulation of more data to reach the high bar of statistical certainty.

So here’s a toast to the great news that vouchers in Milwaukee are making everybody better off!


Education Reform and Colored Maps

April 5, 2010

 (Guest Post by Matthew Ladner)

I read Paul Johnson’s book Modern Times as an undergraduate, and I’ll never forget his description of the Age of Colonial Empire.

Johnson said that while indeed there were vast fortunes to be made for some, that Empire also entailed enormous costs.

In the end, he said, European Empires were about colored maps.

Well, here’s a colored map for you- 4th grade reading scores for Hispanics in Florida compared to statewide averages in other states, via our friends at the Heritage Foundation.

That’s quite an empire that Florida’s Hispanics are creating, don’t you think? Daniel Patrick Moynihan used to joke that high performance on NAEP was directly correlated with proximity to the Canadian border, but notice the long list of predominantly Anglo states on the map: Iowa, Minnesota, Oregon, South Dakota, etc. etc. etc.

Notice also the presence of some states that have been very proud of their K-12 reforms in the past: North Carolina and Texas for instance. Oh, and one of the winners of the Race to the Top, Tennessee. I don’t need to write a sardonic comment, because you’ve already thought of one on your own. Feel free to share it in the comments section.

Here’s another smaller, but rapidily growing empire:

Florida’s African Americans tied or outscored two statewide averages on the 2007 NAEP, but their empire extended to 8 states in 2009. The Deep South may want to pull their heads out of the sand, because they are next on the list unless they pick up their games.

Yes, I’m looking at you Alabama, Arkansas, South Carolina and Tennessee. Florida’s African American students have their hands on the Risk dice and are ready to invade you with their plastic armies. Florida’s Free and Reduced Lunch Eligible kids already outscore your statewide average for all students in your states.


Fordham Foundation on K-12 Economic Segregation

February 18, 2010

(Guest Post by Matthew Ladner)

Fordham has a new study on what they call “private public schools” aka schools that serve hardly any low-income children. Personally I prefer the term Economic Segregation Academies.

Yes kids, calm down, they have data for specific metro areas available online.  So much for the common school myth.


The Silence of the Lame

January 22, 2010

(Guest Post by Matthew Ladner)

Dan Lips on the mainstream media and the head start evaluation. Money quote:

When a Congressionally-mandated study released in 2008 found that President Bush’s favorite reading program was a failure, it was national news.  An article by Greg Toppo in the USA Today blared the headline “Study: Bush’s Reading First Program Ineffective” and reported that the results could be a “knockout punch” for the program.  Similar articles appeared in the New York Times (by Sam Dillon) and Washington Post (by Maria Glod).

But when a similarly devastating report was published last week that undercuts a pillar of President Obama’s education plans, none of these papers has bothered to report it.   As we have reported, the Department of Health and Human Services finally released the results of a national evaluation of the Head Start program that Congress mandated in the late 1990s.


Civic Knowledge Polling Controversy

November 12, 2009

 (Guest Post by Matthew Ladner)

Last summer, I wrote a study for the Goldwater Institute reporting the results of a survey in which we gave 10 questions from the United States citizenship exam to Arizona high school students. The results were terrible, with only about 3.5% of the district students scoring 6 or better correct (passing). A few months later, I replicated the study for the Oklahoma Council of Public Affairs (OCPA) with Oklahoma students, and the results were even worse.

Since then the survey firm I used, Strategic Vision, has come under attack from a group of critics. I believe that it started over a poll from the New Hampshire primary last year. I am in no position to judge the merits of the case involved. Over the past few weeks, the critics have turned their fire on the civics surveys. Strategic Vision’s most aggressive critic claims that someone has replicated the Oklahoma survey and found them far more cognizant of civics than the Strategic Vision reported, concluding that SV simply fabricated the data. Smelling blood, my old friend Leo Casey is waving the bloody shirt. Some are wondering how anyone ever bought into the results in the first place or are feeling buyer’s remorse.

A few comments seem in order.

First, both myself and OCPA are investigating the validity of the survey. We have asked for and received call logs for the surveys, and we are awaiting receipts for the marketing lists employed in the survey. If, it stands to reason, a polling firm were simply fabricating data it seems terribly unlikely that they would pay thousands of dollars for marketing lists. If there has been a fraud, myself, the Goldwater Institute and OCPA were all victims of it.

Regarding the question of “how could anyone have ever believed these results” people should keep an open mind and examine the available evidence.

For the Arizona study, we purchased a poll of both public and private school students. The public results were terrible, but so too were the private school results. If memory serves, 3.5% of the public school students scored six or better and 13.8% of private school students scored six or better. We reported the private results in the study, and essentially characterized them as simultaneously better than the public results and still catastrophic.

If SV were simply manufacturing data, it seems at a minimum odd to make the private school results so terrible.

Second, results from other exams of civic knowledge should be considered. The Intercollegiate Studies Institute has run a series of tests on the civic knowledge of university students. You can look the results of the 2006 study here for both Freshmen and Seniors.

The average score for a freshman at Yale: 69.8%.

Harvard: 67.8%

Princeton: 66%

University of Texas at Austin: 53.8%

University of Michigan: 52.1%

These are all highly to uber selective universities, but strangely enough their students are arriving (and leaving btw-check out the Senior results) profoundly ignorant of American civics. In fact, the Ivy League kids, even the Seniors, score significantly worse than the alleged replication scores (no details provided btw)  in Oklahoma, where the kids supposedly got almost 80 percent correct.

Oklahoma high schools > Ivy League. Things that make you go hmmmmmm…

Next, it is worth considering the differences in testing method. The ISI results were given as multiple choice exams. If they ask you to name the first President of the United States, George Washington is jumping up and down right behind the letter “B.” You’ve really had to have been living on another planet not to get that one right.

The Arizona and Oklahoma surveys, however, emulated the methodology of the United States citizenship exams, directly lifting the questions from INS item bank, and employing their open-answer format. When you are asked “who wrote the Declaration of Independence” it is necessary to answer “Jefferson” without the benefit of four names, one of which is “Thomas Jefferson” literally staring you in the face.

In short, we have no reason to believe that the average high school student to be anywhere near as well-informed as the average student at highly selective universities. As it turns out, students at even highly selective universities know embarrassingly little about civics. Moreover the open answer format of the United States Citizenship Exam represents a higher hurdle of knowledge than a multiple choice exam. You are either carrying around the knowledge of how many Supreme Court Justices there are around in your head, or you aren’t. With a multiple choice exam, you’ve got a shot, but with an open answer format…good luck.

So my request to everyone is to stay calm and give us time to run the traps on this. If I got snookered, I’ll own up to it, but the jury is still out.

 


New Study on Florida Tax Credit Scholarships

August 6, 2009

FL survey table

(Guest post by Greg Forster)

Today the Friedman Foundation releases a new study I co-authored with Christian D’Andrea on the Florida Tax Credit Scholarship Program. You may recall that program as the subject of last month’s rush to judgment.

At the top of this post you can see what the parents participating in the program report about the services they previously received in public schools, and the services they are now receiving in the school choice program. The study conducted a survey of over 800 families randomly selected from the entire population participating in the program, excluding only those who had no prior public school experience (because their children entered the program in kindergarten).

The numbers tell the story. Public schools didn’t deliver for these kids, and school choice does – in spades.

Obviously this doesn’t answer all questions about the program. Indeed, as the first empirical study ever completed on a tax-credit scholarship program (that is, the first to empirically measure the outcomes of such a program measured against a relevant standard of comparison), it hardly could. We all look forward to the completion of the official evaluation when it’s ready. Until then, however, we have to take the information we have. And, if I do say so, I think this is some pretty important information.

Here’s the executive summary:

This study examines the Florida Tax Credit Scholarship program, one of the nation’s largest school choice programs. It is the first ever completed empirical evaluation of a tax-credit scholarship program, a type of program that creates school choice through the tax code. Earlier reports, including a recent one on the Florida program, have not drawn comparisons between the educational results of public schools and tax-credit scholarships; this study is therefore the first step in evaluating the performance of this type of school choice.

The Florida program provides a tax credit on corporate income taxes for donations to scholarship-funding organizations, which use the funding to provide K-12 private school scholarships to low-income students. Over 23,000 Florida students are attending private schools this year using these scholarships. Similar programs exist in Arizona, Georgia, Iowa, Indiana, Pennsylvania and Rhode Island.

Studying a tax-credit scholarship program using traditional empirical techniques presents a number of methodological challenges. To overcome these difficulties, the study used a telephone survey conducted by Marketing Informatics to interview 808 participating parents whose children attended public schools before entering the program. It asked them to compare the educational services they received in public and private schools.

The results provide the first ever direct comparison between the education participants received when they were in Florida public schools and the education they receive in the school choice program.

Key findings include:

• Participating parents report that they receive dramatically better educational services from their current private schools than they previously received in public schools.

• 80 percent are “very satisfied” with the academic progress their children are making in their current private schools, compared to 4 percent in their previous public schools.

• 80 percent are “very satisfied” with the individual attention their children now receive, compared to 4 percent in public schools.

• 76 percent are “very satisfied” with the teacher quality in their current schools, compared to 7 percent in public schools.

• 76 percent are “very satisfied” with their schools’ responsiveness to their needs, compared to 4 percent in public schools.

• 62 percent are “very satisfied” with the student behavior in their current schools, compared to 3 percent in public schools.

• Most participating parents were dissatisfied with their public school experiences on most measurements, and are overwhelmingly satisfied with their current private schools.

• 58 percent had been “dissatisfied” or “very dissatisfied” with the academic progress their children were making in public school, compared to 4 percent in their current private schools.

• 64 percent had been “dissatisfied” or “very dissatisfied” with the individual attention their children received in public schools, compared to 3 percent in their current schools.

• 44 percent had been “dissatisfied” or “very dissatisfied” with teacher quality in public schools, compared to 3 percent in their current schools.

• 59 percent had been “dissatisfied” or “very dissatisfied” with school responsiveness in public schools, compared to 3 percent in their current schools.

• 62 percent had been “dissatisfied” or “very dissatisfied” with student behavior in public schools, compared with 5 percent in their current schools.

• Asked to rate their schools on a scale from one to „„ ten, 94 percent of participants gave their current private schools at least a seven, and 54 percent gave them a ten. Only 18 percent of parents rated their public schools seven or higher, and just 2 percent rated them at the highest level.

• Of the 128 parents whose children are not likely to be in the program again next year, 81 percent said that dissatisfaction with the program played no role at all in their decision, and 100 percent – all 128 of them – said the program should continue to be available for others even though they were not likely to use it again next year themselves.