The Proficiency Illusion

November 13, 2008

(Guest Post by Matthew Ladner)

I had a chance to see John Cronin from the Northwest Evaluation Association present on the Fordham Foundation’s study The Proficiency Illusion at the Arizona Education Research Organization conference last week. It was more than interesting enough to have me check out the study. From the forward by Checker and Mike:

Standards-based education reform is in deeper trouble than we knew, both the Washington-driven, No Child Left Behind version and the older versions that most states undertook for themselves in the years since A Nation at Risk (1983) and the Charlottesville education summit (1989). It’s in trouble for multiple reasons. Foremost among these: on the whole, states do a bad job of setting (and maintaining) the standards that
matter most—those that define student proficiency for purposes of NCLB and states’ own results-based accountability systems.

In short, the accountability and standards reform strategy has morphed into a pig’s breakfast. We’ve all known for some time that most states have failed to set globally competitive standards, and have monkeyed about with their cut scores. One of the revelations of the Proficiency Illusion (to me) is that many states have proficiency standards lacking internal consistency. For example, some states have incredibly low cut scores in the elementary grades, only to amp them way up in 8th grade. Parents will receive multiple notices saying that their child is “at grade level” only to shocked to learn later that they are well short.

Other types of problems exist as well. Two years ago at the same AERO conference, I saw a presentation showing that Arizona writing AIMS test had bell curves that stacked on top of each other rather than being horizontally linked across grades. In short, it was impossible to tell whether 4th graders were writing any better than 7th graders with the state exam.

Cronin’s presentation contained other insights- including just how arbitrary AYP can be. It depends hugely on the N requirement for subgroups state by state- some schools wind up with lots of subgroups and some don’t. This means that some relatively high performing schools miss AYP. In fact, Cronin demonstrated what I take to be a fairly common scenario where middle schools miss AYP but in which they perform at a higher level than all of the public school transfer options in the vicinity.

Checker and Mike go on to argue for national standards as a solution to these problems, but concede that it doesn’t seem likely. My modest suggestion on this front would be to adopt the A-Plus plan, and as states sought alternatives to AYP, to have the US Department require the creation of internally consistent standards as a starting point for negotiations. Given that the states would be able to determine their own set of sanctions (or lack thereof) I can’t see why an increase in rigor would be outside the realm of these discussions for states with absurdly easy to pass tests.

Deeply wedded to inconsistent standards? Fine- have fun with AYP and the 2014 train wreck.

In other words, if the feds would abandon Utopian nonsense like 2014 and the high quality teacher provision, they might be able to play a productive role in providing technical guidance and nudging states into better directions with their testing programs.

I am not a fan of NCLB, but even I will concede that it has to date had a net positive impact increasing transparency in public schooling. This will be lost, however, if the 2014 problem isn’t addressed, or if we go down the absurd road of portfolio assessments, and I do view transparency as vitally important.

The Proficiency Illusion shows us that much of the data we’ve been getting from state testing programs isn’t nearly as useful or reliable as imagined. This is a problem, and it must be addressed.

 

2 Comments | NCLB | Tagged: , , , , | Permalink
Posted by matthewladner

“It’s laissez-faire until you are in deep $#!t”

November 12, 2008

dead-bull(Guest Post by Matthew Ladner)

The brilliant Michael Lewis looks back at Liar’s Poker and the financial meltdown. Lewis worked for Solomon Brothers, which led the way to securitization of mortages in the 1980s. A must read…

3 Comments | economics | Tagged: , | Permalink
Posted by matthewladner

Republicans to Receive Congressional Bailout

November 10, 2008

(Guest Post by Matthew Ladner)

Good stuff

Leave a Comment » | economics | Tagged: | Permalink
Posted by matthewladner

Izumi on the Republicans and Education

November 9, 2008

(Guest Post by Matthew Ladner)

Sweet are the uses of adversity,
Which, like the toad, ugly and venomous,
Wears yet a precious jewel in his head;
And this our life, exempt from public haunt,
Finds tongues in trees, books in the running brooks,
Sermons in stones, and good in every thing.
As You Like It Act 2, scene 1, 12–17

Lance Izumi helpfully suggests a way forward for Republicans on education policy in the New York Times Education Watch Blog.

3 Comments | NCLB | Tagged: , | Permalink
Posted by matthewladner

Safe Harbor Won’t Stop the Race to the Bottom

November 5, 2008

head-in-the-sand

(Guest Post by Matthew Ladner)

Charlie Barone did an excellent job unpacking NCLB’s safe harbor provision in reaction to a report by the Center on Education Policy warning of the looming 2014 train-wreck (where 100% of students must be proficient).

I co-authored a study for the Heritage Foundation also critical of the 2014 requirement. My concern is that what states will obviously do is to simply dummy down their tests in order to comply. I believe this process has already started, and will accelerate as the passing requirements escalate.

Barone however unpacked the arcane “Safe Harbor” provision on NCLB. Safe Harbor basically gives you a pass on AYP so long as you can reduce the number of children scoring below proficient by 10%. If this represents an obtainable standard, then presumably states would not feel overly pressured to drop their cut scores.

So, I decided to put safe harbor to a test- a very forgiving one. I looked at data from Florida’s 67 school districts over the past six years on 4th grade Reading and Math. The NAEP shows that Florida has been making solid progress on both subjects.

So what happens if you assume that we are at the 2014 100% requirement, an apply safe harbor to the overall performance of Florida districts? Judging them solely on 4th grade math and reading, and without judging any of the required student subgroups– like the various ethnic groups, free and reduced lunch kids, special education children, etc- Florida’s districts failed to make safe harbor 71% of the time based on the overall results alone.

I’ll leave the debate over whether public schools should be able to routinely reduce their passing rates by 10% year after year to others. The point I am making is: they don’t. I am confident that if required to break down results by various subgroups, that not a single Florida district would have made AYP during these six years under safe harbor. The number of individual schools making safe harbor is also likely to be negligible.

This, during a period of strong statewide improvement in both reading and math.

The sad reality is that, unless changed, NCLB will create a tremendous and perverse pressure on the state of Florida and others to dummy down their state accountability exams. In the greatest of ironies, a bill which aspired first and foremost to create transparency in public education contains the seeds of its own destruction in encouraging states to lower passing standards.

Gene Hickok and I endorsed Senator Cornyn and De Mint’s A-Plus concept as a way out of this looming train wreck. Under it, states could negotiate a single set of testing with the feds: preferably one better designed than a one-size-fits-all system with the mother of all perverse incentives built in. The Department would have discretion as whether to accept a state model rather than AYP, so no one is proposing to burn down the Department of Education.

The administration was not enthused. Nor, to my knowledge, have they ever acknowledged that 2014 is a problem. Nor to my knowledge have they ever embraced anything that could be considered a solution. Refusing to recognize a problem and thus failing to take action against it while it is still relatively manageable- ring any bells?

The Democrats will run the department and Congress, and powerful constituencies in their coalition want to vastly increase federal funding and replace testing with “portfolio assessments.” Now would be an outstanding time for all who believe in public school transparency-Democrats and Republicans- to pull their heads out of the sand on this issue. Safe harbor is not a solution. If you don’t like A-Plus- fine, propose another solution. Pretending like there isn’t a problem is a recipe for disaster.

7 Comments | NCLB | Tagged: , , , , | Permalink
Posted by matthewladner

Two Days Later…Pain Still Raw

November 3, 2008

(Guest Post by Matthew Ladner)

The Houston Chronicle called it “one for the ages” and it is hard to disagree. The Texas Tech vs. Texas game was an instant classic- plenty of drama and late game heroics.

I’ve discovered that I like classic games better when my team wins. C’est la vie.

Tech deserves a ton of credit. Their coaches out-coached ours, and their players were lights out. Texas starts three defensive ends on the d-line (all the better to sprint through big spreads on the offensive line and crush the qb) so Tech ditched their line splits and (shocker) ran the football down our throat and controlled the clock. Golf clap for strategery!

It would be nice if the multi-million dollar Texas coaching staff would

A. Teach proper kickoff coverage technique.

B. Not call a slow developing run play from the end zone.

C. Milk as much time off the clock as possible on what is certain to be your last offensive drive.

D. Have your offensive linemen hold when the refs absolutely refuse to enforce the penalty.

E. All of the Above

If you answered “E” give yourself a gold star- not that I’m bitter or anything.

Despite all of that, Texas took the lead with 1:26 to go in the game, and came within a dropped interception and single tackle on the games final second of somehow prevailing. Congrats Raider fans- see you in Austin next year.

5 Comments | college sports | Tagged: , | Permalink
Posted by matthewladner

The Upward Surge of Mankind?

October 30, 2008

(Guest Post by Matthew Ladner)

Florida tripled the number of Hispanic and African American students passing one or more AP exams with a program that included a financial incentive for schools and teachers.

Meanwhile, C. Kirabo Jackson finds positive results for a similar Texas pilot program in Education Next:

According to my assessment, the incentives produce meaningful increases in participation in the AP program and improvements in other critical education outcomes. Establishment of APIP results in a 30 percent increase in the number of students scoring above 1100 on the SAT or above 24 on the ACT, and an 8 percent increase in the number of students at a high school who enroll in a college or university in Texas. My evidence suggests that these outcomes are likely the result of stronger encouragement from teachers and guidance counselors to enroll in AP courses, better information provided to students, and changes in teacher and peer norms.

Gordon Gekko for Secretary of Education? I can see the confirmation hearing speech in my head:

The point is, ladies and gentleman, that greed — for lack of a better word — is good.

Greed is right.

Greed works.

Greed clarifies, cuts through, and captures the essence of the evolutionary spirit.

Greed, in all of its forms — greed for life, for money, for love, knowledge — has marked the upward surge of mankind.

And greed — you mark my words — will not only save public education, but that other malfunctioning corporation called the USA.

Just kidding, but I will say this: we need to continue experimenting with programs like this. They certainly seem to beat throwing money at schools in the hope that they will improve.

 

2 Comments | civic values, research reports | Tagged: , , | Permalink
Posted by matthewladner

Illiterocracy?

October 24, 2008

(Guest Post by Matthew Ladner)

Idiocracy is a silly Mike Judge movie about an average Joe who finds himself shot into the future, where the world has suffered a catastrophic decline in mental ability over time. The protagantist, played by Luke Wilson, finds himself to be a relative super-genius, as the denizens of the future can barely speak a sentence and sit around all day watching mindless television programs. Our hero overcomes adversity to become President.

Every once in awhile, I see something that brings that movie to mind.

I’ve written about Arizona’s lowering of AIMS cut scores in order to game school accountability. AIMS however has four levels of achievement: Below Standard, Approaches Standard, Meets Standard and Exceeds Standard. The “Meets” category of course has been the focus of lowering cut scores.

Let’s take a look, however, at the more stable category of “Exceeds Standard.” Cut scores have been much more stable at the top. Figure 1 below presents data from the Tucson Unified School District Reading AIMS. The figure presents the percentage of TUSD 3rd and 6th grade scoring “Exceeds Standard” on the Reading AIMS for the Class of 2012.

For those without an abacus on hand, that’s an 88% decline in the percentage of children scoring at the advanced level between 3rd and 6th grade. Strangely, TUSD finds enough money to spend on a “Raza Studies” program in the midst of such catastrophic failure.


Any interest in running for Governor of Arizona?

Figure 2 presents the statewide figures for all public schools in Arizona for the Class of 2012 for the same grades and years.

Figure 2 presents an 80% statewide decline. Attendance in a typical Arizona public school seems injurious to the ability to perform high-level reading at grade level, at least according to the state’s own standards and measures.

8 Comments | instructional reform | Tagged: , , | Permalink
Posted by matthewladner

Black Market Private Schooling in the Third World

October 23, 2008

 

(Guest Post by Matthew Ladner)

 

Jay, Greg and I all had the chance to see a presentation by Pauline Dixon on private schooling in the third world at a recent conference sponsored by the Friedman Foundation and Clemson Institute for the Study of Capitalism. The information was very compelling.

 

Dixon and her co-author James Tooley explained their research in the 2005 Cato Institute report Private Education is Good for the Poor. Their two-year in-depth study in India, Ghana, Nigeria, and Kenya found amazing results.

The first component of our research consisted of a systematic census and survey of all primary and secondary schools, government and private, in selected low-income areas. The second component examined a stratified random sample of between 2,000 and 4,000 children from each of those areas. Tests in mathematics, English, and (in Africa) one other subject were administered. Children and teachers were also tested for their IQ, and questionnaires were administered to students, parents, teachers, and school managers or headteachers.

What did they learn? For starters, a large majority of students in each of the low-income areas studies in all four countries were attending private, not public schools. For instance, the census of the low-income area of Ghana found 779 schools total, only 25 percent were government schools. The census found that 64 percent of schoolchildren in the area attended private school. In the surveyed area of Nigeria, 75% of students attended private schools.

Interestingly, the majorities of these schools are unregistered, and in many cases illegal. They are neighborhood proprietary operations, run almost exclusively based upon student fees. The vast majority of these schools and students receive no public subsidy whatsoever. In fact, many of them must pay bribes just to stay in business. Many of these schools provide subsidized spaces for the poorest of the poor, and all of them are the precise opposite of the Dead Poet’s Society stereotype of private schooling.

As you can see from the photo above (taken by the research team), these schools are not housed in fancy facilities. In fact, as Dr. Dixon presented a power point presentation of these schools, it brought to mind the old story about Abraham Lincoln learning to read by writing on a shovel with coal.

The story here isn’t that there aren’t public schools to attend- it’s more that those schools are dysfunctional to an unimagined (by me anyway) scale. Parents told the researchers of rampant absenteeism by teachers, meaning that children simply run wild when their teachers decide not to report- which is frequent.

My impression: the public school system in these countries represent blatant jobs programs, rather than schools. This impression seems borne out by the test score results:

The raw scores from our student achievement tests show considerably higher achievement in the private than in government schools. In Hyderabad, for instance, mean scores in mathematics were about 22 percentage points and 23 percentage points higher in private unrecognized and recognized schools, respectively, than in government schools. The advantage was even more pronounced for English. In all cases, this achievement advantage was obtained at between half and a quarter of the teacher salary costs.

So there you have it- much better and much less expensive. Oh, and often illegal and great for the poor.

11 Comments | myth of helplessness | Tagged: , | Permalink
Posted by matthewladner

Memo to Rhee: McKay Scholarships for DCPS!

October 19, 2008

(Guest Post by Matthew Ladner)

Erin Dillion over at the Quick and the Ed has posted a copy of a table on Special Education private placements from a recent research report on parental choice in Washington DC (see below).

So, is it just me, or would a McKay Scholarship program be a fantastic idea for DCPS? Rich kids are already getting private placements with the help of specialized attorneys, often Cadillac placements. McKay creates a maximum amount per pupil matched to their funding, and democratizes the private placement process: no lawyers necessary.

DCPS, like other school systems, has likely been complaining about “inadequate” special ed funding for decades. Typically this narrative involves shunting millions out of general ed funding into special ed. In DC, this story has been reinforced by private placement lawsuits, which often result in far more than the normal per capita funding, and create legal costs.

A  approach would substantially improve this. No lawsuits, no massive awards. If the parent of a child with a disability is disatisfied, they simply walk away with a predetermined maximum amount.

Alternatively- we can leave private placements as mostly something the rich white kids with attorneys access, at a sustantial cost to DCPS.

Leave a Comment » | vouchers | Tagged: , | Permalink
Posted by matthewladner

« Previous Entries
Next Entries »