Netherlanders over alle, of bijna alle

January 21, 2015

(Guest Post by Matthew Ladner)

New Atlantic piece on education in the Netherlands, where teachers call the shots, anyone can open a school, parents have choice, and religion is treated with neutrality rather than viewed as a public menace. Any guesses on what teacher job satisfaction looks like in the Netherlands? How about student test scores? How much do they spend per pupil compared to us?

What is that Mr. DNA OECD? You can answer some of these question? Why yes, please do!

 


Oklahoma Drama Full of Sound and Fury but ends in a whimper

December 28, 2014

(Guest Post by Matthew Ladner)

One bit of 2014 business to address: Oklahoma got their NCLB waiver back after their universities certified their previous standards and tests (which compared relatively well to NAEP) as college and career ready. Note that the story includes the nugget that 40% of Oklahoma college students require remedial education despite those highly thought of standards and tests. Paging Dr. Loveless, Dr. Hanushek! It’s possible that it would have been even worse without good standards and tests but to borrow a line from Sol Stern, good standards and tests are not enough.

On the process issue conditional waivers exceed the authority of the United States Secretary of Education and constitute a piece in a larger mosaic of an attempt to rule by administrative fiat. Unlike the Hotel California, however, you can both check out and leave the CC either without penalty (Oklahoma) after jumping through a hoop. Alternatively states can call Secretary Duncan’s bluff and simply drop their NCLB waiver because the consequences just aren’t that big of a deal (Washington- still no waiver riots on the streets of Seattle). It might at some point occur to someone in Washington or some future waiver-dropping state to file suit over conditional waivers.

It appears that the Secretary has been bluffing with a weak waiver hand, smiling to himself as states all-too-eagerly fold. States wishing to leave the CC however should give some thought as to what they would like to get out of their state testing system rather than adopting a “shoot-ready-aim” approach. My Little Pony connect-a-dot tests may not merit the approval of state university systems, and unlike Oklahoma not all states have decent systems to fall back on. The exit door however is clearly open.

 

 

 

 

 

 

 

 

 


The Anti-Testing Zombie Apocalypse

December 1, 2014

Grrrrrr….testing ruin flavor of BRAINSSSZZZSSSS!!!!

(Guest Post by Matthew Ladner)

While some of the strongest supporters of standardized testing have allowed their minds to wander to counter-productive uses of overstretched waiver authority in the already dying days of a lame-duck administration, rumors have reached my ears of growing support for eliminating annual testing as a requirement under NCLB in Congress.

This may seem implausible to some, but after watching a 30+ year bipartisan consensus on transparency fold like a house of cards in Texas, nothing seems impossible. Discuss among yourselves…

The hour is later than you think…


The Trouble with Baking Achievement Gap Measures into State Accountability Systems

October 17, 2014

(Guest Post by Matthew Ladner)

On March 9, 1974 Yoshimi Taniguchi, a Japanese book merchant and former Major in the Japanese Imperial Army, traveled to the Philippines in order to order a former subordinate, Second Lieutenant Hiroo Onoda to stand down from combat operations. Lieutenant Onoda had received news of Japan’s surrender in World War II, but had concluded that it was mere enemy propaganda. Firm in this misapplied conviction, Onoda carried on a long since concluded war for almost 30 years.

Onoda may have misspent three decades in an island jungle, but on a positive note he at least inspired an episode of Gilligan’s Island (Ginger in fact defeated his doppelganger in a judo battle).  More disturbingly, he may have also passed his die-hard spirit on to NCLB’s last stalwart defenders of achievement gap mania and the dream of mandated perfection.

I should begin by saying that I do believe that achievement gaps are very important. I went back to review a post I wrote a blog post three years ago called In Defense of Achievement Gap Mania and found nothing that I had changed my mind about on this subject.  Getting low-performing American students on a faster academic pace is of the utmost importance.

I am however mystified by the Onoda-style defense of NCLB’s unworkable division of schools into multiple subgroups with targets for narrowing and ultimately closing all achievement gaps on the equivalent of a train schedule. That would have all been enormously beneficial if it had worked, but let’s just say that the trains weren’t showing much sign of arriving on time, sometimes at all. We could discuss various ways states found to escape from the NCLB subgroup noose at length (hello n-size manipulation!) but there are deeper problems to discuss.

Today we find NCLB diehards sprinkled throughout the K-12 reform conversation archipelago. In this recent post on Eduwonk, Anne Hyslop  goes Banzai! corrects factual problems in a recent New York Times story focusing on the flaws of NCLB. My own take on this is that Hyslop is probably completely correct in her assertions. I however believe that they are largely irrelevant to the bigger picture.  When I read a paragraph explaining how technical mumbo jumbo safe harbor confidence intervals actually mean that the mandated 100% proficiency mandate doesn’t actually arrive until 2016 instead of in 2014, it makes me chuckle. Do we imagine that when educators read the fine print they will rush to embrace NCLB’s machine of mandated universal proficiency with open arms? Or would it be more accurate to say that many educators never took AMO schedules seriously in the first place, confident that they would be dropped (they were- through the waiver process).

Ed Trust released a report recently critical of NCLB waivers. I personally don’t like NCLB waivers either given the Secretary’s lack of obvious authority to grant conditional waivers, but that is not what has Ed Trust excited. They think that the Secretary ought to have used his non-existent conditional waiver authority to mandate gap closing measures into state accountability systems.

Stand down Lt. Po…that’s an ORDER!

Ed Trust is careful towards the end of their study to say that they are NOT calling for a return to NCLB’s multiple pathways to failure based on myriad subgroups in pursuit of mandated improvement on a schedule. Hey its 2014 and they aren’t that crazy, you see, they just seem to want Secretary Duncan to work something out with these states that will be the functional equivalent of NCLB’s multiple pathways to failure based on myriad subgroups in pursuit of dramatic improvement by mandate on a schedule. That would do just fine.

Ed Trust (and others) seems sufficiently wedded to NCLB-era mechanics that they dislike an elegant improvement-the super subgroup. Florida policymakers grade schools half on proficiency (the % passing state exams) and half on student progress over time. They double-weighted the importance of the progress of the lowest performing 25% of students from the last year’s test. The students falling behind thus constitute the “super subgroup” and they became the most important students in the building for determining a school’s grade. They count against all three of the main three components of a school grade: overall proficiency, overall growth and the growth of the students who have fallen behind.

The super subgroup doesn’t ask whether you are White, Black, Asian, American Indian, economically disadvantaged, an English Language Learner or a child with a disability, blue-eyed or left-handed. It simply identifies the lowest performing children in the school and puts a special emphasis on their academic gains over time.  It doesn’t create a perverse incentive to ignore an academically struggling child because he or she happens to be White, or because his or her parents make a little more money than this year’s Free and Reduced Lunch standard or because you’ve been reclassified out of SPED. Done properly, super subgroup creates a powerful incentive to identify struggling students regardless of their appearance and/or circumstances and get them making progress.

Oh, and the Ed Trust’s own previous research would lead one to the conclusion that it can help reduce achievement gaps. Not eliminate achievement gaps on a train schedule, mind you, but to make substantial progress on them by creating an incentive for schools to get struggling students to catch up. Who are the kids struggling? Why it is the kids on the short end of the achievement gaps as a matter of fact.  Florida kept A-F school grading up over a good period of time and you see gaps narrow in the best way possible- bottom scoring kids making greater progress than the still progressing top scoring kids.

The Ed Trust report tut-tuts things like Black students in A graded Florida schools scoring lower than White students in C graded schools as evidence that we ought to be including gap closure in state accountability systems.

Should we work ourselves into a froth about this? I personally don’t think so. Ed Trust’s own research has documented Florida’s overall progress in narrowing achievement gaps, but it’s not like they have eliminated achievement gaps. Would anyone be shocked to learn that ELL students at A graded schools score lower than non-ELL students at C schools? What about children with disabilities? Low-income children?  Ed Trust focused only on three states, but you could find similar results in any state.

The super-subgroup mechanism creates an incentive to get all students who have fallen behind to make academic progress. A moment of reflection regarding grading schools based upon various achievement gaps would give any thoughtful person pause. Do we really want to bake perverse incentives to stall the progress of high performing students into state accountability systems? Under the super sub-group, schools have any incentive to get any child that has fallen behind back on track. If states began rating them based on trends in achievement gaps, they could create perverse incentives to ignore their plight if they happened not to have a disability, or if they were a native English speaker, if their family made too much money for a free or reduced lunch, or if they were White.

Against this backdrop, the Ed Trust report seems strategically vague- not in favor the NCLB AYP system, but vaguely in favor of including achievement gaps in state grading systems. The fuzzy nature of these recommendations deftly avoids discussion of how to avoid creating cringe-inducing perverse incentives.

We live in a nation where Black and Hispanic students score closer on PISA to students in Mexico to those in South Korea or their own Anglo classmates. Mexico, btw, has far greater poverty and far lower public school spending than the United States. This is sickening, but we should exercise good judgment in addressing it. Previous Ed Trust research and the NAEP both show that it has achieved commendable gains in narrowing achievement gaps in Florida. In the country as a whole, not so much.

Thinking more broadly, we should recognize the NCLB era as a decentralized learning process. While NCLB created a general accountability rubric, many states had already created accountability systems of their own, creating the opportunity to learn from variations in policy approaches. Florida paid far more attention to school grades than to NCLB’s AYP and achieved greater than average gains among traditionally disadvantaged student groups.  I’m not a fan of conditional waivers, but we need to study and learn from the successes and failures of the diversity of approaches as best we can as well. It is understandable that there are many with a deep investment in NCLB, but we should not allow that attachment to blind us to something more effective at achieving its aims. The importance of achievement gaps should lead us to adopt the most effective methods for reducing them rather than pining for the ones we had hoped would eliminate them in short order.

 

 

 


Bringing Some Coherence to Arizona State K-12 Policy

August 3, 2014

(Guest Post by Matthew Ladner)

The Arizona Republic ran a column by yours truly today calling for making the Arizona Department of Education an executive department reporting directly to the Governor rather than having a separately elected position.

Free from the word limitations of columns, I can provide a bit more detail.

Arizona’s student data system typifies what I describe in the column as dysfunctional policymaking. Officials from school districts and charter schools have complained bitterly about it for many, many years as being riddled with errors. It often crashes, and has remained down for months at a time.  I have spoken to district officials who told me that they despaired of the state system ever working and created their own data system as an attempt to cope. All accounts I have heard say that Superintendent Huppenthal has done a good job in making the best out of an awful situation, but one highly placed source described it as getting inaccurate information to you at a faster pace. The whole system, in short, needs to be replaced, as in at least a decade ago.

Okay, so go and get a new data system, right? Wrong. We’ve seen an endless cycle of finger-pointing on the subject. The Department has requested money to fix the system, the legislature has never provided these funds. One can draw the inference that the legislature has collective doubts about the money being put to good use and, well, it is hard to blame them.

Now imagine a world in which former Governor Janet Napolitano had appointed the head of the department. I had my problems with Governor Napolitano philosophically, but I had no doubts about her overall competence. In this alternative reality, a clown car show of a data system reflects badly on her. People that can’t get basic state functions settled don’t win plum assignments like Homeland Security and the University of California system. Moreover, she would have had her own people in charge and the ability to secure the required financing to get a decent system in place during budget negotiations. The means meets motive, problem solved. It would have been a much better and more enduring K-12 legacy for Governor Napolitano than what turned out to be unsustainable school funding increases. Ah, what use we could have made of the aughts…

Back here in the Arizona of the real world, we had, um, this going on during the aughts.  As you might imagine, it did not end well, nor inspire confidence in the legislative branch.

Honk! Honk! Data system coming through!

At some point, a problem goes on long enough to qualify a symptom of a much deeper disease. Your local Target has a data system that can track every purchase and reorder lollipops just as they start to run low. In 2014, a reliable and secure student data system isn’t exactly rocket science, but it continues to elude Arizona. Appointing a Superintendent won’t dry every tear or fix all of our problems. It may be more necessary than sufficient, and we really need to make good use of the next four years regardless of who wins the next set of elections.

In the end it may not happen because it just makes too much sense, but hope springs eternal even as time grows short.


Fordham Continues to Advocate Playing with Fire

June 25, 2014

(Guest Post by Jason Bedrick)

Imagine the following playground scenario:

Tommy: Hey guys, I have a great idea! Let’s all go play with fire! It’ll be fun!

Cate: No way, Tommy. Playing with fire is very dangerous. Someone could get hurt!

Jay: Cate’s right. I used to think playing with fire was a good idea, but I’ve seen other kids get burned.

Milt: Yeah, plus, there are lots of ways to have fun without playing with fire!

Tommy: Friends, you’ve taught me an important lesson about the dangers of fire. Okay, here’s my new idea: let’s all go play with fire, but if other kids don’t want to, then playing video games is totally cool too. How’s that sound?

If you find Tommy’s response puzzling, then you’re likely to find the Thomas B. Fordham Institute’s “revised” approach to educational choice and accountability equally puzzling.

In the debate between parental choice and top-down government mandates, the Fordham Institute follows Yogi Berra’s advice: “When you come to a fork in the road, take it!” Fordham supports choice, but argues that the only way to prevent parents from choosing “bad” schools is to regulate them out of existence. In January, Fordham released a “toolkit” for policymakers that advocated requiring all private schools to administer the state test (i.e. – Common Core) and publish the results as a condition of accepting school vouchers or even tax-credit scholarships. Lower-performing schools would be forbidden from accepting students with vouchers or scholarships going forward.

Fordham’s proposal elicited a torrent of criticism. Andrew Coulson, Director of the Center for Educational Freedom at the Cato Institute, and I argued that their approach would stifle educational diversity and innovation. Jay Greene noted that standardized tests capture only a fraction of the benefits of educational choice. James Shuls of the Show-Me Institute pointed to the evidence that parents hold a range of legitimate views regarding what constitutes quality. Robert Enlow, President of the Friedman Foundation, reminded Fordham that such top-down accountability has not worked in government schools—something that Fordham itself once lamented when it called certain test-based accountability measures an “illusion.” Rick Hess of the American Enterprise Institute expressed concern that without any clear limiting principle, mandating state tests could easily lead to mandating “certified teachers, a state-approved curriculum, state-approved facilities, a state-approved plan of emergency services,” etc.

Last week, Fordham’s incoming Executive President, Michael Petrilli, offered what he called an “olive branch” to Fordham’s critics:

While we didn’t agree with the all of the arguments forwarded by our friends, we did come to see the risk to private-school autonomy and innovation that a test-based accountability system could create. We also understood the particular sensitivity around using Common Core tests for this purpose.

Petrilli then explained that Fordham has updated its “toolkit” accordingly. But if you expected that recognizing “the risk to private-school autonomy and innovation” would mean abandoning the push to mandate state assessments (i.e. – Common Core tests), then Fordham’s “revised” approach will leave you scratching your head. In the “revised” toolkit, Fordham recommends that state policymakers:

Require that all students who receive a voucher (or tax-credit scholarship) participate in state assessments. (While we prefer state assessments as policy, we think any widely respected test that allows for ready comparison against other schools or districts is a reasonable compromise);

In case you missed it, Fordham’s “revision” is in the parentheses. Like little Tommy, Fordham claims to recognize the risk of playing with Common Core fire but continues advocating for exactly that (unless they need to compromise for political purposes, in which case other tests are totally cool “a reasonable compromise”). If Fordham truly recognizes the “risk to private-school autonomy and innovation” that Common Core poses, then why is it still calling mandatory Common Core testing as an initial preference?

Petrilli concluded by calling for “a round of Kumbaya” and then getting “back to work on expanding great educational options to lots more children nationwide.” However, expanding educational options should mean more than just which school best teaches to the Common Core tests. By all means let’s work on expanding educational options… but let’s do it right.


Let’s Search for Sweet Spots, but with modesty please

June 5, 2014

(Guest Post by Matthew Ladner)

I have a number of friends who have either helped develop or have signed on to a Statement of Principles regarding a three sector reform strategy and what they view as a desirable level of state oversight of private choice programs.  This post will work better for you if you go and read the document first.

The needle starts to scratch across the vinyl for me at:

Even with the expanded choice to the private sector, they also have produced modest results.

This has become a mantra in recent years, but I believe that this statement reflects an incomplete understanding of the research results, and specifically a lack of understanding regarding our random assignment studies of voucher programs. The basic takeaway from the random assignment studies in my view is as follows: the test score impacts are modest but often statistically significant within the three year window that we can reliably study them.

So the Milwaukee Parental Choice Program offered $6,400 vouchers to very low-income inner city parents whose other options were to attend a district spending $14,000 per child and/or charter schools spending somewhere between the voucher and district spending.  We have several random assignment studies of the test score impacts that find that the experimental group basically stays on grade level (a triumph for poor inner city children) whereas the control group declines year by year.  You get to watch this process for about three years before the random assignment breaks down on you.

What happens to test scores after Year 3?  No one knows for sure- these studies fall apart over time.  We do know things however about what happens regarding high-school graduation, college attendance, college persistence, etc.  Borrowing a slide that Pat Wolf presented at the Alliance for School Choice conference:

Slide11

So basically you are less likely to graduate in 5 years (first red column) because you are more likely to graduate on time, less likely to graduate from a two-year college (second red) because you are more likely to be going to a four-year college.  The blue columns are all positive impacts from having been a choice student.

Now if you are determined to cling to the “modest” camp by saying that you wish these impacts were even larger, well, I do too.  I also wish that Chuck Norris’ tears really did cure cancer.  At this point it might be appropriate to raise the question as to just how much a positive impact we should reasonably expect from a program giving profoundly disadvantaged children a $6,400 coupon.  Although we don’t know what happens after a few years of random assignment study, those graduation figures ultimately are far more important than 6th grade math scores.

Being far more likely to graduate from high-school and college for less than half the money sounds like a triumph to me, albeit one that we could and should hope to improve upon through more robust program designs.  The standard here should not be to expect MPCP to transform every last profoundly disadvantaged inner city child into a Dean’s List Ivy Leaguers.  Rather in judging the impact of MPCP we should look at it on a return per dollar invested basis.  When you look at it appropriately through this ROI it is clear that the return on MPCP has been quite good, and that we should be looking for ways to get even more of it.

Then I got to this statement:

We know that smart accountability measures can ensure that public money and young lives are not invested in low-performing private schools.

The statement offers no evidence to support this claim, and moreover the claim itself dodges the more important question of costs and benefits to regulation.  Is it possible for “smart” accountability to keep young lives out of low-performing private schools?  Sure it’s possible.  Smart training can ensure that I could go from being a 46-year-old policy wonk to heavyweight champion of the world. I mean it is possible right? Is it also possible, even highly likely, for the whole enterprise to go south on you in a variety of different ways? Yep, that’s very possible too.

Who is going to administer these smart accountability measures and who will administer them a few years later?  What about 25 years from now?  How often will these people do something they think is smart which proves to be otherwise?  Unless we want to have the Federal Reserve administer these programs, how long will it be until politics will subvert the process of “smart” technocratic policymaking?  Also like the Fed, the costs of technocratic mistakes may prove quite costly.

Even well-intentioned efforts at “smart accountability” could easily backfire.  Let’s take Louisiana as an example.  Louisiana policymakers decided to grade all their schools A-F based upon a state accountability test tied to the state academic standards, and then decided to create a mechanism to remove low-performing schools from eligibility to take new students.  This probably sounds clever at a Georgetown cocktail party, but in Louisiana two-thirds of the state’s private schools have decided to stay out of the program, denying thousands of seats to low-income children attending relatively poor performing public schools in one of the lowest performing states in the union.

Ooops.

Let’s take things a step further. Is it possible that the one-third of Louisiana private schools that chose to participate in the program may have had a selection bias towards being more on the financially desperate side than those that have decided to stay out?  I have no data to support that this in fact did happen, but who would be surprised if it in fact did happen?  The correlation between financial desperation and academic ineptitude often proves strong.  In such a case the initial impact of the regulatory regime might have precisely the opposite of what was intended with many higher performing schools choosing to keep their distance.  Worse still, it might create an incentive for private schools to engage in the same sort of gaming strategies that have been common in states with rising state test scores but flat NAEP scores- teaching to test items rather than to standards (Arizona is waving hello!).  Finally of course it is no triumph if the schools do actually teach the state standards because the whole idea of a choice program is to provide, well, meaningfully varying choices for parents.  If you want state tests and standards in Louisiana you already have thousands of options available to you in the form of district and charter schools.

In the end of the day, policymakers must make decisions about where to draw the line in such matters. We have no wrong or right answers here, only preferences. Personally I believe that choice programs should provide academic transparency to the public in ways designed to have the lightest possible touch on the curricular independence of schools.  I’m willing to sacrifice some level of private school participation in return for transparency.  Preferences will vary and we will learn things along the way through variation between programs.  What I think I have learned however is that Arizona’s transparency-light programs represent a costly obstacle to building broad support, and that the Louisiana and Indiana model has far too many private schools saying “thanks but no thanks.”

To my friends who crafted and signed on to this statement I say only that we should continue the dialogue and gather more information.  I don’t believe in regulation free programs nor do I expect or desire for us to pass any, so I agree with you to a degree. I however strongly suspect that many of you are underestimating the cost of regulation and overestimating the capacity of technocratic regimes.

 

 

 

 

 

 

 

 


Follow

Get every new post delivered to your Inbox.

Join 2,716 other followers