I Only Know One Truth-It is Time for Bossy McBossypants Testing to End

September 5, 2017

(Guest Post by Matthew Ladner)

Last spring I was on my bike and came across this in front of a local middle school. I found it striking enough to take a picture with my phone:

In case you are squinting at your iPhone, the sign says “AZ Merit Testing 4/17-5/3.” Now mind you that the test that these students take to determine whether or not they go to college, and if so what sort of college, takes 4 hours. Comprehensive exams for a Ph.D. took me three days. Somehow in the awesome logic of 2017 it came to pass that it would make some sort of “sense” to disrupt schools for two weeks to give…AZMerit.

I think we all know what tends to happen to the school year starting (in this case) 5/4.

Weeks later I had the opportunity to observe a number of focus groups held on parental choice policy. The groups were from different parts of the country, and included parents, teachers and opinion leaders. Despite the fact that the topic of the convening was never testing, everyone made their feelings on the subject clear during conversations. All groups everywhere deeply dislike the current practice of standardized testing.

I can’t emphasize the next point strongly enough: I never once heard anyone use the phrase “Common Core” or burst into a fit of conspiracy mongering. Rather what I saw repeatedly was that people feel that schooling has become overly fixated on test preparation. People have a rather strongly held belief that schooling is supposed to be more than test prep. Something has gone terribly wrong with education in their view, and they want it to stop. Across the groups I saw, the consensus seemed to be that we should drive a stake through the heart of the current system, fill the mouth with holy wafers, and then burn the sarcophagus to fine ash.

I have seen remarkably little evidence that today’s heavy-handed, standards based testing system is of much utility. There is some suggestive evidence that states that had been doing nothing on the testing front before NCLB got a modest bump in results when they started testing. They may however have received a similar bump from a system with a much lighter footprint. Moreover no less than Hanuskek and Loveless have concluded that the heavy-handed Common Core project resulted in approximately nothing in the way of improved student learning. Given that we live in a democracy, a lighter footprint system seems like a fine idea.

So here is mine:

Preserving campus level academic transparency should be the central goal of testing. The Demos would apparently be happy to sacrifice it in return for slaying the testing vampire, but it would be a terrible loss in my view. States can adopt whatever standards they want (I suggest the old Massachusetts standards) but give their students a three-hour national norm reference exam on the second to last day of school. The last day rather than last month of school can now be the write-off. Do a good job teaching the MA standards, your students will do well/show progress on the nnr test.

Some will want to have their state officials grade or otherwise label schools based on the results. Have at it-but it is worth noting that the defacto accountability system in this country has become the Greatschools rating system given that is where the eyeball traffic resides. State ratings have become little more than an obsession internal to the system. Some will want to continue on the troubled path of trying to move the number of teachers fired for low performance from 1% to 1.5%. My view is that this is an unworkable path to hold schools accountable, but if some state or locality wants to keep it up feel free.

I know some of you continue to feel motivated by the idea that standards are going to lead us to profound improvement and narrower achievement gaps.  Decades into the project it is time to ask- where’s the beef? If you are willing to impose a deeply unpopular system of testing upon American families I must ask why? The burden of proof lies with you. If you (like me) would like to preserve campus level transparency I ask what is your plan? My plan is to adopt a system that is less intrusive and prescriptive and hold for dear life to campus level data-now tell me your plan. If your plan is to hold onto dear life to a system that the public abhors, I want to suggest that you need a new plan.

In my view, voting with your feet represents the most robust form of accountability by a very wide margin. I would like to have those voting decisions informed by test scores, and a great many other things including parent reviews (score another touchdown for Greatschools). Watching the focus group discussions made me realize that the United States House’s decision to enact a deeply misguided federal opt-out was not a fluke, but rather fit with the democratic sentiments of their constituents.

Opt-outs lead to nudge outs which leads to completely unreliable and thus worthless data. They will be passing at the state level soon unless transparency supporters pull their heads out of the sand. As Corwallis wrote to Clinton before the Battle of Yorktown “What is our plan? If we don’t have one, what are we doing here?”

Perhaps I’ve got this all wrong. If so, the comment section awaits.


The Play’s the Thing

September 4, 2017

What do students learn from field trips to see live theater?  As it turns out, quite a lot.  That’s the finding of my new working paper co-authored with Heidi Holmes Erickson, Angela Watson, and Molly Beck that was posted on SSRN this week.

We randomly assigned groups of students to receive free tickets to see a play or to remain in their school to serve as the control. We repeated this experiment for five different plays over a period of two years.  For two of the plays we added a second treatment condition in which students left school to see a movie comparable to the play, while some were randomly assigned to see the play and the rest remained in school as the control.

Across all five plays we found that students randomly assigned to see live theater scored significantly higher than the control students on measures of tolerance and social perspective taking as well as a test of their knowledge of the play’s plot and vocabulary.  For the two plays in which there was also a movie treatment, we found no difference between students who saw the movie and those who remained in school as the control.  Seeing live theater produces important social and cognitive benefits for students that are not realized by showing them a movie instead.

This experiment cannot tell us the exact mechanisms by which these benefits are produced.  Our best guess is that leaving school to see a play exposes students to a broader world, which helps them gain greater understanding and acceptance of that broader world.  For many students, theater is a window to different people, places, and ideas.  Movies may not have the same effect because they lack the personal interaction of live theater.  Perhaps students are more intellectually and emotionally engaged when there are people acting out a story in front of them than when they see that story on a screen.  The relative novelty of theater may also be a factor.

Dan Bowen, Brian Kisida, and I saw similar tolerance and knowledge benefits in a previous experiment in which students were randomly assigned to go on field trips to see an art museum.  There appears to be something about the in-person exposure to cultural activities that affects student values and knowledge of that material.

The new working paper builds on and improves upon an earlier article that presented the results from the first two of the five play experiments we conducted.  With significant help from Cari Bogulski, Hunter Gehlbach, and Thalia Goldstein we revised the original study design to collect pre-treament measures of outcomes, add the movie treatment condition, and to include social perspective taking as an outcome.  The new study presents the results of all five plays combined although the estimated effects are remarkably consistent for each play separately.

Our research of both art museums and theater shows that out of school arts experiences produce significant benefits for students.  Much may be lost if we continue abandoning these activities as schools narrow their focus on math and reading test results.