Jay P. Greene's Blog

I recently recommended the Netflix series, Daredevil, to a friend.  He replied: “Aren’t you tired of comic book-based movies?  They just tell the same story over and over.”  I suggested that the familiarity of comic book characters was precisely what allowed them to be so good.

Comic book stories are just the modern version of Greeks myths or medieval saint stories.  The audience is familiar with the characters and basic plot, but the art is in the particular telling of those familiar stories.  And because the characters and basic plot are mostly known by the audience, much less energy needs to be devoted to explication.  Only the ways in which the characters and plot deviate from the familiar pattern require greater development.

Originality in story-telling is often over-rated.  Shakespeare mostly relied on familiar characters and well-worn plots.  His contribution was in how he used what was already well-known.  Conversely, I love original Chris Noland films, like Inception and Memento, but it’s amazing how much time in those movies is consumed with explaining how dreaming or memory work.  There is an efficiency in using the familiar.

This all being said, good comic book stories still require some development of plot and character, especially to highlight the ways in which the current telling deviates from previous ones.  I thought the Avengers movies fell short in devoting far too little to character and plot development and way too much to mind-numbing action.

But the new Netflix series, Daredevil, is a wonderful example of how comic book stories can be excellent if they are told well.  I remember the boring Ben Affleck movie-version of Daredevil, so it is striking how different the same story can be if it is just done better.  The new Netflix series version benefits from a serialized format to develop its version of the plot and characters more slowly.  It also uses Foggy for great comic relief.

And best of all, it crafts a sympathetic Wilson Fisk — so sympathetic that at times I wasn’t entirely sure whether he or Matt Murdock were actually the villain.  Fisk clearly uses evil methods but appears genuine in his regret about those methods and sincere in his goal for a better city.  Murdock/Daredevil, on the other hand, admits that he derives pleasure from his violence and only differs from Fisk in his methods in his unwillingness to kill his enemies.  Deriving pleasure from torturing one’s opponents hardly seems better than killing them with remorse.

Like all good comic book-based stories, Greek myths, and saint stories, Daredevil’s characters are archetypes worth exploring.  They capture some essential element of humanity, with its mix of tragic flaws and potential for greatness.  Check out Daredevil and let me know if you don’t think it was better than the Avengers and worthy of attention, like stories about Hercules or St. Francis.

Colleagues, students, and I have been studying the effects of culturally enriching field trips on students.  We’ve published research on field trips to see live theater (so far here) and to visit an art museum ( so far here, here, and here) and generally lamented that these kinds of enriching trips are in decline (see here and here).  They are either being replaced with “reward” field trips to places like amusement parks and bowling alleys or disappearing altogether.  Enriching trips not only have the potential of conveying important content knowledge, but also help convey our values and priorities to future generations.

Yesterday, the Minneapolis Star Tribune  carried a piece about a small private school, the Gaia Democratic School, which took its students on a field trip to the Smitten Kitten, a local store featuring sex toys and erotica, as part of their sex education class.

This seems like a culturally enriching field trip to me.  It has potentially useful content knowledge and it is conveying to those children the values and priorities of the parents and educators at that school.  The article does quote one parent who withdrew his children from the school after his two daughters, ages 11 and 13, went on the field trip.  But my guess is that most parents at the Gaia Democratic School know and support exactly what their school stands for.  The Star-Tribune says, “Gaia is a K-12 school with a motto promising academic freedom, youth empowerment and democratic education. Parents say the school has about 25 students, including several described by administrators as transgender.”

That’s the beauty of school choice.  If this school doesn’t teach their values, parents can go somewhere else.  But for those families who want something like Gaia Democratic School, why should they be forced to attend a school that drags their kids to Chuck E. Cheese on field trips?

I wonder if Gaia has thought about having an RCT evaluation of their filed trips, although that would be hard with only 25 23 students.

This is the question raised by a new study by C. Kirabo Jackson, Rucker C. Johnson and Claudia Persico  in Education Next.  Jackson, et al claim to have up-ended decades of school finance research by finding a link between school spending and improved student outcomes.  After reading that article and an earlier, more detailed version posted on the NBER web site, I find nothing to persuade me to abandon the long-standing and well-established finding that simply providing schools with more resources does not improve student outcomes.

Let’s remember how well-established this finding is by noting that Eric Hanushek conducted a comprehensive review of the literature and concluded:

…the research indicates little consistent relationship between resources to schools and student achievement. Much of the research considers how resources affect student achievement as measured by standardized test scores. These scores are strongly related to individual incomes and to national economic performance, making them a good proxy for longer run economic impacts. But, the evidence – whether from aggregate school outcomes, econometric investigations, or a variety of experimental or quasiexperimental approaches – suggests that pure resource policies that do not change incentives are unlikely to be effective. (p. 866)

Jackson, et al acknowledge that past research has failed to find a link between school resources and student outcomes:

Coleman found that variation in school resources (as measured by per-pupil spending and student-to-teacher ratios) was unrelated to variation in student achievement on standardized tests. In the decades following the release of the Coleman Report, the effect of school spending on student academic performance was studied extensively, and Coleman’s conclusion was widely upheld.

But they believe that past research was flawed in two important respects.  First, test scores may be a weak indicator of later-life success, so it would be better to look at stronger measures, like educational attainment, employment, and earnings.  Second, they believe that past studies of school spending may suffer from an endogeneity problem.  That is, extra money has tended to go to schools facing challenges.  The failure to find a link between more resources and better achievement may be because schools with a weaker future trajectory are the ones more likely to get more money.  So, the causal arrow may be going in the wrong direction.  Weak performance may be causing more resources rather than more resources causing weak performance.

Jackson, et al solve the first issue by focusing on longer-term student outcomes, like educational attainment and earnings.  They claim to have a solution to the second problem by finding a type of spending increase that is unrelated to the expected trajectory of school performance.  Court-ordered spending, they say, is exogenous, while regular legislative increases in spending are endogenous.

The surprising findings of the Jackson, et al article hinge entirely on this claim that court-ordered spending is exogenous.  Looking at attainment and earnings by itself does not produce a different result than past research that has focused on test scores.  The thing that allows Jackson, et al to find that spending is linked to better student outcomes is the fact that they do not examine actual spending increases.  Instead, they predict changes in spending based on court-orders and use that predicted spending in place of the actual spending.

This instrumental variable technique developed by James Heckman, however, only works if the instrument is in fact exogenous.  That is, court-ordered spending has to be unrelated to the future trajectory of school performance.  Given how critical this point is to the entire article, you might think Jackson, et al would spend a fair amount of energy to justifying the exogeneity of court-ordered spending.  They do not.

It is completely mysterious to me why we should believe that court-ordered spending differs from legislatively-originated spending in the likelihood that it is linked to the expected future trajectory of school performance.  That is, schools facing challenges are just as likely to get extra money if the spending originates in the courts or in the legislature.  If we are concerned that the causal arrow is going in the wrong direction in that weak performance causes more money rather than the other way around, we should have that concern just as much whether the motivation for the money came from the court or the legislature.

Jackson, et al do not make a proper case for the exogeneity of court-ordered spending other than to describe it as a “shock” to school spending.  But there is nothing more shocking about spending that originates in the courts than in the legislature.  Court cases take years to develop, be decided, and complete appeals.  And then they have to be implemented by legislative action.  The timing of court-ordered spending is no more surprising to schools than regular legislative spending.  Nor is the amount of spending change necessarily more dramatic than those originating in legislatures.  The passage of ESEA and its re-authorizations infused large amounts of money into schools.

Jackson et al need to convince us that court-ordered spending is exogenous to get their unusual result.  If they just used conventional methods, they would confirm the wide-spread finding that extra money does not improve outcomes.  As they describe it:

We confirm that our approach generates significantly different results than those that use observed increases in school spending, by comparing our results to those we would have obtained had we used actual rather than predicted increases as our measure of changes in district spending. For all outcomes, the results based simply on observed increases in school spending are orders of magnitude smaller than our estimates based on predicted SFR-induced spending increases, and most are statistically insignificant.

But Jackson, et al fail to justify the claim that court-ordered spending is exogneous on which their entire article depends nor does such a claim seem plausible.

But even if you were to somehow believe that court-ordered spending is exogenous, it would still be unwise to jump to the conclusion that we now know money matters and should open the resource spigots to K-12 education.  First, the past research Hanushek reviewed includes studies that do not suffer from either of the concerns raised by Jackson, et al.  That is, some of those studies examine later-life outcomes for students and not just test scores and some of those studies rely on experimental methods with which there is no problem with causation.  Why should we disregard those studies for this one new study even if we were to ignore the concerns I’ve raised above?

Second, Jackson, et al are examining the effect of court-ordered spending in the 1970s when spending levels in real terms were much lower and variation in spending across districts within states was much higher.  It’s quite a leap to think that more money now would have the same effect as then.  To my surprise, Bruce Baker made this same point in response to the Jackson, et al article in comments to Education Week:

“[E]xploring such [far-apart] outcomes, while a fun academic exercise, is of limited use for informing policy,” he wrote in an email to Education Week. “Among other things, these are changes that occurred under very different conditions than today.”

Mr. Baker also disagreed with the researchers’ caveat that similar changes might have a much smaller effect if introduced today, in part because total school funding nationwide increased by 175 percent over 43 years, from an average of $4,612 per student in 1967 to about $12,772 per student in 2010, as measured in 2012 dollars.

So does school spending matter after all?  I think the answer is still clearly “no.”

Marty Lueken from the Wisconsin Institute for Law & Liberty (WILL) and a Department of Ed Reform alum sent me this remarkably informative video on the lack of relationship between school spending and achievement globally and in Wisconsin.  Check it out.

(Guest Post by James Shuls)

In the early 1970s the Ford Motor Company designed the Pinto. In addition to being extremely ugly, the Pinto was extremely dangerous. A rear-end collision could cause the gas tank to rupture and ignite. For obvious reasons, the Pinto is regarded as one of the worst cars ever.

At the very least, it was poorly designed.

Many look at Missouri’s interdistrict transfer program, which has allowed more than 2,000 students from the Normandy and Riverview Gardens school districts to transfer to higher performing suburban districts, as if it were a Pinto. It has forced the two unaccredited districts to hemorrhage and rest on the verge of bankruptcy.

Is it ugly? Yes. Is it poorly designed? It depends.

In 2013, the year before students transferred, fewer than 20 percent of students in the two unaccredited school districts were proficient in reading or math. Dropout rates were abysmal, and the prospects were slim for graduates of either district.

Now, let’s imagine that the transfer law was not in place. What would be different today? Chances are, the school districts would not be performing significantly better. What’s more, few outside of the school districts would have taken any note of the quality of education being delivered in these two north Saint Louis County districts.

Since nearly a quarter of the students walked out of these flailing districts, much has changed. No, the districts have not gotten significantly better; nor have they gotten significantly worse. The transfer program, however, has allowed 2,000-plus students to have the opportunity for a better education. Moreover, it has launched a robust conversation around the state about how to turn around struggling school districts.

But to understand if the program was “poorly designed,” we have to determine what it was designed to do.

Was it designed to be a school reform model for unaccredited school districts? If so, it failed, since it has drained these districts of necessary financial resources and could bankrupt both of them.

Was it designed as a long-term fix, a permanent interdistrict program? Again, if this is the case, it failed, since the program lacks a feasible tuition system. Currently, tuition ranges from roughly $10,000 to $20,000, depending on where a student transferred.

Is it possible, however, that the transfer program was designed for another purpose, like to create controversy or to spark change?

As I note in my latest paper, “Interdistrict Choice for Students in Failing Schools: Burden or Boon?” the transfer students were largely absorbed into 24 receiving districts with little disruption. In this regard, the transfer program functioned relatively well.

The program has told families in low-performing schools that their children do not have to be doomed to perpetual underperformance. These families took the right to an education guaranteed by Missouri’s state constitution. Angel Matthews, for instance, experienced the benefits that come from choosing your own school. Angel was one of 175 students who initially transferred from Normandy and Riverview Gardens to the Kirkwood School District, and she has embraced the rigors and opportunities of her new school by taking AP honors classes, cheerleading, and running track.

The transfer program has caught our attention, but it is not a permanent fix. Policymakers must now develop new policies that establish quality schools in every neighborhood. These policies must include some form of school choice, lest we fall back into our old pattern of assigning students to chronically failing schools.

———————————————————————-

James V. Shuls is an assistant professor of educational leadership and policy studies at the University of Missouri–St. Louis and a fellow at the Show-Me Institute.  Follow on Twitter @Shulsie

On Tuesday, the Wall Street Journal published a piece by Jason Riley on “The Soccer Mom Revolt Against Common Core.”  In it, I was quoted offering the analysis that upper-middle class moms were accustomed to having significant control over their kids’ schools.  NCLB may have annoyed these moms with tests that had little meaning for their kids, but by remaining largely agnostic on standards, academic content, and the method of testing, NCLB didn’t interfere with the operational control of suburban moms.

The over-reach of Common Core and federally-sponsored aligned tests is that they impinge to a much greater extent on the operations of schools.  When soccer moms come to school to complain about Rome and Juliet being cut to make way for informational texts, they are being told that the school had no choice in the matter.  Common Core made them do it.  And if they want to do well on the PARCC or Smarter Balanced tests, they have to make these changes.  It doesn’t matter whether CC really requires this change or not.  The issue is that large numbers of upper-middle class parents are being told that they no longer have the same kind of influence over their schools that they are used to having.  And they are pissed.  So, they are starting to boycott the tests.

Not so fast, says Mike Petrilli.  In a post today he argues:

Here’s where Jason’s argument falls apart: Common Core is almost everywhere. Soccer moms are found almost everywhere. Yet the rebellion he describes is limited to one specific area.

As for Jay, maybe the loss of parental control is a real issue, but why do parents in Montclair, for example, feel that their power is being usurped much more so than parents in other states? Again, it can’t be Common Core, or testing, or school accountability policies, because those are almost universal.

Common Core couldn’t explain the opt-outs because they are concentrated in NY and NJ while CC is spread across the country.  The culprit must be the unions, he argues, since they are strong in NY and NJ and managed to enroll these parents in their general fight against accountability.

Let’s try Petrilli’s argument on another situation to see how well it stands up.  The Baltimore riots couldn’t be caused by police abuse, he would have to argue, because the riots are concentrated in Baltimore while police abuse is widespread.  Convinced?

Let’s try another one.  The unions can’t be responsible for the opt-outs because their opposition to accountability is longstanding while the opt-outs are a new phenomenon.  Common Core sophistry is fun!

Of course, mass protests, like opting out or rioting, have to start somewhere even if the source of complaint is widespread.  In addition, agitators typically play a role in motivating and organizing mass protests, but the underlying injury needs to be present or the agitation fails to gain traction.  The unions couldn’t get the soccer moms to opt-out unless they were upset about something.  Before Common Core, the unions tried but failed to elicit upper-middle class action against accountability tests.  Now they are finding a receptive audience.

No amount of sophistry is going to change the political challenge Common Core faces by interfering with soccer moms’ control over local schools.  And no amount of blaming those soccer moms for failing to care about poor and minority students is going to guilt them into surrendering that control.

Below is the text of the testimony I intend to present to the Arkansas Common Core Council on Wednesday.  The Council is chaired by Lt. Governor Tim Griffin and was charged by the legislature with providing advice on the future of Common Core Standards and PARCC testing.  You can watch videos of the Council testimony and discussions here.

—————————————————————————————–

Jay P. Greene’s Testimony to the Arkansas Common Core Council
May 6, 2015

Standards are about what we value. They communicate what we think is important for our children to learn, when they should learn it, and ultimately what kinds of adults we hope they will grow up to be.

Because standards are about values, their content is not merely a technical issue that can be determined by scientific methods. There is no technically correct set of standards, just as there is no technically correct political party or religion. Reasonable people have legitimate differences of opinion about what they want their children taught. A fundamental problem with national standards efforts, like Common Core, is that they are attempting to impose a single vision of a proper education on a large and diverse country with differing views.

National standards can try to produce uniformity out of diversity with some combination of two approaches. They can promote standards that are so bland and ambiguous as to be inoffensive to almost everyone. Or they can force their particular vision on those who believe differently. Either way, national standards, like Common Core, are inappropriate and likely to be ineffective. If national standards embrace a vague consensus, then they make no difference since almost everyone already believes them and is already working toward them. If, on the other hand, national standards attempt to impose their particular vision of a proper education on those with differing visions, then national standards are oppressive and likely to face high levels of resistance and non-compliance. So, national standards are doomed to be either unnecessary or illiberal. Either way, they are wrong.

Some of you may be thinking that education is not entirely about values. Can’t we at least agree, you might be thinking, that all children need to acquire basic competency in literacy and numeracy? And if so, might not standards be helpful in addressing these more technical issues even if they cannot address broader issues of values?

Unfortunately, even when it comes to some of the narrower goals of education, there is no evidence that standards deemed to be higher quality are effective in producing higher levels of literacy and numeracy. I’m aware of four analyses that have examined whether states with standards judged to be better have greater academic achievement. I’ve provided references to these four analyses in the written version of my testimony. None of them show any relationship between the ratings of state standards according to the Fordham Institute and Education Week and each state’s performance on the National Assessment of Educational Progress. I’m not aware of any empirical analyses that show that “better” standards lead to better outcomes for students.

The lack of relationship between the judged quality of state standards and student achievement should raise a number of concerns for this Council. First, it should make you doubt claims about the quality of Common Core standards. How does anyone know whether Common Core standards are good and will contribute to academic achievement if no one has ever found a relationship between Common Core (or any standards for that matter) and student outcomes? Many people claim to be expert judges of the quality of standards but no one’s judgment has been validated by actual improvement in student performance.

Second, perhaps the lack of relationship between expert judgments about the quality of standards and student achievement is explained by the fact that there is not a single path to academic success for all of our incredibly different children. Common Core or other standards might be good for some students in some circumstances, but bad for other students in other situations. The reason why expert claims about the quality of standards have never aligned with student achievement is that there is no single set of standards that could be optimal for promoting even basic literacy and numeracy for all students. Standards, particularly national standards, are then a fool’s enterprise of one size fits none.

Third, the lack of relationship between “better” standards and achievement might be caused by low levels of compliance by schools and educators rather than the unreliable judgment of experts. That is, standards are just a bunch of words in a document. Even if they are the right words and even if one set of words could fit what all children need, there is no assurance that schools or educators would teach to those standards. Schools and educators have their own ideas about the proper goals of education and little can be done to force them to change their practice.

Key backers of Common Core standards are aware of this problem and so the U.S. Department of Education funded the development of new tests that would be aligned with these national standards. If these new tests could detect whether schools and educators were changing their practices in the ways desired by Common Core and if rewards and punishments could be imposed on schools and educators for their compliance with the new standards, then perhaps the empty words of standards could be transformed into a real change in the education system.

The problem with trying to use PARCC or Smarter Balanced tests to drive Common Core changes is that it almost certainly requires more coercion than is politically possible and would be undesirable even if it could be accomplished. If Arkansas tries to use the PARCC test to impose strong enough sanctions on schools and educators to drive changes in their practice, we will witness a well-organized and effective counter-attack from educators and sympathetic parents who will likely neuter those sanctions. If, on the other hand, the consequences of PARCC are roughly the equivalent of double secret probation in the movie, Animal House, then no one has to change practice to align with the new standards.

And even if by some political miracle the new PARCC test could be used to impose tough sanctions on schools and educators who failed to comply with Common Core, it’s a really bad idea to try to run school systems with a test. All sorts of bad things happen when maximizing performance on standardized tests becomes the governing principle of schools. Schools and educators are likely to narrow the curriculum by focusing on tested subjects at the expense of untested ones. If we care at all about the Arts, History, and Science we should oppose trying to run schools with math and ELA tests. And within tested subjects schools and educators are likely to focus narrowly on tested items at the expense of a more complete understanding of math and English.

Common Core is unlikely to produce meaningful changes in practice without an aligned test that punishes schools and educators, but those types of harsh consequences are unlikely to survive the political opposition of educators and parents. And even if PARCC could impose tough consequences to drive changes in practice, the changes would produce a disastrous narrowing in the curriculum of schools.

So what should this Council recommend? Given that there is no technically correct set of standards and given that expert judgment about the quality of standards has never been validated by better student outcomes, there is no reason for Arkansas to defer to the small group of national experts who drafted the Common Core standards. Arkansas policymakers, educators, and parents know as much about effective standards as these self-proclaimed experts. So we should be empowered to write our own standards that reflect our own priorities and values in education. If standards are about values, they should be developed as close to the people to whom they apply as is practical.

But even standards that are developed in a decentralized way will fail to capture all of the legitimate diversity of goals and needs. For that reason, even standards that are developed locally should be humble about what they can accomplish and the extent to which schools and educators ought to change their practice as a result. In the end, it is families, educators, and communities who need to set appropriate goals for individual children, not the state and certainly not the national government or organizations.

Lastly, and perhaps most importantly, we should abandon PARCC and purchase an already-developed, nationally-normed standardized test from ACT or any of the reputable testing companies. The purpose of PARCC is to drive changes in educator behavior in ways that are desired by Common Core. But we should not be using tests aligned with a set of standards to coerce schools and educators to change their practice. What we really need from standardized testing is just information about how our students are performing. This can be accomplished at much lower cost by just buying a nationally-normed test off of the shelf. And lower stakes tests that are primarily about information rather than coercion will produce much less harmful narrowing of the curriculum.
————————————————————————–
References
Charlie M. Belin and Brian Kisida, “Science Standards, Science Achievement, and Attitudes About Evolution,” Educational Policy, September 21, 2014. http://epx.sagepub.com/content/early/2014/09/23/0895904814550069

Eric A. Hanushek, “Is the Common Core Just a Distraction?” Education Next, May 9, 2012. http://educationnext.org/is-the-common-core-just-a-distraction/

Tom Loveless, “How Well are American Students Learning?” The 2012 Brown Center Report on American Education, The Brookings Institution, February, 2012.http://www.brookings.edu/~/media/research/files/reports/2012/2/brown%20center/0216_brown_education_loveless.pdf

Grover J. “Russ” Whitehurst, “Don’t Forget Curriculum,” Brown Center Letters on Education, The Brookings Institution, October, 2009. http://www.brookings.edu/research/papers/2009/10/14-curriculum-whitehurst

Shakespeare can be taught.  Students don’t even have to know how to read to become familiar with the sound and cadence of Shakespeare’s language.  Watch this video of Brian Cox teaching a friend’s toddler to recite a portion of the “To be, or not to be…” soliloquy.

Meaning and understanding comes later, but familiarity almost certainly makes that easier.  And to grasp the meaning, it would help if only we had more teachers follow the example of Geoffrey Tennant in the great Canadian TV series, Slings and Arrows, and declare: “So, let’s get rid of the curriculum and I think we should just f*ck around with some texts.”  I’ve experienced many an English teacher drain  all of the joy and depth from Shakespeare by mechanically having the class take turns reading passages while ticking off what students need to know for the AP exam.

Watch Geoffrey motivate an accountant from the plastics firm to do a better reading of Macbeth’s “Tomorrow” soliloquy: