Jay P. Greene's Blog

I’ve written before about the Character Assessment Initiative (Charassein) at the University of Arkansas, headed by my colleague Gema Zamarro.  This post provides an update about the strong progress they are making with their research.

First, the paper by Collin Hitt, Julie Trivitt, and Albert Cheng that I’ve been telling you about for more than two years is now being published in the Economics of Education Review.  Getting into a top journal takes time, but this paper certainly deserves high placement.  In this paper, Hitt, Trivvit, and Cheng demonstrate across several longitudinal data sets that students who are more non-responsive to survey questions (skipping items or saying “don’t know”) have significantly lower educational attainment and fare less well in the labor market, even after controlling for a broad set of background characteristics and cognitive measures.  Essentially, this paper validates that item non-response is a useful proxy for character skills (probably conscientiousness) and is predictive of later life outcomes.

Second, Albert Cheng and Gema Zamarro have a new paper that demonstrates that teachers actually alter student character skills.  In particular, they look at data from the MET Project in which students were randomly assigned to teachers in the second year of the study.  They find that teachers who themselves have weaker character skills during the first year of the study, as measured by non-responsiveness or careless responses on surveys, weaken the character skills of the students experimentally assigned to them during the second year of the study.  Conversely, teachers who model higher levels of conscientiousness improve the character skills of their students.

This teacher ability to affect student character skills in not related to their ability to improve math and reading test performance.  So teachers who are great at building character skills may not be the same ones who are great at conveying math and reading.  Students are more successful when they learn both cognitive and non-cognitive skills.  If we focus only on retaining and rewarding teachers or schools that do one, students may miss out on having teachers or schools that are good at the other.

I hope it’s not another two years until you see this new Cheng and Zamarro paper in a top journal.  Now that their measure has been validated by the EER piece, future publications should come more quickly.  And Albert is himself headed to great things as he completes his Ph.D. and begins a post-doc at Harvard next year.  Keep you eye on him and this line of research on using survey responsiveness and carelessness as measures of character skills with strong predictive power for student success.

One of the banes of education reform is its obsession with producing uniform reforms at scale.  Donors and policymakers tend to prefer standards and testing reforms that affect all students at once.  And if they are willing to entertain the messiness of school choice, donors and policymakers tend to prefer large chains, like KIPP, over mom and pop charters or the diverse assortment of small private schools.  They want to achieve what they imagine are the economies of scale in education by having the same reform affect large numbers of students at the same time and in roughly the same way.

Unfortunately, there do not appear to be many economies of scale in education.  In fact, the nature of education, like other aspects of human development, appear to be more effectively conducted at small scale.  Education, like child-rearing, relies for success on personal interactions among people with authentic relationships.

We’ve tried a variety of arrangements throughout history for raising children.  People have experimented with all sorts of collective child-rearing approaches but have gravitated back in almost all cultures across long-periods of time to raising children within small family groups.  Despite the allure of economies of scale, we’ve almost universally rejected communal child-rearing.  Even though we could house more children and would require fewer adults in giant, orphanage-like settings, we rightly recognize that proper human development involves the close interaction of caring adults with children.  It’s just not something that is produced well at scale.

We should no more prefer scale in education than desire collective child-rearing.  Children are sufficiently diverse in their interests and needs that uniformly imposed solutions tend not to serve them very well.  Instead, it takes caring adults with an understanding of each child to customize (at least partially) how each child should be educated.  In addition, education requires motivating children to exert effort to learn.  That motivation is much more easily produced by an adult with whom children have an authentic relationship.

So the next time you hear reformers ask if a reform is scalable, remind them that orphanages are scalable — but that doesn’t mean we want them.

The Journal of Policy Analysis and Management just published an article by Sass, et al that I’ve mentioned in a number of previous posts.

This is a very important study using a rigorous causal research design that shows long-term benefits to attending charters in FL on college persistence and later life earnings.  Importantly, these benefits are produced despite the fact that there seems to be little or no test score gain associated with FL charter schools and despite the fact that these are mostly Mom and Pop schools rather than the preferred “no excuses” model of charter school.

So, building our entire education reform strategy around the idea that short-term changes in test scores correspond with long-term changes in life outcomes is inconsistent with a growing body of evidence.  Choice reforms in particular have demonstrated that long term gains in educational attainment (and now earnings) can be produced without seeing short term gains in test scores (and vice versa). Trying to pick the winners among schools of choice based on test scores could lead to horribly wrong policy decisions.  Parents appear to know more than portfolio managers, choice regulators, and other central planners.

Here is the abstract from the now published study:

Since their inception in 1992, the number of charter schools has grown to more than 6,800 nationally, serving nearly three million students. Various studies have examined charter schools’ impacts on test scores, and a few have begun to examine longer-term outcomes including graduation and college attendance. This paper is the first to estimate charter schools’ effects on earnings in adulthood, alongside effects on educational attainment. Using data from Florida, we first confirm previous research (Booker et al., 2011) that students attending charter high schools are more likely to graduate from high school and enroll in college. We then examine two longer-term outcomes not previously studied in research on charter schools—college persistence and earnings. We find that students attending charter high schools are more likely to persist in college, and that in their mid-20s they experience higher earnings.

The Americans for the Arts recently released a poll finding that the vast majority of Americans agree that “the arts are part of a well-rounded education for K-12 students.”  Over half of respondents “strongly agree.”  Unfortunately, the current trend in ed reform is out of sync with this popular support for the arts in education.

The narrow focus on math and reading achievement is driving out other subjects, including the arts.  Ed reformers may offer rhetorical praise for the arts and a broader education, but most quietly believe that math and reading are of such primary importance that shifting away from the arts to attend more to math and reading might actually be a good thing.

Like most other ed reformers, I used to believe this too.  But more research is beginning to show that a broader education, including the arts, may be essential for later success in math and reading as well as the proper development of civic values and character skills, including tolerance, empathy, and self-regulation.  The narrow focus on math and reading may goose math and reading test scores in the short term but at the expense of the longer-term and broader goals of education.  Parents seem to understand how essential the arts and a broader approach to education are even if this has escaped the highly-credentialed minds of “policy experts” trying to manage schools from afar through test results.

Let me briefly provide some evidence to support the claims above.  First, we have good reason to believe that the arts are being squeezed out of the curriculum.  For example, research by Daphna Bassok, Scott Latham, and Anna Rorem has found a dramatic decline in the role of the arts in school between 1998 and 2011.  Using teacher surveys from the ECLS-K, they find that far fewer teachers in Kindergarten and 1st Grade report teaching Music, Art, Dance, and Theater on a daily or weekly basis, and far more teachers report never covering these subjects at all.  (See their Table reproduced below)

What evidence do we have that this shift away from the arts to focus more narrowly on math and reading has negative consequences?  David Grissmer and his colleagues are producing a series of studies that suggest how much later success in math, reading, and science depends on early acquisition of the kind of “general knowledge” and fine-motor skills learned through art and other subjects.

In one of these studies they find: “Whereas the early math and reading tests focused mainly on procedural knowledge, the general knowledge test focused mainly on declarative knowledge (i.e., elementary knowledge or comprehension of the external world). General knowledge was the strongest predictor of later reading and science and, along with earlier math, was a strong predictor of later math. General knowledge measured at kindergarten entrance may reflect early comprehension skills that are necessary when reading changes from a more procedural task in early grades (learning to read) to incorporating more comprehension around third through fifth grades (reading to learn).” This is essentially empirical support for the type of argument E.D. Hirsch, Robert Pondiscio, and the folks at Core Knowledge have been making for years.  It’s important for students to know a lot of things about the world, including about Art, History, etc…, to progress academically.  If we narrow education to the mechanics of math and reading as captured by yearly testing, we short-change the broader knowledge that is the key to academic success later.

Less intuitively, they find that the development of fine motor and other “visuo-spatial” skills are also very strong predictors of later academic success.  These are the kinds of things students learn by playing musical instruments or making art projects — activities disappearing from the early school curriculum.  Yet these fine-motor and coordination skills seem to be an important part of brain development that improves math and reading achievement years later.

Grissmer and his colleagues summarize the implications of their findings better than I could:

Our results suggest that the focus of interventions should shift from a primary emphasis on changing the direct math and reading instructional environment to interventions that build better foundational skills of attention and fine motor skills and a better understanding of the world outside schools. The results suggest that current direct math and reading instruction is insufficient to build attention and fine motor skills. Building these skills may rely more on subjects and curricula that have been deemphasized to provide more math and reading instruction: the arts, music, dance, physical education, and free play. Each of these subjects and curricula may need to be redesigned to focus on building foundational skills in the same way that math and reading have been redesigned in recent years. Building stronger knowledge of the external world also suggests that improving early science and social studies curricula are important. Paradoxically, higher long-term achievement in math and reading may require reduced direct emphasis on math and reading and more time and stronger curricula outside math and reading.

“Building stronger knowledge of the external world” might include going on culturally enriching field trips, which have also been disappearing from schools.  In addition to the general knowledge these experiences convey, the research I’ve done with others at the University of Arkansas on the effects of field trips to art museums and to see live theater suggests that these culturally enriching experiences change student values to promote greater tolerance and empathy.

Given that short-term gain in math and reading achievement are only weakly related to later life outcomes, while a broad education that includes the arts and culturally enriching activities may be associated with long-term success, education reformers should wonder whether they are simply rediscovering what most Americans already know — “the arts are part of a well-rounded education for K-12 students.”

(Edited for typos)

Brookings fellow, Michael Hansen, has a piece blaming high school sports for “distract[ing] public schools from their mission.”  It’s a curious piece because it never actually articulates what the mission of schools should be, nor does it provide evidence that sports undermine that mission.  Instead, it references a study by Marguerite Roza showing that (at least in one district) the “per-participant cost of cheerleading totaled $1,348 and $829 for football. Less than $350 per student was spent on math instruction for the year.”  He goes on to argue:

It is unclear how widespread this disproportionate spending is among U.S. high schools, but it raises the question of whether our spending in public education is consistent with our academic goals. And, from an equity perspective, the proposition of fielding a football team using scarce public resources implies the funds for a few student players comes at the expense of the many other students.

So, his evidence that sports are bad is that they cost more per pupil than math instruction and affect a smaller number of students.  Normally, when economists perform a cost-benefit analysis they consider benefits as well as costs, but in this case Hansen only mentions the costs and fails to consider the benefits.  Presumably he assumes the benefits to high school sports are small or zero, but rigorous evidence suggests the benefits can be substantial.

One analysis by Eric Eide and Nick Ronan uses an instrumental variable approach to estimate the effect of participating in high school sports on long-term outcomes, like educational attainment and earnings.  They find:

…sports participation has a negative effect on the educational attainment of white male student athletes, a positive effect on the educational attainment and earnings of black male student athletes, and a positive effect on the educational attainment of white female student athletes.  We find no effect of participation on the educational attainment or earnings of Hispanic males or black and Hispanic females.

From “an equity perspective,” as Hansen likes to frame the issue, sports seem like a real plus.  Sports may give at-risk students a reason to stay in school so that they can actually receive math and other subject instruction. An amazing randomized experiment of a Chicago program called Becoming a Man — Sports Edition, demonstrates that participating in aggressive sports is particularly effective for disadvantaged youth in improving school engagement and reducing violent crime.   Taking money from sports to increase resources for math instruction might do little good for students who have dropped out or gone to jail.  In addition, we have all sorts of other expensive programs to help black boys, none of which seem to bother Hansen, and I doubt many have been shown to be as effective at improving outcomes as sports.

Sports may also convey leadership skills, which may account for the improved long-term outcomes among white girls who participate in sports.  Again, much ink and and wealth has been devoted to building the self-esteem of girls, particularly in the sciences.  High school sports seem like an effective way to accomplish that.

The per pupil cost figures on sports and math instruction are also misleading because almost all students are enrolled in math while fewer are on sports teams.  Rather than posing a problem for equity, as Hansen suggests, this fact suggests that the total cost of sports is not that high.  I’m sure we could bring the per pupil cost down if we mandated that all students play a sport, just as we mandate that all take math, but we would also raise the aggregate cost.

Of course, programs that serve only some students will be more expensive, but offering a variety of opportunities is precisely the mission of high schools.  Every other elective activity in high school, including theater, band, newspaper, and art, also has higher costs per participating student than required courses, such as math and English.  Does Hansen want to get rid of all of those programs as well?  Does he think they also distract from the mission of schools?

Perhaps if Hansen had thought about what the mission of schools actually  is, he might realize that it involves offering students a variety of experiences rather than the thin gruel of drilling math and reading all day.  Thriving and successful adults aren’t produced by Hansen’s incredibly narrow conception of what it means to be educated.

Many of you have heard the sad news that Andrew Coulson passed away over the weekend.  I thought I would share some personal memories.

I first remember meeting Andrew and his wife, Kay, at a conference in Toronto in 2000.  I have to admit that he felt out of place.  Here was this guy without any university, think tank, or other affiliation and without any formal training presenting on the history of markets in education.  And people did not typically attend these meetings with spouses.  Who was this guy?

As it turns out, this guy was a brilliant autodidact who “retired” after being an early programmer with Microsoft to devote his time to studying and advocating education reform.  And he was really good at it.

The thing that struck me most about Andrew was his incredible optimism and quirky sense of humor.  Liberty-oriented education activists tend to be on the losing side of policy battles.  It can be downright discouraging.  But Andrew never seemed discouraged or became bitter.  It was a long game and he maintained a sunny optimism that freedom worked better and people would eventually gravitate towards what worked.

He didn’t mind standing apart from the crowd.  Just because donors, policymakers, and other scholars were drawn to test-based accountability didn’t make Andrew feel like he had to join them.  He even expressed serious reservations about certain methods of expanding school choice, including charters and vouchers, that he thought would invite excessive government regulation.  I confess that I paid little attention to Andrew’s warnings back then but I wish I had.  The experience and wisdom he obtained from studying history made him more sensitive to these dangers than my narrow practice of social science.  The autodidact had quite a lot to teach highly trained people like me.  As it turned out, choice reforms less prone to excessive regulation, like tax credit funded scholarships and ESAs, are now spreading rapidly — just as Andrew had expected and advised.

His humor often seemed to involve plays on words.  For example I once posted to Facebook a photo of what I (incorrectly) captioned as a “Golden Lion Tamarind.”  He made some sort of joke about how the dish was prepared.  Some people of faith find small typos and errors in language interesting because they think they can be unintentionally prophetic.  I don’t know if that was Andrew’s motivation but the thought of me secretly wishing to eat a small monkey is pretty funny.

I will miss that quirky humor, but more importantly I will miss his wise counsel and good cheer.  It’s a long game that must go on but something will be missing without Andrew as part of it.

(Guest Post by Jason Richwine)

The new experimental evaluation of the Louisiana voucher program poses a challenge to school-choice advocates such as myself. How do we explain the voucher students’ negative test outcomes – including a massive 0.4 SD drop in math scores – when evaluations in other cities showed neutral to mildly positive effects? Supporters have quickly coalesced around the explanation that Louisiana imposed regulations so suffocating that only the worst private schools participated in the voucher program.

I find that explanation unsatisfactory for a few reasons. First, it feels like a post-hoc excuse. Yes, choice advocates have been warning about burdensome regulation for years, including in Louisiana, but how many predicted that the state’s voucher system would go down in flames because of it? The magnitude of the score declines must surprise even the most vociferous critics of regulation.

More importantly, if the participating private schools are so bad – and other people apparently knew they were bad, given the declining enrollments – then why did the voucher recipients choose them? Did the parents fail to research their options? Do they not value academics much at all? Blaming the results on an unusually bad set of private schools is tempting, but it creates the new problem of having to explain why parents made such dubious choices.

Personally, I do not find it plausible that school quality alone could have so much impact, especially in one year. The traits that students bring with them to school – natural abilities, resilience, family support networks  – generally explain much more of the variance in student achievement than school quality. Only the absolute worst schools could have such deleterious effects, but there is no indication that the Louisiana voucher schools were the bottom of the barrel. Even if the one third of private schools that participated really were the worst third in the state, we are still talking about schools that are below average – not uniformly awful.

In trying to reconcile Louisiana with the successful experiments in DC, Milwaukee, Charlotte, etc., I suggest exploring other explanations. In particular, how well did the private schools align their curricula with the demands of the state tests? Maybe the private schools were simply teaching different material rather than teaching the state’s curriculum badly. Also worth examining is whether the randomization process, which was done within a complicated set of priority levels for admission, was conducted appropriately. Another issue is how schools adapt after the first year of statewide implementation. And, remember, it is not uncommon for studies to change significantly from the working paper phase to publication. So let’s be patient. Explaining this anomalous study will require more research.

I came across this video and am convinced that the world is a better place knowing that it exists.

This is not an odd-couple duet, like David Bowie and Bing Crosby singing Peace on Earth/Little Drummer Boy.  Dylan and Cash were actually mutual admirers and there is significant overlap in their musical styles.  So, I am not posting this for its ironic humor.  Instead, I think this is a particularly touching rendition of a beautiful song.  And for anyone who thinks Bob Dylan can’t sing, listen to him harmonize with Johnny Cash at the end.

It’s not a particularly polished performance.  But it sounds very different from Dylan’s solo version and has an air of authenticity around it.  Unlike the artificial feel of the Bowie and Crosby duet and most other TV variety show performances, this actually feels like two friends picking up the guitar and just playing a song together.  Enjoy.

(Guest Post by Ze’ev Wurman and Bill Evers)

In a recent piece on his web site Mike Petrilli, the president of the Fordham Institute, makes a rather strange case arguing that Common Core is alive (and well?). He quotes multiple blogs that argued the recent Massachusetts decision to abandon PARCC and replace it with MCAS 2.0 (Massachusetts’s own test) is a blow to Common Core and PARCC, but instead he argues that the replacement of PARCC by MCAS 2.0 “is no repudiation of PARCC.” His reasoning?

[T]here’s every reason to believe that MCAS 2.0 is going to look much the same as PARCC 1.0. This is akin to a state dropping the “Common Core” label but keeping nearly all of the standards. It’s essentially a rebranding exercise undertaken for political reasons.

Now, we don’t know whether Petrilli is right or wrong about MCAS 2.0, and we are quite willing to believe his sources are better than ours. So what Petrilli is actually saying is that PARCC has been saved in Massachusetts by its own elected officials deceiving their constituents and spending millions of extra dollars to create a fakely different MCAS 2.0. If he is right, this is in line with similar rebranding efforts elsewhere across the nation intended to confuse the parents and keep them in the dark. Yet even if true, one wonders why he brags about deceiving parents, and one wonders how Massachusetts will react to being deceived, given that a statewide ballot to repeal Common Core has just been submitted to the Secretary of State there.

Petrilli then goes on to “prove” that Common Core has been successful and hence is healthy. First, he argues that, with Common Core, standards are “dramatically improved” when compared to the prior state standards. Opinions vary on this point, yet what nobody disagrees with is that nationwide student achievement on the NAEP took a sharp dive across the board this year for the first time ever, precisely five years after Common Core’s “dramatically improved” standards were adopted by most states.

Petrilli continues and praises the “quality of the reading, writing, and math tests in use throughout the country” under Common Core, and their alignment “with college and career readiness,” arguing a “huge improvement” in what he calls the “honesty gap” between what test say and true college readiness. It seems Mr. Petrilli is unaware of the “honesty” of Common Core test results from California. For many years California assessed college readiness of high school students, and the results were slowly but steadily improving until last year. Suddenly, with the new Common Core “college-readiness” Smarter Balanced test, 20% more students this year are suddenly “college ready” in English and more than twice (110% more, to be precise) are “college ready” for math. If these clearly-fake results seems to suffer from Petrilli’s honesty gap, the number of conditionally college ready in English almost tripled(!) from less than 50 thousand to almost 140 thousand students overnight. Perhaps we should rename it Common Core’s “college readiness illusion” instead.

He goes on to acknowledge we have little evidence that Common Core improved instruction in the classroom, yet he argues that “we do know is that higher-quality curricular programs have been developed, and at least one—Eureka Math—is being widely used all across the country.” Actually, we do not know that higher quality programs are in effect, as the recent NAEP collapse indicates. Were truly higher-quality programs being used, we would expect the NAEP scores to rise rather than fall. Moreover, we would expect not to have hundreds of exemplars of math instructional idiocy propagated across the nation and on the Internet, all clearly labelled “Common Core Aligned.”

Finally, Petrilli turns to the comparability among states that Common Core supposedly allows. He somehow forgets to mention states like Washington, or Ohio, or Arkansas, which “re-defined” Proficiency on Common Core test results to suit their own needs rather than to aid comparability.

Based on all that, Petrilli declares that the Common Core standards “are still very much alive” and that cut-scores “are dramatically higher than ever.” We agree that Common Core is still – albeit barely – alive and that millions of students are going to be harmed before it is finally and completely dead. But we also observe that the federally-sponsored consortia are already almost dead and that their college-ready cut-scores are — how to put it politely? — fake.