Jay P. Greene's Blog

My student, Collin Hitt, and colleague, Julie Trivitt, have an amazing paper on how we can efficiently measure an important non-cognitive skill that is strongly predictive of later life outcomes.  A growing number of researchers have come to realize that lifetime success is partially a function of traditional academic achievement (cognitive skills) and partially a function of what are called non-cognitive skills, such as hard work, self-discipline, determination, etc…  Schools may play a central role in conveying both types of skills, but for the most part we have only been collecting information on cognitive skills in the form of standardized test results.  The main difficulty in expanding the types of measures we collect to include non-cognitive skills is that we have not developed efficient mechanisms for doing so.

Hitt and Trivitt have taken an enormous step forward to solve this problem.  They have discovered that student non-response on surveys (not answering questions or saying they don’t know) is an excellent measure of non-cognitive skills that are strongly predictive of later life outcomes.  In particular they examined survey response rates from the National Longitudinal Study of Youth (NLSY) given to students ages 13 to 17 in 1997.  The number of items that students answered was predictive of the highest level of education students attained by 2010, controlling for a host of factors including measures of their cognitive ability.  If students care enough to answer questions on a survey they are more likely to care enough to pursue their education further.

They then examined another data set to see if they found the same relationship.  They did.  The number of items that students in Milwaukee answered in a survey when they were in 9th grade was predictive of whether they graduated high school and went to college later, controlling for their academic achievement and other factors.

If this holds up when examined with multiple data sets, it will be an amazing breakthrough for researchers.  We will finally have a fairly easy to obtain measure of an important non-cognitive skill that is predictive of later life success.

When studying voucher or other school choice programs, for example, we have observed modest test score benefits for participants, but fairly large attainment benefits.  This suggests that school choice has larger effects on non-cognitive skills, but up until now we haven’t been able to observe these non-cognitive benefits without waiting nearly a decade to see if students graduate high school and go on to college.  With the Hitt and Trivitt measure, we will have an early warning indicator of whether students are acquiring non-cognitive skills and are more likely to have higher attainment later.

I am not suggesting that the Hitt and Trivitt measure can be used in an accountability system, since it is certain not to work once high stakes are attached.  But for research purposes it could be incredibly useful.

Developing an accurate and efficient measure of non-cognitive skills is especially important because one commonly considered measure, the self-reported “grit scale” developed by Angela Duckworth, may not be holding up very well.  In the recent Dobbie and Fryer evaluation of the Harlem Promise Academy, it actually appears that the Duckworth scale was a contrary indicator of later life success.  That is, students who rated themselves higher on the grit scale were less likely to succeed.  We have also tried the Duckworth scale in an experiment and found that it was uncorrelated with other, behavioral measures of non-cognitive skills, such as time devoted to a challenging task and delayed gratification.  But the self-reported grit scale was related to a student self-assessment of honesty, suggesting that the Duckworth scale may really measure how highly students will rate themselves more than actual grit or other non-cognitive skills.

Of course, the Hitt and Trivitt measure requires a lot more testing and field research, but it is one of the more exciting recent developments in education research.

When I was a counselor at Sun Fun summer day camp in Glencoe, Illinois we had an annual tradition of “boys day.”  The girls had a sleepover the night before, so they all went home to finally get some sleep, while just the boys and male counselors remained at camp.  We would divide the boys into color-coded teams and have a “color war” pitting each group against the others in every activity from singing to lanyard-making.  At the end of the day, following a giant capture the flag contest, the boys were so riled up that we could have conquered neighboring Winnetka and claimed it for Glencoe.  Hubbard Woods was ours for the taking!

At the time my fellow counselors and I used to joke that “boys day” resulted in an anarchic state like Lord of the Flies, with the only exception being that we didn’t kill Piggy.  But looking back on it, I see that summer camp was probably the closest thing to true liberty that our kids had experienced.  It was certainly more conducive to liberty than school, which gave almost exclusive emphasis to obedience to authority.  School was where kids were trained to obey the state and become cogs in a giant corporate machine.  Camp was where they learned to be free.

Yes, camp has rules.  Yes, camp has authority figures to enforce those rules.  But the rules are quite minimal compared to what we find in schools.  At camp, kids are given remarkable freedom to explore their interests and develop their individual personalities.  They literally can choose to fish in the morning and write poetry in the afternoon.

How is camp able to accommodate so much individual freedom while school seems determined to squash it?  A big part of the answer is that camps are generally organized around clear and strong missions, such as religion, sports, music, dance etc…   Because people usually choose their camps based on their agreement with the camp’s mission, the leaders of the camp do not have to regulate camper life so tightly to ensure that the organization’s mission is advanced.  Schools generally have weaker and less focused missions and so have to create a more oppressive environment to produce compliance.  Similarly, freely chosen governments have greater legitimacy and so do not have to use as much force on their subjects.

I’ve argued before that schools might have a lot to learn from camps.  They are both engaged in the activity of trying to prepare young people for adult life.  But I think camps are much more effective at preparing young people to be free adults.  I even think camps are remarkably effective at conveying traditional academic content.  And they do so at much lower cost. 

As the summer ends and school begins, think about why life has to change so dramatically for kids.  And next time someone starts talking about the benefits of year-round schooling, think about what would be lost if we further crowded-out liberty-loving camp for more oppressive schooling.  Yes, I know disadvantaged kids tend to have less enriching experiences over the summer, but we could address that with expanded camp opportunities — maybe even camp vouchers — rather than expanding the school year.  Let’s not forget the advantages of camp.

I noticed Mike Petrilli’s latest post on Flypaper and EdNext Blog about how Common Core will be great despite signs that it is being hijacked and misunderstood.  If only we implement the standards as he understands them and “we don’t let misguided ideas stand in our way,” it will signal “a return of history, civics, literature, science, and the fine arts to the elementary school curriculum.”

I tried to think of where else I’ve heard this argument and then I realized that Mike’s piece must have been mangled when it was posted.  I think I’ve been able to reconstruct what his original submission must have looked like, and here it is:

Communism’s Best Kept Secret

Shout it from the rooftops, tell all your friends: The Communist era signals a return of equality, decency, and brotherly love. That’s if we don’t let misguided ideas stand in our way.

If this is news to you, you’re not alone. But Karl Marx is doing his darndest to spread the word:

The success of Communism, adopted by more than 29 countries, is supremely important for many reasons, not least because of the recent intensification of global income inequality. But if you look at the actions of those 29+ countries, you will see that they have fallen short of Communist ideals and misunderstood the true spirit of our movement.

There’s a lot about Communism’s implementation that’s tough work and highly controversial. This is not one of them. What workers or vanguard of the proletariat don’t want to usher in the new utopia? Equality, dignity, compassion? Yes please!

Yet the revolutionary leaders might still find a way to screw this up—because they think gulags for dissenters and dachas for the elite are necessary.

So spread the word. As Marx urges in his article’s title, “Workers of the World Unite!  You Have Nothing to Lose but Your Chains.”

Equality is back!

(Guest Post by Collin Hitt)

Harlem Promise Academy is a charter middle school, part of the Harlem Children’s Zone. Previous studies from Will Dobbie and Roland Fryer have found big test score gains. A new paper by the Harvard research pair finds that the school had large impacts on college attendance, even larger than the previous gains in test scores would have indicated. From their new paper:

Attending the Promise Academy increases the probability of enrolling in college by 24.2 (9.7) percentage points, an 84 percent increase. In Appendix Table 2, we show that lottery winners are also 21.3 (5.9) percentage points more likely to attend a four-year college and 7.2 (2.3) percentage points less likely to attend a two-year college.

The charter school not only increases the likelihood that its students will attend college, but it increases the quality of the colleges that they attend. Harlem Promise Academy is considered an exceptional school in many minds because of its inclusion in the larger HCZ neighborhood experiment, which includes “wrap-around” social services meant to address issues of poverty. So Dobbie and Fryer collected lottery records at three other charter schools across the country that don’t feature HCZ-style community services, including Noble Network in Chicago, a personal favorite of mine. They found similar college enrollment gains.

They also tested whether the Promise Academy had an impact on lifestyle choices. Charter enrollment appeared to lower teen pregnancy rates by 71 percent and, for boys, drove the observed incarceration rate to almost zero.

They close with what I think is a crucial point for the academic community and the education reform movement to understand:

The education reform movement is based, in part, on two important assumptions: (1) high quality schools can increase test scores, and (2) the well-known relationship between test scores and adult outcomes is causal. We have good evidence that the rst assumption holds (Angrist et al. 2010, Abdulkadiroglu et al. 2011, Dobbie and Fryer 2011a). This paper presents the first pieces of evidence that the second assumption may not only be true, but that the cross-sectional correlation between test scores and adult outcomes may understate the true impact of a high quality school, suggesting that high quality schools change more than cognitive ability. Importantly, the return on investment for high-performing charter schools could be much larger than that implied by the short-run test score increases.

As discussed on this blog, there is now a litany of gold-standard studies of charter schools that find test score gains. Perhaps these studies provide only a glimpse of the benefits to come. We don’t know yet, which is why Dobbie and Fryer do what every smart researcher does – they call for more research.

For now, we can say one thing: ANOTHER random-assignment, gold-standard study finds impressive gains for charter schools. What is that now, thirteen? It’s actually getting hard to keep track.

P.S. There’s another intriguing finding. Alums of Harlem Promise Academy were given a survey that included Duckworth’s “Grit Scale,” which asked them to self-report their persistence, focus and work ethic. The charter school alums scored far lower than the comparison group.  This suggests that the self-reported Grit Scale may be a bad measure of actual grit, since it suggested the opposite of the grit outcomes that were observed.

I’m back from my hiatus and wonder whether I shouldn’t take a break more often given all of that great posts and discussion that occurred in my absence.

I also feel renewed enthusiasm for work from an item I saw in this morning’s Wall Street Journal.  If you think education policy is being made on flimsy research, just look at what the WSJ says some investment firms are using to make decisions.  In Japan traders have noticed a relationship between the airing of Studio Ghibli films, like Kiki’s Delivery Service and Spirited Away, on Japanese TV and bad economic news.  And they are making investment decisions timed with these broadcasts.

Here’s how the WSJ describes it:

Traders call it the “Ghibli Rule” or the “Curse of Ghibli.”… Believers point to the uncanny accuracy of the “rule.” Since January 2010, NTV has aired Ghibli films 24 times. In the following Tokyo trading session, the dollar fell versus the yen nearly three-quarters of the time.

More:

On July 8, 2011, during a showing of “Kiki’s Delivery Service,” a Ghibli film about a young witch and her cat, the payroll numbers came in 86% below expectations and the dollar fell 1.2%. The following Monday, Japan’s benchmark index fell 0.7%.

“I always factor into my trading that when a Ghibli movie airs on a Friday, the dollar-yen market could get volatile,” says Yukio Nakamura, a senior manager at a French insurance company in Tokyo, who dabbles in foreign exchange on the side. “I don’t watch Ghibli movies on TV myself, but I’m always checking the broadcast schedule as a kind of risk hedge.”

Ummm… O.K.  So if Gates wants teachers to make educational decisions based on glorified mood rings while ignoring their own positive random assignment results on small schools and early college, they look like Socrates compared to Japanese traders.  See?  Education policy-making could be worse.