Jay P. Greene's Blog

I noticed Mike Petrilli’s latest post on Flypaper and EdNext Blog about how Common Core will be great despite signs that it is being hijacked and misunderstood.  If only we implement the standards as he understands them and “we don’t let misguided ideas stand in our way,” it will signal “a return of history, civics, literature, science, and the fine arts to the elementary school curriculum.”

I tried to think of where else I’ve heard this argument and then I realized that Mike’s piece must have been mangled when it was posted.  I think I’ve been able to reconstruct what his original submission must have looked like, and here it is:

Communism’s Best Kept Secret

Shout it from the rooftops, tell all your friends: The Communist era signals a return of equality, decency, and brotherly love. That’s if we don’t let misguided ideas stand in our way.

If this is news to you, you’re not alone. But Karl Marx is doing his darndest to spread the word:

The success of Communism, adopted by more than 29 countries, is supremely important for many reasons, not least because of the recent intensification of global income inequality. But if you look at the actions of those 29+ countries, you will see that they have fallen short of Communist ideals and misunderstood the true spirit of our movement.

There’s a lot about Communism’s implementation that’s tough work and highly controversial. This is not one of them. What workers or vanguard of the proletariat don’t want to usher in the new utopia? Equality, dignity, compassion? Yes please!

Yet the revolutionary leaders might still find a way to screw this up—because they think gulags for dissenters and dachas for the elite are necessary.

So spread the word. As Marx urges in his article’s title, “Workers of the World Unite!  You Have Nothing to Lose but Your Chains.”

Equality is back!

(Guest Post by Collin Hitt)

Harlem Promise Academy is a charter middle school, part of the Harlem Children’s Zone. Previous studies from Will Dobbie and Roland Fryer have found big test score gains. A new paper by the Harvard research pair finds that the school had large impacts on college attendance, even larger than the previous gains in test scores would have indicated. From their new paper:

Attending the Promise Academy increases the probability of enrolling in college by 24.2 (9.7) percentage points, an 84 percent increase. In Appendix Table 2, we show that lottery winners are also 21.3 (5.9) percentage points more likely to attend a four-year college and 7.2 (2.3) percentage points less likely to attend a two-year college.

The charter school not only increases the likelihood that its students will attend college, but it increases the quality of the colleges that they attend. Harlem Promise Academy is considered an exceptional school in many minds because of its inclusion in the larger HCZ neighborhood experiment, which includes “wrap-around” social services meant to address issues of poverty. So Dobbie and Fryer collected lottery records at three other charter schools across the country that don’t feature HCZ-style community services, including Noble Network in Chicago, a personal favorite of mine. They found similar college enrollment gains.

They also tested whether the Promise Academy had an impact on lifestyle choices. Charter enrollment appeared to lower teen pregnancy rates by 71 percent and, for boys, drove the observed incarceration rate to almost zero.

They close with what I think is a crucial point for the academic community and the education reform movement to understand:

The education reform movement is based, in part, on two important assumptions: (1) high quality schools can increase test scores, and (2) the well-known relationship between test scores and adult outcomes is causal. We have good evidence that the rst assumption holds (Angrist et al. 2010, Abdulkadiroglu et al. 2011, Dobbie and Fryer 2011a). This paper presents the first pieces of evidence that the second assumption may not only be true, but that the cross-sectional correlation between test scores and adult outcomes may understate the true impact of a high quality school, suggesting that high quality schools change more than cognitive ability. Importantly, the return on investment for high-performing charter schools could be much larger than that implied by the short-run test score increases.

As discussed on this blog, there is now a litany of gold-standard studies of charter schools that find test score gains. Perhaps these studies provide only a glimpse of the benefits to come. We don’t know yet, which is why Dobbie and Fryer do what every smart researcher does – they call for more research.

For now, we can say one thing: ANOTHER random-assignment, gold-standard study finds impressive gains for charter schools. What is that now, thirteen? It’s actually getting hard to keep track.

P.S. There’s another intriguing finding. Alums of Harlem Promise Academy were given a survey that included Duckworth’s “Grit Scale,” which asked them to self-report their persistence, focus and work ethic. The charter school alums scored far lower than the comparison group.  This suggests that the self-reported Grit Scale may be a bad measure of actual grit, since it suggested the opposite of the grit outcomes that were observed.

I’m back from my hiatus and wonder whether I shouldn’t take a break more often given all of that great posts and discussion that occurred in my absence.

I also feel renewed enthusiasm for work from an item I saw in this morning’s Wall Street Journal.  If you think education policy is being made on flimsy research, just look at what the WSJ says some investment firms are using to make decisions.  In Japan traders have noticed a relationship between the airing of Studio Ghibli films, like Kiki’s Delivery Service and Spirited Away, on Japanese TV and bad economic news.  And they are making investment decisions timed with these broadcasts.

Here’s how the WSJ describes it:

Traders call it the “Ghibli Rule” or the “Curse of Ghibli.”… Believers point to the uncanny accuracy of the “rule.” Since January 2010, NTV has aired Ghibli films 24 times. In the following Tokyo trading session, the dollar fell versus the yen nearly three-quarters of the time.

More:

On July 8, 2011, during a showing of “Kiki’s Delivery Service,” a Ghibli film about a young witch and her cat, the payroll numbers came in 86% below expectations and the dollar fell 1.2%. The following Monday, Japan’s benchmark index fell 0.7%.

“I always factor into my trading that when a Ghibli movie airs on a Friday, the dollar-yen market could get volatile,” says Yukio Nakamura, a senior manager at a French insurance company in Tokyo, who dabbles in foreign exchange on the side. “I don’t watch Ghibli movies on TV myself, but I’m always checking the broadcast schedule as a kind of risk hedge.”

Ummm… O.K.  So if Gates wants teachers to make educational decisions based on glorified mood rings while ignoring their own positive random assignment results on small schools and early college, they look like Socrates compared to Japanese traders.  See?  Education policy-making could be worse.

I’ll interrupt my hiatus to post this video of my lecture to the Georgia Public Policy Foundation as part of their Friedman Day celebrations.  In this lecture I discuss ways in which aspects of the education reform movement are in danger of becoming the thing that they oppose.  In particular, I discuss Common Core, top-down accountability, and narrowing education to focus only on work-related skills.

I’m going to be taking a break over the next month to get some other stuff done, so I don’t plan on posting during July.  I’ve asked a few friends to join Matt and Greg in putting up some posts, just so that the blog is not completely inactive.

See you in August.

(Guest Post by James Shuls)

In a recent Education Next blog post, Peter Meyer wrote about the tendency of Common Core opponents to conflate the idea of content standards with curriculum. He writes, “It is not a small distinction, since standards provide goals and a curriculum provides the day-to-day, week-to-week, year-to-year road map for reaching those goals.” He goes on to say, “From both a pedagogical and political point of view, it is crucial to keep the distinction between standards and curriculum clean and clear. But, most importantly, we need to try as hard as we can to get the facts straight.”

I agree with Meyer in many regards. If Common Core supporters want to build support for the standards among Republicans they absolutely must differentiate the Common Core from curriculum. I also agree that we need to get the facts straight. Unfortunately, this is something that Meyer’s post fails to do.

This conflation of ideas is not simply something Tea Party activists or Common Core opponents are guilty of, it’s widespread. Do a Google search for “Common Core Content Standards” and “Common Core Curriculum Standards.” You’ll get more hits for the later. Curriculum and content standards are often used interchangeably. Teachers, principals, and even assistant superintendents are guilty of as much.

Last year I contacted my children’s school and requested a copy of the curriculum. They sent me a one-page summary of the Common Core and an excerpt from Children’s Mathematics: Cognitively Guided Instruction (CGI)

When I met with them and asked about the curriculum for spelling and reading. The teacher’s response was, “We’re following Common Core.”

When I asked about the math curriculum, the teacher told me they were using CGI to teach the Common Core. The principal even told me that she researched it by asking the district’s assistant superintendent for education about the district’s math curriculum. The answer—Common Core State Standards.

I tried to correct them. I tried to point out that these are standards not curriculum, but they were insistent. The problem is that the distinction between a “content standard” and “curriculum” is only a matter of degree. This distinction is not clear, not even for many educators.

Let’s look at an example. If I say students in first grade should be able to “Express the length of an object as a whole number of length units, by laying multiple copies of a shorter object (the length unit) end to end” Is that simply a standard or is that an activity that could be part of a lesson?

Meyer somehow tries to defend the distinction between Common Core and curriculum by noting that the standards must be complimented by a curriculum; an attempt that falls flat on its face. Demonstrating that a Common Core curriculum must be developed to implement the Common Core Standards simply illustrates that the standards will dictate curriculum to local schools. This strengthens the link between standards and curriculum.

James Shuls is the education policy analyst at the Show-Me Institute

One of the bigger problems in education policy is hubris.  People regularly claim that they know what the right policies or practices are, and things would be better if only others would bend to their will.  The truth is that we know relatively little about effective education policies and practices.  This isn’t for lack of trying.  Despite considerable research effort and policy inquiry, we’ve found remarkably few “universal truths” about effective education.  Part of the difficulty is that knowing what works presupposes that there is a single, best way.  But  it appears much of what is effective in education is contingent on particular needs and circumstances and does not lend itself to broad declarations about the “right”  practices and policies.

Because the scourge of PLDD is endemic, however, we continue to hear claims that “We know what works.”   This was the traditional refrain of teacher union leaders, but now reformers have joined the hubris chorus.  The latest example of this is the ratings of Ed Schools issued by the National Council of Teacher Quality.  NCTQ claims to know what good teacher preparation programs should be doing and judges those programs against NCTQ’s vision of effective practices.

In particular, NCTQ identifies 18 standards by which it judges Ed Schools.  “Our standards for the first edition of the Teacher Prep Review” NCTQ assures us, ” are based on research; internal and external expert panels; the best practices of other nations and the states with the highest performing students; and, most importantly, what superintendents and principals around the country tell us they look for in the new teachers they hire.”

NCTQ describes the research basis for their standards in a lengthy document.  Yet, even according to their own description only 8 of the 18 standards are supported by “strong research.”  And in most of the 8 cases where they do claim to have strong research support, the research does not actually provide them with the strong support they assert.

For example, the “Early Reading” standard assesses whether “The program trains teacher candidates to teach reading as prescribed by the Common Core State Standards.”  None of the studies they cite actually examine the specific standard since none specifically examine what methods of teaching reading, if any, are actually prescribed by Common Core.  As is the case with all 18 standards in the NCTQ rating system, one has to make a series of leaps between the research cited and the actual standard being used to judge teacher prep programs.

In the case of early reading, the “strong research” they cite examines whether teachers are familiar with the “five components of effective reading instruction,” and whether teachers who are certified and have masters degrees are more likely to know those five components.  It turns out teachers are generally not familiar with the five components and are no more likely to know them if they are certified or have a masters.  That’s all very nice, but isn’t the “strong research” supporting the standard supposed to show that knowledge of the five components, which presumably have something to do with teaching “reading as prescribed by the Common Core State Standards,” actually lead to improved reading by students?  The strong research cited by NCTQ says it generally doesn’t: “This study also found no relationship between teachers’ knowledge of these components and their students’ reading growth – with the notable exception of third-grade students.”  This is typical of the “strong research” supporting 8 of the 18 standards by which NCTQ judges Ed Schools.

Standards 1 and 6 address whether teacher prep programs select “teacher candidates of strong academic caliber” and whether “teacher candidates have the broad content preparation necessary to successfully teach to the Common Core State Standards.”  In both cases the “strong research” on which these standards rely is a study by Boyd, et al examining the relationship between teacher characteristics and student achievement.  Let’s leave aside the fact that NCTQ acknowledges that research by Harris and Sass as well as Chingos and Peterson contradict their standard.  Even the Boyd, et al study they do cite does not specifically demonstrate that teachers from more selective programs or with more content training are more effective.  First, Boyd, et al are careful not to make the type of strong causal claims from their work that NCTQ does:

It is not easy to estimate how the achievement gains of students are affected by the qualifications of their teachers because teachers are not randomly sorted into classrooms. For example, if teachers in schools in which students perform best in math are more likely to be certified in math, one might be tempted to conclude that being certified to teach math contributes to higher student achievement. The causal relationship, however, may operate in the other direction; that is, more qualified teachers may be in schools where students perform well in math because they prefer to teach good students and because employers want to staff their courses with in-field certified teachers. Analysts need to be careful not to attribute the test-score gains associated with sorting to the attributes of teachers.

Beyond the fact that Boyd, et al would not make the strong causal claims from their work that NCTQ feels free to do, the Boyd, et al study examines a basket of teacher qualifications and does not claim to be able to distinguish accurately between teacher experience, selectivity of the college they attended, content knowledge, and other characteristics because “many of the measures of teachers’ qualifications are highly correlated with each other.”  In short, the Boyd, et al study is hardly the “strong research” in support of their standards that NCTQ claims it is.

Do we need more examples of how NCTQ misinterprets or stretches research to claim that their standards are supported by “strong research”?  Oh, how about one more…  Standard 13 is “Equity” and judges teacher prep programs based on whether “The program ensures that teacher candidates experience schools that are successful in serving students who have been traditionally underserved.”  The “strong research ” NCTQ cites for support of the claim “that entering teachers learn crucial methods of instruction and management through observation of and supervised practice in schools where staff are successfully teaching students living in poverty” is a piece by Ronfeldt.

Unfortunately, Ronfeldt’s study appears to make the opposite claim.  He finds that it is more important for student teachers to be trained in schools with low staff turnover that tend to have more advantaged students.  He concludes:

Should we place student teachers in “difficult-to-staff, underserved” schools to learn to teach? The main
findings of this study suggest otherwise – learning to teach in difficult-to-staff field placement schools is associated with lower teacher effectiveness and retention. Moreover, the results demonstrate that being trained in field placements with higher concentrations of poor, black, and lowest-achieving students has no significant effect on teacher retention or effectiveness.

I haven’t see this much unreliable citation of research since I read teacher union reports.

To be fair, NCTQ acknowledges that quality research on effective education practices is in short supply: “To the extent that high-quality research can inform how teachers should be prepared, NCTQ uses that research to formulate standards. Unfortunately, research in education that connects preparation practices to teacher effectiveness is both limited and spotty.”  But this lack of evidence does not prevent NCTQ from confidently declaring that they know what teacher prep programs should be doing and judging them on that basis.  If quality research is so limited, how does NCTQ know what everyone else is supposed to be doing?

And I’m sure that there is considerable room for improvement in teacher prep programs.  Many of NCTQ’s recommendations are probably sensible, even if they aren’t backed by “strong research.”  The problem is not so much that NCTQ is suggesting bad ideas as that they are claiming to know much more than they actually know.  And they are willing to boss around everyone else despite not knowing as much as they think.

Maybe we’d make more progress in improving teacher prep programs if we were more upfront about what we didn’t know and encouraged more experimentation and data-collection so that we can learn more.  And given that different circumstances may call for different practices, maybe we should be open to a variety of Ed School approaches rather than attempting to impose the one true way.