Jay P. Greene's Blog

(Guest post by Greg Forster)

OCPA’s Perspective carries my response to folks in Oklahoma who have tried to peel off small-government types (thick on the ground in that state) from the school choice coalition by arguing that choice expands the state:

Now, it is true that ESAs are an “entitlement” in the way that term is normally used in the context of public policy. Like Social Security and food stamps, they create a benefit to which all people in the relevant class (retirees for Social Security, the poor for food stamps, parents for school choice) are entitled under the law.

The key difference is that Social Security, food stamps, and other typical entitlement programs represent the expansion of government into a leading role in areas previously dominated by private savings, employer-paid pensions, church and community organizations, and other non-governmental solutions…

School choice, by contrast, reverses the endless expansion of government by moving us away from a government monopoly…Compared to the government school monopoly, school choice liberates individual initiative, economic interdependence, and spiritual community. It allows parents to take control of their children’s education, becoming stewards over their own lives, instead of treating them like perpetual wards of government—as if they were cattle in the government’s pen. It supports educational entrepreneurs who create new school systems designed to serve the customer base created by school choice. And it allows schools to have a holistic vision of what it means to be an educated person—one that doesn’t yank the leash and stick a gag in teachers’ mouths when students ask big spiritual questions about the meaning and purpose of human life.

Oh, and it also contains this moment:

The email exchange was later made public by subpoena, to Hofmeister’s embarrassment. (Why anyone involved in government uses email or texting is beyond me.)

Come for the well-deserved embarrassment of a politician caught in shenanigans, stay for the political philosophy of the modern state!

I’ve written several times before about the disconnect between changing test scores and changing life outcomes.  In general, even when we can rigorously identify changes in math and reading test results caused by schools or programs, we have weak and inconsistent evidence that this produces commensurate changes in later life outcomes.  Keeping in mind that test-based accountability mostly focuses on the level of test scores, not changes, and virtually never relies upon a rigorous identification of how test scores are caused by schools and programs, we have no way of knowing that that the kinds of schools, programs, and practices that we are pushing in education will actually help kids later in life.  We might actually be hurting them and are certainly foreclosing potential opportunities based on our false confidence that we (policymakers, researchers, and pundits) are better at judging quality than are families.

Rather than contemplate the implications of our ignorance, people in our field are inclined to press ahead.  Yes, our measures are imperfect, they’ll admit, but we have to do the best we can with what we have.  Or they will say we just have to try harder to develop better measures of knowledge acquisition or expand our measures to include non-cognitive skills that provide a more complete picture of the recipe for success.

But what if there is simply no recipe for success?  Or more precisely, what if the recipe for success is highly context-dependent so that distant policymakers, researchers, and pundits are unable to prescribe what programs and schools should be cooking?  Maybe only parents, communities, and local educators are well-enough positioned to make reasonable (if imperfect) judgments about what each child needs.

If you were holding out hope that the expansion of educational measures to include non-cognitive skills would give policymakers, researchers, and pundits a stronger ability to prescribe how and where students should be educated, I have some bad news for you.  The disconnect between educational measures and later life outcomes is at least as severe in non-cognitive measures as it is in test scores.  Let’s leave aside the fact that most non-cognitive measures are too easily gameable to be used for any accountability purposes.  Even if only for research purposes, there does not appear to be a straightforward and consistent connection between non-cognitive measures and later life outcomes.

A new study led by Nicholas W. Papageorge at Johns Hopkins University and IZA examines the connection in Great Britain between teacher reports about behavior when students are 11 and later life outcomes for those students.  Because non-cognitive measures are in their infancy, we aren’t entirely sure how to slice and dice the measures and do not have a clear system for labeling the related concepts we are measuring.  In this study, if we simply lumped all of the teacher reports of misbehavior together we would find that students who misbehave more tend to do worse later in life.

But if we split misbehavior into two categories — one that captures misbehavior directed toward others (externalizing) and another that captures whether students are misbehaving because they are withdrawn (internalizing) — the picture gets more complicated.  Students who score poorly on measures of internalizing misbehavior still seem to fare poorly later in life.  But for students who score lower on the externalizing misbehavior, how they fare later depends on their social class.  If students are from more advantaged backgrounds, externalizing is actually associated with higher earnings, while for more disadvantaged students externalizing seems to have no effect on earnings.

This null to positive effect on earnings for a certain type of misbehavior occurs despite that fact that externalizing is associated with lower levels of educational attainment.  That is, students who misbehave toward others don’t go as far in school, but they earn more in the workplace if they come from more advantaged backgrounds despite that negative effect on educational attainment.

Yet gain, we see that what we think is “good” performance on a a near-term educational measure is highly dependent on context and is not connected to later outcomes in a straightforward way.  Policymakers, researchers, and pundits are inclined to say that scoring higher on a measure of behavior is better, but that is not necessarily the case.  It depends on the type of behavior and who the student is.

In addition, things we do to increase educational attainment may come at the expense of later earnings.  Increasing compliance with school authorities may help students go further in school, but may stifle the initiative and ambition necessary to make larger contributions to the economy later.  Policymakers, researchers, and pundits cannot simply identify a set of educational measures (test scores, non-cog measures, or educational attainment) and judge from afar which programs and schools are going to help students succeed later in life.  You can’t just maximize these measures and expect uniformly good results.

None of this should be surprising to parents engaged in the complicated task of raising their children.  We often want our children to possess certain qualities — but not too much of those qualities.  We want our children to be obedient, but not too obedient.  We want them to be ambitious, but not too ambitious.  We want them to value abstract knowledge, but not too much abstract knowledge.  We want them to value being in school, but not stay in school forever.  And we know that the approaches we take to produce these balanced outcomes vary for each child, even within our own family let alone across all families.

As it turns out, educating children is simply an extension of raising children to be the kinds of adults we hope they can be, requiring all of the same nuance, balance, and judgment.  Solutions imposed by distant policymakers, researchers, and pundits are no more likely to be effective in educating our children than they would be in raising them.  Yes, parents, communities, and local educators will make mistakes in raising children as well as educating them.  But they are better positioned to understand the context and achieve the appropriate balance for each child than are distant policymakers and experts, even if they are well-intentioned and highly knowledgeable.

The German movie, Er ist wieder da (available on Netflix as Look Who’s Back), is no ordinary political comedy.  On one level it’s quite disturbing and not a comedy at all, while at the some time it is a brilliant and hilarious satire.  The premise is that Hitler is somehow not dead and finds himself in modern Berlin.  He’s taken in by a desperate free-lance film-maker who introduces him to a set of conniving TV executives seeking ratings with what they think is a comedy act.  But he’s no comedian.  He’s really Hitler, adapting to our times and re-building political support.

There are many successful movies featuring Hitler, including The Producers, The Great Dictator, and (one of my all time favorites) Inglourious Basterds, but they generally portray Hitler as a buffoon.  In this film Hitler is occasionally buffoonish, but he is also a keen observer of people and politics.  He immediately detects that Germany’s current nationalist party, the NDP, would be an inadequate vehicle for his return to power.   He even storms their headquarters and denounces them as a pack of losers, which they obviously are.  Instead, he sees potential for the rise of authoritarianism in the Green Party.  That’s both astute and hilarious.

The film also mixes scripted scenes with improvised ones in which Hitler encounters real people on the streets of Germany.  He’s shown (for the most part) being warmly received, with people taking selfies and laughing.  Perhaps they also think he is a comedian and are going along with the joke.  But others give him Nazi salutes and describe their complaints about immigrants.  He listens as a very sympathetic and effective politician.

At one point a “man on the street” share his vision of democracy with Hitler.  He says that we need a democracy that is willing to put its foot down more: “That’s how we do it! Point, finished! No discussion!”  Hitler replies, “You’re absolutely right, and that is exactly my kind of democracy!”  Again, both astute and hilarious.

There are many ways a movie like this could go wrong and at times it does go off the rails, but not very often.  The film could become a heavy-handed parable about today’s nationalist politicians.  It avoids that by emphasizing how today’s nationalists lack the skill and energy that Hitler possessed.  At the same time, the film does not attribute to Hitler magical powers to hypnotize us into backing authoritarianism.  It shows us as either wanting authoritarianism or being too easily distracted by frivolous things to bother to stop it.  Hitler is just capable of exploiting that opportunity.

The film’s mixture of real and fictional, comedy and serious commentary, are disorienting.  At one point a TV executive warns what she thinks is comedian Hitler not to do jokes about Jews because they aren’t funny.  Hitler agrees that there is nothing funny about Jews.  That is simultaneously serious commentary and a hilarious joke.

Look Who’s Back at times feels like Borat but it is more like the brilliant 1976 Oscar-winning movie, Network.  It’s a satire that is often more serious than funny and more disturbingly accurately than most dramas.  If you haven’t seen Network you really should.  It’s as if the script writer, Paddy Chayefsky, had a time machine and could see how TV news would turn into its current manifestations of 24 Hour News channels and Twitter.  Let’s hope Look Who’s Back is not similarly prescient.

Some people have been puzzled as to why I’ve been studying how cultural activities, like visiting an art museum or seeing live theater, affect students.  Why don’t I do what almost everyone else in our field does and just study how various interventions affect math and reading test scores?

Well, I’ve been making the argument for a while now that there is remarkably little evidence linking near-term changes in test scores to changes in later life outcomes for students, like graduating high school, enrolling in college, completing college, and earnings.  I have yet to see anyone bother to refute my observation of this weak and inconsistent connection between test score changes and life changes.  No matter, researchers, foundations, and policymakers continue to plod along as if changing test scores should be the focus of our efforts. Whether kids go to art museums or see live theater is at best an amusing sideline or at worst a harmful distraction from the primary goal of education, which they believe is boosting math and reading test scores.

But now we have a rigorously designed study out of Denmark that shows cultural activity among students is strongly (and likely causally) related to later academic success.  The study appears in Social Science Research, a Sociology journal that was co-founded by James Coleman.  It examines a large sample of monozygotic twins in Denmark to see if their cultural activity was related to their teacher-given GPA, exam-based GPA, and rate of completing secondary school.  To measure cultural activity they relied on a survey administered to the mothers of those twins that asked about what their children did when they were 12 years old.  It asked things like: “How often child went to any type of museum” and “How often child went to the theater or a musical performance.”

By comparing outcomes among identical twins, the researchers hope to control automatically for a large set of unobserved environmental and genetic factors.  We could reasonably believe that a large portion of the variation in cultural capital among twins was due to chance and not differences in their upbringing or ability.

The researchers found that the twin whose mothers reported having higher cultural capital at age 12 had significantly higher marks on their end of compulsory school exams at age 15/16.  They also found “an
increase in cultural capital of one standard deviation is estimated to increase the likelihood of completing upper secondary education by 12.5 percentage points.”

Cultural capital was not a significant predictor of the grade point average students received from their teacher when they were 15, which was contrary to the researchers’ expectations.  Earlier theory had suggested that cultural capital might improve academic performance by making students falsely appear more knowledgable, even if their command of the material were no greater.  As they put it: “Bourdieu argued that cultural capital, that is familiarity with the dominant cultural codes in a society, is a key determinant of educational success because it is misperceived by teachers as academic brilliance and rewarded as such.”

This study found that not to be the case.  Instead, their findings are more consistent with the arguments advanced by E.D. Hirsch and others that cultural capital gives students a stronger foundation of broad knowledge that then facilitates future knowledge acquisition.  And the significant increase in completing secondary school may be a function of that broader knowledge, as opposed to the narrow knowledge captured in math and reading standardized tests.  Cultural activity may also increase graduation rates by giving students more ways to be engaged with school on top of traditional academic coursework.

So the next time someone asks me why it matters whether students go to art museums or see live theater, I can tell them that there is at least as much rigorous evidence showing the long term benefits of cultural activity as there is for interventions designed to boost standardized test scores.

(Guest post by Patrick J. Wolf)

We parents all have heard the claim that something wasn’t fair.  “Suzie got a bigger piece of cake than I did!”  “Tommy got to go fishing while I had to clean the garage!”  “Malachi had a lot more money spent on his education because you sent him to a traditional public school and me to a public charter school!”  Well, maybe we haven’t actually heard that last one very often but it would be a more legitimate gripe than the other ones.

Students in public charter schools receive $5,721 or 29% less in average per-pupil revenue than students in traditional public schools (TPS) in 14 major metropolitan areas across the U. S in Fiscal Year 2014.  That is the main conclusion of a study that my research team released today.    Of the cities we examined, some have large and well-established charter sectors, like Houston, Los Angeles, New York, and Washington, while others have more emerging charter school sectors like Little Rock, San Antonio, and Tulsa.

Twelve of the 14 cities have a disturbing charter school funding gap of more than 10%, which earned them a C grade or lower.  Tulsa, Little Rock, Indianapolis, Washington, Los Angeles, Oakland, and Camden earned an F for funding equity since there is a funding gap of more than 30% between what charter schools received versus what TPS received per pupil. Camden had the largest per-pupil funding gap in our study, with charter schools students receiving 45%, or $14,771, less per pupil than TPS students.

Shelby County, TN, which includes the city of Memphis, is the only metropolitan area in the study that funded students in public charter schools at a higher level than TPS.  Shelby County charter students received $10,624 in per-pupil funding in FY 2014 compared to $9,720 per student in the county’s TPS.  Houston’s charter schools were funded just 2% below their TPS, and earned the only grade of A in the study, in part because they were able to raise almost $900 per student in nonpublic revenue.  Funding gaps of $1,500 per student or more for charters remained in 10 of the 14 cities even after excluding all special education expenditures from the comparison.

The main source of the funding gap is local revenues.  Traditional public schools received $7,000 more per pupil in local revenues, on average, than did public charter schools.  Charter schools are public schools, in local communities, that must enroll all students who want to attend (or hold a random admissions lottery).  The parents of charter school students pay local taxes just like the parents of TPS students.  The fact that eight of the 14 cities in our study provided essentially no local education revenue to their public charter schools is shameful.  That’s simply not fair.

Our previous study of charter school funding equity at the state level was criticized for not exempting expenditures on such items as transportation and central administration that are mandatory for TPS but discretionary for public charter schools.  In our view, that’s exactly the point.  Charter schools are permitted to be innovative as an alternative to the more rigidly controlled administrative and spending structure of TPS.  True, the revenue amounts received by charters and TPS are more even once you exclude all of the ways that public schools are forced to be inefficient.  Like I said.  That’s exactly the point.

What are the takeaways for education policy?  Our results support the recommendations of the Fordham Institute and others to fund students directly, using a weighted student funding formula, a.k.a. “backpack” funding.  Placing public charter schools on a par with TPS in receiving local educational funds, as Colorado plans to do, would bring over half the cities in our study to funding parity across the two public school sectors.

States like Massachusetts, Texas, and Denver have tried to compensate for local funding discrepancies in their charter sectors by providing higher state funding to charter students, but that move hasn’t closed the funding gap.  It merely got Houston close enough so that the extra-ordinary fundraising efforts of its charter schools were able to move charter students close to parity.  Such bricolage arrangements are simple guesswork and no substitute for a rational student-based funding policy that treats the same student similarly regardless of the local public school their parent chooses for them.  Ask your children.  Anything else is just not fair.

(Guest Post by Jason Bedrick)

To test, or not to test? That is the question at the center of one of Matt Barnum’s latest and last articles at The 74 (he’s moving over to Chalkbeat). Taking opposite sides of the “testing and educational choice” question are Doug Harris and yours truly.

To test:

“If you’re not going to require anything, we’re not going to know anything about whether these programs worked or not,” said Doug Harris, a Tulane University professor who has done extensive research on school choice in New Orleans and has been a critic of DeVos’s approach to education reform. “When you put it that way, even the people who are somewhat supportive of the idea get a little squeamish.”

Not to (impose the state standardized) test:

“Since there is strong evidence that state testing mandates tend to have negative unintended consequences, such as narrowing the curriculum and distorting how schools teach kids, we’d rather that schools have the freedom to use the tests that are more closely aligned to their curriculum and give parents the freedom to choose the schools that work best for their kids,” said Jason Bedrick, of EdChoice, an Indianapolis-based group that backs voucher and tax credit programs. [links added]

… Bedrick, of EdChoice, says he’s fine with requiring schools to administer a national test, but not the state exam, even if it means that studying the program then becomes difficult.

“As researchers, we’d like the apples-to-apples comparisons that a single test provides,” he wrote in an email. “However, as policymakers, we also have to do what’s in the best interests of kids.”

“The main point of choice policies is to empower parents to choose schools based on the criteria that are important to them, not just to raise [specific] test scores,” Bedrick said, citing factors such as student-teacher ratio, curriculum, and college acceptances.

Later in the piece, Barnum cites me saying, “We don’t think it is necessary to impose standardized tests. If parents demand that information, schools will provide it.” Harris disagrees:

“I think that’s hooey … Schools themselves, even if they wanted to provide that kind of information, can’t provide that in a way that’s comparable across schools — you have to have a coordinating mechanism for information.”

If only the market had a coordinating mechanism for information that parents used! Oh right, not only do such mechanisms already exist, but they’re also growing in popularity, and they get far more web traffic than any state education agency website:

GreatSchools.org is not the only website that ranks pre-K-12 schools. The Internet search engines Yelp and Google offer school ratings, as do websites such as Schooldigger.com and Privateschoolreview.com. But with 40 million annual unique visitors, GreatSchools is the one most used, according to Alexa Internet, which tracks Web traffic.

The site’s founder and chief executive officer, Bill Jackson, says GreatSchools wants to be more than just a school ratings site: He sees it developing into an association that serves parents in the same way that the AARP serves retirees, or that AAA represents drivers.

Perfect information? Of course not. Lots of room for improvement? Absolutely. But they already do more than any state agency to empower parents with the info they want.

And “want” is the key word there. EdChoice’s More Than Scores survey of parents of tax-scholarship students in Georgia found that most parents do *not* place a very high value on test scores:

Student performance on standardized test scores is one of the least important pieces of information upon which parents base their decision regarding the private school to which they send their children. Only 10.2 percent of the parents who completed the survey listed higher standardized test scores as one of their top five reasons why they chose a particular private school for their child.

Just ask real parents, such as, well… Doug Harris:

Research, including Harris’s, suggests that parents do generally place significant value on a school’s test scores, but it may not be the biggest factor driving school preferences.

As a parent himself considering private schools, Harris says he’s never asked any of them for their test scores. [emphasis added]

“Most of these decisions, and this is true public, private, or otherwise, are based more on reputation, on what your neighbors say,” he said.

Tests can be informative for parents, but policymakers should dial back their obsession. After all, they measure only a thin slice of what we want schools to provide. Parents take a more holistic view of their child’s education, and the narrow focus on tested subject can have negative unintended consequences. For example, one researcher recently acknowledged that one reason there “hasn’t been as much actual innovation [in the charter school sector] as maybe the original charter folks hoped” is that test-based accountability is hampering innovation:

[W]hen you have intense test-based accountability it really restricts what you can do and to what degree you can innovate because you have to put so many of your resources towards the same end. There are only so many ways to make test scores go up. So, I think that really restricts what they can do.”

That researcher? Doug Harris.