Cultural Activity Matters

May 20, 2017

Some people have been puzzled as to why I’ve been studying how cultural activities, like visiting an art museum or seeing live theater, affect students.  Why don’t I do what almost everyone else in our field does and just study how various interventions affect math and reading test scores?

Well, I’ve been making the argument for a while now that there is remarkably little evidence linking near-term changes in test scores to changes in later life outcomes for students, like graduating high school, enrolling in college, completing college, and earnings.  I have yet to see anyone bother to refute my observation of this weak and inconsistent connection between test score changes and life changes.  No matter, researchers, foundations, and policymakers continue to plod along as if changing test scores should be the focus of our efforts. Whether kids go to art museums or see live theater is at best an amusing sideline or at worst a harmful distraction from the primary goal of education, which they believe is boosting math and reading test scores.

But now we have a rigorously designed study out of Denmark that shows cultural activity among students is strongly (and likely causally) related to later academic success.  The study appears in Social Science Research, a Sociology journal that was co-founded by James Coleman.  It examines a large sample of monozygotic twins in Denmark to see if their cultural activity was related to their teacher-given GPA, exam-based GPA, and rate of completing secondary school.  To measure cultural activity they relied on a survey administered to the mothers of those twins that asked about what their children did when they were 12 years old.  It asked things like: “How often child went to any type of museum” and “How often child went to the theater or a musical performance.”

By comparing outcomes among identical twins, the researchers hope to control automatically for a large set of unobserved environmental and genetic factors.  We could reasonably believe that a large portion of the variation in cultural capital among twins was due to chance and not differences in their upbringing or ability.

The researchers found that the twin whose mothers reported having higher cultural capital at age 12 had significantly higher marks on their end of compulsory school exams at age 15/16.  They also found “an
increase in cultural capital of one standard deviation is estimated to increase the likelihood of completing upper secondary education by 12.5 percentage points.”

Cultural capital was not a significant predictor of the grade point average students received from their teacher when they were 15, which was contrary to the researchers’ expectations.  Earlier theory had suggested that cultural capital might improve academic performance by making students falsely appear more knowledgable, even if their command of the material were no greater.  As they put it: “Bourdieu argued that cultural capital, that is familiarity with the dominant cultural codes in a society, is a key determinant of educational success because it is misperceived by teachers as academic brilliance and rewarded as such.”

This study found that not to be the case.  Instead, their findings are more consistent with the arguments advanced by E.D. Hirsch and others that cultural capital gives students a stronger foundation of broad knowledge that then facilitates future knowledge acquisition.  And the significant increase in completing secondary school may be a function of that broader knowledge, as opposed to the narrow knowledge captured in math and reading standardized tests.  Cultural activity may also increase graduation rates by giving students more ways to be engaged with school on top of traditional academic coursework.

So the next time someone asks me why it matters whether students go to art museums or see live theater, I can tell them that there is at least as much rigorous evidence showing the long term benefits of cultural activity as there is for interventions designed to boost standardized test scores.


NSVF 2017

May 18, 2017

(Guest Post by Matthew Ladner)

I had the pleasure of participating in a debate/discussion on tensions in the education reform movement at the New Schools Venture Fund Summit yesterday in San Fransico. I attempted to make the following points:

  1. Big Tents are good and disagreements are okay, failure even is okay, but an unwillingness to learn from failure is a huge problem.
  2. If you hold focus groups on K-12 education you learn that the public hates current standardized testing practices and that the deeply misguided federal opt-out provision that passed the United States House of Representatives was no fluke.
  3. The failure of Question 2 in Massachusetts is screaming a warning into our deaf ear about the dangers of excluding non-urban communities from parental choice efforts. Everyone should go back and read the Rick Hess 2011 National Affairs piece “Our Achievement Gap Mania” in light of the Question 2 disaster. School choice needs to decide whether it wants to be Social Security or AFDC. For much of the history of the movement we chose an AFDC model where everyone pays in but only certain people benefit, and, well…http://www.youtube.com/watch?v=Ubw5N8iVDHI
  4. The federal government is deeply in a growing amount of debt and has $75 trillion in unfunded entitlement liabilities to contend with. The pendulum has already swung against a large federal role in K-12, and Uncle Sam’s solvency issues are likely to freeze it that way for a long while if one assumes a prioritization for programs like Social Security and Medicare. We are only five years out from half of the Baby Boomers reaching age 65.
  5. Folks should do their best to remain calm on private choice because it ultimately not a threat to public education and helps kids find a good fit school.

I have a minority viewpoint on K-12 reform, and I appreciate Stacy Childress including me in the discussion. We should have far more discussions like this, and less bomb throwing over social media.

 


Arizona Students with Disabilities thrive during choice era

May 16, 2017

(Guest Post by Matthew Ladner)

I had the chance to meet some of the folks from Raise Your Hand Texas a few years ago, and they seemed entirely delightful and respectable people. I’m afraid however that they have rushed off to form a judgement about the subject of education freedom from students with disabilities without examining the available evidence. We have statistical analyses of the impact of such programs on student outcomes for children with disabilities, surveys of parental satisfaction for participating parents, etc. but in the end this comes down to a gut check: do you believe choice for children with disabilities should be limited to those who can pay for it themselves or hire high-priced attorneys? Or do you believe that everyone can benefit from giving all children with disabilities the opportunity to seek education solutions with their share of funds?

Put me down in the latter category. If you put yourself in the former category, please feel free to explain how Arizona children with disabilities managed to show such strong gains during a period when they were being “oppressed” by not one but two different private choice programs for children with disabilities. You don’t have to trust me- go look up the numbers yourself.

As you can see, a large number of states (including Texas) either made zero progress or else saw declining scores for students with disabilities during this period. None of the states with functioning private choice programs for children with disabilities made it into the “Zero or less” club. Oddly enough several states with such programs operating during this period made it into the top 10, including Arizona, Oklahoma, South Carolina, Georgia and Indiana. Others with long-standing programs-including Florida and Utah- had more modest gains during this period but high overall scores in 2015 (NAEP science scores only go back to 2009 in the current framework).

None of this demonstrates that private choice programs drive academic improvement for students with disabilities remaining in public schools. Far more thorough studies make that case. The opposite proposition that such programs harm the academic progress of children remaining in districts- can survive neither a cursory examination of evidence nor formal statistical evaluation.

It is deeply misguided for Raise Your Hand Texas to attempt to “protect” Texas children with disabilities from more diverse schooling options, the ability to hire certified academic tutors and therapists, assistive technologies etc. Given a 12 year long effort on the part of the Texas Education Agency and 1,054 Texas school districts to undermine the intent of IDEA, I’m inclined to think that these children could use some protection from the public schooling system-as in the option to leave. As for the question of whether participating students have things to gain, you should listen to parents directly involved:


Against Federal School Choice (Even Tax-Credit Scholarships)

May 16, 2017

img_1358

(Guest post by Greg Forster)

OCPA’s Perspective has posted the second of my two articles making the case against any federal school choice program that goes beyond D.C. schools – or other legitimately federal jurisdictions (other territories, military bases, etc.). This is only my own opinion; I recognize the reasons why others, including at EdChoice, are supportive of federal choice or are at least fed-curious. But I’m here to make the case in opposition.

Having already argued against federal vouchers, through Title I or by other means…

If we want to continue living in a democratic republic and not in a technocratic oligarchy, we should be fighting tooth and nail to resist the process of federal takeover, not strengthening it…[Moreover,] it would be the states, not the federal government, which would create systems for parents to access choice through Title I portability. And not just the states, but the education bureaucracies of the states. So the bureaucrats most directly threatened by school choice would be the ones designing the programs. In other words, these programs would be designed to fail.

…in my latest article I argue against federal tax-credit scholarships:

The idea behind federalism is that governance should be kept as close as possible to local communities. That is partly because big, distant legislatures and bureaucracies are not likely to serve people well if they’re not directly connected to them. And that’s still going to be a problem even if you do find a clever way to circumvent the Constitution’s legal barriers to national education policy…

I never thought I’d live to see freedom-loving activists demanding to have the future of school choice put into the hands of the IRS. I feel like Rip Van Winkle. What did I miss here?

Federal choice of any kind also involves a sacrifice of moral legitimacy, which is destructive for any policy and fatal to a reform movement:

Lately I’ve heard a lot of talk from my conservative friends about how wrong it is when distant, powerful elites who are culturally alienated from the population at large shove laws down our throats that we regard as unjust. The question is, do we dislike that because we would rather it was our distant, powerful elites imposing our preferred laws upon populations from whom we are culturally alienated, and who view those laws as unjust? Or because elites shoving things down people’s throats is inherently wrong, whoever does it?

I also canvas the danger we run of a high-profile, national political loss should the bill fail, and other fun topics.

The school choice movement has gained enormous ground by focusing on the states. Let’s stick with what works and not sell our birthright for a D.C. mess.


Texas Implemented a special ed cap, AZ implemented an ESA for special education children. Guess what happened next.

May 15, 2017

(Guest Post by Matthew Ladner)

The Raise Your Hand Texas group has released a white paper opposing an ESA program for special needs students in the Lone Star State. It is alas replete with boiler-plate nostrums etc. but if private choice is terrible for children with disabilities attending district schools, you have an awfully hard time finding evidence for it in the NAEP. We can get NAEP trends for children with disabilities on all six NAEP exams for the 2009 to 2015 period. The Arizona legislature passed a private choice tax credit for special needs children in 2009, and followed that up with the Empowerment Scholarship Accounts program in 2011.

Texas meanwhile during this period had the Texas Education Agency implementing a defacto cap on the number of special needs students in districts, without the slightest apparent protest from Texas districts, who implemented the program quite effectively. Ah, well, at least those Texas districts should have been doing a better job delivering special ed students for the children with disabilities they served, right?

Wrong.

Arizona authorities decided to expand options and increase freedom. As you can see, Arizona students with disabilities have demonstrated academic progress much better than the nation as a whole, which has either been treading water or actually declining. This looks pretty bad until you examine the scores for Texas students with disabilities, which are not only consistently worse, but which failed to show improvement in any of the six subjects covered by NAEP.

These trends obviously have factors other than choice which impact them, but if the theory is that ESAs are terrible for children with disabilities in public schools, we can reject the hypothesis. Texas has a special education disaster on its hands, while Arizona is making progress far and away above the national average. No student group has more to gain from choice than children with disabilities- including those who choose to remain in districts.

 


That’s Not Fair!

May 10, 2017

(Guest post by Patrick J. Wolf)

We parents all have heard the claim that something wasn’t fair.  “Suzie got a bigger piece of cake than I did!”  “Tommy got to go fishing while I had to clean the garage!”  “Malachi had a lot more money spent on his education because you sent him to a traditional public school and me to a public charter school!”  Well, maybe we haven’t actually heard that last one very often but it would be a more legitimate gripe than the other ones.

Students in public charter schools receive $5,721 or 29% less in average per-pupil revenue than students in traditional public schools (TPS) in 14 major metropolitan areas across the U. S in Fiscal Year 2014.  That is the main conclusion of a study that my research team released today.    Of the cities we examined, some have large and well-established charter sectors, like Houston, Los Angeles, New York, and Washington, while others have more emerging charter school sectors like Little Rock, San Antonio, and Tulsa.

Twelve of the 14 cities have a disturbing charter school funding gap of more than 10%, which earned them a C grade or lower.  Tulsa, Little Rock, Indianapolis, Washington, Los Angeles, Oakland, and Camden earned an F for funding equity since there is a funding gap of more than 30% between what charter schools received versus what TPS received per pupil. Camden had the largest per-pupil funding gap in our study, with charter schools students receiving 45%, or $14,771, less per pupil than TPS students.

Shelby County, TN, which includes the city of Memphis, is the only metropolitan area in the study that funded students in public charter schools at a higher level than TPS.  Shelby County charter students received $10,624 in per-pupil funding in FY 2014 compared to $9,720 per student in the county’s TPS.  Houston’s charter schools were funded just 2% below their TPS, and earned the only grade of A in the study, in part because they were able to raise almost $900 per student in nonpublic revenue.  Funding gaps of $1,500 per student or more for charters remained in 10 of the 14 cities even after excluding all special education expenditures from the comparison.

The main source of the funding gap is local revenues.  Traditional public schools received $7,000 more per pupil in local revenues, on average, than did public charter schools.  Charter schools are public schools, in local communities, that must enroll all students who want to attend (or hold a random admissions lottery).  The parents of charter school students pay local taxes just like the parents of TPS students.  The fact that eight of the 14 cities in our study provided essentially no local education revenue to their public charter schools is shameful.  That’s simply not fair.

Our previous study of charter school funding equity at the state level was criticized for not exempting expenditures on such items as transportation and central administration that are mandatory for TPS but discretionary for public charter schools.  In our view, that’s exactly the point.  Charter schools are permitted to be innovative as an alternative to the more rigidly controlled administrative and spending structure of TPS.  True, the revenue amounts received by charters and TPS are more even once you exclude all of the ways that public schools are forced to be inefficient.  Like I said.  That’s exactly the point.

What are the takeaways for education policy?  Our results support the recommendations of the Fordham Institute and others to fund students directly, using a weighted student funding formula, a.k.a. “backpack” funding.  Placing public charter schools on a par with TPS in receiving local educational funds, as Colorado plans to do, would bring over half the cities in our study to funding parity across the two public school sectors.

States like Massachusetts, Texas, and Denver have tried to compensate for local funding discrepancies in their charter sectors by providing higher state funding to charter students, but that move hasn’t closed the funding gap.  It merely got Houston close enough so that the extra-ordinary fundraising efforts of its charter schools were able to move charter students close to parity.  Such bricolage arrangements are simple guesswork and no substitute for a rational student-based funding policy that treats the same student similarly regardless of the local public school their parent chooses for them.  Ask your children.  Anything else is just not fair.


Do Parents Care About Test Scores?

May 8, 2017

yhvkb2a1iyrt6oh4hoys

(Guest Post by Jason Bedrick)

To test, or not to test? That is the question at the center of one of Matt Barnum’s latest and last articles at The 74 (he’s moving over to Chalkbeat). Taking opposite sides of the “testing and educational choice” question are Doug Harris and yours truly.

To test:

“If you’re not going to require anything, we’re not going to know anything about whether these programs worked or not,” said Doug Harris, a Tulane University professor who has done extensive research on school choice in New Orleans and has been a critic of DeVos’s approach to education reform. “When you put it that way, even the people who are somewhat supportive of the idea get a little squeamish.”

Not to (impose the state standardized) test:

“Since there is strong evidence that state testing mandates tend to have negative unintended consequences, such as narrowing the curriculum and distorting how schools teach kids, we’d rather that schools have the freedom to use the tests that are more closely aligned to their curriculum and give parents the freedom to choose the schools that work best for their kids,” said Jason Bedrick, of EdChoice, an Indianapolis-based group that backs voucher and tax credit programs. [links added]

… Bedrick, of EdChoice, says he’s fine with requiring schools to administer a national test, but not the state exam, even if it means that studying the program then becomes difficult.

“As researchers, we’d like the apples-to-apples comparisons that a single test provides,” he wrote in an email. “However, as policymakers, we also have to do what’s in the best interests of kids.”

“The main point of choice policies is to empower parents to choose schools based on the criteria that are important to them, not just to raise [specific] test scores,” Bedrick said, citing factors such as student-teacher ratio, curriculum, and college acceptances.

Later in the piece, Barnum cites me saying, “We don’t think it is necessary to impose standardized tests. If parents demand that information, schools will provide it.” Harris disagrees:

“I think that’s hooey … Schools themselves, even if they wanted to provide that kind of information, can’t provide that in a way that’s comparable across schools — you have to have a coordinating mechanism for information.”

If only the market had a coordinating mechanism for information that parents used! Oh right, not only do such mechanisms already exist, but they’re also growing in popularity, and they get far more web traffic than any state education agency website:

GreatSchools.org is not the only website that ranks pre-K-12 schools. The Internet search engines Yelp and Google offer school ratings, as do websites such as Schooldigger.com and Privateschoolreview.com. But with 40 million annual unique visitors, GreatSchools is the one most used, according to Alexa Internet, which tracks Web traffic.

The site’s founder and chief executive officer, Bill Jackson, says GreatSchools wants to be more than just a school ratings site: He sees it developing into an association that serves parents in the same way that the AARP serves retirees, or that AAA represents drivers.

Perfect information? Of course not. Lots of room for improvement? Absolutely. But they already do more than any state agency to empower parents with the info they want.

And “want” is the key word there. EdChoice’s More Than Scores survey of parents of tax-scholarship students in Georgia found that most parents do *not* place a very high value on test scores:

Student performance on standardized test scores is one of the least important pieces of information upon which parents base their decision regarding the private school to which they send their children. Only 10.2 percent of the parents who completed the survey listed higher standardized test scores as one of their top five reasons why they chose a particular private school for their child.

Just ask real parents, such as, well… Doug Harris:

Research, including Harris’s, suggests that parents do generally place significant value on a school’s test scores, but it may not be the biggest factor driving school preferences.

As a parent himself considering private schools, Harris says he’s never asked any of them for their test scores. [emphasis added]

“Most of these decisions, and this is true public, private, or otherwise, are based more on reputation, on what your neighbors say,” he said.

Tests can be informative for parents, but policymakers should dial back their obsession. After all, they measure only a thin slice of what we want schools to provide. Parents take a more holistic view of their child’s education, and the narrow focus on tested subject can have negative unintended consequences. For example, one researcher recently acknowledged that one reason there “hasn’t been as much actual innovation [in the charter school sector] as maybe the original charter folks hoped” is that test-based accountability is hampering innovation:

[W]hen you have intense test-based accountability it really restricts what you can do and to what degree you can innovate because you have to put so many of your resources towards the same end. There are only so many ways to make test scores go up. So, I think that really restricts what they can do.”

That researcher? Doug Harris.