If Every Instinct You Have is Wrong…

April 9, 2019

(Guest Post by Matthew Ladner)

In the Seinfeld episode “The Opposite” Jerry discovers that he is “Even Steven” in that setbacks are quickly followed by gains and his life basically remains the same. Meanwhile, George slowly morphs from a loser living in his parents basement to getting hired by the New York Yankees after giving George Steinbrenner a dressing down about the poor management of the franchise. Meanwhile Elaine ruins a merger of her company and finds herself unemployed. “I’ve become George,” she glumly observes. I often think of this episode when reading odd summaries of Arizona’s K-12 which are opposite of reality. Arizona Congressman Ruben Gallego for instance recently wrote a letter to Secretary of Education Betsy DeVos making a variety of claims about education in Arizona. The letter however makes a variety of claims about which are demonstrably mistaken. I’d like to address the following paragraph in particular:

Arizona public schools have improved performance over time rather than seeing performance decline. Student performance, as measured by the National Assessment of Educational Progress (NAEP) has improved both in the aggregate and across a variety of subgroups as shown in the figure below, which shows NAEP data from the first NAEP exam that includes all states (2003) to the most recent data available (2017) in 8th grade reading and math.

NAEP gains have improved Arizona’s subgroup rankings in overall proficiency by subgroup. For instance the chart below shows where Arizona Black students ranked compared to Black students in other states in 2003 on the left, 2017 on the right on 8th grade math. Moreover, as shown in the figure above, the largest Arizona gains were made by Black students in math (16 points), Hispanic students in reading (14 points) and American Indian students in Reading (16 points). Each of these student groups displayed a command of math and reading at 8th graders in 2017 that we might have reasonable expected their 2003 peers to have shown as 10th graders.

 

Here is the breakdown for Hispanic students:

Here is the breakdown for Anglo students:

If these levels of gains and proficiency represented the “fifth worst” school system,  the rest of the world would be looking at America’s international test scores with envy.  It seems profoundly unlikely that Arizona students would be making these enviable academic gains if choice harmed their education. Most Arizona students in Maricopa County attend a school other than their assigned district school. Students attending other district options however outnumber charter students by nearly two to one. Statewide charter student vastly outnumber private choice students in Arizona. Individual district schools both lose and gain students through family decisions. Choice is being done primarily by district schools rather than to district schools. Schools which fail to gain the confidence of families as the best option for their child do lose enrollment, but the positive academic trends show that Arizona schools are rising to the challenge rather than wilting under pressure.

Opponents of choice often conflate it with spending, which is misleading. A great many factors influence public school spending- the wealth of a state, the relative priority placed on K-12 compared to contending priorities like health care and higher education, local and state elections on funding and age demographics-states with lots of elderly and young people. One of the factors influencing per pupil funding trends is enrollment growth. Fast growing states have a harder time in accommodating growth and spending more per pupil at the same time. Arizona had the largest increase in per pupil spending among states with a 20% or more growth in enrollment between 2000 and 2015.

I will however agree with Representative Gallego to this extent: Arizona’s academic improvement is not a coincidence. They represent a huge amount of hard work put in by Arizona students and teachers and some very underestimated policies.  This is not to say there isn’t more progress needed (there is) but these gains don’t have “man-hands” or eat their peas one at time and shouldn’t be taken for granted.


William N. Sheats for the Higgy

April 8, 2019

William N. Sheats, Florida Superintendent of Public Instruction

(Guest Post by Patrick Gibbons)

William N. Sheats was Florida’s very first elected superintendent of public instruction, serving from 1893 to 1904 and again from 1913 until his death in 1922.

As the state’s leading educator, Sheats worked tirelessly to modernize Florida’s education system. He drafted the first statewide curriculum, reformed teacher training and required teachers to pass exams to prove subject-area mastery. He passed the state’s compulsory-attendance law in 1919 and made Florida’s public school system one of the best funded among southern states.

Contemporaries called him the “father of Florida’s public school system.”

As a chair of the education committee during Constitutional Convention of 1885, Sheats nearly caused panic among Democrats when he proposed allowing taxes to support the creation of common schools for black students. Thirty-two members of his own party voted against him. He invited Booker T. Washington to speak before white educators and helped secure public funds for the American Missionary Association’s Fesseden Academy, a college and career preparatory school in rural Florida. Sheats even lost his re-election in 1903 after Florida Education Association Vice President Clementine Hampton, along with the editor of the Gainesville Sun, smeared him as a “Friend of the Negro.”

One can argue that black students may have been worse-off without Sheats as the state superintendent, but the Higgy is not about recognizing the worst or most evil human being. For all the good he may have tried to accomplish, Sheats’ paternalistic racism left lasting scars.

Education, Sheats believed, would “make the vast number of idle, absolutely worthless negroes industrious and self-supporting.”

Sheats enshrined segregation into Florida’s Constitution of 1885, personally writing Article XII, Section 12 which states:

“White and colored children shall not be taught in the same school, but impartial provision shall be made for both.”

It would remain in Florida’s constitution until 1968.

Sheats fervently believed in racial segregation. As he saw it, “any effort to enforce mixed education of the races would forever destroy the public school system in one swoop.”

Fearing teachers might sway impressionable young minds, Sheats used the power of his office to outlaw hiring teachers trained at racially integrated northern colleges. He also outlawed white teachers from educating black students in public schools. The combination of strict certification requirements, a ban on white teachers educating black students, and a rule preventing teachers trained at integrated colleges meant many public schools for black students with a shortage of teachers. By 1924, two years after his death, the rules he left in place meant only 1 out of every 4 black teachers were state certified.

When Sheats learned of Orange Park Normal & Industrial School, a racially integrated private school operating outside of Jacksonville, he lashed out, calling the school a “social and moral blotch,” and a “vile encroachment upon our social and moral system.” With the constitution mandating racial segregation only in public schools, Sheats lobbied the legislature to pass a bill outlawing whites from educating black and white students within the same building.

The New York Times reported in 1896 that the law “provided that it should be a penal offense” for any person or organization to run a school, public, private or parochial, “wherein white persons and negroes should be instructed or boarded within the same building or taught in the same class, or at the same time, by the same teacher.” Those found guilty could be fined between $150 and $500 or imprisoned for three to six months.

With the law passed, Sheats ordered the arrest of the school principal, three patrons, five teachers and even the local minister. In October 1896, a judge in the Fourth Judicial Circuit Court of Florida tossed out the law on a technicality and Sheats went to work trying to pass a new law. He wouldn’t succeed until after returning to office in 1913. Known as “Sheats Law” the state of Florida now banned “white persons from teaching Negroes in Negro schools.” The law also prohibited black teachers from educating white students.

Facing declining enrollment, attacks from the KKK, wariness over legal expenses and threats of arrest, Orange Park Normal & Industrial School closed its doors for good. After two decades, Sheats had finally succeeded in shuttering Florida’s first and only racially integrated school. A Catholic school under the leadership of 34-year old Bishop Michael Joseph Curley stepped up to fight the law. When Sheats asked Bishop Curley to remove white teachers from the school he refused and vowed to fight the law all the way to the Supreme Court.

Arrests wouldn’t happen until April 1916 when Gov. Park Trammell, at the instance of Sheats, ordered the arrest of three Catholic sisters who taught at St. Benedict the Moore School in St. Augustine, Fla. They were charged with “unlawfully teaching negroes in a negro school.” The arrest of the three sisters attracted national attention, but it also forced black private schools throughout the state to temporarily cease operations or risk arrest.

Fortunately, the case was resolved quickly. On May 20, 1916, Judge George Cooper Gibs ruled the “Sheats Law” unconstitutional, declaring,

“Has a white teacher any the less right to sell his services to negro pupils than a white doctor to negro patients, or a white lawyer to negro clients, or a white merchant as a right to sell his goods to negro customers, and vice versa?”

Sheats enshrined segregation into Florida’s constitution and fathered a system of uniform public schools that were ultimately a system of separate and unequal schools. From 1885 until well into the 1960s, Florida’s public-school system required separate attendance zones for white and black students, even if they lived in the same neighborhood. Segregation became “so entrenched that school superintendents were required to keep separately the books used in white and Negro schools.” Even the tax dollars used for white and black schools could not comingle.

Although he dared to fund education for black students when many contemporaries of his time would not, he ultimately created and enforced a regime of racial segregation and inequality that lasted 83 years. For that, he deserves the dishonor of the Higgy.


The Achievement-Attainment Disconnect Strikes Again!

April 2, 2019

I’ve lost track of how many studies with rigorous causal identification find that improving test scores is not associated with improving later life outcomes, like graduating high school, attending and completing college, and getting a job and earning higher salaries.  In the area of school choice alone we now have dozens of such studies, but you can also find this achievement-attainment disconnect in rigorous studies of pre-school and other interventions.

You can add to this list the latest study from Mathematica on the long-term effects of attending a charter middle school. Mathematica examined lotteries at a sample of charter middle schools around the nation, comparing outcomes for lottery winners to those of lottery losers.  When the original results were completed almost a decade a ago, they found no overall effect on test scores.  But when they disaggregated results, they found that urban charters had a significantly positive effect while suburban charters had the opposite result.  This prompted leaders in the charter and ed reform worlds to conclude that we should focus our efforts on urban charters, since those were the ones that “worked.”

Now Mathematica has followed-up on those students to see whether they eventually attended and completed college.  The new results show that attending a charter middle school has no effect on attending or completing college, just as it had no overall effect on test scores.  But — and this is the important part for our discussion — they also concluded:

The success of an individual charter middle school in improving college outcomes was not related to its success in improving middle school achievement. The study schools that improved middle school achievement were not consistently more successful than others in boosting college enrollment and completion.

So, all of those technocratically-minded ed reformers who thought we should focus on urban charters because test scores showed they “worked” were guilty of mis-judging long-term success based on unreliable short-term measures.  The evidence shows that changing test scores is not a particularly good indicator of schools that will improve their students’ lives.

City Fund, NACSA, and others who support portfolio management, harbor-mastering, quarterbacking, or whatever marketing term they are using nowadays are once again left trying to explain exactly how they intend to distinguish the “good” charter schools from the “bad” ones better than parents can.  But take comfort, their political ineptitude matches their technocratic inclinations so that I expect they will burn through their $200 million without successfully installing and maintaining any multi-sector portfolio management systems. So children will be safe from their falsely-guided superior judgement.


It’s Time for “The Higgy”

March 31, 2019

William Higginbotham

It is time once gain to solicit nominations for the William Higinbotham Inhumanitarian Award.  Below I reproduce portions of the first announcement of “The Higgy” in 2012, so you have an understanding of the historic significance and criteria for this dishonor.

——————————————————————–

As someone who was recognized in 2006 as Time Magazine’s Man of the Year, I know a lot about the importance of awards highlighting people of significant accomplishment.  Here on JPGB we have the Al Copeland Humanitarian Award, but I’ve noticed that “The Al” only recognizes people of positive accomplishment.  As Time Magazine has understood in naming Adolf Hitler, Joseph Stalin, and Ayatullah Khomeini as Persons of the Year, accomplishments can be negative as well as positive.

(Then again, Time has also recognized some amazing individuals as Person of the Year, including Endangered Earth, The Computer, Twenty-Five and Under, and The Peacemakers, so I’m not sure we should be paying so much attention to what a soon-to-be-defunct magazine does.  But that’s a topic for another day when we want to talk about how schools are more likely to be named after manatees than George Washington.)

Where were we?  Oh yes.  It is important to recognize negative as well as positive accomplishment.  So I introduce “The Higgy,” an award named after William Higinbotham, as the mirror award to our well-established “Al.”

Just as Al Copeland was not without serious flaws as a person, William Higinbotham was not without his virtues.  Higinbotham did, after all  develop the first video game.  But Higinbotham dismissed the importance of that accomplishment and instead chose to be an arrogant jerk by claiming that his true accomplishment was in helping found the Federation of American Scientists and working for the nonproliferation of nuclear weapons.  I highly doubt that the Federation or Higinbotham did a single thing that actually advanced nonproliferation, but they sure were smug about it…

I suspect that Al Copeland, by contrast, understood that he was a royal jerk.  And he also understood that developing a chain of spicy chicken restaurants really does improve the human condition.  Higinbotham’s failing was in mistaking self-righteous proclamations for actually making people’s lives better in a way that video games really do improve the human condition.

So, “The Higgy” will not identify the worst person in the world, just as “The Al” does not recognize the best.  Instead, “The Higgy” will highlight individuals whose arrogant delusions of shaping the world to meet their own will outweigh the positive qualities they possess.

We will invite nominations for “The Higgy” in late March and will announce the winner, appropriately enough, on April 15.  Thanks to Greg for his suggestions in developing “The Higgy.”


Setting the Record Straight on Choice

March 28, 2019

469EADCA-45D4-4475-8720-5953C9D0B5C2

(Guest post by Greg Forster)

OCPA carries my article fisking an especially sloppy smear job attacking school choice:

I call it a strange document because it’s trying to present itself as some sort of scholarly press release. It’s published by something calling itself Scholars Strategy Network, and the byline is from two academics, with their academic affiliations and emails listed at the top. But it’s not a work of scholarship, nor is it informed by scholarship. It’s two pages of emotive bullet points, unsubstantiated bumper-sticker assertions, shoddy reasoning, and deceptive characterizations of the empirical research. An impressively long list of “sources,” formatted to look like scholarly citations, is supplied at the end in the desperate hope of simulating gravitas.

Come for the blatant dishonesty about easily checkable facts:

Assaulting private school choice, the authors appear to be afraid to make their own assertions, but quote someone else’s claim that no “independent studies” have ever found that students using private school choice in Milwaukee, Cleveland, or Washington, D.C. performed better than children who remained in public schools. But the official study in Washington, D.C. looked at exactly this question, using random-assignment methods (the gold standard), and found huge increases in high school graduation and college attendance rates. No doubt the report being cited here doesn’t count this study as “independent,” because it was a federal program and the study was federally funded as part of the program. That’s blatantly dishonest cherry-picking. And what is their excuse for leaving out the two—two!—gold-standard studies of this question finding academic improvements in Milwaukee?

Not to mention that it’s cherry-picking to include only selected cities. Across all private-choice programs, there have been a total of 18 gold-standard studies (no cherry-picking). Of these, 14 found academic improvements, two found no visible effect, and two (both examining a poorly designed program in Louisiana) found negative results.

Stay for the outrageous ideological claptrap!

The authors complain that schools in choice programs are not “transparent.” But parents have the power to demand whatever information they think important, or not attend the school. This is why private schools are already more transparent, by orders of magnitude, than organizations are typically required to be when participating in other kinds of government programs. Look at the reams of hard data on named, particular private schools on GreatSchools.org or in the U.S. Department of Education’s Common Core of Data. Then try getting comparable data on apartment buildings that take Section 8 housing vouchers, or grocery stores that take food stamps.

Send me a scholarly press release letting me know what you think!


New Arts Studies Lost in Busy Week

March 15, 2019

Image result for lost in the shuffle

It’s been a busy week with the publication of an op-ed by me and Rick Hess in the Wall Street Journal and a study in Education Next documenting the monolithic partisan composition of education reform advocates and those who conduct research on those efforts. One might think that folks committed to evidence-based decision-making would be very interested in facts about their field, but their social media response has generally been counter-productive and fact-free.  Those responses have focused on how they are not to blame, how Republicans are icky anyway, and how many of their best friends are Republicans.  I’m not bothering to link to those responses because there really is no point.  If folks are happy with a uniformly Democratic movement, then they are welcome to keep it… as long as someone continues to be willing to pay for this party.  Given the groupthink and political ineffectiveness that is likely to result from this lack of heterodoxy, I can only wonder why and for how long funders will subsidize it.

Lost in the shuffle of this busy week, some graduate students and I released two new studies of the medium-term effects of students receiving multiple arts-focused field trips to the Woodruff Arts Center in Atlanta.  We randomly assigned school groups to a treatment that involved three field trips per year to visit an art museum, see live theater, and listen to the symphony, or to a control condition.  Among the treated students, some received 3 experiences over 1 year and some received 6 experiences over 2 years.

We split the analyses into two separate reports.  The first, led by Heidi Holmes Erickson, found that these arts-focused field trips improved school engagement, as measured by disciplinary infractions and survey responses, as well as increased standardized test scores in math and reading. These benefits persisted even one year after treatment ended for the first cohort in the study.

The second study, led by Angela Watson, examined social-emotional outcomes.  It found that exposure to multiple arts-focused field trips increased social perspective taking and tolerance.  It also found evidence of an improvement among treated female students in their conscientiousness, as measured by survey effort.

Heidi and Angela will be presenting these results at the Association for Education Finance and Policy conference next week.  Please attend their sessions to learn more about this research and to provide suggestions for improving their papers.  And if folks at AEFP are also interested in engaging in a productive discussion of how to improve the intellectual and ideological diversity of the organization, that would also be wonderful.


Picture Yourself in a State by the Ocean with Really Low Scores and Nothing to Lose

March 4, 2019

(Guest Post by Matthew Ladner)

Believe it or not 20 years have passed since Jeb Bush kicked off the Florida K-12 reforms during the 1999 legislative session. In a kickoff post at Redefined for a series of articles I lay out the reasoning behind the Florida Supreme Court’s decisive rejection of a recently concluded challenge to these reforms.

A highly coveted Jayblog “No-Prize” goes to whomever leaves the best Beatles lyric pun in the comment section that hasn’t already been used on social media.


Too Many Social Scientists, Too Few Truths to Discover

February 17, 2019

Image result for too many fish

I’m teaching a seminar for the Honors College this semester on BS. It’s been a lot of fun and the students have been great.  Last week we were discussing the prevalence of BS in social science.  In particular, we were discussing the problems of file drawer and publication bias, p-hacking, and spurious relationships.  While considering why there is so much of this BS in social science we stumbled upon a possible explanation: perhaps there are just too many social scientists under too much pressure to regularly discover and report truths about human behavior when there just aren’t enough truths to be discovered.

Roughly estimating, there are at least 2,000 institutions worldwide that give priority to research and expect their faculty to produce it regularly.  And there are at least 50 active researchers in the social sciences at each of those institutions who depend on publishing novel insights about human beings, sometimes annually, in order to obtain and keep their jobs as well as receive promotions. In my back of the envelope calculation, there is demand for “discovering” roughly 100,000 true things about human behavior each year.

Now let’s consider the supply side.  My general worldview is that there is a very limited number of universally true things we could say about human behavior.  I’d wager that there are no more than several dozen true things that generally apply to human beings across time, place, and context.  And perhaps there are several hundred more contingently true things, observations that would be true for specified groups of people in particular circumstances.  The number of universally or contingently true observations we could make about human behavior may not exceed a thousand.

I understand that this description of the supply side is merely an assertion with which many may disagree.  But if you accept that much about human behavior is truly random or the function of idiosyncratic factors that make them impossible to predict, then you’d have to accept that the number of true observations about human behavior is quite limited. Even if it is more than a thousand, it is almost certainly well short of the number being demanded by social science researchers.

This shortage of truths to discover about human beings is especially severe when you consider that many of the possible true things have already been discovered.  Social science may only be about a century old, but the search for true observations about human beings goes as far back as we have written records.  Poets, philosophers, artists, and historians have been casting their nets for generalizations about human beings for millennia, so it is questionable how many fish are left in the sea.

I don’t want to be understood as making the equivalent of the claim that all discoveries have been made so let’s close the patent office.  I’m confident that there are new and interesting observations to be made about human behavior.  And I’m even more confident that we can do much to confirm or to dis-confirm previously made observations.  I just can’t escape the impression that the number of these true things to be said is dwarfed by the number of people with professional pressures to uncover them.

If I’m right, several implications follow.  The prevalence of BS in social science, including replication problems, file-drawer and publication bias, p-hacking, etc…, cannot be addressed with improved training or enforcement of more rigorous standards.  The pressure to make claims that are not really true is simply too great to be controlled by ethics or peers facing the same pressure.

Even if we could remove the external incentives for generating dubious findings by altering standards for promotion and tenure, we are left with the problem that searching for these findings is built into the self-definition of social scientists.  Social scientists think of themselves as explorers and they will continue to sail the world’s oceans shouting “Land!” at every mirage on the horizon even if much of the Earth has already been mapped.

I suspect that a main solution is to reduce the number of people engaged in social science research.  I don’t see this as particularly likely to happen given our desire to reproduce ourselves.  But pressure to reduce the number of social science researchers might come from those who foot the bill for these adventures, including taxpayers, tuition-paying students, and benefactors.

It might also be helpful if the social sciences begin to change how they think of what they do to be more like the humanities.  Scholars in the humanities are not so much focused on making new discoveries as they are on documenting and disseminating the insights of the past.  This makes them more interested in teaching and longer-form scholarship, like books.  Of course, given the celebrity-worship status hierarchy in the social sciences, how will we know who the coolest kids are if they do things that are hard to rank and compare, like teaching and only periodically writing books?

I don’t hold out much hope for the social sciences beginning to thin their ranks of researchers or shifting to a humanities orientation.  But thinking about the mismatch between how many people are searching for generalizations about human behavior and how many valid generalizations they are able to find is still useful for diagnosing how the social sciences may have gone astray.

 


Class Sizes, Again

February 6, 2019

classroom-2787754_640

(Guest post by Greg Forster)

OCPA carries my article on class sizes:

California enacted a big class-size reduction policy in 1996. It sounded easy when it was pitched to voters, but it ended up costing the state billions of dollars. And it produced no measurable improvement in any education outcomes—not test scores, not graduation rates, nothing.

Alas, the lesson was not learned. Florida enacted an even more ambitious class-size reduction policy in 2002. It cost the state $20 billion to implement as it was scaled up over eight years, and costs between $4 billion and $5 billion to maintain every year. And it produced no positive effect on education outcomes.

Let me know what you think!


Only Retributive Justice Is Restorative: Plummeting Outcomes Edition

February 6, 2019

chuttersnap-193778-unsplash

(Guest post by Greg Forster)

Rand reports that a big randomized trial of super-lax discipline policies in Pittsburgh schools had negative effects on student achievement and big negative effects on African-American student achievement. Of course, Rand buries the negative finding under mountains of politically palatable pablum, but if you dig (search for “not all PERC impacts were positive”) you can find it just barely acknowledged. Or, if you grok geek, check out Table 6.7 on p. 56.

Via this thread, which reviews other studies with similar findings in LA and Philly. Backfill from Max Eden at (sigh) Fordham.

As I’ve written before on JPGB, the most destructive aspect here is that advocates have chosen to label the loosey-goosey discipline policy “restorative justice.” They accept a false and outrageous distinction between “restorative” and “retributive” justice, and cling to the absolutely unsupported prejudice that lax policies will be more restorative, while it is the desire for retributive justice that produces harsher policies. The reality is that in most cases, justice must be both retributive in intention and strict in application to be truly restorative. There is a place for mercy, but not too much and not too often. Nothing ruins young people more effectively, or prevents their “restoration” more completely, than lax discipline.

This is destructive because the lesson most people will take from their failures is that justice should not be restorative if we want it to be either retributive or effective:

The increasing tendency of some to dehumanize criminals and demand harsher and harsher treatment of them cannot be fought by advocacy of lax punishments in the name of “restorative justice.” It is directly caused by advocacy of lax punishments in the name of “restorative justice.”

Only retributive justice, which affirms that punishment is not an arbitrary tool of social control but a just and necessary consequence of the crime that the criminal is morally obligated to suffer, can be effective in restraining the abuse of criminals – and promoting their genuine restoration.

As C.S. Lewis once said, I plead for retributive justice not primarily for the sake of society, or for the sake of crime victims, but for the sake of the criminal.