Too Many Social Scientists, Too Few Truths to Discover

February 17, 2019

Image result for too many fish

I’m teaching a seminar for the Honors College this semester on BS. It’s been a lot of fun and the students have been great.  Last week we were discussing the prevalence of BS in social science.  In particular, we were discussing the problems of file drawer and publication bias, p-hacking, and spurious relationships.  While considering why there is so much of this BS in social science we stumbled upon a possible explanation: perhaps there are just too many social scientists under too much pressure to regularly discover and report truths about human behavior when there just aren’t enough truths to be discovered.

Roughly estimating, there are at least 2,000 institutions worldwide that give priority to research and expect their faculty to produce it regularly.  And there are at least 50 active researchers in the social sciences at each of those institutions who depend on publishing novel insights about human beings, sometimes annually, in order to obtain and keep their jobs as well as receive promotions. In my back of the envelope calculation, there is demand for “discovering” roughly 100,000 true things about human behavior each year.

Now let’s consider the supply side.  My general worldview is that there is a very limited number of universally true things we could say about human behavior.  I’d wager that there are no more than several dozen true things that generally apply to human beings across time, place, and context.  And perhaps there are several hundred more contingently true things, observations that would be true for specified groups of people in particular circumstances.  The number of universally or contingently true observations we could make about human behavior may not exceed a thousand.

I understand that this description of the supply side is merely an assertion with which many may disagree.  But if you accept that much about human behavior is truly random or the function of idiosyncratic factors that make them impossible to predict, then you’d have to accept that the number of true observations about human behavior is quite limited. Even if it is more than a thousand, it is almost certainly well short of the number being demanded by social science researchers.

This shortage of truths to discover about human beings is especially severe when you consider that many of the possible true things have already been discovered.  Social science may only be about a century old, but the search for true observations about human beings goes as far back as we have written records.  Poets, philosophers, artists, and historians have been casting their nets for generalizations about human beings for millennia, so it is questionable how many fish are left in the sea.

I don’t want to be understood as making the equivalent of the claim that all discoveries have been made so let’s close the patent office.  I’m confident that there are new and interesting observations to be made about human behavior.  And I’m even more confident that we can do much to confirm or to dis-confirm previously made observations.  I just can’t escape the impression that the number of these true things to be said is dwarfed by the number of people with professional pressures to uncover them.

If I’m right, several implications follow.  The prevalence of BS in social science, including replication problems, file-drawer and publication bias, p-hacking, etc…, cannot be addressed with improved training or enforcement of more rigorous standards.  The pressure to make claims that are not really true is simply too great to be controlled by ethics or peers facing the same pressure.

Even if we could remove the external incentives for generating dubious findings by altering standards for promotion and tenure, we are left with the problem that searching for these findings is built into the self-definition of social scientists.  Social scientists think of themselves as explorers and they will continue to sail the world’s oceans shouting “Land!” at every mirage on the horizon even if much of the Earth has already been mapped.

I suspect that a main solution is to reduce the number of people engaged in social science research.  I don’t see this as particularly likely to happen given our desire to reproduce ourselves.  But pressure to reduce the number of social science researchers might come from those who foot the bill for these adventures, including taxpayers, tuition-paying students, and benefactors.

It might also be helpful if the social sciences begin to change how they think of what they do to be more like the humanities.  Scholars in the humanities are not so much focused on making new discoveries as they are on documenting and disseminating the insights of the past.  This makes them more interested in teaching and longer-form scholarship, like books.  Of course, given the celebrity-worship status hierarchy in the social sciences, how will we know who the coolest kids are if they do things that are hard to rank and compare, like teaching and only periodically writing books?

I don’t hold out much hope for the social sciences beginning to thin their ranks of researchers or shifting to a humanities orientation.  But thinking about the mismatch between how many people are searching for generalizations about human behavior and how many valid generalizations they are able to find is still useful for diagnosing how the social sciences may have gone astray.

 

Advertisements

Teaming with Goldwater’s New Improved Matt to Tackle the Subject of AZ District Space Glut

February 12, 2019

(Guest Post by Matthew Ladner)

The Arizona Chamber of Commerce Foundation and I teamed up with the Goldwater Institute to create a white paper on vacant district space. Arizona has one of the fastest growing student populations but oddly finds itself with a large glut of underutilized district space.

How large is it?

Ah well no one really knows because of severe flaws in reporting but the statewide floor starts at 1.4 million sq feet but the Arizona Auditor General found more than that in a single district by poking around a bit so the ceiling is much much higher. In any case Arizona’s district space increased by 2004 and 2017 by 22.6 million square feet—a 19 percent increase—despite a student enrollment increase of only 6 percent during this same period. Arizona not only has a glut of underutilized district space it appears to be growing.

Research from MIT of co-location of charters within district space demonstrates both financial and academic benefits to districts-specifically in increasing district resources and classroom spending in districts. Arizona has tens of thousands of students stranded on waitlists at high demand district and charter schools, millions of square feet in underutilized district space, and a need to increase resources for classroom use. Mutually beneficial arraingements are there for the taking between high demand schools with waitlists and districts with underutilized space. The scale of these gains are of a scale that Goldwater’s Matt Beienburg and I swallow our pride to point to legislation in California and New York (someone just yelled “get a rope and find a big cactus!) to serve as possible models.

Anyhoo- check it out here. It’s fun to be back writing with GI again.

 


Class Sizes, Again

February 6, 2019

classroom-2787754_640

(Guest post by Greg Forster)

OCPA carries my article on class sizes:

California enacted a big class-size reduction policy in 1996. It sounded easy when it was pitched to voters, but it ended up costing the state billions of dollars. And it produced no measurable improvement in any education outcomes—not test scores, not graduation rates, nothing.

Alas, the lesson was not learned. Florida enacted an even more ambitious class-size reduction policy in 2002. It cost the state $20 billion to implement as it was scaled up over eight years, and costs between $4 billion and $5 billion to maintain every year. And it produced no positive effect on education outcomes.

Let me know what you think!


Only Retributive Justice Is Restorative: Plummeting Outcomes Edition

February 6, 2019

chuttersnap-193778-unsplash

(Guest post by Greg Forster)

Rand reports that a big randomized trial of super-lax discipline policies in Pittsburgh schools had negative effects on student achievement and big negative effects on African-American student achievement. Of course, Rand buries the negative finding under mountains of politically palatable pablum, but if you dig (search for “not all PERC impacts were positive”) you can find it just barely acknowledged. Or, if you grok geek, check out Table 6.7 on p. 56.

Via this thread, which reviews other studies with similar findings in LA and Philly. Backfill from Max Eden at (sigh) Fordham.

As I’ve written before on JPGB, the most destructive aspect here is that advocates have chosen to label the loosey-goosey discipline policy “restorative justice.” They accept a false and outrageous distinction between “restorative” and “retributive” justice, and cling to the absolutely unsupported prejudice that lax policies will be more restorative, while it is the desire for retributive justice that produces harsher policies. The reality is that in most cases, justice must be both retributive in intention and strict in application to be truly restorative. There is a place for mercy, but not too much and not too often. Nothing ruins young people more effectively, or prevents their “restoration” more completely, than lax discipline.

This is destructive because the lesson most people will take from their failures is that justice should not be restorative if we want it to be either retributive or effective:

The increasing tendency of some to dehumanize criminals and demand harsher and harsher treatment of them cannot be fought by advocacy of lax punishments in the name of “restorative justice.” It is directly caused by advocacy of lax punishments in the name of “restorative justice.”

Only retributive justice, which affirms that punishment is not an arbitrary tool of social control but a just and necessary consequence of the crime that the criminal is morally obligated to suffer, can be effective in restraining the abuse of criminals – and promoting their genuine restoration.

As C.S. Lewis once said, I plead for retributive justice not primarily for the sake of society, or for the sake of crime victims, but for the sake of the criminal.


Creating and Managing Schools = Feature Not a Bug

February 6, 2019

(Guest Post by Matthew Ladner)

Over at the Chamber Business News I pose a few questions to charter opponents who want to force Arizona charter management organizations to competively bid out the management of new schools. Questions for the hive mind: can anyone think of a reason this should be done to charters but not districts? I mean you know given stuff like this:

…wouldn’t most reasonable people take a look at that dot on the top right, learn that it has a majority-minority student population, gets only a modest level of funding, and take in kids from districts in their state with below average test scores and conclude “hmmm these schools seem to be pretty well managed?” Given Arizona’s growing school population, am I nuts to think such a provision would stop new school construction dead in its tracks if applied to either districts or charters? I’m trying to imagine either CMOs or districts raising millions to build a new school only to RFP off the management of that new school, and I’m coming up empty trying to imagine a case to build the new school. Maybe you can help me out in the comments.


“Get Me Roger Stone”

January 26, 2019

(Guest Post by Matthew Ladner)

I do as much as I can to avoid paying any attention to Washington DC. The House passes bill with full knowledge that the Senate won’t do anything other than greet themselves in the mirror each morning as “Good morning Mr./Madame President.” It’s a unique combination of pointlessness and vanity imo. Anyway I had never heard of Roger Stone until CNN was there with a news camera to arrest him on something to do with Russiagate. I heard something about him having a tattoo of Richard Nixon’s face on his back. A friend strongly recommended that I watch “Get Me Roger Stone” on Netflix, which I did this morning.

I’m still trying to process it.

If you mix Forrest Gump’s proclivity for being around big events with a Denny Crane scale ego and a large dash Idiocracy politics, that maybe kind of sorta is a starting place, but only a start. I’ve never seen anything remotely like it. It is both profoundly disturbing and occasionally hilarious, and it leaves people like me politically homeless living in a VAN down by the RIVER.

Go watch it. Like now! And then see if you can console me in the comments section. This is all going to be ok like somehow right?

 

 


The Hour is Later than Osborne Thinks

January 22, 2019

(Guest Post by Matthew Ladner)

Over at RedefinED I make the case that David Osborne’s fears of universal education savings accounts have in fact already come to pass with universal public schooling. Osborne opposes universal ESAs because he fears that wealthy families would top-off, well, guess what has been going on since the early 1970s:

The age of multi-vendor education dawned for upper income families decades ago. John Stuart Mill made a respectable case that government should restrict education activity to providing subsidies for the poor, but the justice and political viability of such a system seems deeply suspect given that no one has ever been told they can’t attend a education institution because their parents paid too many taxes. The best solution imo is to provide broad access with a significantly higher level of subsidy, but consider this your invitation to read the piece and reveal the flaws in thinking in the below comments section.