Jay P. Greene's Blog

(Guest post by Greg Forster)

In NRO today, Rick Hess and Sofia Gallo accuse the school choice movement of using “overheated” and “alarmist” rhetoric. They assert that since school choice is the opposite of neighborhood schooling (which all smart people know to be true, since we say it and we’re the smart people), choice advocates need a new, toned-down rhetoric to convince jumpy soccer moms that choice is no big deal and won’t have any big effects on anything.

How is this wrong? Let me count the ways:

1) “School choice advocates are wild-eyed ideological extremists” is a timeworn smear used by the Blob to demonize reformers, and anyone paying attention should have seen through it by now. For Hess and Gallo to resort to this lazy stereotype is offensive. The examples they bring to justify their claim that choice advocates are overheated and alarmist (choice is like Uber for schools!) redefine the whole concept of “weak tea.”

It is Hess and Gallo whose rhetoric is overheated and alarmist, in claiming that choice advocates are overheated and alarmist.

2) Hess and Gallo admit that choice is not only increasingly successful politically, it is growing more popular over time. Their efforts to create the impression that choice has a public perception problem (choice underperforms when compared to pie-in-the-sky hypothetical utopian alternatives in heavily biased survey questions formulated by the servants of the Blob at PDK!) don’t change the basic facts.

By Hess and Gallo’s own showing, choice is winning in statehouses, winning in governors’ mansions, and winning in public opinion polls. No doubt that success will ebb and flow in the future, as it has in the past. But choice has better public perception today than at any time in its history.

3) School choice is not in tension with local control, it is local control. For half a century, governance of district schools has moved further and further out of the local neighborhood and up the bureaucratic ladder, from the building to the district to the state and federal levels. There is no plausible plan for reversing that movement, other than school choice.

The only possible future of “neighborhood schools” is neighborhood schools of choice. Nothing else but choice will return governance of schools to the neighborhood level.

Because guess what neighborhoods are made up of? Parents. And parents who are given school choice exercise that choice as members of their local communities, gathering information and forming relationships in neighborhoods.

4) Parents tend to like their own schools, so Hess and Gallo recommend that choice advocates adopt a message along the lines of “choice won’t change schools.” Because that’s how you sell a reform – argue that it doesn’t matter and won’t change anything.

Or perhaps the message will be, “choice won’t change schools for people like you, it will only change schools for those other people. You know the ones we mean.”

Jay has been pointing out for years that the biggest mistake education reformers have been making is to argue that their policies will benefit a small and relatively powerless portion of the population, and offer no benefits to larger and more powerful constituencies. Hess and Gallo want choice to double down on that strategy.

5) To the extent that choice advocates could do better in framing their rhetoric, the problem is not that choice is percieved as a threat but that choice is percieved as of limited value because it is disconnected from moral imperatives like justice, equal opportunity, diversity and freedom. School choice advocates often assume that when they talk about markets they are affirming those imperatives, but in fact the language of markets does not and will not invoke those commitments for most people. A new language of school accountability through choice is needed to connect school choice to the things that matter most.

We shouldn’t talk as if choice should matter less, but as if it should matter more. Because it does.

Museums and theaters should stop telling me what to think about art.  I know that the folks who run museums and theaters think they are just providing context and facilitating discussion, but too often they are actually attempting to control what their patrons think about art works and plays with excessive gallery text and after show “talkbacks.”

I have no expertise in curating galleries or presenting plays, but I can speak as a frequent consumer of the arts that this well-intentioned, but ultimately bossy, deluge of information interferes with my direct experience and enjoyment of the art.  And I’m not the only one who feels this way.  Last year the playwright, David Mamet, forbid talkbacks following his plays.  He was mocked by some in the theater community for this, but I understand what drove his action.  Too many theaters were hosting talkbacks after his plays in which the theater staff or an expert they selected were obviously steering the audience toward particular and simplified interpretations of his work that might make it less controversial.  As another playwright, Christopher Shinn put it: “Broadly speaking, theaters use talkbacks to protect the audience from uncomfortable feelings the play may have aroused.”

I’m all for discussing plays after you see them, but that’s why you should go for dinner or drinks afterwards. When theaters host the discussion they cannot help but use their authority to drive the discussion in certain directions.  I don’t want my theater to tell me what to think about the play I just saw.  I want to develop my own thoughts and talk to others without the mediation of self-appointed experts on its meaning.

Shaping what people think about a play is especially likely if the theater-facilitated discussion immediately follows the performance.  That’s why some theaters hold events at separate time or in separate locations, more clearly demarcating the interpretation from the performance itself.  This seems like a reasonable compromise for theaters concerned about expanding audience engagement without being too controlling.

Some art museums have also drifted away from excessive gallery text.  The Isabella Stewart Gardner museum in Boston, for example emphasizes: “Isabella created her installations to evoke an emotional response in visitors. That’s why, unlike at other institutions, there aren’t conventional labels in this museum. She wanted you to find your own meanings.”  But other museums, while trying to avoid “the priestly voice of absolute authority,” still feel obliged to cover their walls in verbiage about the social and historical context of the works they display.

I sympathize with the impulses of museum staff to try to help their patrons, but I fear that they have too little trust in the ability of the art to communicate without mediation.  In addition, social and historical contexts are complicated and often disputed, so when museums try to convey that context they are inevitably making choices about what the correct understanding of history and sociology should be.  I am no more interested in having my museum tell me what to think about the world than having it tell me what to think about art.

Also posted on the University of Arkansas’ NEA Research Lab Blog.

NAEP results are being released next week, but state departments of education have already been briefed on their results.  State education officials are leaking like sieves, so many edu-pundits have at least some inkling of what’s coming.  Rumors from multiple sources suggest that the results generally look bad — with a decline nationally.  Aware that they may be blamed for declines, a number of folks are anticipating the release by placing their own spin on the soon-to-be-released results.

Exhibit A in this pre-spinning is John White, who is the superintendent of education in Lousiana.  According to Matt Barnum at Chalkbeat White sent a letter to the U.S. Department of Education on March 23 raising concerns about the comparability of NAEP results over time given the transition to computer-administered testing.  Although he acknowledges that NCES made adjustments to ensure the comparability of paper and pencil and computer administered tests and found insignificant differential effects by sub-group, White still raises questions about whether differential effects may distort results for certain states.  Barnum notes: “Even though researchers warn that it is inappropriate to judge specific policies by raw NAEP results, if White’s letter is a signal that Louisiana’s scores have fallen, that could deal a blow to his controversial tenure, where he’s pushed for vouchers and charter schools, the Common Core, letter grades for schools, and an overhaul of curriculum.  White said his state’s results are not what’s driving his concerns.”  Hmmm.  Maybe it’s just a remarkable coincidence that White has suddenly developed these technical concerns about the validity of NAEP at about the same time that he was briefed on his state’s results.  How much do you want to bet that there is a decline in LA?

Exhibit B is Arne Duncan taking to the pages of the Washington Post to defend the idea that ed reform has contributed to significant improvement.  He focuses on trends over the last few decades.  That would be a smart move to focus on long-term gains if recent trends — you know, in the wake of Duncan’s tenure as U.S. Secretary of Education — have been taking a nose-dive.  NAEP results slipped for the first time when 2015 results were released.  How much do you want to bet that national results have declined again?

Barnum is right to warn people away from inferring too much from changes in NAEP as an indicator of the success of any particular policy or education leader, but these folks live in a political, not a research, world.  Both White and Duncan’s political standing in education policy was built on over-claiming from NAEP results, and those who live by the NAEP sword may die by it.  That’s why they better start spinning.

(Guest post by Greg Forster)

Ah, yes, it’s once again that magical time of year when a young man’s thoughts turn to fools. I refer of course to The Higgy – the William Higinbotham Inhumanitarian of the Year Award. For the next two weeks, we’ll highlight our (un)favorite inhumanitarians; please join the fun and submit your nominations by email or comment! This year’s most (un)deserving nominee will be (dis)favored with The Higgy on Tax Day, April 15.

The award is named for history’s greatest monster, William Higinbotham.

For reference, here once again are the official criteria for selecting the winner:

“The Higgy” will not identify the worst person in the world, just as “The Al” does not recognize the best.  Instead, “The Higgy” will highlight individuals whose arrogant delusions of shaping the world to meet their own will outweigh the positive qualities they possess.

In other words, we’re looking for PLDD not BSDD.

In that spirit, I’m nominating pointy-headed academic and bloviating edu-blogger Jay P. Greene for The Higgy this year. His arrogant delusions of shaping the world include endorsing Common Core, union corruption, communism and even slavery. Last year, though, he went too far – he took all the fun out of The Higgy by giving it to Plato. He said the Republic could earn The Higgy even if it was actually a satire of irresponsible utopianism. Well, guess who else writes satires of irresponsible utopianism!

April Fool’s, of course! My real nominee will be coming in the next few weeks – as, I hope, will yours. May the best fool win!

(Guest Post by By Douglas J. McRae and Williamson M. Evers)

Did Smarter Balanced mishandle the bank of test-questions for 2017? Test scores dropped in virtually all states using Smarter Balanced national tests in their statewide testing programs in 2017.

States that used the Common Core-aligned Smarter Balanced tests showed English/Language Arts and Mathematics composite declines for 11 of the 14 states using these tests spring 2017, neither a loss nor a gain for 2 states, and a very modest gain for only a single state. Looking only at E/LA scores, there are declines for 13 states and only a tiny fraction of one percent gain for one state.

Tony Alpert, executive director for the Smarter Balanced Assessment Consortium, argues these results show that scores from its states are on a plateau. No, instead, there has been a substantial consortium-wide decline in scores.

We can compare the 2017 declines to consortium-wide composite score gains for 13 out of 14 Smarter Balanced states in 2016, and to composite gains for the parallel PARCC consortium for 2017 for all except one state. Both of these comparisons make the 2017 Smarter Balanced declines look like a sore thumb pointed downwards.

Yet Smarter Balanced continues to stonewall against releasing actual evidence or independent analysis of data contributing to the 2017 declines in test scores.

Alpert’s Jan. 26 opinion piece acknowledged for the first time that the 2017 item bank was changed from the 2016 item bank in a significant way. All information released by Smarter Balanced prior to Jan 26 indicated that the 2016 item bank was unchanged from 2015, and there was no public notice that the 2017 item bank was in fact modified from the 2016 item bank.  This lack of transparency from Smarter Balanced adds to concerns that the 2017 declines may be traced to changes for the 2017 test-question bank.

Alpert says that the item bank was “similar between the two years.” Well, “similar” isn’t good enough for valid, reliable gain-scores from year-to-year or trend data over multiple years — certainly one of the major goals for any K-12 large-scale statewide testing program. We need more evidence than an assertion of similarity.

To generate valid reliable gain scores from year-to-year, a test maker has to document that changes made for any item bank do not change alignment to the academic content standards that are being measured, as well as not changing the coverage of what is in the blueprints for the test. In addition, the balance of easy-medium-hard test questions has to match the prior item bank, or scoring adjustments need to be made to reflect changes. This information has to be available before a modified item bank can be used for actual test administrations.

A glimpse into this information surfaced on Feb 9 in a document linked to an Education Week post on this issue. This link was to Smarter Balanced subcontractor (AIR) technical report dated Oct 2016 (but not made public by Smarter Balanced until recently). It includes an appendix on changes for item characteristics for Smarter Balanced operational item banks for 2016 and 2017; these charts showed the addition of more “easy” items for E/LA and Math for grades 3 and 4, and addition of more “hard” items for E/LA and Math for grades 5, 6, 7, and 8. This mix of additional items for the 2017 testing cycle indicates the 2017 item bank had more difficult items than the 2016 item bank, which unless Smarter Balanced adjusted their scoring specifications for the 2017 test, would be consistent with the decline in scores from 2016 to 2017 documented in late September 2017. Smarter Balanced has not released information to date on whether the scoring specifications were adjusted for differences in difficulty of the tests on a grade-by-grade basis for both E/LA and Math for the spring 2017 test administration cycle.

In addition, a test maker should monitor the item-by-item data for a revised item bank early during an actual test administration cycle to insure the new items added to the bank (for either replacing former items or expanding the size of the bank) are performing as anticipated, in order to further modify scoring rules as needed to ensure comparability of results from year-to-year. According to the Feb. 9 Education Week post, Smarter Balanced said they were now doing these analyses, long after-the-fact.

The Smarter Balanced lack of transparency for critical information on this issue is quite troubling. So far, Smarter Balanced has released no information confirming these routine test integrity activities were done prior to scoring and releasing spring 2017 test results for 5-6 million students from 14 states.  Smarter Balanced has the behind-closed-doors data from the 2017 testing cycle; those data are required for informing any changes for the 2018 testing cycle which is already underway. If the 2017 data does not inform changes for the 2018 testing cycle, lack of routine comparability activities will affect Smarter Balanced annual gain scores and trend data for years into the future.

If Smarter Balanced has the evidence outlined above to justify their claim “we have every reason to believe that the scores accurately describe what students knew and were able to do in spring 2017,” then the professional thing to do was to make that information available concurrent with the release of 2017 test results in the fall of 2017, with perhaps more comprehensive analyses released before the beginning of the next test administration cycle.

Such transparency would have informed students, teachers, parents, school & state administrators, state policy makers, the media, and the public of the integrity of previously released test results and would have offered justification for any changes for the upcoming testing cycle. More transparency from Smarter Balanced would also allow independent experts to review and validate their currently behind-closed-doors data. What is Smarter Balanced hiding? Why isn’t it transparent like any other professional testing organization?

We repeat our call for Smarter Balanced to open the wall of secrecy for the information needed to investigate their 2017 consortium-wide score decline problem, and allow their claims to be examined by independent experts. The Smarter Balanced January 29 opinion piece falls woefully short of providing evidence their 2017 tests provided comparable scores from year-to-year upon which to conduct gain and trend analyses. We deserve better analysis and explanations from Smarter Balanced, along with much greater transparency for all parties interested in statewide K-12 test results across the country.

Douglas J. McRae is a retired educational-measurement specialist from Monterey, California.  Williamson M. Evers is a research fellow at Stanford University’s Hoover Institution and a former U. S. Assistant Secretary of Education for Planning, Evaluation and Policy Development.