The Way of the Future in Assessment

July 17, 2014


We control the SAT, ACT, GED and AP. Who the hell are you?

(Guest post by Greg Forster)

Hope Matt doesn’t mind me borrowing his Way of the Future theme, but there’s no better way to point y’all to this fascinating article on people experimenting with ways to measure non-cognitive traits, like “heart” and “grit,” that have a huge impact on education and life outcomes.

While the article focuses on colleges using such measures to predict collegiate success of applicants, the measures are just as badly needed (if not more so) to change the way we measure success in K12. There is really no question that these traits, just like the cognitive outcomes we currently measure with standardized tests, are partly a result of genes and environment but also partly a result of school performance. We need a revolution in thinking about K12 that puts non-cognitive outcomes back at the center of education, where they belong. That isn’t likely to happen until we can measure these outcomes.

The early methods are still riddled with challenges, of course, as you would have to expect at this stage. And the people involved (as well as the reporter) have an unfortunate attachment to some of the usual nonsense about the evils of standardized tests. These may or may not be the people who invent the assessments we need. Often the first people to take on a tough new task are only clearing the way for greater lights to come. But there is no doubt about the need, and every little bit helps.

Epic Fail in Arizona

November 14, 2013

(Guest Post by Matthew Ladner)

The Arizona Board of Regents put out a tracking study on the High School Class of 2006.  Arizona Republic reporter Ann Ryman lays out the relevant data in the first couple of paragraphs:

Half of the state’s public high schools saw 5 percent or fewer of their graduates from 2006 earn bachelor’s degrees, a new study finds.

And 62 percent of the college degrees earned by the high-school Class of 2006 went to students from just 40 of the state’s 460 high schools.

The report out today from the Arizona Board of Regents is the first in the state to provide a snapshot of college-completion rates for individual high schools. For six years, the regents tracked 53,392 Arizona students who graduated from high school in the 2005-06 school year, regardless of whether they moved or attended college out of state.

Using data from colleges nationwide, the report found that 57 percent of the Arizona students who graduated from high school in 2005-06 went on to college, but only 19 percent graduated from a four-year institution within six years.

An additional 6 percent graduated from a two-year college or trade school.

So after six-years we are looking at 25% getting some sort of credential. Half of Arizona high-schools get 5% or fewer of their graduates to earn a BA.  These results, while shocking, are actually consistent with the very low reported completion rates at Arizona’s three universities and the even lower rates reported by community colleges.

Where does one even start with this?

Perhaps with higher-education itself. This study takes aim at Arizona’s incredibly dysfunctional K-12 system, and rightly so. Let’s not however divert our attention from the role that higher-ed plays in all of this. The universities do not require the use of a college admissions exam for students graduating in the top quarter of, oh yes, those Arizona high-schools they just so effectively bashed.  Community colleges have even lower admission standards, some exercising an “open door” policy that don’t even require trivial little things like a high-school graduation.

This sets the tone for K-12 and in so doing sets up many Arizona children to fail. The universities and colleges have no problem taking money from unprepared kids and flunking them out in droves, but (call me crazy) it might serve them better by setting some minimum standards for entry and communicating those standards forcefully down to the K-12 system.

As you might expect in a state with half of the high-schools getting 5% or fewer of their kids to graduate from college after six-years, the K-12 system is just a mess. Most of the few bright spots are among schools of choice in the state, but on the whole we are looking at a catastrophe.  Defenders of the system will be quick to claim that it is Arizona’s relatively low spending per pupil that is to blame, but this won’t do for two broad reasons. First Arizona schools spend beyond the dreams of avarice of their predecessors from previous decades.  Second the state is relatively poor with wealth concentrated among retirees who came here from somewhere else with housing standing as the state’s main industry. You can guess where that winds up in terms of residential property tax rates for a state whose main industry is keeping retirees out of the cold.  Finally the state has a large number of old people and a large number of young people- translating to one of the highest age dependency ratios in the country. More than is normally the case around the country, Arizona taxpayers are either not working age yet, or past their prime earning years.

Finally even if the state had a huge amount of money burning a hole in its pocket (it doesn’t) it isn’t remotely clear that Arizona’s districts deserve anyone’s confidence in doing good things with the money. Better to create incentives for improvement and deliver additional funding upon the documentation of that improvement, which is the path that Arizona Governor Jan Brewer has endorsed.

The state’s accountability system jumped the shark a decade ago.   The initial AIMS test was a rigorous exam that told Arizonans information that they didn’t want to hear, especially those working in the system. This brought on to the biggest dummy down in cut scores in the recorded history of the United States.  The testing system devolved into a bad joke- rampant item exposure and drilling to individual test items.  Our kids got better and better at taking a dummy downed AIMS exam while our NAEP scores flat-lined and very few students make it through college.

This is what is so sad about Arizona activists spending their time fighting Common Core. Um, guys, Arizona is not Massachusetts.  I have no idea how Common Core will turn out or even if it will stick around but it would boggle my mind if it somehow turned out worse than the status-quo here in our patch of cactus.  Arizona has a huge problem regardless of what happens next on standards, and btw, our current set of tests and standards did approximately nothing to prevent this problem.  Simply being against Common Core without any thought about what should be done to replace AIMS is a luxury that Arizona cannot afford.

Arizona adopted A-F school grading a few years ago, but in 2012-13 61 percent of schools received an A or a B grade.  Some cruel person could have a great deal of fun cross listing the Arizona Republic’s data base on college graduates with the school grades, but let’s resist such temptation for now. I will simply note that the NAEP exam shows very low percentages of Arizona students reading with full grade level proficiency and the Arizona Board of Regents has now found catastrophically low college completion rates. We would do well therefore to set challenging standards for school grades rather than throwing around A and B grades like beads at a Mardi Gras parade.

In short, I believe that Arizona needs a coordinated effort at the K-12 and higher education levels to toughen up what is an incredibly soft system.  Arizona’s educators policymakers are not bad people, and it was not wicked motivations that got us in to this mess. It seems nice not to require high-school students to do much of anything to graduate from high-school. It feels egalitarian and democratic to have open door policies in higher education. We can hope against hope that the handful of Arizona schools getting C grades will strive to get A/B grades, but it feels kinder and gentler to rig the game in such a way that profoundly mediocre results can get you a good grade. The road to hell-in this case backwater status- is paved with good intentions.

The problem with the delicate approach is that it systematically puts a higher priority on the comfort level of adults rather than the needs of Arizona’s children.  You can’t paper over illiteracy and the consequences of all this softness is a system that is failing to prepare students for the future.

Administrative Bloat in the WSJ

December 30, 2012

This weekend the Wall Street Journal had a front page piece detailing how administrative bloat in higher education is causing costs to spiral higher.  The piece carefully dissects hiring patterns at the University of Minnesota to illustrate their general conclusion that:

Across U.S. higher education, nonclassroom costs have ballooned, administrative payrolls being a prime example. The number of employees hired by colleges and universities to manage or administer people, programs and regulations increased 50% faster than the number of instructors between 2001 and 2011, the U.S. Department of Education says. It’s part of the reason that tuition, according to the Bureau of Labor Statistics, has risen even faster than health-care costs.

This conclusion is not new, but it is great to see it getting the attention it deserves.  As regular readers of JPGB will remember, Brian Kisida, Jonathan Mills, and I released a report on administrative bloat at the nation’s 200 leading research universities.  That report elicited a tizzy fit from Arizona State University President Michael Crow, including a letter to the chancellor of my university accusing me of academic fraud.  University leaders who should be the guardians of academic freedom are too often its greatest oppressors.

But the obvious facts about administrative bloat cannot be suppressed.  Johns Hopkins professor Benjamin Ginsberg recently published an excellent book on the topic.  And despite a lousy attempt by the professional association of State Higher Education Officers to spin the data, a subsequent analysis by the Pope Center successfully replicated our results and confirmed out conclusions.

And now we have a front page article in the Wall Street Journal reporting the same thing.  Michael Crow might try writing an angry letter to the editor but university leaders won’t be able to shut this story down.  The good university leaders are already taking steps to reign in runaway non-instructional, non-research costs.  See for example Erskine Bowles efforts at the University of North Carolina or the new leadership at the University of Minnesota.  The bad university leaders will bluster, brow beat, and continue to expand the mission of universities beyond their core missions of teaching and research.

The Ultimate Debate!

June 27, 2012

(Guest post by Greg Forster)

Old and Tired: Star Wars versus Star Trek – which is better?

New and Fascinating: Greg Forster versus Rich Vedder – whose evisceration of the student aid regime is more devastating?

I vote for Rich. The constraints of the WSJ debate required me to address a more narrow set of questions. Rich has marshalled a more comprehensive takedown in the new Imprimis. It’s your one stop shop for everything that’s dysfunctional and destructive about the student aid regime.

HT Basic Instructions, my favorite webcomic – see more legible versions of these comics here and here!

WSJ Also Hosts College Financial Aid Showdown

June 24, 2012

In addition to the debate between Checker and me on national standards, the Big Issues in Education series that the Wall Street Journal is publishing features a variety of great debates.  I just saw that one of those is a debate between our very own Greg Forster and Mark Kantrowitz about college financial aid.  Here’s a taste:

Mark Kantrowitz:

Without some form of aid, the cost of a four-year college is beyond the reach of most low-income families. Even taking grants into account, the average annual yearly cost is more than one-half the average annual family income for low-income students….

Millions of students never even enter college due to their own limited financial resources, inadequate need-based grants or both. In a 2006 report by a congressional advisory committee on student financial aid, only 54% of college-qualified low-income students enrolled in a four-year college and 21% in a two-year college. For high-income students, the comparable figures were 84% and 11%.

Claims that there is no gap in college access for low-income students are based on a flawed analysis that understates college readiness and overstates enrollment figures. College-readiness figures like the ones my opponent cites look only at 17-year-old high-school graduates who satisfy minimal entrance requirements for four-year colleges. But people also qualify who are older than 17, some of whom didn’t graduate from high school but possess the high-school equivalency credential known as the GED.

In reality, more than half of low-income college-ready students don’t enroll in four-year bachelor’s degree programs because they can’t afford the cost.

Greg Forster:

For 60 years, we have been pouring more and more money into collegiate financial aid with little or no regard to the academic merit of the recipients. Unlimited, unmerited college financing has produced skyrocketing tuition rates, lower academic standards, and runaway spending on collegiate administration and services that deliver no visible academic benefits….

Proponents of need-based aid cite studies suggesting there is a big gap in college-enrollment rates between low-income and high-income students with the same qualifications. But those studies are misleading because the qualifications they cite don’t match up with the real entrance requirements of colleges. They omit transcript requirements, for example, like numbers of years of English and math.

In fact, virtually all low-income students who aren’t going to college aren’t qualified to go to college. Empirical research by myself and others has consistently found that the number of high-school graduates who meet the academic requirements to attend a traditional four-year college and the number of students actually entering traditional four-year colleges is almost identical.

The problem for low-income students isn’t a lack of aid—it’s a lack of quality education at the K-12 level. Almost the only way to expand educational opportunity to the truly needy is through academic, not financial, reform. Too many kids at the K-12 level never have a chance to become college-ready….

Of course colleges jack up their tuition. They’re capturing the subsidies we provide. Like any other service provider, colleges will raise prices until the market clears. Flooding the market with subsidies gives customers more purchasing power. The market clears at a higher price.

But higher tuition is only the obvious symptom; putting all that financial aid on the table with no connection to academic merit creates a huge incentive to dumb down academics. Colleges can pick up a lot of free money by relaxing admission standards. This helps explain why more than a third of freshmen now take remedial courses.

And since colleges no longer compete on price, they compete on amenities. That’s why we see so many new buildings and services on campuses, but so little improvement in educational results. From 1993 to 2007, according to a study by Jay Greene, an education professor at University of Arkansas, college administrative spending per student grew at twice the rate of instructional spending.

Administrative Bloat Study Successfully Replicated

March 26, 2012

Replication is the engine of scientific progress.  That progress feels especially good when it confirms one’s work.

A little more than a year ago I wrote an analysis for the Goldwater Institute along with Brian Kisida and Jonathan Mills on the growth in non-instructional professional staff at major universities — or administrative bloat.  Then last year the State Higher Education Executive Officers (SHEEO) issued what appeared to be a rebuttal analysis in which they claimed that “public colleges and universities are operating more efficiently than before, and with appropriate numbers of staff.”

Recently the Pope Center examined both of these studies and then conducted their own new analysis.  They concluded:

the Pope Center analyzed the two studies and also roughly replicated both of them for the 16 campuses of the University of North Carolina system. While we do not claim to be the definitive voice on the matter, we discovered that one of the two studies—the one that said excessive staffing is a serious problem—seemed to be on the mark. The other contained some truth but also raised a few questions about its objectivity….

Our findings, which focused entirely on the UNC system, corroborated the Goldwater study for the most part. Between 1993 and 2010, total UNC system staffing indeed grew faster than enrollment: 51 percent against 42 percent; the number of total staff members per 100 students grew 5.9 percent….

The failure to mention the more recent upward trend in staffing [in the SHEEO report] was puzzling—certainly anybody who has looked at statistics professionally would be able to pick up the trend reversal and realize its significance. Such an important omission raises the possibility that the SHEEO researchers also “cherry-picked” 2001 as a starting point in order to show an overall decline in staffing, rather than the real long-term trend that staffing is rising. (There are no such concerns about the Goldwater study—the researchers chose 1993 because that was the first year for which this type IPEDS was available.)

Ahh.  Vindication is sweet.

The Miseducation of a University President

November 16, 2011

(Guest post by Jonathan Butcher)

Aesop tells us that every man carries two bags, one in front and one behind, both full of faults. The bag in front contains the faults of others, while we carry ours in the one behind. As a result, we always see someone else’s mistakes and rarely look at our own.

Writing in the Washington Post earlier this month, ASU President Michael Crow chastises universities for not being more innovative during the current financial crisis. “Their lack of creativity in adjusting to the reduction of resources has shocked governors and business leaders alike who want to see universities innovate in order to educate more students better, faster and cheaper,” said Crow.

But ASU hasn’t exactly been a model of efficiency. Across the country, colleges are hiring massive numbers of administrators and ASU is no exception — in fact, the Sun Devils are an example of this “administrative bloat.” Between 1993 and 2007, ASU increased the number of full-time administrators per 100 students more than 167 other comparable universities while the number of instructional staff and researchers actually decreased. Goldwater research finds “[n]early half of all full-time employees at Arizona State University are administrators.”

Considering these hiring practices, Crow’s warning that proposals to turn universities into businesses are “ill-conceived” is remarkable. Also “ill-conceived” are programs that would “send our kids to college in the basement with the local online university.” Yet the University of Phoenix reports that some 75 percent of higher education students today “are older…work full or part time and have family responsibilities, including financial obligations,” which means online access to college classes may be the only access they have.

ASU and other state universities should focus on core academic programs and direct spending not to administration but on practices directly tied to student instruction. In addition, Arizona colleges and universities should align tuition more closely with the actual costs of providing an education, pursue more private funding, and make themselves more financially self-sufficient.

Jonathan Butcher is education director for the Goldwater Institute.

True to Her Traditions – At Last

November 11, 2011

(Guest post by Greg Forster)

On Veterans Day two years ago I posted a sharp condemnation of my grad school for its contempt of the military, even in defiance of its own traditions. In the comments, I made a promise that I would be prepared to post something more cheerful for Veterans Day “when the Ivies quit spitting on the people who fight and die to preserve their right to spit on them.”

Yale is bringing back ROTC, along with Harvard and Columbia. Princeton refuses to budge. Brown is still considering its position. Cornell, Dartmouth and Penn had already brought it back before this year.

I’m not sure at what point my stated obligation to “post warm fuzzies about mom and apple pie” kicks in, and I’ll admit that I don’t think the series of events leading up to these developments generally augers well for civil/military relations. But that war is over and now is a time for reconciliation. Six of the eight Ivies now offer military training within their for-credit educational curricula. That is progress.

One Book, But Why That Book?

September 13, 2011

In the last decade a large number of colleges and universities have initiated a “community reading” program, where everyone in a university and its neighboring community is required or strongly encouraged to read one book and discuss it over a series of events in an academic year.

In principle the One Book idea sounds great.  Even as core curricula in higher education are being eviscerated, this appears to be an effort to have a shared intellectual experience on issues that are central to the missions of each participating university.

The practice, however, has not met that potential.  As Harold Bloom put it, “I don’t like these mass reading bees… It is rather like the idea that we are all going to pop out and eat Chicken McNuggets or something else horrid at once.”  Of course, we don’t have to select the book equivalent of Chicken McNuggets, but in practice that’s what universities appear to be doing when they choose their One Book.

The National Association of Scholars (NAS) has a report that documents what books universities actually choose based on a review of One Book programs at 245 universities and colleges.  The results are incredibly disappointing.  Rather than choosing high quality and intellectually stimulating books, universities tend to pick current, shallow, popular books. In particular the NAS report finds:

First, almost 90% of the books selected were published since January 2000.  If important works tend to stand the test of time, almost none of the One Books have passed that test.  Once you look at the list of what was selected, I think it’s safe to bet that almost none of them will be read a hundred years from now.  Rather than exposing the university community to enduring truths or works of enduring quality, the One Book programs almost always picks a topic that is likely to be a passing fad or a disposable work.

Second, the topics are remarkably skewed toward what is considered politically correct.  Out of the 245 selections, 58 were on African American themes, another 18 on African themes, 10 on Native American themes, 7 on Latino themes, 5 on East Asian themes.  24 One Books were about environmentalism, 10 about Hurricane Katrina, 10 were comic books or graphic novels, and 8 were self-help books or about the pursuit of happiness.

Third, memoirs and biographies dominated the list.  There were 79 memoirs and 62 biographies, more than half of the total.  Why so many memoirs?  The NAS report answers:

… memoirs are “a genre familiar to students.” In high school English courses, students are taught to base their interpretation of works of literature on their own personal experiences. A recent study on high school literary study finds that this emphasis on the personal “may be contributing to the high remediation rates in post-secondary English and reading courses.”

Training students to write from the perspective of personal reflection gives them a taste for more of the same. This is one explanation for the popularity of the memoir in common reading programs. Another is that our society has an appetite for true stories. The growing number of reality TV shows is evidence of this. Getting to hear from the author in person at a scheduled campus speech is part of the allure of the memoir. The emphasis on memoir may also reflect the rise of post-modern sensibilities in American higher education. A memoir often presents “my truth,” rather than “the truth.” It is a way of asserting the primacy of self and the importance of opinion as trumping common judgment, authority, and hard-won facts.

The most popular One Book is The Immortal Life of Henrietta Lacks(2010) by Rebecca Skloot, which was selected at 39 of the 245 institutions.  The NAS report describes the work:

The book does make a history of complex scientific research accessible to average readers, and Skloot explains biological jargon in simple terms. Readers will come away from the book having learned new things, but the writing itself is journalistic, not intellectual. Judging by what they say about it, some colleges seem to have chosen The Immortal Life of Henrietta Lacks in part because it can be read as a story of racial injustice.

Tied for the next most popular is Zeitoun by Dave Eggers, which was selected by 9 institutions. Wikipedia summarizes the plot:

 It tells the story of Abdulrahman Zeitoun, the Syrian-American owner of a painting and contracting company in New Orleans who chose to ride out Hurricane Katrina in his Uptown home. After the storm he traveled the flooded city in a secondhand canoe rescuing neighbors, caring for abandoned pets and distributing fresh water. Soon after the storm, Zeitoun was arrested without reason or explanation at one of his rental houses by a mixed group of National Guardsmen and local police. He was not immediately charged with a crime but was imprisoned for 23 days without having stood trial. During that time he was accused of terrorist activity presumably because of his ethnicity, was treated inhumanely, and was refused medical attention and the use of a phone to alert his family. His wife and daughters, staying with friends far away from the city, only knew that he had seemingly disappeared from the face of the earth.

Also selected by 9 institutions was This I Believe: The Personal Philosophies of Remarkable Men and Women, which is a self-help book by Jay Allison and Dan Gedimen.  Seven institutions picked The Other Wes MooreOne Name, Two Fates by Wes Moore.  NPR summarized the plot:

 In the book, author Wes Moore tracks his own life, alongside the fate of another man of the same name.

While both Wes Moores grew up in poverty in Baltimore, the two men had dramatically different fates: The author became a Rhodes Scholar, while the other Moore is serving a life sentence in prison for murder.

And 6 universities or colleges chose No Impact Man by Colin Beavan about a New York City family that attempts to have no impact on the environment for an entire year by buying nothing newly made, producing no non-compostible trash, and only buying food produced within 250 miles of their apartment.

Another 6 chose The Absolutely True Diary of a Part Time Indian, which Wikipedia says “is a novel for young adults written by Sherman Alexie. It is told in the first-person, from the viewpoint of Native American teenager and budding cartoonist Arnold Spirit, Jr. (better known by the nickname “Junior”). Detailing Arnold’s life on the Spokane Indian Reservation and his decision, upon encouragement from a reservation high school teacher, to go to an all-white high school in the off-reservation town of Reardan, Washington, the novel deals with issues such as racism, poverty, and the following of tradition.”

If you didn’t notice Shakespeare, Camus, Ellison, or Plato on the list, you’d be right.  But the NAS helpfully compile a list of 37 suggested books that includes these authors and would be far better for One Book programs.

As the old United Negro College Fund commercial used to say, “A mind is a terrible thing to waste.”  So is the opportunity for everyone at a college or university to read and discuss a quality book.

(edited for typos)

Benjamin Ginsberg on Administrative Bloat in Higher Ed

September 9, 2011

Johns Hopkins political scientist, Benjamin Ginsberg has a new book out: The Fall of the Faculty: The Rise of the All-Administrative University and Why It Matters.  In it he documents how universities have experienced explosive growth in the number of administrators and other non-faculty professionals and how this administrative bloat is making costs soar while distracting universities from their primary mission.  His argument is virtually identical to the report I wrote last year with Brian Kisida and Jonathan Mills that was released by the Goldwater Institute.

I’m thrilled that Ginsberg is getting more attention for this issue.  Here is a taste from his Washington Monthly article summarizing his book:

Between 1975 and 2005, total spending by American higher educational institutions, stated in constant dollars, tripled, to more than $325 billion per year. Over the same period, the faculty-to-student ratio has remained fairly constant, at approximately fifteen or sixteen students per instructor. One thing that has changed, dramatically, is the administrator-per-student ratio. In 1975, colleges employed one administrator for every eighty-four students and one professional staffer—admissions officers, information technology specialists, and the like—for every fifty students. By 2005, the administrator-to-student ratio had dropped to one administrator for every sixty-eight students while the ratio of professional staffers had dropped to one for every twenty-one students.

Apparently, as colleges and universities have had more money to spend, they have not chosen to spend it on expanding their instructional resources—that is, on paying faculty. They have chosen, instead, to enhance their administrative and staff resources….

Every year, hosts of administrators and staffers are added to college and university payrolls, even as schools claim to be battling budget crises that are forcing them to reduce the size of their full-time faculties. As a result, universities are now filled with armies of functionaries—vice presidents, associate vice presidents, assistant vice presidents, provosts, associate provosts, vice provosts, assistant provosts, deans, deanlets, and deanlings, all of whom command staffers and assistants—who, more and more, direct the operations of every school. If there is any hope of getting higher education costs in line, and improving its quality—and I think there is, though the hour is late—it begins with taking a pair of shears to the overgrown administrative bureaucracy.

I also particularly enjoyed this bit Ginsberg had on strategic planning at universities:

Another ubiquitous make-work exercise is the formation of a “strategic plan.” Until recent years, colleges engaged in little formal planning. Today, however, virtually every college and university in the nation has an elaborate strategic plan. This is typically a lengthy document— some are 100 pages long or more—that purports to articulate the school’s mission, its leadership’s vision of the future, and the various steps that are needed to achieve the school’s goals. The typical plan takes six months to two years to write and requires countless hours of work from senior administrators and their staffs.

A plan that was really designed to guide an organization’s efforts to achieve future objectives, as it might be promulgated by a corporation or a military agency, would typically present concrete objectives, a timetable for their realization, an outline of the tactics that will be employed, a precise assignment of staff responsibilities, and a budget. Some university plans approach this model. Most, however, are simply expanded “vision statements” that are often forgotten soon after they are promulgated. My university has presented two systemwide strategic plans and one arts and sciences strategic plan in the last fifteen years. No one can remember much about any of these plans, but another one is currently in the works. The plan is not a blueprint for the future. It is, instead, a management tool for the present. The ubiquity of planning at America’s colleges and universities is another reflection and reinforcement of the ongoing growth of administrative power.

Be sure to check out Ginsberg’s book.  We plan to have some meetings to discuss it, will form a study group to consider recommendations, and will then issue an action-plan that is aligned with our strategic priorities.


Get every new post delivered to your Inbox.

Join 2,519 other followers