In case you missed the 60 Minutes segment on the Khan Academy. You can watch the video and read the transcript here.
And the New York Times reports on a study conducted in New York City comparing student achievement at 10 schools using a Core Knowledge approach against the achievement at 10 schools using existing (mostly Balanced Literacy). It find greater gains in the Core Knowledge schools in reading comprehension as well as content knowledge in social studies and science.
Leave a Comment » | 
instructional reform | Tagged: Core Knolwedge, Khan Academy | 
Permalink
Posted by Jay P. Greene
Comic book geeks are familiar with Bizarro World, a place where everything is the opposite of what it is in the normal world. In Bizarro World, people would abandon a policy strongly supported by rigorous evidence while embracing an alternative policy for which the evidence showed little promise.
I was thinking about Bizarro World and then it struck me — Perhaps the Gates Foundation has somehow fallen into the Bizarro World. It’s just about the only thing that makes sense of their Bizarro choices with respect to education reform strategies.
The dominant education reform strategy of the Gates Foundation before 2006 was to break large high schools into smaller ones, often using school choice and charter schools. As a Business Week profile put it:
The foundation embraced what many social scientists had concluded was the prime solution: Instead of losing kids in large schools like Manual, the new thinking was to divide them into smaller programs with 200 to 600 students each. Doing so, numerous studies showed, would help prevent even hard-to-reach students from falling through the cracks. The foundation didn’t set out to design schools or run them. Its goal was to back some creative experiments and replicate them nationally.
But the Gates Foundation wasn’t patient enough to let the experiments produce results. Instead, they hired SRI and AIR to do a very weakly-designed non-experimental evaluation that produced disappointing results. Gates had also commissioned a rigorous random-assignment evaluation by MDRC, but it would take a few more years to see if students graduated and went on to college at higher rates if they were assigned by lottery to a smaller school.
Gates couldn’t wait. They were convinced that small schools were a flop, so they began to ditch the small school strategy and look for a new Big Idea. Tom Vander Ark, the education chief who had championed small schools, was out the door and replaced with Vicki Phillips, a superintendent whose claim to fame, such as it was, came from serving as Portland’s superintendent where she consolidated schools (not breaking them into smaller ones) and centralized control over curriculum and instruction. As one local observer put it:
In her time in the famously progressive, consensus-driven city, she closed six schools, merged nearly two dozen others through K-8 conversions, pushed to standardize the district’s curriculum, and championed new and controversial measures for testing the district’s 46,000 children-all mostly without stopping for long enough to adequately address the concerns her changes generated in the neighborhoods and schools where they played out. During her three years in Portland, Phillips’ name became synonymous with top-down management, corporate-style reforms, and a my-way-or-the-highway attitude.
Under Phillips and deputy education director, Harvard professor Tom Kane, the Gates Foundation has pursued a very different strategy: attempting to identify the best standards, curriculum, and pedagogy and then imposing those best practices through a national system of standards and testing.
And here is where we see that Gates must be the Bizarro Foundation. The previous strategy of backing small schools has now been vindicated by the rigorous random-assignment study Gates couldn’t wait for. According to the New York Times:
The latest findings show that 67.9 percent of the students who entered small high schools in 2005 and 2006 graduated four years later, compared with 59.3 percent of the students who were not admitted and instead went to larger schools. The higher graduation rate at small schools held across the board for all students, regardless of race, family income or scores on the state’s eighth-grade math and reading tests, according to the data.
This increase was almost entirely accounted for by a rise in Regents diplomas, which are considered more rigorous than a local diploma; 41.5 percent of the students at small schools received one, compared with 34.9 percent of students at other schools. There was little difference between the two groups in the percentage of students who earned a local diploma or the still more rigorous Advanced Regents diploma.
Small-school students also showed more evidence of college readiness, with 37.3 percent of the students earning a score of 75 or higher on the English Regents, compared with 29.7 percent of students at other schools. There was no significant difference, however, in scores on the math Regents.
Meanwhile, as part of their newly embraced top-down strategy, the Gates effort to identify the secret formula for effective teaching has failed to bear fruit. The Gates -operated Measuring Effective Teachers Project failed to identify any rubric of observing teachers or any components of those rubrics that were strongly predictive of gains in student learning. And the Gates-backed “research” supporting the federally-orchestrated Common Core push for national standards and testing has been strikingly lacking in scientific rigor and candor.
In short, the Gates Foundation has ditched what rigorous evidence shows worked and is pushing a new strategy completely unsupported by rigorous evidence. They must be in Bizarro World. Somebody please get me some blue kryptonite.
14 Comments | instructional reform, research reports | Tagged: gates foundation, Measuring Effective Teachers, Smaller Schools | Permalink
Posted by Jay P. Greene
For a higher quality version that I cannot embed, click here.
3 Comments | instructional reform | Tagged: You Can Do Anything | Permalink
Posted by Jay P. Greene
(Guest post by Greg Forster)
Since we’re so deep into the subject of value-added testing and the political pressures surrounding it, I thought I’d point out this recently published study tracking two and a half million students from a major urban district all the way to adulthood. (HT Whitney Tilson)
They compare teacher-specific value added on math and English scores with eventual life outcomes, and apply tests to determine whether the results are biased either by student sorting on observable variables (the life outcomes of their parents, obtained from the same life-outcome data) or unobserved variables (they use teacher switches to create a quasi-experimental approach).
Finding?
Students assigned to high-VA teachers [i.e. teachers who produce high "value added" on test scores] are more likely to attend college, attend higher- ranked colleges, earn higher salaries, live in higher SES neighborhoods, and save more for retirement. They are also less likely to have children as teenagers. Teachers have large impacts in all grades from 4 to 8.
Let’s bring that down to reality:
Replacing a teacher whose VA is in the bottom 5% with an average teacher would increase students’ lifetime income by more than $250,000 for the average classroom in our sample.
But here’s what I want to pay the most attention to. Note the careful wording of the conclusion:
We conclude that good teachers create substantial economic value and that test score impacts are helpful in identifying such teachers.
Note what they don’t say. They don’t say that increasing math and English test scores by itself leads to improved life outcomes. They say good teachers lead to improved life outcomes, and value-add is one relatively good way to identify good teachers.
You’ve heard the saying that the map is not the territory? (If not, that means you haven’t seen Ronin, in which case shame on you.) Well, it’s true. What raises life outcomes is good teaching, and good teaching can’t be reduced to test scores. (See here, here, here, here, here, here, here and here.)
But if you want to find your way around the territory, you need a map. If you want to help those kids stuck with lousy teachers who are out a quarter million, you’re going to need a tool that identifies them. Value added analysis is the best tool we’ve come up with yet – other than parental choice, of course.
And where the tests are freely selected and voluntarily adopted by schools, the tests provide helpful data for parents, so parent choice is strengthened by voluntary testing. That’s why over 90% of private schools use testing in some form. On the other hand, forcing teachers to use a test they don’t believe in is a self-defeating proposal.
But how do you get schools to want to use a test? Parent choice, of course! Choice is what creates the external standard of performance that makes assessment tools seem legitimate rather than illegitimate. So testing and choice are like chocholate and peanut butter – they’re two great tastes that taste great together.
4 Comments | instructional reform, research reports, vouchers | Tagged: Greg Forster | Permalink
Posted by Greg Forster
Steve Cantrell, a senior researcher at Gates, sent me an email last night in response to my post from yesterday asking for the MET results to be released. He said that I was right in suggesting that large, complicated projects sometimes take longer than originally planned. He said that final scores for coding the videos had just been delivered to the research team and that the full results for the 2009-10 year were now scheduled to be released January 5, 2012. It’s unclear whether that report will also contain information for the 2010-11 year as well. The MET web site will be changed to reflect this new schedule. (Update: According to another email from Steve Cantrell, the January release will only have the full 09-10 results. The final results including 10-11 and are scheduled for release in early summer of 2012 .)
Steve also clarified information on the cost of the project. Last year I repeated the New York Times and LA Times description of the project costing $45 million. More recently I’ve repeated the Wall Street Journal description of the project cost as $335 million. Steve resolved the confusion by saying that the MET study costs about $50 million and the $335 million figure includes grants to the partner districts.
Let me be clear that I think Gates has a lot of good and smart people working on the MET project. My concern is not that these are bad people. My concern is that Gates has a flawed strategy based on centrally identifying what educators should do and then building a system of standards, curriculum, and assessments to impose those practices on the education system. I don’t think this kind of centralized approach can work and I fear that it creates enormous pressure on good and smart researchers to toe the centralized line — even if it becomes obvious that it is not working. Everyone at Gates can see what happened to the folks who pushed small schools when the Foundation decided that approach was not working.
And unlike Diane Ravitch, Valerie Strauss, and the Army of Angry Teachers, I am not criticizing the Gates Foundation because I think Bill Gates is in the “billionaire boys club” and therefore somehow disqualified from using his wealth to try to improve education. I am critical of recent Gates Foundation efforts because I believe Gates can and should try to improve education by adopting a more fruitful strategy.
(corrected typos)
11 Comments | instructional reform, research reports | Tagged: gates foundation, Measuring Effective Teachers | Permalink
Posted by Jay P. Greene
Bill and Melinda Gates mentioned again in the Wall Street Journal the Measuring Effective Teachers (MET) project that their foundation is orchestrating. Bill and Melinda may want to check on the status of the MET research they’ve been touting since full results were promised in the spring of 2011 and have yet to be released.
Just to review… In an earlier interview with the Journal, MET was described as follows:
the Gates Foundation’s five-year, $335-million project examines whether aspects of effective teaching, classroom management, clear objectives, diagnosing and correcting common student errors can be systematically measured. The effort involves collecting and studying videos of more than 13,000 lessons taught by 3,000 elementary school teachers in seven urban school districts.
The motivation, re-iterated in the new piece by Bill and Melinda Gates is to identify what “works” in classroom teaching to develop systems that train and encourage other teachers to imitate those practices:
It may surprise you—it was certainly surprising to us—but the field of education doesn’t know very much at all about effective teaching. We have all known terrific teachers. You watch them at work for 10 minutes and you can tell how thoroughly they’ve mastered the craft. But nobody has been able to identify what, precisely, makes them so outstanding….
The intermediate goal of MET is to discover what we are able to measure that is predictive of student success. The end goal is to have a better sense of what makes teaching work so that school districts can start to hire, train and promote based on meaningful standards.
As I’ve argued before, using research to identify “best practices” in teaching only makes sense if the same teaching approaches would be desirable for the vast majority of teachers and students, regardless of the context. And as I’ve also suggested before, I don’t believe this effort is likely to yield much in education. Effective teaching is like effective parenting — it is highly dependent on the circumstances. Yes, there are some parenting (and teaching) techniques that are generally effective for almost everyone, but those are mostly known and already in use.
This doesn’t mean we are completely unable to measure effective teaching (or parenting). It just means that we have to judge it by the results and cannot easily make universal statements about the right methods for producing those results. To make a sports analogy, there is no single “best practice” for hitters in baseball. There are a variety of stances and swings. The best way to judge an effective hitter is by the results, not by the stance or swing. And if we tried to make all hitters stand and swing in the same way, we’d make a lot of them worse hitters.
It is because of this heterogeneity in effective teaching practices that I think the MET project is doomed to disappoint. And according to inside sources, I’ve heard that results are being delayed because they are failing to produce much of anything.
According to the MET web site, the full results for the 1st year should have been released in the spring:
In spring 2011, the project will release full results from the first year of the study, including predictors of teaching effectiveness and correlation with value-added assessments.
It is almost November and we have not seen these results. I understand that in very large and complicated projects, like MET, things can take much longer than originally planned. If so, it would be nice to hear that explanation. It would be even nicer if the Gates Foundation released results if they have them, even if those results were not what they had hoped they would find.
Some inquisitive reporters should start asking Gates officials and members of the research team about the status of the MET results. Reporters should go beyond talking to the media flacks at Gates HQ and actually talk to individual members of the team confidentially. If they do that, they may confirm what I have been hearing: MET results have been delayed because they aren’t panning out.
(UPDATE: Gates responds.
8 Comments | instructional reform | Tagged: gates foundation, Measuring Effective Teachers | Permalink
Posted by Jay P. Greene
2 Comments | instructional reform | Tagged: Arne Duncan, common core, Marco Rubio, national standards, NCLB waivers | Permalink
Posted by Jay P. Greene
Digital learning has significant potential but it also faces significant political barriers. Existing regulations, such as seat-time requirements, teacher certification requirements, and the immobility of student funding, all stand in the way of rapid expansion of digital learning in K-12 education.
Notice that I did not include the lack of a national set of standards as a significant barrier to the expansion of digital learning. I understand that a number of backers of digital learning support the national standards movement because they believe it will allow digital learning providers to achieve scale and offer products in all 50 states without having to contend with 50 different sets of state standards.
But at the recent Harvard conference, Shantanu Prakash, the head of Educomp Solutions, one of the largest digital learning providers in the world, was asked whether different sets of standards were a major obstacle to his company’s operations. He conceded that the markets in which they operate, principally India, have numerous different standards. But he also said that this was a trivial barrier because one of the strengths of digital learning is that it typically consists of many small modules that can easily be added or dropped to fit every set of standards.
If backers of digital learning think we need to streamline state regulation to achieve scale, they should be focusing on teacher certification and seat-time requirements rather than standards. But would any of them really support the idea of having teacher certification and time requirements decided at the national level? Wouldn’t the opponents of digital learning be able to seize a national regulatory regime to block the expansion of digital learning everywhere? If so, why is the same concern not true for national standards?
The reality is that the biggest opponent of digital learning will be the teacher unions, who must recognize that digital learning allows cost-savings by replacing labor with capital. Digital learning backers will have to fight the unions in each state to ease teacher certification, seat-time, and the immobility of funding. At least now they have beach-heads in states that have a more accommodating regulatory environment. But if digital learning folks support the construction of a national regulatory regime, they may be marginalized everywhere.
9 Comments | instructional reform, National Standards | Permalink
Posted by Jay P. Greene
(Guest post by Greg Forster)
Duncan now triple-dog swears that you don’t have to sign up for Common Core to get a waiver . . .
. . . as long as you can “prove” you have “high standards,” as defined by Duncan.
http://blogs.edweek.org/edweek/campaign-k-12/2011/08/duncan_states_dont_need_to_joi.html
Wow, I never thought of that! Just think how much change we could effect with that method! And why let the government have all the fun? Anyone can play thus game – I hereby personally offer a million dollars cash to any school that can prove it has high standards, as defined by me.
In other news, the Legitimate Businessmen’s Neighborhood Business Protection Program hinted privately that I’d better join or my legs would be broken, but when the police asked them about it in public they said it was all a big misunderstanding. So I guess that proves they’re innocent! After all, what other possible explanation could there be?
More to the point: do you think people will stop fearing them now? A leak followed by a disavowal is a great way to intimidate people into doing what you want without getting called on it.
Like I said last week, now that the self-appointed champions of high standards have foolishly chosen to start a war over nationalization, this won’t really be over until that war has been fought to a clear conclusion. Way to go, guys!
Leave a Comment » | instructional reform | Tagged: Greg Forster, national standards | Permalink
Posted by Greg Forster
Paul the psychic octopus sez: “Toldja so!”
(Guest post by Greg Forster)
With the shooting war that’s emerging betweeen Arne Duncan and Rick Perry over national control of education, some of the people who helped facilitate the movement toward nationalization are now saddened to see that creating a giant lever in DC that has the power to impact every school in the nation leads directly to a vicious, snarling political war over education policy, such that education can’t be discussed and debated dispassionately because culturally aliented partisans who don’t trust each other are all too busy trying to be the first person to seize the lever.
Surely no one could have predicted this unforeseeable outcome! Oh, wait.
National control over curriculum creates a single lever you can pull to move every school in America. Would conservatives trust progressives, and would progressives trust conservatives, not to try to seize control of that lever to inculcate their religious and moral views among the nation’s youth? And if you don’t trust the other side not to try to seize the lever, is there any reasonable alternative to trying to seize it first?
And this would not be just a single conflict that would happen and then be over. Like the Golden Apple or the One Ring, national curriculum and testing will continuously generate fresh hostility and cultural warfare as long as they exist. And once you forge this ring, there’s no Mount Doom to drop it into.
See also. Plus Neal here. Not to mention Neal’s eternal platonic beauty queens.
The whole idea of “high standards” is now irreversibly associated with nationalization. Now that the standards people – most of them, anyway – have been foolish enough to start it, this war over nationalization is going to have to be fought to its conclusion before we can circle back and talk about “high standards” in any other context.
6 Comments | instructional reform, politics | Tagged: Greg Forster, national standards | Permalink
Posted by Greg Forster
You are currently browsing the archives for the instructional reform category.
