Jay P. Greene's Blog

(Guest post by Reid Lyon)

“The implementation of Reading First has been a hot topic, especially since the release of the recent impact study. Has the implementation of the program been successful?  What were the most important issues that arose regarding the implementation of Reading First? What lessons have been learned?”

With respect to whether the implementation of the program has been successful, the short answer is it depends.  Amber Winkler, the research director for the Fordham Foundation, recently reported in the Gadfly that Reading First is “perhaps the best-implemented education program in federal history.”  This may be true in some states, but not in others.  Dr. Winkler highlights Georgia, Oregon, and California as states that got the implementation process right.

On the other hand, Texas and several other states made successful implementation unbearably difficult.  In Texas, for example, a process was initially in place to award districts Reading First funding if their proposals were rated above a particular score by the grant review group.  Proposals from districts scoring below a threshold value were to be resubmitted following technical assistance to equip the district with in depth understanding of Scientifically Based Reading Research (SBRR) and to incorporate those concepts in their resubmission along with assessment and accountability requirements.  For some reason, a decision was made to award all districts Reading First funding irrespective of the quality of their applications.  Moreover, eligible districts received Reading First funding before any technical assistance was in place and before any baseline reading assessments could be administered for program evaluation purposes.  Not good.

But there is a great deal more involved in the implementation of a complex program beyond ensuring the quality of grant applications, and few states and districts had their hands around all of the essential conditions that must be in place to embed and bring to scale an initiative as intricate as Reading First.  At first blush, it would seem that the existence of a converging body of evidence relevant to reading development, reading difficulties, and reading instruction would have facilitated implementation fidelity.  But a substantial research base, bipartisan political support, and hefty funding only go so far.  The devil is in the details, and strategies for implementation at the federal level and in many states either were not appraised of the details or simply felt, as in Texas, that the money had to flow immediately no matter what, reflecting education’s love affair with entitlement programs.

One detail that gets right in your face immediately when you are implementing a program as complex as Reading First is that you have to manage coordinated systems change at federal, state, district, school and classroom levels.  Complexity theorists like W.L. Miller and his team use the metaphor of a jazz band when discussing how individuals within an effective system perform their own tasks in concert with others in achieving a desired goal.  Each contributor is responsible for certain tasks, but always listening to the other members of the orchestra to determine how their own actions contribute to the whole. This is tough to do when the band members come from different generations and different musical perspectives.  Getting into the groove takes a good deal of practice and a willingness to expand one’s thinking.  These are not features that characterize public education.   Basically, any evaluation of a program like Reading First must drill down into how this coordination played out and how long it took.  

Another common-sense detail that was not planned for in the implementation schedule was the fact that many hard-working folks in schools and classrooms who have gotten used to doing things in a certain way were being asked to change their routines and to try something new. In some cases this meant a decision was made to stop certain programs and replace them with others that many were unfamiliar with. When this occurs, teachers and leaders must have confidence that what is being implemented provides advantages to students over existing practices – and, in many instances, the case for this was not made prior to implementation.  While this may seem a bit fluffy, it is important to understand that public-school educators are under a constant barrage of new magic bullets, fads, and aggressive textbook company representatives all selling materials “based on SBRR.”  Implementation experts will tell you that without teacher and leader buy-in, any program, no matter how effective, will not realize its potential.  Fortunately, the majority of those leading and teaching in Reading First districts and schools saw the clear advantages offered students by the program as they observed poor readers become good readers.  But this took a while. 

In my interactions with many Reading First programs over the past six years, I did notice some common conditions that were in place when implementation fidelity was strong.  In addition to the details noted above, strong implementers embraced data and accountability for results.  States like Alabama integrated robust professional development with continuous coaching and feedback for both teachers and leaders.  Instructional programs were selected not only on the basis of their alignment with SBRR but because they were practical, useful, and beneficial to students.  Teachers were treated as self-determined professionals and responded by taking ownership and responsibility for their parts in the jazz band.  And building-level leadership ensured that teachers and coaches had the necessary time to plan, review student data, and collaborate in differentiating instruction for individual students based on their performance data.

The lessons learned are many, but I can think of three big ones.  Let’s start with the way Congress and the feds typically expect complex programs to be in place, in full operation, as soon legislation is passed.  Reading First embodied so many new concepts and requirements that, in my view, the first year should have been spent in providing technical assistance and professional development to states and districts even prior to the submission of Reading First grants.  I can’t tell you the number of times I saw the thousand-yard stare following my mention of SBRR, progress monitoring, data-driven instruction, or comprehensive reading programs.  We are talking significant mismatch between the requirements of the legislation and the background knowledge of many grant applicants; not to mention that the grants were competitive – a novel concept in education formula funding.

And if you were on the ground during the first two years of the program, this is what you would typically observe:  Teachers were first learning to understand, administer and use the results of assessments to inform instruction.  As they were learning these new concepts, they were also taking part in state reading academies to learn more about  the foundation of SBRR (in 5 areas of reading in k-1, in 4 areas  of reading in 2-3). In addition, as they were learning and using new assessments and taking part in professional development academies and workshops, they were simultaneously learning how to use a new approach to instruction and how to integrate core program instruction with additional interventions when required to meet individual student needs. This was done at the same time they were learning about center activities, grouping students for instruction and aligning and using supported classroom libraries.

It is important to ask whether any program that has added this amount of new learning to a teacher’s other responsibilities – including going to IEP meetings, attending parent conferences, preparing for their instruction in math, social studies and science, serving on school-wide committees and a host of other tasks – could demonstrate substantial gains after three years.  Give me a break.  What is amazing is that despite this unbelievable load, Reading First coordinators, teachers and their leaders rose to the occasion and have done and are doing a superb job.

Lesson Number One: Take a year to develop the infrastructure essential for program implementation. 

Lesson Number Two: During this first year,  make sure that all involved at every level understand the essential conditions that have to be in place to coordinate and implement a massive and unique program and to anticipate the need to customize some of its features based on individual district and school characteristics.

Lesson Number Three – and this is for the Department of Education: The next time Congress gives you $25 million dollars a year for six years to carry out an ongoing evaluation of a program, for God’s sake design and implement the evaluation commensurate with the initiation of the program.  This was no time to carry out a delayed and abbreviated evaluation when the complexity and uniqueness of a program demanded comprehensive, continuous and systematic feedback to ensure improvements in implementation where needed.