Both the United States and the state of Maryland have made important shifts in how they conduct business for students with disabilities.
Until now, the U.S. Department of Education’s primary focus has been to determine whether states were meeting procedural requirements for students with disabilities. The requirements included timelines for evaluations, due process hearings and transitioning children into preschool services. While these compliance indicators remain important for federal funding, under a new framework known as Results-Driven Accountability (RDA), the Department now includes educational results for students with disabilities in making each state’s annual determination under the Individuals with Disabilities Education Act.
Following this requirement, the Maryland State Department of Education (MSDE) also is moving from accountability for compliance towards accountability for compliance plus results for children with disabilities. Notably, this shift is reflected in the Division of Special Education/Early Intervention Services strategic plan – Moving Maryland Forward – designed to narrow the gaps in school readiness, school achievement, and readiness for adult life after school.
To support both of these shifts, the MSDE applied for and won a U.S. Department of Education State Personnel Development Grant (SPDG) in 2012 to help address the national mandate through one of the goals of the State’s strategic plan:
By 2018, implementation of equitable services, universally designed instruction, evidence-based interventions, technology, and accessible materials will result in increased access to instruction based on Maryland College and Career-Ready Standards and improved academic achievement and functional outcomes.
With this grant, the MSDE, in partnership with the Center for Technology in Education at Johns Hopkins University, is working with Implementation Teams in Prince George’s and Charles County Public Schools to bring into play an evidence-based instructional delivery system. This system, with the creative title Buzzz – Be United in the Zone, is designed to improve mathematics outcomes for students with disabilities in elementary schools through:
- Universally designed instruction
- Structured cooperative learning
- Positive behavior strategies
- Formative assessment strategies
These are delivered through a unique Team-Based Cycle of Instruction, a six-step protocol that highly structures class time so teachers can better organize resources, keep momentum high, maintain a smooth lesson flow, and engage students, particularly those with special needs. The TBCI was developed by faculty at the Johns Hopkins University School of Education.
Implementation Teams Powered by Implementation Science
Research indicates that even when an evidence-based intervention with proven results is implemented in a school, it is not certain that the school will get improved student achievement results.
The key is support.
Research also shows that with Implementation Teams in place, 80% of interventions will have improved outcomes in as little as 3 years of implementation. By contrast, after 17 years of implementation only 14% of interventions will have improved outcomes without implementation teams.
This is why MSDE is working with local districts to form strong Implementation Teams and build members’ knowledge about implementation science or the ways to bring research findings into practice. The teams include:
- State Department experts in Special Education and Instruction;
- County Directors of Mathematics and Special Education;
- Principals, department chairs and teachers; and
- Experts from Johns Hopkins Center for Technology in Education
The Implementation Science frameworks they are learning about include:
- Usable interventions – sufficient detail about an innovation. With detail, you can train educators to implement it with fidelity, replicate it across multiple settings and measure the use of the innovation
- Implementation stages – steps along the path of implementation
- Implementation drivers and teams – common components of successful programs
- Improvement cycles – an iterative trial and learning approach that leads to higher standards
- Systems Coaching – providing coaching support with key stakeholders across a district’s systems
- TAP-IT – a decision support data system
.* (Fixsen, Blasé, Timbers, & Wolf, 2001; Balas & Boren, 2000; Green & Seifert, 2005; Saldana & Chamberlain, 2014
How can the Implementation Science frameworks help us get the desired results?
Through Implementation Science we understand that the implementation process has multiple stages demanding different activities. Since implementation is not an event, activities will occur in stages that overlap and that are revisited as necessary. We also need to pay attention to the implementation drivers. to ensure practitioners implement the selected intervention with fidelity. After we have selected and described the core components of a usable intervention like The Team Based Cycle of Instruction, Implementation Teams focus on supporting effective implementation and enabling contexts.
Effective implementation means that TBCI is implemented with correctly, so that students can experience it as intended. This is accomplished by implementation teams who ensure that practitioners have the support they need, such as training and job-embedded coaching, to implement TBCI. Implementation teams actively work to create positive environments for the new intervention through facilitative administration and/or systems intervention and also ensure that a decision support data system is in place. By following this process, local school system Implementation Team utilizes implementation science’s formula for success.
The Formula for Success (from the Active Implementation Hub)
As part of the BUZzzzInitiative, teachers and administrators also are learning how to review and analyze classroom data more effectively, so that they can plan interventions for students with disabilities.
Using the TAP-IT* process, (i.e., they tap into data), a team analyzes student and teacher data to plan an intervention for a student, implements the intervention, and then tracks its impact. The Implementation teams work with a specialized digital portfolio platform to collect and share data and documents, analyze data, make recommendations, track interventions, and receive coaching tips from district-level members.
The final implementation science framework, improvement cycles, uses the Plan-Do-Study-Act (PDSA) cycle to promote purposeful implementation of the selected intervention. PDSA is a strategy employed by implementation teams to help them identify any barriers to implementation and to engage in problem solving so that identified barriers can be alleviated. The Maryland team has expanded this traditional improvement cycle by integrating two frameworks from implementation science – the Implementation Stages and Implementation Teams – into a single process. This coherent process is called TAP-IT – Team, Analyze, Plan, Implement, and Track. Central to this process are the strategic protocols for building the team and actualizing the remaining TAP-IT stages.
An implementation team is needed to develop, oversee and support the change process. For this grant, Team refers to the Local school system Implementation team.
The Analyze step takes place whenever the team uses data to assess needs in relation to a problem of practice, examine possible solutions for that problem, and assess how proposed solutions might work.
The Plan step of TAP-IT aligns with the installation stage of implementation. During installation, the team plans to:
- acquire resources for tasks aligned with the new intervention;
- prepare the organization by making sure that structural supports (funding, human resource strategies, policies and procedure development and reporting frameworks for expected outcomes) are in place;
- select and plan for training staff in the new intervention; and
- develop an Implementation Action Plan which outlines the tasks, names of staff responsible for completing tasks, and a timeline for completion.
The Implement step of TAP-IT aligns with initial and full implementation stages. During initial implementation, data is collected and used to make program adjustments if needed. It is also time for the team to learn from mistakes and develop system solutions when appropriate rather than allowing problems to re-emerge and re-occur (State Implementation & Scaling-up of Evidence-based Practices). During this step, sources of implementation data are collected from regular assessment of fidelity of the intervention and monitoring of student learning.
The Track step involves the Implementation Teams meeting quarterly to track implementation progress by examining implementation data and student progress data. These TAP-IT meetings provide information about implementation barriers and successes that enable adjustments to policy, structure, procedure, and practice
Implementation Teams have a big job to do, so they too need a support system to ensure their success. According to Allison Metz from NIRN (2015), a systems coach, often working at the district level, can provide this support by helping to develop the capacity of the school district to effectively implement a program, practice, or approach to enhance student outcomes.
The coach provides support for organizational change related to choosing, implementing, sustaining, and improving evidence-based approaches to academic instruction and interventions that need to be addressed.*
Improving results for children with disabilities is a complex process because, in many cases, it requires the implementation of new interventions to get the desired result. The Maryland project is unique in that it not only provides teacher support for the implementation of an effective intervention – TBCI – it also provides district support to leaders to learn about implementation science, systems coaching, and TAP-IT so they have the capacity to sustain and scale-up once the project grant is completed.
(* Balse’, K. A., Fixsen, D. L., Sims, B. J., & Ward, C. S. Implementation Science: Changing Hearts, Minds, Behavior, and Systems to Improve Educational Outcomes: NIRN).
TAP IT In Action at Thomas Stone Elementary School
The Implementation Team at Thomas Stone Elementary School began the 2015-16 school year with informal walkthroughs of classrooms to gather data on each teacher’s level of implementation using a checklist of specific elements of effective instruction or “look fors”. Based on those results, and an additional 4 days of training, the team broke the teachers into three groups, each with a different implementation plan.
The team met in the first quarter to set team goals about the number of monthly classroom walkthroughs they would conduct to gather data on teacher implementation of the TBCI. Using the Scholastic Math Inventory Assessment (SMI), they set a student achievement goal:
all students would improve their score by at least 1 grade level by June 2016.
After each classroom evaluation, the team met to analyze the implementation data and determine the most important needs for professional development. These needs were addressed in weekly staff meetings and supported by instructional coaches in the building during instructional time. This allowed the team to address needs quickly and efficiently thereby leading to better results.
Each quarter, the team reviewed the SMI data to analyze student performance. Based on the annual goal and the number of students and rate of growth, instructional goals were set for the next quarter. The team then determined which interventions were working best with students and which alternative interventions might address challenges. Teachers received professional development on these the following quarter.
By using a cyclical approach, the implementation team and coaches were consistently reviewing data, making changes in strategies, and supporting teachers so that the intervention had the best chance to produce strong results.
The contents of this brief were developed under a grant from the US Department of Education, #H323A120010. However, those contents do not necessarily represent the policy of the US Department of Education, and you should not assume endorsement by the Federal Government. Project Officer, Tina Diamond.