When asked publicly or privately about high stakes assessments for teachers and schools, we always say the same thing: don’t go there. Using value-added models based of student test scores to reward or punish teachers misdiagnoses educator motivation, guides educators away from good assessment practices, and unnecessarily exposes them to technical and human testing uncertainties….
We are pleased to deepen our work of evaluating educational programs serving highly mobile students through three new Department of Defense Education Activity (DoDEA) MCASP grants: to Hillsborough County Public Schools, Socorro Independent School District and Fairfax County Public Schools. These new projects extend our prior work on behalf of highly mobile students through the evaluation…
Our clients are by and large genuine change-makers, motivated to measure and achieve the positive outcomes they seek. And one of our most important jobs is helping them develop and use appropriate data to enhance discovery, analysis, insight and direction. But client commitment and our professional responsibility don’t always avoid some common data collection pitfalls….
Our past two posts covered both the “why” of measuring implementation and some of the common challenges to doing so. In this third and final post, we’ll look at what is most useful to measure. Implementation measures are particular to each program and should take into account the specific actions expected of program participants: who…
In our last post, we shared four reasons why educators should be measuring implementation: here we’ll look at four common challenges to strong implementation measurement. 1. Differential definitions. What happens when different units of your program operate with different working definitions of a measure? Take tutoring, for example, in a multi-site program, where each site…
Understanding implementation is critical to both program improvement and program evaluation. But measuring implementation is typically undervalued and often overlooked. This post is one of three in a series that focuses on measuring implementation when evaluating educational programs. “Fidelity of implementation” ranks next to “scientifically based research” on our list of terms thrown about casually,…
The Department of Defense recently announced the 2016 round of Grants to Military-Connected Local Educational Agencies for Academic and Support Programs (MCASP), designed to support military-connected schools. These have been important programs for schools that serve military families and students, and our evaluations of MCASP programs have shown them to be effective. If your school or…
That women are traditionally underrepresented in technology-related careers will surprise no one. It is something we’ve seen in our work, our research, and with our own children. In the 2015 ARS evaluation of STEM, Inc., a coding and entrepreneurship project designed for middle school students, only 32% of participants were girls and they were almost half as…
Data mapping is about more than key points; it’s about the many sources of data that can help predict future outcomes. For a project in Clark County Public Schools, we sought a way to show how various data sources across the district could be useful in informing progress toward their strategic objective of producing graduates who are “Ready by Exit.”…
Together with Drs. Raymond Barclay and Barbara Weschke, Managing Director Kirk Vandersall recently released results from an Arroyo Research Services study of online teacher education in the Journal of Asynchronous Learning Networks published by the Online Learning Consortium (formerly the Sloan Consortium). The study presents findings from an investigation of the impact of teachers who graduated from a…