Measures of Effective Teaching: A new look at how to evaluate teachers
August 14, 2013
ANN ARBOR – One of the most extensive collections of videos showing teachers at work in the classroom will be available this fall at the University of Michigan to help researchers learn more about how to evaluate good teaching and how to develop excellent teachers. The videos of 3,000 teacher volunteers from around the country are a crucial piece of the Bill and Melinda Gates Foundation-funded Measures of Effective Teaching (MET) project, a multi-year study that set out to determine how to identify and advance strong teaching.
MET used three kinds of measures to evaluate teaching—classroom observations, student surveys, and student achievement gains. Typically for classroom observations, one or two observers mark items on a checklist while watching a teacher at work. A key innovation of the MET project was to use video cameras in place of human observers; teachers set up the cameras and did the recording themselves.
Now U-M’s Inter-university Consortium for Political and Social Science Research (ICPSR), part of the Institute for Social Research (ISR), has compiled the videos and the quantitative research that accompanies them into a collection known as the MET Longitudinal Database. “The ability to gather huge quantities of observational data on classrooms was a real advance,” said Brian Rowan, the Burke A. Hinsdale Collegiate Professor of Education at U-M’s School of Education, professor of sociology, and research professor at ISR. “Now we have instruments that can show teachers, ‘Okay, this is where you need to improve.’”
Using videos allows for a richer approach to evaluating what teachers do, Rowan said. For example, researchers can “double code” a video after the fact, marking good and bad examples from different areas of teaching, such as activities and content, and then link it to other data such as student performance. “We think they’ll have a big impact on teacher education,” Rowan said.
Another major use of the videos will be to train observers in how to do effective evaluations. In fact, while the initial purpose of the videos was to evaluate teachers, researchers are already proposing a broad range of projects that will draw on the collection.
The MET Longitudinal Database will be kept in a virtual data enclave at ICPSR in order to protect the identities of participants, Rowan said. As such, researchers who want to work with the data will need special clearance. Ten early career researchers already have received access to the database—and $25,000 each—after winning a grants competition earlier this year sponsored by ISR, ICPSR, and the National Academy of Education. (See a list of the winners and their research topics.)
In addition to the original restricted research data, U-M will soon offer more publicly accessible videos. The Gates Foundation provided funding to follow 396 teachers—most of them participants in the original MET study—during the 2011-12 and 2012-13 school years. The videos in this MET-Extension study will be available in October through the School of Education’s Brandon Center Digital Archive. The student and teacher participants gave consent for the broader education community to view their classroom sessions. “We’re building a set of tools that people can use to search, clip, analyze, play back, and work in groups with the videos,” Rowan said.
Users can stream or use virtual clips of the MET-Extension videos, but can’t download them to their desktops. The videos aren’t scored with quality ratings, but are searchable by practices and content, according to Lesli Scott, U-M’s project director for MET-Extension activities.
Scott said there could be considerable demand for the videos, particularly among third party organizations that want to include them in professional development products or evaluation tools. More than 30 states now require annual teacher evaluations that include observations of teaching.