I completely agree that relying on VAM for teacher evaluation was a huge mistake. It was a very complicated, black-box model that very few people truly understood, and, as you mention, it could only be used in certain content areas and grade levels. It also failed to provide teachers with any useful data to act upon in terms of improving their practice. I also think some of the researchers using VAM made some questionable interpretations of their results or failed to consider the limitations of the data. For instance, a consensus arose among some researchers and reformers that teacher effectiveness plateaus very quickly and almost all of the improvement in teachers' ability to raise test scores happens in the first two years. Thus, perhaps policy should focus on weeding out ineffective novice teachers (and perhaps even loosening entry requirements to increase the supply, as long as you used VAM to weed out the ineffective ones and keep the effective ones). What researchers didn't consider enough was that their data reflected the existing (and historical) quality of professional development and support for teachers and teaching in schools and districts. If the average quality of PD was pretty bad (which is often was) and most schools and districts weren't organized to adequately support and develop novice teachers (which they mostly weren't), then of course many novices' teaching skills would plateau early after all the benefits of individual trial and error was exhausted. And this would be reflected in the data researchers analyzed. What researchers and reformers failed to consider was the role organizational context played in teacher's student outcomes, often attributing VAM scores solely to individual ability. And years later, John Papay and Matt Kraft at Brown have shown that this consensus was off. Teachers can continue to improve for many years, and this improvement varies according to how supportive their school contexts are. I use this extended example to make the larger point that while reformers often based their proposals on research, they sometimes did not think critically enough about the findings (or their assumptions) and were quick to brush off many of the issues and concerns raised by teachers and other practitioners. This idea that teachers only improve in the first two years or so did not pass the smell test for anyone who had ever taught.
Interesting series, but one that misses the main issue: Whether the purpose of schools is to help children learn or to provide teachers secure, reasonably well-paid jobs. Firing incompetent teachers may help children, but it hurts the people who are fired and scares many others. After peak-COVID, K-12 schools received more than one hundred billion dollars from the federal government, ostensibly to fight learning loss and help kids overcome mental health challenges. That money has done nothing to improve either; instead, it was used to increase staffing, raise salaries, and upgrade facilities. The rhetoric is all about saving the children, but following the money reveals the actual priorities.
I worked with a charter school during this time that took the multiple measures approach as recommended by Gates. There was a lot of garbly gook math that took a lot of time and money to execute. I just kept thinking that the time and money it takes to get to this evaluation number is costly and we could just put it into teacher paychecks instead of going through all these hoops so we can pay teachers more. This organization was not trying to solve the problem of identifying and dismissing low performers, but rather how can we create a fair pathway that leads to higher pay and could drive teacher retention.
Thanks for a great article, and a great series! I learned a lot.
I don't dispute anything that you've said here, and of course it all makes a great deal of sense. But I wanted to add one small comment on the issue of why principals are reluctant to dismiss low-performing teachers. It's hard to recruit and hire teachers! There's not a huge supply of these people, and the hiring process is arduous. In the end, there's no way to know if someone is an effective teacher until they get into the classroom. So it's easy to stand on the sidelines and say, "You should fire bad teachers", but in practice it's a much more complicated and difficult undertaking.
Great article overall. The question I've never seen or heard an answer to is how much training in the process do evaluators receive. I've never had an insightful evaluation from the actual process, which is disheartening. But that didn't mean I wasn't eager to learn and improve my teaching, or that my informal discussions with administrators didn't provide valuable input into my teaching.
Evaluation, even in systems that embrace it, is distinctly separate from the process of improving practice. The learning curve for new teachers is long and slow - three years at the very least - but the learning by no means stops at that point. The constant change in instruction and curriculum alone created necessary new learning experiences for veterans. Teaching is the only profession where it is assumed that those with years of experience are worse than a totally inexperienced young person. Why is that?
In my experience veteran teachers aren't intimidated by the process - not sure where that came from. In this day and age, admins have minimal classroom, if any, and are not in the least intimidating.
People need to move on from COVID. It happened. No one knew what was going on - we forget that four years ago it was a novel virus with no vaccine, no medications. We have no idea what would have happened if large public schools were allowed to remain open throughout 2020. Maybe we would have had a mass exodus of educators. Who knows? What we do know is that we blew the reentry to in person schooling. Rather than taking care of the social/emotional losses, we dove right into "getting students caught up" and have paid for it dearly since.
As for student absenteeism, whose responsibility is it to get kids to school? Why are parents allowing their children to stay home? I've read many articles about it, and in not one does the author question why a parent would allow their kids to stay home from school unless they were sick. While it's a bigger problem in low income areas, it's a problem in suburban communities as well. I have adult children and the idea that children get to stay home when they're not sick is incomprehensible to me.
a refreshingly honest self-assessment that basically all of the pre-conceived notions of the 'reformers' were incorrect...true enough...one directive that could and SHOULD have risen from the ashes, however, was the movement to establish a Common Core Standard across subjects - english, math, science, social studies, etc. for all 50 states...with enough philanthropic dollars / Dept of Ed partnership this could have survived.... as a result, the real loser is America...the winner is China and the like... signed, Ron Tupa (former HS Social Studies teacher / DFER Director of State Legislatures and Vice-Chairman, CO Senate Education Committee)
There were Common Core standards for English and math, and the majority of states adopted these (with no observable effects). While there were no standards for "science, social science, etc." under the Common Core brand, there were in fact other standards that were developed for these subjects e.g. the Next Generation Science Standards (NGSS). So I don't really understand what mean when you say "one directive that could and SHOULD have risen from the ashes, however, was the movement to establish a Common Core Standard across subjects - english, math, science, social studies, etc. for all 50 states". That did happen!
agreed...I was referring to the fact that the effort has largely fallen by the wayside along with Teacher Evaluations specifically and Ed Reform in general...it would have taken years of funding and political commitment by Ds and Rs alike to see measurable gains in the Common Core standards across states...since the national effort is absent, states will scrap CC if they haven't already (do all states have NGSS?) and just return to their own standards...bringing us right back to square one as a nation.
I completely agree that relying on VAM for teacher evaluation was a huge mistake. It was a very complicated, black-box model that very few people truly understood, and, as you mention, it could only be used in certain content areas and grade levels. It also failed to provide teachers with any useful data to act upon in terms of improving their practice. I also think some of the researchers using VAM made some questionable interpretations of their results or failed to consider the limitations of the data. For instance, a consensus arose among some researchers and reformers that teacher effectiveness plateaus very quickly and almost all of the improvement in teachers' ability to raise test scores happens in the first two years. Thus, perhaps policy should focus on weeding out ineffective novice teachers (and perhaps even loosening entry requirements to increase the supply, as long as you used VAM to weed out the ineffective ones and keep the effective ones). What researchers didn't consider enough was that their data reflected the existing (and historical) quality of professional development and support for teachers and teaching in schools and districts. If the average quality of PD was pretty bad (which is often was) and most schools and districts weren't organized to adequately support and develop novice teachers (which they mostly weren't), then of course many novices' teaching skills would plateau early after all the benefits of individual trial and error was exhausted. And this would be reflected in the data researchers analyzed. What researchers and reformers failed to consider was the role organizational context played in teacher's student outcomes, often attributing VAM scores solely to individual ability. And years later, John Papay and Matt Kraft at Brown have shown that this consensus was off. Teachers can continue to improve for many years, and this improvement varies according to how supportive their school contexts are. I use this extended example to make the larger point that while reformers often based their proposals on research, they sometimes did not think critically enough about the findings (or their assumptions) and were quick to brush off many of the issues and concerns raised by teachers and other practitioners. This idea that teachers only improve in the first two years or so did not pass the smell test for anyone who had ever taught.
Interesting series, but one that misses the main issue: Whether the purpose of schools is to help children learn or to provide teachers secure, reasonably well-paid jobs. Firing incompetent teachers may help children, but it hurts the people who are fired and scares many others. After peak-COVID, K-12 schools received more than one hundred billion dollars from the federal government, ostensibly to fight learning loss and help kids overcome mental health challenges. That money has done nothing to improve either; instead, it was used to increase staffing, raise salaries, and upgrade facilities. The rhetoric is all about saving the children, but following the money reveals the actual priorities.
I think one of the reasons TN was successful is that they had had an approach to value-added data for years: https://www.nytimes.com/2017/05/19/upshot/the-little-known-statistician-who-transformed-education.html So that was not a hard shift for folks to wrap their heads around.
I worked with a charter school during this time that took the multiple measures approach as recommended by Gates. There was a lot of garbly gook math that took a lot of time and money to execute. I just kept thinking that the time and money it takes to get to this evaluation number is costly and we could just put it into teacher paychecks instead of going through all these hoops so we can pay teachers more. This organization was not trying to solve the problem of identifying and dismissing low performers, but rather how can we create a fair pathway that leads to higher pay and could drive teacher retention.
Thanks for a great article, and a great series! I learned a lot.
I don't dispute anything that you've said here, and of course it all makes a great deal of sense. But I wanted to add one small comment on the issue of why principals are reluctant to dismiss low-performing teachers. It's hard to recruit and hire teachers! There's not a huge supply of these people, and the hiring process is arduous. In the end, there's no way to know if someone is an effective teacher until they get into the classroom. So it's easy to stand on the sidelines and say, "You should fire bad teachers", but in practice it's a much more complicated and difficult undertaking.
Great article overall. The question I've never seen or heard an answer to is how much training in the process do evaluators receive. I've never had an insightful evaluation from the actual process, which is disheartening. But that didn't mean I wasn't eager to learn and improve my teaching, or that my informal discussions with administrators didn't provide valuable input into my teaching.
Evaluation, even in systems that embrace it, is distinctly separate from the process of improving practice. The learning curve for new teachers is long and slow - three years at the very least - but the learning by no means stops at that point. The constant change in instruction and curriculum alone created necessary new learning experiences for veterans. Teaching is the only profession where it is assumed that those with years of experience are worse than a totally inexperienced young person. Why is that?
In my experience veteran teachers aren't intimidated by the process - not sure where that came from. In this day and age, admins have minimal classroom, if any, and are not in the least intimidating.
People need to move on from COVID. It happened. No one knew what was going on - we forget that four years ago it was a novel virus with no vaccine, no medications. We have no idea what would have happened if large public schools were allowed to remain open throughout 2020. Maybe we would have had a mass exodus of educators. Who knows? What we do know is that we blew the reentry to in person schooling. Rather than taking care of the social/emotional losses, we dove right into "getting students caught up" and have paid for it dearly since.
As for student absenteeism, whose responsibility is it to get kids to school? Why are parents allowing their children to stay home? I've read many articles about it, and in not one does the author question why a parent would allow their kids to stay home from school unless they were sick. While it's a bigger problem in low income areas, it's a problem in suburban communities as well. I have adult children and the idea that children get to stay home when they're not sick is incomprehensible to me.
a refreshingly honest self-assessment that basically all of the pre-conceived notions of the 'reformers' were incorrect...true enough...one directive that could and SHOULD have risen from the ashes, however, was the movement to establish a Common Core Standard across subjects - english, math, science, social studies, etc. for all 50 states...with enough philanthropic dollars / Dept of Ed partnership this could have survived.... as a result, the real loser is America...the winner is China and the like... signed, Ron Tupa (former HS Social Studies teacher / DFER Director of State Legislatures and Vice-Chairman, CO Senate Education Committee)
There were Common Core standards for English and math, and the majority of states adopted these (with no observable effects). While there were no standards for "science, social science, etc." under the Common Core brand, there were in fact other standards that were developed for these subjects e.g. the Next Generation Science Standards (NGSS). So I don't really understand what mean when you say "one directive that could and SHOULD have risen from the ashes, however, was the movement to establish a Common Core Standard across subjects - english, math, science, social studies, etc. for all 50 states". That did happen!
agreed...I was referring to the fact that the effort has largely fallen by the wayside along with Teacher Evaluations specifically and Ed Reform in general...it would have taken years of funding and political commitment by Ds and Rs alike to see measurable gains in the Common Core standards across states...since the national effort is absent, states will scrap CC if they haven't already (do all states have NGSS?) and just return to their own standards...bringing us right back to square one as a nation.