Is Quantifying Teacher Performance Akin to Flipping A Coin?

Last week, on the way home from school, I tuned into a story on the radio titled: “Seattle Releases First Teacher Ratings Based on Student Performance.” Data is great, but if you paid attention to the elections a few weeks ago, there were two kinds of math going on. Nate Silver’s Five Thirty Eight blog predicted 50/50 states. Karl Rove’s analysis of the data had him flummoxed. The difference was that Rove was emotionally attached, was eager to win, and for some reason his analysis of the same polls was way off. Alternatively, Silver simply plugged numbers into complex algorithms.

Mathematicians have noted that test scores and teacher performance don’t necessarily have a strong correlation, yet an incredible weight and cost is attributed to these standardized tests. Math professor Johh Ewing says, “You might as well look at all the teachers and flip a coin and those that get heads, say, are good, and those that get tails are bad, and it’s not much different from using one year of growth to measure teachers,”

Ewings paper, “Mathematical Intimidation: Driven by Data,” Looks at the potential pitfall of trying to create Value Added Measures to teacher evaluation.

Like the election examples earlier, we often attach a lot of emotion to the data creating a lot of noise. This noise had the potential to lead to bias. When a teacher says, “But I’ve done this for 20 years. I know this works,” it is evident that experience plays an important role. But is there bias involved. During those 20 years, did that teacher ever once control the experiment by not utilizing a particular skill? If so was the result the same, better, worse. Without trying to control for various things, how does one really know if what you do works. Is it just a feeling or is it based on empirical data.

Finally, there are so many things that make a good teacher: relationships with students, high expectations, integrity, care, leadership, collaboration, etc. Yet all of these traits can’t be tested for.

Standardized test scores are a reality and here to stay. As long as graduate schools use test scores as a tool to help with admissions, and undergraduate schools do the same, high schools and middle schools won’t have much of a choice. Elementary schools just follow.

There’s a dark side to this. Children as early as Pre-K are getting tutored in test preparation. Like the qualities of teachers, students have many amazing strengths and skills. However just because they struggle with test taking, potential doors my be closed without even giving the child a chance to show the brilliance that lies within.

And what about those 21st Century Skills – Critical Thinking, Communication, Collaboration, Creativity, etc. Will teachers drop integrating teaching these skills in order to meet the demands of the test scores? I hope not.

How Do You Measure Success?

The London Olympic games are coming to a close, and I’ve noticed a few themes/issues throughout the games that seem to spill into the realm of education: 1) How do we measure success? 2) In 20 years, other sites may push twitter or Facebook aside, but I’m pretty confident social media is here to stay. How do we promote digital citizenry and prepare kids to use these tools productively and? 3) Privilege and equity – does every country have a fair shot at a sailing or equestrian medal? Does every child have access to a good education? 4) Standards: what are the standards for commentary on the Olympics? I know very little about gymnastics, but I don’t need someone to point out that a fall off an apparatus is not a good thing. Did the opening ceremony need a play-by-play? Can you imagine giving students the answers rather than providing opportunities to grapple with, discover, and construct their own knowledge? There are many more themes that have emerged from these games, but the first one I mentioned resonates with me the most. How do we measure our own success and the success of our students?

After Michael Phelp’s fourth place finish at his first event, the USA Today had a story titled: “Sluggish Michael Phelps is not swimmer we expected in London.” Since his first event, Phelps has become the most decorated Olympian in history, but I guess if you look just at the one fourth place finish, sluggish it must have been.

Why is it that some athletes cry for joy after winning a silver and other athletes are visibly disappointment, often with tears in their eyes for winning a silver medal.

The most emailed article in the New York Times over the past three days has been one titled: Raising Successful Children. It’s a parenting article about the importance of not over-parenting and allowing children to make mistakes and build resiliency on their way to success and confidence.

I’m not a parent, but I completely agree with the statement, “HANGING back and allowing children to make mistakes is one of the greatest challenges of parenting.” It’s a challenge of teaching as well. Not all failures are equal. They need to be ones that lead to growth. So what kind of mistakes should parents and teachers let kids make?

“In this gray area of just beyond the comfortable is where resilience is born.”

Perhaps being called ‘sluggish’ and then coming back to win 4 gold and 2 silver medals can qualify as a good measure of success.

I just came back from a great three day summer planning inservice with my colleagues where we spent a lot of time looking at and practicing how we assess and give feedback to our students and to each other. I wish us all a successful school year that can be measured by the risks we take ourselves in that gray area just beyond the comfortable and by the resilience we develop in our students. 

 

Am I An Effective Teacher?

There are many ways to evaluate the effectiveness of a teacher: observations, rating forms, informal drop ins, student work, student surveys, and student tests. Many different organizations keep looking for different ways to look at teacher improvement and thus school improvement.

Some ratings are tied to compensation, and others to tenure. Others seem to be done as a formality. Some are pass/fail while others use a rubric: exemplary, proficient, adequate, ineffective. Regardless of the system, for it to be effective, an action path for growth needs to take place. Even if you are an exemplary teacher, there’s always room for growth.

At the same time, positive feedback is only good if it is specific. “Good job!” doesn’t mean anything more than ‘you did well.’ What part specifically? The whole entire thing? Really? I had a good morning today as my head of school came in and gave me direct feedback with concrete examples. It was positive, but specific. Now I know and can think to myself, “That part was good, so Implement it next year. Ask yourself is there a way to make it better?”

I’m excited as our new school at more effective ways to evaluate our teachers. I’m hoping with the intent of developing growth mindsets. Perhaps if we look at all the data about the kinds of assessments we collect from children, the same kinds could apply to teachers.

There are so many ways to collect evidence that you’re reaching your goals. Student work, portfolios, student surveys, individual assessments, their daily writing, observation, questioning., etc. Wouldn’t it be nice to get past the old teacher observation evaluation model and incorporate elements of the assessments we place on kids. Observation and feedback should just be a piece of the pie.

What is Assessment Literacy?

“Assessment illiterates do not understand how to produce high-quality achievement data and do not evaluate critically the data they use.”

 

Richard Stiggins, whose been a educational leader in assessment research wrote that in 1995. He has spent over two decades combatting years of “assessment training neglect for teachers and administrators.” He coined the phrase ‘assessment literacy’ and urges us to use assessments in meaningful ways. A huge focus of his is assessment ‘for’ students rather than ‘of’ students.

When it comes to assessment literacy, there is so much to consider:

  • What exactly are you assessing? A product? A performance? Mastery of a standardized skill?
  • Does your assessment line up with what you’re teaching? This does’t mean teaching to the test, but does the scope and sequence of what students are learning align with your assessment?
  • Have you included your students in creating criteria for their assessments? Do you use rubrics? How much are teacher generated? How much are student generated?
  • How do you communicate these assessments to students? To parents? To other teachers? Do you do this through portfolios? Report Cards? Conferences?
  • What information are you getting from a standardized test? How are you using that information? Is this information used for student improvement? School improvement? Teacher improvement?
  • What does it mean to be 2 standard deviations above the mean? How valid is the assessment?

I’ve only just scratched the surface, but you can begin to see how complicated assessment for student learning can be. I used to consider myself literate in assessments knowing that it was something that would continue to evolve and require me to learn more about it. That is until now.

My students used to be given a standardized test in the fall of each year and we’d get results back in the winter. We could analyze the results, look for trends and gaps in the school as well as confirm any gaps there may be in student learning, and try to address them. While this isn’t bad, Stiggins noted that instructional decisions based on an assessment that happens once a year does not have the greatest impact on student learning. And what about the students? Were they being given this information as a tool to set new learning targets?

So this year, when our school decided to move to a computer adapted assessment that would be issued at least three times a year, I got excited. Not only would the assessment take less than 30 minutes (the old format took about 6 hours over the course of a week), but we’d get data back immediately. Unfortunately, my excitement has turned to frustration. Mostly because I can’t make heads or tails out of the data. I feel like I’ve become assessment illiterate, but I know that it’s not true.

If I’m going to give my students an assessment at least 3 times a year, I want to know how it aligns with our curriculum and what action my grade-level team can take immediately. Over the course of the year, sure we can use the data as we had previously, but in that case, why would we subject our students to it multiple times in a year. Saying that it gives kids a chance to practice filling in bubbles to prepare them for future standardized tests is an argument I never bought. It is clear that isn’t the case now as they presently ‘click’ their selection.

This time of year, we always engage the children in an author study unit. How great it would be if we could use data to fine-tune this unit and communicate this to our students. I’ll leave you with another Stiggins quote. This one from a more recent article (2009).

“Let me be clear about my mission here. The arguments I advance do not arise from a desire to end accountability- oriented standardized testing. Such tests do provide op- portunities for educators to reflect on what is and is not being achieved. If educators don’t take advantage of these opportunities, it is not the fault of the tests. I will suggest specific ways for users to take far greater advantage of standardized tests in the future. But for assessment to become truly useful, politicians, school leaders, and society in general must come to understand the gross insufficiency of these tests as a basis for assessment for school improvement. “

 

Elite New York “Pressure Cooker” Schools are Rethinking Homework

An article about homework in this weekend’s nytimes couldn’t have happened at a more appropriate time, as I continue to search and explore ways to make homework (now called “home learning” in our second grade classes) be meaningful.

Sending a worksheet home, so that it can be returned the next day for the sake of compliance is not the message I want students to get. If I have to assign it, and kids have to do it, it has to have purpose beyond that. Being prepared to share something in class with their peers is valuable.  So is practicing various skills as long as there is immediate and meaningful feedback. Unfortunately, over the course of a regular busy schoolday, one often doesn’t have a chance to check a child’s homework until after the end of the day, so by the time a child receives feedback, was the home learning task really that successful?

Towards the end of last year, I thought I’d give Kahn Academy a try, and while it worked for some, it didn’t achieve what I was hoping. What I did like was the immediate feedback kids were getting at home. Sal Kahn spoke at our regional conference this year and I was surprised how novel his ideas seemed to many teachers.  He isn’t the only one who’s been trying new things, but he’s been endorsed by Bill Gates and has also done a TED talk, so he’s definitely more visible.

So this year, my teaching partner suggested some other online tools which were more age appropriate than Kahn’s, covered multiple subjects so that kids could have some choice in their learning, used tech in a way that allowed for immediate feedback, and allowed us to still included elements that required kids to be prepared to share as well as take some responsibility to bring certain things back to school (even though it might not be daily).

Well, I wouldn’t call it a complete success after the first month. There were a lot of elements to consider, and some we have reconsidered.  Many things, however did work. There are elements that really seem to be doing what we hoped, and they just need to be revised and tweaked. In the classroom, my students have begun to start appreciating the idea of process and revision and not always about getting it right on the first try. It’s great that I can show kids that it is also how adults learn. We didn’t get it right on the first try, but we’ll see how the adjustments go, and report back. Thanks to all my students’ parents who provided excellent feedback in helping us refine it.

It’s nice, though, to know the most elite independent schools in New York (not that it should be a measure of anything) are also working on similar issues. We too, will be giving a “Home Learning” holiday on October 31st!

Got Character?

Cover of Today's New York Times Magazine

Today’s NYTimes Magazine is the Education Issue. Our Head of School forwarded one of the magazine’s featured articles to the faculty earlier this week: What if the Secret to Success is Failure? A Radical Re-thinking of how Students Should be Taught and Evaluated. It’s a thought provoking article, but if you’ve been following some of the changes in education over the past few years, it doesn’t seem all that radical.

Daniel Pink has explored zest, grit, and optimism in his work Drive along with empathy (social intelligence), play (curiosity) in his book A Whole New Mind.

Carol Dweck, in her book Mindsets discusses self-control as an important factor in developing growth mindsets.

Nel Noddings has been writing about the ethic of care for years.

I was able to catch a few of the TEDxLondon talks that were live-streamed this weekend, and there was definitely a call to spark curiosity in our students. Hopefully, the videos will air soon, but Ewan McIntosh posted the transcript of his talk about creating a generation of ‘problem finders’ on his blog. I encourage you to read his post.

Character Ed. isn’t new, but what I found compelling about the article was how they broke down the list of character traits into two categories ‘moral character’ and ‘performance character.’ I also liked how the article mentioned many of these character traits can backfire. “Too much grit…you start to lose your ability to have empathy for other people.”

I also liked the Head of Riverdale’s “philosophical issue with quantifying character.” It’s true that the last thing we need are people trying to game the system with test prep on character traits. Also, if too much of a certain trait can backfire, how would you measure what is best?

Another great question brought up in the article is: How do you teach these traits? I don’t know the answer, but it’s definitely one worth exploring. I know you can’t do it with carrots and sticks and you can’t do it simply by putting quotes around your school. You can start by modeling these traits (I’m 41 and I’m still learning how to grow some of these traits and moderate others), getting to know your students, and creating supportive relationships with their families. I suppose what’s radical is that more an more people and schools are thinking about these questions. It’s exciting to see some start to try new things.

I’m looking forward to hear what others at my school think, as our Mission and Values have both the moral and performance character traits we strive towards.

An App a Day: Stick Pick

There are many teacher tool apps out there, but not all of them work well. If you’re an elementary teacher, drawing sticks with kids’ names on them out of a can isn’t a new idea. It’s a great tool to randomize the way you select kids. If this app simply replaced this concept electronically, it wouldn’t be that innovative. The person who designed this app actually put some thought into it so it does much, much more.

There’s no point describing it here when their website does it so well.  It was developed by a 6th grade teacher. I can’t wait to use it with my new class this year.