This isn't quite the same thing, but whenever we run a job search I try to go back and look at what happened to the finalists we didn't hire in past searches. If nothing else, this is a good exercise in humility: years ago the department unanimously rejected one candidate in large part because of doubts about research, and that person would now be one of our three most successful department members in scholarship if they'd been hired.
It's very nearly the same thing--as you say, a crucial exercise in humility. (Which in my mind is part of what assessment should be rather than an attempt to reach perfect via incremental adjustment.)
This isn't quite the same thing, but whenever we run a job search I try to go back and look at what happened to the finalists we didn't hire in past searches. If nothing else, this is a good exercise in humility: years ago the department unanimously rejected one candidate in large part because of doubts about research, and that person would now be one of our three most successful department members in scholarship if they'd been hired.
It's very nearly the same thing--as you say, a crucial exercise in humility. (Which in my mind is part of what assessment should be rather than an attempt to reach perfect via incremental adjustment.)
That's a great idea. I've never heard anyone propose that before!