One of my favorite things about managing a team was interviewing candidates. Anyone who ever interviewed for a job for any of my teams got to know my penchant for asking strange questions and doing strange things - the main purpose was always to overcome the fact that most interviews are a complete and total sham.
For most interviews, the decision on whether to make the hire or not is made within the first 30 seconds of the interview, and has absolutely nothing to do with any of the rational parts of the decision-making process. While we may think that we're completely rational, what research has shown is that the interview process is mostly a matter of the interviewer having a first impression, and then spending however long the interview lasts gathering evidence to back up that first impression.
I remember a particular story from my past where this became all too clear to me. Early in my career, I was on a team that was interviewing for a trainer who would be responsible for all customer-facing training. We saw a few mediocre candidates, and then we interviewed a relatively attractive woman. The woman had no training experience, and was absolutely terrible in the group interview. When the interview ended, we all re-grouped around my then boss's office while he saw her out and started discussing. It was clear that we all thought she wasn't a fit - we all thought she was too much trying to rely on her looks and had very few skills.
That was when my boss returned, and started extolling her virtues. I argued with him for a few minutes, until he uttered the line that those who were on the team make jokes about to this day: "She will be our trainer." Suffice it to say, she didn't exactly set the world on fire with her performance after she was hired. And, after making some pretty big mistakes and generally being a disruption at times, she was eventually let go.
What was most interesting to me that day was that it was as though my boss (who was normally quite sharp and a decent senior manager) was in a completely different interview than the rest of us. Gladwell reported on this research in the New Yorker, and again in his book Blink. From the article:
"She took fifteen seconds of videotape showing the applicant as he or she knocks on the door, comes in, shakes the hand of the interviewer, sits down, and the interviewer welcomes the person," Bernieri explained. Then... Prickett got a series of strangers to rate the applicants based on the handshake clip, using the same criteria that the interviewers had used. Once more, against all expectations, the ratings were very similar to those of the interviewers. "On nine out of the eleven traits the applicants were being judged on, the observers significantly predicted the outcome of the interview," Bernieri says. "The strength of the correlations was extraordinary."
The difficulty with interviews is that we end up having our unconscious mind hijack the interview process - we end up in rapport with the person, and our unconscious tells us that we should like them and that they're worth hiring because of it. Or, we don't develop rapport (or even actively dislike the person) and choose not to hire them because of that.
Almost everyone who has ever interviewed for a job can think of a time when they were completely qualified and the opportunity was absolutely perfect, but they didn't get the job. And almost everyone who has ever interviewed someone has been hijacked by their unconscious this way.
The only way around this is to set up a process that ensures that your conscious and unconscious mind can work together on the process of interviewing.