When it comes to making hiring decisions, relying on intuition can be tricky — and sometimes flat-out wrong, according to Michael Rosenbaum.
He has created two businesses from what started as a research project for a fellowship at Harvard, using algorithms and statistical models to predict how successful a person might be in a specific role or at a specific institution or company. He also worked at the White House and State Department as an economist before applying his methods to the technology industry with his company Catalyst IT Services.
Now, at Pegged Software, Rosenbaum is expanding to health care organizations and, eventually, beyond. The Baltimore company recently drew a $7.5 million investment as it processes 3 million job applications each year for 119 institutions.
The company says it has cut its clients' employee turnover by at least half, saving them the time and hassle of repeatedly going through the hiring process. It does that with the help of big data, Rosenbaum told The Baltimore Sun.
What are the biggest things hiring managers look at that actually don't matter?
Interestingly, one of the things we find is that frequently there's no such thing as a "good fill-in-the-blank." There's such a thing as a good nurse in a particular role at a particular hospital, or a good software engineer for a particular engagement in a particular organization. But the person who is successful in one place may not be successful somewhere else.
That isn't how we as hiring managers approach the labor market. "This person looks like they've done these things, which means they'll be good." Even if you have the exact same job description in different places that might even be right next to each other, like two hospitals right next to each other, something predicting success in one job might predict failure in the job across the street. That's just not the way we think about things. I think, as an overarching theme, that's probably the most significant thing we see that hiring managers do, whether consciously or subconsciously.
How did you figure out what matters and what doesn't?
Originally it was trial and error. I'd love to say it was more systematic than that. We said, "Let's collect a bunch of information on an employment outcome" — say, in a hospital someone gets promoted quickly or the people they serve tend to have better health care outcomes. And then you just experiment with stuff. What data can we look at on someone? What data can we find on someone publicly while they're applying for a job?
The idea of collecting keystroke information just came from an experiment. We said, "I wonder what would happen if we collected keystroke information when someone was applying for a job," and we found it told us things. Looking at the text in a free-form answer in a job application, basically what we've done is say, "OK, I think that the use of X word in this answer might tell us something," and then we test that. Sometimes it tells us something and sometimes it doesn't. Sometimes we think it's going to tell us someone's going to be successful, but it tells us someone is more likely to fail.
Our gut intuitions about what's important in a job sometimes aren't right. The process of challenging our assumptions and testing them against the actual data is probably the biggest lesson we've learned.
What do you find leads to turnover?
There are a lot of bad hires. When you look at total turnover in health care, a very large portion of the total turnover in the hospital happens in the first year someone is employed. That's just a bad hiring decision. When you look at why, I have some theories on it, but I think often people look at a resume and say, "Well this person has a degree in what I'm looking for and they've worked for X years in that role. I'd rather [hire] the person with the degree and the person with more years of experience for my job." But it turns out the answer is more nuanced than that. In some jobs, more years of experience can be better, but it may be that 12 years is no better than five, and yet when as individuals we look at resumes, we bring our own subjective perspective to that, and it's hard as an individual to look at that systematically.
We're looking at 3 million job applications per year, so it's easier. As an individual looking at many fewer applications, it's hard sometimes to pull at those threads. As a result, we make hiring decisions based on very small amounts of information. It's very imperfect.
The other thing people do is they like to hire people like themselves. It turns out frequently that if as a manager you hire people who are just like yourself, you end up succeeding less frequently a lot of the time. When you ask me why I think that is, I think it's probably because you end up a with a team with too much homogeneity on it. Increasing the diversity of the team increases the odds the whole team succeeds.
The software is deployed over the Internet into the HR departments of health care organizations. Then whenever someone applies for a job, Pegged will collect a whole bunch of information on that applicant. We'll generate a prediction using an algorithm of whether or not that applicant is likely to succeed in the job they applied for. It lets the hospital do a search through a very large pool and hand over to a recruiter the small number of people they think are likely to succeed. For those who are not likely to succeed, it will look at whether those applicants are likely to succeed in another job in the organization, even if it's not what they applied for. It essentially allows hospitals to hire people into jobs they didn't apply for.
What was it like going from academia and the White House to running your own businesses?
It was a very big adjustment. When you pick up the phone and you say, "Hi, my name is Michael Rosenbaum. I'm an Olin Fellow at Harvard," people take your call. When you say, "Hi, I'm Michael Rosenbaum, CEO at Pegged Software," people say, "Who? What?"
Also, it's really difficult to do something that's totally different in a big organization without getting in trouble. One of the things I loved very quickly about doing the work I do is being able to challenge every assumption I have and to create an organization where we challenge assumptions. That's just harder to do in a bigger organization. In some ways it was awesome; in some ways it was a difficult transition.