43

In engineering fields it can be quite difficult to come up with good, consistent questions that enable an interviewer to consistently get a good overview of what a candidate is capable of. Unfortunately, in these fields, recruiters are very desperate to get placements because it can be quite lucrative, and so they do whatever they can to give their candidates an edge - including quizzing them on the questions that they were asked so that they can prime their candidates for success at answering the questions. The end result is that our questions become useless, as if a candidate already knows the answer (or has a chance to research it ahead of time) we can't get any data about their actual ability to solve problems themselves.

Short of having to come up with new questions for every candidate (bad because it increases the interviewing workload and makes it difficult to make objective comparisons) or asking HR to bar candidates who come from these bad recruiters (they are unwilling to do that), how can we avoid having recruiters "poisoning the well" in this way?

EDITED TO ADD: The questions we ask are not simple "memorization" or "trivia" questions. They are problem-solving and high-level systems design. However, this question still applies to fields where memorization/trivia questions are an important part of the screening process, and the nature of the problem itself is not the part of this situation that needs to be changed in any case.

Also, the recruiting firms are hired by us to find candidates - we're the ones paying their hiring bonus, not the candidates. Thus, they really should not be sending us low-quality candidates that they have prepped to seem like high-quality candidates.

Improve this question
12
  • 27
    An interview should not just be questions. It should be a conversation, using some questions to start that conversation.
    – Ed Heal
    Commented Oct 27, 2015 at 19:07
  • 4
    @EdHeal The OP is completely valid/on-target. If you want to go create a pool of dozens of distinct questions that is admirable. But unlikely. So for someone who does not have infinite time on their hands -and who justifiably places heavy emphasis on coding ability - this is real conundrum. A comparison would be if you were the TA for a class and needed to generate many sets of quizzes because the answers keep getting leaked. Not optimal/fair use of your time and effort. In addition it does detract from apples to apples comparisons.
    – WestCoastProjects
    Commented Oct 28, 2015 at 0:11
  • 2
    @EdHeal You're assuming a lot about our interview process.
    – fluffy
    Commented Oct 28, 2015 at 1:25
  • 5
    I think the problem is with the recruiters your company is using.
    – DA.
    Commented Oct 28, 2015 at 4:46
  • 2
    @Brandin Often the recruiters will speak to the applicants afterwards (at least that is what i encountered being an appcilant brought together with a company through a recruiter) and ask them what where the topics of the job interview, what questions where asked and if there was anything said about wages or similar.
    – Aron_dc
    Commented Oct 28, 2015 at 9:32

6 Answers 6

Reset to default
46

What questions are you asking?

I'm assuming you aren't asking trivia questions but have more meaningful questions.

If you have design types of questions, you can switch small details and really trip someone who doesn't actually know what they are doing up. Changing small details of your interview questions can really prove a few things:

  • Non-qualified candidates who "studied for the test" will fail miserably
  • Qualified candidates who know the material will easily be able to adapt and thus prove themselves

If you don't want to do this, ask lots of "why" questions. "Why are you doing X?" "What about Y?" as followups. Good candidates will be able to answer, or at least discuss intelligently, regardless of preparation - bad candidates won't.

... have you talked to the recruiter?

You may also want to ask your recruiter about this and see why they are doing it. That may be not aware it is causing you problems. You may quickly resolve this by a short conversation.

Otherwise, if they tell you they aren't/refuse/whatever, get better/different recruiters. It sounds like you are working with bottom of the barrel recruiters.

Also consider the possibility your questions are posted online (and not from the recruiter). I'd suggest first looking into this before blaming your recruiter. If you work for an even remotely high profile company chances are questions are posted all over the Internet.

You can also consider including NDA type of agreements in your interview process. If you have a dedicated HR/legal department approach them about this issue.

Improve this answer
4
  • 13
    I disagree with the "bottom of the barrel" remark. A recruiters' job is not to make sure the best candidate gets your job, it is to make sure their candidates get the job. That's their job. Just as a lawyer's job isn't to see justice done, it's to give their clients the best defense possible.
    – Francine DeGrood Taylor
    Commented Oct 27, 2015 at 18:06
  • 10
    @FrancineDeGroodTaylor any recruiter who blatantly copies questions and gives them to candidates without asking/telling the company is... not really doing a great job of providing a quality service to fluffy's company.
    – enderland
    Commented Oct 27, 2015 at 18:10
  • 8
    There could be any number of reasons for why they provide questions to their candidates but I don't see how that particular discussion is relevant to this question and Enderland's excellent answer. My take on it is that if you have good and useful questions and a good interviewer it won't matter that the questions are publicly available.
    – Lilienthal ♦
    Commented Oct 27, 2015 at 18:25
  • 2
    @FrancineDeGroodTaylor - I strongly disagree. A bad recruiter is there to collect their commission as fast as possible. A good recruiter provides a service. The fact that people even consider the former, however, is a sign of how truly woeful the recruitment industry is. Having encountered both good and bad recruiters, the difference is stark.
    – Jon Story
    Commented Oct 29, 2015 at 16:11
19

Instead of coming up with different questions for every candidate, come up with questions that are going to be different for every candidate. The best interviews I ever had involved an open ended project "Show me how you would design a game of monopoly" "write a method to check to see if a string is a palindrome" "I'm a client who wants you to design a system for them...ask me questions and figure out what I want", etc.

You don't have to do a lot of footwork to simply swap between monopoly and solitaire (or whatever the engineering equivalent would be), and what kind of prep work are they going to do when the test is basically "show me how you go about doing engineering work"?

An example might be to come up with a list of issues that you've run into in the job that you will be asking him to do. Ask him how he would go about troubleshooting and solving the problem. Listen to his answer and judge him on it, but don't tell him what the "real" answer was. So what if the candidate might have a little extra time to research the problem...that's more realistic than expecting him to know it on the spot. Real engineers solving problems have the same time and resources that your candidates will have if they have advanced warning of the nature of the problems.

And if you really want to level the playing field, pass out a list of real world problems to the candidates a day or two beforehand. After all, what you really should be testing is their ability to use the resources available to solve problems, not their ability to have facts memorized.

Improve this answer
1
  • The problem with switching out different scenarios in the same general theme is you won't be able to reuse experience about how hard the question is, where the common misunderstandings are and how to avoid them, where the hidden pitfalls are (and whether the interview is best served by steering towards or away from them). You can survive without that stuff, but it makes your job much harder than it needs to be.
    – Ben Millwood
    Commented Oct 28, 2015 at 14:54
15

Start asking questions the recruiters can't answer. Otherwise, the questions are just like Trivial Pursuit or Let's See If You Know What I Know sort of games.

Questions based on experience or require the ability to apply knowledge are going to be difficult for recruiters to help with too much prep. There's nothing wrong with someone preparing themselves anyway.

You have to decide the importance of memorization in your field. Maybe you should have the candidates do a small task that is typical of your projects to demonstrate they can do what the job requires.

Improve this answer
9
  • 21
    +1 I really dislike being asked canned questions that have a researchable answer. It's objective, but it measures the wrong things. Ask questions that don't have a specifically correct answer. Ask candidates to describe a problem they ran into and how they solved it. Questions like that can be prepared for, but can't easily be cheated on if the folks interviewing know the area they're asking about and ask follow up questions based on what has been said.
    – ColleenV
    Commented Oct 27, 2015 at 17:32
  • 2
    This is also a much better approximation of the work they will do for you. After all, when was the last time you or one of your developers actually coded an insertion sort? :)
    – cdkMoose
    Commented Oct 27, 2015 at 17:34
  • @ColleenV, you're absolutely right. That's not an interview, that's Trivial Pursuit. Hate those.
    – Nolo Problemo
    Commented Oct 27, 2015 at 19:08
  • 3
    We are most definitely not asking memorization-type or trivia questions. We are asking high-level problem-solving questions that one isn't likely to have come across before. And of course we try to adapt the question based on how the candidate is doing. But if someone has a week to prepare for every aspect of a question and memorize existing solutions to it, that highly dilutes our ability to differentiate good candidates from bad.
    – fluffy
    Commented Oct 28, 2015 at 1:28
  • 2
    I also hate interviews that put you on the spot like that. Problem solving or technical abilities are better assessed through testing or some other means in addition to interview. An interview is just a chat to get to know the candidate personally and see how they behave/work.
    – Luke
    Commented Oct 28, 2015 at 12:13
5

Make sure your contract with the recruiters states that the bonus is only paid if the person stays in your company for, say, 3 months. If a candidate comes through your screening process by memorising questions he/she will probably not be with you for long...

Improve this answer
1
  • This has been part of the contract with recruiters everywhere I've ever worked (sometimes on a sliding scale). UK based, for what it's worth.
    – Philip Kendall
    Commented Oct 28, 2015 at 19:45
2

The other answers give short shrift to the severity of the issue presented by the OP. The OP is valid/on-target. If you want to go create a pool of dozens of distinct questions that is admirable. For an interviewer who does not have infinite time on their hands -and who justifiably places heavy emphasis on coding ability - this is real conundrum.

Given all this - if I were the interviewer I would put a significantly higher bar on those candidates coming from that particular agency. The actual questions would likely be more off the cuff: in any case they are going to be "leaked" back to their agency so they do not deserve as much attention.

You can probably discern in any case if the candidates had heard the questions before.

Improve this answer
1
  • 3
    The other answers are fine. None of them suggest creating pools of dozens of answers; they all suggest either modifying details in the questions or using more open-ended questions that require more than a memorised answer. Trying to come up with an off-the-cuff question for a candidate you suspect may have been fed the answer to a previous question, is much more difficult to do consistently than preparing a good question ahead of time.
    – Logan Pickup
    Commented Oct 28, 2015 at 1:23
0

Rather than asking them to come up with a design (which is the kind of thing you can learn for), let them look into someone else's existing design and perform a peer review of it. That's a very different skill, a very important one and you can just pull up the last few commits made to Github on some public project and review that with the candidate. There's literally no way to prepare for a code review of a piece of code that didn't exist yet when you started the interview.

(This of course requires that the interviewer be at least at capable as the interviewee, because he'll also have to be able to analyse an unknown codebase in fairly short time.)

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .