Having been both the interviewer and the candidate in this kind of situation, this is really a big interviewer training failure.
The general way to handle this as an interviewer is really simple: acknowledge that the interviewee gave a good answer, but ask that for the purposes of evaluating their technical design skills that you'd like for them to design a new system/code a new implementation to solve this problem.
If the candidate isn't willing to suspend disbelief for the exercise, then you can consider that alongside all of the other signals your interviewer team gets about the candidate. I generally take it as a negative signal, not because I need conformance, but because I need someone who can work through honest technical disagreements.
As a candidate, what's worked for me before was to ask the interviewer if they'd prefer that I pretend ____ doesn't exist and come up with a new design, but it makes me question whether I want to join that team. IMO it's the systems design equivalent of the interviewer arguing with you about your valid algorithm because it's not the one the interviewer expects.
A good interviewer won’t be looking for a single solution to the problem. I’d expect them to entertain the Google Sheets answer - it’s good signal that the candidate will consider what already exists in the world. I’d rather extend the problem: the team is spending considerable time iterating with manual entry, what would you do?
"Well I would probably go home and work on my resume because that's a fool's errand."
I hate going to work and reinventing wheels all day because the company I work for thinks it's so special that every business function needs a 100% tailored solution to solved problems. I much prefer working somewhere that's able to tailor business processes to conform to existing standards.
I’ve interviewed a few hundred people. Probably approaching a thousand, if not already. An interview is a scenario, and if you aren’t willing to engage in the scenario that we all agreed to partake in, that’s a huge warning sign that you’re going to be difficult later down the line. The point of the question is to have something remotely understandable for both sides to talk about, that’s it.
I'd call it an interviewer failure, not an interviewee failure.
I absolutely want people I hire to be "difficult" when the moment calls for it. If the scenario is one where the right business/user choice is "let them keep using Google Sheets", then the answer I want is "Google Sheets seems fine to me", no matter what people with more power start out wanting. Too many developers have been encouraged to be minions, not professionals.
Ditto for ones who act like everything is a nail for their coding hammer. A developer who can save a company a couple hundred thousand dollars by not turning something simple into a big coding project is a rare and precious commodity. Or should be, at least.
The thing to do isn't to give demerits for "being difficult". The thing to do is to then add something to the scenario where they get into the thing you want them to get to. "For this, we need better access control than Google Sheets allows us." Or, "We need this to be more closely integrated with our accounting system."
Unless, of course, what you're hiring for is the willingness to roll over for unreasonable requests from people with more power. Which, honestly, a lot of places are.
Humans are primates, and primate dominance dynamics are going to be the default absent some conscious choice otherwise. Our whole executive/worker dichotomy is a descendant of the British class system. (E.g., note that airlines specifically have a "business class".) And MBA-driven business culture is focused on short-term managerial interest, not societal value or long-term business success.
I think all of those tendencies come to the fore at any organization that doesn't have either a strong sense of mission or a sufficiently desperate need for success that they pay attention to material reality rather than social reality. With a possible partial exception for things like co-ops and other places where the culture is fundamentally different enough. E.g., Mondragon, or Zingerman's.
I think Google, back in its don't be evil/organize the world's information era, probably qualified. They started with a very strong mission-driven culture rooted in academics and engineering. It took a fair bit of time for MBA dogmas to make it like most other places. But from everything I hear, what once felt almost like a calling now is just another job.
> MBA-driven business culture is focused on short-term managerial interest, not societal value or long-term business success.
This is a common refrain I also believe in and there's an interesting open question that comes up here about whether or not an engineering department should or shouldn't execute an order that intentionally destroys the product for short term gain.
Agreed. To me that's related to the question of minions vs professionals.
If I go to a doctor and say, "Hey, please prescribe me a lot of morphine," the answer will be some version of "hell no". That's because doctors, even if you pay for the visit, have responsibilities to the patient, the profession, and society at large. Responsibilities that should not be overridden by money or power.
The same is true for actual engineers, like the ones that build bridges. But although we often call ourselves engineers, a lot of us don't act like it. We're often more like the minions in a supervillain's volcano lair: whatever the boss says, we do.
We could, as a profession, agree to follow those. We could build an organization that supports people who do the right thing in the face of managerial pressure. We could censure those who don't. I'd love to see it happen, but I'm not going to hold my breath.
>The same is true for actual engineers, like the ones that build bridges. But although we often call ourselves engineers, a lot of us don't act like it.
Because software is the wild west. Maybe there's some exceptions in medical tech, but there's no license at risk nor ethics assossiation to be ousted from (nor to vouch for us) if one day we receive something like: "hey, we need you to triangulate and calculate the parameters needed to bomb this children's hospital. Get to it".
Either we do it and go along with our day. Or we don't and get moved or fired.
In an interview when you’ve been explicitly asked to discuss a topic to have a technical discussion about something is not when the moment calls for it. Doubly so if you’ve been asked twice. If you’re not willing to put aside being technically correct when you’re trying to show off your best self, it’s pretty likely that when things get tough, you’ll behave the same.
> unless of course what you’re being for is the willingness to roll over for unreasonable requests from people with more power
D, do you think that someone saying “can we please talk about a technical topic, here’s an example we’re both likely familiar with” is looking for yes men? I actively want my team and coworkers to challenge me, but I absolutely don’t want to work with that person who appears at every meeting with a list of reasons why we shouldn’t do X.
When I ask an interviewee a technical question, what I want is an answer that is correct technically.
If I want them to give me a different kind of technical answer, then I think it's on me to ask a question that actually requires what I'm looking for. It's not hard! All the Stripe interviewer had to say is, "Ok, great. It sounds like you have a good sense for system capacity. Now let's add another zero to all the load numbers." And then keep increasing orders of magnitude until they learn what they're looking to learn.
I am, just to be clear, not defending people being willfully obtuse or contrary jackasses. But that's not the scenario being described in either the Stripe story or the Google Sheets story I'm responding to. Two apparently reasonable people were asked technical questions and they gave answers that were the right thing for the business.
I think that's good and I like to hire people like that. I get lots of others don't, and I get the POSIWID reasons behind it. But I'm not going to pretend I think it's a healthy way to run an organization. And I also get that the people who like pretense and deference in interviews are not going to like me saying so. ¯\_(ツ)_/¯
>do you think that someone saying “can we please talk about a technical topic, here’s an example we’re both likely familiar with” is looking for yes men?
Probably. You say "likely familiar with" but interviews are conducted as if it's a pop quiz. Which I never understood.
If you want to have a decently technical discussion, why not just tell me ahead of time what topic and I come to the interview with research? Why do I have to guess that we're talking about dyanmic programming and be punished if you really cared about graph traversal? (meanwhile the interview is for an embedded programmer. Definitely reflects what you'll really do on the job).
I really hate how few initerviews really felt like they were testing my knowledge related to proper fundamentals and not treated as some pseudo-SAT schlock.
but also maybe its a green flag in that this employee might see the wood for the trees and save the company a lot of money later down the line. In my experience, a lot of engineers can waste a lot of time dicking around re-inventing wheels and whatnot.
While you consider it a huge warning sign, have you ever employed someone who would answer that way or are you assuming that you're not capable of making hiring mistakes? I can't help but think this "huge warning sign" might simply be a cognative bias where the interviewer is misdirecting their frustration in the poor design of their own process at the candidate [0].
For reference, I think both answers are fine and both perspectives (its a positive or a negative) are equally valid. Its just that I don't think we can confidently state either way.
> While you consider it a huge warning sign, have you ever employed someone who would answer that way or are you assuming that you're not capable of making hiring mistakes? I can't help but think this "huge warning sign" might simply be a cognative bias where the interviewer is misdirecting their frustration in the poor design of their own process at the candidate
Yes, I did. More than once. I always regretted it. Sure it could be a cognitive bias, but the entire interview process is essentially trying to figure out “can I work with this person”.
> I think both answers are fine and both perspectives are equally valid
I disagree - refusing to engage with the interview because you don’t like the question is perfectly valid to do, but don’t expect me to want to work with you over it. We’ve only got an hour, maximum, so any scenario we come up with is going to be contrived and simplified - if you can’t accept that then I’m going to make my decision based on that.
> Yes, I did. More than once. I always regretted it.
Fair.
> I disagree - refusing to engage with the interview because you don’t like the question is perfectly valid to do, but don’t expect me to want to work with you over it. We’ve only got an hour, maximum, so any scenario we come up with is going to be contrived and simplified - if you can’t accept that then I’m going to make my decision based on that.
Sure but lets not forget the other perspective. Candidates have to interview for a cumulative many hours over the course of a job hunt, only to have many interviewers batter them with an array of 1337code, pop quizes or contrived examples, none of which reflect the day to day work of the position they will fill. From their perspective their answer could well be a good one, albeit I agree that having some level of willingness to engage in the theatre is a positive sign.
In an auto-interview I recently did, I was given extremely limited time to "refactor" a bunch of code that was clearly broken. I chose not to refactor and instead fix the brokeness of the code, however I entirely expect to fail the interview because I fixed the problems instead of removing a couple of obviously duplicated code blocks. I can see why I would fail by not "following orders" but their async code was broken and the awful exception handling botched all their telemetry. From a "big picture" perspective I did do the right thing but it might be the case they're too stupid to know that I was doing the right thing (they're a multi-language company, so I assume they're less good at the language I specialise in).
Personally I think due to lack of industry organisation around certification or any sort of guild or union, we have a seriously difficult problem around hiring across the industry. In response to the extremely challenging task of vetting programmers I feel like orgs are simply fishing for reasons to disqualify candidates, as a reaction to this problem.
The rare positive experiences I've had interviewing were Amazon, who act like they want you to succeeed instead of fail or orgs that just half-ass it with low bar challenges, who seemingly accept that they're not capable of perfectly vetting a candidate.
and if you hire only based on solely on employee compliance then you are also probably missing the wood for the trees. I've worked in such orgs and they're extremely vulnerable to cargo culting.
I’m not hiring on compliance. I’ve accepted that his answer is correct but asked for the purposes of the exercise if he can put that to a side so we can talk about it. I’ve worked with and hired people like this and they tend to turn every molehill into a mountain, which is just killer on a small team.
I think you missed the point in GP's post. Not all organizations optimize for problem solving. Some organizations prefer subordinates who follow orders (or better, is able to read the mind of the boss to decipher what order he is actually making) than those who breaks out of the box and says ”just use gsuite, boss."
sure but if its not a privately held business then using gsuite is better for the shareholders. Ultimately its the bosses choice, but for the board to fire them its worth knowing they were aware of being able to use gsuite instead of pissing away resource on a needless project.
and if you don't understand my position then you've failed to interview. Some people just seek reasons to disqualify candidates and I think that's a pretty basic approach to interviewing. Remember, we all have a cognitive bias to hire ourselves and part of improving interviewing process is about trying to mitigate that by creating an environment where the interviewee can show the best of themselves, which may not necessarily reflect our own strengths. This is why pop quiz questions are kinda crap and while 1337code is better, its still kinda crap
> What would you do if two different people were emailing a spreadsheet back and forth to track something?
> I’d use google sheets
> Excellent answer, that is what I would do as well, now what if we wanted to build it in-house?"
> Well I would probably go home and work on my resume because that's a fool's errand.
I’ve not failed to interview. The candidate has been a jerk. Could I have asked a better question? Sure. Could the candidate not have sneered at it and thrown a strop - definitely.
Most real-world scenarios aren't so arbitrary, and hardly any have a "right answer". If I had a candidate that broke out of the box of our interview to give a good answer, and that's not the answer I "want", I'd be more likely to believe the interview question is the problem, not the candidate.
remember that we already did the "Excellent answer, that is what I would do as well, now what if we wanted to build it in-house?" part.
the "good answer" was already acknowledged, the "real-world scenario" answer was accepted.
the second part ("what if we wanted to build it in-house") is purely hypothetical to gauge how the interviewee would approach the specific technical challenge (shedding some of the "real-world" constraints so that the focus is technical).
if they again say "well that is dumb i would just use sheets", that is absolutely an interviewee problem.
Depends on the dyanmic. If you have an excellent candidate you're trying to poach, it becomes an intervewing problem because you're wasting both you and their time.
If they are a dime a dozen, then it becomes their problem. Whether or not they care it's their problem depends on their circumstance.
I also feel it's very easy for a good interviewer to bring the conversation back to the desired scenario.
Anything from "imagine we are in a parallel universe where Google Sheets has not been invented yet" to "how would you design a google sheets competitor" would do the trick.
Yeah - for sure. When I’m interviewing I want to give the candidate the best chance of success and to show off both what they know and how they will work with me.
I generally give it three goes with questions like these - the initial ask, and two clarifications. If we’re not getting anywhere I move on.
> The point of the question is to have something remotely understandable for both sides to talk about, that’s it.
I think a lot of people miss this point.
Real projects are complex and have tons of context at the historic layer, political layer, and technical layer. If I have one hour to do the interview, I need to get to some shared context with the candidate quickly, or else it'll just be an hour of me whining about my job. And I usually don't need someone who is already a senior subject matter expert, so I'm not going to ask the type of question that is so far down the rabbit hole that we're in "wheels haven't been invented yet" territory.
Fundamentally, that's why I'm asking a somewhat generic design question. I do also dig into how they navigated those layers in their past experience, but if I don't see them in action in some way then that's just missing signal I can't hire on, and that helps neither me nor the candidate.
In another company or timeline perhaps I could run a different interview style, but often you're working within the constraints of both what the candidate is willing to do and what the company standardized on (which is my current situation).
Depends on the dynamics here. Remember that an interview is 2-way.
Someone giving an answer like "that's a silly, unrealistic scenario" is more likely than not someone who isn't in need for that job to begin with. I'm sure it's something many wish they could say, because the interview pipeline can be very grating. But not everyone needs to play that game.
I think the contrived scenarios you come up with need to not have a trivial solution. Everything about my brain is optimized for KISS, it breaks everything to turn down simple solutions to reach for something more complex.
Think of it this way: they're paying you lots of money to build something boring that has a lot of prior art/research available to you for free. This could be the easiest money maker in your life.
It's not your problem they're hellbent on building a new wheel. They're willing to pay you!
Chances are, you've thought of your own pain points in whatever they've asked you to build and you've now got an opportunity to shine by solving them and demonstrate your expertise.
There is a tool invented lately, that's very good for solving problems, that are well-researched and had been solved multiple times already.
This tool is actually why there is a RAM shortage in the world right now.
Some even say, this tool will replace a lot of workers soon(sic!).
"That does sound like something nice to have. However, recreating Google Sheets is a substantial undertaking. First, we need to evaluate the business case for duplicating something that already exists to ensure that there is a net benefit in doing so. Second, we need to determine if the business has sufficient capital to see the project through."
I suspect a lot of businesses are going to make this mistake in the "SaaS is dead" era as companies try to eliminate $50k/mo subscriptions for boring business software, and they figure it's easier to burn AI tokens creating an internal solution they didn't plan on maintaining in the far future.
While I agree, how much training does anyone get as an interviewer? I spent 10+ years doing interviews at all sorts of orgs (including Fortune 500s, government, etc.) without a single hour of interviewer training.
Now that I think about it, none of those organizations ever trained me at anything at all. Huh.
> none of those organizations ever trained me at anything at all
They trained us repeatedly not to bribe foreign government officials, even though I had zero access to anybody like that. There was also some mandatory training against harassing coworkers. I.e. "protect the company from lawsuits" training, not "here are some ideas for how to do your job more effectively" training. They were megacorps, too.
Yeah that's proven by the fact they get degree educated level engineers and force feed them videos designed for people working entry level positions. Its a crying shame because there's actually a lot of interesting discussions around nuance that are just sidelined by these videos creating basic bitch absurdities:
> During the lunch break, Jim has dipped his penis into Samantha's yoghurt
> is this:
> a) entirely acceptable, its just his culture
> b) a borderline issue
> c) something that someone should report to HR
Instead of:
Jane is developing feelings for someone that reports to her, they meet up outside of work and have a one-night stand. What should Jane do next?
> While I agree, how much training does anyone get as an interviewer?
TL;DR: not enough training.
As a hiring manager, whenever we start a hiring period I have a conversation with my interviewer team about what qualities we're looking for and review the questions they plan to ask in order to normalize the process. Stuff like "what does a good answer look like, and why? what does a bad answer look like? is this something easy for a candidate to engage with or will you spend half the interview just explaining the background? is this coding question unreasonably hard?" and so on.
I also look at the evaluations that interviewers give relative to other interviewers. Almost every hiring cycle I've done I've had to deal with some interviewer (all over the seniority spectrum) asking unreasonably hard questions.
Yeah, I had no training on being an interviewer before I started doing interviews. My manager gave me some tips, and I came up with two security bug-hunt exercises (was hiring AppSec engineers), but that was it.
Now, I wonder if I had rejected earlier candidates that I would have passed if I was a better interviewer.
The best advice I've had on interviewing is: Find an actual problem that your team is actually working through and ask the candidate how they would approach it.
Then to jazz it up: simplify the proble. To get to the root stuff that needs to be covered (e.g. ignore creating Jira tickets and focus on connecting to a database with cross-refion replication).
You also want it to be simple enough that it can be solved in <30 minutes
That's unusual. Maybe that's a US thing? In Europe anywhere I've had to interview people I've received at least a couple of hours of training and then usually sat in as the shadow on at least one interview.
Quality varies, but I think it's only the super small outfits where I've been expected to just wing it.
Seriously? I worked at startups and research institutions. We trained people on interviews. I know Google used to invest quite a bit on interview training.
Sounds common to have training in big tech but I never received any training either. Sometimes we’d discuss changes we wanted to make to the interview process, which suppose could be considered adjacent to training.
I prefer pushing the constraints to motivate a different solution instead of asking them to do an unmotivated exercise.
“Google Sheets is a great solution for two people, but let’s say the department expands and now it’s ten people. How does this change your answer?”
It’s easy to break Google Sheets as a workflow by increasing the number of users, adding complex business logic, etc.
It’s interesting to see what candidates come up with and how they think. Sometimes the solutions are genuinely interesting. Mostly they’re not, which is okay. If you don’t give yourself the opportunity to learn as an interviewer, you’re missing out.
It takes a lot of practice to become good interviewer and majority of ICs especially at small shops never get the required mileage. I don't think i really knew what i was doing until like 100 sessions in...
Thats fair, it definitely sucks for those seeking employment to have a really awful interview. How do you turn it around without looking like an a-hole…
At google where i started interviewing it worked pretty well at least initially when recruiters assembled panels such that you only get 1-2 inexperienced interviewers out of 6. They also didn’t let you do screens as fresh interviewer where it’s much harder to get signal. Every other place i worked it was basically a crapshoot
That's a good part of why I want to stay in the IC track. I don't care about having the power to hire/fire people. That's a manager/lead skill in my eyes.
Maybe we have a small conversation with candidates, but I see no reason I need to be quizing people on their talents.
If I would be the interviewer in this kind of situation, I would just follow up with something like this: "that might be a good option, but let's assume you need to build a tool to replace those excel sheets, ..."
Surprisingly often you do get an interviewee who just won't accept the premise of the hypothetical. I've had people get hung up on the equivalent of 'but I'd just use excel' even with prompting/nudging/explanation that this is an exercise.
“Good answer! Now, assume we’re in a regulatory environment where we can’t use Google for legal reasons, and must develop a solution in house. With that new constraint, how would you solve the problem?”
The general way to handle this as an interviewer is really simple: acknowledge that the interviewee gave a good answer, but ask that for the purposes of evaluating their technical design skills that you'd like for them to design a new system/code a new implementation to solve this problem.
If the candidate isn't willing to suspend disbelief for the exercise, then you can consider that alongside all of the other signals your interviewer team gets about the candidate. I generally take it as a negative signal, not because I need conformance, but because I need someone who can work through honest technical disagreements.
As a candidate, what's worked for me before was to ask the interviewer if they'd prefer that I pretend ____ doesn't exist and come up with a new design, but it makes me question whether I want to join that team. IMO it's the systems design equivalent of the interviewer arguing with you about your valid algorithm because it's not the one the interviewer expects.