How This Candidate Turned Ambiguity into Insightful Metrics to Uphold High Standards

Published Tuesday, August 19, 2025
Live Interview
Expert Analysis Included
Full Transcript

Loading Video...

Preparing the interview

Complete interview transcript & analysis below

Enhanced transcript with interviewer insights

INTERVIEWER

Interviewer

Um, in this projected role, uh, you'll be responsible for what I would call, uh, cutting edge, new cutting edge and new business models, right? It's cause they are, this company is making a transition to, sorry, that's my phone ringing, that's supposed to be on Do Not Disturb, but my wife invariably always calls when I'm doing these, and she's the only call that goes through on Do Not Disturb. So sorry about that. Let me, let me let it stop ringing and then I'll ask the question again. I apologize. No problem. It literally goes without fail that I put on do not disturb and her, she's the person who calls when I'm doing these.

CANDIDATE

Candidate

My wife has that magical power as well. Absolutely

INTERVIEWER

Interviewer

it's great text exchange we have after this. Um, OK, so, uh, this, in this particular role, it's very much, uh, what I would say new business model, cutting edge, right, they're trying to transition to being a sass company when they've very largely traditionally been a hardware company. Uh, they have a lot of data exhaust it's coming off their products and trying to figure out how to use that in a, uh, we'll call it a more friendly way for their customer base, so it's government agencies and and others, right? And so this is gonna be mean likely just based on the the JD that you sent me that you're gonna be working with or having to define kind of performance metrics and operation metrics to to to keep the teams operating at a high pace and tempo, right? This the the they brought in a bunch of senior leaders from other companies that are known. Being fast pace into a company that did not start with that DNA, right? So they're definitely trying to change the nature of the company. So walk me through how you as a leader, uh, determine targets, like appropriate targets, uh, to set by area of your team to ensure high operational tempo.

CANDIDATE

Candidate

Sure, and, and so by operational tempo, are you referring to, um, like contractual SLAs, operational SLAs um or product, uh, you know, product projections or, or trends that you wanna see or both?

Interviewer Insight

interview is being intentionally vague and won't clarify

INTERVIEWER

Interviewer

I, it's an ambiguous question

CANDIDATE

Candidate

intentionally

INTERVIEWER

Interviewer

how

CANDIDATE

Candidate

you

INTERVIEWER

Interviewer

answer it, yeah.

CANDIDATE

Candidate

Yeah, so So I'm, I'm asking a clarifying question. Yeah, and and

INTERVIEWER

Interviewer

it's, and it's, but it's intentionally vague and it won't be clarified because it's intentionally vague.

CANDIDATE

Candidate

OK, so you're not gonna, you're not gonna clarify. OK. Um, yeah, OK, so, so I, I see kind of, uh, the, the, the two primary categories that I just pointed out, which are the, the product. Um, contractually, uh, obligated SLAs that, that we want to provide to our customers or that, that we are obligated to provide to our customers and those are things like up time, uh, reliability, um, And, and so for, for up time, you know, you, you, you, you know, we have the, the traditional number of nines, right? I think, uh, for, for operational stability, I'll be asking the team to, to begin measuring, um, uptime, right? So, uh, you know, shooting for, let's, let's start with 3 9s. Like I don't know where they are now or I don't know what the contracts say, but let's say 3 9s, uh, for now. Um. And, uh, and, and we'll be measuring, be measuring that. And then we'll, we'll also measure uh reliability and that would be the, uh, the number of hours that elapse while uh a product is up. Um, so we'll, we'll, we'll, we'll have some sense of, of, uh, you know, the gaps when the product is down, um, how reliable it is. Um, those, those are the, the two kind of primary operational metrics that I'll be looking for. And then on the product side, um, There are, there are a lot of different things that, that we would want to, uh, a lot of metrics we want to collect to present to the business, um, that, that show the, uh, the usage, uh, the adoption of our APIs. Um, so we wanna see, you know, per, per agency, uh, per unique user per agency, how many, um, how many hits our APIs are getting. Uh, we also are, are gonna, uh, kind of going back to the Uh, contractual side, we wanna make sure that customers aren't, um, abusing the license, um, agreement that we've, uh, created with them. I know that Axxon has a, uh, kind of a licensed user model, so we wanna make sure that, that we're counting the number. Appropriately and either, you know, overcharging charging them for overages or uh preventing, preventing them from going over. Um, but yeah, so, uh, but back to the uh business metric side, right? So we wanna, we wanna see adoption, we want to see consumption, um, or, or the lack of adoption, you know, the lack of use, um, that, that might indicate, um, either a, a misunderstanding with the product, um, or a product or an API that is, uh, you know, not performing, not doing the job that We're we're wanting it to do.

Interviewer Insight

This was a starting point for using metrics and performance frameworks for measuring success against goals, but the content didn't develop much past a surface discussion in this first answer block.

INTERVIEWER

Interviewer

OK, but so what is your process for working through this with your managers, right? If you're, if you're coming up short, how do you work through this with your managers when you're when you're not achieving what you expect to be achieved?

CANDIDATE

Candidate

Yeah, so, so I've been, I've been a dev manager, uh, for a large chunk of my career. Um, and I've had to have those difficult conversations many times. Um, and, you know, I've done, I've done a lot of hiring and firing and correcting. Um, so I'll give you an example. I, I, I worked with, with a guy who I, he, it's kind of, kind of a, a weird situation where um I was about to go on a, on a trip to Europe for a couple of weeks and I just hired this um very senior developer, um, and he was coming in to work on a greenfield project with the team. Um, he was, I hadn't hired him, he's a contractor and he You know, we interviewed him. He was a super bright guy, and he had a lot of really fresh and cool ideas to contribute, you know, using cloud technologies and so forth. And so, I talked to him, um, a couple of days before I left, and I said, look, I know that you're brand new. Um, we have a lot of senior engineers on this team. Um, while I'm gone, I'd like you to, um, you know, bring your experience, talk to the team about, uh, about You know, the kind of the, the new cloud methods, uh, cloud technologies that you're familiar with. And when, when the developers are talking about architecture and design, you know, don't, don't be afraid to interject and, and don't, you know, don't feel like you have to accept the status quo. Um. And, and, you know, he said, OK. And then I, I left, I was on vacation, I came back. And when I got back, I learned that he, um, he, he had been kind of, uh, combative the entire time that I was gone. And, and he took my question, the status quo as, um, don't believe anything anybody says. Um, and You know, don't do it their way, do it your way. And, and, and, and he kind of got the wrong message. So, um, I had to, I had to, you know, level set it again with him. I had to reset expectations. And really, as a, as a manager, that is your number one job. It's my number one job is to set clear expectations for the team. Um, and, and then hold them accountable to those expectations. I will write them down, I might set them as goals, um, but, you know, being very clear about expectations is, is job one. Um, and then it's easy to follow up on those expectations, um, you know, at a daily stand-up meeting, for example. Um, I can tell if a, uh, if, if an employee is, is meeting expectations, if they're getting their work done, if, if they're hitting their, hitting their targets or not. OK.

Interviewer Insight

the "sell to" process for gaining approval and agreement was light on details.

Interviewer Insight

this is a case of where the answer is "right" but for the level at which the candidate is interviewing it will prove insufficient.

CANDIDATE

Candidate

Been a dev manager for a large chunk of my career. Done a lot of hiring and firing. Had one situation where I had a sr dev who was coming in on a greenfield project. He was a contractor. I wanted to get his input to our team about new cloud tech with which he was familiar and asked him to make sure to push on things that felt like status quo. I left on vacation and when I came back it turns out he had been rather combative the whole time. He took my input as "don't believe anything anyone says" - he got the wrong message. I had to reset expectations. #1 job is to set clear expectations, communicate them, and follow up regularly.

Interviewer Insight

There was no actual problem resolution here. This example is a case of an employee not performing to expectations, but there was no resolution presented. In an ideal case the preamble would have been used to setup the problem but more of the discussion would have been spent on how this led to a performance improvement opportunity for the contractor, as well as candidate realizing that their own communication style may need to be adapted to prevent future occurrences of this sort from happening.

Expert Assessment

Interviewer assessment - would be used in a hiring meeting

This was another case where the candidate likely has the content which would demonstrate that they have High Standards but it was not presented during the interview. For the companies which operate at the highest level, Insist on Highest Standards means having a mindset of ensuring that the goals are known ahead of time, how they will be measured, having the telemetry and systems in place to gather and surface the relevant data, and mitigation steps in the event that things go off the rails. Beyond the product standards, there is a requirement of managers to be able to manage the standards placed on their employees in a logically consistent and performant manner. This was not demonstrated with the content provided.

Use this feedback to improve your own approach
How This Candidate Turned Ambiguity into Insightful Metrics to Uphold High Standards | Insist on High Standards | CalmInterview