The Quest for Questions

This post is for engineering leaders of all stripes, but I’m sure that it will also resonate with product management colleagues who, like Adrianna, work closely with engineering. Last time, I wrote about the value of code visibility, and the situations which drive engineering leaders to dig into their codebases. I gave a few examples of business drivers, but didn’t really dig into how we select and answer the questions.

_{Everyone loves a good origin story}

In her latest post, Rachel wrote about how seemingly simple questions like “where are the TODOs” are actually much more complicated when you break them down. This is true. But how do you even know how to break the questions down? Knowing how to do this well requires an understanding of what the overarching questions are. In other words, what are the questions behind the questions?

I’m going to go back up to the origins of some of these questions. No sales or compliance spreadsheet I’ve seen asks “where are the TODOs?”. With apologies to Taiichi Ohno, I’m only going to get three levels in:

• Why are we asking this? Because TODOs often indicate technical debt in a codebase. Developers make notes to themselves about future improvements.
• Why does that matter? Because technical debt is an indicator of work that needs to be done on a codebase to avoid downtime, bugs, and other major issues in production.
• Why does that matter? Because we’re looking to buy this company, and I want to know how much risk we’re taking on when we start supporting their code in production.

That’s for another company looking to acquire yours. How about sales?

• Why are we asking this? Because TODOs often indicate areas of the code where there’s a known vulnerability.
• Why does that matter? Because the process by which companies identify, triage, and fix vulnerabilities serves as an indicator of their security posture.
• Why does that matter? Because if we buy this company’s product, we’re trusting that they are adhering to a high standard of security risk management.

Run it in reverse. The way you break down and answer questions like “where are the TODOs” will differ greatly depending on who is asking the initial question and what they’re really trying to answer.

_{It’s like flying with the Riddler!}

So many questions. Maybe you’ve had this experience before: your colleague in sales sends you a link to a web form that has some technical questions from a prospect. “Don’t worry,” they say, “it’s not a 256-line spreadsheet. It’s just 12 questions! Should take you 15 minutes.” And it is 12—until you answer the first one.

Then the question tree expands.

Go back to Rachel’s post and scroll down to the image titled “Higher Order Thinking—What do the TODOs say?” That’s the sort of question complexity we’re talking about here. Questions beget questions.

But that’s not the hardest problem.

_{Gaining XP}

The real problem is that without experience, you not only don’t know how to answer these questions well. You don’t even know what questions to ask.

So that means that you can’t just pawn the high-level questions off on someone junior on your team. I have tried this once and only once. This is not a knock on the junior engineer—we all started there, and I’m the one who made a mistake—they worked so hard on the rock fetch that is information gathering. But it took five times longer than it would have taken a senior engineer, the question breakdown wasn’t quite right, and the answers were often at the wrong level—way, way too detailed.

And you can’t just have your sales team or customer success team own them—I’ve also tried enabling both groups to answer incoming questions. They can take a reasonable pass, but it still requires a thorough review from engineering. Sometimes the pattern-matching fails (you can’t account for every variant of an incoming question), and sometimes the answer is out of date.

You and senior members of your team—you are the ones with the proper experience to ask the right set of questions, dig into the details, and then provide answers at the right level for the audience.

At Flux, we’ve each been through this before. We know the process and we know the time it takes. That’s why we’ve been building a platform that helps engineering leaders and their teams ask and answer the right questions about their codebases. We know the right questions to ask, provide insight into the details, and help summarize at the level you need to answer the question behind the questions.