The AI Code Understanding Imperative

The Problem: Code is exploding and we can’t keep up

A host of factors, including the ongoing torrid pace of business process digitization, automation, and the cloud, continue to drive the amount of software code that most modern organizations have under management. Now AI is further accelerating the pace of code creation, and it’s only getting started. In short, code is exploding at an increasing rate.

At the same time, so are the questions that organizations have about their code. Compliance wants to understand any risk associated with the code. Marketing wants to learn what new capabilities the code provides customers. Security wants to identify any new risk in the code. Customer success wants to validate that the code solves customer problems without creating new ones. Sales wants to verify that the code aligns with customer security and process requirements, as detailed in the often extensive questionnaires they receive. Product wants to make sure that code does what they intended. Yet despite decades of innovation, questions about code are typically answered the same way as when we started coding: manually.

To cope with this, we’ve thrown bodies at the problem. Each business function has staff dedicated to finding answers to these questions. This staff has defined processes and built (or bought) tooling to help them collaborate and track their extensive work. This collective apparatus bombards Engineering with a torrent of questions. In response, Engineering pulls developers off their core work to search for answers amidst the onslaught.

Obviously, this approach doesn’t scale. From increasing security incidents, to growing legal and compliance crises, and more, the evidence is all around us. And since code creation isn’t going to slow down, and nor are the questions about that code, unless something changes, this will only get worse. Absent a better approach for understanding our code, collisions between these opposing forces will increase in frequency and severity.

The Solution: Recruit computers to help us understand and explain code

The current wave of AI, including but not limited to large language models (LLMs), provides a purpose-built solution to this dilemma: enlisting computers in our effort to understand their code. Programming is, after all, done in languages. And they are much more structured and bounded than prose. So it should be no surprise that even though we are at the dawn of AI-driven code evaluation, existing LLMs that understand code can provide surprising insight. A compound approach that blends LLMs, static analysis, and other data science and enrichment sources with human feedback is the most promising of all. Each component helps make the others better, yielding insight it’d be impossible to obtain otherwise. For example, passing the results of static code analysis to an LLM provides powerful context that improves question answering beyond what the LLM could do by itself. Adding a human in the loop provides further context and feedback, driving additional improvement.

Take a reorganization scenario, for instance: you’re an engineering leader who has just been moved to take over a new department that has had some challenges. You need to get up to speed quickly on the codebase, in particular to identify the major areas of pain (e.g. tech debt, code churn, under-testing, out of date documentation). You could run static analysis tools like SonarQube or PMD, but would then be left processing the output. Another common approach is to sit down with architects and team leads and ask them, but then you’re relying on word of mouth and not cross-checking their assessments against the actual codebase.

A compound approach would let you pull from multiple data sources (codebase and its history, the team’s technical documentation, etc) and let you ask questions and surface insights. In cases where it couldn’t give you the complete answer, it could suggest a member of your team (e.g. architect) who could provide the answer. This means your team is getting involved only as strictly needed, and their responses can help yield better insights in the future.

Figuring out the right components for this compound approach—and getting them to work together optimally for each use case—is substantial work. But the benefits are extraordinary.

• Speed
  • Instant answers to your questions with no need to interrupt Engineering. By unshackling the ability to answer questions from manual analysis, you gain insight in moments, not days / weeks / months.
• Scale
  • You can scale your ability to interrogate code like you scale the cloud, not like a hiring plan.
  • Colleagues in other organizational functions (e.g. legal) can get Engineering-approved answers to questions without requiring Engineering to be interrupted. Legal has new questions about a major release? They can get answers via self-service, not place additional burden on the development team.
• Coverage
  • Broad and growing support for languages, components, etc. as LLMs expand and improve—plus integrations with other tools for more context—means firms will soon have broader and deeper code coverage than most companies could ever afford to staff, and at a fraction of the cost.
• Accuracy
  • Code understanding is always updated based on evaluation of an ever-changing, ever-evolving codebase.
• Cost
  • Doing the work via software frees your team up from grunt work and enables first-class results at economy pricing.

Conclusion: Unlock code understanding with AI

In an era where the volume and complexity of code are increasing exponentially, traditional manual methods of understanding code are proving inadequate. The multifaceted questions posed from all corners of the organization cannot be sustainably addressed by continually diverting engineering resources. This growing misalignment highlights the critical need for a better approach.

The solution lies in harnessing the power of AI, specifically large language models (LLMs), static analysis, and data science. By leveraging these techniques, we can create a compound system where each component enhances the others, providing unprecedented insight into code. This system delivers instant answers, scales effortlessly, and offers broad coverage, ultimately reducing the burden on engineering teams and ensuring more accurate and timely insight.

Adopting this AI-driven approach will revolutionize how organizations interact with their code, transforming the process from a source of constant challenge into a manageable and valuable asset. This shift not only alleviates current pain points – like security incidents and compliance crises – but also equips organizations for future demands, enabling them to thrive in the face of accelerating code growth.