Why AI Engineering Needs Old-School Discipline  🔗 **Source:** [Why AI Engineering Needs Old-School Discipline](https://share.snipd.com/episode/167d8a30-6601-489c-b2ef-c3b8de6dab79) ## Highlights - **Shift From Speed To New AI Capabilities** - Companies ask “How do we go faster?” but should instead ask what new capabilities AI enables that weren’t possible before. - Nimisha Asthagiri urges reimagining products and measuring collaboration with AI, not just deployment velocity. Transcript: Nimisha Asthagiri Think it's the question that is being asked. The question that we're hearing a lot from executives and others is, how do we go faster? How do we go faster? How do we keep relevant? Where I think the right question or another alternative, better question here might be, what do we build given the latest technology that we couldn't build before? So reimagining and reinventing what we couldn't do before. And that's where we focus our energy rather than everything. And then secondly, how do we build differently? Now that we do have AI and it's not just as an assistance, but actually even a part of the team with human and machine agents working together, what does that look like? And that is not, therefore, just a tool change, but a systems change. Rethinking your models of what to build and how to build. And that also then implies how do you measure? And the measurements in the past where we might have just measured output and production of, you know, how quickly are people bringing in pull requests or, you know, code into production. Instead of that, it could be more about, for instance, iteration cycles and interactions with your AI and with others. Those collaboration and interaction metrics that are also your first past acceptance rate, right? How often is that code that is generated? ([Time 0:03:06](https://share.snipd.com/snip/05155613-48b6-416a-bfb7-487f2df7bd43)) - **Build Strategic Platforms As Paved Roads** - Invest in platform capabilities as paved roads to accelerate teams and embed governance and efficiency. - Use platforms to free humans for higher-order judgment and unlearn repetitive roles replaced by AI. Transcript: Nimisha Asthagiri And so therefore, that then supports the system at large. And you're using therefore your platform as paved roads that people can use to accelerate their work, but also becomes that the conduit to the governance that you also need to apply And the efficiency gains. So all of that kind of coming together and for ThoughtWorks, like we've been progenitor of a lot of platform and engineering excellence capabilities. And that is now, once again, coming to the forefront. We're like, yes, there's a lot of fundamentals from the past that need to be actually, and at this point, reinforced, if anything. And so then you start thinking about if you have that harness and the system, technical system in place, then that's where now you're elevating the people to say, hey, use more of your Human judgment. ([Time 0:05:22](https://share.snipd.com/snip/a7c13131-ab5d-4875-be33-df6ba3702ab5)) - **Bring Back Test Discipline For AI Agents** - Reinstate engineering fundamentals like test-driven development and mutation testing to create tight feedback for AI agents. - Design tests and sensors so autonomous coding agents can continuously learn and produce viable code. Transcript: Nimisha Asthagiri So for instance, it might be everything from like mutation testing, right? So test-driven development, right? So we're really thinking about creating those feedback sensors for AI. So that the AI, when it's now with autonomous coding agents, and there's been a drastic change, right, in December with the latest models and so forth. So, and there's a lot of, therefore, keen interest in generating and designing these autonomous coding agents. But you want to, what do you want to bring back? So I think there are things like mutation testing. So a lot of our testing principles. And there might be things with test-driven development, for instance, designing that tight feedback loop for the AI so that it can continuously learn and evolve. So that being a key component. ([Time 0:06:54](https://share.snipd.com/snip/f3236233-6511-41e6-bc5a-bae3dc2c272a)) - **Measure AI Collaboration Not Just Output** - Traditional metrics like DORA still matter but should be complemented by collaboration metrics with AI, such as iteration cycles and first-pass acceptance rates. - Measure how often AI-generated code is accepted and needs minimal rework. Transcript: Nimisha Asthagiri So I think those types of things with deployment frequency, lead time, change failure rate, these are our lagging metrics that will then ensure that, okay, yes, we are moving the needle In the right place. But I think there's also other things like zero trust security architecture, right? And really thinking about ensuring that we have proper identity management and security as well. So when we're seeing the propagation of agents and now they're coexisting along with you when you're working on your laptop and your desktop. And, you know, a lot of that changes that are happening, there's zero trust architecture is critical. And, you know, being able to know who did what, as well as the authentication and the authorization of the work that is happening. ([Time 0:07:48](https://share.snipd.com/snip/9d08b32d-46be-460f-a5da-6f2318cf1e07)) - **Require Agents To Produce Architecture Records** - Coding agents change topology and architecture, requiring forward engineering constraints like architecture decision records for agents to document choices. - Nimisha recommends agents produce records humans can review since reading all code won't scale. Transcript: Nimisha Asthagiri And you mean coding agents, right? Frédéric Lardinois Yeah, coding agents. Sorry, coding agents. Specifically coding agents. Nimisha Asthagiri Yeah. So I guess seeing that in especially two different ways. But one is the you can think about it from the topology shifts that we're seeing as well as architectural shifts. And so I'm part of this global team that puts together the ThoughtWorks technology radar where we're, you know, finding trends and we're learning from on the ground actual experiences From thought workers, from our thought workers globally of 10,000 plus employees. And one of the things that we're finding is that we're using, we're putting together techniques to ensure that the code base is not shifting or drifting away from architectural principles. So those types of things are very important when we're also thinking about coding agents. And so I think there is the latest technologies with agent skills, as well as agents at MD was from before. So a lot of these things that help provide the context to the agents. We're thinking about that as essentially forward engineering, right? Like forward constraints for them in terms of the guiding principles. What are those architecture decision records that we would ask the agent itself to document so that the humans could review that and could reference that? Because at this point, humans reading all the code, that's going to become, you know, just, yeah, unsurmountable. So having those ways to be able to, you know, to validate that. ([Time 0:11:49](https://share.snipd.com/snip/eeab98f9-2b39-4890-91ac-dfabdde71c3e)) - **Design Agent Teams Deliberately** - Experiment with agent topologies deliberately; assign role-specific agents and avoid wholesale agent swarms without testing. - For regulated teams, proceed cautiously and evaluate maturity before wide adoption. Transcript: Nimisha Asthagiri And, you know, so there we're seeing these teams of coding agents where it might be a lot more strategic and intentional with deliberate design, you know, who might be orchestrating Very role specific, Okay, this machine agent for back end versus front end and whatnot. So that we are finding and we put in our radar as a mechanism, as a technique that people can assess. Just don't jump in without some other additional experimentation and testing are the coding agent swarms that are coming out where it's like hundreds of agents being tasked to do the Same thing potentially. And then you have to think about collaboration and, you know, they might run into conflicts that they have to resolve. And so they're, you know, right now we're like, it's still, it's still maturing. So for organizations that are on the forefront of trying these things out, then, okay, fine, go ahead and continue to evolve and provide more practice, best practices for the industry. ([Time 0:13:51](https://share.snipd.com/snip/337f0a92-9796-484d-aaab-dd83df2663a7)) - **Modularity Reduces Human And Agent Cognitive Load** - AI will vastly increase code volume and human reviewers will become bottlenecks, raising cognitive load for both humans and agents. - Modularity and clear boundaries reduce context window pressure and make reviews feasible. Transcript: Nimisha Asthagiri Yes. And we blip this actually in the past radar and also we reinforced it here as well. First of all, I think a perspective of cognitive load, cognitive load for the human agents as well as for the machine agents. And this is where, once again, good architectural principles like thinking about the boundaries of your code. And, you know, from before we've talked about modularity and, you know, from 1970s, you know, Parnas's paper, like once again, it becomes very evident here as well. So why that as a technique is important is that for the machine agents themselves, I think for their own, you know, what we feed into their context windows, but also for human cognitive Load to be able to review and understand the code, right? A lot of that modularity helps and comes into bear. And I think once you have that, then you can start also thinking about the harness that you are developing. And that harness, including, you know, those guardrails or architecture guardrails, what are those feedback sensors, right? ([Time 0:17:39](https://share.snipd.com/snip/a36b9300-5cdc-4094-ab1e-e1aa8f3a68b3)) - **Treat Some Code As Ephemeral Dark Code** - Expect lots of 'dark code' and treat code by volatility: explicitly document lifecycle and delete ephemeral code when appropriate. - Generate one-off code dynamically for single-use features instead of retaining low-value artifacts. Transcript: Nimisha Asthagiri So what I mean to say is, first of all, this is why strategic thinking is important about what's actually valuable for the organization. Where do you want to build versus buy? How do you want to actually differentiate your organization, right? Should we even bother building it? So that is very important because it's so easy to build these days that it's going to get even easier that, because code is going to become a commodity to generate, that you don't necessarily Need to spit. It becomes, there's going to be a lot of dark code, right? Like we have a lot of dark data. We've been collecting a lot of data through our big data initiatives. But now, you know, there's a lot of dark code more more so than even before. But not that there wasn't dark code. I think the other thing is what is the volatility of the code? Because it's so quickly to also create POCs, how might you architecturally or from your system standpoint think about documenting that code as having a life cycle that would then eventually Get deleted? So the retainment of that code being very explicit about it. But secondly, the thinking about code that could be dynamically and more ephemerally generated. ([Time 0:19:58](https://share.snipd.com/snip/070c797c-9cd0-4e61-93f2-93e8d8dd37a4)) - **Use Progressive Context Disclosure For Specs** - Focus on precise specifications and progressive context disclosure instead of massive monolithic specs. - Reveal only necessary context to agents progressively to manage machine and human cognitive load. Transcript: Nimisha Asthagiri Of… But then we're going to still shift the needle then to the spec. And the spec uses cognitive load, which we're also seeing as well. And for that, I think we do have a technique, by the way, that our thought workers propose what we're calling about progressive context disclosure. Okay. Where, know, progressively, because remember, like I said, the machine's cognitive load as much as important as our human ones. And the progressive context disclosure, you know, we are being explicit and intentional, I'd say, about what matters for this particular request. Frédéric Lardinois Right, right. ([Time 0:22:12](https://share.snipd.com/snip/b9b85d93-7924-4d75-bee8-b4896eb10df3))