How Better Developer Experience Drives Results ![rw-book-cover](https://i.ytimg.com/vi/54bg7kXqWLQ/maxresdefault.jpg) ## Metadata - Author: [[LeadDev]] - Full Title: How Better Developer Experience Drives Results - Category: #articles - Document Note: --- tags: [#developer-experience, #DX, #onboarding, #business-case, #open-source, #metrics] - #developer-experience - #DX - #onboarding - #business-case - #open-source - #metrics Author: Lead Dev Webinar Panel (Jennifer Riggins, Martin Reynolds, AB, Christos) Source: [Lead Dev Webinar: How better developer experience drives results](https://www.youtube.com/watch?v=54bg7kXqWLQ) Reference: [YouTube Video](https://www.youtube.com/watch?v=54bg7kXqWLQ) date created: 2025-05-15 16:35 --- # How Better Developer Experience Drives Results ## Table of Contents: - [📝 Summary](#-summary) - [📖 Main Content](#-main-content) - [🗒️ Notes](#️-notes) - [📖 References](#-references) - [🗣️ Paraphrase](#-paraphrase) - [📰 Newsletter](#-newsletter) - [✅ Action Items](#-action-items) - [🧪 Hypotheses](#-hypotheses) - [📊 Methodology](#-methodology) - [❓ Questions](#-questions) - [🔮 Future Research](#-future-research) - [💡 Implications](#-implications) - [🧠 Learning Styles Assessment](#-learning-styles-assessment) - [📊 Accessibility Assessment](#-accessibility-assessment) - [🌍 DEIB Assessment](#-deib-diversity-equity-inclusion-and-belonging-assessment) - [✅ Logical validity](#-logical-validity) - [🔍 DARVO Analysis](#-darvo-analysis) - [🔍 SEO Evaluation](#-seo-evaluation) - [🎯 Conclusion](#-conclusion) --- ## 📝 Summary The Lead Dev webinar explores the direct impact of *developer experience (DX)* on business outcomes. Through case studies and real-world practices from regulated industries (fintech, gaming, platform engineering), the panel shows how robust DX—driven by automation, transparent metrics, fast onboarding, and peer advocacy—yields faster time-to-market, higher retention, better code quality, and stronger business cases for further investment in developer platforms and internal tools. --- ## 📖 Main Content ### 🗒️ Notes #### [0:13–3:01 – Panel Introduction & Context](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=13) - *Jennifer Riggins* introduces the focus: making a business case for DX. - DX is not just a "nice-to-have"—it's about real business value. - Panelists: - *Christos*: Senior tech manager (iGaming, Kaisen, international teams). - *AB*: Engineering manager at Cleo (fintech), manages chat/LLM teams, previously DX lead at Factorial. - *Martin Reynolds*: Field CTO at Harness, formerly dev, DevOps, platform engineer. #### [3:09–4:49 – First-Hand DX Experiences](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=189) - *Christos* describes DX roots: post-M&A, led R&D in Asia to align processes, engineering culture, fairness. - Emphasizes the link between streamlined workflows and equality across distributed teams. #### [4:50–6:06 – Industry Backdrop](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=290) - Panelists cover industries with strict regulatory/compliance needs: - iGaming: operates only in regulated markets, must move quickly. - Fintech: compliance (security, privacy) drives tooling decisions. - Platform engineering: must meet the same high standards as their financial/healthcare customers. #### [6:07–10:52 – Balancing External & Internal Demands](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=367) - All industries see compliance as a "design parameter," not just a constraint. - *AB* (7:02–8:59): Security/compliance = easy path for devs ("yellow brick road" analogy). - Internal tools: Simulate user conversations for testing, avoiding exposure to sensitive data (8:00). - *Martin* (9:24): "Easy to do the right thing, hard to do the wrong thing"—guardrails, not blockers. - Use of bumpers/guardrails in workflows: protect security, speed up delivery, keep internal/external standards aligned. #### [10:53–12:34 – Treating Developers as Internal Customers](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=653) - *Christos* (11:11): Developers are "first-class citizens"—treat them as VIP users. - Platform as Product: Developers are involved early in tool design, providing feedback and shaping direction. #### [12:35–15:56 – Deciding What to Build for Internal Devs](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=755) - *Martin* (13:49–15:11): Prioritize automating workflows that generate the most support tickets. - Example: Self-serve access to dev/prod environments, automate pipeline provisioning. - Focus on "quick wins" to build momentum and reduce friction. - Balance quick wins with larger business objectives (15:23–15:56). #### [16:00–18:21 – Building Engagement & Adoption](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=960) - *AB* (16:16–17:40): Four phases—Identify pain, Engage devs, Build tools with their feedback, Evangelize tools. - All-hands meetings recruit early adopters and gather broad feedback. - Peer testimonials drive wider adoption ("devs trust devs more than tool creators"). - Tool adoption is monitored through direct feedback and usage. #### [18:22–19:08 – Peer Advocacy & Cross-Team Buy-In](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=1102) - *Christos* (18:36): Cross-team voices amplify buy-in; collective feedback is louder and more persuasive. #### [19:13–22:36 – Business Value & Quantifiable Outcomes](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=1153) - *Christos* (19:41–21:40): CI/CD case study: - Surveys found CI/CD was too slow (20:03). - Data (DORA metrics) confirmed increasing lead times (20:26). - Solutions: Parallelize test suites, cache containers/libraries, autoscale jobs. - Results: CI/CD pipeline time cut from 20–25 minutes to 4–5 minutes. - Direct business impact: faster time-to-market, improved user experience in a high-churn industry. - *Martin* (23:00–24:42): Bank onboarding case study: - Onboarding time reduced from weeks to days—engineers productive on Day 1. - Better onboarding boosts retention, speeds up value delivery, saves money. #### [24:43–28:59 – Employer Brand, Retention, and Onboarding](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=1483) - *AB* (27:03): Cloud-based dev environments cut onboarding time by 90%. - Devs onboard in hours, not days. - Positive employer brand: devs share experiences on Twitter, Glassdoor. - *Martin* (24:44): Developer happiness = retention = cost savings. - NPS for external users correlates with internal DX metrics (25:00–25:35). #### [29:02–30:28 – Documentation, Technical Debt, and Information Discovery](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=1742) - *Jennifer* (29:42): Devs cite documentation and tech debt as top friction points. - AI aids documentation and onboarding, but clear ROI is essential. #### [30:29–33:37 – Metrics, Transparency, and Continuous Improvement](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=1829) - *Christos* (31:02–34:17): Measurement starts with a cultural foundation of trust. - Use DORA metrics, onboarding times, and before/after snapshots. - Open, transparent sharing of metrics builds trust. - Compare internally and externally (benchmarking). - *Martin* (34:34–37:40): Translate metrics for business audiences. - Speak in KPIs that execs understand (NPS, feature velocity, cost savings, bug leakage). - Use reductions in “time spent on non-innovative work” as a value proposition. - Business-aligned metrics secure ongoing investment. #### [37:41–40:28 – Role of AI in Developer Experience](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=2261) - AI reduces time spent on documentation and discovery. - Measure ROI in terms of business outcomes (hours saved, features shipped, satisfaction). - The language gap between business and tech is real; DX teams act as translators. #### [40:29–45:09 – Onboarding Metrics, Real-World Benchmarks, and Feedback Loops](https://www.youtube.com/watch?v=54bg7kXqWLQ&t=2429) - *AB* (41:25–42:44): “Time to first PR” for new hires: 1–3 days at top companies. - *Martin* (43:01–44:11): Immediate setup, with compliance and code access, enables fast productivity. - *Christos* (44:17): It takes 1 week to “deliver something meaningful”; domain mastery takes longer. - Survey response rates are a “trust barometer”—rising rates mean devs believe leadership will act on feedback. #### [45:10–46:14 – Final Takeaways](https://www.youtube.com/watch?v=54bg - Document Tags: [[developer experience]] - Summary: In this webinar, hear from engineering leaders who have transformed DevEx in their organizations, their strategies for gaining buy-in from non-engineering stakeholders, and the metrics they used to build a compelling case along the way. - URL: https://www.youtube.com/watch?v=54bg7kXqWLQ Hello everyone and welcome. A huge thank you to everyone that's joining us today live or those that are watching it after the fact. We're having an active interactive discussion on how better developer experience drives results and how to make that case both us making the business case for developer experience and how to make the business case to your business side of things because this time is money. So let's try to find a way to make the best of these very well-played hopefully engineers lives and also that you get maybe investment to help invest in these engineers lives that we hope you will be convinced by and you will have the power to convince by the end of this call. So this is a lead dev webinar created in partnership with harness. The webinar will last about 45 minutes and during throughout I welcome everyone on the call to please add Q&As's. We want to keep this as robust and relevant to you as an audience as you would like and we want to make sure we answer the questions you like. So let's get started with some introductions. I'm Jennifer Riggins. I'm a technology journalist and for the last 10 years it's been a lot around the developer experience. So this is nothing too new to me but the debate the debate goes on and today I'll be joined by Martin Ay and Christos as they share their experiences and insights. So uh Christos why don't you start by introducing yourself first and tell us a bit about you and what developer experiences in your life. Okay so hello all I'm glad to to be here along with with all of you. So I'm I'm Christo. I'm a senior technology manager currently in the I gaming industry and more specifically in Kaisen gaming. Maybe you have heard it with the brand Pano. So I have an experience of more than 10 years in different industries from telecommunication to to entertainment, different uh you know positions and crafts, software engineering, um data engineering, um and and that's me and for the last four to five years I'm having let's say leadership position more you know manag I have followed more the the managerial path. So I'm leading teams across different countries, different even continents from Asia to US and to Europe. And that's me. Other than that, I'm I'm I'm used to train for for Hierox and I also like traveling a lot and you know spend some time with my family and my two years old kid. Fair enough. That's for me. Okay. And regarding my first let's say experiences um regarding the developer experience right it was started when I when I you know I worked in another company and we we finalized an M&A right so uh I was responsible for an R&D department uh in Asia and more specifically in Sri Lanka right so my main job was to try to to streamline the processes the engineering culture and also uh to be let's say uh to ensure fairness and equality between the different departments across the rest of of let's say our R&Ds and also and that was my first let's say experience with developer experience in general right um of course all this stuff was binded with um OKRs etc etc but that's it for me and I would like to to give you the uh you know the word to to the rest of the the speakers for now please. Thank you Christos. Now AB can you introduce yourself and provide your context? Of course. My name is AB. Um currently an engineering manager at Cleo a fintech in the UK. I manage two teams in the chat pillar. We are in charge of the chat experience and chat quality and evaluations. Currently we run evaluations with human annotations and LLM evaluations. Um before I transition into management I was also I was a software developer at factorial on the DX team. That's the developer experience team in charge of streamlining um processes and workflows to improve developer productivity and experience. So, I'm so excited to be here to talk about how DX drives business outcomes and results. Absolutely. And it should be very interesting because you you're in more of a technical of fintech, but financial services in general are what I call an accidental tech company that suddenly woke up and have tens of thousands of engineers and are working on some of the most interesting problems that are actually real world problems around AI and data. So it should be great and we definitely want those developers to be happy. Now, Martin, can you introduce yourself to the crowd, please? I can indeed. I'm Martin Reynolds. I'm the field CTO at Harness where I've been for about 18 months. And, uh, prior to that, I I worked in software development, uh, DevOps and platform engineering teams. And, you know, developer experience has been pretty constant for the 14 15 years before I joined um, joined Harness. And you know the flip is really I get to take all that experience and and and then work with different companies now to help them uh improve their developer experience. Okay. Fantastic. Each of you come at this conversation and the idea of improving and concentrating on it all developer experience with different internal customers. Martin, you're dog fooding and building an internal developer platform product for other people's internal developers. Very meta lowercase meta. Uh AV, you're within the constraints of this well-re regulated financial services industry and Christos in data engineering in the I gaming space. Can each of you explain how you find the right balance between those external demands, your customers and your restrictions, your your like the financial services is a well- reggulated community, things like that. and then how you can then also prioritize your internal developer customers. Amy, why don't you go first? Okay. So, as we all know, the fintech industry has a lot of security and compliance requirements. So, one thing that we have learned to do is to treat these all these requirements as design parameters. we we choose to not see them as restrictions because of course they are in some way but then they also form the design parameters for the tools that we build. So the the the um most the most important thing is to make the secure and compliant path the easy path for them so that that's the path that they just naturally gravitate towards. For example, right now my team is building something called an AI test suite. And it's it's a tool that helps devs to be able to create test cases and test groups to simulate user conversations in chats and then test what we call outcomes. So it's it's we have the the constraints of not accessing our like the data for users except it's extremely necessary. So we don't want the devs going to prod to say okay what did this user say um what was their configuration and all of that. So now we have built a tool internally that simulates conversations. you can't tell the um LLM, give it instructions and say this is exactly what you want to say to Cleo and then as we all know LLMs are nondeterministic. So you could try to run that a million times and have so many like the results would just vary. And so what we do is we take these constraints, we let them make us be innovative in the kinds of solutions that we build such that we can have the easy path as also the secure and the compliance path. Okay. Yeah. The golden path has to be compliant in your field which is very important. I like to call it the yellow brick road. You can go off, you can get high in a field of poppies, but your life will be easier if you just stay on the yellow brick road, not singing to inflict any copyright infringement for your drums. So, Martin, how does that balance in your industry? So, so that's really interesting because I I I mean AB almost used a phrase that we use quite often uh which is easy to do the right thing and hard to do the wrong thing you know and uh that is something that you know we we talk about quite often because that having that that golden pathway uh to to production making it easy for people to to get there uh and and you know and I I like the idea of calling them design you know design parameters over say um you know we will often call them guard rails. They're really like you here's the bumpers on the side of the bowling alley that you can you can you can roll the ball how you like down the middle but you know we're going to we're going to protect the other the other bowlers uh by not letting it bounce out. you know do that and and interestingly if I talk about how we do it internally too it's for us we have a lot of financial customers and so we have to hold ourselves to the same standards as the banks do and health care services do and insurance firms do so we absolutely have to meet those same standards internally we have the same kind of process of you know a golden pathway to production um you know kind of welldesigned secure architectures that we're trying to meet and we have all those policies is uh you know and guardrails in place to make sure that we're hitting that path to production and we're doing it securely time and time again. Absolutely. And then this idea how does that differ or does it not in the gaming industry if I'm honest Christos I know less about the online gaming industry I than I do about writing code maybe if possible. Okay. Yeah. So more or less it's it's it's the same. I think that the team covered almost everything right again in the IM industry there are you know strict regulations that we have to follow. So we are operating only in the in a regulated market. So market so this is a common that we have with with Abby and also maybe with Martin due to the banking sector right and um more or less it's it's it's the you know it's the the same it's a very competitive uh industry you need to go fast to have you know the golden the golden paths uh to be you know instead of you know ahead of of the curve and the and the competition and in order to do so I think you have um uh to have you know the people that solving problems like the developers happy. So this is where developer experience you know comes into the play right because for us we also we have the the users of the application okay and then we have also the the developers and we are treating our developers like our internal users. So it's not different for us maybe the capacity bit different superization but it's not that different uh you know in a sense. So yeah, it's it's it's pretty important for us to you know to treat developers like first- class citizens and of course like you know VIP users that is kind of in the theme of platform as a product where you are not just you are bringing your developer customers on early as you're designing the services for them. You're understanding their frustrations and then you're getting their feedback once you build it and you're trying to sell them on it. You're giving them demos. you're getting their opinions across surveys. It's definitely very important. So, I just want to speak to our fifth panelist, the audience. I welcome you and wanted to remind you in case you want to ask questions, ask follow-ups, you can add comments in the notes if you want me to vocalize on your behalf. And of course, if you have any questions, you have access to these experts for the next half an hour. So, please ask them. In the meantime, I just with this we've talked about this friction a bit where you've got multiple or friction or the blissful world you all seem to work in where you have engineers that you know you need to build products for but you also need to build products for the future world so or for the external. So how do you decide which internal developer centric features do you build? Which comes first? Yeah. So that is quite an interesting one because I think uh you know and I'm going back to my previous experience because I I I think it varies from organization to organization. I know for you know a lot of my personal experience it was actually you know what we wanted to do was one of the first things we want to do is make it easier for engineers uh and you know developers to selfs serve uh to be able to solve the problems themselves rather than raise a ticket. So we actually some of the first things that we prioritized were the things that got the most tickets raised that the DevOps team or the platform engineering team were having to then pick up and you know they were waiting for a response for them to do something you know. So what we actually did is we just looked at the number of tickets that were being raised looked at the ones that were most common and said you know what this ticket gets raised you know 100 times a day across our engineering estate. How do we automate this and make it so they can self-s serve that that functionality or that that need? So whether that was I need access to a dev environment to look at this or I need temporary access to a production environment to review this to help me solve a a problem or you know actually I just need a new pipeline spot, you know, it depends. But we were picking off our first ones that we were targeting were the ones that were the quick wins for the engineers. The things that saved them, you know, maybe not loads of time, but saved them frustration and friction. So, pull out that friction. Uh, and then it's really good to be able to then have that follow- on conversation with them to say, hey, you know, what else would help you? But I think I do think though you have to balance that with, you know, what are the business and organizational objectives? Because you know at the same time we were focusing on how do we build out that golden pipeline or pipeline versions. You know what are the frameworks and things and and you kind of have to be you want to be supportive and focus on the customer but you also want to have those objectives and maybe a little bit of opinion on how you want to build this going forwards. And also the idea of early winds is another platform as a product because you want to get your customers your internal developers on board as quickly as possible. And what was interesting, yes, tickets, but what I'm hearing something to look at is anytime a team relies on communicating with another team to get something done and that's a great area to start. What are some other areas you've started in your journey, AB? Okay, so I think I like to see the entire process in four phases. identify, engage, build, and then evangelize. Identification is what um is what he just spoke about in terms of finding out what's the most painful pain point, per se, and then being able to say, "Let's pick that up and let's fix it." But then, let's say you decide that you want to fix this pain. What I have found that works especially when it's time to build a a new tool is to be able to communicate with the developers. Um at Cleo we use the all hands to say we have identified this problem and we will be fixing this problem. So is there anyone who's interested in helping us fix this problem or giving us ideas on how to do it? Please join this channel. That way that's exactly how you find the early adopters, people who are excited about this thing. And then we begin to build, communicate, ask them for feedback and then once that is done and we have the feedback from them, they've been able to use it, we then say, "Okay, this thing is out there. We have to then evangelize and start telling everyone like this new tool is here. Can you please use it?" But then there's there's also the part of there's nothing like hearing a fellow developer say this thing saved me two hours using this tool saved me five hours. You would find that devs will jump to use something when when a testimonial comes from another dev as opposed to you who built it who is saying oh it's it's really cool. You're like of course everything you build is cool to you. So finding ways for people to share their testimonials and to say oh this thing helped me. That's how we then build adoption for it and then that's also how we identify based on adoption if it was the right tool or not. Great marketing advice whether internal or external that not all tech companies are great at case studies and testimonials but it's what people care about most. Christos, do you have anything to build on that? No, but I really liked the you know Aby's Aby's comment because I think that you know cross team let's say voices are louder than a solo voice right so having let's say Alice uh that are you know facing the common problems or they they going to contribute to to find the solution this gives you you know power to to pitch the idea get the buy in and deliver something that is it makes sense let's say for the department so I really like this kind of of AB's view yeah absolutely and so reducing friction is sounds great for developers improving developer experience but the point of this talk is very much on the business case for that so Christos can you share a specific instance where there's been an improvement in developer experience that directly led to some sort of positive business outcome come be revenue it could be retention repeatability cut cost speed to market something else I'm not thinking of what how has that helped developer experience in the bottom yeah I have some examples of on top of my mind but I would pick let's say maybe a common uh let's say example to many people I guess in this audience so it's regarding the CI/CD so there was a case a period actually when in surveys we get um consistent feedback that our CI/CD uh is is slow or you know it doesn't meet let's say the expectations of our developers. So we try to to take this as an input uh to try to somehow correlate it with quantitative data let's say with some dora metrics and indeed we saw that the lead time was a bit increasing over the the previous let's say months. So we validated that and um what we did I mean in the actual uh problem solving part was to try to parallelize some of our testing suites in order you know to run in parallel let's say have um less time to be consumed at this point and uh also we have um um make some libraries and uh containers cast in order to to build it once and reuse it instead of build all the time. So this uh was further improve the the time. And last but not least, we we inserted an autoscaling mechanism in order to avoid let's say having um let's say uh some jobs that are waiting for others to finish. So we try to uh to take this kind of uh jobs in a realtime m let's say in order to um to improve the the overall time. So in terms of a numbers because I think are important from 20 25 minutes when we start we we managed to be around four to five minutes. So I think it's it's a very good um example of you know uh hearing the the developer's voice and try to to help him with the actual day-to-day work and then how did that yes speed is good but how did that speed impact business because we have very good so yeah very good question thank you thank you Jennifer so having let's say a faster pipeline it affects the time to market right so we have a closer feedback loops we are not waiting for um a check let's say or a potential fix uh 20 minutes and all these kind of you know activities are also affecting the time to market and also the user experience because we are a user experience heavy right so we need to have let's say solutions fast we have we need to have quality in our in our software because you know the the users of the gaming industry is not that patient right so yeah this is a factor for Okay, Martin, what about you? What have you seen are features that have come out of following this developer is a customer mindset platform as product mindset business? Yeah, I I have one example I'm going to hold off on for a second, but I I I we see lots of I mean honestly we see lots of different uh impacts for this. So you know uh certainly some of the larger banks and you mentioned these earlier you know they have huge numbers of developers and huge number of new starters and changing of products and you know one of the things that they were targeting was saying hey it takes us weeks to onboard an engineer you know and uh that onboarding of an engineer experience was not a great experience you know they were uh not not able to even know what laptop they had and in fact um uh I know specifically Ly's banking they uh they they took that that whole process down from weeks to to pretty much the developer having their laptop in front of them kind of on the day that they arrive with all the access to the tools that they need you know and actually you know that kind of thing just makes it a good experience from day one. Um but interestingly that same so you know in terms of how does that translate that translates into two really useful things. one those those engineers that you just spent a lot of money hiring are more effective in the business much faster right so you know it means that it is days until maybe they are committing their first line of code rather than weeks or months in some instances you know and uh I think the other side is is you know those people that have those great experiences also are more likely to stay within an organization and you know expensive the engineers happy, engaged uh and enjoying what they do. And one of the things that uh I I really loved and it kind of relates to what we've been talking about, but um we had a a CIO from a bank come and talk to us at Harness about what they were doing uh with their their developer experience and they they said, "You know what? We we actually rate uh we look at our external facing MPS as a as a direct correlation metric to our internal developer experience success." So they were basically saying when we have happy developers and uh good developer experience processes they tend to write better solutions and so if we're doing DevX right uh what we deliver out of the other end you know that our end users our customers uh are picking up is also better they can see you know the the correlation between the two and so they were tracking that as part of their overall metrics and business case saying hey we invest in this developer experience stuff that customers, our external customers get a better experience too, which I I genuinely loved. It was one of those moments when you hear somebody say something and think that's awesome. That is exactly how it should be. That full holistic view end to end. I see a you nodding along excitedly, so I need to hear what you have to say. Yeah. Yeah. Um that's so true. I have an um an example from when I was at Factorial that kind of blew my mind and everyone else's. We we transitioned to a cloudbased um preconfigured development environment. And before that time, devs would have to run several commands to have their sessions open and use all the internal tools just to get their dev environment started. And this was for devs who were already with us. Then you can imagine how long it took for onboarding. So we transitioned to this cloud-based um preconfigured um environment and all you had to do was go on GitHub and click a button that said open and it opened an um a web instance of of um VS code and then all the terminals were already started and you would just see a link to the app and they just clicked on the app and they found everything right there. So this meant that on body time was cut by over 90%. Time to first PR could be in hours as opposed to days because you just needed to add them to GitHub. They just clicked the button and all of that was done. So we had desks tweeting about this. We had a glass door review that stated that the company having a dedicated DX team was a pro. And I think that's when it dawned on us that this wasn't just something we were doing internally. It was gradually becoming our brand that actually we do support developers. So we could see that EMPS improved for um engineering developers were happier. Retention was there of course because they were happy and as Martin said when you are not hiring because you are retaining the devs that means you are actually cutting cost. So for us that was like a good instance of where investing in the X actually led to to tangible business outcomes. Let's be honest it helps with recruitment too because it's kind of the t-shirt test. People won't wear maybe they will on a call but they won't wear to the gym or tweet about their company if they don't love it. So yeah. Yes. I'm completely aligned with the with the rest of the team. But um you know because once you you invest in the in the hiring and you want to to scale up uh let's say or have more people um is for a very um well justified reason, right? you have a need that you need to fulfill and the on boarding time is the the dimension that you need to shrink in order to to have these people um to left city let's say uh the activities and have these people solve the problems that is the necessity of you know and the justification of why you are hiring more people so this is a very important dimension I think and let's be honest speed to on board is you just a lot of companies it can be three months. It can take a month sometimes to get a laptop like you do not feel good at your job. You already feel vulnerable in this economy in this market and then as you start a new job. So I know for example at Spotify that was one of their goals. I think onboarded like definition of onboarded is that you did your 10th pull request. So it went from 60 days to 20 days. So all these companies are benefiting. That's a really good thing to measure also because that's a lot of inefficiency and let's not forget uh one thing that we haven't talked about that much is discovery information discovery which I find tends to be in the last 15 years all I've heard complaints about from do from developers is documentation and in the last 10 years it's been technical debt and these are things that AI are actually useful at explaining complex systems and finding information. And when that equals a day, a week you spend finding stuff that's demotivating whether you're and I'm sure it's higher. It's a probably hopefully something that goes down the longer you're within a company and you have that institutional knowledge, but when you're starting, all you're doing is trying to find stuff and that's very frustrating and that's a waste of a very high hourly rate. So, we've talked about all these great success stories, but one of the things that's going to help you make that business use case and to build that trust with developers is measuring the impact of everything you're doing. So, can we talk about any qualitative or quantitative metrics that you're using to track all of your developer experience? Christos, do you want to kick off on that one? Uh, yes, of course. Um so before going to the measuring part I think we need to to pay attention to to something also important is is a you know we discussed a bit about the trust that we have to um to have between the developers etc. And this is somehow correlated to to how we are measuring results, right? Because there are some occasions that developer experience or developer productivity h is somehow misconcepted, right? Is somehow correlated with measuring outcome or performance. But most of the most of the times shouldn't be this case, right? So we need to um we need a mental shift. we need to set up the ground and uh make you know the whole team aligned on why we are you know investing in this kind of of activities. Um this is I think the baseline. Then we maybe uh we start as I said with some qualitative and quantitative data. we're picking our our um cases and then in terms of the evaluation and the actual measurements you can use uh you know main stuff from Dora metrics if we are you know want to um to dive deep a bit in the delivery aspect let's say uh to more um quantitative uh data like surveys uh to hear the actual uh developers problems or to even um compare between snapshots. So let's say that uh let's consider the previous example regarding CI/CD and that we needed to to improve this this process. So we have a snapshot we are measuring through DORA uh the time to first commit or whatever and then after the the decision and the actual items that um and action items that we um we did we compare the new result with the previous um result from the previous. This is somehow how we can uh check the progress and also another interesting part is to also try to uh to measure um the engineering level compared compared to the rest of the industry. So if I'm in the budgeting sector, how the budgeting sector is performing in this in the same let's say um measures uh Dora metrics whatever and then to you know to elevate a bit the uh the the process to compare it with different you know sectors that may be better in some uh in some specific cases. So I think that's that's all from my side. Do you keep these metrics secret or do developers see these metrics? How are how are business over these metrics? Who sees these metrics? Thank you for that. I think that you know help me remember something that I missed. No, no, it's complete transparent and this is also another aspect of uh building trust with developers, right? I don't want to have um uh let's say special access or specific access to information that developers cannot uh cannot have. Ideally in my world of developer experience this should be open to all of them to not only to compare let's say or check uh between the teams but between the different departments at the same company. So I would like to have you know the same access into into data despite of the of the role and the the level that you you are in the company. Right. Okay. And Martin especially how do you then take this information but also advocate on the developers behalf that they can do these things that are not necessarily shipping features that they have time for that. So how do you what other questions are you asking? Yeah. So that that literally where I wanted to kind of carry on because I always feel like durometrics is a great place to start. But durometrics especially if you're talking to a seuite doesn't necessarily make a lot of sense to them. And so what you need to do is be able to translate that into something that is meaningful. Often uh a good way to do that is to understand uh what the business drivers are you know in their current you know fiscal period. So whether that is actually you know we want to improve our MPS score because we have lots of customer race bugs or actually we want to try and maintain costs at current levels but we still want to deliver more features right and so then what you need to do is you know look at the things in the dorometrics that that you can measure and report back on that show improvements right so you know if they want to I know for example maintain current costs but deliver more you can say well actually you know what if we reduce the amount of time that developers are spending say maybe with CI/CD or maybe that they're spending uh manually documenting versus say auto documentation or or the time that they're spending you know uh remediating security issues you know here is an investment case that says hey we can invest this a little bit now I know you want to you know save money but over the long term this would you know we should reduce this time spent by I know say 10% that's 10% across you know 500 a thousand 10,000 developers uh suddenly becomes you know a lot of hours and you say hey but that actually flips right because now that means they're spending more time innovating and solving the problems of the external customers right so you know you you have to be able to translate those but I think finding the metrics that actually um make sense to the business goals right you have to be able to translate those between the two uh so that because you know generally when you're talking especially to kind of seuitees or directors or you know often uh the language of the developer doesn't always make sense to them right so you you need to be able to translate those things you know so um you know whereas we might talk about code quality uh and QA quite often you know as far as the they're concerned that's the MPS or bug leakage I know I've worked in businesses and they would call it bug leakage so how many so we would measure for example how many bugs do we find when we do our testing versus how many bugs the customer finds in that uh for every release that we do and then you have that like that graph of hopefully we find lots and fix them and the customer finds very few but that percentage makes sense because they know that when that's really low the MPS is improved so improving those processes is a good investment for them so you know there's there's lots of ways I could keep Also, these you're getting developers back to doing interesting work, not what I like to call bang your head up against the laptop work. Yeah. And that's kind of the point across the board. And something we haven't and I can't believe I'm the one mentioning it, but AV like and everyone, we're talking about AI. We're not talking about AI, but how are we not? Because when we're talking about the business use case all I'm hearing from sea levels is those two letters. So how can you then measure uh these whole ideas and do testing and apply this methodology because AI affects the developer experience whether positively by hopefully improving productivity or increasing technical debt and place and the time to find bugs. So, how do you apply measurements there? Okay. Yeah, that's that's an interesting question and I think that we've already um slowly transitioned into how people can make a case for the X to the seauite. Somehow we just transitioned into that and I think that it's crucial to be able to translate the X improvements into clear business outcomes. So it's not about saying, "Oh, we need this new AI tool, but why? How does it impact the business? What is it going to help us with?" For example, um you did say something earlier, Jennifer, about information discovery and how that's the thing that you hear all the time. Now we know that AI can help us with um summarizing a codebase and explaining to a new joiner to say oh this is exactly what um this domain is about. AI can also help you write your test cases. It can help you um identify bugs and all of that. But the thing is yes it can do these things but how do these things translate into clear business outcomes which is so I think the key is to be objective and datadriven the the goal ultimately is to be able to say that now because developers don't spend two hours every day searching for documentation they can invest those two hours in shipping features and that means means that time to market is improved. So being able to translate those clearly into key business outcomes, that's the way forward. That's exactly how you can make a case for AI or any other tool. And these tools that we're giving developers, this information can help them finally learn this language because it is a I would dare I say a almost completely different language, business versus tech. But they have to work together. But also it helps give developers purpose because they're understanding the business. They're connected to the business. So that's great. So we had a question from the audience that I thought was pretty interesting and it felt like a good one to wrap up. Uh what is the average time that it takes for a developer to complete onboarding with your team? That is a great question. Uh that's a great metric. And I want to put it to y'all if you are willing to answer that question. Go if we know that or how to and how you figured out the answer to that question would be very useful to those on the call. I think um I can go first. So I think that measuring on boarding is actually in different sections and phases. There's the time to first PR that okay this person has been onboarded but there's also the um understanding of the domain and that also takes time at what point are they comfortable to explain the concepts in in their domain. So onboarding is in different levels but let me speak to one of them which is time to first PR. So for engineers you want to say that okay they have been perfectly onboarded in this regard if they have their dev set up they're able to push their first PR they are able to merge and everything is in production when I was at factorial and we moved to the um the cloud-based M in a day a new dev would be onboarded have everything set up and they could actually contribute at Cleo I remember onboarding three graduate engineers and they had their first PR on the third day and it was a win because they were new engineers. This was their first job. They needed a little of support. So they had everything ready on the third day. The third day is when they they pushed their first PR. So this means for experienced um devs it could be less. Martin. Yeah, I I I was I was actually just thinking through that process in my head of everything that that like makes up all those steps. And you're right, there is different levels of it, but uh I will stick to roughly the same measure uh that AV is is talking about there. And I I I would say generally um we do have to do some compliance training in that forms part of that kind of you know before they can do their first thing. So generally I would say it's day it's day two day one they have everything. In fact one of the things I love and I know we're not the only company that do this you know I love that when somebody starts when they accept the job they get the things like oh what laptop do you want what you know so that actually all of those things are on the day you open your laptop you do your first login and everything is there for you already. And by the way, here's the list of those few little trainings you have to do. Here's your first meeting, you know, having all of that kind of, you know, that's all part of that onboarding process. But actually seeing it, you know, and then having like, you know, the cloud development environments, access to the code, all of that is already sorted so that you are, you know, able to commit quickly. I I think that's it's it's good. But I would say generally it's a day two. They get everything they need in day one, but by the time they're by at the end of day one, it's going to be day two before they can actually Yeah, absolutely. Christos, to finally add on this topic. Yes. I I think you know if we we take a very a simple case of just you know pushing a a change a simple change in the production maybe it's day one or day two but I would say it's a bit of more like one week in order to to deliver something meaningful right I mean to also having this kind of try to understand the business aspect that takes time I mean months maybe as I also also commend it and I will insist on you know the first week because the the tricky part here is to to take let's say a thumbs up by by the the PR reviewer for the business case for for the business aspect of of this. So I would say maybe you know fourth fourth day for this in order to understand and familiarize a bit the code. Not bad. That's not bad at all. No. And I just want to add another different kind of metric which is response rate to your surveys. That should be going up. If that's going down, you are wasting your time and money on the survey because no one believes you're going to do anything with it. So involve your developers as much as possible. In the entire world of developer experience, they are your customers. That's means you can't make them use what you're building. You need to build what they want to use. So constantly have that conversation and then I hope this helps you everyone on the call start that conversation with business to kick off your developer experience journey. I recommend you connect with all of us on LinkedIn so we can continue the conversation. There were some questions that we didn't get to because we ran out of time but I am so grateful to every participant. Thank you, Martin and Christos. And thank you to our listeners and lead dev for having this conversation and harness for sponsoring it. Have a great day, y'all. Good luck., How Better Developer Experience Drives Results, LeadDev, 01jvas8bfygczabxnek8wed9k4.