Europe's AI Build-Out  🔗 **Source:** [Europe's AI Build-Out](https://www.youtube.com/watch?v=ntgm4Aw8xPc) If you're building in Europe, think about multiple things, right? Not choking GPUs, tearing your storage from nearline to to hot storage, capacity planning, access to power and ultimately start thinking about uh you know becoming macroeconomic, supply chain, architectural wizard and psychic. uh welcome our audience. Thank you guys for taking the time and joining us uh for the 30 minutes panel. I know time is precious. Uh and I want to welcome our panelists uh four brilliant people representing global massive very important companies for this uh sector. Please guys take a minute to to introduce yourself. >> Yeah, thanks Martin. Uh Sean Barley, chief evangelist at Amper Computing. We're now a soft bankank company and uh also working very closely with our sibling company ARM who you probably know. Um but yeah, I basically look into all of the uh enabling of customers and the ecosystem to kind of bring out the best in our technology and of course in the age of AI that's we do a lot of. >> Hi everyone, good afternoon. Roger Carell. I'm with Solidime and I head up uh something called our AI and leadership marketing team and I want to give a shout out to a lot of my Solid Posi over here. Uh thanks for the support guys. Um and we're we are Solid is a pure play enterprise SSD company. So every single resource at the company is hyperfocused on building the best drives for enterprise. Uh we're also very focused on AI. Um, and we're really excited to be here because we kind of think Europe is um, consistently takes the lead in how to do efficient buildouts and there is a storage opportunity there. There's a storage play there. So, we're excited to be here. >> Yeah. Um, my name is Vincent Lander. I'm actually from Europe. So, yeah, nice to see some people that I know and I've been working with for many years. Um, I'm from Seagate Seagate Technology. I think most of you know from hard drives but also from storage systems as well as uh cloud object storage. So everything that is storage related which is highly relevant right now in the era of AI. So uh I'm very proud and happy to be part of this panel today. >> My name is Prasad Aluri. I'm with Micron Technologies. I lead our rest of the world data center business. uh prior to taking this role I ran our corporate strategy for about three years. uh you know Micron Technology is the leader in memory and storage technologies and you know AI is clearly the thing that actually gave us u lot of the visibility and you know you probably have been hearing that you know uh memory bound computing and storage bound computing and so we are truly the ones that are you know the center of the AI revolution now >> thank you very much. Welcome gentlemen uh our audience. Thank you guys again for joining. I assume you understand that we're going to be talking here about AI specifically with a focus on Europe. Uh there's a lot of things going on and I'm assuming that's why you're actually here to listen and to to see what uh what our specialists have uh have to share. Uh our goal is to discuss uh not the press release versions, not the uh greenwashing stuff, but really to look deeper at uh um the constraints that Europe offers today, the current situation, what's happening where the the flap D situation, which is almost a legacy concept, right? uh uh Paris, London, Frankfurt, Amsterdam, Dublin, constraints in power, uh lack of power, legacy facilities that can't even like even going as deep as not being able to have the flu uh floor being able to uh to carry the loads that that modern infrastructure requires, weight of uh modern C uh GPUs, the entire stacks, liquid cooling, all of these things together uh require completely different u concept from design perspective and modern uh infrastructure executive really becomes uh a lot less focused on the technology itself but on commercial aspects as well as the fact that they need to be uh basically a visionary on supply chain perspective and forecasting and future planning and basically the the psych the the the cycles of purchasing that were very comfortable quarterly maybe yearly base that's all gone away. Now you're going to have to be dealing with with things that procurement of two three even beyond that capacity. So without further ado let's uh see what our guys have to say. Uh the first question when you look at uh Europe right now what is the real limiting factor in your from your perspective in AI buildout is the GPU's power or time to sight >> all of the above. Um but I mean that's a serious answer. They're all big concerns. Um I think you talked about some of the flap D kind of the core data center areas. You know, when you consider that literally getting that first um piece of IT equipment plugged into power, plugged into the grid can take 8 to 10 years. I don't know, maybe power's got an argument is in terms of being the the top challenge. Yeah, power definitely number one in Europe and we see um the Nordics we see a big uptake now a lot of data centers being opened there because power availability but also uh connectivity to the grid right which is another thing is is uh more available there than in in the cities where or in the locations where you just mentioned um also in Spain you see u more uh data centers being being built for that for this reason power availability follow the power. >> Yeah, >> say power and time to sight are both the limiting factors. Um, of course GPU supply is definitely you know tight but what governs the scale of AI in Europe is you know the availability of power cooling solutions and also permitting. you know Europe is known to be quite slow in actually overall you know permit process and all of that. So all all those three things will determine you know how fast the buildout is going to happen. So if you think about the implication to you know from our perspective from Micron's perspective what does it mean it's you know the conversation now changes to how many GPUs to how fast you can feed those GPUs right so if you look at the overall infrastructure when bits are moving across the infrastructure the amount of energy consumed roughly twothirds of the energy is actually consu consumed by the bits moving across Compute only takes about a third you know in that regard actually compute is cheap. So now again coming back to Micron's perspective on it you know because of the fact that the energy consumed is a lot more by the data movement um you know memory and storage actually falls right along in that path and so you know looking at the energy efficient solutions and making sure that uh when you are architecting how you actually uh use memory and storage is going to be helping you in that power effic efficiency. >> I really don't have anything to add to that except for I do want to poke my friend Prasad because he didn't mention the current, you know, memory capacity problems that are also limiting everybody in the world trying to actually, you know, get compute out there and up and running. But I don't know if you have anything to say. >> I didn't want to poke more pain, cause more pain to people. I mean, they're already in pain. So >> all right um I completely agree but that moves us to the next question. Where is the AI capacity actually landing in Europe today and what does that tell us about the next weight of site selection me? >> Go ahead. So if you just look at the overall data center energy consumption itself you know right now annual consumption of data centers is somewhere around 600 to 700 terowatt hours by the end of the decade it's going to get to close to about 900 terowatt hours about a third of which is going to be consumed by modern AI data centers and Europe is probably going to have a meaningful share out of out of that about 15 to 20%. And uh you know the three things that are probably going to decide um where the buildout is going to happen in Europe is probably at the intersection of again coming back to the power availability and cooling capabilities. And the third and equally important is uh alignment to sovereign interests. Right? It's all those three things. >> Yeah. >> The intersection of those three things is where the buildout is going to happen. So more specifically if you look at um you know where those three things intersect from a uh if you look at the Nordics for example because of their scale of power and uh cooling capability training is being you know is is being focused there in that region >> and then UK and western part of the uh Europe is lot more skewed towards training and so I think the next phase of the buildout is probably not going to be one singular thing probably it's going to be governed by multitude of things and you'll have to probably optimize for the different you know workload be it training or >> absolutely >> yeah I um I was just going to kind of build a little bit on what Prasad said um from our perspective we kind of look at it as a uh strategic bifurcation so you've got the flap D regions that's where the population centers are that's where latency matters um you know maybe a little bit less uh power intensive in terms of the compute workload and then as Prasad points out you know power availability uh maybe even space availability more maybe less regulatory we see secondary markets um you know Nordic states southern Europe very similar thing going on now in the US in terms of Virginia's kind of doing their thing they're running out of power and here's Texas land power availability alternative fuel sources etc ETA. So kind of similar bifurcations between North America and Europe here. >> Yeah. >> Yeah. Certainly with I mean if I may add it here as well uh we definitely see Europe Nordics uh building their uh fortresses so to speak free cooling almost there right? Uh happening uh a lot less >> Iceland as well. Yep, definitely. Uh, southern Europe, right? >> Barcelona recently with Core Weave in the Merlin data center. So, yeah. >> Yeah, it's it's what we see and the the same thing happens exactly in the US, right? You see the the Southwest gaining power, right? uh in in the space. Uh but definitely in Europe we're talking Nordics, Iceland, uh uh Spain, uh Portugal, Italy, just moving from the flap D, uh especially if you want to do training, etc. because uh it's easy to really run cables, but you can't just will by sheer will produce additional, you know, 2 megawatts of power, >> right? >> That's very difficult and it takes many decades to build, you know, reactors, etc., etc., right? Uh so considering Europe is power constraint, where should buyers look first for efficiency gains? And this is where you guys really come in. Compute, memory, storage, >> all of it. That's what you're going to say, right? Well, I was going to say I'm from a storage company, so of course I'm going to say storage. But in all seriousness, you know, whether you're looking at how do I maximize my compute density in terms of direct attach storage, certainly moving towards um direct to chip liquid cooling uh for um you know for that direct attached storage. We're very proud of the partnership uh that we had with Nvidia and bringing the first solution to market for that. uh kind of um you can do either approach air cooled storage or liquid cooled storage in the uh GB300 area but you move to Vera Rubin Ver Rubin ultra you better start planning for liquid cooling um yeah so oh and then on network attach I mean that is all about density and power efficient performance and take that footprint take that power savings take that infrastructure savings and give yourself an option to scale um you know to have compute scale within in whatever power budget you have from the grid. So that's our perspective. >> Yeah. From a storage perspective, since I also work for a storage company, I will I will take that one. There's a lot of new technology being developed. Looking exactly at what you mentioned, right? We're looking at power consumption. We're looking at data center footprint. Uh we're looking at higher capacity points um per basically per wreck. Uh that's exactly what we are working on in the industry. making sure that we are reducing that cost per terabyte when it comes to power consumption that the footprint of the data center is getting smaller. So when planning ahead look at the latest technology right that that is extremely important uh the being released a couple of new technologies in terms of hard drives uh that are extremely relevant for the AI workloads doing exactly what I just mentioned. So from my perspective, there are probably two things to consider when you're building the AI data center. Power utilization efficiency is super important. This is at a macro level, right? How you actually architect your cooling. If you're off on your PUE, you can easily be off by 10 to 20%. Right? And so since you are a data center operator, you probably understand that very well. So that's a macro level. But now coming down into the data center itself uh you know how you architect uh to optimize for each of the workloads you know how you disagregate the infrastructure itself to run a specific workload is going to determine how efficiently you're going to be using energy. So let me give you one specific example, right? So if you look at what's happening on the inference case, I want to use that as just as an example. You know there are parts of the inference that are compute bound. So in that case you know you don't want to you know use something like a HBM type memory and feed you know you're you're basically going to be wasting the the energy and you know instead you are better off probably using something like a low power DRAM um you know to to feed the compute. Now within inference there is also going the the stage where it's more of memory bandwidth bound. So in which case of course you use um you know the the high bandwidth memory HPM to feed the compute. So now you you'll have to actually segregate these both stages and build the infrastructure and orchestrate your workload to be placed in all in both of these types of infrastructure and that's how you achieve overall optimization. So I think those are the two I would like to point out. name of the game is of course, you know, feeding the GPU, but feed it efficiently. >> Yeah, >> I like to take a little bit of a different u, you know, track with this one. Um, for me, we can always bring down individual component level um, you know, power and and raise up the efficiency. Another way to say that, right? But I think what's facing us as an industry right now is workload planning. Now, let me say more about this. when we're looking at uh infrastructure that is designed to deliver world models for example right and what I mean by a world model is something that's trillions of parameters right that's going to be a a GPU inensive delivery infrastructure but that particular um sort of planning exercise is probably not going to be space constrained it's going to be power constrained And and it might also be, you know, constrained by some of the bit moving pieces that you were talking about earlier, Prasad. But if you start to look at what I've been calling lately, I've been calling business productivity. Business productivity is where enterprises and new business comes in and they take advantage of AI in its newest form. Right? These are going to be inherently specialist use cases where smaller models are going to be utilized. Many many more models in conjunction with one another are working together. Those types of workloads can actually be processed by CPUs which might be a lead into your next question. >> Yes. >> And um and that particular kind of planning is going to be very different because it's going to be a combination. It's going to be a combination of GPUs and CPUs and all of those sorts of things. in which case space might still come into a consideration. Those sorts of things are going to be very different. So it's going to be workload planning. That's going to be my answer. >> Heterogeneous environment that you'll have to cope with. >> Y definitely need to look at whether you're you're in the business of training models. You're dealing with frontier models. Are you running rags where you know vector databases? Uh how much memory you need? some of that can be possibly offloaded to uh to hot storage as well because otherwise it's you know like I was saying before zetabytes of of data if you wanted to store it on on uh hot flash probably entire globe couldn't pay for it >> right uh so there's definitely storage tiering workloads uh tiering uh and uh you know small models uh mixture of experts right all of these things uh definitely require future planning how you do your storage >> happen in real time right with uh Nvidia's G3.5 uh context memory extension um I mean just speaks to you know everybody I think um has talked about the industry at large has talked about data set size for training um but let's also talk about data set size for inference and increasingly long, you know, uh, chain of reasoning, um, you know, endless context windows, um, rag databases, etc. And so, I mean, the industry is is in real time rearchitecting that storage stack in response to it. So, you you have I think the industry is in a position where now where like storage was an afterthought >> and now it's like it's in the critical path. Yeah. like you better start treating storage as as a strategic asset. >> Yeah. >> And how can it help you achieve your business goals? >> Yeah. What's the what's the figure? Is it like 10x um to vectorize, you know, a piece of data and then get it vectorized into ra databases? >> Roughly 10x. Yes. >> So >> I agree with Roger. Very similar um thought process. So if you look at what's happening you know from a storage perspective it is real time evolving and so there are probably the best way to look at it is pro you know there are three tiers so the first is you know um direct connected storage with u with the GPU itself compute more generically >> so when you're doing snapshotting of the data or you know you're doing checkpointing you need to have nonvolatile memory directly connected to the to the to the compute itself and so that needs to be highly performant because you know the bandwidth matters >> absolutely >> and uh energy efficiency matters and so their you know gen 6 and leading gen 5 type uh SSDs is what's going in that particular tier and the next year is about you know the this is more evolving and as Roger mentioned uh this is the you know KV cache Once the context sizes grow, the KV caches are becoming so large that you can't actually fit them within the memory, not in the HPM memory, not even in the CPU connected, you know, head node connected memory. So you'll have to, you know, shift some of that KV, you know, key values to uh storage. And so that's going to be residing on the east west, you know, portion of the traffic. And then uh of course you know you still have to you know have the capacity storage for all the all the data links and you know uh all the long old contexts that you want to store and all of that has to go on the north south portion of the storage. But one overarching statement I would like to make is if you look at the modern architecture AI data center itself it's actually the AI native architecture is evolving on all flash right so that doesn't mean hard this drive is going to go away it still is you know you still need to archive the data and some of the older data that still needs to be there and so enterprises are are still going to use some part of the hardest types but I think the AI native stuff is all all flash >> I think right now around 80% % of any data center environment not specific specifically only AI is still running on hard drives and we we don't see a change there that's not just related to performance because I think the SSD tier has its own right there for the performance part the data lake part that is all about scale and about cost and if I look right now how difficult it is to plan for us with our partners to get enough availability of drives into the market we are we are now forecasting almost a year uh ahead right so um I don't see any changes coming in the in the very near future uh when it comes to the taring and also the importance of hard drives for exactly what I just mentioned the data lake environment yeah absolutely if if you're in a business of uh of training right checkpointing is is lifesaver it's if you're not implementing properly right skimping on that will cost you millions easily You don't want to be recmputing the whole thing. I mean that's again >> goes back to inefficient use of absolutely >> energy. So >> yeah right. So that gets us to the next question. Uh there are many cases that not every AI workload needs a GPU. >> Help the audience to understand where CPU based inference fits in the stack especially in a power constraint European market. >> I like this. >> Yeah. Go ahead. >> I led into it for a reason. Uh no I mean seriously some of the things I said earlier are the considerations right and and this is going to be changing right um one of the things that's changed I think radically is the use case around AI this is not some you know singular monolithic you know beast we're talking about models that range from tens of millions of parameters to trillions right and it's an exponential scale so essentially what you're when you're when you have computer vision for example, right? These are actually very reasonably sized models. They're they're smaller than what you know we're you know just banding about with you know 8 billion level you know kind of models but lots of models lots of models lots of specific models being trained lots of uh fine-tuning going on around enterprise specific data which also brings it down you somebody mentioned mixture of experts earlier yeah Martin thank you I mean all of these sorts of things are factors in reducing model size what does that do that makes it much easier for lower memory bandwidth types of devices to actually compute those models and and so now you get into something that's a much more um easily uh you know kind of built infrastructure first off right it's very easy to manage the cloud technology that exists around CPUs is decades old right it's very mature we're talking about virtualization of virtual machines. We're talking about containerization, orchestration, connectivity to storage, everything, right? So that smaller model kind of um universe is really really ripe for CPUs. And now you start to see it coming back into vogue. Even Nvidia is talking about it with Vera, right? So now we're looking at, you know, kind of a landscape that spans CPUs all the way up to the biggest GPUs. And it really kind of depends on what kind of thing you're trying to compute. And so that's where we'll see CPUs kind of come back into their um, you know, their heyday uh, or at least back into respectability. From my perspective of having been a CPU guy for a long time, we sort of lost our mojo for about five, six years and, you know, just sort of took a backseat to GPUs. Everybody wanted GPUs, but now they're coming back. And I think you'll see that that evolves with what I'm calling business productivity AI, not world model AI. >> Hey, you haven't lost your hair. 25 years since I've known you. >> Hair is amazing. >> You still have your you still have your mojo. You haven't lost. >> Thank you. Thank you. I appreciate that. >> All right, gentlemen. We we're running out of time a little bit. So I'd like you all uh to take maybe 45 seconds tops to uh do some final thoughts something that you want our audience to thank take home and uh when they wake up Monday morning pray you know after they brush their teeth what should they be thinking of business side what to do next after this session >> go this way >> so um rethink your architecture in one sentence So you know we've talked about Europe being energy limited or you know power limited. So truly thinking about energy efficiency how you can achieve that and you know we talked about perhaps uh the way to achieve that is you know focusing more on how to minimize the bit movement which consumes majority part of the energy as I said the pjles per bit you know roughly 67% of uh the energy goes in just those that movement so think about you know architecting in such a way that the movement is minimized And there is memory and storage that plays big role because it's you know it's it it it goes in the way of that bit movement. So that's all I want to say. >> Thank you. >> Yeah, we we we we named this or we call this a panel discussion but it's almost a panel agreement that we have here >> which is not a surprise right we we seeing the same things in the market happening. Uh so to answer your question very in line with what you just mentioned um it is all about all about power to start with of course it's about your cooling your thermal plan and definitely and now I'm talking about the storage bit look at the tearing how you plan your storage look at your your your flash your NVME layer to work with your CPUs to make sure you don't have CPUs idling use time efficiently and of of course your data lake tier underneath that for uh the larger data sets on a more cost- effective and scalable platform that is what I would like to uh to add to that >> not much to add maybe a way of uh I'm in marketing so maybe a repackaging um start thinking yeah here we go uh I don't know think of storage as a strategic asset right you've you've got objectives uh in the AI buildout in Europe Um, and so what are your power constraints? What are your space constraints? What sustainability concerns did uh do you have? Um, what's your water usage effectiveness goals? And storage, believe it or not, storage has a role in all of those areas. And it can do it in a way that keeps your GPUs utilized um and um and it done in the most efficient way can save power and space to enable GPU scale within whatever your budget is. And and there is a role for storage to help overcome or help address all of those really tough challenges. >> I already said it. Workload planning. Workload planning. What are you building for? Right? And that's going to dictate almost all the other things that my colleague said, right? Whether it's the storage capacity needs or the power and cooling needs, all these sorts of things because if you're building, you know, uh, Vera Rubin clusters, you're going to need a lot of power. You're going to need a you're not going to probably need that much space. Actually, there's a lot of of of data centers in the United States that have basically given up square footage because they ran out of power way sooner than they ran out of space. >> So, workload planning. Um, otherwise you're going to have a lot of empty space. Uh, and I think that what goes along with that is some attention to kind of the next generation of technologies because believe it or not, there's a whole another raft of technologies coming at us that is going to drastically change how inference gets done and inference will be the workload 85% of the workload going forward. >> Thank you gentlemen. Um, let me close off uh starting with uh what Sean just said. Inference is pretty much if you're not a hyperscaler, your play is going to be inference. That's where you're going to make your money. That's where your opportunities lie. Uh with that said, if you're building in Europe, think about multiple things, right? Not choking GPUs, maybe not necessarily needing to use GPUs. uh tearing your storage from uh from nearline to to hot storage, capacity planning, access to power and ultimately start thinking about uh you know becoming macroeconomic supply chain architectural wizard and psychic. >> Thank you so much for your attention. Thank you for joining us guys. have a wonderful uh rest of of the cloud fest and hope to see you on uh any other sessions that we attend. Thank you very much., Europe's AI Build-Out, YouTube, 01krc7kd06xp14xv1mk43yapmk.