Where Is All the A.I.-Driven Scientific Progress?  ## Document Notes ## Highlights - **AI That Feels Like A Scientist** - Cosmos feels like an actual AI scientist that delivers deep, sometimes wrong, insights about 80% of the time. - Edison measured Cosmos by giving it datasets and objectives and finding it reproduced months-long human results overnight. Transcript: Sam Rodriques Yeah, yeah, exactly. And actually, I will just start out by saying that when I got that six-month number, my reaction originally was like, there is no way that this is true, right? And we've now measured it in a bunch of different ways. I can walk you guys through that. But basically, just to take a step back, so we've been working for two years on figuring out how to build an AI scientist. And the concept here is there's so much more science that we can do than we have scientists, right? And so how do we scale up science? And the thing that happened with Cosmos that is pretty cool is Cosmos is like the first thing that I think that we've made that actually really feels like an AI scientist when you're working With it, right? Which is to say that you go in, you give it a research objective, it goes away and it comes back with insights that are actually really deep and interesting and sometimes wrong, but about 80% of the time, right, which is like kind of similar to like, you ask a human to go away and do something, comes back like similar percentage of the time is right. And it's like it's a kind of new experience working with it. So that's very exciting. The six-month number specifically, the way that we measured this was we had a bunch of academic collaborators, you know, scientists who had done a bunch of science previously that They had not published yet. And we basically gave the same research directive and the same data set to the AI, to Cosmos, and we asked it, you know, to go away and just make new discoveries. And it would come back and it had found the same things that the researchers had found overnight. And then you go and you ask the researchers, you know, how long did it take you to find this in the first place? And they would say like three months, five months, like six months, whatever. ([Time 0:08:28](https://share.snipd.com/snip/b49d67d3-bc9c-4fcb-a784-a71ec174d4c5)) - **Use Structured World Models** - Build long-running structured world models so agents can maintain state and coordinate complex tasks over hours. - Orchestrate many parallel and sequential agents toward a coherent goal rather than relying on single short prompts. Transcript: Sam Rodriques So it is indeed a box that you basically type into. You ask a research objective. It's not a chatbot, right? It runs for 12 hours or so before eventually coming back to you with its findings. In terms of how it's built, we build on top of a bunch of different language models from OpenAI, from Google, from Anthropic. Like in any given run, we use models from all the different providers. We also have like our own models for specific tasks that we've trained internally, where those models are like much better for the specific tasks that we train them on than the models That the frontier providers make. Got it. And then the key insight in Cosmos is basically this use of what we call like a structured world model. So one of the main limitations with AI systems today is that they're just limited in the length of the task and the sophistication of the task that they can carry out before they kind of Go off the rails. They like, you know, forget what they're doing. They no longer are on task. And what we figured out was a way to have them contributing to this world model that gets built up over time that basically describes like the full state of knowledge about the task that They're working on, which then means that we can orchestrate hundreds of like different agents running in parallel, running in series, and have them all working towards a coherent Goal. And that was like the real unlock. ([Time 0:10:29](https://share.snipd.com/snip/b4f3d77f-ab97-443d-8982-1b1d2803c778)) - **High Compute, High Value** - A single Cosmos run reads ~1,500 papers and writes ~42,000 lines of code, which drives its high compute cost. - Scientists often view the service as cheap relative to the thousands spent gathering experimental data. Transcript: Sam Rodriques Answer is it uses a lot of compute, right? Give us a sense of how much. Well, so an individual run from Cosmos will write 42,000 lines of code and read 1,500 research papers on average. Like if you run Claude, it might write like a few hundred lines of code, right? ([Time 0:12:00](https://share.snipd.com/snip/38028ed3-6802-47da-b4f3-045601c3ce54)) - **AI Finds Mechanisms In Genetic Data** - Cosmos produced several novel contributions, including identifying a mechanism for a noncoding genetic variant linked to type 2 diabetes. - The model connected protein binding, gene expression, and pancreatic function to propose a plausible mechanism. Transcript: Sam Rodriques So like one of the ones that we really like, the human genome contains millions of genetic variants, right? These are differences between different people's DNA that are associated with disease. And for the most part, we know that a variant is associated a disease, but we have no idea why, right? And so we asked Cosmos, we gave it a bunch of raw data about a huge number of different genetic factors. So like what the variants are, what proteins bind near the variants, right? Like all these kinds of things. And just asked it for type 2 diabetes to go and, you know, identify a mechanism associated with one of these variants. And it came back and it identified this was a variant that was not in a gene. And Cosmos identified that this is actually somewhere where a different protein binds. It was able to identify what protein binds and what gene is being expressed and connected that to the actual mechanism of that gene, SSR1, which is involved in the pancreas in secreting And so on, right? Casey Newton Okay, so in this case is what I'm hearing that your model was able to do some very fancy reasoning over some existing data and identify something that sort of no other human scientist Sam Rodriques Had gotten around to and might not have for a really long time. Yeah, that's right. Okay. ([Time 0:14:05](https://share.snipd.com/snip/0c84ac09-7163-46d1-aca5-a10cc78f7422)) - **Validate AI Findings With Experiments** - Always validate AI-generated discoveries with experiments and cross-checks before acting on them. - Treat AI outputs as hypothesis-generating steps that require human review and follow-up experiments. Transcript: Sam Rodriques You have to go and validate it. And so that's actually one of the things also in the paper where actually we describe how we went and validated that particular variant. In general, when people are using it, yeah, you go in. I mean, actually, literally, when you run a Cosmos run, the first thing you have to do is you have to understand what it's telling you. Because it has just done something that scientists think is like six months worth of work. And you're going to sit there for a long time just like reading and understanding it. Once you've read it and understood it, then yes, indeed, you're going to go and you're going to run, you know, various experiments, do your own analysis, cross-reference to try to, Like, convince yourself that this is true. And then based on what your research objective is, you'll decide next steps, right? You know, in this case, I think it's probably low likelihood there's a new drug target, like, from this particular finding, right? But you could go and you could run this on other findings, and then eventually maybe you find a new drug target, you start a drug program, that's, you know. ([Time 0:15:51](https://share.snipd.com/snip/e86c00e9-dc9a-4150-988a-8fc419eb79ba)) - **Trials, Not Ideas, Often Limit Progress** - Clinical trials remain the fundamental bottleneck for translating discoveries into human cures, regardless of AI speedups. - AI can improve trial design and surface hidden insights from existing data to make expensive experiments more effective. Transcript: Sam Rodriques So absolutely. I actually like, you know, I really agree that like the bottleneck at the end of the day in solving medicine is basically, you know, clinical trials. I mean, and the easiest way to see this is if you look at the number of diseases that we like know how to cure in mice, right? It's like astronomical because obviously you can just like run experiments. And in humans, things are just slow. That said, if you think that every experiment that is being run right now by pharma companies, every clinical trial that's being run is optimally planned and optimally conceived, Given the full state of knowledge, you are off your rocker. There's no way. And those experiments cost hundreds of millions of dollars. And so the question is, we do, at the end of the day, have to run clinical trials. How do we make sure that those experiments are the best experiments we could possibly be running, given all the knowledge that we have, given all the data we have? There's so much data that we have that has insights in it that are waiting to be found, where we just do not have people to go and find them. ([Time 0:17:18](https://share.snipd.com/snip/3d259dcf-3fb1-4504-b882-535c56c6dcea)) - **Apply Agents To Data-Heavy Tasks** - Use agents like Cosmos for heavy data analysis and literature review when you have large datasets you can't manually parse. - Expect to spend time reading and validating the AI's findings after it generates hypotheses. Transcript: Casey Newton Is it the sort of thing where like I have completed my experiments and now I want some help doing some analysis? Is it I have all these old experiments that I only did a little bit of analysis on and I'm curious if I can like sort of squeeze any more juice out of them. Or like what other ways are you seeing the AI being like really good right now for a working scientist? Sam Rodriques Yeah, yeah, great question. So going back to me in 2019, which is when I was wrapping up my PhD, right? I had this gigantic data set and I wanted to graduate because I was a PhD student, which meant that I was making like, you know, $40,000 a year or something on. And like, there were a ton of great opportunities to go out and like, don't be a PhD student anymore. Okay. So I spent six months literally just like sitting at my desk, like trying to analyze the data and drawing conclusions, reading papers, right? For right now, that's what, Cosmos fits in. It's like, you know, you would just take that data that you give it to Cosmos. It comes up with a lot of findings. Right now, you need to go and do a bunch of manual work to validate those findings and so on. ([Time 0:18:31](https://share.snipd.com/snip/0807f7c3-5ceb-49f2-adc0-bb457c4b0f88)) - **Two Paths In AI For Science** - AI for science splits into modeling nature (proteins, organisms) and modeling the scientific process (agents, reasoning workflows). - Edison focuses on the latter: building agents that generate hypotheses and orchestrate analysis across datasets. Transcript: Sam Rodriques Order to understand the landscape of AI and science, the first thing like fundamentally that you have to understand that AI is about building models, right? So, for example, right, like, a language model, like, what is a language model? A language model is fundamentally a model of human language. It just so happens that when you build a model of human language, it, like, learns how to think like a human in some sense, because humans, like, encode their thoughts in language. This is one of the greatest discoveries, certainly in the 21st century, maybe of all time. So similarly, when we talk about AI and science, what you have to think about is that you are modeling things. That is what AI does. And there are kind of two fundamental categories. There's modeling the natural world, right? And there's modeling the of doing science. These things are fundamentally different. And the reason to make this distinction is because, you know, what we are doing, right? We are modeling the process of doing science. The other side of the AI for science world is building models that can, for example, predict the structure of proteins, that can generate a new antibody, that can create a new organism From scratch, which are all things that have kind of like happened in 2025, where there's just a huge amount of momentum. ([Time 0:20:15](https://share.snipd.com/snip/74a707da-a3ed-459c-9df0-c981fd7abeea)) - **Generative Models Are Transformative** - Generative models that create proteins or antibodies from scratch represent a revolutionary capability in biology. - These models enable de novo design, potentially cutting months or years from traditional discovery pipelines. Transcript: Sam Rodriques What's most exciting right now, I think, without a doubt, is this trend towards what we call generative models. So these are things where, these are models that can produce examples of, you know, proteins or antibodies or whatever that have desired characteristics, basically from scratch. This is a new capability that we have never had before, and it's huge. ([Time 0:21:50](https://share.snipd.com/snip/55952a55-1add-4358-b4cd-95c953dd501c)) - **Verification Is Faster Than Creation** - Checking AI outputs remains faster than producing them manually, but not free: scientists must still thoroughly verify results. - The goal is parity with human-quality analysis, not perfection; errors will persist as with human work. Transcript: Casey Newton I mean, look, you have to spend a lot of time going back and checking. Yeah. Sam Rodriques But like, to be clear, this is true regardless of whether or not an AI does it or whether you ask a friend to do it. If you're going to publish a paper, you damn well better go back and check it and like be sure that you are confident. And it's never going to be 100%, right? The best you're going to do is you're going to get to a place where it is similarly good to if you were doing it yourself, which is not 100% because you're not infallible, right? And checking the work is always going to be faster than producing it in the first place. ([Time 0:22:46](https://share.snipd.com/snip/52ff5526-134c-4606-b972-1b52cc3418c7)) - **Keep Windows Open For Serendipity** - Serendipity and 'mistakes' have historically driven breakthroughs and may persist with AI if systems retain some randomness. - Sam suggests keeping experimental 'windows open' and preserving noise to allow unexpected discoveries. Transcript: Sam Rodriques Yeah, this is a great question. And the fact of that is we just really don't know yet. This is going to be a really important core question that a lot of people are asking. Kevin Roose What's your intuition on that? Sam Rodriques I mean, I think they probably will because— They probably will— They probably will preserve it. Kevin Roose Preserve it. Sam Rodriques Penicillin, my understanding is that basically the window was left open on some agar with no antibiotic in it. Obviously, they didn't have antibiotics. This was the discovery of the first one, right? So the window was left open with some agar and some spores flew onto it and began growing and they observed that the bacteria was inhibited, right? That's a mistake. Someone screwed up. Right. And that mistake led to something fantastic. And you will have mistakes, I think, that will be preserved. But in the meantime, scientists should always leave their windows open. Casey Newton You never know what's going to happen. Sam Rodriques You have no, you know, seriously, though, like there's so much when you get a graduate student in academia. Right. When you get graduate students, first year graduates, they have no idea what to do. They have no idea what to do. And that is a huge source of scientific progress because they just do the most random kooky stuff that no one who knew anything, who knows anything would ever think to do. And it's actually, it's actually really important. Kevin Roose You almost want your like AI scientist model to hallucinate a little bit. Totally. So that it doesn't lose that quality of like- Or just add noise, right? Sam Rodriques We talk about this as just like adding noise in order to, this is actually important for like biological evolution also, right? Like the genome has a lot of noise, and that's how the evolution randomly comes up with like new stuff. There's a protein that is just totally random, doesn't do anything. Then one day, all of a sudden, oops, it does something. And that's great, right? ([Time 0:23:45](https://share.snipd.com/snip/441fc5c6-8f02-4572-9a6e-dfda3eca3729)) - **Cures In A Decade Are Implausible** - Bold claims that AI will cure most diseases within a decade are implausible because of experimental timeframes and knowledge gaps. - Sam thinks major leaps are plausible in 20–30 years, but a ten-year universal cure timeline is 'crazy.' Transcript: Sam Rodriques Is crazy. Oh, and I'm happy to take a very strong stance on this because if I'm wrong, it's a great thing, right? But if I'm wrong, everyone wins. But like a decade is crazy. Why is it crazy? Because for the reason that we were talking about before, you have to run clinical trials, right? If we had a drug right now that prevented aging, completely halted aging in humans between the ages of like 25 and 65 or something, you would not know for 10 years. Because you can't detect in humans in that age range whether or not they're aging for like at least like, you know, five or 10 years. You don't detect from one year to the next that you're aging. So you won't know if the thing is working. Casey Newton I don't know. Some people at my 10-year high school reunion were already looking pretty rough. This is fair. I hate to say it. Sam Rodriques I did say 25. Yeah, okay. Fair enough. Fair enough. But right. I mean, you know, so we have to conduct experiments. Those experiments will take time. Now, will we, like 30 years, I think is very plausible. We don't know what is going to be possible. We don't know if it's possible to halt aging. We don't know if it's possible to, like, cure all diseases or whatever. ([Time 0:25:32](https://share.snipd.com/snip/34166f92-1108-4e0e-82b3-79285b005837)) - **Benchmarks Drive Research Focus** - Labs chase math benchmarks like the IMO because they give clear, measurable progress signals. - Biological breakthroughs are harder to benchmark because true validation (e.g., human approval) takes years. Transcript: Sam Rodriques Are two reasons. I think that one of the reasons is exactly what you just said. It's just familiar, right? But the other reason is that you can measure progress, right? So ultimately, like what drives progress in machine learning, a big part of what drives progress is benchmarks. With math, you can tell whether or not your proof is right. And there's kind of like an infinite number of things to go and prove. So it's just like really easy to tell whether or not you're getting better. And things like the IMO just present like great opportunities. By contrast, if you look at like some of the biggest breakthroughs recently, you know, biggest breakthroughs this year in AI for biology, right? Things like, you know, Chai Discovery, Nabla coming up with these like extremely good for producing antibodies de novo, right? Huge breakthrough. But ultimately, the win for them is going to be when it's approved in a human, and that might be another five years or something. ARC Institute putting out the first time anyone has designed an organism from scratch. They designed a bacteriophage. It's the kind of virus that infects bacteria. Incredible, right? But like just harder to evaluate, like how good is it? Like you're not going to release it into the wild. And so it's harder to evaluate, whereas like the IMO is just like super clean. ([Time 0:31:41](https://share.snipd.com/snip/829479bc-6ce3-4e3e-9ad5-f6a590ecf988)) - **Start With Low-Friction AI Wins** - Expect slow adoption in wet labs; start with low-friction wins like coding assistance and literature search. - Use AI to remove coding bottlenecks and parse literature before applying agents to core experimental work. Transcript: Sam Rodriques That will be a year from now? I think that you'd be shocked to the extent that they have not yet. Scientists in general are extremely conservative people because if you're running an experiment, you never actually fully know, in biology at least, you usually do not fully understand Why the experiment works and why not. There are some things that you've inherited from protocols that you've run in the past and where it's like, we do it this way. You could go and test it, but there are way too many things to test. So you're just kind of like locked in in your methods and it's what works and you just want to do what works. And so for that reason, like biologists just adopt new methods slowly. I think most labs around the world are still probably doing science the way they've done it before and probably will continue to do so for a while. And that's OK. You know, one place, I think, with coding, a lot of people are already adopting it because in biology, historically, coding has been a big bottleneck. It's a huge unlock now that biologists who didn't know how to code can, like, do a lot of coding using cloud code, using OpenAI's models, Gemini, etc. So that's a huge unlock. I think that's going to see a lot of adoption quickly. Literature search, right, like being able to parse the immensity of the scientific literature, that's a huge unlock. That's going to get adopted very quickly, right? The tools like what we're building are like a little bit more frontier. Ultimately, people will adopt them when they see other people using them and getting great results. ([Time 0:35:00](https://share.snipd.com/snip/c4228d00-4ae3-45bd-8f51-088cd6074179)) - **2025: The Year Of Agents** - 2025 was the year agents emerged as a dominant trend in AI-for-science, alongside de novo antibody and organism design. - Sam lists agents, de novo antibody design, and organism generation as top breakthroughs of the year. Transcript: Sam Rodriques Yeah, I think that, honestly, this year has been the year of agents. This was the year when people discovered agents. And so I do, in good faith, have to put myself, I have to put us on that list. Also with Google co-scientists, I mean, we're not the only people who are working on this. You know, Google has been doing a great job. There are a bunch of other people. So AI agents for science, definitely. And then like generative design is just having a huge moment, right? So the other ones would probably be the work that Chai has been doing, the work that Nabla has been doing, and many others on de novo antibody design. I'm really glad you defined de novo earlier in the broadcast, by the way. It's come up a lot. Yes. Sorry. When I say de novo, I just mean like literally you just like, it generates it from scratch. You don't give it anything, right? You just like, or you give it a target that you want to bind to and it generates it from scratch. This is huge because like basically the promise that companies like Chai, Nabla and so on are going after is a world in which you can say like, we know to cure this disease, we have to target That protein. You click a button and you have an antibody that you can go and put in humans tomorrow. It's huge. It cuts out an enormous amount of what people had to do previously. So that's a huge one. And the third one, I just what brian hee patrick shu and so on at the ark institute have done with like generating organisms to know sorry generating organisms we can say it we know what It means now that's the important thing this is our like peewee's playhouse word of the week the de novo design of organisms. Is it useful? I don't know. Is it awesome? ([Time 0:39:14](https://share.snipd.com/snip/29b57868-ae1a-4170-be8e-5ddb1cf1b0f3))