Is AI What Robotics Needs? With Jasmine Lawrence Campbell  ## Metadata - Author: [[Hanselminutes with Scott Hanselman]] - Full Title: Is AI What Robotics Needs? With Jasmine Lawrence Campbell - Category: #podcasts - URL: https://share.snipd.com/episode/8ad3230f-87be-4b5c-98bc-9c4452f10d4f ## Document Notes ## Highlights - **Robots Struggle at Home Tasks** - Robots excel in constrained factory settings but struggle with dynamic home environments. - AI and human input are essential for home robots to adapt and perform complex tasks like folding laundry. Transcript: Scott Hanselman Like I think about the stuff that I dislike the most, it's the dishwasher, and it's the folding the clothes. But you said single purpose robots. And that feels like from a manufacturing perspective, maybe this is an overstatement, but a solved problem. Like I've seen robots move at speed, pick and placing stuff, but I still can't get one in my house to fold my laundry. Jasmine Lawrence Campbell Yeah, that's a very great question. Thank you so much for this softball. So in the factories, there are very constrained environments, right, that are not dynamic at all. There are strict boundaries and regulations around how humans and robots come into contact or interact with each other or not in these environments. You want a robot in your home that can adapt to your home every single day. And, you know, your kids are going to leave their shoes one place in one day. Red's going to be in a position another day. And so it's a combination of like, it's not the shortage of the capability of the robot to do it, but to do it in a dynamic and adaptive way that is honestly only made possible by machine Learning and a combination of human in the loop, right? We've seen, I've recently read an article about Roombas, right? They've been in the market for 20 years even, and they're still having challenges with object detection, where the robot is designed for its single purpose of cleaning the floor, and It will see some dog poop on the rug and say, I should clean that. And because it's not effective at cleaning that, you'll have more poof in your house than you ever wanted. So it's really figuring out how do you make this thing the best at candidates' job. Now, if this Roomba also had to fold your laundry and put your shoes away and dust your countertops, the complexity of the problem really raises. ([Time 0:04:00](https://share.snipd.com/snip/257c269c-df7e-45c4-a797-41fa1461bec5)) - **Why Phones Are Smarter Than Robots** - Phones have vast computing power but robots like Roombas remain limited by hardware and narrow sensor use. - Robots often don't fully utilize sensors or data due to manufacturing and algorithm constraints. Transcript: Scott Hanselman And it was dumb when it was bouncing around randomly. And now it has LiDAR and radar and cameras. And it's still dumb. Help me understand why my phone is so powerful and my Roomba is so dumb. Jasmine Lawrence Campbell That is a great question. That is a great question. I don't know. I have to put my speculative product hat on. I think, like I said, it's the variance of the different environments. I don't know why it's not smarter yet. It's probably older algorithms. And honestly, I'm not sure if every person... Is your Roomba connected to the internet? Scott Hanselman Yeah. So I have a Roborock and it's quite sophisticated and I love their app. And there's also open source software for people who don't want a cloud enabled robot because people don't want their inside of their house being off to a foreign country, which is valid. Jasmine Lawrence Campbell No, these are the trade-offs, right? Because if everything has to happen on the device itself and it has limited compute because of how it's being manufactured, it really limits what it can do, what it can see, how it can See, right? It's maybe not doing... Also, sorry, some of the sensor data might not be leveraged for each one of the capabilities. So, for example, it might have that LIDAR or time of flight sensor, but it might only be used for detecting ledges and making sure it doesn't fall down the stairs or get trapped under a Couch that it might not fit under. Yeah, there's a lot of options. I think there's just been a lot of innovation in the last even five years on what sensors and computing. And, you know, there's also this challenge of like, hey, maybe your Rula has like a perfect slam based map of your your house. Doing any kind of predictive forecasting of like, hey, this was here, this was there, or even having the knowledge of like, this is a movable object versus a fixed structure. So even our understanding of the world could be incomplete in making your device seem pretty dumb. ([Time 0:06:16](https://share.snipd.com/snip/b3601f3b-5b79-4407-a52f-071dc6d831ff)) - **SLAM Mapping Explained** - SLAM (Simultaneous Localization and Mapping) helps robots create rough maps of unknown spaces indoors. - Robots combine sensor data and odometry for positioning but struggle when environments change dynamically. Transcript: Scott Hanselman And I want to expand one of your acronyms there. You just said a SLAM-based map. Like you just threw it out like everyone would know that. Simultaneous localization and mapping, right? So this is how to map an unknown environment. Talk about that because I know it's doing that kind of work. Yeah. Jasmine Lawrence Campbell So basically it'll walk around, zoom around and kind of taking, I think a lot of people use lasers nowadays, calculating the distance between itself and other, I think we call them like Landmarks in the environment and assess like, hey, I think I'm here. And in relation to whatever surface this laser is bouncing off of, I'm going to draw a boundary that this might be an edge or a wall and it will go around this environment as many times as Needed to form a complete map. So if you think about like a top down map that you might use, not even at the fidelity of like a Google Maps, but like something like a pirate map level, right? So like outlines and edges, honestly. That's essentially what you get out of SLAM. Now what you can do is really pair that map with some of the other sensors on the robot, like these sensors that track how many rotations the wheel has had, like using the odometer. Combining a lot of this information, you can know where you are in the environment and where these landmarks that you're running into might be. ([Time 0:08:10](https://share.snipd.com/snip/7e8ca65b-089a-49f5-aa17-7cb290e07fe3)) - **Human Intelligence Beats Robots** - Humans unconsciously integrate multiple senses and proprioception to navigate environments seamlessly. - Robots lack this complex, flexible intelligence, making tasks like simple navigation very hard. Transcript: Jasmine Lawrence Campbell One of the most exciting and endearing aspects of robotics to me is that in your example, people have no conscious sense of how they navigate the world. They completely underestimate all the parts of their brains and the proprioception and blah, blah, blah that goes into us being able to navigate the world. The rumor only seems dumb because we underestimate what intelligence is, which honestly has yet to be defined. But when you try to quantify these things, it gets really challenging. But what I love is that it helps me to pursue a deeper understanding of the world of physics, of my body and other bodies like animals and nature, and really figure out what we can learn From these precedents that have been set by the world around us. Scott Hanselman Yeah. You use one of my favorite words, proprioception, right? This idea of kinesthesia, the ability to like, for example, walk on a path of uneven stones. It just works. Like we just do it and babies can do it within a couple of years of life and robots still have trouble with that. Jasmine Lawrence Campbell It's so true. It's like, you know, that pin the tail on the donkey is a great game that I like to use to explain to people like why people can be so good at that. They really understand where they are in the world relative to themselves and other things in the environment. Yeah. So robots really do struggle with that. They struggle with interpreting what they see and understand based on the limited sensors. And not all the sensors are given the same fidelity. It's like today we have these super high definition photos that our phones are able to take. And, you know, some robots don't have the ability to process that large volume of data or make smart inferences or guesses based on that data. ([Time 0:10:18](https://share.snipd.com/snip/46c738b4-f54d-4cb4-b9d6-aea73b30d265)) - **Use Human Feedback in Robotics** - Incorporate human-in-the-loop feedback to improve robot learning and error correction. - Use apps or verbal communication to help robots re-map or adjust when environments or tasks change. Transcript: Jasmine Lawrence Campbell And I think that, you know, what an area that I got really passionate about when I was at Google was human in the loop, right? That, you know, you would never just leave your toddler out and say, hey, yeah, why don't you just explore the world and figure it out? Why don't you just put everything in your mouth and do that? We learn a lot that way, but there's a lot of coaching and mentoring that goes into human development. And I think robotics is in the same realm. So if you went into your app that controls and supports your robot, you could probably say, hey, I think you got the wrong idea. Refresh the map. Walk around again and see what reality truly is. Or there's been a problem. Why don't you check again? If you could verbally communicate or nudge the robot to do the right thing, it would be able to learn easily. But just like you said, you change the environment. It goes, that's different. ([Time 0:12:57](https://share.snipd.com/snip/5c1a49e1-e4ed-4995-b760-f08c5ac38b55)) - **Robotic Senses Are Limited** - Robots have limited senses compared to humans; for example, smell remains underutilized. - Advanced touch sensors can detect force and texture but integrating these inputs is a complex challenge. Transcript: Jasmine Lawrence Campbell Yeah, absolutely. And there are sensors that it could use that we don't have a digitized version of it, like smell. Smell is still something, an underutilized sense that we're not leveraging robotics. Touch, we might have like force, right? There are like force torque sensors, like if you, or bump sensors or things like that, that can tell us when there's been contact. But what degree of contact, what temperature of contact, what is the texture of that contact? Is something slipping out of my hand? Do I have a decent grip on it? These types of things, there are only early prototypes and models of getting this type of feedback from the environment for a computer. And then once you get that feedback, you have to figure out how do I integrate that into all of the other knowledge that I know? How much do I weight that input of how hot something is to whether or not I should or can pick it up? ([Time 0:13:57](https://share.snipd.com/snip/573e8390-c41d-4415-bb83-93f1dc6d6507)) - **Grocery Bags Highlight Robotic Challenges** - Jasmine struggles to carry all groceries in one trip, like many do. - She questions how complex algorithms would handle grasping many varied items with limited robotic fingers. Transcript: Scott Hanselman Tell me if you think that this analogy works or this is a story that you can relate to because I'm going to guess based on your personality that when you bring your grocery bags from the Back of the car to the house, you were like, I am not doing this in two. Yeah. One trip mentality. I am not doing it. One trip. So then you're just like, okay, I've got this many knees and I've got 10 fingers and I'm going to like, and then the ability to hold all of these things and not have them slip. Yeah. That's, that is, I always think to myself, what an interesting robotics program. Jasmine Lawrence Campbell Yeah. Yeah. Scott Hanselman The robot would totally get like just the milk and go and then just the chips and then go. But I'm like, I'll be damned if I'm going to go and make more than one trip. But I'm like, it would be impossible to imagine what kind of algorithm would be involved to go and get all of those things on just two hands. ([Time 0:14:45](https://share.snipd.com/snip/820476ab-7b2f-415e-b1a7-58a78a2cd254)) - **Adapt Environments for Robots** - Humans adapt their environment for ease, robots need similar adjustments to assist effectively. - Robots can’t replicate human adaptability or dexterity so redesigning environments benefits robot integration. Transcript: Jasmine Lawrence Campbell So when I think about what a human can do, I don't always think that is the best way. Honestly, we have to take our little narcissistic hats off and really think about, is there another way that that task could have been done, right? Because when you were at the grocery store, you could have put all the groceries in a cart. And that cart could have been really easy for a forklift or two hand robot to pick up that entire cart, rather than you have any expectation that it has to weasel its arms and end effectors Through all those tiny little paper bag loops and all that tension, right? So there are ways that we can adapt to wanting to integrate AI and robotics into our lives. And so I really love what happened. I think it's the 35th anniversary of the kind of ADA laws in the U.S. And so all of these things like curb cuts of alternative feedbacks of like lights for our fire alums, all these different things are actually really beneficial in robots being able To not have to fit our expectations of them, but for us to make adjustments in our world so that they can help us. Scott Hanselman That's a really, that's a good reminder. Cause sometimes I'll be watching a movie or I'll be looking in the mirror and I'll just go, Hey man, we're some funny looking people. You know what I mean? Like what a weird decision it was to have the thumbs oriented this way. Jasmine Lawrence Campbell No, really. I mean, even in school when I, so I'm, I left hand when I write, but I'm right hand dominant for almost every other activity. Same. And so like even living in a world where you're like, wow, 95 percent of scissors weren't made for me or like doors open in an awkward direction. Like I can eat and brush my teeth with different. It's just a very strange world. Assumptions that we make based on how we think people are supposed to be. We put these expectations on devices that are a 10th or 1% as intelligent or dynamic and flexible as we are, and then we call them stupid. ([Time 0:15:40](https://share.snipd.com/snip/212929fd-b391-4473-9184-d6281c87c5aa)) - **Start Robotics Learning Early** - Use platforms like Adafruit and KiwiCo to start robotics education at any age. - For advanced learners, explore courses on Udacity, Coursera, and support STEM programs like Black Girls Code. Transcript: Jasmine Lawrence Campbell Oh, man, there are so many. There are so many. I really love Adafruit. It's a great website with a lot of projects that you can do out of the box. If you're, I would say, like maybe high school and college level, they have kits with step-by instructions. They have videos on YouTube. It's really great. And then there's KiwiCo if you've got kids like middle school and below, like even up to five years old. I'm actually in their robot coder starter kit that really helps you develop a career of robotics without any code. Right. Really understanding what is path planning? What are sensors? Different things like that. And then I would say if you are like high school or college level and above, there's platforms like Udacity and Coursera that have amazing programs that were co-developed with leading AI and technology companies, really helping you grow job-ready skills, right? I actually teach a course on Udacity for AI product management that a lot of people have taken, not because they, you know, want to go out in the field, but just to better understand the Industry and better understand how they might influence the direction that it goes, either as a creator or even as a consumer. So those would be some of my resources. There's definitely more on my website. I'm a huge proponent of STEM education in general. ([Time 0:31:16](https://share.snipd.com/snip/3c603d5b-91c7-4ef5-9e50-a32ad08dd959))