Large Language Models with Don Rosenthal

I'm excited to share this episode with Don Rosenthal. Don is a seasoned product leader with extensive experience in AI and large language models. He has led product teams at Google AI research, Facebook Applied AI, and Uber's Self-Driving Technology division. During this conversation, Don shared his insights on the anatomy of an LLM, ways to incorporate LLMs into products, risk mitigation strategies, and taking on LLM-powered projects. LinksDon on LinkedInAttention is all you needThe Illustrated TransformerTranscript[00:00:00] Don Rosenthal: please, please, please do go out and do come up with unique and exciting, uh, important new applications, build stuff that solves important problems we couldn't even try to address previously. I just want you to be sure that, uh, you're going into this with your eyes open and that you've prepared your stakeholders properly.[00:00:21] Don Rosenthal: There, um, there are a lot of successful applications that have been built with these LLMs and a lot of the pioneers have discovered all the pitfalls and where all the dragons hide so that we can We can avoid them. [00:00:35] Himakara Pieris: I'm, Himakara Pieris. You're listening to smart products. A show where we, recognize, celebrate and learn from industry leaders who are solving real world problems. Using AI.[00:00:46][00:00:47] Himakara Pieris: , today we are going to talk about large language models. And I can't think of a better person to have this conversation with than Don Rosenthal. So Don has [00:01:00] spent most of his career in AI.[00:01:02] Himakara Pieris: He started out as a developer. building ground support systems for the Hubble telescope, um, including being part of the team that built the first air ground system ever deployed for a NASA mission. He then went on to build and manage NASA's first AI applications group, where his team flew in, flew the first two AI systems in space.[00:01:22] Himakara Pieris: And he worked on prototype architectures for autonomous Mars rovers, done then commercialized. Uh, the air technology from Hubble Telescope in two of his air companies that he founded. He was the group product manager for autonomy at Uber 80 G. Uber's autonomous vehicle spin off in Pittsburgh. He was the PM for face recognition at Facebook.[00:01:43] Himakara Pieris: And most recently, Don was the group product manager for conversational at a I research [00:01:50] Himakara Pieris: done. Welcome to the smart production.[00:01:53] Don Rosenthal: Thank you very much. I'm really, really excited to be here. You might. Thank you for inviting me.[00:02:00][00:02:01] Himakara Pieris: So let's start with the basics. What is an LLM? [00:02:05] Don Rosenthal: Good place to start. Um, let me start out by saying that, uh, LLMs have finally solved, and I don't think that's really an exaggeration.[00:02:14] Don Rosenthal: They finally solved one of the longstanding foundational problems of natural language understanding. Understanding the user's intent. Um. What do I mean by that? Um, uh, any one of us who's used the recommender system for movies, TV, music, which pretty much all of us, um, we know how frustrating it can be to try to get the system to understand what we're, we're looking for.[00:02:40] Don Rosenthal: These systems have all trained us to dumb down our queries. Uh, in order to have any chance of a successful retrieval, you can't talk to in the way you would to a