Designing Memory with Benjamin Zweig August 2026 Not long ago, LLM chatbots were a blank slate. Each new conversation meant starting from scratch, forgetting everything from your prior chats. This changed when ChatGPT gained memory, opening up new possibilities for LLMs as personal assistants and collaborators. This month we sat down with one of OpenAI’s first designers, Benjamin Zweig, who now works at The Bot Company. He was part of the team that built the first version of memory for ChatGPT. While memory may seem simple in retrospect, its development process sheds light on some novel challenges of designing for AI, as the boundaries of a feature can be unusually hard to define. Memory has to cover a huge number of use cases for a massive, diverse user base. How do you start to solve a problem this broad? Let’s find out. Ryan Mather Hey Ben! Before we get into your work at ChatGPT, could you give our readers a little background on yourself and what you were doing before OpenAI? Benjamin Zweig I fell in love with interface design in middle school. My first interaction with great design was a 2nd-gen iPod Touch. It felt different and better in ways I couldn't explain, and I wanted to understand why. The way to do that is to take things apart, so I got heavily into the jailbreaking scene back in the day, before all of the good ideas had been officially implemented by Apple, and there was still some fertile ground. After high school, I was at a weird juncture where I had applied to colleges and done that whole thing, but I felt like I knew exactly what I wanted to do and thought, like, would it be crazy to just start working? So I did just that. When I was still a teenager, I started full time at Twitch. I worked there for about two and a half years on chat. Then I went to Facebook and worked on chat. A few jobs later, I got an email from Maddie at OpenAI, and the rest is history... RM This guy can't get enough of chat. BZ Yeah, that's the joke friends always make. It keeps coming back to chat. I'm doing a little less of that now, but I'll never escape. I worked on ChatGPT for two years and change, and am now working on human-robot interaction for the home. RM Well, we can't wait to see how you design chat for robots, but we should get into the meat of the interview, let's talk about your time at ChatGPT and the process of designing memory. BZ When we started working on memory, no consumer chat product had shipped it yet. It's kind of expected table stakes now because it is so obviously a good idea. And not only did I think it was super valuable, but you ran into a funny problem: when [ChatGPT] could speak so convincingly to anything, and when hallucinations were more prevalent, people would assume that it could do things that it couldn't actually do. [So] it's a funny situation where it's easier to just build the thing users expect rather than trying to explain why it can't do what they think it can. Building it is more beneficial long term than trying to explain around the lack of it. RM How did you decide to work on it? What was the first version, and how did you assess if it was any good? BZ We had myself and Noah, who was also a huge proponent of memory, and Joanne, who was the PM for model behavior at the time. We talked with her a lot about how important we thought it was, and we all got it on the roadmap. That degree of conviction around "this is so obviously correct" is a luxury to have. No one wants a coworker with amnesia. No offense towards any coworkers with amnesia—but I don't think a coworker with amnesia is particularly desirable. For most people, an assistant [with amnesia] isn't particularly desirable either. There are whole swaths of relationships and types of work that weren't practically possible until something like this had been built. And to validate demand for this type of thing, people were going to kind of crazy lengths to keep some semblance of a consistent chat session going. Back then, context windows were much more constrained, and hitting those limits was a real problem. So, many people were saying, "I tell it to summarize everything about our relationship, or its essence of self, into something that I can copy and save to my desktop. And then next time, when I start a new chat, I put that in so I can have some sense of continuity." It made it pretty easy to say "I really think we should build this," even beyond our gut conviction. A first version of this is what I think anyone would have thought to build: there's a little notepad, and the model can write to it, read from it, or edit its contents over the course of multiple conversations. You do run into some problems: does it apply retroactively? How do you make that work? When people bounce between different chats, is the expectation that it'll only apply to new chats created going forward? Probably not. Ideally, I want it to apply everywhere whenever I send a message. There were technical challenges to make that work, but the first version was really clear: the model can choose to write something to this notepad, which it then has persistent access to, and can choose to read from. We had ideas for more complex systems, but this was a good place to start. And there were already interesting problems! How do you figure out what's worth saving, and when to read it? When context windows were much more constrained, you didn't want to just dump in the entire notepad. Not only do you need to watch the length of your chat, but it might over-index on irrelevant memories. How do you design it to work well? The mantra of "design is how it works" demands designers to engage with this technology and process deeply if they want actual ownership and the ability to set product direction. So if you're working on anything like this, you have to dive head-first, especially if you have the luxury of working at a model company, into how these decisions are actually made. Design is how it works. I don't think I would have been satisfied and considered it a finished job if all I did was think about how the UI state... RM ...is called, and if it is memory in settings. BZ You do have to consider: how is it represented? What is the icon in the settings panel? Should you show when it saves or reads? But the essense of designing memory was training these decision boundaries such that it remembers useful things and recalls them in a useful way. You could call that "model design." You could call it "model behavior." There're a couple different terms floating around now. It's fun that "model designer," I think, is a new term, in much the same way that Facebook invented "product design." I think of design holistically. Anyone can jump into this stuff. And the core of it all really is, how do you turn qualitative decisions into a quantifiable rubric? That's exactly how you train a model to get better at any task. Here, this meant formalizing the very human skill of determining what's worth remembering, both in terms of what you save and when you recall it. If you ask a person to explain how they make that decision, they're not going to have a clear answer for you. It's an inherently human thing. So what you do is you look at a bunch of examples, apply your human intuition to make a decision, and then look for patterns and ways to turn that intuition into a heuristic that you can train against. That's a new skill for many designers, and a new way of thinking and working. Federico Villa When you were working on something as intangible as memory did you work a lot on diagrams or off a prototype? Or were you just going through these philosophical conversations about what it means to remember something? Can you speak to your design process at that stage? BZ Initially we had conversations as a group, cross-functionally, across the whole team that launched memory. Some were philosophical, yeah. When you're working on something as inherently human as this, you want diverse perspectives. Once we had an actionable plan, we put up a dogfood model: a test that we could chat with, and we paid very close attention to how it behaved. Did it match our expectations? Did it not? We gathered a list of golden examples of really good "saves" and really good "applies" that we all wholeheartedly endorsed and were great examples of the feature working as intended. Then, it's an iterative process of refining the instructions that data labelers use in order to decide what is a good application of memory or a good memory write. This was one of the most interesting problems then and probably still is now, because it's so complicated but so effortless to us as human beings. We do it every day, but don't definitively understand how it actually works. We know that spaced repetition is a good technique for remembering information long term. We know memory can be heavily influenced by other senses, especially smell. That said, everyone has probably read The Bitter Lesson, and scale is what matters. It's really tempting to apply bespoke structures to the model based on our best guesses of how our minds work. When you're in a position where that kind of thinking actually makes sense, it's really fun. For instance, we know that memories are reinforced by recency and reptition. It likely does make sense to record when a memory was created and last referenced, because that could make the model treat the same piece of information totally differently. If that metadata isn't available, it can't do as good of a job of making sense of your memories in a way that feels natural, useful, and expected. Normally, I'm not sure it makes sense to work that way... but when you can lean into that a little bit, I think that's cool. RM Let me make sure I'm clear. Which way? BZ Applying some version of your understanding of the structure of the human mind, like the way that we think about memory and decide what's important to remember or recall. "Frecency," or frequency plus recency, is a good, sensible signal. Some version of our REM sleep cycle, where you give the model a chance to reason over and organize everything you've told it over the course of the day, also makes sense. I've seen other, more elaborate takes, like "we made the model navigate a metaphorical mind palace, because that's the way we think human beings remember things." That, I think, is probably not bitter-lesson-pilled enough. I'm not sure that's going to be advantageous long-term. But deciding which types of structure, support, and additional information you think the model needs in the short term is another fun challenge. Robin Chen You mentioned how everyone has their own interpretation of how memory should work. What's your particular flavor and take on memory, and how did that affect the iterations you were working on? BZ The thing that's most missing now for most memory implementations I've tried is productive and effective forgetting. Human beings forget the vast majority of the information they take in. The benefit of digital storage being relatively cheap is that even if something is forgotten, it can still be archived somewhere. Imagine your memory continues to work as it does today, but everything you've seen is in a neatly organized drawer in a storage facility that you never have to think about. If you need something in there, you can go take a look. That idea of forgetting is just part of doing a good job of applying memories and not applying memories that don't make sense to apply. Most of the issues I see now in any product's memory implementation are more about over-application than under-applying. RM There have been a bunch of these tweets lately of memory use being like, "oh, and since you like anime, let me add some miso to this dinner..." BZ Totally. This is a problem that we thought about specifically and did our best to make the model good at. But there's definitely more progress to be made there. It's already gotten much better. The team has done a great job since. My personal flavor of memory is honest and aggressively forgetful. And maybe that's how I'd like to be, too. Because I think that's how you can be happy. RM Talking about memory, it sounds like as a designer there were at least three different layers you were working on. The model itself: sometimes it's harness design, but sometimes the model just needs to be smarter about understanding memory. The harness layer: how do we think about this as triggering enough or not enough? The interaction layer: how do we give the user feedback of "this memory thing." Which layers did you focus on more than the others? How do they interact? BZ I really wanted to work as evenly as I could across all of them, to the extent that it made sense. Obviously a researcher is going to be able to do a better job of directing model behavior technically than I could, at least for now, until the AGI in a box does all research for us. To the extent that it was possible, the answer is all of them. I worked with a super talented researcher on making the model better at applying preferences generally, be they from a memory or manual user input. Custom instructions existed before memory, and there were already existing opportunities for making the model better at applying what it knew about the user. Then there was that full-team effort across research and product, figuring out when and how the model should read and write, identifying good and bad examples, and writing labeler instructions. How do we teach labelers to form the shape of the data that will determine the model behavior? And then finally, once all of this is in a spot where it makes sense, then I can think about how to present this behavior to the end user. Working on the actual model behavior informed my approach to the end user interface. If you want to really engage with the raw materials you're working with to design something good, it's necessary to get your hands dirty, if you're lucky enough to be in a position to do so. My dream version of a memory management interface experience was no special interface at all. You're already chatting with this thing. If you want to know what it remembers about you, just ask, and it'll tell you. Maybe it outputs something a little more structured than pure text. But there's no special little panel I have to go hunt down. If I want to tell it what to remember, the right interface for that is the way I would normally do anything else. You're painting on air. It's not somewhere out of the way. That was my dream. And I think it makes sense to get there. We did try it at first. RM I think it works like that now. BZ Yeah. It wasn't good enough then, in our opinion. I wanted the answer to be that it "just worked." But we added a special "memory updated" label whenever it created a new memory, and you could hover over that to see exactly what it had saved. From there, you could click directly into the management interface to delete it, or see exactly in which chat memories had been created. That was the right call, because we weren't 100% satisfied with what it was choosing to remember, and giving people the opportunity to say, "oh, it remembered something, but did I want it to? Does this look right to me? Do I want to delete it?" was important. That's a degree of cognititve load that I didn't want to put on people if we could avoid it. We talked [earlier in the interview] about how [ChatGPT] is a super ambiguous tool that can kind of do anything, but maybe it can't always do everything, and maybe it doesn't always know what it can and can't do. I think the role of traditional interface in AI tools is to be a grounding, directly user-controllable source of truth in this wild, generative world. When you see this special interface element, the special little label in chat, you know for certain that a memory was created. You can see exactly what it was. And you learn that it definitely can remember things, so it also serves as a form of product education. Hypothetically, the raw model output could just say, "okay, got it, I'll remember that." But would you be certain that it's really true? RM Hopefully, yeah. But I find myself typing that sometimes. I'll be like, "remember this," and I feel a little bit stupid when I'm typing it, because from my tone, you should remember this. BZ Exactly. So traditional interface does still have a role, and it was sort of a funny moment. A little humbling, but also nice: like, oh, I really wanted this to just work, but it turns out there's still a lot of value in traditional interface for very clearly communicating the truth. RM Do you think this is purely a function of performance? Is it the case that you have these feedback mechanisms because feedback mechanisms are good and the user must understand how the technology works? Or is this a case where you have them only because the natural interface has some failure modes? I had one where ChatGPT said to me, "as a mechanical engineer, you might be interested in the airplane museum," and I'm like, I'm not a mechanical engineer. So I was very grateful to have the "memory updated" UI for that. But in the world where it never gets things wrong, would you say the best interface is no interface? BZ Definitely. Some of the most powerful interaction design principles like direct manipulation already feel like not having any "interface" at all. Feedback mechanisms are necessary right now because these tools aren’t fully reliable yet. Even as they become more reliable, it’ll take a while for our belief in them to catch up. So when you hit the edge of the “jagged frontier” of model intelligence, more "complex" interfaces are great. If you’re doing something seriously weird or seriously complicated, it means you’re always inspecting the work and making sure it’s doing the right thing, and potentially even taking it over the finish line yourself. But I don’t think anyone’s happy about doing that! It’s work. Figuring out when you can or can’t trust your tools is work too. So if you can trust the tool, I think simple always wins in the long run. RM On memory specifically, did you have to navigate this tension around letting the user edit the memory? And [the question of] how do we let them? There's this obvious thing where if it's doing it automatically, you could have the user and the model step on each other's toes. How did you think through those intricacies? BZ There are a lot of questions. What if I tell the model to remember NOT to remember something? How do you represent a negative memory? Then, if I ask it "what do you remember?" it's gonna say, "well, I remember not to remember anything about this," which is a problem. What if memories have dependencies on one another? I tell it to remember a trip that I took with someone, and then I tell it to forget that person. Does it also forget the trip? How does it communicate that gap in the most delicate way, but also the most honest way? All of these are cases of the user either doing or asking for something that is challenging for the system. We did think through these questions as best we could. We couldn’t solve all of them, because we really wanted to get something shipped. We felt this was a super valuable capability, and we wanted to get the most common use cases out the door. But as these products become more capable and are managing more aspects of our lives, especially autonomously, all of these questions become more and more relevant. How does it handle these cases in the way you want, in a humane kind of way? The personalization team has done a ton of great work since, I’m excited to see where they take it. RM My thought would be, okay, as long as the model has create, read, write, delete, if the model is smart, it'll be like, "okay, I'm not supposed to remember anything about this person, but I have this memory. I'll delete the old one and make a new one that remembers the good parts of the trip, but not the bad parts." Is that how you thought about it? BZ There is also a luxury in being able to say: "we expect to have to solve this in the future, and by the time it's necessary, the model should be smarter." If it turns out it isn't, then we'll need to revisit. It should be honest and follow your instructions as best it can, and respect your wishes, but not make anything up. My personal feeling, having talked this problem over with a couple other folks in the industry, is it would be okay to say, "I know you took this trip, but I can't remember with whom. Someone was there, but I don't remember who it was." If you want the model to forget that someone else was there at all, that's a different request. There's another challenge when you're trying to design a system that the model administers to and manages: if you let people into that little sandbox and they mess around with it, you have to be ready to handle whatever changes they make. What that means is if they make changes that are totally out of distribution, they can cause the product to behave poorly or unexpectedly. So, imagine we made the format in which the model records memories very specific and brittle, but then also said "hey, it's really important that people have ultimate control" (which I do believe!). If they go in and mess around, the model doesn't know what to make of it, because the memories are no longer in this special format that it exclusively knows how to read. That would be a problem. We were always operating on the assumption that some degree of user observability and editing would be required, and that meant hardening the model against exactly this sort of situation, and making sure that memory didn't rely on any one specific syntax. The way you do that is by making sure that the examples you train on are diversely phrased and structured, so it doesn't learn to expect any one specific format. Any researcher worth their salt would make sure that was covered. There are many other examples of cases like that, like when you're training a model to respect preferences, you need a bunch of examples where you force it to apply a perference no matter what, and a bunch of examples where you ask for a reply and don't give it any sort of preference at all. You make sure it can handle all of those cases really well, and that it doesn't regress or over-specialize on one specific skill. Again, design being how it works, it's really cool and beneficial to have an understanding of this process. It allows you to develop an intuition for what it's easy to get the model to do, and what would actually be hard. RM Go to the moment you launched the first version of memory. What did you do? How did you decide whether or not it was working? BZ Once it launched externally, obviously we paid attention to metrics. You want a new feature to be doing well. Internally, we used it a ton ourselves, and we'd regularly have a new versions to test. Often we'd have multiple running concurrently and try the same sort of input on both and see how they compared. Evals are also super helpful here. If you have really good examples of things you think are or are not worth remembering, you can figure out an automated way to decide whether a model is doing a good or a bad job. The very beginning of the project was figuring out: is the model good enough to power this right now? If not, what are the specific ways in which it needs to be improved? Then, what are the specific mechanisms that we're going to use to get it there? Then, how do we make sure that it can make judicious use of those mechanisms? And then let's try it out! Let's see how it feels. And then, how do we explain it to the world, keeping in mind that some people think it can already do this, some people have never even thought about it, and some people know that it couldn't before. We need to cater to all audiences. How do you help people feel in control and get the most value that they can? I went through a bunch of different versions of how to show people what ChatGPT is remembering about them. Do you try to make it into a fun profile and make it feel more like content? Or, is it a very bare-bones and technical-feeling source of truth, more like a utility? I thought about how the medium of its presentation impacts what people think it's for. Is this a screen I feel like I'm supposed to visit, and it's fun, and I should check in on it? Or does this feel technical and like I should only be in here if I have to? Those are two very different positionings of the same information. And you've got to move quickly. So what's easy to build that gets the job done well is also an aspect of "good design," in this case. RM It sounds like you landed on the simple list of text strings. BZ Yeah, it was the fastest and also the clearest. In order to do a good job of the profile direction, other things have to happen. To give you context, at the time we were shipping this, you didn't even have a name in ChatGPT, or a profile picture. There was a lot of foundational stuff we had to figure out about the nature of an account before something like that would've made sense. Launching anything like this is a risk, and you need to figure out the MVP to determine whether people actually find this valuable in practice. We're really confident it's a good idea—are we right? Is this version of it something that people find useful? Shipping something is so much more important than shipping the best thing. I was very confident that someone was going to put out a version of a model that remembers your conversations. A core principle of OpenAI was, and I think still is, iterative deployment. Not for deployment's sake, but so people can start to better understand what these technologies are, how it's going to impact their lives, what it can do, what it can't do, build up any antibodies they have to build up, to understand "oh, it says it can do this, but maybe it can't." And also the positive version of that: "oh, it's useful in this way that I didn't necessarily expect." Getting some version of this out quickly was valuable in many ways. RM That connects to something you touched on earlier about scaffolding. When you're building on top of a model that's going to get smarter, how do you decide which structures and techniques are enduring, and which won't generalize? BZ If you believe in a takeoff of intelligence, the answer is the model is smart and it will scale, and the amount of special scaffolding that you need to, or even should, apply will become less and less. That's been true in terms of instruction following. We used to have to structure prompts in very specific ways and swap context in and out to get it to actually pay attention to what you wanted it to pay attention to. In many cases, all of that scaffolding is now completely useless, not only because contexts are larger but because instruction following is better. RM Rest in peace: Prompt engineering, 2020 to 2026. BZ Exactly. Ideally, eventually, it doesn't matter at all. And what matters are your communication skills. Ideally, the model is the smartest, most understanding entity in the world, and can try to draw out the true meaning of whatever you're saying. The skill of "prompting" goes away, and all that matters is the skill of being a communicator. Part of what you're getting at is another one of the really interesting things about working on anything involving this technology. I worked on Messenger for maybe three and a half, four years, and the core technologies we were working with never changed, and still haven't. So it was very traditional design work built on top of that. There were safe assumptions you could make around what was possible. This idea we have [now is]: if you believe in this trend of intelligence increasing at scale, when does it make sense to invest in special scaffolding, and when do you throw it all away and say, "model's smart, it's just going to get it and all of this investment will have been... not wasted, but no longer relevant." That is also a unique challenge to working within the space, especially at OpenAI. You have to think in terms of multiple timescales at once: A. What do I think will be possible in two years? B. A year? What do I think will be possible in six months? C. What can I ship next week? Those really are the three things you have to think about. You don't want to make a decision in the thing you ship next week that really gives you trouble six months later, or a year later, when the capability you hoped was around the corner comes around. You don't want to hamstring yourself. But you also do need to ship something, and it does need to work, and you need to make compromises and build something now. That was a new skill for me and a new muscle to figure out. It was fun and challenging, and, I think, useful. FV I want to go back to something you said earlier, Ben: The art of prompt engineering. Do you really believe that in the future no one will need to be a prompt engineer? BZ The most valuable skill is going to be being a clear communicator. When models were less good at understanding your intent, there may have been specific formats that made sense to learn and practice and speak in. I'm very confident that's going to become less and less necessary, and what matters is being an effective communicator generally, knowing what to ask for, knowing the axes of the project that matter to you. The best representation of this is the area in which models have currently made the most progress, which is writing code for you. It's a really magical time to be a designer, especially a technically-literate designer, because now what matters is knowing what to ask for. "These are the technologies I want you to use, I want you to structure it generally in this way. I'm not going to write it, I don't have to know how to write it." You have to be a good communicator and a good director. And those skills would be valuable even if AI didn't exist. So it's a win-win. But that matters much more than any specific way of writing a prompt, and I really don't expect the discipline of grammar-specific prompt engineering to be around long term. And it's already much less important than it has been. I mean, if I offer to tip 200 bucks, will it perform better? If I say my grandma's going to blow up if you don't help, maybe that works. RM Focusing now on more practical advice: When you're designing AI products day-to-day, what have you found actually works? BZ The first is very boring, which is: prototype. Go from idea to something built. It's much more possible to do that now than it was even a year or two ago. This is not novel, and it's why tools like Origami have been so powerful for as long as they've been around. Anything that reduces the time from "I have an idea" to "other people can understand exactly what I mean" is really significantly valuable. That's easier than ever now. Do more and more of that. Design is about communication, and making something is the clearest way to express your idea. It really doesn't have to be perfect. The first version of memory was very far from perfect. The first version of ChatGPT was very far from perfect, but gave people enough of an idea of what it could be that they got excited about it, and it worked out. One of the reasons I thought Origami was so cool, beyond its fidelity, was your ability to incorporate real data and real logic into your prototypes. That is so much easier now through code than it has ever been before. Take advantage of that. The model will just do it by default. RM So nice. You're like, "please, onboarding flow." And it will mock up the onboarding flow and put realistic user data in it. BZ And read from whatever array, and I can go in there and modify it, or populate it from a call later. So that's super cool. Another thing: if you are designing an AI-involved product, use the actual model in your prototypes. Very doable, very informative. If you're working on a new capability, or something that isn't very clearly covered, you will quickly see what it can and can't do. Design is how it works. It's just as important as the interface or the framing you're building around it. Even in Origami, it's super doable to drop a network request in and do it for real. So the more specific part of what I mentioned before: if you're designing something that uses a model, use a model. Maybe you'll deal with outputs that are much shorter or much longer than you expected. You'll discover ways in which it works really well or works very poorly. Don't wait to do that until you've crafted the perfect container. Do it concurrently. RM What do you think you learned at OpenAI that translates to your new job in robotics? BZ I mentioned before that if you dig in a little bit, you can start to get a sense of what would be easy and what would be hard. That has been incredibly helpful around sequencing what I propose we build and knowing what I have to worry about next week, versus what I have to worry about six months to a year from now. That's probably the most helpful takeaway: there's some degree of intuition you can get from being involved with that process [of digging into model capability]. And a firsthand understanding of how quickly things are changing and have been changing. There are some problems we can expect will be hard for the foreseeable future. But if I think that something can get better, it's probably going to get better in a timeframe that's relevant to my current work. So being aware of, and even learning to appreciate and rely on the shifting sands you're working upon is helpful and relevant, and they'll usually shift to your benefit. For example, imagine something like the AI Pin using today's models. It would have been received very differently. It would have been much faster and much more accurate. That probably would have been helpful for them. RM Maybe those products will come back soon. BZ Maybe! I think AGI is sort of an overloaded term now, and I don't really know what it means anymore, but I do believe that there are many ways in which I want and expect the model to improve. And when I say "the model," I really mean any model made by anyone. There's something to that, too—it's very generic, there's no special allegiance. You can shop around. Someone will often make a model that's better at the things I care about, and that's a cool non-constraint to design around. You're just guessing how far out that's going to be. For instance, we talked earlier about real-time understanding of a bunch of different input signals. I'm confident someone is going to put out a model that does a great job of that, because I think it's possible and clearly valuable. So I feel comfortable at least starting to put some effort into thinking about how I would leverage that, even if I can't do so today. RC You worked on so many chat products. How did your experience designing chat for people inform how you design chat for machines? BZ I worked on Twitch chat, which is one of the most chaotic, utterly nonsensical forms of chat that exists. What did I learn from that? Good question. Let's talk about Messenger. RM Well, you probably had a decent sense of what kinds of deranged things people would type in a box. BZ Okay, totally correct. It helped inform me as to the broad distribution of human beings. The other answer to your question is, thankfully, very real and appropriate. On Messenger, I worked on a bunch of patterns around figuring out how to make chat make sense as an interface for complex interactions and getting things done. What are the input types? Booking appointments, answering questions, changing previous choices. How do you do a better version of a chat for getting things done? This was when Messenger was working on an assistant they called "M," which was in actuality a bunch of remote workers answering your questions, with the idea that eventually they could train some type of automation. There was a really fun thing people did (I did it too!) where you could ask "draw a picture of me," and someone would send back a sketch they had actually drawn with a pencil at their desk, which was crazy. Thinking back now: God, would we have loved something like a generative pre-trained transformer chat model. Messenger gave me a sense of what degree of complexity works within a chat. How do you handle using the structure of a chat for complex work, "tasks," that span multiple turns? So I was working on the interface without the underlying technology a couple of years early, and it didn't always work out because the technology wasn't there yet. But the idea that chat is a very natural and flexible interface that comes easily to people, that they understand, that feels lightweight, that they feel comfortable using for many different tasks—that was a design space I was working on by happenstance, way back when. It was super helpful when jumping into ChatGPT. RM You had a sense of what would work and what wouldn't work as an inline UI. BZ Yep. And it was very helpful when I was deciding on ChatGPT patterns like keeping human messages bubble-wrapped, right-aligned, and in a very familiar iMessage or Messenger-like style. Meanwhile, model responses can be bare text and feel less human in that way, closer to the material of the page, and also more flexible, because without that containing bubble what you can stick in exposed content is much more open. It gives you a lot more flexibility around how the model can respond. RM What did you find hardest about designing ChatGPT as a product? BZ The hardest thing about designing ChatGPT was always that it had to be everything for everyone, which is also a huge strength. Part of why it's so popular is because it's the ultimate multi-tool. That's really hard to design against. Great design generally comes from serious constraints, and we were working within a very broad mandate of "this can't break for anyone, it has to be for everyone, but you need to help them understand what they can do with it." The blank input field problem was always the most challenging part of working on ChatGPT. Also, there's this thing that seems really smart, but the intelligence is super jagged, so how do you have people understand what it can and can't do now, and design for a world where capabilities may be radically different in a year, or six months, or three months? And the model, ideally, knows about itself and its capabilities and limitations, and has knowledge of the product. All of those were challenges for building open-ended AI products, but the hardest part is: how do you even have a concrete problem shape to work off of when your demand is so broad? So in that way, robotics is easier, even though the specific domain is also challenging. It's really fun to take this intelligence, even a more limited version, and embody it physically. You have all these problems of, how do you communicate what it's thinking about, what it's paying attention to, what it's doing, in physical space? Some opportunities and axes are also shared with ChatGPT: how much of a "person" do you make the thing? When should it communicate more like a person versus not? The more person-like you make it, the more people assume it can do everything a person can do, because that's the face it's presenting. When that doesn't match up, it feels really bad. FV Speaking of the move to hardware: why did you decide to leave OpenAI when you did? BZ I was really excited to work on hardware, and this was the fastest way to do that in a way that I believed in and would quickly make it to market. Model labs are ideally a cool balance of long-term research and immediate product, and trying to bring the future forward into the market. I like making things, and it's more fun when you can share those things with the world, validate whether or not you did a good job, and figure out how you can improve. This seemed like the fastest way to do that for model-powered hardware, for tasks that I care about, in a context that I believed in, in a form factor that I would want around me. DOGFOODING Dogfooding is the practice of testing out a product internally before it is released to the public. DATA LABELERS Data labelers are people who create datasets of examples that AI is trained on to learn a certain pattern or behavior. MVP Minimum Viable Product THE BITTER LESSON A seminal paper, by Richard Sutton, which claims that simple methods, scaled, perform better than bespoke methods, that are tuned to a specific domain. HEURISTIC A heuristic is a set of explicit rules that can govern behavior. For example, whenever my favorite fruit is 50% off, I buy some. It's straightforward, and explicit, compared to intuition. OUT OF DISTRIBUTION If the model is not trained to know how to handle or expect something, something completely different than anything it's ever seen before, it doesn't know how to generalize or treat it. DIRECT MANIPULATION Direct manipulation is when you can directly interact with what you see on screen and get immediate feedback Hot Takes #1 What is something you hope never gets automated? Creative work, like music production or graphic design. Part of the reason I wanted to work on the bleeding edge of hardware deployment is that I'm excited for the model to take care of the things that I don't want to do: cleaning, laundry, etcetera. I love creative tools because the experience of the process and the discoveries you make along the way are just as cool as the final product. You can totally have a collaborative dialogue with a machine, but it needs to give you opportunities to meaningfully engage with what's going on. Also, bartenders. #2 Do you think design is a solvable problem by intelligence and raw compute? Yes or no? I believe that LLMs will allow for a degree of automation that will meaningfully alter the world. If I say that, then I think I need to say that with a sufficient degree of data and compute, many people's idea of functional design could be considered solvable – by the commercial consensus definition of "design." But you decide if that's true for you, by your understanding of what it means to design. #3 What do you want to be remembered for? Oh, god. If I do it all right, if I figure it all out, I think I want to be remembered for goofing around with the best of them. And being extremely honest, and gracefully forgetful.