'In Silver's vision, truly intelligent agents would need long time horizons. Chatbots have brief interactions: The user asks a question; the bot answers. But humans are forever conceiving long-term objectives, and planning what they need to do next week and next month in order to realize them. Future AIs, Silver believed, would behave in the same way. Taksed, for example, to help solve energy scarcity by inventing a superconductor, and AI might draw up a reading list, conduct experiments, invent novel materials, and so on, pursuing its goal over the space of a year or more. As Silver and Sutton wrote, the models of the future would "actively explore the world, adapt to changing environments, and discover strategies that might never occur to a human."
"The kind of AI that we have today doesn't have a life. It doesn't have its own stream of experience in the way that an animal or human might have. And that needs to be changed, so that we can have systems that keep learning and learning. We'll have coding agents that are just there, continusly improvng worlds code, and predicting which tools you'll find more useful. They'll just be beavering away on all this stuff in the background.
Or let's say you tell your agent that you want to learn a new skill - speaking Japanese, for example. It will go off and build an app for that. And then it will teach you in a way that's optimized for you. And you'd get better on some tests. And your improvement would reward the system, so that it learned how to be a better teacher even as you learned to be a better Japanese speaker.'