AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
Dota 2 logo nvidia keynote9/12/2023 ![]() The Finals version plays with 17 heroes-we removed Lich because his abilities were changed significantly in Dota version 7.20. We believe these issues are fundamentally solvable, and solving them could be interesting in its own right. ![]() Imagine how hard it is for a human to learn a new hero when everyone else has mastered theirs! We haven’t yet had time to investigate why, but our hypotheses range from insufficient model capacity to needing better matchmaking for the expanded hero pool to requiring more training time for new heroes to catch up to old heroes. NVIDIA Reflex is a must-have in games, reducing system latency so your actions occur quicker, making you more competitive in multiplayer matches, and making single-player titles more responsive and enjoyable. Although they were still improving, they weren’t learning fast enough to reach pro level before Finals. We spent several weeks training with hero pools up to 25 heroes, bringing those heroes to approximately 5k MMR (about 95th percentile of Dota players). We hypothesized the same would be true going to even more heroes, and after The International, we put a lot of effort into integrating new ones. We saw very little slowdown in training going from 5 to 18 heroes. This isn’t the end of our Dota work-we think that Dota is a much more intrinsically interesting and difficult (and now well-understood!) environment for RL development than the standard ones used today. We are retiring OpenAI Five as a competitor today, but progress made and technology developed will continue to drive our future work. But we think decreasing the amount of experience is a next challenge for RL. ![]() ![]() This limitation may not be as bad as sounds-for example, we used Rapid to control a robotic hand to dexterously reorient a block, trained entirely in simulation and executed on a physical robot. The surprising power of today’s RL algorithms comes at the cost of massive amounts of experience, which can be impractical outside of a game or simulated environment. The results exceeded our wildest expectations, and we produced a world-class Dota bot without hitting any fundamental performance limits. To build OpenAI Five, we created a system called Rapid which let us run PPO at previously unprecedented scale. ![]()
0 Comments
Read More
Leave a Reply. |