NVIDIA’s Isaac Sim: Preparing for the coming era of robotics
When NVIDIA began to aggressively move towards self-driving cars, they also laid the groundwork for self-driving robots of all kinds.
At Dell Technologies World a few years ago they had a session on the future and robotics was to be one of the few very disruptive changes the market had to go through this decade.
We now have robots in development for manufacturing, telepresence, military use, law enforcement, security, such as in shopping malls and large buildings and event centers, and aids. digital technology for people with disabilities.
The remaining robotic problem
But our path to robotics is strewn with pitfalls. From actuators and cameras, to power, to just making things smart enough, there have been huge obstacles to moving forward. We have worked for the most part, but a problem persists.
This problem is how to train them effectively without having to physically guide them through each task. The fix was to use simulation where training days could be done in minutes, as you can speed simulation to machine speed and way beyond what a physical build could do.
But not all simulations were created equal, and while the use of game-based simulations seemed promising, that promise did not translate well into practice. The simulation, to function correctly, must reflect through the virtual cameras to the virtual robot all the richness of the physical world. Otherwise, the robot drive will be faulty.
For example, game-based simulations for training resulted in weird anomalies when robots were programmed with what was learned. They would do things like think shadows were solid objects and bypass them, dramatically reducing the efficiency and productivity of robots, while also creating potential dangers when robots attempt to dodge non-material shadows on objects.
The solution, designed to work with NVIDIA’s Jetson Xavier robotics platform, which 800,000 developers have adopted from 120 ecosystem partners for 3K customers, is emerging as one of the de facto robotics solutions.
Isaac Sim was developed to solve this problem. This is a simulation built on the Omniverse digital twin platform, which provides a realistic and accurate virtual representation of the natural world, allowing the import of various types of objects, including CAD files, to generate a simulated environment that is logically indistinguishable from the real world.
You can then tweak things and test pretty much anything you can imagine, from power outages and weather events to alien and zombie invasions if you want to. You can adjust the combinations of cameras and sensors, adjust the size and nature of the virtual robot under test, and even create the elements using elements that can only be theoretical if you can define the attributes of those theoretical elements. .
While this uses synthetic data, which requires a process to ensure that the simulated data is consistent with the world you are emulating, this level of flexibility is essential for creating the next generation of robotics at scale.
One thing that struck me was that this solution could be used in an exciting game which, in turn, could be used to develop skills with the platform.
The process of creating a robot that would fight monsters, zombies, or aliens is potentially the exact process needed to create a robot that will perform more menial tasks. If you can do something fun, more people will learn the skills to use the tool.
Imagine how much fun it would be to use a realistic location like, say, your home or office, and then build a robot, or a team of robots, to defend it on its own from zombies? You can also have virtual robot wars and battles, while learning the basics needed to design, build and train real robots.
I think NVIDIA is on the cusp of something exciting, and they have relationships with most of the major game companies that could help make this theoretical game possible.
As we moved to robotics, and AI for that matter, the critical issue of training became visible.
At least with robotics, one solution was to use the Omniverse platform to create a digital twin of the places you needed to train the robot, and then switch to a training solution that ran at machine speed.
This solution is called Isaac Sim, and the next phase is to train the next generation of trainers on this tool to accelerate the time-to-market of the robots that have been promised to us. I think the tool lends itself to gamified training as well, and I think eventually this is perhaps how most of us will become familiar with the tool.
Either way, the creation of Isaac Sim confirms that Dell Technologies is talking about years ago, making it more certain that the next big wave of technology, at least when it comes to hardware, is likely to be the next big thing. robotic.