Designing NVIDIA's Isaac

I had the privilege of working on the Isaac project at NVIDIA. I was responsible for the concept design. I also directed the various aspects of the art team including environment, assets creation and animation. Big thanks to Rev and Jensen for the opportunity.

I was asked to design a robot based on Rethink Robotics’ Baxter. The initial design brief included words like “friendly” and “adorable”. The name Jensen chose for this new robot is “Isaac” in honor of Isaac Asimov. Isaac would be an AI-enabled robot that would be trained using a simulation environment called Isaac-Lab. Once trained in the Isaac-Lab environment, the knowledge can then be deployed and transcended between the physical and the virtual realms. By developing and training robots in a simulated world and then working with those robots in a virtual reality environment, researchers can deploy them to the real world in a way that is safer, faster, and more cost-effective. 
Whatever Isaac would learn in the virtual environment, it would then be transferred to the physical robot - Baxter. That is why the design had to be based on Baxter.

The reason Jensen requested Isaac to look ‘adorable’ is because we wanted to communicate the idea that the robot is still in a ‘kid’ state, a learning stage which is connected to its goal of being trained in the Isaac-Lab.

I felt that ‘innocence’ was going to be a crucial part of its feel. It is the same type of innocence that we perceive when observing children being curious in front of something new that is presented to them. While I was provided with a 3D mesh of Baxter for reference, I wanted to get acquainted with how the design ‘felt’ in real life.
Taking a look at a picture of an object, observing its 3D model, and analyzing it in real life are all vastly different experiences.

As I was going around Baxter and observing it move the wooden cubes and toy cars up and down, I understood that the original design tried, in its own way, to be ‘friendly’-looking, – which is the same goal  I was tasked with in my design.

There’s a few design choices in Baxter that are worth noting. First and foremost the shapes are forcefully “roundified”.  It is quite evident when looking at its arms, the cylindrical casing is not simply cylindrical. Instead it presents features that I personally call “lumpy”. This is a choice typical of children toys and in general products for infants, which is something that has some value in that context, but does not necessarily translate well on a 5' 10” robot. I understand 'why' they did it, but I do not personally agree with the decision. Below more about that.

Another major design component is the stall where the upper body sits, it is surprisingly quite uninspired. It is just an office chair looking element that doesn’t fit with the rest of the design.

 Not much love left for the stall

Not much love left for the stall

The only constraint I had to keep in mind when designing Isaac was related to the arms. Isaac’s arms needed to keep the same joints and functionality layout that Baxter had since the arms would behave like the real life version. Other than that, I was given carte blanche on what I wanted to achieve design wise

Baxter uses an LCD screen as face, which arguably has certain advantages in real life. However my design would need to work for the VR experience and training, so an LCD screen would not be relevant.

Designing for a virtual product, rather than a real one is quite different. There are certain freedoms you can have in a virtual design that would not work in a physical one. The two also have different constraints and requirements. Isaac in this instance is not just a “product” but a character as well, a being which we can feel, perceive and interact with.

One thing I kept in mind since the beginning was the design of its face, and particularly its eyes. The eyes would have very large and round features to keep the cylindrical tune going.
The arms would have cylindrical shapes as well. I tend to sketch in small quantities, just enough to get some good vibes about the direction I want to pursue. I spent about 15 minutes on this.

I then hopped on Fusion360 and quickly made a block-out of what I wanted to do.

I kept going from there entirely in 3D and developed the rest of the upper torso. In my first iteration of the torso, instead of keeping some similar negative spaces to the ones in the real torso (which I think give it a really nice harmony and break-up) I kept the front purposefully flat. To create contrast with the rest of the rounding shapes, I placed four cylindrical headlights on its front. I was picturing how it would look if it was placed in a sort of low-light scenario and I thought it would look pretty cute with some tiny headlights on.

I also decided to add two little electrical engines at the back that would make it look like its power source, because as I mentioned,the robot I am designing lives in a virtual world where he is a living character, so adding elements like the motors help convey that idea.

This was the first iteration for Isaac. I felt pretty good about it but was not entirely satisfied. At this point, I wanted to start to work with the actual color scheme that I had planned. Adding color and materials changes the perception about the shape and how we feel towards the product we're creating. In Isaac's case, being a product and a character this is just as important.

Baxter uses a bright red as its main color, blended with mostly neutral grays and black across the body. I personally don't shy away from color when designing, however when I do, I try to do it with awareness of its function or because of the feeling I want to communicate.
When I look at Baxter's red, I think of red and its strength as a color. It's one of the primary colors. It is visually strong and vigorous, and due to this, it is often used for emergency or medical signs. I think ultimately the color scheme Rethink Robotics picked goes against their own body design language which was clearly trying to be "approachable" and "friendly". It just feels a bit overwhelming to look at.

Combining the fact that this was NVIDIA's first internally designed robot with Jensen's brief, the concepts I had in mind were "innocence", "life", and "purity". White is the choice that I went with for Isaac's main color. White is perceived as the color of life, the color of birth.
White is the color of bones, the color of eggs, white is the color of milk. It is also the color perceived to be connected to the creation. Not by coincidence usually the entrance to the afterlife is presented as a glowing white.
White is a color which communicates purity, deity, but as well as innocence and quiet. Going back to the character element, Isaac needed a secondary color for its accents.

I know it might sound odd, but I had been thinking about white life items such as bones, milk, eggs and it struck me that, the orange yolk was the suited choice for the secondary color.

 Digital egg yolk

Digital egg yolk

Sometimes I love to render colors and materials I am going to use in the form of simple cubes sitting on a surface, to get feedback on how I feel about those being together. At this point I decided to do a quick render pass where I would apply the following criteria: All main elements would be white. Orange would be used as an accent color, in a few spots to create the right amount of contrast. I also added two more materials that would make sense for its character. A rough aluminium and a matte grey. Neutral non saturated colors that would not disturb the balance of the white and the orange.

At this point of the full design of the eyes and many other parts is in progress and will be fully addressed later. But placing basic materials and colors helped me "feel" the character more.

 First rough iteration with a material and color pass

First rough iteration with a material and color pass

Jensen wanted to nail the design of face first, so I started to do different 'mouth' variations. Neither of us was sold on the first shape I created yet. Below are the five most significant variations of the mouth that I created.

I felt like each mouth had pros and cons but ultimately decided to go with the variation number four. This zoomorphic variation is inspired by a tortoise's mouth. Tortoise mouths, to me, look charming.

At this point, it felt right to check how everything would look with the arms. As mentioned, the arms would have the exact same layout, except that I wanted to get rid of the "lump" features. The arms would be cylindrical and there would be nothing forcefully roundified in the casing.

In the above image I highlighted some of the "lumpy" features I was describing earlier, and in the same image just below, there's the arm design for Isaac. Streamlining these features was essential to get a more elegant look.

To their merit, Rethink Robotics has launched another robot called "Sawyer" in which they addressed this issue and removed the "lumpy" features entirely. Needless to say, it's a much more elegant design than Baxter was.

The torso piece of Isaac is extremely important and it's strongly connected to the arms. I am fond of the connection that the two flat circles that peek on the side at the back have with the rest of the arms; those shapes are there to echo all the other circle shapes on the arms joints.

They help balance the overall silhouette. At the back, as I mentioned, there's a few fictional electrical motors which power Isaac. As you can see, simplifying the arms shapes to a more honest cylindrical shape makes them more visually powerful because we can read these shapes better. It flows better and suits the rest of the design with better harmony.

At this point Jensen requested that the gap between the eyes be reduced, something which I agree was the right call. On top of that, at this stage the "soft" body neck was introduced to allow its neck to bend in all directions. Lastly, Isaac gained eyelids.

 The new face presents less gap in between the eyes and as well a soft neck

The new face presents less gap in between the eyes and as well a soft neck

When these changes were made I did some quick Photoshop paint-overs of the face, changing the expression slightly. It was the first "expression" and "emotions" test I conducted. The "ears", the eyelids, the eyebrows, and the mouth are all vital parts that will make Isaac feel alive when animated

 I did a quick over the 3ds Max viewport to test Isaac's expressions

I did a quick over the 3ds Max viewport to test Isaac's expressions

Since Isaac did not need to sit on a stand, and due to the fact that its arms had to be very long, I did not feel like having a regular "leg" design would fit the rest of its body. It would have looked extremely unbalanced and irregular. I decided to equip it with tread tracks, which I felt were perfectly in line with the type of character I was shaping.

While these were my favorite tracks, Jensen requested a flatter version which ended up in the final design. The very last component to change ended up being the chest.

Even though I liked the flat chest with the headlights, a smoother chest only made sense to further match the flow that Isaac had on the rest of the arms and the body. Jensen suggested that the chest should be more 'inviting' than the original fatter version I had created. One thing I added was the two round elements in the back that peek out like flat cylinders. I did that to echo the round parts of the arms and make it all match and tie together.

Isaac was modeled in Fusion 360 and the CAD model I had worked on was cleaned up with perfect quad topology by the amazing Andrej Štefančík. Andrej has been freelancing for NVIDIA, working with me on multiple projects and he is in my opinion one of the best sub-d modelers that exists, period. His meshes never fail to amaze me for how clean and tight they are. Isaac was no exception as you can see from these viewport screenshots. He was also responsible for the modeling of the high tracks.

Isaac was finally born!

At this point the whole concept and mesh was done. You can see the final results below. Texturing and lighting of the final renders executed by Andrej Štefančík, who did a fantastic job.

After this process was done, another pretty big chapter started. I was art director of the art team that put together the VR environment in which Isaac would be playing dominoes against people. That part itself deserves its own entry. Lots of work was done by a team of super talented people. One thing I wanted to share are some of the animations that Isaac performs which I absolutely love and were done by the super talented Brian Robison, who is the lead animator in my team. In the final VR experience (I will make a separate entry to talk about that) Isaac is sitting in a kindergarten classroom, playing dominoes vs people. The mesh was skinned and rigged by the super talented Johnny Costello, huge thanks to him too. He was involved in the low-poly mesh of Isaac quite a lot! The low-poly Isaac mesh and final texturing was done by Reno Levi who did an amazing job as well.

The VR version of Isaac needed to be adjusted in height to match Baxter’s height. When I originally designed Isaac, I was told it’d be OK to attach the arms in a lower spot, but ultimately that was not the case. This means the final design in VR the arms are attached at a slightly taller point instead of the metal belt on top of its pelvis.

I like both versions for slightly different reasons but I am overall more fond of the one with the original proportions I had created first. The design foundation was solid enough to allow for these changes on the torso and arm attachment and did not compromise its look and feel compared to the original stance.

In the animation above you can see stitched: Isaac performing a "Hello" wave animation, which he does when he spots people wanting to play with him, Isaac pointing out that it's your turn to move, and Isaac a bit confused because he did not understand the move. Other than a few scripted animations like these ones, the arms move on their own during the domino game powered by our amazing AI. So many people responded positively to the face that NVIDIA staff at Siggraph didn't have to explain the demo. People perceived the robot as a character, even that it had some sort of feelings they should sympathize with when something went wrong.

The demo was very well received at Siggraph and I am super proud of my team and also everyone else who made it work and made it possible, especially all of the engineers without whom none of this could have happened. More about that in a different entry. I wanted to keep this one strictly about the design decisions behind Isaac.

Please read more about the Isaac-Lab and Isaac here