Simon Yuen, NVIDIA

Tell us a bit about yourself – how did you get your start in visual effects and/or animation? 

I was in my third year in college, majoring in Computer Science and minoring in Art and at that point, I was not quite sure what I wanted to do after I graduated.  I remember thinking – I don’t want to write database all my life.   Then one day in the computer lab, someone was playing around with a 3D program – 3D Studio version 4 on DOS.   And I was in love.  It was my first time seeing a 3D program and right away I knew that is what I wanted to do.  It was a great combination of technical skills and art.

From there, I tried to find any resource I could at bookstores or magazines with anything related to 3D.  This was over 20 years ago so there were no schools or even much material on learning 3D.  So it was a lot of trial and error and learning on your own by just playing around with the software.  I had to figure things out without any manual or documentation.  Which was great and that philosophy gelled very well with visual effects back then.  It is this same skill that allows creators to create new effects that have never been done or seen before.  This is before tons of videos online that show you how to make fire or water.  Or how to make a realistic creature.  You have to figure all that out yourself through experiments, observation of nature, your own creativities, and the technical skills to execute that vision.

From there, I spent most of my personal time honing my skills to get to the point where I could start applying for jobs in the visual effects industry and eventually got my first break.

What was the first film or show you ever worked on? What was your role?

Zoolander.  I was a Generalist 3D artist.  Handled everything from modeling, texturing, shading, to animation, effects and rendering.

What has been your favorite film or show to work on and why?

Hancock.  Hancock at Sony Pictures Imageworks was a very under the radar type project.  People knew it was Will Smith attached to it, but it was not Spiderman where there is a lot of big buzz and expectations from everyone.  And the team was quite lean with a lot of experienced veterans on it.  With that setup, we were able to do some great work and really test out many of the character technologies we had been developing to great lengths and it all worked out really well.  At the end we were able to create a very high quality replica of Will Smith and other digital human characters.

What do you work on currently? 

I am currently the Director of Graphics and AI at NVIDIA, leading our Digital Human efforts.  NVIDIA Omniverse is an open platform that leverages the advances of  real-time 3D technologies and AI to simplify and accelerate the process of creating high quality, realistic Digital Humans and allows  many types of digital humans to coexist in the same world or worlds together.

What do you like about open source software? What do you dislike?

Likes:

Openness for the whole community, sharing, community development effort, can lead to industry-wide standardization more easily since many different parties / companies participate together.

Dislikes:

Maintenance, reliability, rate of improvements especially for production needs.  Since this is an open community development effort, getting good contributions from participants and agreeing on a standard can be time consuming.  But this is on a project-by-project basis and not true across the board.  Still the pros far outweigh the cons.

What do you think is the biggest challenge facing open source developers/projects across the industry? 

Open Standard Format is a big challenge that can really benefit from open source projects.  Standardizing on a universally accepted format for all 3D data has been difficult.  Areas like materials and rigging have been especially hard to standardize for their own reasons.  Open source projects and formats is probably the only way to achieve this global standardization.  Pixar’s USD (Universal Scene Description) has been one of the most successful and best attempts at standardizing a universal 3D scene description and all related components to a 3D scene.  This problem is by no means solved even with USD but it’s many steps in the right direction and now we have a platform to continue our work on.  It’s an evolving project that is gaining traction.

MDL and MaterialX are all great efforts to standardize material across the board.  It is also still in progress but recent years have made good strides in making these standards compatible with each other so they can talk together seamlessly.  I think we will see the fruits of this very soon across material and rendering solutions where going between different renderers carrying the same materials across will be much easier than today.

A good example of using both USD and MDL to help solve this issue with interexchange and overall 3D data standardization is NVIDIA Omniverse.  At its core it is based on USD and MDL and is a real-time simulation and collaboration platform.  Anyone with USD data can immediately use it and collaborate with different applications that normally can’t talk to each other without lots of export and data conversion steps.  With Omniverse and its support of USD, data can be shared and worked on live in real-time.

How do you think the Academy Software Foundation can help solve that challenge?

The Foundation is a great effort to unify the industry and standardize on standard practices, data exchange, and library compatibilities specifically for movie and TV related entertainment productions.  I think the Foundation’s role in having industry-wide veterans running this effort to the broad network it has with all the studios involved is already making a big difference in adoption of standards across the board and in some cases getting people talking together that might otherwise not be having these conversations.  Making exchange not only of 3D data, but process, core libraries and language compatibility possible is already a big step forward.

I think the Foundation just needs to continue what they have been doing for the two areas mentioned above, materials and rigging, to get a more cohesive standard across the board.  I think there will always be proprietary data and standards – that is hard to avoid completely.  But if we can even standardize on 80-90% use cases as a start, that would already be a big win for all.

What do you think is most important for the Academy Software Foundation to focus on in the next year?

Same as the above challenge.  Materials and rigging standards.  And including that with USD in mind would be most ideal, as that is already gaining traction to be the standard 3D interexchange format for the industry.

Where do you hope to see the Foundation in 5 years?

Unification of standards for data and software for content creation.  Data exchange should be a thing of the past.  And an easier exchange or adoption of 3D software, custom tools, and technology sharing when studios choose to do so. 

Another area to start putting more focus on is AI.  There will be more and more AI-based solutions and they will get better very quickly for production usage.  Keeping an eye on them early so we can create standardization and best practices early on is only going to help as AI technologies grow.

What advice would you offer other developers or software engineers interested in getting started with the Foundation?

Pick a problem that’s still a bottleneck today.  The things we spend most time on processing or ingesting.  Whether that is taking data from another vendor, or recreating toolsets to do the same thing – and use these to identify needs for standardization.  What would you like to see differently if most of your time is not spent on the creation process but rather doing mundane ingestion / conversion work.  Start there and come up with solutions that may serve the industry as a whole.  Engage with the Foundation or someone in your company who is active in the Foundation to get traction.  Get your efforts out there to help make this a better industry.