Senior Software Engineer - Architecture Lead
Walt Disney Animation Studios
Tell us a bit about yourself – how did you get your start in visual effects and/or animation?
I’ll have to start at the beginning. My parents immigrated to the US from Ecuador shortly before I was born. My father, now retired, was a civil engineer, my uncle a dentist, and my mother an accountant, and they brought with them a strong entrepreneurial work ethic that they instilled in us from an early age. The trio started and continue to run their own business to this day, which has always been an example to me as to how you can create something from nothing when you have the space to experiment, fail, and try again with the wisdom from past mistakes. I didn’t speak a word of English at a young age, but the neighborhood kids were so friendly that we basically picked it up in no time after we moved to a new neighborhood at around age 3. I can’t discount the fact that my ability to learn new programming languages was probably helped by having to quickly learn a new language as a kid.
I’ve always been a tinkerer. When I was a kid I was always doing things like taking electronics apart to see how they work, see what’s inside, and see if I could “fix” them or at least put them back together without breaking anything. I remember going through a phase where I would mix all kinds of random household chemicals / solutions together to see what happens, and luckily, I didn’t hurt myself or my siblings in the process. I did end up learning that stuff can be toxic and corrosive, as the plastic lens of one of my play microscopes was one of the unfortunate tragedies.
In grade school, my 3rd/4th grade teacher was really into computers and had our school hooked up to what I now realize must’ve been an early pre-internet darpa net. Her name was Mrs. Gilkenson, and I didn’t realize how much of an influence her curiosity had on me. I have a fond memory of receiving an ascii-art Christmas tree email from our school principal; that must’ve been sometime around 1987/1988, which is kind of amazing to think back on.
When I was in high school in the mid-90s, my best friend Peter Ta turned me onto computers, the DIY spirit of “build your own box” computing from that era, the fun of DOS shareware/warez games, and the pre-internet bulletin board system (BBS) culture of dialing into local BBS’s on our modems. Peter sadly died in a tragic car accident alongside another friend before they finished high school, and I’m forever grateful for the world of wonder that he imparted in me in our brief time together.
Like many who grew up in that era, I was heavily influenced by ID Software’s Doom series and spent many hours playing death match games with Peter and friends on a local BBS called “Jungle ][“. Beyond computers, playing music and drawing were my favorite pastimes as a kid, so it wasn’t a far leap to start exploring how to be creative on a computer. One of the really fun things about these early games was that they were heavily user-“moddable” and the tools for creating your own graphics, music, and game levels were just a BBS download away. You didn’t have to have specialized tools or anything beyond just a regular PC from the era. Making the leap from a player to a creator was easy and fun, and I think that’s something that might not be so accessible these days in the closed walled gardens of iPads and consumer computing.
I don’t completely remember if the image file format was called “pcx” or something along those lines, but I do remember drawing in a DOS-based image editor to create the sprites for a Doom level we were making with some friends from the BBS. I remember creating the level design, sprite graphics, and providing some of the silly sound fx for the “Jungle ][” wad, which still exists online it seems! Disclaimer: this was created by kids, so it’s completely fun and silly:
The joy of creating something, being able to share it with friends or anyone who wanted to download it, and getting to see it “for reals” by playing it with friends online is really an addictive experience. That little computer had so much potential. The BBS scene then led to me becoming aware of the “demoscene” and computer graphics demos from groups like Future Crew that seemed like magic. Again, I was in awe of what could be done, and it sparked more curiosity and learning.
I was always into playing and creating music, and one of the artifacts from games and the Amiga demo scene was the idea of “mod trackers,” which were truly unique programs with a powerful and quirky interface (Pixar’s internal “mdt” editor gives me happy flashbacks of mod trackers). They would turn your PC into a music sampler + sequencer capable of taking a handful of small sound loops and instrument samples and composing them into songs that could be played in real time using computers of the era. I came in during what I consider the “2nd wave” of mod trackers, once the ideas and file formats from the Amiga had been implemented for the less-interesting but more ubiquitous PC hardware of the time. Electronic music was kicking into full swing in the 90s, and realizing that I could create my own music using just my computer and imagination is another pivotal influence that led to where I am.
When I went to university, I took all of these experiences with me, and had an interesting experience in that I had signed up to be a Computer Science + Physics double major. Some of the worst advice I ever got was from my university counselor who told me that I “wouldn’t ever need that CS stuff,” and he convinced me to drop CS and become a pure Physics major. Little did I know that the reason for this “advice” was that there were only five other students in the entire Physics program, and I was being duped to try and get more funding for the department. I would have enjoyed it, except I’m not a morning person and had a really hard time getting up for 8AM chemistry lectures. I was an over-achiever as a kid, and not doing well in a college course due to my immunity to alarm clocks was a shocking experience, so I panicked. This was the worst advice because it led me to completely switch gears and switch to a Fine Arts major, which while it was something I enjoyed, it was not satisfying my inner tinkerer. After a couple of years into the program, I realized the mistake I had made by following someone else’s advice and took it upon myself to get back into Computer Science, which is a field of study that just came naturally for me. I “hacked” my way into a grad school program by skipping all of the “Intro” and “boring” intermediate classes and signing up for all of the upper-division CS classes as electives; these were the very same classes that were pre-requisites for the graduate CS program. While I didn’t have an “official” CS minor, I was able to tread a path where I could get into the CS masters program and finish what I had originally started. I was the only non-CS major in all of my graduate courses, and my circle of friends tended to be from the Art / Film circles, so it’s interesting to think back on how this might influence where I ended up in the future.
Needless to say, I continued tinkering the entire time, and during the early 2000s, the Web was full steam ahead. I had picked up Perl on my own and after a couple of small jobs out of school, I randomly stumbled upon a posting on jobs.perl.org for a role at Walt Disney Animation Studios. I was lucky that they were looking for someone to maintain some Perl-based systems at the time, and I landed a gig on their “dev-ops” (before “dev-ops” was a thing) infrastructure team at a pretty young age. Everything that came after was basically a continuation of that early opportunity. I’ve experienced what it’s like being the youngest in the room, the oddball, and the only one with my background. I’m certainly not the youngest anymore, but I very much understand what it’s like to be the “only one” in a room.
What is your current role?
I’ve spent the bulk of my career at Disney Animation, and it’s been a great experience in that I have been able to work on a lot of different projects. I started in 2004 on a team that was focused on Build Engineering, but have had a number of great experiences on different teams over the years. I had the pleasure of being a founding developer on our internal Playback/Review system, which is still in use to this day both at Disney Animation and ILM, and a lot of our foundational code still lives on to this day.
I got to work on an internal version of Nothing Real’s (and later Apple’s) Shake compositing system and ported the entire system to x86-64 Linux, which was a fun exercise in learning about x86-64 assembly and evolving a medium/largish C++ project in a tight feedback loop with artists. We were still evolving the system up until we switched over to Nuke for Tangled in 2010. Some of the usability improvements we made to Shake were immediately requested as custom plugins when we migrated to Nuke, which was satisfying since it showed the little things like an attention to usability is something that artists really appreciate. I’ve worked on our pipeline team, dove deep into Renderman and Ri Procedurals, and generally tried to absorb as much as I could from the talent around me. I had a great deal of fun working on our Hair / Procedural Geometry system starting with Wreck-It Ralph and got to learn first hand about the rewarding challenges that come along with evolving legacy software systems to take advantage of modern hardware. Legacy software is ripe with performance opportunities; making a single-threaded application into a NUMA-aware multicore application is both challenging and enlightening due to the tradeoffs that have to be made between time constraints and implementation choices.
I’ve since circled back around and am now an Architecture Lead at Walt Disney Animation Studios for a small “Code Force” team that is focused on developer tools, CI/CD, devops, and evolving our developer workflows. One of the projects I worked on in 2007 was taking a big bet on Git and moving all of our software development workflows to a distributed Git-based model, instituting code reviews, and pushing the needle forward on automation.
I’ve always worked towards enabling a culture of experimentation and developer freedom to play and try wild ideas with minimal effort by removing all technical roadblocks that would prevent someone from trying something new. For years, I’ve had to ignore naysayers that have perpetuated myths like “Git is hard,” and generally are resistant to change. Reality is much more nuanced, and the truth is that artists and developers alike are able to be extremely productive, with little friction, when the right guidance and tooling is provided.
What was the first film or show you ever worked on? What was your role?
Chicken Little! This was when we were first planning out our x86-64 plan, and our pipeline was heavily Perl-based. I was mostly evolving our build system at the time. Our next film, Meet the Robinsons, was the first film that used my x86-64 version of Shake, and thus was the first movie where I was getting pixels on the screen. We had fully migrated onto x86-64 Linux by then and were ready to shut down our last 32-bit Linux and older Irix SGI workstations by the time that movie wrapped.
What has been your favorite film or show to work on and why?
Moana and Zootopia. We did a lot of work on our procedural hair/fur system for the furry animals in Zootopia, and seeing it all animated was really awesome. That movie helped us scale up the performance of the system immensely, and seeing the results were really awesome. Moana is still one of my favorite movies — especially the music. We did a lot of work on our procedural geometry/hair/fur system so that we could handle the curly hair, huge island-scale procedural geometry, and crowds that were in the movie. We couldn’t have made that movie without the improvements we made to the system, and the collaborative nature of tackling the problems alongside technical directors and artists was extremely satisfying.
Here are some links about our work on Zootopia:
Web “1.0” and Perl is what got me into open source. Once I realized that I could ditch the buggy Windows 98 and Win2k OSes that I had been lugging along and install Linux on my computer, everything started to click. I realized I could run my own Apache web server, which was much better than the IIS stuff I tried initially on Windows. I grew up on DOS so there was something very familiar with typing away at a terminal, except it was powerful and not janky like I remember DOS being as a kid.
Having my own Linux environment allowed me to avoid having to walk halfway across campus to use the Sun Workstations for my C/C++ coding assignments, and that made me very productive. I got to the point where I could write most of my assignments on my desktop in my dorm room and then only make a final trip to the Engineering Lab and its Sun Workstations to make minor portability adjustments to account for the differences in the compilers / environments. I avoided Visual Studio and the predominant Microsoft culture at the time, as I didn’t want to invest any time in locking myself into a walled development garden. I really enjoyed being able to fully understand how a system worked, and the unix model of small composable tools just made more sense to me.
Learning about and becoming proficient in the Linux ecosystem was a breath of fresh air and freedom. I’ve always been into the ideas of DIY, punk, and making things, so it didn’t hurt that the philosophy behind GNU/Linux was aligned with my principles.
What do you like about open source software? What do you dislike?
When you start out, you can build upon the shoulders of giants with nothing but an internet connection. That is huge. It’s great that you can naturally progress from being a user of tools to being able to enhance them and contribute back to the community.
I dislike the movement away from the DIY spirit of knowledge sharing and creating things towards one where the big players build off of open source projects and then want to collect rent inside their walled gardens. New developers these days are being trained to be cloud proficient rather than being exposed to the DIY approach of running things from scratch and learning. I see this as the single biggest problem in open source — the reliance on cloud and someone else’s computing resources. Centralization is horrible. The VC culture from Silicon Valley has basically enabled and rewarded this behavior, unfortunately.
The resilience of open source has been able to withstand forks due to change in ownership, for example Hudson to Jenkins and MySQL to MariaDB, but it’s a troubling pattern where companies try to shut down or take ownership over community projects.
Which open source projects do you currently use or contribute to?
I created Git Cola as a way to learn about PyQt and Git, and to fill in the usability gaps that existed at the time. We still use Cola a lot at the studio, and being able to keep it an open project has been a great thing since it is a tool that I use daily both at work and at home. I’ve learned a lot from contributors, and have managed to keep it going for over a decade now. As part of that, I’ve been able to contribute back to Git itself, especially during the early days of the project, by writing tools that our developers requested or that Cola would benefit from, and getting them upstreamed into core Git itself.
Contributing to Git was an extremely rewarding experience. Their code reviews were always extremely helpful, and I learned so much about code quality and maintaining a community project by involving myself in the community. I haven’t had as much time lately, but I do still have a handful of patches that I will be working on sending back upstream in the future.
I have a handful of other projects that I help maintain (some not so well, sorry) and contribute to on my GitHub page, but none of them are quite as well known or used as Git Cola, jsonpickle, or Git.
What advice would you offer other developers or software engineers just getting started?
Always try to dig a few levels deeper than the tools that big companies would want you to use. Invest time in free software tools that are not tied to any particular job or workplace. Contribute back to Debian if you can, and run it at home. Stay independent from tools that don’t allow you to look under the hood and improve on your own.
In your experience, what has been the biggest barrier to entry for participating in an open source project?
Luckily this is one of the strengths of open source — if you have a computer and free time then you can start participating. You can email the project maintainers directly and participate in development. The biggest barrier these days is projects that are tied to a commercial service, and the project is a gateway to being locked into a particular company’s service. As we build complexity, we make it harder for tinkerers to master a system and be fully self-reliant.
Some development communities do have a higher barrier to entry, particularly older ones with established email-based workflows that can seem foreign or odd to new developers. There’s a balance between bringing on new developers and making it so easy that projects become inundated with low-value contributions. There’s a balance to strike here, and even the Git project has done a good job of enabling new conduits into their established development models by creating things such as https://github.com/
Another barrier to entry is having to sign CLAs in order to make contributions. The DCO model is simpler and less friction, and doesn’t require developers to involve a lawyer in order to upstream their contributions.
How do you think we can encourage more diverse talent to participate in open source communities?
Expose students to open source development communities earlier in their development career. I wish I had known about open source earlier on. Universities should stress the importance of open tools, and they should incorporate them into their curriculum. My high school would have been better off with a lab full of Linux machines rather than Windows workstations.
Projects that are tools for creators (music, graphics) will probably have the easiest time attracting diverse talent. Focusing on some of the users, especially ones with diverse backgrounds, and spotlighting how they use the tools to create something tangible and accessible can go a far way. Exposure is the first step. The talent pool exists; connecting the dots is by no means an easy task though. It requires deliberate steps by educators, users, and professionals alike to expose, mentor, and teach. Projects like Google Summer of Code are also a great way to bring students into the culture. More programs like that would be a boon.
We also have to find ways to make software fun for kids. I find myself looking for fun ways to instill the tinkerer spirit in my nieces and nephews, and I am all ears on ways to bring the fun creative spirit that I experienced as a kid into computing.
Is there something you wish you had known earlier in your career that would have helped with your professional development? Or, are there resources you wished you could have had access to?
I wish I had known that I could get a priceless education in C and Unix from experts in the field at no cost, by contributing to open source projects. I learned more by working on Git than I did from any courses or other formal education.
What is your vision of diversity for the Open Source community and the Academy Software Foundation?
That anyone, with minimal equipment and commodity hardware, can use and build upon projects for the purposes of making new tools and running their own experiments. We should highlight the underrepresented and favor the use of open systems and tools so that efforts can be put towards improving the community’s tools to the benefit of everyone. Software can be used for anything; there’s no reason that open source communities should be male-dominated or perpetuate stereotypes about who is involved.
ASWF should strive to have participation from a balanced community, and that includes more than just software developers. One of the biggest eye-opening experiences for me was when my tools became internationalized and translated into multiple languages. We suddenly started receiving contributions from users from around the world because we started talking their language. Internationalization is a good topic because it will naturally result in friction because projects are often not written with i18n in mind. Any effort to move the needle on that topic will ultimately result in many changes to existing stable code bases, and the ASWF can help projects by championing these kinds of efforts and by putting quality controls in place that guarantee that any new experimental feature will not result in regressions.
This plays into portability in that projects should, within reason, support a wider range of platforms / compilers to maximize their potential user base.
What is your involvement within the Academy Software Foundation? Can you sum up your experience so far?
I’ve contributed a few patches to some ASWF projects, but most notably I’ve been involved with the ASWF’s Python 3 Working Group. Our general mandate is to help studios large and small make the transition to Python 3. We’re aiming towards providing guidance on how to approach a Python 3 transition from a studio’s perspective, as well as getting the alignment across third-party vendors on how they plan to incorporate Python 3 compatibility into their commercial packages. An important aspect of this is getting alignment from vendors as to when they will eventually retire Python 2 support in the future so that studios can plan ahead accordingly.
What do you think is most important for the Academy Software Foundation to focus on in the next year?
ASWF should work with distro maintainers from Debian and Fedora to make sure that our projects are easy to integrate into their ecosystems. Diversity and inclusion are every bit as important for the long-term health of the community.
Where do you hope to see the Foundation in 5 years?
ASWF will have adopted a handful of other projects. Python 3 will be behind us, with the majority of large and small studios having already made the switch. Projects will be receiving a steady flow of improvements from contributors around the world, and the contributions will be from a wide diverse range of individuals with different backgrounds and expertise. We might even have an open ASWF Conference that is free for students and subsidized by studios so that everyone can come and meet in person.
What do you like to do in your free time?
It goes without saying that I like tinkering with software in my free time. I like to draw sometimes, though it’s true that I do tend to spend too much time in front of a computer these days for my creative pursuits. I enjoy producing music with my friends, and we’ll sometimes use Schismtracker when we do it. One of my great joys was introducing a non-technical musician friend to something as arcane as mod trackers and watching them run wild with it and use it for creating new music. I have a small collection of analog synthesizers and musical toys which are always fun to use as sample sources when writing.