Putting the science in fiction - Dan Koboldt, Chuck Wendig 2018
CGI is not made by computers
Things to know for when Skynet takes over
by Abby Goldsmith
Does your novel or story involve computer-generated imagery (CGI)? Does it take place in virtual worlds, like The Matrix films, or books such as Neal Stephenson’s Snow Crash (Bantam Books, 2000) or Scott Meyer’s Off To Be the Wizard (47North, 2015)? What about massive multiplayer video games like the one in Ernest Cline’s Ready Player One (Crown, 2011)? Does your story include technological illusions, or digital manipulations of people’s sensory input, like in TV series such as Westworld, Dollhouse, or Lost?
As a 3D animator, I shake my head with exasperation every time I see an author’s wrong interpretation of how video games or virtual realities are created. Hyper-realistic graphics don’t pop into existence by magic. Unprecedented visual effects cannot be created or maintained by one lone genius, or even by a tiny secret crew.
Yet this is the implication in novels such as Ready Player One, The Three-Body Problem by Liu Cixin (Tor, 2015), and Daemon by Daniel Suarez (Dutton Adult, 2009). These impressive, immersive virtual reality games were apparently created by a secret team that is so improbably small, they’re able to remain untraceable. In the Otherland series by Tad Williams and the Wool series by Hugh Howey, the secret team is apparently small enough to vanish without anyone noticing. Oh, and the amazing graphics are apparently self-sustaining, now that the team is gone. The massive virtual reality doesn’t need upgrades or maintenance at all. Ever.
I’m left to suppose that magical artificial intelligence fairies are adding creative graphics and artistic touch-ups, as needed.
The misconception about CGI
In these fictional examples, and in many others, the book or TV show implies that a tiny team of artists—let’s say no more than twenty—are able to populate an entire world with graphics so amazing they’re indistinguishable from reality.
This conceit betrays the author’s ignorance of how visual effects are made. Computer-generated imagery is a misnomer. The impressive visuals you see in films and games are not generated by computers, but by large teams of human artists who put in long hours of work. They always have been, and they always will be, unless our technology passes a singularity.
Sure, the artists use software tools, such as products by Autodesk and Adobe—software written by human programmers—but the artists using those tools have traditional training in painting, sculpture, or animation. They’re often overworked and underpaid. Many have aspired all their lives for a career in the entertainment industry and have practiced their art skills from childhood through college.
Where are they, in the mega-corporations of fiction and TV shows? It’s as if they don’t exist.
During the Renaissance in Italy, artists such as Botticelli and Donatello pioneered realism techniques in their artwork, imbuing their paintings with dimensions of reality that had been lost to Europe during the Middle Ages. As Leonardo da Vinci and Michelangelo competed for projects of epic scope, they impressed the mass audiences of their day. A Renaissance-era visitor to the Sistine Chapel might have attributed the magnificent ceiling to divine intervention. Perhaps the artist’s hands were guided by an immortal presence, or perhaps he drank special wine or used special paintbrushes.
I believe we’re undergoing a similar renaissance in art today. Visually spectacular films such as The Lord of the Rings and video games such as The Elder Scrolls V: Skyrim, look too authentic to have been created by human hands. Backgrounds and creatures are more detailed than anything painted by the old masters. So mass audiences assume there must be a secret ingredient. Unaware of cutting-edge proprietary tools at the disposal of digital artists, and unaware of film-grade training programs such as the Gnomon Workshop, the average person attributes hyper-realistic otherworldly imagery to “computers.” They credit the tool, as if the artist is just an unskilled, interchangeable automaton.
The vital role of artists and animators
It’s easy to ignore artists and animators. They’re often stashed in overcrowded buildings in unremarkable industrial parks. The teams are so large that no single person can be wholly credited for a particular scene or a particular character. Burnout rates are high, and many artists are fresh out of college or immigrants from other countries. A lot of film work is outsourced. On top of that, showrunners, and actors such as Andy Serkis, take credit for creating CGI characters. They claim responsibility for bringing these characters and fantastic worlds to life without acknowledging the enormous teams of artists who do most of the work.
When you see Spider-Man swinging between skyscrapers or King Kong roaring in chains, you can bet that some 3D animators made those characters move in a realistically convincing way. Motion capture can help artists, but it’s only a starting point. Human facial expressions and motions do not wholly translate to an ape, or to a cartoonish human, or anything else that isn’t real. Motion-captured facial expressions and movements are tweaked by animators. Every frame of film footage is artistically altered or exaggerated in order to make the character look less fake and more alive.
The process of creating a CGI character starts with a screenplay and storyboards, or a game design document. In pre-production, concept artists are paid to sketch visual concepts of every character, environment, weapon, and whatever else will be involved in the final product. A director or a committee approves the final character designs, ensuring that each character will fit within the overall mood and world of the film, game, or franchise.
Whether they’re creating superheroes, dinosaurs, or talking cars, the general process is the same. Someone models the character or creature using a 3D program, such as ZBrush, Maya, Blender, or Autodesk 3ds Max. Skin and clothing textures get mapped to the 3D wireframe mesh, sometimes by a different artist. A specialist rigs the character to give it an underlying skeleton for the ease of pose manipulation. Each skeleton has to be custom fitted to the 3D wireframe mesh or the character will deform in unnatural ways during motion.
Once the rig is finalized and tested, the 3D animators can begin their work. Animators spend a lot of time studying motion, timing, and the way quadrupeds and bipeds move. Cinematic animators practice their craft with an emphasis on acting. Many take acting classes. Game animators tend to specialize in smooth loops, or cycles, as well as exaggerated motions that read well at a tiny size on the screen.
A programming team will script code that enables fur, hair, or scales to look realistic. Another team handles physics, such as fire, or objects that flop about. Another team handles lighting, with the purpose of making scenes look natural given the simulated weather conditions or environmental conditions.
The hyper-realistic backdrops of SF/F films are painted with a blend of photography and digital enhancements, plus the traditional painting techniques taught for generations. Costumed actors and sets are then composited onto those backdrops. It’s called digital matte painting. A similar process is used to create the environments in high-end video games.
I spent my twenties as a 3D animator, and fairly often I’d hear non-industry people try to guess what my job entailed: “So, you draw on a computer?” Rather than launch into a long explanation, I’d say, “Yep, pretty much.” But although I love drawing and spent years honing my sketching craft, those skills are just a subset of what I actually do. CGI work has more in common with sculpture and stop-motion animation than with drawing. Animation is more about a sense of timing, and of weight and motion, than painstaking illustration. Animators tend to be sketch artists rather than detail-oriented illustrators. I’m not meticulous; I’m impatient! I want to make a character kick ass rather than gaze off into space with photorealistic beauty.
However you define CGI, it is creative work, and it requires artistry. Computer code lacks an imagination. Code can translate data into visual representations of terrain or geometry, but it cannot visualize spaceships or dragons. A skilled programmer can write a script to simulate aspects of visual elements, such as lighting and physics, but programmers do not create the visual environments or the visual characteristics of characters. Software cannot pull together all the data of human perceptions and create something imaginary from it. That’s up to human artists with creative minds.
Pre-made versus custom CGI
It’s possible that a lone genius can avoid hiring a huge team of artists by buying prepackaged environmental elements and prepackaged CGI characters. Online marketplaces such as TurboSquid, CGTrader, and Renderosity offer loads of 3D assets for sale. However, keep in mind that other low-budget gamemakers and filmmakers purchase and use the same assets. They tend to look prepackaged. If your virtual world needs customized anything, this is not the way to get it.
For customized work, you need to hire someone. And much like hiring a writer, it’s hard to find someone who is the perfect match for your needs. The 3D artists and animators who advertise their services are doing so because they want to build up their portfolio or demo reel. They’re inexperienced. Professional high-end artists with an impressive track record tend to be employed full time, and they value career stability over freelance gigs. They’re unlikely to want to juggle extra work hours, and they’ll be suspicious of sketchy corporations or weird geniuses. There are plenty of those in the entertainment industry. You learn to be wary.
The budgets of major films and games sustain hundreds of skilled professionals who are working intensively, full time, for several years. If your story includes a project with visual effects or immersive entertainment, please remember the team that must have created it. Every element in your digital world was planned, sculpted, textured, rigged, animated, and implemented by a lot of artists.
It’s not a magical process. Only the final render looks like magic.