Art Is Not Mere Simulation
On the Quest for The Real in Video Games
Recently, Nvidia announced a quantum leap in graphic technology: DLSS 5. This new tech leverages artificial intelligence to achieve immense visual fidelity. By using neural networks, the AI can generate details on the fly in a consistent way.
As the press release put it,
“Twenty-five years after NVIDIA invented the programmable shader, we are reinventing computer graphics once again,” said Jensen Huang, founder and CEO of NVIDIA. “DLSS 5 is the GPT moment for graphics — blending hand-crafted rendering with generative AI to deliver a dramatic leap in visual realism while preserving the control artists need for creative expression.”
As evidence of their new technology’s effectiveness, they presented side-by-side screenshots of games like Resident Evil 9: Requiem.
From a purely technical standpoint, DLSS 5 is a remarkable step forward. I am reminded of the way gaming was when I was a child. Growing up, every new generation seemed like a dramatic visual improvement.
I remember my jaw dropping when I saw the water in the first Uncharted game, or the first time I played Warframe on my brand-new PS4. Every time, I told myself graphics could not get better. And then they did. DLSS 5 feels like a return to that era.
At the same time, this technical accomplishment has created backlash. There is no shortage of bloggers blasting Nvidia for creating more AI slop.
The conflict over DLSS 5 is a conflict at the heart of the video game industry. By its very nature, video games are both an art and a science. They blend together the humanistic, expressive side of our being and the technical, scientific side—the “right brain” and the “left brain.”
Sometimes art and science conflict with each other. The scientist wants to innovate. They want to push the technology as far as it can go. The artist, however, wants to imagine. They want to invite others into a space that does not yet exist.
These goals stand at cross-purpose with each other. The better technology gets at visualizing ideas, the less space there is for the audience’s imagination. They no longer have to imagine Grace Ashcroft because there she is, in glorious 4K resolution.
Video games, then, embody a duel between the Artist and the Scientist. This is a duel over how best to reach Reality. Is the faculty for perceiving reality our imagination, or is it our senses? The Artist will defend imagination as the prime way of perceiving Reality. Consequently, their main tool for perceiving reality is art. The Scientist, by contrast, will argue that our senses are the only way to truly grasp Reality. In cases where direct perception is unavailable, then, the best way to perceive the Real is through simulation.
I am going to argue the side of the Artist in this essay. My argument is that the quest for realism in games is a mistaken pursuit of simulation over art. It is a mistake because true meaning resides in the art, not merely the simulation of some event. And games are at their best when they communicate meaning.
The Impertinence of Detail
Why think that art, rather than simulation, is the best way to encounter the Real? To answer that question, it is actually better to start with the study of oratory.
Richard Weaver was a 20th century rhetorician. Today he is most often remembered for his seminal book, Ideas Have Consequences, a key text in conservative thought. But his main scholarly contribution came in the field of rhetoric.
Weaver was a Platonist. He believed that language had its own existence. The goal of rhetoric is to use the power of language to create a desire for the Good in the audience. Thus, the art of rhetoric was about learning how to work within the confines of language to create this “appetition for the Good.”
His project of Platonist rhetorical theory led Weaver to ask why great oratory had disappeared. We live in a world of TED Talks and long, meandering political speeches. But it was not always so. Once upon a time, men like Martin Luther King Jr., Abraham Lincoln, and Cicero walked the Earth. Their oratory was epic in its power— “spacious,” to use Weaver’s term. Why are there not more of these guys?
Weaver thought that one factor contributing to the decline of great rhetoric was an obsession with detail. Details, he argued, can actually detract from the main point.
If one sees an object from too close, one sees only its irregularities and protuberances. To see an object rightly or to see it as a whole, one has to have a proportioned distance from it. Then the parts fall into a meaningful pattern, the dominant effect emerges, and one sees it “as it really is.”1
Great oratory disappeared, in other words, because audiences got nit-picky. They started demanding proofs of every little claim instead of giving the speaker room to tell a grand story. And when the speaker is too caught up in the detail of statistics, the epic meaning disappears from view.
The great speech requires what Weaver called “aesthetic distance.” The audience stands far enough away from it for the whole epic speech to hit them. They are not so hung up on one aspect that they miss the forest for the trees. The pieces of the speech only come together into a spacious whole if the audience is distant enough to see that whole.
But when the audience cares more about soundbites than thematics, the culture of great oratory dies. The audience is too obsessed with the parts to experience the whole.
Detail, then, is “impertinent” when it distracts from the unity of a great speech.2 And that’s one reason great oratory has disappeared.
The Distortions of Photorealism
I believe something very similar has happened to video games. As technology has advanced, the gaming community has become overly concerned with the details of games, rather than their aesthetic unity. The result has been more games that sacrifice interesting ideas on the altar of visual fidelity.
Game developers strive to make games at the highest graphical fidelity because they believe that more detail makes a better game. They assume this because they implicitly accept the Scientist’s ideal: That reality is fundamentally something you perceive with your physical eyes, not your mind’s eye. But as Weaver pointed out, that is a dubious assumption. Meaning is perceived through the mind’s eyes—the imagination.
Ultimately, people care about what the details mean, not the details themselves. And as games get obsessed with detail, they tend to forget this important point. That is how you get a game that looks photorealistic but is utterly uninteresting from a visual perspective.
Grace Ashcroft from Resident Evil 9: Requiem is a great case study in how detail can actually distract from the conceptual Reality in a game. Consider her appearance as Capcom designed her:
Look at her frayed, straw-like hair. See the subtle bags under her eyes. These stand out against her smooth skin and round face.
More detail than these notable features are impertinent to the character’s design. That’s because her round face, smooth skin, dark eyes, and messy hair communicate the truth of Grace. She is a terrified, innocent young woman in a nightmarish situation. And that situation is taking its toll on her soul. Everything works together to communicate the idea of Grace Ashcroft.
Now, take a look at DLSS 5’s reimagining of her:
Notice how the higher resolution adds details.3 Her hair has more dark streaks, her face is more angular, and her lips are fuller than Capcom’s design. You can also see the wrinkles in her hand and the ligaments beneath her skin.
Details like these have to exist at a higher resolution, for higher resolutions convey more information. However, the addition of these details does not at all improve the design.
This version of Grace seems less innocent. Her hair is darker, her face is less round, and her lips are pulled almost into a vulpine smirk. This is not a woman out of her depth. This woman is more like Jill Valentine or Claire Redfield, someone experienced with difficult situations.
The expert artists at Capcom would probably have rejected this design for the character. It does not convey the truth of who Grace Ashcroft is: A traumatized young woman barely surviving a harsh world. The bleach-blond, thin-lipped, round-faced version does—even if she has less detail.
The Truth in Imagination
What this exercise shows is that the real Grace Ashcroft does not exist in the player’s field of vision. She is only visible in the imagination. The details on screen are only helpful as guides to the player’s act of imagination, which pulls them together into the character of Grace Ashcroft.
Gaming, like all artistic endeavors, requires an act of the imagination. The true meaning is not passively received; it is actively deduced. Everything a player sees, hears, or interacts with merely serves to inspire this act of imagination, which unifies the elements into a coherent whole.
Pursuing visual fidelity for its own sake robs the player’s imagination. It makes the imaginative act harder by adding extra details that distract, rather than empower, the player. Consequently, the player is barred from the meaning present in the game’s design.
The AAA gaming space has fallen for this trap over and over again. Game development companies have tended to treat simulation as the goal. They keep building games that requires as little imagination as possible—big, shiny, and bland. These games lack deep meaning because they are too caught up in the details.
Meanwhile, indie games and AA games go from strength to strength because they cannot afford too many details. They cannot show the pores on character’s faces, so they have to make the face look interesting instead. They are forced to rely on the player’s imaginative abilities.
And then there are the rare few who truly understand that art is the goal, not simulation. The developer that comes to mind is Fromsoftware.
The genius of Miyazaki is that he understands gaming is an act of imagination. The truth of a game is apprehended through effort, not passively received. Therefore, he deliberately creates games that are ambiguous and obscure. His worlds have stories; the designs have meaning. However, he will not just give them to the player. They must actively play with the world to understand it.
Games like Elden Ring are so good partly because they have the resources of AAA games while retaining the philosophy of indie games: That truth resides in the imagination, not in the senses. Games should be art—not simulations.
Conclusion
For games, pixels must be deterministic, delivered in real time and tightly grounded in the game developer’s 3D world and artistic intent.
— Nvidia Press Release
It is unlikely the world of game development will ever truly internalize Richard Weaver’s insight. The quest for graphical fidelity is too deeply built into the industry. Even if artists grasp Weaver’s point, tech executives will not. It is all too easy to equate more detail with more value. And value sells.
For this reason, the technical side of game development and the artistic side will always exist in tension with each other. The technical side is dominated by Scientists who think in terms of simulation. The Artist thinks in terms of imagination.
This tension is embodied in Nvidia’s press release. Jensen Huang wants to reconcile the imagination with simulation. He wants to find a balance between AI technology and “the game developer’s 3D world and artistic intent.”
But the very decision to AI-alter Grace shows where this technology’s heart really lies. DLSS 5 is currently a tool of simulation, not imagination. It is a device prone to generating impertinent details that detract from the meaning of characters like Grace Ashcroft.
It need not be this way. I encourage artists working in the game development world to appropriate this tool to empower new designs, not just add details to old ones. Use it to capture the essence of characters and worlds previously hidden to us. Create art to stir the imagination, not mere simulations of observable reality.
As Weaver taught us, Reality exists in the whole, not just the collection of parts. And if we are to reach that whole, we will need our imagination as well as our senses. Hopefully, one day the industry will learn that lesson.
Richard Weaver, The Ethics of Rhetoric, p. 175.
Weaver, The Ethics of Rhetoric, p. 175.
By “resolution,” here I just mean more visual information.