Deep Breath: Okay, Let’s Talk About That Controversial DLSS 5 Demo

from the here-comes-the-comments dept

The polarization over any and all uses of artificial intelligence and machine learning continues. And, to be clear, I very much understand why this is all so controversial. Any new technology that has the chance to be transformative will also necessarily be disruptive and that causes fear. Fear that is not entirely unfounded, no matter your other opinions on the matter. If that’s you, cool, I get it.

I’ll start this off by pointing to the latest edition of the Techdirt podcast in which both Mike and Karl engaged in a fantastic discussion about the use of AI. I’ve listened to it twice now; it’s that good. And, while I found myself arguing out loud with the both of them at certain points during the podcast, despite the fact that neither of them could hear my retorts, it presents a grounded, often nuanced conversation, which we need much more of in this space.

And now, in what might be a subconscious attempt by this writer to commit suicide by comments section, let’s talk about that controversial demo of NVIDIA’s forthcoming DLSS 5 technology. What DLSS 5 does compared with previous versions of the technology is indeed new, but what is not new is the introduction of AI and machine learning into the equation. DLSS 2 and 3 had that already, in the form of pixel reconstruction and frame generation. DLSS 5, however, introduced what is being labeled as “neural rendering”, which uses machine learning to alter the lighting and detailed appearances in environments and, most importantly, character rendering over the engine on top of the 2D image output. Here’s the video demo that got everyone talking.

The backlash to the video was wide, immediate, and furious. There was a great deal of talk about the alteration of artistic intent, about whether this changed what the original developers were attempting to portray when they created the games, and, of course, industry jobs. I want to talk about the major complaint pillars seen across many outlets below, but this backlash also supposedly came with death threats foisted upon NVIDIA employees. I would very much hope we could all at least agree that any threats of that nature are completely inappropriate and absurd.

With that, here is what I’ve seen in the backlash and what I’d want to say about it.

Get your damned AI out of my games!

Perhaps not the most common pushback I saw in all of this, but a very common one. And a silly one, too. As I mentioned above, DLSS versions already used some version of AI and machine learning. That isn’t new. How it’s applied is certainly new, but that isn’t the same as the demand to keep AI entirely out of the video game industry.

And if that’s where you are, go ahead and shake your fist at the clouds in the sky. AI is a tool and, as I’ve now said repeatedly, the conversation we should be having is how it’s used in gaming, not if it’s used. That’s because its use is largely a foregone conclusion and it is an open question as to whether its use will be a net benefit or negative overall to the industry. Dogmatic purists on AI have a stance that is understandable, but also untenable. We’re too far down this road to turn around and go home. And if the tech were able to lower the barriers of entry to the gaming industry, acting as the fertilizer that allows a thousand indie studios to sprout roots, would that really be so bad for the gaming ecosystem?

I can appreciate the purists’ point of view. I really can. I just don’t see where they have a place in the conversation when it comes to gaming.

It overrides artistic intent!

Does it? If it did, then hell yes that’s bad. But if it doesn’t, then this concern goes away entirely.

DLSS 5 is built with options and customizable sliders for game developers. That’s really, really important here. At the macro level, a developer that has decided to use DLSS 5, or decided and customized how it’s used in their games, is exercising consent over their products. That should be obvious.

But then we get into really interesting questions of art, the actual artist, and the ownership of that art, because those last two are very different things. As Digital Foundry outlines:

It may even raise consent and other questions surrounding artistic integrity. On site and witnessing the demos in motion, concerns about this seemed less of a problem when the games we saw had been signed off by the studios that made them – the contentious assets we’ve seen, likewise. Nothing from the DLSS 5 reveal released by Nvidia has not been approved by the studios that own those games. But perhaps the issue isn’t just about specific approvals by specific developers on agreed DLSS 5 integrations, but rather the whole concept of a GPU reinterpreting game visuals according to a neural model that has its own ideas about what photo-realism should look like.

While we’ve seen endorsements from Bethesda’s Todd Howard and Capcom’s Jun Takeuchi, to what extent does that consent apply to the entire development team and other artists associated with the production? And by extension, there is also the question of whether now is the right time to launch DLSS 5 at a time when the games industry is under enormous pressure, jobs are on the line and cost-cutting is a major focus in the triple-A space. The technology itself cannot function without the work of game creators – it needs final game imagery to work at all – but the extent to which it could be viewed as a worrying sign of “things to come” cannot be overstated bearing in mind the reactions elsewhere to generative AI.

That strikes me as a valid and interesting ethical question when it comes to the use of this technology, but one that is probably overwrought. Individual artists who work on video games already have their artistic output live at the pleasure of the game developers they contract with. Those developers already can use this game art in all kinds of ways that the individual artist may not have had in mind when creating it, or indeed have even considered such possibilities. DLSS 5 is just one more version of that, with the main difference being that it involves AI making changes to game images. That’s an important thing to consider, sure, but there are cousins to this ethical question that we’ve all come to accept already. This strikes me more as part of the “all AI is bad all the time” crowd finding a foothold in something other than dogma to grab onto.

Developers and publishers own their games. If they want to use DLSS 5 in those games, there is little other than specific work for hire or other contractual stipulations with individual artists that would keep them from implementing it. If artists don’t like that, I completely understand that point of view, but that’s what contract negotiations and language are for.

Bottom line: I have been as vocal as anyone arguing that video games are a form of art for well over a decade now and I struggle to agree that an optional technology that has approved buy in from game developers and publishers equates to “overriding artistic intent”, writ large.

The faces in these examples look like shit, are “yassified”, or suffer from the uncanny valley effect!

Look, here we’re going to get into matters of opinion. I have to say that when I viewed the demo video myself, I had the opposite reaction. And, yes, this opens me up to claims that I am somehow a massive fan of AI-created pornography (this is where the yassified comments come in), or that I just want all the characters to look “hot” (I’m too old for that shit), or that my older age of 44 means I’ve lost touch with what video games should look like. Despite my genuine respect for the dissenting opinions here, allow me to say this: bullshit.

The caveat to all of this is that the demo revealed very little in the way of this technology working within these games in motion. It’s also certainly true that NVIDIA chose the best potential images to show off its new technology. If the DLSS 5 rendering sucks out loud in a larger in-motion game, or if the images it creates end up being inconsistent throughout gameplay, or if it does just end up looking shitty, then I’ll be right there with you with a torch and pitchfork in hand.

And here’s the other thing to consider with this particular complaint, combined with the previous one about artistic intent: do any of you use visual mods in your games? I do. A ton of them. For a variety of reasons. I have used them to alter the faces and models for games like Starfield and Skyrim, among many others. Do I need to feel bad for altering the artist’s intent? Do I need to apologize for incorporating mods to make characters and environments appear in a way that helps me better connect with the game I’m playing?

Because I’m not going to do either. And I don’t expect you to. Nor do I expect game developers that choose to use this optional technology to beg for forgiveness for their own output.

The hardware demands to run all of this are insane!

Fine, then you’ll get what you want and nobody will be able to use this technology anyway. But I don’t think that will be the case. NVIDIA knows what it will take to run this tech once it leaves the demo stage and goes into production. The idea that they would hype up technology that nobody can use strikes me as unlikely in the extreme.

Conclusion: everyone take a breath

This still strikes me as more of a “all AI is bad” crowd grasping at lots of other things to buttress their pushback than anything else. AI has plenty, plenty of potential pitfalls. Worried about jobs in the gaming industry and elsewhere? Me too! But if you’re not also looking at the potential upsides for the industry, then you’re engaging in dogma, not conversation.

Will DLSS 5 be good? I have no idea and neither do you. Will DLSS 5 alter previously released games in a way that fundamentally alters how we play these games? I have no idea and neither do you. Will it negatively impact the gaming industry when it comes to the number of jobs within it? I have no idea and neither do you.

This was a tech demo. Details on how it works are still trickling out. Most recently, there has been some clarification as to the 2D rendering nature of the technology and what that means for the output on the screen. As an early demo of the technology, feedback is going to be important, so long as it’s informed and reasonable feedback.

The technology may end up being trash and hated for reasons other than “all AI is bad all the time.” If that ends up being the case, I trust the gaming market to work that out for itself. But a lot of the hand-wringing here looks to me to be speculative at best.

Filed Under: ai, developers, dlss 5, rendering, video games

Companies: nvidia