Idea: Using Nvidia Flex Gameworks for Flesh Physics Simulation

[Diluted]

Nvidia Flex Gameworks is a GPU accelerated physics system that is available on most Nvidia graphics cards.  Few games have really made much use of this tech beyond using it as eye candy.  As is gratuitously demonstrated in the video below it specializes in fluid physics.  Although it can cause glitches in my experience, you can enable Nvidia Flex in Fallout 4 to handle extra effects like shells being ejected from guns.

 

But I think guns and gore are just scraping the surface of what Nvidia Flex could add to the game with a little creativity.

 

 

I could design the hell out of the concept I have but frustratingly I'm unable to code it.  What you see below is an example of how a game designer could collaborate with the gamedev team on exactly how they want the element to behave - an adjustable system of sliders in the demo.

 

In this case, I demonstrate how you can transform a tank full of simulated water into a gelatinous semisolid state using a combination of settings.  In this case, what I'm trying to replicate is Ballistics Gel.  Frustratingly if left to my own devices this will always just be a concept; I'm hoping someone in the modding community has experience with Flex Gameworks and may be able to put this idea into action.  

 

 

In the above video, you can see how the simulated fluid is capable of expanding and contracting, becoming more dense and staying in a solid form in response to falling with adjustments.  I'm trying to make it behave like the real ballistics gel you can see below:

 

 

Ballistics gel is used in forensic tests because it highly resembles the physical properties of human flesh, and can be used to accurately predict how bullets behave when they hit it.  What if you could make a mod in Fallout 4 that used Flex Gameworks to add properties of ballistics bel to your player model? 

 

It's a shooting game after all, and there are already mods available for FO4 that can simulate the ballistic paths of bullets.  Until now when we shoot a weapon at something, it will still do a predetermined amount of damage when it hits.  If you could use a simulation of ballistics gel instead to simulate a bullet's impact, it could add a whole new element to the game.

 

In all honesty the simulated ballistics gel on its own would just be a part of it; a skeleton and critical damage areas could also be used.  Existing ballistics dummies like this are one of my inspirations:  

 

 

This scene from Sniper Elite 4 is a great example of how I'd want the concept to feel if I could develop it myself.  Like this would be the hidden mechanic with every shot:

 

 

While this might seem like overkill when playing on a conventional display, if you take a medium like VR when your character and targets are simulated at life-size a new approach makes more sense.  The Reality part of Virtual Reality is something I feel is being neglected and I'd like to see that change.

 

https://developer.nvidia.com/flex

 

 

I'm impressed with what it's possible to accomplish through my experience of modding using existing mesh based physics solutions, but it has its limitations. 

 

To my mind, there comes a point where it becomes more complicated to try to create an empty skin that behaves like it's full of flesh; than it may be to simulate what's actually inside the skin using different tech.  Breast and body physics could be approached in a new way too.

 

I also do a lot of screen archery and one thing I've noticed is that while most poses look fine, there are some that look unnatural because of how the model's arms for instance are arranged.  It's also possible to simulate muscle movement and position using Flex, so this could add a bit of authenticity to posing.

 

This is all something I'd like to see someday, maybe more:

 

Gore

Ballistic simulation

Jiggle physics?

 

I can only really give a design illustration of how this could work but maybe there are people out there who have knowledge of mod development and Flex?

 

 

 

 

[Diluted]

Did I mention jiggle physics?

 

 

I personally experimented with Flex Gameworks fluid simulation on a GTX 980 graphics card so as a potential framework for mods in the future it could reach a good audience of Nvidia users.

 

Of course I was already thinking in this direction when I googled for some recent physics demonstrations, but what did I find but this beauty?  To me the red one kind of resembles the properties of potentially a breast implant, although bouncier.  And it would probably be fairly tunable to make it resemble silicon (or natural breast tissue) more than this if you had that aim.

 

You could have a system that simulates the elasticity of skin as an outer surface, and have individually modeled layers of fat; realistic muscles; and a skeleton (similar to the XPMS system) to support it.  When I say "skin" hopefully I mean the outer texture of a character whose assets exist in the game, just with these properties added to it.

 

Whether you're playing a game or (moreso perhaps) taking screenshots, you want character models that look as natural as possible, in as many situations and poses as possible.  While I love the current gen of Bodyslide, 3BA physics and using SMP physics for hair and clothing, they have their limitations.  Flex offers a potential futureproof structure for handling many of these in one programming solution.

 

If you're taking screenshots of a model in some poses currently, or especially when watching them attempt to dance; their arms sometimes look like this:

 

 

It's not so much that the poses of the skeletons of the mods are wrong; more that the current character models we're using for posing have limbs have no muscle system to change in response to these positions.  Nvidia Flex Gameworks could help this.  

 

Given that it would make characters look just as good whether you're on active duty or having some in-game "R & R" with your love interest, the uses for this tech could obviously also extend to more "adult" purposes but I'll stay on-topic...

 

 

[Diluted]

I just want to keep contributing to this thread because I could get hit by a bus tomorrow or something and it would be kind of a shame not to relay this stuff IMO

 

(I'm pretty sure an approximation of this stuff would be possible on AMD graphics cards, but this exact design concept is focused more on Nvidia's tech)

 

Why does it make sense to use Nvidia Flex Gameworks?

 

Texture based physics create the illusion that a character's skin texture contains soft flesh, and use a complicated system to handle how points in the texture react during collisions.

 

In all honesty I believe using Flex Gameworks would be a better way to go because it offers a unified particle representation system, which means the way simulated tissue would act could be coded into the flesh itself. 

 

 

This is a copy of my original video, but this time I'll explain the process I imagine you could use in a kind of character creator studio that could be based off this SDK.  From a design perspective, the main thing that's missing is a mould for the final substance to fill so it can take shape.  Similar in concept to a jelly mould.  I can imagine a studio process where you generated a "mould" file from whatever Bodyslide preset you're using, fill the mould with the simulated liquid in its default liquid state; and then adjusted the sliders in the software so you could perform the next step of transforming the simulated liquid into the semisolid fleshlike state using sliders exactly like I used to generate this big chunk of the stuff.

 

Instead of just using a texture on the outside though, I believe it would be possible to take the texture data for the character and "paint" it onto an outer layer generated for the model that could simulate the elasticity of skin (I won't post the jiggle physics video again but scroll up to see how the Flex Gameworks system could possibly handle this).  A fluid physics simulation based on this could make breasts for instance still react realistically at any size and have customizable volume and bounce, and offer customizability if you wanted breasts to behave more like silicon for example.

 

 

Staying on the concept of moulds, if you could simulate the properties of breasts you could simulate the physical form and physics properties of a Fleshlight like this in theory and use it to handle collision physics in SexLabs or Ostim.  

 

I don't know how to program so I'll never be able to create a "character studio" mod like this myself, but breaking the concept down into elements it doesn't seem impossible.  It's the kind of thing I'd like to see one day in my screenshots anyway.  

 

 This is Nvidia's page for the Gameworks tech:

 

https://developer.nvidia.com/flex

[Pamatronic]

2 hours ago, Diluted said:

I don't know how to program

Which perfectly explains why this entire thread is just dumb.

[xcrunner15]

All interesting ideas but probably not practical and definitely not worth the enormous effort of bringing into an 8 year old game that already has some form of flesh sim

[sen4mi]

22 hours ago, xcrunner15 said:

All interesting ideas but probably not practical and definitely not worth the enormous effort of bringing into an 8 year old game that already has some form of flesh sim

I disagree.

 

The ideas are pretty, but the interface is insufficiently documented to really be "interesting". OpenGL (various versions), DirectX (various versions) or Vulkan are most likely what any game is going to be using, and most of would need a lot of documentation (of seemingly "obvious" issues) and examples before we could do anything with this stuff.

 

The age of the game (along with the documentation on other games deployed with variations on the engine) increase the likelihood that some of us could pull together enough information to figure out how to wire things together.

 

And, unless there's too much squabbling between implementors, this could be made available in an update to some existing tool (like maybe aaf or if it ever gets enough work put into it naf or maybe even a revived fourplay).

 

I totally understand that most of us lack sufficient clues to make headway. But every bit helps.

[xcrunner15]

On 2/23/2024 at 7:16 AM, sen4mi said:

I disagree.

 

The ideas are pretty, but the interface is insufficiently documented to really be "interesting".

 

Ok I was being nice so the whole comment wouldnt just be negative. should I have said the ideas suck

 

Quote

(like maybe aaf or if it ever gets enough work put into it naf or maybe even a revived fourplay).

 

if the authors could play nice and NAF wasn't abandoned on this forum and by modders that would be cool. Doesn't seem like it will happen though. We went through the squabbling phase and are now in the ignoring each other phase which may seem like a good compromise to some but is wholly unproductive.

 

Quote

The age of the game (along with the documentation on other games deployed with variations on the engine) increase the likelihood that some of us could pull together enough information to figure out how to wire things together.

 

I mean this a game that has a VR port but doesnt even have VR support with any animation frameworks. Why don't we start there and then think about bringing in other code libraries. Not sure if a project last updated in 2017 is really gonna be the cutting edge library for performant physics though.

 

 

 

[sen4mi]

2 hours ago, xcrunner15 said:

I mean this a game that has a VR port but doesnt even have VR support with any animation frameworks. Why don't we start there and then think about bringing in other code libraries. Not sure if a project last updated in 2017 is really gonna be the cutting edge library for performant physics though.

 

This reflects the technical abilities of the bulk of the people involved (though it also reflects something about some other economic and VR issues). Old games have advantages in terms of fan base support (because everything worth doing takes time). Older games tend to be a bit difficult for getting paid support (people have high expectations for paid support, in many various ways - flaming and trolling are both popular activities, but so is getting paid enough to live and there's always new people involved - and that... those problems accumulate crud).

 

A big part of the problem, from my perspective, is that documentation efforts these days don't get much traction, and we are mostly relying on tools and systems developed back when we had much better documentation support. (And, granted, simpler game formats to document.) But, nowadays, most of us have stronger tendencies to give up early.

 

There's lots of reasons for this, of course. Everybody has "good reasons". And, like you've been hinting in what you wrote: those good reasons tend to not be helpful.

 

But "it sucks" isn't even meaningful in modding. Everything sucks in some way or another, and if we let that stop us from improving things, we would do nothing.

[Diluted]

Thanks for the contributions and reads on my idea. 

 

I already admitted and have had it pointed out to me that my primary flaw is that I myself haven't yet learned how to execute the code in-game; I'm limited to experimenting in Nvidia's Flex Gameworks SDK environment instead of being able to say this idea is truly viable.  I respect coding and modding as an art form in themselves; think of this thread as trying to bring fresh canvas and paint to create something new with.

 

I've been gaming since the Atari 2600 days and have seen trends developing, and there has never been a right time to even post my idea on any forum until now.  It's actually thanks to the modding community in the first place that I started this thread because it gave me a relatable physics foundation in the form of SMP.  This is more of an idea of where I see the trend going:

 

First if it's possible to use Flex Gameworks for fluid physics in Fallout 4 it will be relatively easy to hook in and at least set up some basic experiments.  You won't see any performance hit; it just represents a new modality instead of the standard solid-body collisions supported in the base game.  So the FleX framework is tested and I know that at least is stable and hopefully usable.

 

 

Part of what inspired this idea is the concept of generative design - taking as precise a human input as possible to describe what you want to create, and then having Artificial Intelligence working with that data and generating concepts.  The way current-gen Nvidia GFX cards use AI to enable Ray-Tracing & other features demonstrates they can do this in an existing arena.

 

  

 

How I believe AI could be helpful in a design concept could be taking existing SMP meshes and converting them into a format friendly for current and next-gen games, using a generative design concept.  Check out the image below:

 

 

What you see is the illusion of curves that have been made up of straight lines.  The curves could be made rounder than this image by adding more lines, but it would take more effort to create.  It's became evident fairly early on that curves are being formed.  

 

 

All I've done in the above image has been take the existing curve made by many lines and made a near-perfect circle in black using a fill tool in a photo editor.  This represents what the AI on most Nvidia GFX cards could potentially do - take a complex form made of curves and generate a simple rounded form inside it.  When I say "simple" I mean simple in that it has taken a complex shape that tries to be curved, but will always be made of flat surfaces; and interpreted it into what the programmer was trying to express.  Generative AI.

 

In this case all the AI has to do is fill in the complex curved form made by the external straight lines and it's created a round inner mould.

 

 

I'm using my Skyrim character as an example but in the above example you can see how I've got my character set up to use a high-poly head, with the mesh file and end result.  The face texture file alone isn't enough because in the vanilla game you'll end up with a deformed result unless you have a custom mesh file to fit the texture, and specifically enable the high-poly face part of the correct character race.  Not following any one of these steps properly spoils the result.

 

What this could mean for end users in a potential Gameworks Physics Studio system assisted by AI is this procedure could hopefully be possible using the concept of simplification demonstrated by the circle diagram.  The job of setting up custom face parts could be made easier because the software could potentially take the raw data of the original mesh file made of polygons, and create a rounded inner form to fill it.  Then the texture itself could possibly be reformatted and turned into a rounded outer skin to match the mould.

 

Getting high-poly heads to reliably work in Skyrim from my experience is a chore sometimes because it's not enough to just have the high-poly face: without the right mesh files the textures are useless.  If end users had a tool that could simplify the procedure by generating a custom inner mesh form for the texture it could make the experience better.

 

 

Specifically what leads me to believe now is a good time to look into Flex Gameworks tech, is that the latest version of SMP has the experimental option to use CUDA architecture on current-gen Nvidia graphics cards.  The problem is that even at the highest physics quality settings, some of the physics look unnatural and would continue to look unnatural under the current SMP model no matter how complex the simulation.  

 

With existing SMP physics all it takes is for a character to cross her arms wrong and her breasts get the "physics jitters".  It would be very complex to anticipate every possible collision using texture data alone because current-gen models are hollow.  It's a question of a complex job using a more simple tool, or making a more sophisticated tool to make that task easier.  Instead of trying to anticipate & code for every possible collision situation, physical collision & gravity properties could be build into the internal structure of the simulated breast.

 

If it's possible to take an ageing physics model in the form of SMP and get it to run with CUDA architecture in-game, it may be possible to use the more modern Gameworks system for a more organic result.  So the missing link to this system is there - SMP is getting old as a physics system, but looking for a new approach using features on the same CUDA hardware this concept would use.  In all honesty there's not much to improve using the existing SMP system but breast physics; the rest of the body isn't as affected by gravity.  

 

While new, this could be a more organic approach.  Fallout 4 is in the "Goldilocks Zone" for this idea given that it's had Flex Physx support since launch, and an established modding community.  As I've been quoted as saying, I don't code.  Another saying is "Those who can't do, teach" so I guess that's what I'm trying to do.  It's a topic I've been wanting to discuss for a while.  It's unlikely Bethesda themselves will ever develop this tech; bare flesh much less the physics of flesh is something ignored in their vanilla games. 

 

I know this thread has been kind of like proposing a chicken dinner, then holding out an egg and saying "Maybe one day..." but things have to start somewhere.  I just want to see if this idea can incubate