Incorporating Sounds, Facial Expressions into AAF animations

[Gray User]

I've been trying to build sound and facial expressions into the animation packs for some of my projects, but I'm not having much luck. In skyrim, I would have handled everything using scripts and it would have been kind of a mess. Given how awesome AAF is, I'm hoping there's a way to do it without resorting to scripting. So, my questions to you all are:

 

1) Does anyone know how to link sounds to animations that are controlled via AAF? @Leito86's animation pack has sounds (and the still work, even when the animations are controlled via AAF), but even after digging around the esps, I can't figure out how he managed it.

 

2) The documentation for AAF indicates that it is possible to control facial expressions that way. If I'm following correctly, I set up the animation file to point to an action file that points to a reaction file that points to an expression file, like so:

animation.xml -> action.xml -> reaction.xml -> expression.xml

 

I think I understand how make the reaction -> expression link, and the animation -> action link, but the action -> reaction link (using a custom defined stat?) escapes me. Does anyone have (or know of) a working example of this? If so, I can probably work it out from that example.

 

 

[Guest]

I havent dug into question 2 at all yet, but I can tell you that Leito's sounds are a result of notating in 3ds max, telling the animation file to play a sound file at certain keyframes.

[Gray User]

6 hours ago, Flashy (JoeR) said:

I havent dug into question 2 at all yet, but I can tell you that Leito's sounds are a result of notating in 3ds max, telling the animation file to play a sound file at certain keyframes.

Well I'll be damned. I didn't know that was possible, but that would explain why they work even when triggered by AAF. I shall have to look into that as an option. Thanks!

[Guest]

4 minutes ago, Gray User said:

Well I'll be damned. I didn't know that was possible, but that would explain why they work even when triggered by AAF. I shall have to look into that as an option. Thanks!

Head over to the AAF Discord server and ask to be flagged as a mod author. It'll give you access to the mod author channel and you will be able to pick people's minds to get the low down on how to do this.

 

Im sure @dagobakingwill be pleased to circle you in.

[dagobaking]

The sounds you hear in Leito's pack are baked into the animations. You can do this by adding "annotations" within 3DSMax that trigger sounds at specific keyframes. I'm not much of an animator. So, I couldn't detail out the process. There probably are more people that could help with that on discord as Flashy mentions.

 

In addition to that, there is a "topic" system in AAF that hasn't been used yet by anyone. That allows you to make audio with lip-sync files. So, the characters would look like they are saying things. I think, ideally, physical sounds should be baked into the animations with the annotation method. But, verbal sounds should not be and should instead be set up through the topic system. Then people can control voices (gender or individual) and the lip-syncing would be more immersive.

 

One caveat with the topic system is that because nobody has attempted to use it, it hasn't been fully tested yet. I need someone to make a first set of those files so that it can be verified as working all the way through.

 

You are also on the right track regarding the actions > reaction > topic process.

 

Those are also important files that have not been set up by anyone yet. I have internally tested them as working with placeholder versions. But, it's up to the community to make those files for actual gaming.

 

There has been a lot of private discussion about this subject recently. Many concepts in AAF aren't obvious on the surface, even with the wiki. So, I am looking at sort of "branding" groups of files so that they make sense to everyone in simple terms. For example, I am now planning to introduce the concept of having "Gameplay Themes". A Gameplay Theme is a collection of specific XML files for AAF that pre-define actions, overlays, equipment handling, etc. in a way that makes sense for a specific type of playthrough. So, there might be an "Adult GamePlay Theme", "Romance GamePlay Theme", "Vanilla GamePlay Theme", etc. Again, I will not be making these themes. That is for the community to do. But, there are people already working on them. So, hopefully, there will be working examples to learn from soon.

 

I think that these themes will be easier for people to understand and install. And it will give animation packs a larger target of XML references to make instead of needing "This pack for Stats" and "this pack for tags" and "this pack for actions". It will all be part of a GamePlay Theme instead.

[Gray User]

23 hours ago, dagobaking said:

Those are also important files that have not been set up by anyone yet. I have internally tested them as working with placeholder versions. But, it's up to the community to make those files for actual gaming. 

Interesting...You wouldn't be willing to PM me the placeholder example xml files would you? (unless they are just what's in the examples on the wiki, in which case I can just keep copying from there).

 

[dagobaking]

2 hours ago, Gray User said:

Interesting...You wouldn't be willing to PM me the placeholder example xml files would you? (unless they are just what's in the examples on the wiki, in which case I can just keep copying from there).

 

They are the examples given in the wiki. Only difference is that I made the reference ID names generic so the user would understand which parts they need to tailor.