K1 Endgame Cutscene Improvements

[DarthParametric 3,782]

Testing out the Dodonna model sidetracked me to a new tangent - fixing those god-awful stunt models in the endgame cutscenes. First up is the Dark Side version, replacing the droids:

 

     

 

     

Interestingly, my framerate in this scene is pretty terrible even in the vanilla version. I have a GTX 1080Ti so I'm not exactly hurting for raw GPU horsepower, but it drops to sub-40 fps in certain places. The replacement version of the droids can see it dip below 30 with shadows enabled (it's about 5fps better with their shadows disabled).

Next up is the cardboard cutout people. Bioware didn't use a flipbook (animated texture) for those. Instead they have a bunch of planes with static textures in slightly different poses, and they animate the alpha values of the meshes to blend between them. It seems an odd choice, but perhaps that was a more efficient use of memory or GPU for the Xbox. The DS versions are particularly terrible, as they just sort of wobble side-to-side. At least in the LS version they are clapping, which looks a lot better.

[JCarter426 1,214]

@Jenko did this before and found that once everything was replaced with actual models the frame rate drops into the totally unacceptable range. We tested various scenarios with different things changed and it seemed to be specifically the poly increase. My guess is it's when they're animated. The area geometry can have tons more polys, and those droids might be ok because they're just standing there. But when the humanoids were upgraded from rectangles to actual models, even an idle pose was too much for the game to handle.

Maybe you could make higher quality cardboard cutouts, though.

[DarthParametric 3,782]

It seems like it runs pretty terribly even in its vanilla state though, as I said above. I wonder what it is about it. It shouldn't be too taxing, theoretically. Most of the level geometry is a single room model running about 40K polys, but there's nothing particularly outrageous. The heaviest individual meshes are the Rakata statue parts, weighing in around 800 polys each. 

Edit: So I did a test to see what happens in the best case scenario. Editing the room model with all the references that spawn in the stunt models and deleting them all, I still get a 5fps drop in the final shot as it pans over the now non-existent crowd.

Swapping out about half of the sprites for static meshes with shadows disabled gave around a 25-30fps drop.

I think you could probably cull a bunch of those anyway. There's something like 140 sprites, but you can't even see a good chunk of them because of the camera angles. Sprites would be fine up the back where you can't really tell the difference anyway. Then closer to the front could be a thinner crowd of actual meshes. The droids could probably be thinned out by 50%. They are packed in pretty tight.

[DarthParametric 3,782]

The positioning needs to be adjusted to better fit the camera angles, and the sprites at the back are probably a bit too dense, but this arrangement seems to give a pretty acceptable level of performance. Dropped maybe ~10fps in the final shot, which is consistently the worst performing section.

I used lite models for the Sith officers. I could possibly play around more with that, see if I can bulk out the numbers whilst camouflaging the low poly stuff as much as possible. It's also lacking any sort of Sith/Dark Jedi, so some of those need to be sprinkled in.

 

[Salk 374]

Fascinating.

I didn't remember noticing any noticeable lag when I played DS a couple of years ago. Using a much less powerful GPU than you have, DarthParametric.

[DarthParametric 3,782]

Being so ancient, the game is single threaded. As I am running a Ryzen, the game is likely entirely CPU bound.

[Salk 374]

Have you tried ImageCFG to force the usage of a single core?

[JCarter426 1,214]

9 hours ago, DarthParametric said:

So I did a test to see what happens in the best case scenario. Editing the room model with all the references that spawn in the stunt models and deleting them all, I still get a 5fps drop in the final shot as it pans over the now non-existent crowd.

[...]

Swapping out about half of the sprites for static meshes with shadows disabled gave around a 25-30fps drop.

Sounds like you're having better performance than we got before, maybe thanks to the lite models or the static poses. With full quality (well, KOTOR quality) animated models for everything it was dipping down to the single digits on that pan. But it's dropping even with nobody there? There must be something weird going on in that area. It's always been a mystery to me why some areas perform worse than others. As you said, this is a tiny area and I remember the stunt module lagging more than the regular game one, so that's doubly mysterious. @VarsityPuppet recently suggested it might have to do with how many elements the models are broken up into, but I don't know if anyone has ever properly investigated what's really going on.

[VarsityPuppet 1,085]

Could also be the bumpmaps. How many textures have bumpmaps in that module? Looks like most of the building does and there's also all of that water

[Squall Lionhart 81]

Running a Ryzen CPU, and I notice sometimes on both games it will give me a bad fps if I'm filming on the same drive I'm playing on. Other than that I've never had any issues with the Dark Side ending. My question DP is after the Dark Side ending, are you going to do the Light Side ending as well. I ask since I notice some of the "cutouts" look to be clapping and use Bastila's style of robe (90SK did a version of it if I remember correctly)

[DarthParametric 3,782]

1 hour ago, Squall Lionhart said:

are you going to do the Light Side ending as well.

Yeah, the LS version is on the agenda. At the very least I want to swap out the "Council" for the actual Dantooine Council. The weird thing though is loading that module into Max, all the references for the NPCs are way up in the air and I seem to be lacking the ramp where they all stand on.  Not sure what the deal is there. I guess I need to check I used the right layout.

5 hours ago, VarsityPuppet said:

How many textures have bumpmaps in that module? Looks like most of the building does and there's also all of that water

Not sure, will have to check. I also wondered about the water. I'll have to try a version with the water plane deleted.

6 hours ago, JCarter426 said:

@VarsityPuppet recently suggested it might have to do with how many elements the models are broken up into

As I referred to in a previous post, the main chunk of geometry is broken up into lots of tiny pieces. I can only assume this was done deliberately for performance reasons on the Xbox, because it would be an absolute bastard to try building the level that way.

6 hours ago, JCarter426 said:

But it's dropping even with nobody there?

Not exactly nobody. In addition to the player and Bastila, there are also the four Sith officers. In the latest version I removed the two at the back, as they are more or less unnoticeable anyway. Trying that with the empty version, I gained a few fps back in that last shot, so just them standing there with idle animations was having an effect.

[JCarter426 1,214]

44 minutes ago, DarthParametric said:

Not exactly nobody. In addition to the player and Bastila, there are also the four Sith officers. In the latest version I removed the two at the back, as they are more or less unnoticeable anyway. Trying that with the empty version, I gained a few fps back in that last shot, so just them standing there with idle animations was having an effect. 

Ah. I'm surprised there'd be noticeable lag with only that many of them, but I guess not too surprised.

45 minutes ago, DarthParametric said:

As I referred to in a previous post, the main chunk of geometry is broken up into lots of tiny pieces. I can only assume this was done deliberately for performance reasons on the Xbox, because it would be an absolute bastard to try building the level that way.

It might not be just the number of objects on the model - it could also be the number of elements on each object. It looks like a lot of faces, particularly on the pyramid, have been needlessly detached as separate elements. If that is the source of the problem, though, I don't know of a quick and reliable way to fix that.

[DarthParametric 3,782]

9 minutes ago, JCarter426 said:

It looks like a lot of faces, particularly on the pyramid, have been needlessly detached as separate elements. If that is the source of the problem, though, I don't know of a quick and reliable way to fix that.

For any elements that share the same texture and light map, it should be a simple matter to join them back together.

[JCarter426 1,214]

Oh I know it can be done, but I'm not sure how to go about it in a way that's quick or easy. I don't think you can do a quick welding of vertices without potentially messing up other things, so some of it would have to be done manually... a lot of effort just to fix one area, and any other areas would be need to be fixed on a case by case basis.

[DarthParametric 3,782]

I can drop the mesh count from ~330 to ~60 before it starts getting too tedious to bother. But given the way the light maps are split, there's actually not much to be gained by welding. A lot of the rejoined meshes are physically separate chunks. And presumably most of the welded geometry would be split again on export based on UVs anyway. I'll export this version and see what effect, if any, it has on framerate.

Edit: This may require the assistance of @bead-v and @ndix UR, as MDLEdit chokes on my condensed version.

[HK-47 84]

Something I've learned is that sometimes a game can be taxed in FPS simply due to the Engine. The engine itself can only handle so much and will cause FPS drops. It was SWTOR that I found out about that. The current lead... head... person for SWTOR Kieth Kanning mentioned in a live stream that some of the War Zone and Operation FPS issues are due to the engine being taxed.

Since the engine KotOR uses is designed for the original Xbox, I'm not surprised at all that there is FPS issues when there seemingly shouldn't be.

In this instance, I would say the water has something to do with it as well since you still had FPS issues with that hardware and only the 4 officers.

Really, the whole thing shouldn't be surrounded by water anyway. It's always bothered me because it's inaccurate. You're not teleported to a building out in the middle of the ocean nor are you 500+ft in the air. There should be visible land. Maybe some low res texture land to make up half/most of where the water is. Perhaps you could get away with a mostly flat looking/not animated ocean and low res "land" texture. Make it look distant as it should, yet not like it's miles away. Additional details like the crashed republic ships and other features are up to the discretion of the modder.

[JCarter426 1,214]

48 minutes ago, DarthParametric said:

This may require the assistance of @bead-v and @ndix UR, as MDLEdit chokes on my condensed version.

Yeah, the same thing happened to me. I just did a quick select by material to attach everything, but MDLEdit wouldn't load the model. I remember something similar cropping up with a few specific areas during the MDLEdit beta testing so maybe bead-v can explain it.

2 minutes ago, HK-47 said:

Really, the whole thing shouldn't be surrounded by water anyway. It's always bothered me because it's inaccurate. You're not teleported to a building out in the middle of the ocean nor are you 500+ft in the air. There should be visible land. Maybe some low res texture land to make up half/most of where the water is. Perhaps you could get away with a mostly flat looking/not animated ocean and low res "land" texture. Make it look distant as it should, yet not like it's miles away. Additional details like the crashed republic ships and other features are up to the discretion of the modder.

Manaan seems to run fine though? That certainly could be an contributing factor, and it may be worth removing it for the other reasons you mentioned, but I doubt it's just the water. Some things I could maybe write off due things like water or grass or the size of the area or lots of NPCs spawned, but that wouldn't account for everything. I've never been able to nail down a pattern on the areas that have had particularly bad lag, though, and my hardware has changed over the past 15 years too.

[DarthParametric 3,782]

The water appears to be having a negligible effect. I deleted the water plane and at absolute best I maybe gained 1fps, but that could easily be explained away by the margin of error in my pretty sloppy testing.

I also tried changing the CPU affinity to a single core. It didn't really make any noticeable difference when farmed out to an unused core, but I did notice that it maxed out utilisation the entire time it was running. Although that was for an empty scene. I'll need to try it again with a model-heavy scene.

[DarthParametric 3,782]

Ahah! After what seemed like an eternity (must have taken like 15+ mins), MDLOps finally compiled the model. Much to my surprise, not only did the game not crash, but the framerate (for an empty scene) was now locked at 60fps except for one very brief drop of a frame or two in the pan to Revan's face. Very interesting indeed....

It seems like @ndix UR and @bead-v will have to compare notes on their level model compiling.

[HK-47 84]

5 minutes ago, JCarter426 said:

Manaan seems to run fine though? That certainly could be an contributing factor, and it may be worth removing it for the other reasons you mentioned, but I doubt it's just the water. Some things I could maybe write off due things like water or grass or the size of the area or lots of NPCs spawned, but that wouldn't account for everything. I've never been able to nail down a pattern on the areas that have had particularly bad lag, though, and my hardware has changed over the past 15 years too.

When inside the temple, it's nothing but water outside too, and that runs smoothly as well. I agree that it's not likely *the* cause, which DP has pretty much confirmed. I've often wondered if part of it is what's causing the temple texture to shine (I forget what KotOR uses for that(shaders?)) is contributing. But Manaan has a shine to it so that doesn't really make sense.

I had also wondered about the fact that parts of the temple are different pieces (and as revealed the elements on those parts) might be part of the cause. I have always noticed the lines of light in the shadow at the beginning of that segment.

[bead-v 251]

1 hour ago, DarthParametric said:

Edit: This may require the assistance of @bead-v and @ndix UR, as MDLEdit chokes on my condensed version.

Which model? Is it only the condensed version that chokes? May I get the ascii?

[DarthParametric 3,782]

48 minutes ago, bead-v said:

Which model?

The source model is m44ad_01a. MDLEdit can successfully load and compile the vanilla ASCII. It crashes on loading the condensed ASCII. Initially I thought this may have been because one monolithic mesh contained more than half the level's polys (~26K out of 44K), so I split that out in to what I figured was more manageable chunks, topping out at around 4K for the largest. But it still crashed MDLEdit when loading the ASCII.

https://www.darthparametric.com/files/kotor/misc/K1_m44ad_01a.7z

Edit: @bead-v - When trying out variations of my crowd model, I managed to crash MDLEdit again on load by virtue of the condensed version where I combined all meshes sharing a texture into single meshes. I think this must be some upper face or vert limit per mesh that MDLEdit can't deal with.

[DarthParametric 3,782]

So with the performance gains I decided to try getting rid of all the sprites and seeing what happened. This is what I came up with, 180K polys worth of static meshes:

            

Somewhat surprisingly, the performance hit was fairly minimal. Something like 5-7fps. I'm interested to see if that improves with a condensed, monolithic mesh version, assuming I can get MDLOps to compile it.

I'm not a fan of the apprentices/Sith, but they are placeholders. I'm wondering whether I should use some of the TSL models, maybe in conjunction with a separate mod that replaces all the K1 Dark Jedi with those as well.

Edit: OK, god damn, it took MDLOps well over an hour (!) to compile, but I finally managed to test the monolithic mesh version of the crowd. As with the level model, it was a noticeable improvement, once again returning to a solid 60fps. So it seems that the real issue at play here is entity count? Too bad there's no practical way to animate the crowd as a single mesh (I think it would only be possible with PLA - point level animation).

@ndix UR: What is your analysis of this performance issue?

[bead-v 251]

5 hours ago, DarthParametric said:

https://www.darthparametric.com/files/kotor/misc/K1_m44ad_01a.7z

Edit: @bead-v - When trying out variations of my crowd model, I managed to crash MDLEdit again on load by virtue of the condensed version where I combined all meshes sharing a texture into single meshes. I think this must be some upper face or vert limit per mesh that MDLEdit can't deal with.

Thanks! It looks like kotormax exported a bunch of UVs that aren't even used on one of the nodes, bumping the index so much it went into negative because the numeric variable was signed rather than unsigned. Anyway, a quick and simple fix, won't be a problem anymore. The model compiles now, but it takes its time...

[DarthParametric 3,782]

Weird. How did it manage to do that? Due to something Max does? And how did MDLOps manage to deal with it? There were no noticeable issues in-game with the compiled models it produced.