MDLedit bug reporting thread

[DarthParametric 3,782]

OK so I just went back and checked again and it seems that your hex edited vanilla model with added alpha node set the value at 1.0. That doesn't crash. However, if the value is edited to, say, 0.5, then it does crash. That would apparently rule out MDLEdit and put it back to being a fundamental game/modern OS problem.

[bead-v 251]

4 minutes ago, DarthParametric said:

That would apparently rule out MDLEdit

Phew!

So, do we know of any vanilla models with a non-1.0 alpha on skinmeshes?

[DarthParametric 3,782]

I know that the Dodonna and Vandar holograms (c_holododonna and c_holovandar) have animated alpha to fade in and out their holograms during a couple of animations. I switched those to use a base mesh alpha of 0.5 and didn't get any crashes. Not sure on others. Nothing that instantly springs to mind in K1.

Edit: Just (literally) dusted off an old laptop and tested my fully alpha'd up version on XP running the 4CD version and it doesn't crash. So some sort of OpenGL or other modern system issue I guess.

[bead-v 251]

Well, there's one obvious difference between the dodonna holo model and the rakata holo model – all the skin nodes (= all the meshes with alpha 0.5) come before all the trimesh nodes (= all the meshes with alpha 1.0) in the dodonna model, but not the rakata model. That's one thing you could try.

Also, mdledit fails to load dodonna's supermodel binary  will this ever end....

[DarthParametric 3,782]

7 minutes ago, bead-v said:

all the skin nodes (= all the meshes with alpha 0.5) come before all the trimesh nodes (= all the meshes with alpha 1.0) in the dodonna model

In the vanilla model? Where are you seeing that? I don't see that in MDLEdit or the decompiled ASCII. The skins are at the bottom of the hierarchy, same as pretty much any other vanilla model.

[bead-v 251]

4 minutes ago, DarthParametric said:

In the vanilla model? Where are you seeing that? I don't see that in MDLEdit or the decompiled ASCII. The skins are at the bottom of the hierarchy, same as pretty much any other vanilla model.

No, your edited model, the one you released.

[DarthParametric 3,782]

Huh, you may be on to something. I just tried editing the vanilla holo Rakarta model, moving all skins above the trimeshes and adding alpha 0.5. It didn't crash. Bizarre. Even though the vanilla holo Dodonna model has all its skins at the bottom of the hierarchy and uses animated alpha without crashing.

This partially explains an issue I reported in my initial testing, that I was getting crashing with one hierarchy arrangement, but then the crashing stopped when I split the head mesh up further and further swapped around the hierarchy. I wonder if it is a specific mesh.

[bead-v 251]

It appears it's not about the animated values but about the base value. On the vanilla c_holododonna.mdl, all the skins have an alpha value of 1.0, same as all the other meshes, and that appears to be fine, even when they're at the bottom. Now I wonder if the same also applies for meshes, i.e. that the base alpha value needs to always be equal to or larger than the previous one throughout the model.

Hopefully this wasn't just a case where it randomly doesn't crash, though.
 

[DarthParametric 3,782]

Son of a bitch. That works. And that's with the eyeball and lid trimeshes also having alpha. Actually every single node has alpha 0.5 because I was lazy and did a find & replace in the ASCII.

So now the million credit question, why?

[bead-v 251]

Not sure what you mean with the eyeballs, but I'm glad it appears to work!

As for the million credit question, that's gonna have to be someone else.

[DarthParametric 3,782]

23 minutes ago, bead-v said:

Not sure what you mean with the eyeballs

I mean it is not about skins with alpha 0.5 being above trimeshes with alpha 1.0. All the trimeshes also have alpha 0.5 (at first I thought it was just the eyeballs/lids, but it is everything). I think the general rule must be if a skin node has alpha <1 then it must be above all trimeshes, regardless.

23 minutes ago, bead-v said:

As for the million credit question, that's gonna have to be someone else.

Paging @ndix UR.

Edit: Food for thought perhaps. Unlike the vanilla holo Dodonna and Vandar, the vanilla holo Rakata is classified as an Effect, not a Character. Is this of significance? I changed my custom one back to Character.

[LiliArch 115]

For the record, if you need raw-ish comparing done on binary files, I have a software for that. In case you want checking done on differences between vanilla and compiled models. I’m in the process of moving right now though, but that should ease in a couple of days. Just in case.

[JCarter426 1,214]

On 8/24/2018 at 10:13 AM, DarthParametric said:

Food for thought perhaps. Unlike the vanilla holo Dodonna and Vandar, the vanilla holo Rakata is classified as an Effect, not a Character. Is this of significance? I changed my custom one back to Character.

I've only ever seen this make a difference on the NWMax or KOTORMax side of things. You can take a character model and make it an effect or a placeable or whatever and the game doesn't care as long as it's a properly-compiled model (though it may not function properly due to lacking animations and such) so I'd be surprised but interested to hear if that made a difference anywhere.

For example, one of the stranger things on the list of things I've done is replacing one of the Gamorrean models with a plasteel cylinder. And then running around the Refugee Quad with angry containers trying to bludgeon me. All for the sake of testing.

[DarthParametric 3,782]

I will preface this by saying it is not a MDLEdit bug, but seemingly a GPU/driver issue. However, @ndix UR asked me to describe it here so that there is a record for future reference.

Discord user Deltm reported an issue with my recent TOR Republic uniforms mod for TSL. They were having rendering issues with the hologram effect, like so:

         

As with my previous hologram mods, these are both full body models, so the face meshes are part of the same model. It appears that the user's GPU, an nVidia GTX 970, is incapable of rendering the hologram effect on meshes using a normal map/tangentspace 1. The uniform meshes are ported from TOR and use a modified version of the source's diffuse texture and normal map (the rank bars on the captain receive the hologram effect because they use a different texture). This was fixed by setting the tangentspace flag to 0 and swapping the offending diffuse to one that didn't point to a normal map (i.e. TXI edit, or removal actually). I personally didn't experience the issue during testing on a 1080 Ti on Win 10. I haven't seen any other reports of this issue as yet, although the mod was only released recently and the download count is small.

[JCarter426 1,214]

I got another problem that I can fix, but a tool fix would be preferred.

Some backstory first. I've been porting areas since the MDLEdit beta testing, and it worked for the Dantooine courtyard (m14aa) right away. But then, for every attempt after that, there would inevitably be something wrong. I've now nailed down the reason for this.

Here's my latest attempt, with m02ab on Taris:

Spoiler

Several rooms are not visible at all. The visibility and layout files all check out, since they're direct copies of the ones from the game. I even tried rebuilding them with new room names and still had the same problem. And of course I compiled all the models, copied over all the textures and lightmaps. Everything should be working, just like Dantooine did. And yet, several rooms are invisible.

I noticed that it seemed to be limited to rooms that don't have a walkmesh, like the skybox and some of the background buildings. All the walkable areas are rendering, and the others aren't. This was consistent with my earlier broken attempts in other areas, though I didn't make the connection before. So I tried adding a walkmesh to one of the rooms (m02ab_02r) to see what would happen:

Spoiler

And there it is. Now that room renders.

I've noticed before that many rooms have walkmeshes when they shouldn't need to, like the skyboxes on Citadel Station. Yet I went back and checked m14aa and its skybox does not have a walkmesh, and my ported version of it did render. So I wanted to confirm whether the issue was K2 requiring walkmeshes when K1 didn't, or something MDLEdit was doing. I kept the compiled WOK file and reverted the MDL & MDX files for that room, and that turned it invisible again. I also tried removing the WOK while retaining the newer MDL & MDX files and the room was visible with that. So the fault seems to lie in MDLEdit. When it compiles a room that doesn't have a walkmesh, there's a chance that it won't render in the game.

[DeadMan 103]

On 8/28/2018 at 9:47 AM, DarthParametric said:

As with my previous hologram mods, these are both full body models, so the face meshes are part of the same model. It appears that the user's GPU, an nVidia GTX 970, is incapable of rendering the hologram effect on meshes using a normal map/tangentspace 1. 

That's really strange. I have GTX 970 too, and your bumpmapped models work fine in holograms for me. 

[DarthParametric 3,782]

I believe the declaration of card model was later revised by the user to a 960, but regardless I find it hard to reconcile it as being a hardware problem. Possibly some combination of driver and OS? Who knows. The individual in question does appear to have a singular ability to break the game in unconventional ways.

[ebmar 893]

11 hours ago, JCarter426 said:

Here's my latest attempt, with m02ab on Taris:

  Hide contents

 

Apologize for being off-topic but- wow. Taris looks very good at night, isn't it? 😮

Edited August 30, 2018 by ebmar

[JCarter426 1,214]

But it's not supposed to be night.

bead-v is working on it, though....

[ebmar 893]

7 minutes ago, JCarter426 said:

bead-v is working on it, though....

*gulp*

[ndix UR 218]

18 hours ago, JCarter426 said:

I noticed that it seemed to be limited to rooms that don't have a walkmesh, like the skybox and some of the background buildings.

It seems like there's some Kotormax-relative terminology here. I use the older terminology where 'walkmesh'es are WOK files, and the walkmesh in-model are AABB nodes; bead-v folded them into one thing in the interface so that people couldn't get them out of sync w/ each other. In these terms, m14aa's skybox does have a walkmesh, it does not have an AABB node. When you're discussing issues around this stuff, it is very helpful to differentiate.

bead-v chose to collapse the walkmesh functionality around the AABB nodes, ignoring the WOK files as much as possible (except for the information that uniquely resides within them), so, for these models where the area doesn't have an AABB node, it seems highly likely that there would be some kind of breakage, and I would expect MDLEdit not to be able to produce a WOK file (during ASCII=>Binary compilation phase). Probably the fix for bead-v will be to auto-add an AABB node from the WOK file (during Binary=>ASCII decompile) if the model is lacking an AABB but a WOK is present (assuming he hasn't already had to add such a feature).

Unfortunately, in several places within the tools (particularly the editor components), the presence of the AABB node is used to classify area models. Meaning, within the model-processing toolchain, there's a decent chance that the WOK file wouldn't even be loaded in cases where there's no AABB node, because we don't even know that the model in question is an area. It's pretty annoying that there's no classification for area models... I'm sure kotorblender would also experience issues with area models lacking AABB nodes, based on the shared use of that AABB-node == area model classification method. There's a chance that mdlops would work where MDLEdit doesn't though because it uses an ASCII WOK file during compilation, while MDLEdit never looks at such a file.

18 hours ago, JCarter426 said:

I've noticed before that many rooms have walkmeshes when they shouldn't need to, like the skyboxes on Citadel Station. Yet I went back and checked m14aa and its skybox does not have a walkmesh, and my ported version of it did render. So I wanted to confirm whether the issue was K2 requiring walkmeshes when K1 didn't, or something MDLEdit was doing. I kept the compiled WOK file and reverted the MDL & MDX files for that room, and that turned it invisible again. I also tried removing the WOK while retaining the newer MDL & MDX files and the room was visible with that. So the fault seems to lie in MDLEdit. When it compiles a room that doesn't have a walkmesh, there's a chance that it won't render in the game.

In these cases, you have to (often with difficulty) determine whether something "didn't render" or whether it's location was just messed up. They often seem similar. The most common thing that gets messed up is that instead of using the LYT/WOK position, an area model gets placed at 0,0,0 (the second most common is that its location gets 'double' applied, so instead of being at 30,30,45, the model's origin will be at 60,60,90). So sometimes I will load up the LYT in Blender, and move the misbehaving model to 0,0,0 and figure out in there where I might be able to see the model in-game if that were the situation. Anyway, that's just a debugging tip.

I've heard before something related to area models without WOK files or AABB nodes (I can't remember which) not rendering in-game, which I assume is what you're experiencing. bead-v was involved in debugging that and only relayed it to me secondhand, so he'll probably have something to say about it when his time permits. I think the idea that K2 requires an AABB node is probably right (or currently prevailing understanding), and that K1 not requiring an AABB node might be something we were unaware of. There might be more to it though, which would be nice to know. Seems like an easy enough thing to test w/ a hexedit... I'll give it a go when I get a bit of time.

[JCarter426 1,214]

22 minutes ago, ndix UR said:

It seems like there's some Kotormax-relative terminology here. I use the older terminology where 'walkmesh'es are WOK files, and the walkmesh in-model are AABB nodes; bead-v folded them into one thing in the interface so that people couldn't get them out of sync w/ each other. In these terms, m14aa's skybox does have a walkmesh, it does not have an AABB node. When you're discussing issues around this stuff, it is very helpful to differentiate. 

Right, yeah, they do have WOK files, but as far as I can tell they are totally empty. When imported in KOTORMax, they have no mesh with an OdysseyWalkmesh modifier, and when compiled with MDLEdit they do not produce a WOK file for that reason. There is, essentially, no walkmesh data at all for these rooms.

I hadn't considered this before, but I was not including the original WOK files in the folder I had the ASCII files in when exporting. I had them all there when I converted to ASCII, but then I moved the ASCII files to their own folder. This may or may not make a difference.

22 minutes ago, ndix UR said:

The most common thing that gets messed up is that instead of using the LYT/WOK position, an area model gets placed at 0,0,0 (the second most common is that its location gets 'double' applied, so instead of being at 30,30,45, the model's origin will be at 60,60,90).

Mm, I've checked for that. I zoomed out with GLIntercept when the rooms should have been visible, and they're definitely not rendering at all. Even if the skybox had been offset by that much, I would think at least some of it would be visible given its size and the coordinates.

I already sent the files to bead-v via Discord, but if you or anyone else wants to take a look, they're attached. Remember the binary models were compiled for K2, not K1, because of sinful porting.

Edit: bead-v had me check for the presence of aabb nodes. There isn't one on either the original K1 room or my ported v1. Obviously there is one on v2 since I manually added a walkmesh to the room.

m02ab_02r.zip

[ndix UR 218]

IIRC, the WOK files have a specific byte or uint32 set (or unset, or something like that) when their mesh data is empty. Maybe K1 used that info to ignore whether a model has AABB, while K2 just ignores it and requires the AABB... I guess it's unlikely because the whole division between WOK & AABB node is a remnant of the 'client-server' architecture, with the WOK being used (largely) to establish & enforce positioning on the 'server' side and the AABB node used for the 'client' side.

9 minutes ago, JCarter426 said:

Mm, I've checked for that. I zoomed out with GLIntercept when the rooms should have been visible, and they're definitely not rendering at all. Even if the skybox had been offset by that much, I would think at least some of it would be visible given its size and the coordinates.

Cool, sounds like you're definitely aware of this. I wasn't really expecting it to be a factor here, mostly putting it out there as a general debugging tip. Using GLIntercept is another nice thing to add to that apparently.

[JCarter426 1,214]

28 minutes ago, ndix UR said:

Maybe K1 used that info to ignore whether a model has AABB, while K2 just ignores it and requires the AABB... I guess it's unlikely because the whole division between WOK & AABB node is a remnant of the 'client-server' architecture, with the WOK being used (largely) to establish & enforce positioning on the 'server' side and the AABB node used for the 'client' side. 

Ah, that's interesting. I was wondering why even have a WOK format when some of the collision is on the AABB node on the model anyway... but I guess legacy NWN fussiness is always the explanation... when it isn't the Xbox's fault, anyway.

So far the only thing that does work is if the model has an AABB node... but it's not that K2 requires it. I checked the skybox of m14aa and neither the original K1 model nor my working ported version have an AABB node.

Still doesn't explain why K2 skyboxes tend to have them, but eh.

28 minutes ago, ndix UR said:

Cool, sounds like you're definitely aware of this. I wasn't really expecting it to be a factor here, mostly putting it out there as a general debugging tip. Using GLIntercept is another nice thing to add to that apparently.

Yeah, I'm used to that because I've caused the doubling-down on the layout coordinates and other problems before, like cutscene animations being stuck at origin. Breaking things is how you learn.

[bead-v 251]

15 hours ago, ndix UR said:

I would expect MDLEdit not to be able to produce a WOK file (during ASCII=>Binary compilation phase)

I confirm, you only get a wok if you feed it a model with an aabb node.

 

15 hours ago, ndix UR said:

there's a decent chance that the WOK file wouldn't even be loaded in cases where there's no AABB node, because we don't even know that the model in question is an area

Well, at least this much mdledit does. It won't read a .wok.ascii though, even if it's present.

 

14 hours ago, ndix UR said:

IIRC, the WOK files have a specific byte or uint32 set (or unset, or something like that) when their mesh data is empty.

In the K1 version of the wok for this particular file, that uint32 is 0. It's also unset for other woks (same module) where mesh data ( and therefore wok data) is present. So that's probably just some uninitialized memory getting in there for some reason. So maybe K1 simply takes the presence of a wok file as an indicator? Or maybe it doesn't even need that and always renders.

EDIT:
So me and @JCarter426 solved the puzzle (for now). There's gonna be an update about it in the area research thread (where most of the above really belongs), but I wanted to continue talking in this thread about the part that requires the tools to be modified.

Through testing, we discovered the following:
1. If there is no .vis file, K2 will render all area models, no exceptions
2. If there is a .vis file, K2 will render only the area models with aabb nodes, whether or not .wok files also exist.

K1 appears to not have the aabb requirement on rendering area models, so the aabb node is missing from a number of area models (though the .wok is present). So directly porting such models to K2 won't work, because it won't render since there's no aabb node.

In order to automate this, I could fairly easily do the following in mdledit: on decompilation, if there is a .wok file and no aabb node, generate a dummy aabb node. This could be an option that you turn on or off in preferences. This would make the area model render in K2, and wouldn't affect K1 as far as I can see. If my understanding is correct, MDLOps will also need an update to allow for this.

@ndix UR do you see any other solution?