MDLedit bug reporting thread

[DarthParametric 3,782]

9 hours ago, bead-v said:

Will this be a good model for testing? @DarthParametric

Perfect, thanks. I was trying to do exactly that. While I could add controller nodes 9 and 10 via a hex editor, I didn't know what to do to adjust the changed offsets for the camerahook nodes.

Anyway, a test with this model with VBOs enabled did not result in a crash. Seeing as a version to achieve the same thing compiled with MDLEdit did crash, I must therefore assume that something MDLEdit (and MDLOps) is doing is causing the crash.

[bead-v 251]

9 hours ago, DarthParametric said:

Anyway, a test with this model with VBOs enabled did not result in a crash. Seeing as a version to achieve the same thing compiled with MDLEdit did crash, I must therefore assume that something MDLEdit (and MDLOps) is doing is causing the crash. 

Wait, so you decompiled and recompiled the model I sent you with mdledit and it crashed?

[DarthParametric 3,782]

No, I decompiled a clean copy of the vanilla model, added the same alpha, recompiled and it crashed. The hex edited version you sent doesn't crash, so in the absence of other evidence MDLEdit must be doing something that induces a crash in that scene.

Edit: Wait, I don't think you actually added a <1 mesh alpha to that model. I just noticed it looks like the head is sitting at ground level rather than transparent. Decompiling your model

node skin Head
  parent C_HoloRakata
  position 0.0 0.0 0.0
  orientation 1.066e-005 0.0365348 0.999332 3.14159

Versus decompiling the untouched vanilla model:

node skin Head
  parent C_HoloRakata
  position 7.58e-006 0.025986 0.710792
  orientation -0.0 0.707107 0.707107 3.14159

 

[bead-v 251]

Okay, in the mdledit-compiled version, the following types of bytes are different:

• Float bytes (green) – These are the most likely cause for the crash if the crash is caused by the binary. These are the reasons I can see why they might be different:
  • mdledit's calculations result in different numbers
  • a minimal difference on the float is introduced just from converting it from binary to ascii and back to binary
  • the total area of a mesh is 0 in the vanilla binary
  • animated UV floats have random values in the vanilla binary
• Some adjacent faces (purple) – unlikely
• Unused bone index bytes (purple) – unlikely
• Bone indices padding bytes (light grey) – unlikely
• Bone garbage array bytes (light grey) – unlikely
• Supermodel reference number bytes (light grey) – unlikely
• Padding bytes in the header, animation, controllers ... (light grey) – unlikely
• String padding bytes (dark grey) – unlikely
• MDX padding bytes (black) – unlikely

The recalculated floats are:

• In the MDL: face normals, tbones, qbones, orientation controller data if compressed.
• In the MDX: vertex normals, tangent space vectors.

My suggestion is to first try using the vanilla .mdx with the mdledit-recompiled .mdl (with alpha added to the Head node).

Unfortunately, I can't test this myself because my game crashes as soon as I enable VBOs.

[DarthParametric 3,782]

24 minutes ago, bead-v said:

mdledit-recompiled .mdl (with alpha added to the Head node)

That's my point though, there is no alpha added to the head node. It seems like the position and rotation controllers were edited rather than adding an alpha controller.

Edit: I removed all alpha, self-illum, and scale controllers from my custom model and it still crashes, so it seems like it's none of those at fault at any rate.

[bead-v 251]

29 minutes ago, DarthParametric said:

That's my point though, there is no alpha added to the head node. It seems like the position and rotation controllers were edited rather than adding an alpha controller.

Not sure what you're talking about here. You said:

1 hour ago, DarthParametric said:

I decompiled a clean copy of the vanilla model, added the same alpha, recompiled and it crashed

My question is, does it still crash if you use the vanilla .mdx instead.

Have you also simply tried recompiling the vanilla model without any changes?

[DarthParametric 3,782]

To clarify - the edited model you uploaded doesn't add an alpha controller to the head, it just positions the head on the ground. It doesn't crash though.

Decompiling the vanilla model and recompiling with no changes results in a crash at the end of the scene.

Decompiling the vanilla model and recompiling with no changes and then swapping the MDX for the vanilla one does not result in a crash. So it seems you are on the right track.

[bead-v 251]

1 hour ago, DarthParametric said:

Wait, I don't think you actually added a <1 mesh alpha to that model.

Yes, I missed some values in the new binary... here's a fixed version. This one definitely has alpha added to the Head node.
 

2 minutes ago, DarthParametric said:

Decompiling the vanilla model and recompiling with no changes and then swapping the MDX for the vanilla one does not result in a crash.

Awesome! So, the MDX contains the vert coords, vert normals, UV coords, tangent space vectors, weights and the padding which may be different after recompiling with mdledit. It can't be tangent space vectors, because there aren't any on the model and it'd be weird if it crashed because of vert or UV coords. The MDX padding is also unlikely. So that leaves the vert normals and the weights.

c_holorakata_vanilla_alpha_v2.zip

[DarthParametric 3,782]

Thanks, but I think the alpha thing was just a bad (if seemingly logical at the time) assumption on my part.

Is there anything I can do to further the MDX investigation?

[bead-v 251]

3 minutes ago, DarthParametric said:

Thanks, but I think the alpha thing was just a bad (if seemingly logical at the time) assumption on my part.

Is there anything I can do to further the MDX investigation?

I just checked all the weights on the model, they all seem to be exactly the same. I looked very quickly, but there's also no reason why the should be different and they seem to work on the model, so I'm excluding that.

I'm attaching an .mdx, if you can try that and it crashes, it'll exclude the padding bytes.

That leaves the vert coords, UV coords and normals, the normals being the most likely cause.

The algorithm fails to find some cross-mesh normals in that model, I will add those to another .mdx and send it for testing.

c_holorakata_alphaed_v2.mdx

[DarthParametric 3,782]

No crash with that MDX.

[bead-v 251]

How about this one?

c_holorakata_vanilla_v1.mdx

[DarthParametric 3,782]

No crash with that MDX.

[bead-v 251]

Another one...

Also, what version of mdledit did you use to recompile the model?

c_holorakata_alphaed.mdx

[DarthParametric 3,782]

The last one you uploaded. The zip was labelled as 1.0.10 but the program itself displays 1.0.99 (same as the previous version).

Edit: No crash with that one either.

[bead-v 251]

Ha! Well, something's fishy, because the last one is a straight recompile after adding an alpha to Head (which doesn't affect the MDX at all and is therefore the same as a straight vanilla recompile).

[JCarter426 1,214]

4 hours ago, bead-v said:

So, the MDX contains the vert coords, vert normals, UV coords, tangent space vectors, weights and the padding which may be different after recompiling with mdledit.

Would that be why the visual effect models included in one of the official patches have MDLs but no MDXes? I've always found that strange.

[bead-v 251]

Just now, JCarter426 said:

Would that be why the visual effect models included in one of the official patches have MDLs but no MDXes? I've always found that strange.

Yes, all that the MDX contains is data for every vertex of every mesh, which includes:

• vertex coords
• vertex normal
• vertex color
• uv1 coords
• uv2 coords
• uv3 coords
• uv4 coords
• tangent space 1 coords
• tangent space 2 coords
• tangent space 3 coords
• tangent space 4 coords
• weight values
• weight indices

+ an extra empty vertex structure + sometimes some padding.

So if there is no mesh on the model, then there's no MDX data. In such cases there is still an .mdx in the game files, but it's just an empty file. So visual effects, animated cameras, etc... all don't need .mdx's at all.

[JCarter426 1,214]

So you're saying I've installed literally hundreds of MDX files for no reason. Lovely.

[DarthParametric 3,782]

So I am guessing the MDX issues, whatever they are, are probably also the cause of VP's Ajunta Pall's ghost model crashes. Have you narrowed it down to a specific subset yet @bead-v? The normal data?

 

Edit: On a different note, I see that when loading a binary model and switching to hex view, you are adding data that doesn't actually exist in the loaded model. For example:

I gather MDLEdit is showing what would exist in a model it compiled (since you can look at the hex view of a loaded ASCII as well) rather than what actually exists. In this case, some of the "imaginary" data seems to be TSL-related, stuff like the hide in hologram flag. But this is a K1 model.

[bead-v 251]

3 hours ago, DarthParametric said:

So I am guessing the MDX issues, whatever they are, are probably also the cause of VP's Ajunta Pall's ghost model crashes. Have you narrowed it down to a specific subset yet @bead-v? The normal data?

Nope, what we found so far is inconclusive. You say the vanilla model crashes when recompiled with the latest mdledit, I sent you an .mdx which is identical (we should check that actually, but it should be identical) to the .mdx that caused a crash, and apparently this one didn't cause a crash. If the crashes don't happen regularly, which is possible, it may be that the issue is not even the .mdx and we were just lucky to not get a crash while testing the .mdx's.

 

 

3 hours ago, DarthParametric said:

Edit: On a different note, I see that when loading a binary model and switching to hex view, you are adding data that doesn't actually exist in the loaded model. For example:

I gather MDLEdit is showing what would exist in a model it compiled (since you can look at the hex view of a loaded ASCII as well) rather than what actually exists. In this case, some of the "imaginary" data seems to be TSL-related, stuff like the hide in hologram flag. But this is a K1 model.

If you load a binary model, the current game in mdledit changes to match the one of the loaded binary model. If you load an ascii model however, it compiles the model to whatever game was selected. And yes, mdledit always has compiled model data in memory (which is what it displays in the hex view. Also, when you save the model as binary, all it does is print those bytes to a file. All the processing happens when the model is loaded). Some actions in mdledit (changing the game/platform, adding bumpmapping) will cause a recompilation of the model, so the hex view will also show these changes. Apparently you changed the game to K2 (because it says so in your screenshot), so you caused the model to recompile in memory. At this point, you could change the game again, then load the original model with 'Compare to ...' to see what changes mdledit introduces to the model during recompilation.

[DarthParametric 3,782]

18 minutes ago, bead-v said:

we should check that actually, but it should be identical

Yes, you are correct. The MDXs are a match. I just tested a straight decompile/recompile of the vanilla and it didn't crash. I was certain it did crash the last time I tested it, but I have so many different versions now I can't keep track. I tried a fresh attempt at decompiling the vanilla, adding an alpha 0.5 to the head mesh, then recompiling. This did crash, but the MDX was identical to the straight recompile, so maybe we are back to where we started with it being a MDL problem. Here are the files if they are of use.

The_Ongoing_HoloRakata_Saga.7z

21 minutes ago, bead-v said:

Apparently you changed the game to K2

Oh god damn it.....

OK, that's my bad. I must have accidentally hit it when selecting hex mode in the menu. I have done that a few times actually. Maybe you can add "move buttons above menu bar" to your future ideas wishlist.

 

[bead-v 251]

Me and VP had a cutscene in 903MAL for MVI which crashed often, but at very specific times during the cutscene. It didn't crash always though... at the time we didn't have the necessary knowledge to figure it out. This seems like it's more or less the same. So first we'd need to establish the following:
1. The vanilla model doesn't ever crash.
2. Does a straight mdledit v1.0.100 recompile sometimes crash or not?

Do you feel like you've repeated the scenarios enough times you could confirm/answer those?

 

25 minutes ago, DarthParametric said:

I have done that a few times actually. Maybe you can add "move buttons above menu bar" to your future ideas wishlist.

Happens to me too... That's one thing I can't do without putting in A HUGE AMOUNT of energy, so much it's definitely not worth it. Besides, you'd still hit them by accident like that. I could move them between the two panels, but I bet people would hit them by accident there as well.
 

[DarthParametric 3,782]

16 minutes ago, bead-v said:

1. The vanilla model doesn't ever crash.

Almost certainly the vanilla doesn't crash. Or at the very least I have never had it crash. I believe both @JCarter426 and @jc2 tested it and reported no crashes with VBOs enabled either when I had them test my custom model (which does crash).

16 minutes ago, bead-v said:

2. Does a straight mdledit v1.0.100 recompile sometimes crash or not?

I have run several tests of it in a row with no crashes. It may need 3rd party confirmation, but it at least appears that a straight recompile does not crash.

 

My question is if you do a hex comparison of a straight recompile MDL with a recompile MDL with a single alpha controller added, why is it so significantly different?

[bead-v 251]

12 minutes ago, DarthParametric said:

I have run several tests of it in a row with no crashes. It may need 3rd party confirmation, but it at least appears that a straight recompile does not crash.

Okay, so it appears at least that the changes a mdledit recompilation introduces aren't the culprit. So I guess we're back to suspecting the added alpha controller.
 

14 minutes ago, DarthParametric said:

why is it so significantly different?

If you're using mdledit's comparison function, it doesn't take offsets into account because that'd be way too much work to implement. So it's basically only useful for comparing models that should be for the most part 100% identical. If you add one byte in the beginning of a binary and compare with the original for example, it will mark the entire file as different. Anyway, if you add the alpha on the last mesh (Head), only the very last part of the MDL suffers from this, while before that the only changes are to the total MDL size (the first four yellow bytes in the header), the offset to the camerahook (at offset 73073) and the offsets and counts of controllers and controller data for the Head node (block 218471–218494,  this is the part I forgot when I sent you the first file above). So it really is minimally different.