Tackling Model Seams

[VarsityPuppet 1,085]

Gents,

 

 

One of the many loose ends in modeling for KOTOR is those pesky modeling seams on custom models.

 

For most uses, they're not that visible, but it's especially jarring if displayed on a head model.

 

 

 

 

 

 

So this is where Fair Strides and I are at currently.

 

 

1) Extracted PMHC01.mdl and PMHC01.mdx from KotorTool.

 

2) Use Mdlops6 to decompile, get ascii

 

3) Recompile the ascii to make PMHC01-ascii-k2-bin.mdl and PMHC01-ascii-k2-bin.mdx (compile with super model s_female01)

 

4)Place PMHC01-ascii-k2-bin.mdl and PMHC01-ascii-k2-bin.mdx into the override (rename to PMHC01.mdl and PMHC01.mdx)

 

 

5) Take the original non-modified version of PMHC01.mdx and put that into the override, replacing the new compiled version and the result is...

 

 

FIXED. M*THER. F*CKING. SEAMS.

 

 

So the key to fixing the smoothing on models is in the mdx, we believe.

 

 

 

 

The issue we're still having though is... what part of the mdx data controls the seams?

 

In a test model, we've been playing with calculating vertex normals with no real success in eliminating seams. Could it be we're calculating vertex normals incorrectly? Not sure yet.

 

What I do know is that in contrast to our test mdl which has 32 bytes of vertex information, the head model has 64 bytes of information, that information I'm not entirely sure what that is yet.

 

¯\_(ツ)_/¯

 

Anyways, this will just be a progress thread. Anyone is welcome to contribute, files are attached below.

 

 

I'll be looking at the PMHC01.mdx files to see what different information is included that is smoothing out these meshes.

PMHC01.zip

PLC_metalbox.zip

[newbiemodder 37]

Nice work guys.

[DarthParametric 3,777]

The root cause of the problem was detailed by cchargin (original MDLOps author) back in 2005:

 

If you were to peek into an .mdx you would see the data stored this way:

 

vertex x, vertex y, vertex z, normal x, normal y, normal z, texture u, texture v

 

As you can see geometry, shader, and texture information are stored together. This is why we have that annoying 1 to 1 relationship for geometry vertices and texture vertices. This one is kotors fault. If you want more texture verts you have to break vertices apart, which brings us to the next problem.

 

Since NWMax does not calculate normals for us and kotor can't compile models on the fly, MDLOps has to do the calculations. Right now MDLOps assumes that all polygons with welded vertices should be smoothed together. And of course all unwelded vertices will not be smoothed, even if they are in the same smooth group. This is my fault.

 

Now that I look at an ascii model output from NWMax It does appear that NWMax outputs smooth group information. I will have to see if I can use the smooth group info when calculating the vertex normals.

http://www.lucasforums.com/showpost.php?p=1973627&postcount=11

 

Unfortunately it seems that he never got around to addressing it, nor did JdNoa in the subsequent revised version.

[VarsityPuppet 1,085]

The root cause of the problem was detailed by cchargin (original MDLOps author) back in 2005:

 

 

http://www.lucasforums.com/showpost.php?p=1973627&postcount=11

 

Unfortunately it seems that he never got around to addressing it, nor did JdNoa in the subsequent revised version.

 

I've been running tests on a low poly model, 8 vertices total. It's a pyramid with no bottom, one side is a detached mesh.

 

So of the 8 verts, 3 are duplicates.

 

 

Anywho, I've been playing around with generating the vertex normals, only to have no real changes done to the model in-game. So I'm led to believe that I must be calculating them wrong? Not sure ¯\_(ツ)_/¯

[DarthParametric 3,777]

Playing around how? By hex editing the compiled MDX?

[VarsityPuppet 1,085]

Playing around how? By hex editing the compiled MDX?

Yes.

 

Reading the ascii vertices and faces and then computing the vertex and face normals, then writing a custom mdx file to disk. I'm using C# to parse a section of the ascii data and write it as mdx data

[DarthParametric 3,777]

You could try getting the normal info from Max via MaxScript. There are no doubt plenty of existing scripts that you could use/modify. At the very least they may be a useful reference.

[Fair Strides 509]

You could try getting the normal info from Max via MaxScript. There are no doubt plenty of existing scripts that you could use/modify. At the very least they may be a useful reference.

If we were to go that route, I could edit NWMax to print out the vertex normals in the exporter...

 

As it is, doing it in 3DSMax could possibly be done: https://www.opengl.org/discussion_boards/showthread.php/128451-How-to-calculate-vertex-normals?p=966252&viewfull=1#post966252

[DarthParametric 3,777]

Yeah, briefly poking through the scripts it seems to my uninformed mind that NWMax is actually calculating normals/tangent normals/binormals, it just doesn't export any of that info in the ASCII MDL.

 

Interestingly, I was looking at Eshme's Dragon Age Origins export script to see what it does (as it does export normal info), and I noticed that the Eclipse Engine appears to still have the same underlying issue, namely the 1:1 of verts and texture verts. I suppose it's not completely surprising, given that Eclipse is just an evolution of the Aurora/Odyssey codebase.

[VarsityPuppet 1,085]

Good news!

 

Turns out, I'm terrible with mixing up override files...

 

Before:

 

After:

 

 

Fixed!

 

There are still some loose ends though. The algorithm being used for vertex normals is not the optimal for visual experience. More is explained here.

 

On the other hand, if 3DSmax is exporting normal info, then we might as well just use that

 

 

Basically, what we need to do (and I feel yet again that Fair Strides has already mentioned this in the past) is, check the smoothing groups for faces and overlapping vertices and factor those into vertex normal calculations.

 

 

If 3dsMax already does this properly, we are good to go. Just gotta update that mdlops

[DarthParametric 3,777]

I did a quick play around with it. Made your 4 faced pyramid with 1m sides, split each face out to its own UV island, left the verts welded (5 total) and set a single smoothing group. Exported it as a DA model via Eshme's script to generate normal data and split the faces to match the number of UV verts, then reimported the model. Set it up as a KOTOR model and exported via NWMax.

 

 

On import the normal data is stored in an Edit Normals modifier, so once I deleted that it reverted to hard edges at element breaks (as per MDLOps).

 

Here is the generated data -

 

 

NWMax:

  verts 12
    0.499756 -0.499756 0.0
    -0.499756 0.499756 0.0
    0.0 0.0 0.999512
    0.499756 0.499756 0.0
    -0.499756 -0.499756 0.0
    0.499756 -0.499756 0.0
    -0.499756 0.499756 0.0
    0.0 0.0 0.999512
    0.0 0.0 0.999512
    0.0 0.0 0.999512
    0.499756 0.499756 0.0
    -0.499756 -0.499756 0.0
  faces 4
    2 4 0  1  2 4 0  1
    7 5 3  1  7 5 3  1
    8 10 1  1  8 10 1  1
    9 6 11  1  9 6 11  1
  tverts 12
    0.990234 0.00976563 0
    0.495117 0.504883 0
    0.747559 0.495117 0
    0.990234 0.504883 0
    0.504883 0.00976563 0
    0.504883 0.504883 0
    0.0100021 0.00976563 0
    0.747559 0.989998 0
    0.252686 0.989998 0
    0.252686 0.495117 0
    0.0100021 0.504883 0
    0.495117 0.00976563 0

DA Import/Export Script:

<Data ElementCount="12" Semantic="POSITION" Type="Float4">
<![CDATA[
0.4999 -0.4999 0.0 1.0
-0.4999 0.4999 0.0 1.0
0.0 0.0 0.9998 1.0
0.4999 0.4999 0.0 1.0
-0.4999 -0.4999 0.0 1.0
0.4999 -0.4999 0.0 1.0
-0.4999 0.4999 0.0 1.0
0.0 0.0 0.9998 1.0
0.0 0.0 0.9998 1.0
0.0 0.0 0.9998 1.0
0.4999 0.4999 0.0 1.0
-0.4999 -0.4999 0.0 1.0
]]>
</Data>
<Data ElementCount="12" Semantic="TEXCOORD" Type="Float2">
<![CDATA[
0.99 0.99
0.495148 0.495149
0.747426 0.504852
0.99 0.495149
0.504851 0.99
0.504851 0.495149
0.01 0.99
0.747426 0.0100001
0.252574 0.0100001
0.252574 0.504852
0.01 0.495149
0.495148 0.99
]]>
</Data>
<Data ElementCount="12" Semantic="TANGENT" Type="Float4">
<![CDATA[
0.816497 0.408248 -0.408248 1.0
-0.816497 -0.408248 -0.408248 1.0
1.0 0.0 0.0 1.0
-0.408248 0.816497 -0.408248 1.0
0.816497 -0.408248 0.408248 1.0
0.408248 0.816497 0.408248 1.0
-0.408248 -0.816497 0.408248 1.0
0.0 1.0 0.0 1.0
-1.0 0.0 0.0 1.0
0.0 -1.0 0.0 1.0
-0.816497 0.408248 0.408248 1.0
0.408248 -0.816497 -0.408248 1.0
]]>
</Data>
<Data ElementCount="12" Semantic="BINORMAL" Type="Float4">
<![CDATA[
0.154303 -0.617213 -0.771517 1.0
-0.154303 0.617213 -0.771517 1.0
0.0 -1.0 0.0 1.0
0.617213 0.154303 -0.771517 1.0
-0.154303 -0.617213 -0.771517 1.0
0.617213 -0.154303 -0.771517 1.0
-0.617213 0.154303 -0.771517 1.0
1.0 0.0 0.0 1.0
0.0 1.0 0.0 1.0
-1.0 0.0 0.0 1.0
0.154303 0.617213 -0.771517 1.0
-0.617213 -0.154303 -0.771517 1.0
]]>
</Data>
<Data ElementCount="12" Semantic="NORMAL" Type="Float4">
<![CDATA[
0.57735 -0.57735 0.57735 1.0
-0.57735 0.57735 0.57735 1.0
0.0 0.0 1.0 1.0
0.57735 0.57735 0.57735 1.0
-0.57735 -0.57735 0.57735 1.0
0.57735 -0.57735 0.57735 1.0
-0.57735 0.57735 0.57735 1.0
0.0 0.0 1.0 1.0
0.0 0.0 1.0 1.0
0.0 0.0 1.0 1.0
0.57735 0.57735 0.57735 1.0
-0.57735 -0.57735 0.57735 1.0
]]>
</Data>

 

The vert order and position match (leaving aside rounding), but interestingly texture vert position seems to have a few anomalies. Anyway, I thought the normal info may be of interest to you as an example of the sort of thing that can be extracted from Max.

[Fair Strides 509]

Well, I know what I'm doing when I wake up later... (It's 1:58AM here)

 

Going off of VP's "more info" link, I should be able to add this to both MDLOps and KAurora quickly, so we could be seeing minor updates soon.

[Sithspecter 909]

Well, I know what I'm doing when I wake up later... (It's 1:58AM here)

 

Going off of VP's "more info" link, I should be able to add this to both MDLOps and KAurora quickly, so we could be seeing minor updates soon.

 

What would be the point of adding this feature to KAurora?

[VarsityPuppet 1,085]

What would be the point of adding this feature to KAurora?

Why not? Vertex normals are pretty common to most 3D engines, and clearly the Odyssey engine was making use of them. The way I see it, it was just a feature that was not implemented properly in mdlops and KAurora.

[redrob41 86]

Good news!

 

Turns out, I'm terrible with mixing up override files...

 

Before:

 

After:

 

 

Fixed!

Can you post/share the mdl & mdx of the before and after, so that we can compare the two? I just want to visualize it (either ascii or hex).

 

I'm excited by the possibilities

[Fair Strides 509]

Before:

 

 

 

After:

 

[redrob41 86]

Is it just the seam on Mira's hip that was fixed? I'm not sure if I see any others that've been fixed.

 

Will this new method work on Mira's head model? I've never been able to get my female Devaronian working, because Mira's head model doesn't play nice with mdlops.

 

Hmm, I wonder if I'd be able to add her shirt tassels to the underwear mod I did for her (once upon a time)?

[Fair Strides 509]

Is it just the seam on Mira's hip that was fixed? I'm not sure if I see any others that've been fixed.

 

Will this new method work on Mira's head model? I've never been able to get my female Devaronian working, because Mira's head model doesn't play nice with mdlops.

On her body model, the hip/waist is the most noticeable part.

 

Running Mira's head through MDLOps once to decompile and once again with no edits yields this:

 

Before (Compiled with MDLOps 0.5 and then ran through VP's Headfixer)

 

 

 

After (Compiled with the new MDLOps and then ran through VP's Headfixer)

 

 

 

As you can see from Mira's upper lip, things still aren't quite perfect. VP and I want to add another extra leg or two to the calculations that will hopefully fix that issue.

[VarsityPuppet 1,085]

On her body model, the hip/waist is the most noticeable part.

 

Running Mira's head through MDLOps once to decompile and once again with no edits yields this:

 

Before (Compiled with MDLOps 0.5 and then ran through VP's Headfixer)

 

 

 

After (Compiled with the new MDLOps and then ran through VP's Headfixer)

 

 

 

As you can see from Mira's upper lip, things still aren't quite perfect. VP and I want to add another extra leg or two to the calculations that will hopefully fix that issue.

The vertex shading seems to be working fine there. What you're seeing is the teeth mesh clipping through the skin.

 

Also, the eyes are wonky AF

[DarthParametric 3,777]

I believe that is related to one of the issues I was having with custom droid models, namely that vert positions seem to end up offset from their original position even if left untouched. I've run into it a few times with heads as well, although it seems less pronounced in those cases. Mission's head in K1 springs to mind. I remember trying to change the UV map and ending up with a face clipping horror show. I've also encountered it with high poly versions of meshes. When I was playing around with the weighting for the Nautolan head, I tried a 5K vert/10K poly smoothed version of the mesh (vs 1.2K vert/2.5K poly unsmoothed original) and it ended up with all sorts of mouth clipping as well.

[VarsityPuppet 1,085]

Success.

Separated Atris head with proper smoothing.

This is the same Atris head I separated long ago, around the time of HeadFixer.



I've included Atris' Head as a modder's resource for download. No permissions required.

 

http://http://deadlystream.com/forum/files/file/773-modders-resource-atris-head/

 

I may include an install package to replace Atris' Full body model at some later point.

[redrob41 86]

On her body model, the hip/waist is the most noticeable part.

 

Running Mira's head through MDLOps once to decompile and once again with no edits yields this:

 

Before (Compiled with MDLOps 0.5 and then ran through VP's Headfixer)

 

 

 

After (Compiled with the new MDLOps and then ran through VP's Headfixer)

 

 

 

As you can see from Mira's upper lip, things still aren't quite perfect. VP and I want to add another extra leg or two to the calculations that will hopefully fix that issue.

 

It looks like her eyes, eyelids and teeth have all been offset forward. It's hard to see from that angle, but I think that the hair mesh (the part that is a separate mesh from the main head) has also been offset forward. With the new MDLops, do you really still need to use the HeadFixer? Can't it be incorporated into the new MDLops?

 

I believe that is related to one of the issues I was having with custom droid models, namely that vert positions seem to end up offset from their original position even if left untouched. I've run into it a few times with heads as well, although it seems less pronounced in those cases. Mission's head in K1 springs to mind. I remember trying to change the UV map and ending up with a face clipping horror show. I've also encountered it with high poly versions of meshes. When I was playing around with the weighting for the Nautolan head, I tried a 5K vert/10K poly smoothed version of the mesh (vs 1.2K vert/2.5K poly unsmoothed original) and it ended up with all sorts of mouth clipping as well.

I've always had problems with Mira's head (as well as Bao-Dur and some of the other party's heads).

[Fair Strides 509]

 

It looks like her eyes, eyelids and teeth have all been offset forward. It's hard to see from that angle, but I think that the hair mesh (the part that is a separate mesh from the main head) has also been offset forward. With the new MDLops, do you really still need to use the HeadFixer? Can't it be incorporated into the new MDLops?

 

 

I've always had problems with Mira's head (as well as Bao-Dur and some of the other party's heads).

The offsets could very well be there, but I could try compensating for that in 3DSMax.

 

As far as Headfixer, it works by hex-editing specific spots in the binary model, spots whose position doesn't change between models. I'd need to ask VP what the new values are, but there is also the issue of finding out where the hex-edited info lines up in the ascii model/MDLOps' compiling process.

 

Short answer: Headfix is still needed, since right now trying to find where its binary data is in the ascii/compilation is not an easy thing.

[redrob41 86]

The offsets could very well be there, but I could try compensating for that in 3DSMax.

 

As far as Headfixer, it works by hex-editing specific spots in the binary model, spots whose position doesn't change between models. I'd need to ask VP what the new values are, but there is also the issue of finding out where the hex-edited info lines up in the ascii model/MDLOps' compiling process.

 

Short answer: Headfix is still needed, since right now trying to find where its binary data is in the ascii/compilation is not an easy thing.

Fair enough. As long as a tool works, I'll use it

 

So, will it now be possible to make Jedi robes for Bao-Dur? With the arm?  I hope so. Maybe then I can improve my armors that I started and never finished - http://deadlystream.com/forum/topic/3377-request-missing-bao-dur-body-models/

[Fair Strides 509]

Fair enough. As long as a tool works, I'll use it

 

So, will it now be possible to make Jedi robes for Bao-Dur? With the arm?  I hope so. Maybe then I can improve my armors that I started and never finished - http://deadlystream.com/forum/topic/3377-request-missing-bao-dur-body-models/

 

I would assume yes it would work now. I just need to get one or two more things working with VP before I could think about releasing it.