MDLOps

[redrob41 86]

Ok, now that I've downloaded 0.7a2, I'll start posting some of my experiaments. I've been holding off so far, so that FS could have a bit of a break.

I hadn't seen that there were no shadows when I was doing my testing, because I usually have them turned off. Plus, for testing, I've made the Dantooine ground transparent, and the skybox is 128 Grey. That way, I don't have to work so hard with Photoshop to separate foreground from background. Anyway...

 

http://deadlystream.com/forum/gallery/image/449-mira-test-mdlops-7/

In the image for Mira's underwear, I've got in-game shots of her with various mdls.
- The one using Taina's replacer has smooth edges, but no shirt tie, and some of the vertices are weighted strangely (by default).
- Using MDLOps 0.6 you can see the hard edges (the back of the arms is the worst, but no screen shot), but at least the shirt ties work, and the weighting issues have been fixed (at least the ones that bothered me the most).
- Using MDLops 0.7a1 or a2 (only difference was the shadows) you can see that the edges are smooth along the wrist, fingers, inside leg, trapezius, but there is a hard edge below the deltoid / above the bicep. This is where the Larm/Rarm mesh object meets the Torso mesh object. I've checked a pair of verts, and they seem to be identical between Larm and Torso. Those faces were all set to the same smooth group.

 

http://deadlystream.com/forum/gallery/image/450-male-underwear-test/
- Using Taina's replacer, the model retains the original smooth groups. This results in smooth edges at the seams, but there's a noticible semi circle around the collar (this is where the vanilla DS skin has red cloth). This is th model that is included in my underwear mod.
- Using MDLops 0.7 fixes edges and removes any different smooth groups. I found that it didn't matter if I compiled the binary mdl straight from the unedited ascii.mdl or after I'd edited it in 3ds max 2011. It seems to me that mdlops is swithcing all faces to the same smooth group before calculating normals. The downside, is that the faces on top of the neck (where the head mdl sits) needs to be a different smooth group than the rest of the body. If they are the same smooth group, then it tried to round the edge, and thus it puts a highlight around the neck, rather than a shadow on one side (like the Taina model).
- Also, you can see the hard edge where the arms and torso meet when using 0.7a

So, I think that the new mdlops still isn't perfect (hence alpha). Hopefully, my testing can help out.

As a side note, when I was checking for shadows, I noticed that Mira's head didn't cast a shadow when in her undies or when wearing armor. Her head did cast a shadow when wearing her default clothes (I think that her body model includes bones for the head, but the other body models don't).

[Fair Strides 510]

In the current release of MDLOps, we don't bother to check the smooth group at all... And what are the exact coordinates of a vertex on Mira's torso and a vertex on her left arm that seem to overlap?

[redrob41 86]

In the current release of MDLOps, we don't bother to check the smooth group at all... And what are the exact coordinates of a vertex on Mira's torso and a vertex on her left arm that seem to overlap?

I checked a few verts around this seam:

As shown, X=-22.522cm Y=-4.118cm Z=132.47cm and the matching vert on the Larm had the same coordinates.

 

and I made the smooth groups the same for the skin of the shoulder (Torso object) and arm (Larm object):

[DarthParametric 3,795]

You'd have to re-weight the mesh, but one solution would be to split the mesh at any point where you want a seam and merge any pieces you want to have unified smoothing. So cut off the little bits of upper arm and join them to the main arm meshes.

[redrob41 86]

You'd have to re-weight the mesh, but one solution would be to split the mesh at any point where you want a seam and merge any pieces you want to have unified smoothing. So cut off the little bits of upper arm and join them to the main arm meshes.

sure, I could do that. And I could delete the mesh where the neck hole would be. As long as there remains the same Torso, Larm, Rarm configuration that all the body mdls are set up for, it could be a decent work around. But there are times when you want different smooth groups on the same mesh (like the shoes), but they can't be detached into their own object. Afaik, the game engine isn't set up for that (it expects Torso, Larm, Rarm). Of course, I could be completely wrong (I know next to nothing about game engines ).

 

If MDLops can make use of smooth groups and calculate normals across mesh seams, that'd be more useful in the long run.

[JCarter426 1,220]

For the model I was working on, with the shadow issues, I did have the shoes detached from the torso:

...and I haven't noticed any problems. I did just notice the same problem with the upper arms on mine, though.

 

Also, out of curiosity, what model(s) are you using for Mira there?

[DarthParametric 3,795]

There should be no reason that a split between torso/left arm/right arm as individual meshes is required as such. The model will animate identically regardless of whether it is one mesh or twenty, as long as the verts are all weighted the same. I suspect it is more an artefact of their production pipeline than any inherent requirement of the engine.

 

As to smoothing groups/normals, the only limitation of the MDL format is that there must be a 1:1 relationship between geometry verts and texture verts. It seems like NWMax exports smoothing group data correctly, so there should be no reason why MDLOps couldn't compile models with smoothing across UV seams at least. Theoretically that should extend to physically separate meshes sharing vert co-ords as well, but I'm not sure if that is necessarily the case in practice.

[VarsityPuppet 1,085]

I can't quite remember, but I think I told Fair Strides to do the checking for overlapping vertices between different meshes originally (in an attempt to get the Atris head working) but then realized we didn't need it.

 

Not sure if he reverted it then? Sounds like he did.

 

Either way, what would be the ideal solution? Check overlapping vertices on separate meshes for faces that have the same smooth groups? We would have to do that, I think. Keep in mind you'd have to make sure that any other vertices aren't overlapping as well (i.e. on a bone mesh and a skin) because that will also mess with the vertex normals.

[Fair Strides 510]

Actually, I kept that data, but I checked that there is less than 0.0001 between each axis of both vertices... Maybe try 0.01?

[JCarter426 1,220]

3ds Max only displays three decimal points for me... I don't know if that's just a setting that could be changed, or if it's a limit of the version I use. (3ds Max 8... no, not 2008. 8.) I usually use the snap toggle to have vertices overlap exactly, but even so, giving it a range three decimals or less would be safer. 0.01 sounds too broad to me, though. That's one centimeter, so accidental overlaps could happen. I'd say something between 0.001 and 0.005 should do it.

[DarthParametric 3,795]

Yeah the rounding is normal, regardless of version. Other 3D apps do it as well.

[redrob41 86]

Also, out of curiosity, what model(s) are you using for Mira there?

That's the standard PFBAM mdl, but heavily modified to match Mira's BB model almost perfectly (like the pronounced clavicles). My previous underwear mod was good, this one will be better.

 

There should be no reason that a split between torso/left arm/right arm as individual meshes is required as such. The model will animate identically regardless of whether it is one mesh or twenty, as long as the verts are all weighted the same. I suspect it is more an artefact of their production pipeline than any inherent requirement of the engine.

 

As to smoothing groups/normals, the only limitation of the MDL format is that there must be a 1:1 relationship between geometry verts and texture verts. It seems like NWMax exports smoothing group data correctly, so there should be no reason why MDLOps couldn't compile models with smoothing across UV seams at least. Theoretically that should extend to physically separate meshes sharing vert co-ords as well, but I'm not sure if that is necessarily the case in practice.

That's good to know. I'll keep it in mind as a last resort, since it would mean a whole bunch of extra work to re-weight the shirtless male & Mira undies models. If MDLops is tweaked to work the smooth groups better, then they would be ready now (I still have to re-do her dancer model before releasing it).

[JCarter426 1,220]

That's the standard PFBAM mdl, but heavily modified to match Mira's BB model almost perfectly (like the pronounced clavicles). My previous underwear mod was good, this one will be better.

 

That's good to know. I'll keep it in mind as a last resort, since it would mean a whole bunch of extra work to re-weight the shirtless male & Mira undies models. If MDLops is tweaked to work the smooth groups better, then they would be ready now (I still have to re-do her dancer model before releasing it).

 

I figured it had to be that, but I couldn't figure out how you had gotten it to be so accurate. You're scary sometimes.

 

And I'm seconding the request for the smoothing groups. I've been fiddling around with it some more and while everything does seem to be doable, the way many the models are set up would require a lot of editing just to retain the smoothing as is. A lot of them have the forearms on a separate mesh, and in my experience editing the arms is one of the trickiest parts. Simply running it through MDLOps loses the smoothing because of that. In contrast, a lot of them have the feet on the torso mesh and splitting them is required to keep everything as is.

[xander2077 80]

im about to run my own models through mdlops, but i see here the file is updated.

 

from what i remember you have to put all three files for a model in the same directory. but i forgot which button to push once i choose the model i want to convert back to binary. is it the read/ write? or the replace?

 

never mind, found the instructions on the DL page, and boy is that gui nice!

 

not knocking 3dsmax 8, it was my favorite before i got 9 years ago, and i wish i could still run both in linux. but anyway, in blender i can take it down to 0.0001 for the decimal points for proximal vertices i want to weld, however it jumps in increments of .001 so it has to be entered manually to tweak it just right. by default it is set already to 0.0001... anyway i always use that on import because the faces are all separated. if nwmax or neverblender do the same thing and export them as smooth groups, then it shouldn't matter if they are welded before export or not.

 

i cant remember what it was called in max, but in blender i can select all faces/vertices, and set the faces as smooth material. if that is something that can help the appearance i would suggest it, but to do so the verts must be welded or it wont work.

 

now something i do not know whether it effects the models or not is the sharps. if i mark something sharp it sometimes helps the appearance but im not sure if it breaks the model up or not. im not sure about seams either. i often just leave the original UVs alone unless i add geometry, and then if there is new geometry i only unwrap that portion or the island they logically belong to and keep the rest the same. if you can create seams from island then that is a trick that might help. if not then they need to be manually added.

 

once i export these i will give my models a test in game to see if they suffer from any anomalies, but first to download the updated mdlops.

[ndix UR 218]

I just spent waaaaaaay too long debugging MDLOps and I have a fix for you!

 

Setup: working on character models with blender using neverblender import/export

 

Symptom: recompiled characters don't have shadows in game. if they are playable NPCs (handmaiden say) and you enter combat with them active, the camera freaks out, doing a combination of moving inside the model's head and orbiting them really closely as if some kind of collision were happening.

 

Cause: neverblender exports models with classifications in ALL CAPS. It took me *so* long to figure this out. I tried just about everything else (returning weights, tverts, faces, orientations, all things neverblender changes on a simple import/export).

 

Solution: in MDLOps.pm change line around 1795:

    } elsif (/\s*classification\s+(\S*)/i) { # look for the model type
      $model{'classification'} = $1;

to

    } elsif (/\s*classification\s+(\S*)/i) { # look for the model type
      $model{'classification'} = ucfirst lc $1;

This is similar to how keys are used in the nodelookup hash.

 

Anyway, I hope this will save some blender character modders some grief somewhere down the line. Thanks for maintaining this essential software!

[xander2077 80]

I just spent waaaaaaay too long debugging MDLOps and I have a fix for you!

 

Setup: working on character models with blender using neverblender import/export

 

Symptom: recompiled characters don't have shadows in game. if they are playable NPCs (handmaiden say) and you enter combat with them active, the camera freaks out, doing a combination of moving inside the model's head and orbiting them really closely as if some kind of collision were happening.

 

Cause: neverblender exports models with classifications in ALL CAPS. It took me *so* long to figure this out. I tried just about everything else (returning weights, tverts, faces, orientations, all things neverblender changes on a simple import/export).

 

Solution: in MDLOps.pm change line around 1795:

    } elsif (/\s*classification\s+(\S*)/i) { # look for the model type
      $model{'classification'} = $1;

to

    } elsif (/\s*classification\s+(\S*)/i) { # look for the model type
      $model{'classification'} = ucfirst lc $1;

This is similar to how keys are used in the nodelookup hash.

 

Anyway, I hope this will save some blender character modders some grief somewhere down the line. Thanks for maintaining this essential software!

 

This is pretty cool that you tracked it down. I may have to pick your brain on blender import/export and some of the things to make sure are right before export. One of my biggest obstacles is skinned models and weights. Since you use neverblender, like I do, I think it is worth a discussion. Incoming PM.

[Purifier 22]

@ndix UR and xander2077

 

 

What version of blender are you guys using? I only use the old 2.49B, mainly for other games I have, but I could never get the Neverblender export made back in 2003 to work with the 2.49B version. And when was the NeverBlender import created? I thought the 2003 Neverblender export was all that was made long ago and that was it. 

 

I've been using 3Ds Max 7 for a while now as a 3d editor, but I would much prefer Blender to edit the character models and put them back in the game, if that's possible?

[xander2077 80]

@ndix UR and xander2077

 

 

What version of blender are you guys using? I only use the old 2.49B, mainly for other games I have, but I could never get the Neverblender export made back in 2003 to work with the 2.49B version. And when was the NeverBlender import created? I thought the 2003 Neverblender export was all that was made long ago and that was it. 

 

I've been using 3Ds Max 7 for a while now as a 3d editor, but I would much prefer Blender to edit the character models and put them back in the game, if that's possible?

 

 

I use blender 2.8 native to linux and i also have blender 2.49b portable that i use in wine. 2.8 is the one i have neverblender in, and it is a simple zip file install from within blender, so it is really easy to install.

 

For blender 2.49b, i cant figure out either how to get even the old version of neverblender working... so instead, i use a 2.49b blend file that opens the script window properly so i can run the import script for Kotor mdls, and this is the one i use only for problem mdl files that i have to force into blender. Most of them import fine into 2.8 with no issues. Once i open problem files in blender 2.49b, i can then save them and open them again in 2.8

 

If you want both you can also look for the portable app version of 2.49b and then do a full install of the latest version, that way you have both.

 

You can move files between new and old blender, but beware, if you save them in old blender they will work in both, but if you save them in new blender they will screw up in old blender... UNLESS: you save the file in new blender as a "legacy blend" file, however this destroys any animations and sometimes skin weights and bones. If you try to open a 2.8 file in 2.49b, it will import only the edges with no faces, and the rest of the things that were with the model will be gone, and sometimes whole sections of model will be missing. Also convert to tris before saving as a legacy because any quad faces will be deleted on opening in 2.49b.

 

Also the 2.49b import method i use, for some reason assignes a bijillion materials and other weird things in the file, so it has to be cleaned up before it will work as an aurora model.

[Purifier 22]

Ok, I think see. I take it somebody in the blender community made or updated NeverBlender import\export files for Blender 2.8? Which comes with the blender 2.8 package? And you can import the Kotor mdl model characters into Blender 2.8, edit them however you wish (but not all of them), then export them out, putting them back into the game?

 

But with the few models that won't work when you try to edit them in Blender 2.8 and put them back in the game, with the usual way of importing and exporting through Blender 2.8, you still import those models into Blender 2.8, but instead export them out of Blender 2.8 as a old blend file, so you can import them into the old blender 2.49 and then edit them in that old version of Blender, while also removing the ungodly amount of assigned materials to that model. Then you export it back out of blender 2.49 as a old blend file again and back into Blender 2.8 to export the model as a mdl file with the latest version of the NeverBlender for 2.8.

 

Is that right?

[xander2077 80]

no lol, i only use old blender (2.49b) to import some mdl files that wont import with the new (2.8) blender plugin.

 

There are two version on the neverwintervault site. one is for 2.6 (but is supposed to work in 2.49b even though it does not) and the newest one which i believe will work in 2.7 and up.

 

I would probably not want to export anything from blender 2.49b. The new blender plugin seems to handle export better in my opinion , unless you have better results than i do, but YMMV. It probably also depends on which operating system you have, since i can only use one import script and the rest will not work for me in linux. If you use a different script then your OS probably has a lot to do with the success rate of a plugin.

 

What happens when importing it to old blender (with the plugin i use) is a lot of extra materials get added to the model that dont belong. so i only use it in extreme cases and immediately save it as a blend and open that in blender 2.8. then i do my work there and export only from blender 2.8. the reason i even tried going the other way was because of the vertex paint feature in 2.49b that i really like which is the fake ambient occlusion. but when i open it again in 2.8 that is wiped so it was pointless to save it as a legacy blend file.

 

I just made mention of all that nonsense because i was trying out different things and so i know what will happen if you pass the model between the two versions of blender. best to import in 2.8 and export, and only import in 2.49b if you absolutely need to. Now there are plugins that 2.49b has that 2.8 does not (yet) but i use that more for morrowind and my workflow for that is the exact opposite. But that is a moot point for Kotor.

[Purifier 22]

Ah okay, I see what you're saying now. Well I have XP and that's about all I can use as a operating system for right now - been trying to save up and get a better computer with Win 7 or build one that I can put Win 7 on.

 

But meanwhile, I'm going to browse over to the NeverWinterVault website and look at those NeverBlender python scripts, probably download both versions you mentioned and then try messing with the Kotor models with the latest NeverBlender import\export python script version in Blender 2.8. 

 

I suspect the reason why the 2.6 NeverBlender import\export script would not work in the old blender 2.49 is because the programming writing format for python had changed since Python 2.6, because Python 2.6 is what the old 2.49 Blender runs on; and I think that change may have occurred when they introduced Python 3.0. several years ago. Which I think was about the same time Blender 2.5 or 2.6 started coming out. So the Blender community adapted to Python 3.0. for the latest versions of Blender.

 

Anyway, I'll take a look at the NeverBlender 2.6 in Python's Idle editor and see how the author wrote the import\export python scripts and see if he\she wrote it with python 3.0.

 

Thanks for taking the time to respond and explain all that, Xander2077. I appreciate that. If I can at least get some Kotor models in Blender 2.8 and edit them to where they will work in the Kotor games, that will be very helpful to me somewhat, because I'm a Blender fan and was a Blender user long before I ever fiddled with Gmax and 3Ds Max. Blender has always been my favorite 3D editor. 

 

[xander2077 80]

Ah okay, I see what you're saying now. Well I have XP and that's about all I can use as a operating system for right now - been trying to save up and get a better computer with Win 7 or build one that I can put Win 7 on.

 

But meanwhile, I'm going to browse over to the NeverWinterVault website and look at those NeverBlender python scripts, probably download both versions you mentioned and then try messing with the Kotor models with the latest NeverBlender import\export python script version in Blender 2.8. 

 

I suspect the reason why the 2.6 NeverBlender import\export script would not work in the old blender 2.49 is because the programming writing format for python had changed since Python 2.6, because Python 2.6 is what the old 2.49 Blender runs on; and I think that change may have occurred when they introduced Python 3.0. several years ago. Which I think was about the same time Blender 2.5 or 2.6 started coming out. So the Blender community adapted to Python 3.0. for the latest versions of Blender.

 

Anyway, I'll take a look at the NeverBlender 2.6 in Python's Idle editor and see how the author wrote the import\export python scripts and see if he\she wrote it with python 3.0.

 

Thanks for taking the time to respond and explain all that, Xander2077. I appreciate that. If I can at least get some Kotor models in Blender 2.8 and edit them to where they will work in the Kotor games, that will be very helpful to me somewhat, because I'm a Blender fan and was a Blender user long before I ever fiddled with Gmax and 3Ds Max. Blender has always been my favorite 3D editor. 

 

Yes the decision to focus on python 3.0 dependency is something i disagree with entirely. And this effected Nifscripts in a bad way, but Neverblender actually works right in spite of that misdirection. In linux, the distro has the default python set to 2.6 or 2.7, but it has provisions for 3.0 even though it is still largely in the beta stages for supporting older applications.

 

The problem comes in when older apps that need a lower version of python call for that version, and even if it is present (because linux keeps them all in separate directories and does not overwrite older versions) for some reason applications cannot "see" that version of python and fail.

 

While it is ok for blender to support 3.0, some of the scripts still depend on 2.6, and the way it is handled is so different there is no way to patch between the two reliably. So when some applications decided to move exclusively to 3.0, they jumped the gun before the bugs were worked out, and the python 3.0 dev team had to backtrack and try to figure out backward compatibility that they overlooked terribly, since some of the scripts that we all use were still dependent on lower versions. and even then the way they try to convert the python scripts to work in older versions is terrible. So there is a huge mismatch in versions and how fast people update their applications or scripts to the new language, nobody seems to be in line yet completely. they are all over the place. and then to top it off, some of the scripts have dependencies that have been omitted from the linux base, so they wont work any more, so it seems like to me that the application developers would try and bundle everything with the dependencies and older version of python but that is hit or miss...

 

Plus they did not foresee the issues that 3.0 would cause, and i think they are pretty much fixed now, but becuase they changed so much in python and because of the way linux handles python, it can be a huge hurdle to get everything working in concert.

 

i have suggested to them that they make some kind fo a library that can tell what version of python an app or script needs and direct it to that version without just brick walling, but i am not sure if they took it into consideration. In windows there may not be an issue with python, since you can always just install the necessary stuff in the same directory as the exe and get it working. Linux is not always so easy. sometimes they discard things before they should be becuase they assume everyone is up to speed, but that is never he case when it is open source you are talking about.

[LiliArch 115]

Ah okay, I see what you're saying now. Well I have XP and that's about all I can use as a operating system for right now - been trying to save up and get a better computer with Win 7 or build one that I can put Win 7 on.

Well I don't know about Blender 2.8, but 2.7 runs fine on XP (after installing that C++ runtime or whatever it was, if it's not installed already). I'd suppose the new neverblender should work with it.

[Purifier 22]

Well I don't know about Blender 2.8, but 2.7 runs fine on XP (after installing that C++ runtime or whatever it was, if it's not installed already). I'd suppose the new neverblender should work with it.

 

 

Yeah I found both 32 bit and 64 bit versions of the Blender 2.69 and downloaded that first. Learning the in's and out's of 2.69 right now, as I type this reply. Hmmm...probably should have downloaded Blender 2.7 instead. 

 

LiliArch, have you tried both versions? - and if you have - In your opinion, is there much difference between the Blender 2.69 and 2.7 version? 

 

Thanks for giving me notice about 2.7 running fine on XP, BTW. 

[xander2077 80]

To me there is a difference. It seems like every version of blender is different, and they only stayed the same for a very short time. (Maybe the period was somewhere between the 2.4 to right before 2.6.) The basic controls are the same, but they still need to refine the controls a bit. so that is why certain key combos don’t do the same things any more, or they stopped using certain functions that were good and adoped some that were similar but not as good, but overall the changes have been decent, better than it used to be. I just wish they would make it so you could drag the controls on the mesh and it was more interactive like max, but they have already been told they need to do that way back in 2.6 and it is now 2.8.