MDLOps

[Fair Strides 509]

 

File Name: MDLOps

File Submitter: Fair Strides

File Submitted: 14 Jan 2016

File Category: Modding Tools

 

 

-----------------------------------------------------------------
-->--
-----------------------------------------------------------------

 

-------------------------
-->--
-------------------------

July 1 2004: First public release of mdlops.pl version 0.1

July 17 2004: Added support for vertex normals (thanks JRC24)

 

August 4 2004: Fixed division by zero bug in vertex normals (thanks Svosh)

 

October 18, 2004: Version 0.3
-Now ignores overlapping vertices
-fixed a bug that caused some controllers
to be ignored (thanks T7nowhere and Svosh)
-updated docs on how texture maps work in
kotor (thanks T7nowhere and Svosh)
-new tutorial written by bneezy
-Svosh updated the quick model tutorial
November 18, 2004: Version 0.4
added replacer function (idea originally suggested to me by tk102)
gui does not get built when using command line (thanks Fred Tetra)
added ability to rename textures in binary models (thanks darkkender)
cool new icon created by Svosh. Thanks Svosh!

 

March 8, 2005: Version 0.5
figured out that some meshes have 2 textures (thanks Fred Tetra)
added fix for meshes that have 0 verticies (thanks Fred Tetra)
added support for Kotor 2. The model is bigger by only 8 bytes per mesh!
the program will auto-detect if a binary model is from kotor 1 or kotor 2

 

March 9, 2006: Version 0.6alpha4
Added aabb support (thanks Fred Tetra)
Vastly improved ascii import speed (optimized the adjacent face routine)
Added partial support for aurora lights
ANIMATIONS! MUCH MUCH thanks to JdNoa for her cracking of the compressed quaternion format
and for writing the animation delta code!

 

May 21, 2007: version 0.6.1alpha1 (changes by JdNoa)
Added support for compiling animations. Code mostly ported from Torlack's NWN compiler.
Added controllers for lights and emitters, but not tested yet.

 

January 13, 2016: Version 0.7alpha
Reworked calculations of face normals
Reworked calculations of vertex normals

 

Summer, 2017: Version 1.0.0
Reworked calculations of face, vertex normals, plane distances, adjacent faces
Added tangent space calculations
Added emitter and finished light node support
Added walkmesh support (DWK/PWK/WOK)
Added lightsaber mesh support and conversion
Added bezier controller support and fixed existing controller list
Added normalization of vertex data into MDX form
Added detection of real smoothgroups
Added reference node support
Added super model node number construction
Fixed replacer for many cases
Many more small fixes and features

 

 

 

 

-----------------
-->--
-----------------
This script is released under the GPL, see the included
GPL.txt.

 

----------------
-->--
----------------
MUCH MUCH MUCH thanks to Torlack for his NWN MDL info!
Without his info this script could not exist!

 

Thanks to my testers:
T7nowhere
Svosh
Seprithro
ChAiNz.2da

 

Thanks to all at Holowan Laboratories for your input
and support

 

file browser dialog added by tk102

 

AABB, animations, lights and emitters, and speed-up by JDNoa

 

Calculations of vertex and face normals by VP and Fair Strides

1.0 version by ndix UR, thanks to bead-v for inspiring many of the added
features, figuring out a number of critical format algorithms, and sharing
the journey

 

-----------------------
-->--
-----------------------

This is a Perl script for converting
Star Wars Knights of the Old Republic (kotor 1 for short)
AND Star Wars Knights of the Old Republic, The Sith Lords (kotor 2 for short)
binary models to ascii and back again.

 

Binary models are converted to an ascii format compatible
with NeverWinter Nights.

It can also do some other operations on models,
like renaming textures and replacing meshes.

 

------------------
-->--
------------------
-Automatic detection of binary model version
-Automatic detection of model type
-node types supported:
-trimesh
-danglymesh
-lightsaber
-skin
-emitter
-light
-reference
-dummy
-model properties supported:
-diffuse
-ambient
-shadow
-render
-alpha
-self illumination
-many, many more
-supports compile/decompile pwk/dwk/wok walkmesh files
along with their associated models
-when reading in a binary model a text file is created
that lists all the textures the model uses.
-replacer function lets you replace 1 tri-mesh in a binary
model with another tri-mesh from an ascii model
-renamer function lets you rename textures in a binary
model

read the tutorials "KotOR_Tutorial.txt" and "Quick_tutorial.txt"
for an explanation of how to get your models into kotor

 

----------------------------
-->--
----------------------------

command line usage of the compiled perl script:
View all command line options:
mdlops.exe --help
mdlops.exe [-a] [-s] [-k1|-k2] c:\directory\model.mdl
OR
mdlops.exe [-a] [-s] [-k1|-k2] c:\directory\*.mdl

command line usage of perl scripts:
NOTE: you must first copy the MDLOpsM.pm file into your \perl\lib or
\perl\site\lib directory
View all command line options by running:
perl mdlops.pl --help

 

perl mdlops.pl [-a] [-s] [-k1|-k2] c:\directory\model.mdl
OR
perl mdlops.pl [-a] [-s] [-k1|-k2] c:\directory\*.mdl

 

For the command line the following switches can be used:
-a will skip extracting animations
-s will convert skin to trimesh
-k1 will output a binary model in kotor 1 format
-k2 will output a binary model in kotor 2 format

 

Notes:
1: The script automatically detects the version
of the input binary model.

 

2: mdlops by default DOES extract animations and DOES NOT
convert skin to trimesh.

 

3: The script automatically detects the type
of model.

 

4: For binary models you must have the .MDL and .MDX
in the same directory

 

5: For importing models that have supermodels, the super model or
the original model must be in the same directory as
model being imported.

 

6: Running uncompiled version requires perl 5.12+ and, on Windows, will
use the following CPAN packages when available:
Win32API::File, Win32::Autoglob

 

 

 

 

-------------------
-->--
-------------------

Import/export usage:
1) In a command prompt: perl mdlops.pl OR double click mdlops.exe
2) click 'select file'
3) browse to directory that has your .MDL file.
Select the .MDL and click 'open'
4) To quickly convert the model click 'Read and write model'
NOTE: The script will automatically detect the model type.
5) If you started with a binary file (ex. model.mdl) then the
resulting ascii model will be model-ascii.mdl
6) If you started with an ascii file (ex. model.mdl) then the
resulting binary model will be model-bin.mdl
7) The 'view data' button will let you view the raw data for
a model loaded from binary source. This does not work
with models loaded from ascii source.

 

NOTE: reading in ascii files that have models with lots of
polygons will be slow! You can watch the progress
in the command prompt window.

 

NOTE: Running uncompiled version requires perl 5.12+ and, on Windows, the
following CPAN packages (and all required for Command Line):
JSON, Win32::FileOp, Tk, Tk::Tree, Tk::Frame
To get Unicode path support on Windows, you need a customized version
of Win32::FileOp.

 

Renamer usage:
1) start mdlops
2) click 'select file'
3) browse to directory that has your .MDL file.
Select the .MDL and click 'open'
4) click on 'read model'
5) click on 'renamer'
6) type in a new texture name in the "New name" box
7) click on the mesh that needs its texture renamed
8) click "change name"
9) when you are done changing names, click "write model"
The model will be written to the same directory as
the original with the name model-rbin.mdl and
model-rbin.mdx

 

Replacer usage:
see the included 'replacer_tutorial.txt'

 

 

 

 

-------------------------------------------
-->--
-------------------------------------------

For those of you familiar with texturing the bad news is the
way the polygon vertices and texture vertices are stored
it is not possible for a single polygon vertex to have
multiple texture vertices.

 

For example, if you had a simple cube where all the polygon
vertices were welded you would have 8 polygon vertices.
So when you texture your cube you would only have 8 texture
map vertices to work with.

 

If you wanted your UV map to look like 6 separate squares
you would have to split the polygon vertices, but weld the
polygon vertices for each side. This would give you 24
polygon vertices and 24 texture vertices.

 

If you wanted to have separate polygons you would have
36 polygon vertices and 36 texture vertices.

 

If you use the 'validate vertex data' option, mdlops will do all
the work to make your model come out correctly.

 

 

 

 

--------------------------------------
-->--
--------------------------------------

To get models out of kotor files:
Kotor Tool
http://deadlystream.com/forum/files/file/280-kotor-tool/
OR
Xoreos Tools
https://github.com/xoreos/xoreos-tools

 

To edit the models
GMax (it is free, windows-only)
http://www.discreet.com/products/gmax/
OR
Blender (it is free, cross-platform)
https://www.blender.org

 

To get models in and out of Gmax, Max, or Blender
KOTORMax (to get models in and out of Gmax or Max)
Kotorblender (to get models in and out of Blender)

 

 

 

 

----------------------------------
-->--
----------------------------------
This script may only be hosted from sites that do not claim
ownership of files they host. In other words, any site that
claims "All files submitted to this site become property of
the site owner" can not host this script.

 

You are free to host this script from your website as long
as the distribution contains only the files listed below.

 

You are free to submit this script to any public download
site as long as the distribution contains all the files
listed below.

 

GPL.txt
icon.bmp
KotOR Tutorial.txt
mdlops.exe
mdlops.pl
MDLOpsM.pm
Quick_tutorial.txt
readme_mdlops_1-0.txt
replacer_tutorial.txt

 

I also ask that if you do host or submit this script to a
site send me an e-mail to let me know. My e-mail address
is at the top of this file.

 

 

Click here to download this file

[VarsityPuppet 1,085]

I simply can't like this enough!!!

[1Leonard 134]

This is great guys, it's amazing that the kotor series gets new and updated tools more than a decade after release. Thank you both.

[redrob41 86]

 I just used this last night on my mullet man head model, and it worked beautifully, so I've uploaded a new version of that mod. Thanks a ton.

[JCarter426 1,215]

That's great work, guys. I can't understate the amount of mod ideas I've given up on because of smoothing issues, or the amount of time I spent looking through models in a hex editor to see if I could figure out the issue, to no avail. So this is a real boon to the community.

 

However... yes, of course there's a however...

 

However, I have a bug to report. I just edited a body model, and while the smoothing was retained very nicely, I noticed that the model's shadow is now gone, as you can see below.

 

 

There's the head's shadow, but the body's shadow is nowhere to be seen. Converting the model with version 0.6 does not result in this problem, but of course I lose the smoothing that way.

 

I've yet to determine if I can get around this using Taina's Replacer, but even so it would be nice to have it all in one package. (EDIT: Taina's Replacer did NOT work. Same problem.)

 

Since 0.6 didn't lose the shadow, I'm hoping it's just something missing from 0.7 that can be added back in.

[VarsityPuppet 1,085]

I'd have to consult with Fair Strides.

 

I don't think any of the shadow flags were touched, but it's possible that something was changed when he was cleaning up code? Hard to say.

[Fair Strides 509]

JCarter426, could you perhaps post the first 20 or so lines of the skin Torso part of the ascii? Just the part up to the verts section. I'm thinking you might try changing the Shadow flag on the ascii if it's 0, but if that doesn't fix it, I could take a look at the source code again...

[JCarter426 1,215]

Sure:

node skin Torso

  parent JC_HandA

  position 0.0 0.0 1.11

  orientation 1.0 0.0 0.0 0.0

  wirecolor 1.0 1.0 1.0

  alpha 1.0

  ambient 1.0 1.0 1.0

  diffuse 1.0 1.0 1.0

  specular 0.0 0.0 0.0

  shininess 0.0

  bitmap JC_HandA

  verts 564

[Fair Strides 509]

Trying adding the "shadow 1" after the the alpha, and if that doesn't work, try "Shadow 1" instead.

[JCarter426 1,215]

Nope, neither of those worked.

 

And I just tested again to be sure... converting with MDLOps 0.6 does produce shadows without editing the ASCII at all.

[VarsityPuppet 1,085]

Could you actually just post the entire ascii for the model?

[JCarter426 1,215]

All right... it's still a WIP though, so don't judge.

 

https://www.dropbox.com/s/nmd8lzq2hruh6w1/JC_HandA-ascii.mdl?dl=0

[Fair Strides 509]

All right... it's still a WIP though, so don't judge.

 

https://www.dropbox.com/s/nmd8lzq2hruh6w1/JC_HandA-ascii.mdl?dl=0

If you've done a retexture, could we see that and the original .mdl and .mdx files? (Can't compile it right now)

[JCarter426 1,215]

Hadn't done the retexture yet... but the original model was PFBAM (in K2). The texture mapping is more or less the same, just the feet and the neck different.

[VarsityPuppet 1,085]

Checked the ascii. The shadow flag is set to 1 on all of the trimeshes it makes sense to render a shadow for.

 

See, the bones are used to render shadows instead of the skin meshes. Not sure why really, but it might have to do with being being easier to compute.

 

I wonder if it's possible that messing with the vertex weights on the bone meshes could impact the way shadows are rendered..? ¯\_(ツ)_/¯

[JCarter426 1,215]

Yeah, I noticed the shadows were coming from the bones. I was wondering about that. Like I said, though, I was still getting shadows with 0.6. And I've done it before without ever losing shadows. So I don't think it's a result of my messing... or not exclusively my messing, at any rate.

 

I hadn't touched the bones when I tried running it through Taina's Replacer, so I could try that later. Might not have access to the files for the rest of the night though.

[Fair Strides 509]

We've fixed the issue (either the face normals or plane distances) and uploaded a fixed version (now it's MDLOps 0.7 Alpha 2).

[DarthParametric 3,785]

bones are used to render shadows instead of the skin meshes. Not sure why really, but it might have to do with being being easier to compute.

Yes, it's to reduce overhead for the Xbox1 the game was designed to run on. Still, even on modern PCs the game will sometimes crash if you try to force shadows on a mesh with too many polys.

[VarsityPuppet 1,085]

Yes, it's to reduce overhead for the Xbox1 the game was designed to run on. Still, even on modern PCs the game will sometimes crash if you try to force shadows on a mesh with too many polys.

Yikes! Yeah, that totally makes sense. I think I remember trying that on some skin meshes back in the day, with varying results.

 

Anyways, Fair Strides fixed it and it seems that face normals are part of what help determine shadows... ¯\_(ツ)_/¯

 

Also, too bad we never have figured out how to get the models to work on the xbox :/

[Fair Strides 509]

Yikes! Yeah, that totally makes sense. I think I remember trying that on some skin meshes back in the day, with varying results.

 

Anyways, Fair Strides fixed it and it seems that face normals are part of what help determine shadows... ¯\_(ツ)_/¯

 

Also, too bad we never have figured out how to get the models to work on the xbox :/

 

Makes even more sense considering how blocky the "bones" are would help with giving the shadows shape while conserving polies.

 

Also, it was either the Face Normals (which I doubt) or the Plane Distance on the faces, which it looks like I might have zeroed out by accident.

 

As for the XBox models, did anyone ever do a filedump and check the .mdl files? I know that K1 and K2 both have different seemingly-random numbers taking up the first twelve bytes of a .mdl file, so it could be that these numbers differ between the PC and Xbox versions...

[DarthParametric 3,785]

I think I still have the Xbox discs for TSL somewhere....

[Dastardly 31]

I got both xbox game files on my pc.

[DarthParametric 3,785]

Didn't Team Gizka pull some stuff that was only in the Xbox version for use in TSLRP? Did the TSLRCM team do anything in that regard?

[VarsityPuppet 1,085]

Didn't Team Gizka pull some stuff that was only in the Xbox version for use in TSLRP? Did the TSLRCM team do anything in that regard?

HK Factory, I believe... and M4-78 I think as well..?

[JCarter426 1,215]

We've fixed the issue (either the face normals or plane distances) and uploaded a fixed version (now it's MDLOps 0.7 Alpha 2).

 

Ah, that's great! Thanks for the update.