Malkior 476 Posted November 11, 2012 I've been curious ever since I found the "bumpmaptexture" string in the text field on the Hutt texture, so I finally opened the C_Hutt01b file in Photoshop and here's what I saw. Now, I'm not too familiar with the NWN engine's complexities, but that looks like a proper Normal Map! This leads into my overall goal of possibly adding bumps to more texture geometry if someone could decipher whatever code is in the texture text under the preview window of the KotorTool. and then possibly try a different normal map and see if it works. Perhaps when they add a different bumpmap to a different texture they can reference it using that code. 1 Quote Share this post Link to post Share on other sites
InSidious 237 Posted November 13, 2012 Fairly substantial amounts of work were done on this over at LF. Nothing has ever come of it. Quote Share this post Link to post Share on other sites
Malkior 476 Posted November 13, 2012 Fairly substantial amounts of work were done on this over at LF. Nothing has ever come of it. Was it too unfeasible to implement or was it stalled due to a lack of interest? Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted November 13, 2012 From my own experiments - even vanilla tpc's such as Rakghouls that use these - don't add geometry but create more complex (more than just alpha channel) shading effects. The difference is barely noticeable from my point of view. They probably ripped out support for this to make it X-box compatible, seeing that the X-box is way less powerful. However, I'm going to experiment more and see what I can find. Quote Share this post Link to post Share on other sites
Dark_Ansem 12 Posted December 3, 2012 From my own experiments - even vanilla tpc's such as Rakghouls that use these - don't add geometry but create more complex (more than just alpha channel) shading effects. The difference is barely noticeable from my point of view. They probably ripped out support for this to make it X-box compatible, seeing that the X-box is way less powerful.However, I'm going to experiment more and see what I can find. And if anything relevant is out I will make bumpmaps. and specular ones as well. Quote Share this post Link to post Share on other sites
Nsinger998 17 Posted January 6, 2013 Didn't xarwars add bump maps to his retextures? Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted January 6, 2013 i heard something about that actually, was he also the guy who used the dds format instead of tga? that might help actually considering there are various options when saving as dds in ps, one of them, iirc, had to do with normal maps, or in the kotor engine, bumpmaps. we also have some information on the wiki about it though more info is being added. Quote Share this post Link to post Share on other sites
Sith Holocron 2,479 Posted January 6, 2013 Was he also the guy who used the DDS format instead of TGA? Yes, he was. However, most folks requested he change his files back to TGA. I think he used the DDS format for space-saving reasons, IIRC. Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted November 27, 2014 Sorry, to resurrect this thread, but I felt this is a better place to put what I found than into a new thread. Where to begin? Well, I have found out something for sure, we can make bump maps, they have to be in grayscale, that means not rgb mode but grayscale mode in photoshop, with no alpha channel And as for dimensions, they must be a power of 2 squared, (2x2, 4x4, 8x8, ... , 256x256, 512x512, 1024x1024, 2048x2048... etc.) So now we have a texture that is considered a valid bumpmap, it won't crash your game, and will work when called/defined as a bumpmap. In its txi there are some fields we can now fill namely: isbumpmap 1 // Mandatory, without this it won't be treated as a bumpmap bumpmapscaling 1 // this is for how high or low the bumps should be (0.5, 0.75, 1, 2.25 etc.) The following are some commands, from Never Winter Nights, that are in the kotor/tsl .exe but I haven't tested out in game yet // Specify what kind of bumpmapping this bump map will do: specular, diffuse, or both. These values are ignored if this texture is not a bump map. isdiffusebumpmap 1 isspecularbumpmap 1 // Specify a color for specular bumpmap highlights (RGB). This value is modulated with the bumpmapping light's color to produce the final highlight color. 1=255, 0=0 specularcolor 0.5 0.5 0.8 So, here comes the theortical part. How does kotor do bump maps? in two ways, A from a grayscale texture then renders a tangent normal map, B from a pre rendered tangent-space normal map. Almost all the bumpmaps in kotor are prerendered, they come out in rgb when we extract the with ktool, but they also always crash the tpc viewer. It seems this is the more efficient way to do it, but it is not something currently we can do, (we can keep experimenting) but what was needed for efficiency in 2003, is a luxury, I'd say we can afford to take in 2014. in the end the gpu get's the same thing from either method, (according to glintercept). Lastly here is the experiment I ran, let's see what we can do now with bump maps! my experiment: 3 Quote Share this post Link to post Share on other sites
DeadMan 103 Posted November 27, 2014 OMG! YOU DID IT!!! That's an amazing find, DS. I think I can put it to good use;) Quote Share this post Link to post Share on other sites
Malkior 476 Posted November 27, 2014 You do realize many of our dreams can now be reality.. Basically Kotor's one big weakness graphically is how flat it looks. That is now quite likely a thing of the past. I've a million ideas, but first I think fixing most of the clothes is in order. Also, you know, Sion... 1 Quote Share this post Link to post Share on other sites
MrPhil 58 Posted November 27, 2014 Can't wait to see what you'll come-up with, Malkior! Quote Share this post Link to post Share on other sites
Fair Strides 509 Posted November 27, 2014 You do realize many of our dreams can now be reality.. Basically Kotor's one big weakness graphically is how flat it looks. That is now quite likely a thing of the past. I've a million ideas, but first I think fixing most of the clothes is in order. Also, you know, Sion... And with the new cubemaps, you might replace CM_Baremetal in your Realistic Aliens and Eyes mod(s)... Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted November 29, 2014 The bad/good news, The good news: the games both use bump maps the exact same way, The bad news: Not all models accept the bump maps, For example, I couldn't get any sort of custom bump map to work on any models that don't support it (space suit, robes, Nihilus, Mandalor,e etc.) Some that do for sure are the Twi'lek male head, the Quarren, the Rakghouls, etc., in TSL. In KotOR1, Hutts do too (for some reason Obsidian cut this for Vogga) , I think this is something we need to edit models for, but that is far from my area of expertise. Help me [unknown super modder], you're my only hope! Quote Share this post Link to post Share on other sites
VarsityPuppet 1,085 Posted November 29, 2014 So, it is possible to get envmaps to work on those models. (I've done it before). Can't remember what I did, but I think it's best to cover as many bases as possible. Txi, appearance.2da, there's a property in the .mdl you need to make sure is set.... etc Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted November 29, 2014 So, it is possible to get envmaps to work on those models. (I've done it before). Can't remember what I did, but I think it's best to cover as many bases as possible. Txi, appearance.2da, there's a property in the .mdl you need to make sure is set.... etc well i can crossout txi & appearance 2da, so they're either hard coded, or they're in the model, i really doubt them being hard coded, in all the string-dumping i have done on swkotor2.exe i have only seen bumpmaps mentioned in the txi code. Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted November 29, 2014 after doing some thinking i feel almost as if it's in the mdl of the file, specifically it has to do with materials, i think the game loads the model , the model gives it a list of textures, with various parameters, one of them having to do with bump mapping, for example the quarren, is all one model right? but we can only apply a functional bump map to his head. a visible example (not my screen btw, bottom right on the wall) notice how only the one texture/material has a bump map?, also when i tried earlier to bump map the dantooine cliff, nothing happened, but the water definitely has a bump map going on. iirc they are on the same model. Quote Share this post Link to post Share on other sites
redrob41 86 Posted November 30, 2014 after doing some thinking i feel almost as if it's in the mdl of the file, specifically it has to do with materials, i think the game loads the model , the model gives it a list of textures, with various parameters, one of them having to do with bump mapping, for example the quarren, is all one model right? but we can only apply a functional bump map to his head. That makes sense to me. Once upon a time I modified the male Twilek head mdl (so that it doesn't have the neck sac and added mask and goggle hooks). Taina's replacer method never worked with that model, so I used VarsityPuppet's HeadFixer method. While I did get the head to work in game, it lost two important qualities: the edge smoothing groups was lost, and apparently the connection to the bump map was also lost. I suspect that it has more to do with exporting from 3DS Max rather than the HeadFixer, and it probably has more to do with the mdx not the mdl. I'd guess that since we need the NWN plugin to import Kotor models into Max, that we might also need some kind of plugin to export the model with ALL the correct info intact. Or maybe the problem is with MDLOps converting the ascii file into binary. Quote Share this post Link to post Share on other sites
VarsityPuppet 1,085 Posted November 30, 2014 That makes sense to me. Once upon a time I moified the male Twilek head mdl (so that it doesn't have the neck sac and added mask and goggle hooks). Taina's replacer method never worked with that model, so I used VarsityPuppet's HeadFixer method. While I did get the head to work in game, it lost two important qualities: the edge smoothing groups was lost, and apparently the connection to the bump map was also lost. I suspect that it has more to do with exporting from 3DS Max rather than the HeadFixer, and it probably has more to do with the mdx not the mdl. I'd guess that since we need the NWN plugin to import Kotor models into Max, that we might also need some kind of plugin to export the model with ALL the correct info intact. Or maybe the problem is with MDLOps converting the ascii file into binary. I am offended! Haha, jk. I think it may have to do with the ascii model. I exported my armored robes and those seemed to retain specular mapping (on the armor part at least, since that was the part of the model that originally used the specular mapping) Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted November 30, 2014 so i have sent MagnusII a pm on LF, here is what he had to say: This is what I know of the trimesh header structure of kotor's MDL files: public class TriMeshHeader { public static int KotorI_Size = 332; public static int KotorII_Size = 340; #region Private Members private int _KotorVersion; /*0*/ private UInt32 _FunctionPointer0; /* 4216656 K1, 4216592 K1 SkinMesh, 4216640 K1 DanglyMesh * 4216880 K2, 4216816 K2 SkinMesh, 4216864 K2 DanglyMesh * */ private UInt32 _FunctionPointer1; /* 4216672 K1, 4216608 K1 SkinMesh, 4216624 K1 DanglyMesh * 4216896 K2, 4216832 K2 SkinMesh, 4216848 K2 DanglyMesh * */ private UInt32 _OffsetToFaces; private UInt32 _FacesCount; private UInt32 _DuplicateFacesCount; private KotorVertex _BoxMin; private KotorVertex _BoxMax; private float _Radius; private KotorVertex _AveragePoint; /*60*/ private float[] _DiffuseColor = new float[3]; private float[] _AmbientColor = new float[3]; private UInt32 _Shininess_Tentative; /* 0 (mostly), 1, 2, 3, 4, 5, 7, 8, 13 * Blend Factor, SamplerStageStates, TextureOperation? */ private byte[] _Texture1Name = new byte[32]; /*120*/ private byte[] _Texture2Name = new byte[32]; private byte[] _Texture3Name = new byte[12]; private byte[] _Texture4Name = new byte[12]; private UInt32 _OffsetToVerticesIndexesCounts; /*180*/ private UInt32 _VerticesIndexesCountsCount; // = 1 private UInt32 _DuplicateVerticesIndexesCountsCount; private UInt32 _OffsetToVerticesOffsets; private UInt32 _VerticesOffsetsCount; // = 1 private UInt32 _DuplicateVerticesOffsetsCount; /*200*/ private UInt32 _OffsetToTrimeshInvertedCountersArray; private UInt32 _TrimeshInvertedCountersArrayItemsCount; /* * = 1, always one UInt32 * sequence goes like this: first * trimesh node is 98, then: * 98..00, 100 * 199..101, 200 * 399..? */ private UInt32 _DuplicateTrimeshInvertedCountersArrayItemsCount; private Int32[] _Unknown1 = new Int32[3]; // -1, -1, 0 /*224*/ private byte _SaberUnknown1; // 3 for non-saber private byte _SaberUnknown2; // 0 for non-saber private byte _SaberUnknown3; // 0 for non-saber private byte _SaberUnknown4; // 0 for non-saber private byte _SaberUnknown5; // 0 for non-saber private byte _SaberUnknown6; // 0 for non-saber private byte _SaberUnknown7; // 0 for non-saber private byte _SaberUnknown8; // 0 for non-saber, 17 for sabers private Int32 _SparkleTentative; /* 0 for most, 1 for water and clouds and * one piece of the GUI * Maybe some sparkle or reflex effects, * or a specular lighting modifier? */ private Rotation _UnknownRotation = null; private Int32 _MDXDataSize; /* mostly 24, 32, 40, 76 with a smattering of other values * if textures = 0 almost always 24, except 15 models with -1 and 23 with 60 * if textures = 1, almost always 32, with several 64 and 68, and a bunch of 100 and 0 * if textures = 2, it's either 40 (mostly) or 76 */ private UInt32 _MDXDataBitmap; /* which data is present in MDX * according to MDXDatabits */ /*260*/ private Int32 _OffsetToVerticesInMDX; // = 0 for most, -1 for lightsabers or when verticescount = 0 private Int32 _OffsetToNormalsInMDX; // = 12 private Int32 _OffsetToUnknownInMDX; // = -1, never present private Int32 _OffsetToUVsInMDX; // = 24 if present private Int32 _OffsetToUV2sInMDX; /* -1, 24, 32 * if mdxdatasize = 24 then it's always -1 * if mdxdatasize = 32 it's almost always -1 except for 14 models where it's 24 * if mdxdatasize = 40 or 76 then it's always 32 */ private Int32 _OffsetToUV3sInMDX; // = -1, never present private Int32 _OffsetToUV4sInMDX; // = -1, never present private Int32 _OffsetToUnknownStructInMDX; /* -1, 24, 32, 40 * if mdxdatasize = 24 or 32 or 40 then it's always -1 * if mdxdatasize = 76 then it's always 40 * this is basically whatever info is in the MDX * after the last texture UV coordinates and * before the bones. It is 36 bytes long. * the last three of these values seem to be some sort of * "weighted normals"... they definitely have something * to do with normals as at least one of these triplets * was equal to the normal in c_hutt --> eye01, and the others * have similar values */ private Int32 _OffsetToUnusedMDXStructure1; // -1 private Int32 _OffsetToUnusedMDXStructure2; // -1 /*300*/ private Int32 _OffsetToUnusedMDXStructure3; // -1 private UInt16 _VerticesCount; private UInt16 _TexturesCount; private UInt16 _HasLightmap; //0, 1 private UInt16 _Shadow; // 0, 256 private UInt16 _Render; // 0, 256 private UInt16 _Unknown2; // ?? private float _TotalArea; // sum of faces' areas /*320*/ private UInt32 _Unknown4; // = 0 private UInt32 _Kotor2Unknown1; private UInt32 _Kotor2Unknown2; private UInt32 _OffsetToMDXData; private UInt32 _OffsetToVertices As you can see from the comments in the code, there are several fields I never managed to fully decode. What I do know is that theoretically the thing supports up to 4 textures applied to the models. First one is the texture proper, second is the lightmap where present. I don't remember all that well since I've not touched this stuff in years but if I'm not mistaken I never found anything in either Kotor I or II having more than two textures applied to it. A potential candidate for some sort of bump mapping info would be _OffsetToUnknownStructInMDX, a pointer to some kind of small data structure which I could never fully decipher. i'd wager the four textures are, the base, the lightmap, the bumpmap and the environment map. 1 Quote Share this post Link to post Share on other sites
redrob41 86 Posted December 3, 2014 here's something interesting. I opened twilek_m01b.tpc and twilek_m01b.tga with a hex editor. The tpc had a bunch of empty bytes (is that correct term? they all say 00) for the first eight lines. Then at the end of the file came the isbumpmap 1 bumpmapscaling 1 xbox_downsample 128. The tga (converted by MDLOPs) only had one line of mostly 00 at the beginning, while at the end it had isbumpmap etc... then 76 hex digits, then it repeated isbumpmap etc... The result is that the tga had 18 extra hex digits over the tcp version. Then, I opened the tga with Photoshop, and saved it as a copy tga. When I checked it with a hex editor, this version had 87534 fewer characters than the original version. I have no idea what any of that means . If you need some pixels or vertices pushed around, I'm your guy. If there is eldritch languages, call Robert Langdon to figure out the DaVinci Code I honestly wish I was more help 1 Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted December 3, 2014 here's something interesting. I opened twilek_m01b.tpc and twilek_m01b.tga with a hex editor. The tpc had a bunch of empty bytes (is that correct term? they all say 00) for the first eight lines. Then at the end of the file came the isbumpmap 1 bumpmapscaling 1 xbox_downsample 128. The tga (converted by MDLOPs) only had one line of mostly 00 at the beginning, while at the end it had isbumpmap etc... then 76 hex digits, then it repeated isbumpmap etc... The result is that the tga had 18 extra hex digits over the tcp version. Then, I opened the tga with Photoshop, and saved it as a copy tga. When I checked it with a hex editor, this version had 87534 fewer characters than the original version. I have no idea what any of that means . If you need some pixels or vertices pushed around, I'm your guy. If there is eldritch languages, call Robert Langdon to figure out the DaVinci Code I honestly wish I was more help actually your post over at LF sent me in the right direction, with the greyscale etc. so really you've been the biggest help yet you got me going on this, indirectly, but still. Quote Share this post Link to post Share on other sites
LiliArch 115 Posted December 3, 2014 i'd wager the four textures are, the base, the lightmap, the bumpmap and the environment map.I believe you're right. See, I tried to make a set of Sith Armors that have a different cube map for every color variant for testing purposes, but they all ended up using CM_BareMetal, even after I set the "envmaptexture" column (or whatever it was, something like that anyway) to DEFAULT for that appearance row. So it looks like the Sith soldier model has that info stored in somewhere. Also, I tried to replace the water texture on Dantooine (LDA_water03) with another one, and dropped the changed file into the Override without a .txi file. It appeared just like it was if it had a .txi file that has all the same .txi info than the LDA_water03.tpc has. Quote Share this post Link to post Share on other sites
redrob41 86 Posted December 3, 2014 so i have sent MagnusII a pm on LF, here is what he had to say: public class TriMeshHeader { public static int KotorI_Size = 332; public static int KotorII_Size = 340; #region Private Members private int _KotorVersion; /*0*/ private UInt32 _FunctionPointer0; /* 4216656 K1, 4216592 K1 SkinMesh, 4216640 K1 DanglyMesh * 4216880 K2, 4216816 K2 SkinMesh, 4216864 K2 DanglyMesh * */ private UInt32 _FunctionPointer1; /* 4216672 K1, 4216608 K1 SkinMesh, 4216624 K1 DanglyMesh * 4216896 K2, 4216832 K2 SkinMesh, 4216848 K2 DanglyMesh * */ private UInt32 _OffsetToFaces; private UInt32 _FacesCount; private UInt32 _DuplicateFacesCount; private KotorVertex _BoxMin; private KotorVertex _BoxMax; private float _Radius; private KotorVertex _AveragePoint; /*60*/ private float[] _DiffuseColor = new float[3]; private float[] _AmbientColor = new float[3]; private UInt32 _Shininess_Tentative; /* 0 (mostly), 1, 2, 3, 4, 5, 7, 8, 13 * Blend Factor, SamplerStageStates, TextureOperation? */ private byte[] _Texture1Name = new byte[32]; /*120*/ private byte[] _Texture2Name = new byte[32]; private byte[] _Texture3Name = new byte[12]; private byte[] _Texture4Name = new byte[12]; private UInt32 _OffsetToVerticesIndexesCounts; /*180*/ private UInt32 _VerticesIndexesCountsCount; // = 1 private UInt32 _DuplicateVerticesIndexesCountsCount; private UInt32 _OffsetToVerticesOffsets; private UInt32 _VerticesOffsetsCount; // = 1 private UInt32 _DuplicateVerticesOffsetsCount; /*200*/ private UInt32 _OffsetToTrimeshInvertedCountersArray; private UInt32 _TrimeshInvertedCountersArrayItemsCount; /* * = 1, always one UInt32 * sequence goes like this: first * trimesh node is 98, then: * 98..00, 100 * 199..101, 200 * 399..? */ private UInt32 _DuplicateTrimeshInvertedCountersArrayItemsCount; private Int32[] _Unknown1 = new Int32[3]; // -1, -1, 0 /*224*/ private byte _SaberUnknown1; // 3 for non-saber private byte _SaberUnknown2; // 0 for non-saber private byte _SaberUnknown3; // 0 for non-saber private byte _SaberUnknown4; // 0 for non-saber private byte _SaberUnknown5; // 0 for non-saber private byte _SaberUnknown6; // 0 for non-saber private byte _SaberUnknown7; // 0 for non-saber private byte _SaberUnknown8; // 0 for non-saber, 17 for sabers private Int32 _SparkleTentative; /* 0 for most, 1 for water and clouds and * one piece of the GUI * Maybe some sparkle or reflex effects, * or a specular lighting modifier? */ private Rotation _UnknownRotation = null; private Int32 _MDXDataSize; /* mostly 24, 32, 40, 76 with a smattering of other values * if textures = 0 almost always 24, except 15 models with -1 and 23 with 60 * if textures = 1, almost always 32, with several 64 and 68, and a bunch of 100 and 0 * if textures = 2, it's either 40 (mostly) or 76 */ private UInt32 _MDXDataBitmap; /* which data is present in MDX * according to MDXDatabits */ /*260*/ private Int32 _OffsetToVerticesInMDX; // = 0 for most, -1 for lightsabers or when verticescount = 0 private Int32 _OffsetToNormalsInMDX; // = 12 private Int32 _OffsetToUnknownInMDX; // = -1, never present private Int32 _OffsetToUVsInMDX; // = 24 if present private Int32 _OffsetToUV2sInMDX; /* -1, 24, 32 * if mdxdatasize = 24 then it's always -1 * if mdxdatasize = 32 it's almost always -1 except for 14 models where it's 24 * if mdxdatasize = 40 or 76 then it's always 32 */ private Int32 _OffsetToUV3sInMDX; // = -1, never present private Int32 _OffsetToUV4sInMDX; // = -1, never present private Int32 _OffsetToUnknownStructInMDX; /* -1, 24, 32, 40 * if mdxdatasize = 24 or 32 or 40 then it's always -1 * if mdxdatasize = 76 then it's always 40 * this is basically whatever info is in the MDX * after the last texture UV coordinates and * before the bones. It is 36 bytes long. * the last three of these values seem to be some sort of * "weighted normals"... they definitely have something * to do with normals as at least one of these triplets * was equal to the normal in c_hutt --> eye01, and the others * have similar values */ private Int32 _OffsetToUnusedMDXStructure1; // -1 private Int32 _OffsetToUnusedMDXStructure2; // -1 /*300*/ private Int32 _OffsetToUnusedMDXStructure3; // -1 private UInt16 _VerticesCount; private UInt16 _TexturesCount; private UInt16 _HasLightmap; //0, 1 private UInt16 _Shadow; // 0, 256 private UInt16 _Render; // 0, 256 private UInt16 _Unknown2; // ?? private float _TotalArea; // sum of faces' areas /*320*/ private UInt32 _Unknown4; // = 0 private UInt32 _Kotor2Unknown1; private UInt32 _Kotor2Unknown2; private UInt32 _OffsetToMDXData; private UInt32 _OffsetToVertices i'd wager the four textures are, the base, the lightmap, the bumpmap and the environment map. Here are some tables from cchargin's site where he describes the binary .mdl and .mdx structurehttp://home.comcast.net/~cchargin/kotor/mdl_info.html Hopefully it will be useful to someone with more experience than I to make sense of it combined with MagnusII's info, so that the unknowns can be filled in. Quote Share this post Link to post Share on other sites
Darth_Sapiens 90 Posted December 3, 2014 Here are some tables from cchargin's site where he describes the binary .mdl and .mdx structure http://home.comcast.net/~cchargin/kotor/mdl_info.html Hopefully it will be useful to someone with more experience than I to make sense of it combined with MagnusII's info, so that the unknowns can be filled in. iirc you brought up the nwn binary format in the chat, it directly commented on bump mapping, we can do this guys! Quote Share this post Link to post Share on other sites
