DrMcCoy

Sure, your choice

If you do use the xoreos-tools, I'd be happy to hear any feedback you can offer.

...But that's about a completely different issue?

Well, my KotOR2 has several main menu GUI files: mainmenu10x7_p.gui mainmenu12x9_p.gui mainmenu16x12_p.gui mainmenu8x6_p.gui mainmenu_p.gui Which of them are loaded depends on your resolution. There's also mainmenu_x.gui and mainmenu8x6_x.gui, which, I guess, might be for the Xbox versions.

Well, for binary 2DA, the ones KotOR use, the number of colums has a hard limit of 2^32, because the count value is given as an uint32_t. For symmetry's sake, I bet the row counter also uses a 32-bit integer in the binary. I'm too lazy to check though. If they're sloppy with their arithmetics, you might want to limit yourself to 2^31, though. That said, there's a more important limit: the offset to the cell data is given by an uint16_t. So your 2DA file will get issues when it reaches 65535 of raw cell data. This is not necessarily the file size, because that is excluding the row counters and column headers, and the cell data offset table. Also, the same data in multiple cells is only saved once. 64kb of unique cell data is difficult to reach, though. My original, unmodded appearance.2da for KotOR2, which is 148884 bytes big (i.e. ~ 146kb), only reaches a max offset value of 19222 (~ 18kb). Yes, 129662 bytes, 87% of the file, is taken up by the row counter, colum headers and the cell data offset table. That said, if the file is actually exceedingly large, and has a lot of unique cell values, you certainly can reach that, if you tried. If the program you use to write the 2DA files doesn't check for that, it can theoretically overflow these offset. But: these broken offsets will most probably not point to the beginning of a valid cell, but in the middle of one. So you won't get duplicated values, but bogus ones. (It is probably possible to intentional rig up a file to do surprising things, though.) EDIT: Oh, but if you're using ASCII 2DA files (does KotOR still read those?), this all is of course not applicable. Unless the first converts them into binary ones in memory. Which is possible, but I don't think it's likely.

(Just as a heads up: if you saved after the change happened, the change might have propagated into the save file.)

Yes. The menu code in the executable is quite hard-coded. The game basically knows how the menus are laid out, i.e. which buttons and other widgets exist. It creates the general layout, and then only looks into the GUI GFF for details, like color and size.

Oh, and while it's not really on-topic for this thread, I did an interview with "The Linux Gamer" about xoreos: https://www.youtube.com/watch?v=kDEmpUJviRM It's more a general overview of what xoreos is, the current state, etc. In my badly accented, mumbling English.

Yeah, Torlack's docs (and Skywing's NWN2 JIT NWScript thing) provided the base for my understanding of NWScript.

That's quite likely, yes.

Yes, that would be the decompiler step I talk about. Basically, iterate over the blocks, check what control structures they belong to (which is already detected) and generate NWScript source code from that. That's something that's planned and should be reasonably possible. I did already experiment with that without the control structures: https://gist.github.com/DrMcCoy/6c3ce1131707bd97ebef (though the code that did that is not public, as it's a big hack). Now think while-loops and if-conditions linking the code segments instead of goto sprinkled everywhere, and this would be the output you get. Somebody (or I, in the future) will "only" have to connect that codegen with the control structures and clean it up. As you can see there, there are a lot of unnecessary duplicate temporary variables. The next step then is to eliminate those, by simplifying and unifying expressions. Especially for conditions in ifs and whiles, that would be interesting. And currently, structs and vectors aren't recognized as such (instead, you get their separate members as single variables). Again, if somebody is interested in taking up these tasks, please contact me. I can say, though, that right from the get-go, my decompiler would fail on some scripts too. Namely, scripts with recursion. I.e. functions that call themselves. My approach of analyzing the stack (which is already done) fails for these scripts, because it needs to finish analysation of a function before continuing analyzing its caller [1]. So when I have a function that calls itself (even over multiple levels), I'm biting my own tail, so to speak. ncsdis currently already recognizes that and stops deeper analysis of these scripts. So you won't see function signature comments (like that "; void main()" in your example there) in these scripts, only the bare disassembly. There's not really anything I can do about it as such, as that's similar to the halting problem. There might be some clever approach I could switch to then that works for certain cases (maybe even most that occur in exiting scripts?), but so far, I haven't found one. Again again, anybody with a lead and/or interesting in implementing anything there, please contact me. [1] Because I need to know how the function manipulates the stack. A function pops off its arguments at the end.

In case anybody wants to try this out, but couldn't compile xoreos-tools from sources: xoreos and xoreos-tools 0.0.4 has been released, which includes binaries of ncsdis for GNU/Linux, Windows and Mac OS X. It is still a command line tool, though, without a GUI.

Over at xoreos.org I wrote a blog post about Disassembling NWScript Bytecode. This is basically a progress report of me writing an NWScript disassembler, which might be extended into a full-fledged decompiler. It might be of interest to people here.

Ah, yes, in case anybody didn't notice: I did end up writing an xml2tlk tool myself, that can recreate a TLK out of an XML file created by tlk2xml. It's able to write the versions 3.0 (NWN, NWN2, KotOR, KotOR2, The Witcher) and 4.0 (Jade Empire) of the TLK format. It does not currently support writing the GFF4'd versions 0.2 and 0.5 (Sonic Chronicles, Dragon Age: Origins, Dragon Age II) of the format, because those need a GFF4 writer first. I only did a cursory testing of the produced files, though, and some simple binary diffing between original and recreated file. They should work, but I can't give a 100% guarantee. If you do find cases that do not correctly work, I'd be very much interested in samples that break, and descriptions how and where, so that I can fix the issues.

Also, for some shameless self-promotion, Jade Empire is also a target for xoreos. Earlier this year, I figured out the general model format, at least the static geometry (I'm still not completely sure about material properties and shader parameters, and the animations). As far as I am aware, this hadn't been done at all previously. Since the, Farmboy0 did some more sleuthing on the model format, and has created a more mapped out description file for 010 Editor. I'm still looking for people to join the xoreos efforts. People who want to extend the tools in the xoreos-tools repository, or add further tools to it, so that this sub-project is more useful for modding purposes, are also more than welcome. EDIT: I went ahead and checked the Jade Empire binary template file for 010 Editor into the xoreos-docs repository, in the off-chance it might be useful to somebody. The model loader in xoreos itself can be found here.

Btw, this might be relevant to your interests: I've been writing an NCS disassembler the last few weeks, which can disassemble all NCS scripts found in Neverwinter Nights to Dragon Age II, including KotOR and KotOR2. Doesn't need an nwscript.nss either. It's assembly output looks like this. It also does some stack analysis and marks subroutines with a full signature, i.e. the numer of arguments and their types, and the return type (or types, in case of a subroutine returning a struct). It can also create a DOT graph description file, which can be used by Graphviz to draw a control flow graph, like this here, which makes scripts either to understand, I find. I've also added a simple control structure analysis, which makes it detect do-while and while loops, as well as if and if-else. An artificial example would be here. The next step is adding simple C-like code generation, to expand it into a proper decompiler. I.e. the result will be something close to the original NWScript source. Minus variable names, of course. And it'll be a lot more verbose at first. And I'm not yet handling structs and vectors as joined either. There's no binary release yet, but the source can be found in the xoreos-tools repository. Specifically, ncsdis.cpp and the nwscript/ directory. Anybody interested in expanded this is of course always welcome; please feel free to contact me. I also have an assembler and a compiler (to replace the old OpenKnights nwnnsscomp in all its different flavours) on my TODO list; that'd be up for grabs too.

Also: - mainmenu8x6_p.gui adds the "More Games" button, that opens a browser in Steam to the Aspyr website - optgraphics_p.gui adds an option to disable the Force Speed Effect - saveload_p.gui adds a toggle to disable/enable cloud saves

You mean the distortion based on the view position and angle? I don't think there's a way to get rid of that, because that's one of the reasons reflective environment maps exists at all. From what I've seen in the disassembly, all render paths enable that. It's done by calling glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_REFLECTION_MAP), and also for GL_T and GL_R. I.e. it auto-generates 3-dimensional texture coordinates according to a reflective model and based on the view angle. Oh, neat, so KotOR's TGA loader supports that by having the 6 sides one below the other. Very interesting. I assume the code depends on height / width == 6 there as well, and that would explain the crash Jorak Uln had when trying that with a square TGA. From what it looks like, the order is the same as with the TPC (which is slightly different from the order suggested by OpenGL tutorials, since KotOR's in-game coordinate system is rotated to fit a top-down viewpoint; a remnant from the NWN days). But are the TGA cube sides rotated as well? It's a bit difficult to see in those images. I've seen that the TPC loader rotates the sides in 90° steps (In order: 3, 1, 0, 2, 2, 0; each is the number of 90° steps clock-wise), and NWN seems to be similar with its cubemap files.

Just to clear it up a bit, you're using that square texture as a reflective environment map. But it's not a cubemap. A cubemap consists of 6 square sides that wrap around a cube. That can be used as a skybox, i.e. you're constructing a cube around your position and put the 6 textures on that sides, emulating the environment you're in. You can also use that as a reflective map. You're basically saying "this cube represents the environment as rendered from the perspective of the object", and then tell your graphics card to distort the texture according to the position and angle you view the object at. This fakes a reflective effect, and you can see an approximation of the object's surround reflected in the object. Likewise, you can use that texture as a refractive map. The idea is similar, only that you math on how to distort the texture is different. See this article for a more in-depth description how it works, the math behind it and how to do this in modern OpenGL (i.e. not quite how KotOR does it). You can also use a normal square texture to for this effect (like you did), but this is even more faked. Instead of having six sides that approximate the suroundings of your object, you'd then only have a generic "metal-y" texture with, say, some light spots. You won't, per se, see anything reflected on the object. If you do use a normal square texture with discernable things from the surroundings on it, you will see those things reflected even where the object faces away from those things. Keep that in mind. And yes, you shouldn't add a TXI with "cube 1" for those square textures. The game will expect a cube map with 6 sides. No idea how to use a TGA for real cubemaps, though. NWN used "cube 1" and "filerange 6" for a foo.txi, and then foo0.tga - foo5.tga. Possible that KotOR still supports that. For the TPC, they do actually contain 6 textures (each with their own set of mip maps), and the game detects that because height / width == 6 (and it divides height / 6, to get square sides). Interestingly, I've been experimenting with environment mapping in KotOR for xoreos in the last few days. KotOR combines this texture with such an environment map, and the blended combination then looks like this. That's the CM_BareMetal.tpc, and those don't contain any visible environment features, though, because CM_BareMetal.tpc is used all over the game. Reflective geometry in areas (like the streets of Taris) are different there, they use Taris-specific textures. And if you forget to turn off the viewpoint-distortion calculation for the default textures, you get this.

Ah, okay, that would make more sense, yes (and again matches that huge list of mine with unverified assumptions ). Thanks.

Wait, what? Does that mean that GetObjectByTag() reads all object template files (.ut?) the game has access to, and checks their tags? EDIT: And since GetObjectByTag() returns an object, does it mean it creates an object according to the template?

Okay, thanks. So those assumptions I made (based on my more in-depth knowledge of NWN) are correct, then.

Hmm, I have a question there, regarding how the game works: does that work, moving an NPC through a door into another module? I.e. after the creature left the module, does it appear in the target module when the PC then also moves through the door? Or do you need to fake that, by creating a copy of the creature in the target module (either statically, or from a script)? Somewhat relatedly, does GetObjectByTag() (and similar functions) work for objects outside the current module? I.e. if the PC is in module Foo1, and there's a module Foo2 with an object Bar2 in it, can I call, while inside Foo1, GetObjectByTag("Bar2") and have it return the object inside Foo2? I've always assumed that doesn't work; is that assumption correct?

Depending on the exact search order the game uses, placing GIT files into the override folder might also be very dangerous for save files. When you save your game, the game puts the local variables it assigns to objects into the GIT files inside the save archives. If the resources in the override folder have a higher search priority than current save files, that will potentially reset the local variables on all objects within that area. Now, I for one don't know if that's the case, but I think it's quite likely. ARE files are save in that respect, though.

To me, that sounds like the OpenGL path of your graphics driver doesn't like the high native resolution of your 4k monitor. It fails creating an accelerated OpenGL context, and then just creates a software one, or lets Windows create a compatibility one that maps to Direct3D, or something like that. Those then often lack a lot of the features, leading to weird graphical glitches and bugs. One thing you could try is to enable windowed mode in KotOR. For that, you need to edit the "swkotor.ini" file in the installation directory of the game. In the "[Graphics Options]" section, there should be a "FullScreen=1" line. Change it to "FullScreen=0" and add a new line with "AllowWindowedMode=1" below that as well. Also, you can set "Width" and "Height" (a couple of lines above), but try 800x600 or something like that for a start.