AmanoJyaku

My concern with nwnnsscomp is that we now know its output differs from that of the original compiler. That leads to three possibilities: The output files are fully reversible, even if they differ slightly from the source. This is acceptable. The output files are fully reversible, but require additional code to detect and handle the differences from the original compiler. This will be very annoying. The output files are not reversible. This is unacceptable. At the moment, I have no idea which of these is the case. With over 2,500 files, there is no reasonable way I can develop a reverse compiler while inspecting each script in a timely fashion. I feel it's something to take note of, and look into once the reverse compiler is complete. I don't know who taught you this, but it's a practice that's frowned upon even with modern compilers. We try to learn good habits rather than relying on the compiler to handle things we could easily do ourselves. It's not uncommon for bugs to creep into compilers, and you can track these down more readily with proper code. This advice comes from the best programmers at places like Apple, Google, and Microsoft, and even the ISO C and C++ committees. Why not create variables local to the else branch's scope? That's the whole point of having scopes, to create variables only where they're needed. Variables placed in too high a scope, e.g. the global scope, are at risk of being overwritten erroneously. As for the compiler knowing better than we do, that's true of professionally written compilers. Which nwnnsscomp is not. I've already listed some of the basic things it fails to optimize, e.g. failing to remove empty functions. To be fair, the original compiler from BioWare has an even bigger problem in that logical AND and logical OR are incorrect. Which nwnnsscomp doesn't fix...

I'm confident they can be, and I'm currently in the process of confirming that. What I believe will happen is that vanilla game files can be reversed perfectly, but anything that was compiled by nwnnsscomp will be altered from its original source. As DrMcoy said, the result, while syntactically different, should functionally be the same as the original source. For example, the source of k_contain_unlock from TSLRCM contains the following: string GetTreasureBundle (int nItemLevel, int nItemType = 0) { string sTemplate = ""; if (nItemType == 0) {} else { if((nItemType % 100) == 0) {} else if((nItemType % 10) == 0) {} else {} } return sTemplate; } But, the NCS file is written like this: string GetTreasureBundle (int nItemLevel, int nItemType = 0) { string sTemplate = ""; if () {} else if (){} else if() {} else {} return sTemplate; } The number of branches is the same across both, so the two forms should behave the same when executed by the game. If this proves to be true, then it won't matter if the reverse compiler produces the second form even if the first is the original. If I understand correctly, the original intent of the reverse compiler was to retrieve the sources for vanilla game scripts. Those are not compiled with nwnnsscomp. Which means the only files that are of concern are mods. I don't know what the policy is for reverse engineering mods, but I assume that's frowned upon?

Happy Life Day!!! The Last Problem The Sub Awakens The Rise of Subwalker tl;dr The reverse compiler is on its way to being complete. I was honestly about to give up, but I'm glad I didn't!

@Red Hessian I think it's k_inc_treas_k2, which is an include file for other scripts. For example, k_def_spn_t_jedi includes it and creates treasure on Jedi opponents when they spawn. (In theory, as I never confirmed.) If you look through k_inc_treas_k2 you can find which scripts call its subroutines and then confirm which enemies and quests call those scripts.

Time for an update! Now that I've recovered from the nwnnsscomp-induced heart attack, I'm back to solving the last of the reverse compiler challenges. (When I complete the reverse compiler, I'll provide a detailed explanation of NCS, how NCS maps to NWScript, how the reverse compiler works, and why I'm so critical of nwnnsscomp.) The current problem I'm working on looks rather simple: RSADDx JSR JSR We see a reservation on the stack, followed by two subroutines. NCS rules for subroutines are as follows: Optionally, reserve one or more values on the stack Optionally, place subroutine arguments on the stack Call the subroutine You can see the disaster coming: it's those optional bits. In the example, the first subroutine might return a value. If so, that value is the argument of the second subroutine. But, RSADDx is also used to create uninitialized named variables in NWScript, for example: int i; //RSADDI float f; //RSADDF string s; //RSADDS You should never create uninitialized variables, but it was once a common practice. Numerous examples exist in NCS files, and that means the example code can also be read as an uninitialized variable, followed by two subroutines that do not take parameters and return void. In other words: Sub1(Sub0()); //vs int i; Sub0(); Sub1(); So, how do we figure this out? Unfortunately, we have to look inside each of the JSR calls. Which means yet another rewrite of my reverse compiler. It's not the worst thing in the world since the purpose of a reverse compiler is to reconstruct JSR calls. But more work must be done to determine dependencies, change the order in which things are analyzed, and store the resulting information for later use. I don't work this hard in my paying job...

I use a wireless Xbox controller without any problems, not sure why yours doesn't work. What controller do you have? Additionally, I almost always kill 100% of the soldiers. I can do it with a controller or keyboard, so you just need to practice. I don't think you'll find a mod to alter the turret behavior as I believe it is hard-coded into the game engine. Meaning, someone would have to rewrite the game.

I believe the point was that, if possible, you should uninstall your Steam mods and use the non-Steam versions. Mods that use TSL Patcher correctly do all the work for you, no need to muck around with the Registry. They also have better support here as the community is more familiar with using mods outside of Steam. And in case it's not clear, you can use the game that's installed from Steam. It's just the mods that you want to get outside of Steam.

I'm familiar with compiler optimizations, I considered that possibility. However, I don't believe this is the case given the other issues like the lack of optimizations for logical AND and logical OR, the existence dead code, and empty functions. There is so much more that could have been optimized that I can only conclude this is sloppy coding. Additionally, it's highly unlikely the game benefits from such optimizations. The "optimized" files are larger (i.e. no cache savings), and the lack of an additional JMP instruction saves a few dozen cycles (which wasn't even a problem in 2003). Anyway, it's not my intention to argue over this. It won't prevent me from making the decompiler, but it means it won't be possible to guarantee the decompiled output matches the original source for files compiled by nwnnsscomp. I assume those files would only be found in mods. The main purpose for making this decompiler is to access original game files, which I assume were not compiled by nwnnsscomp.

I never used nwnnsscomp until I suspected mod files have "incorrect" scripts. Have you seen other issues? BTW, I realize my last post might be confusing to those who don't code in C and assembly. I didn't mean to alarm people by giving the impression scripts compiled with nwnnsscomp don't work. As far as I can tell, they work correctly in the game. They just don't decompile correctly, and that makes it difficult for someone who is using them to learn how to write a decompiler. There is a certain structure that a compiled file must adhere to in order to be decompiled. I finally realized that structure is different between the files in my KoTOR installation, which has no mods, and my TSL installation, which has the TSLRCM. I don't remember if TSLRCM saves the original scripts anywhere, so I'll reinstall a clean copy of TSL on another machine to examine the original files there.

I said there's always another problem... TL;DR Any scripts created by nwnnsscomp are likely to be broken. That probably means all scripts in TSLRCM. Oh, joy. The Gory Details

You may find @bead-v's stats guide useful. It's what brought me to DeadlyStream.

Hope everyone got their fill of roast porg yesterday! The Gap - No, We're Not Selling Pants Do You Have A Reservation? The Next Problem - Because There's Always Another Problem These are control statements, the remaining set of statements to decipher. Expect an update in the next few days.

I forgot to do the monthly update. 🤯 I was all set to finish the decompiler, and of course I ran into trouble. First, I had a family problem that has since been resolved. The second is a mundane issue: my laptop has been suffering from BSODs for the past two months, and it's happening with increasing frequency. Sometimes when I wake the machine from sleep or hibernation it BSODs, and when I cold boot it doesn't see any storage. This only happens when it's on battery, and it seems to be when the battery is below 50%. Now that I have an idea of what triggers this I can work around it, but it was affecting my productivity. The third is an issue with NCS files. My decompiler relied on files being written "correctly", so of course it bombed when fed a poorly written file. For example, k_inc_npckill: Nothing wrong with the source. Let's look at the NCS: The lines with asterisks are dead code, they never get executed and the game is fine with that. Problem is, my decompiler was expecting the last operation of the true branch of an if statement to be part of the control path. The last op of the true branch is what tells you if you're looking at a regular if, an if-else, or an if that exits the script as seen above. The decompiler was looking at the wrong thing, and returning the wrong results. I've now fixed that with an additional set of evaluations. Now I'm back to working on producing output. This is tedious as it requires keeping track of data, individual operations, and the context in which the data and operations are being used. Believe it or not, a block of code will decompile differently based on code that comes before and after it. (For example, RSADDx does not mean "create a named variable" as I incorrectly assumed months ago. It could create a named variable or a temporary variable, based on the context.) This morning I had an idea as to the overall rule for determining the beginning and end of statements, so I'll be working on that this weekend. I hope to have a status update in a few days.

I haven't been on in weeks, and by coincidence I log on and my assistance is required. I must have felt a disturbance in the Force... I ran into this issue with Carth, as well. I had always assumed it was deliberate or an oversight. I'll take a look at the scripts and see what I can find.

Sorry, I wasn't clear. You are correct about the compiler substituting defaults into the bytecode, the engine does not substitute at runtime. (I'm preparing a post to better explain NCS in case someone wants to port this to a different language, or make a better decompiler.) This is a silly example. But, it demonstrates the impossibility of identifying the type and value of an unused parameter from inside a function. I can't say I've seen this in practice. There are 2,500 files, I'm not looking at them all that closely. But I have to code for the possibility, or risk corrupting the stack.

Sorry that this is taking longer than I wanted. I was lucky enough to get some contracts, but that also means very little free time. While a decompiler is an all-or-nothing program (hence the reason there won't be a beta), I can show off some sample output: It's not much, but you can see following: It identifies the NCS as a Kotor2 file, which is reliant on the code including K2 engine functions There are six subroutines _start(), which is automatically included by the compiler void main() void DamagingExplosion( object oCreature, int nDelay, int nDamage ) int GR_GetGrenadeDC(object oTarget) void NonDamagingExplosion(object oCreature, int nDelay) void KillCreature(object oCreature, int nDelay ) The types of return values are found by examining the called subroutine The types of parameters must be found by examining caller subroutines (strange as it may seem, there's no guarantee a parameter is used) The local variables are listed in the order in which they are created As simple as this may seem, it's been hell trying to figure out how NCS works due to limited documentation, time, and mental capacity (🤪). That said, I think I now know everything there is to know about NCS. Even how to deal with recursion (it took less than an hour), identifying vectors and structs (they do disappear in the bytecode, but there are code patterns to look out for), and handling certain errors in the game's code. So, the two major tasks left are: Including block scopes, e.g. if-else and while statements (I can identify them, I just haven't put them in among the locals) Handling expressions, commonly known as operator precedence and associativity, e.g. int d = (a + b) * c

You attack the Sith throughout the game with Bastila in your party, and none of them says "Lord Malak wants her captured alive!" Malak surely wouldn't prioritize a soldier or officer over Bastila, it's obviously game mechanics. So, the limited game mechanic means Sith should attack Bastila on sight. OTOH, the Sith officers and soldiers are often stupid...

It's time for a monthly update, and I said I might have a beta ready by now. Well, there won't be a beta. Will have more news next week.

@Salk I preferred your version! Your accent is great, and the professional seemed lifeless. Did he view the scene in-game to see what was happening? @Sith Holocron Nice job!

I thought Adrenas' excuse was an interesting one. I never considered the possibility that a human/near-human species would have grown up in a region speaking something other than Galactic Basic. I wish this was expanded on since he says he can speak Galactic Basic, just with a thick accent. It's understandable that he doesn't since he'd be made fun of, which would be a sign of weakness among the Sith.

I considered that, but the differences are obvious and the detection simple. Are you aware of any pitfalls, or are you just being cautious? This is a beta release, there's plenty of time to change functionality. Given the fact that one file out of 2,500 has a do-while loop, it's low priority. I'll get around to it after more pressing matters are dealt with. Thanks, I thought as much. I first saw it in use destroying two elements of a vector, so I figured its purpose was for destroying unused elements of aggregates. I just need to identify which elements are preserved.

@JCarter426 Thanks! I'll look at them later just to be sure I haven't missed anything. I was supposed to do it earlier, I just forgot about it until I was Force Persuaded to. 😋

No, there isn't. That's why I had to write a detection algorithm.