AmanoJyaku

Sometimes, I learn more from failure than from success... I spent two weeks troubleshooting an algorithm that didn't always work. So, I added some functionality to assist in diagnosing the problem. Which lead to a few features updates! Directory Mode Now, when you supply a path the type is detected as either a file or directory. If it's a directory, all subdirectories are scanned and all files are processed! Version Detection The algorithm was fine, the problem was the input. I was using K2 function definitions, some of which had changed from K1. Now, NCS files are identified as K1 or K2! Conditional Input Detection The algorithm in question was alluded to in the September 4 post section about control structures. You will often see code like this: This is a pain to parse. The programmer sees one if statement, but the NCS file has 15 potential if statements! Seven of those appear to be extra, I haven't tested the logic to see if I can safely remove them from the compiled code. Still, that leaves us with eight potential if statements [e.g. if (sModule == "851NIH") { /*do stuff*/ }], which must be evaluated to see if they can be combined. That's what the algorithm does. And now that K1 and K2 functions are correctly identified, the algorithm correctly combines them as input to the actual if statement! To Do That leaves the following tasks: Evaluation of switch statements I haven't done this yet because ignoring them doesn't change the way the rest of an NCS file is processed, and my time is better spent on other tasks Evaluation of for and while loops They look identical, with the only potential difference being an incremented or decremented value at the end of a for loop Now that the algorithm described above has been completed, it should be easy to determine the rest of the code that makes a for loop unique I've temporarily given up on do-while loops because only one file has one, and I'm not entirely certain it's in the compiled NCS file Merging the algorithm with the virtual stack Once all control structures have been correctly identified, it will be necessary to identify variables created and destroyed inside scopes This will allow for the identification of subroutine input parameters and return values I've avoided using the virtual stack since I believe this is what has caused problems working with recursive functions for other decompilers Evaluating subroutines This is a big one, because callee input parameters and return values affect the caller subroutines I've been hard-coding both as a shortcut, but that only works for the files I'm currently testing Figuring out the DESTRUCT op code I've seen this used to destroy variables on the stack that aren't necessary, but prevent the top-of-stack behavior used by NCS Although this shouldn't be necessary, since the desired variable could just be copied to the top of the stack using the CPTOPSP op code... I should clarify that the algorithm does more than detect conditional input. It combines NCS op codes that make up NWScript statements. For example: RSADDS CPTOPSP -8 4 string IntToString(int) CPDOWNSP -8 4 MOVSP -4 Is correctly grouped as: string String = IntToString(Int);

The short answer is we'll never know because the game was unfinished. It was supposed to be larger. MUCH larger. But LucasArts wanted the game released in time for Christmas, so the game's development was cut short by almost a year. The budget was cut, too. The reason is shortly before TSL came out, LucasArts was restructured and a lot of their game development was canceled. Even before the restructuring, BioWare saw trouble and passed on developing the sequel. They recommended Obsidian, who had worked with them in the past. Unfortunately for Obsidian, plotting of TSL was started BEFORE KoTOR was finished and LucasArts refused to allow Obsidian access to the developing game. When KoTOR finally came out Obsidian had to go back and redo material that didn't fit with the original game's story. They had to start over, giving them even less time to write the story. This is the reason for so many plot holes and unfinished content, which is why modders have been working to restore whatever they could find. KoTOR and TSL are examples of what happens when a company (LucasArts) has crappy management. A whole book was written about it, in fact.

@DarthParametric I did say I will not fix the game scripts, right? When I'm done with the decompiler, I will look into each script to determine what, if anything, needs to be fixed. It's entirely possible that the NCS files are different from the NSS files, such as k_trp_generic which has a bit of dead code. Of course, with 2,500+ files I'm in no rush to try and fix things. Especially since I haven't even completed the decompiler. Which now has a name, btw: FEANCS. You get a cookie if you can figure out the origin.

Yes, because identification of the mine level occurs inside the if block. The problem is the if condition to get to that block always evaluates to "true". int TRAP_BASE_TYPE_POISON_GAS_MINOR = 9; int TRAP_BASE_TYPE_POISON_GAS_AVERAGE = 10; int TRAP_BASE_TYPE_POISON_GAS_DEADLY = 11; nTrapID == TRAP_BASE_TYPE_POISON_GAS_MINOR || TRAP_BASE_TYPE_POISON_GAS_AVERAGE || TRAP_BASE_TYPE_POISON_GAS_DEADLY (nTrapID == 9) || 10 || 11 (true or false) || true || true In practice, this isn't a problem because it's the last statement in the chain of if-else statements. By this point, the only mine type that hasn't been checked is poison gas. The condition isn't even necessary, since it must be a poison gas mine. However, if the poison gas test was ahead of any other mine type test then all subsequent types would be treated as poison gas. //FLASH STUN MINE if( nTrapID == TRAP_BASE_TYPE_FLASH_STUN_MINOR || nTrapID == TRAP_BASE_TYPE_FLASH_STUN_AVERAGE || nTrapID == TRAP_BASE_TYPE_FLASH_STUN_DEADLY ) {} //GAS POISON TRAP //BAD!!! else if(nTrapID == TRAP_BASE_TYPE_POISON_GAS_MINOR || TRAP_BASE_TYPE_POISON_GAS_AVERAGE || TRAP_BASE_TYPE_POISON_GAS_DEADLY) {} //FRAGMENTATION MINE else if(nTrapID == TRAP_BASE_TYPE_FRAGMENTATION_MINE_MINOR || nTrapID == TRAP_BASE_TYPE_FRAGMENTATION_MINE_AVERAGE || nTrapID == TRAP_BASE_TYPE_FRAGMENTATION_MINE_DEADLY) {} //LASER SLICING TRAP - THIS IS NOW THE PLASMA MINE else if(nTrapID == TRAP_BASE_TYPE_LASER_SLICING_MINOR || nTrapID == TRAP_BASE_TYPE_LASER_SLICING_AVERAGE || nTrapID == TRAP_BASE_TYPE_LASER_SLICING_DEADLY ) {} I'm being a nitpick, because the game is unlikely to be modified to add new mine types. But, with things like Xoreos it's not impossible.

You're saying that they aren't attacking you like they're supposed to? It could be they didn't see you, you have to be in their line of sight I think. You could force the issue by killing Saquesh and turn everyone hostile. 😁

@JCarter426 Nah, I just read some more code and confirmed that my original post was wrong. The "extra" RSADDx is for subroutine return values. Speaking of which, once I confirm that I know what the hell I'm doing I plan on documenting NCS. So far, the best resources are: Tim Smith’s documentation on the op codes, which is incomplete and has a few errors Ken Johnson (Skywing)'s documentation on his own unfinished decompiler DrMcCoy's documentation on their own unfinished decompiler I will explain how NWScript translates to NCS op codes. For example, it seems as if the only use of JNZ is for switch statements. There's no reason I can think of for why it couldn't be used in other ways, but it looks like the compilers all default to JZ for boolean conditions. Even if JZ ! true could be optimized to JNZ true. So wasteful...

Yeah, that was what I wrote in my edit. I left the original post up in case someone with experience could explain what I'm seeing. Thanks!

Edit: I think the following is wrong. I haven't worked on subroutines yet, so I think the [?????] is the reserved space for the return value of GetTreasureBundle(). I'm not sure if this is the source of the stack bug, but I did find something interesting: Now, compare the compiled code: That [?????] is a string, and an extra stack object that doesn't exist in the NSS! Might not have seen this if I hadn't been using this file to test my decompiler for the last 6 months. I guess it could be changed to a NOP, which is the same size and doesn't do anything. Still, I now need to count the stack objects and confirm it isn't doing anything weird.

@Salk If possible, please remove extra datapads upon completion of quests. For example, Calo Nord's after you leave Taris. Also, the Mandalorian datapads probably shouldn't duplicate on Dantooine.

*flips table* I am not fixing the game scripts. I am NOT fixing the game scripts!!! OK, I'll fix the game scripts.

Could be someone used an old version of the compiler, even for the official game.

I doubt performance is a concern. We're talking dozens of wasted cycles out of billions. It wasn't a problem back in 2003, it's even less of a problem today. OTOH, bugs need to be dealt with. Tsk tsk. Placing side effects into your conditions. Shame on you. 😛 Fortunately, I haven't seen any in scripts, nor do I see it in NWLexicon. Goto is bad, anyway, so I would just be an ass an not translate it. 😜

What's Taking So Long? CPUs and NCS Bytecode NWScript and NCS Control Structures Difficulties and Errors All of this was taken into account when I said I think I can get this done by the end of September. Just thought people might be interested in the gory details. Also, as I work on this I'm encountering what appears to be poorly coded scripts. For example: That last one is doesn't look right. May need to create a second WIP: the Script-Fixing Project. Sigh...

Eh! This is what happens when you read a forum as soon as you wake up. I don't remember if that happened to my character. Will pay attention the next time I play. I finished KoTOR earlier this year, but didn't play much of KoTOR2. Something about a decompiler... Will also pay attention when I get a chance to play.

@StellarExile It was me vs. four rakghouls. My companions ran in place and watched while I died. After that, I made sure to check that my party was with me before I jumped into battle. If you're far enough away they usually respawn behind you. If not, I quick-save then quick-reload. That usually fixed it, but if they got stuck again I just quit the game and started up again. It didn't happen on every world, so I never took notes. Not going to try to reproduce it now since I'm trying to finish a mod tool.

@StellarExile K1 Undercity was the first place I encountered it. "Rakghouls! Time to kick some ass! Um... Where is every- Huagh!!!" I don't remember this in the original play through 15 years ago. Wonder if it's related to the Steam re-release...

Can confirm that I've seen all three. Not sure if it's a script problem or an engine problem.

I'm a suspicious person by nature, and I never like to promise anything before I deliver... But, I may be done by the end of September. I just completed a few tests and things are going well. Like, "woah, did that actually decompile???" well. There are three tasks remaining: Differentiating the different types of loops (they look really similar in byte code) Pairing the logic with the stack to refer to the proper variables Generating output whose style doesn't offend the reader When that's complete I'll test against all 2,500+ K1/K2 NCS files. As long as life doesn't get in the way, I'll have a beta for K1/K2 soon. 😣 Wakanda Forever

@JCarter426 This is what happens when you hire college kids to write professional software... Anyway, I think I figured out the problem that caused me to create this topic. This is going suspiciously well... 🤔

Me, too. Because it is hard. It's hard to stay awake! Maybe now I don't have to figure out a way to remove it.

@JCarter426 Sigh... This means I have to check every script that has logical OR. The snippet from k_contain_unlock is fine since it doesn't change anything, but any script that looks like the short circuit test is guaranteed to do the wrong thing. Whether or not that breaks the game remains to be seen... I knew I should have stuck with stupid weapon and armor mods! 😂

@JCarter426 Nah, that's not it. The problem I ran into is logical OR. //Contrived example int Index{ 0 }; if (true || ++Index) std::cout << "Index " << Index << '\n'; The output should be "0", because ++Index should not be evaluated. Yet, according to the NCS bytecode ++Index is always evaluated. //Code snippet from k_contain_unlock, included from k_inc_treas_k2 //First expression in logical OR //sModule == "851NIH" 2993 CPTOPSP -4 4 3001 CONSTS 6 851NIH 3011 EQUALOO 3013 CPTOPSP -4 4 3021 JZ 3041 //Code to execute if expression evaluates to true 3027 CPTOPSP -4 4 3035 JZ 3061 //Why the heck are we evaluating the next epression if the previous expression is true??? // || sModule == "852NIH" 3041 CPTOPSP -8 4 3049 CONSTS 6 852NIH 3059 EQUALOO 3061 LOGORII If this pattern exists in all files, then all files are "wrong".

@verb8im Globals->Booleans->003EBO_VISAS_ARRIVES should be set to "Yes". Globals->Booleans->VISAS_ARRIVES_CONV should be set to "No". Globals->Numerics->000_Before_Visas_Trn should be "0". There are other Numerics for Visas, but I don't think you need to check them. If you do, they should all be set to "0". If you still don't get her, then I'm out of ideas. Good luck!