DrMcCoy

Members
  • Content Count

    58
  • Joined

  • Last visited

  • Days Won

    3

DrMcCoy last won the day on April 8 2018

DrMcCoy had the most liked content!

Community Reputation

26 Jedi Padawan

About DrMcCoy

  • Rank
    Jedi Apprentice
  • Birthday 08/05/1985

Contact Methods

  • Website URL
    https://drmccoy.de/

Profile Information

  • Gender
    Male
  • Location
    Braunschweig, Germany
  • Interests
    Geek. Atheist+. Leftist. Metal-Head. Discordian. Lefty.
    ScummVM dev, xoreos lead.
    Software freedom zealot.

Recent Profile Visitors

2,801 profile views
  1. To explain what this warning means: ncsdis tries to run through the whole disassembly to analyze the stack. Essentially, how the stack looks after each operation: how many stack slots are allocated and what types (int, string, float, ...) are in there. When there is a fork, for example due to branching in an if-clause, ncsdis follows both paths. And when the paths merge again, for example after the if when the excution continues on linearly, the stack has to look the same for both paths. Otherwise, I wouldn't know what to make out of this situation. (ncsdis only checks that the size of the stack matches, and ignores the types at the moments.) In this case, for some reason ncsdis found that the stack sizes of two paths don't match. It's 2 slots for one path and 5 slots for another path. This might be a bug in ncsdis. This could also be a bug in the compiler that produced the NCS in question. For example, IIRC I found the same message for one of the bugs I wrote about in my blog post, the one where the short-circuiting boolean OR. In that case, it didn't really matter that the stack is broken for one path, because that path was logically dead anyway, it would never have been called. So to work around this bug in the original compiler, I added a dead branch check, and the stack analysis won't follow dead branches. It might be that this here is a similar problem and my dead branch check is not catches it and needs to be expanded. Or this script is broken after all. (Or it could still be a bug in ncsdis.) I'd need to check that, I'll add it to my TODO list. Or if anybody else would like to tackle this, feel free
  2. I don't remember seeing this exacty case. Then again, it's been a while since I touched that code and I'm not known to have a good memory. :P Also, my disassembler just follows the starting segment and IIRC I don't even check for unreachable blocks without an edge leading into them. That's maybe something that could use improvement. I take GitHub pull requests! ;) However, I can say that I have found a few bugs in BioWare's script compiler (I explained two in my replies above, I think), so I wouldn't particularily rule that out. And while I can't name any specifics right now, I also never got the feeling that their compiler was all that great in optimizing, fusing instructions, or removing outright unnecessary instructions. I don't think they focussed on that at all, rather keeping it simple and working without any surprises (bugs nonwithstanding).
  3. As for operators, you should probably also make sure you know the precedence rules. I'm not sure NWScript follows the C rules there exactly. I haven't yet looked at that, since it's unambiguous in the bytecode. Oh, and also, there's another bug I've found, but I'm not sure in which games it was present. It had to do with parameter shadowing. I.e. void foobar(int blah) { int blah = barfoo(blah); } (excuse the quirky formatting, this editor is weirder than I remember) I've seen the produced bytecode use the uninitialized value of the just created local variable blah as in input for barfoo(), instead of the parameter of foobar(). Maybe something to have an eye on. Hmm, depends on how DeNCS operates. Does it also try to decompile functions that are not called? I.e. if you have an include with foobar1(), foobar2() and foobar3(), and the script itself just calls foobar1() (and that doesn't call the other functions), does DeNCS also decompile foobar2() and foobar3()? Because IIRC ncsdis completely ignores foobar2() and foobar3() in that case and wouldn't see any recursion there.
  4. Yeah, do have a look at the nwscript directory: https://github.com/xoreos/xoreos-tools/tree/master/src/nwscript Specifically, the instruction.h: https://github.com/xoreos/xoreos-tools/blob/master/src/nwscript/instruction.h , which contains a list of opcodes (including 2 each that were introduced by Dragon Age: Origins and Dragon Age II). You might also want to look at https://github.com/xoreos/xoreos-tools/blob/master/src/nwscript/game_kotor.h and https://github.com/xoreos/xoreos-tools/blob/master/src/nwscript/game_kotor2.h for a list of engine functions and their signatures (return type + types of parameters). Alternatively, you can parse the nwscript.nss from the games yourself. If you haven't already, have a read over my blog post about disassembling the bytecode: https://xoreos.org/blog/2016/01/12/disassembling-nwscript-bytecode/ . It also shows a bug in BioWare's compiler (though that particular bug is not present in the KotOR scripts, it was fixed before KotOR development started it seems). If you want to decompile all scripts, you'll find that you're going to have problems with recursion, i.e. functions that call themselves. You need to analyze how a function leaves the stack to be able to continue with the code after the call, so you need to branch into a callee first before continuing with the caller. If the callee is the caller itself (or A calls B calls C calls A again)... that's a problem. This is essentially the halting problem, and there's no general solution, unfortunately.
  5. (Btw, please feel free to tag me in when talking about xoreos or something related. Google takes a while until it picks up new posts :P) In either case, I'd be happy to have any help willing to work on xoreos and its sub-projects (xoreos-tools and Phaethon). It'd very much like xoreos-tools and Phaethon to be a collection of tools helpful for modders as well. My general idea was that xoreos-tools was to be a CLI collection and Phaethon a tool incorporating complimentary GUI functionality. Both, I feel, are a good base to add more things that can edit and create, as opposed to just read, the files for the targetted BioWare games. I'd also be willing to have similar sub-projects under the same xoreos banner as well [1][2][3]. Basically, what I'm saying: I for one think xoreos would be the ideal grounds for modernized KotOR modding tools [4]. We're not quite dependency-free, because there are libraries we depend on. Especially Boost is somewhat of a beast, I'm aware. In the future, I do want to phase it out as stuff gets approved into new the C++ standards and those become mature to be used in userland code. [1] Though it might maybe be a good idea to migrate some of the code into a library first. Right now, xoreos-tools and Phaethon contains (mostly) unmodified copies of the xoreos files and that's a bit eeeeeeh [2] I do want them to be portable, though and of course license-compatible with the GPLv3'd xoreos code. I can understand if people disagree with me on these points :P [3] And yes, I am sometimes a stickler for form and following a consistent coding style. Again, I can understand if people feel I'm too strict there and wouldn't want to me near any of their code [4] And also the other targetted BioWare games, to be honest
  6. Huh, interesting. That might be a bug in xoreos-tools' erf packer? I might be able to look a bit at that during the holidays. Unless you want to tackle debugging this for me?
  7. Wait, why did I have to (re?)upload my profile photo? Or did I just completely forget to do that when I joined ages ago? Am I getting old? O_o

    1. DarthParametric

      DarthParametric

      I think some older images got nuked during the site update a little while back. A lot of attachments got the Thanos treatment.

    2. DrMcCoy

      DrMcCoy

      Ah, okay, thanks

  8. I didn't say that. No one is currently actively working on an Android port. But I have that on my radar and I do have ideas and plans. So I'm afraid I can't give you an estimate. Heck, I can't even give you an estimate on when KotOR is playable in xoreos, or when xoreos is "user-ready", or anything, really. I can only say that more contributors = faster work, and we really need more contributors. I know, I probably sound like a broken records, but it is what it is.
  9. That...depends. In C, you can't. In C++, you can overload a function, as long as the signatures, i.e. the number of parameters and their types, are different (internally, C++ "mangles" the identifiers, using a scheme that encodes the parameter types into the name). In NWScript, you can't either. The compiler wouldn't know what to do. When you write a script in NWScript, you use the compiler (for example, a modifed nwnnsscomp for Kotor, based on a open source reimplementation by the "OpenKnights Consortium" of the original NWN compiler, based on the work of Torlack who RE'd the stuff back in the day) to turn the C-like NWScript source (nss) into NWScript bytecode (ncs). Internally, the bytecode doesn't know much about the engine functions anymore. The compiler looks into the nwscript.nss file to look for the name of the function, and then counts at the index of the function within the file. I.e. if it's the first function in nwscript.nss, the second, the third, etc. That number it puts into the bytecode file ncs. So in the bytecode, you only have pushes onto the stack, for the parameters, that the compiler arranged there (and space for the return value), and then "call engine function 523". The game, when running the script bytecode, takes that number, then knows what to do, and which parameters it takes. I.e. with NWScript, you can't change the number of parameters, and the types, because that knowledge is hardcoded into the game. (Well, it's not impossible, just quite hard.) What you could, theoretically, do is add a new function to the bottom, then manipulate the look-up function in the game to recognize it. You'd still have to add the functionality itself, though. It's still hard, just slightly less so. (This could also be used to emulate overloading, by extending the compiler to map the overloaded functions onto later indices. Might not be worth the hassle, though.) We, err, don't have that implemented yet. It's not really that it's forbidden. It's just that you're setting yourself up for a lot of hard work and pain. And potentially failure, of course. I know, coming from me, who's RE'ing games for fun, and who has too many projects and stuff to do for 10 lifetimes, that's a bit ironic. Still, I still think the "best" way to resolve hardcoded things in KotOR is going over xoreos: getting xoreos in a playable and "done" state, so that improvements can be build upon xoreos. Not that this isn't a massive amount of work either, far from it, just that this opens up so many more possibilities. If you're adding a single thing to KotOR using the manual binary-hackery method, it's way harder to be sure you didn't miss anything, you can't verify easily when you continue to work on it that you're not breaking it, it's harder to integrate with other people's work, etc. You could instead do something similar, figure out how things for in the game and reimplement that in xoreos, bringing xoreos further along. It will not give you the gratification of having changed something in the original game as quickly, and still need to wait for other people to do similar things for other areas, but once it's all done, that's a far more stable base. At least that's how I see it. YMMV and all that jazz. And to be fair, I started xoreos low-key in 2010, and while there has been progress, it's still far from done, nearly a decade later. It is what it is.
  10. In C/C++-parlance, these are called "function declarations". They declare the name and parameters of a function, so that the function can be called. (Additionally, you have a comment explaining the function, which is not strictly necessary to call the function but helps the programmer.) Yes. The whole functionality for these functions is inside the engine, inside the EXE. It can't be changed readily at all. Wrong. The code and documentation of xoreos show explictly that the functionality is within the engine. That's pretty much expected, yeah. These engine functions usually check the parameters to be within their valid ranges. The rest of the code expects them to be still within the valid ranges, too. If you suddenly violate this expectancy, a crash is the most likely thing to happen. What you are doing is most probably invoking what's called "Undefined Behaviour", and the program might actually do everything it wants to in such a case. Common joking examples include eating your grandmother or letting demons fly out of your nose. On a more serious note, the game might also not crash immediately, and just leave your game in a broken state (even carrying on in future save states), so that you might, for example, always get a crash in a later section of the game, or just not progress through the game at all. Because you're changing the game's memory to values the original programmers never thought about. Essentially, what you'd need to do is to check every single location in the game that uses the value you changed, and verify that it still does the originally intended actions. And if you want it to do something additional or different, you need to make sure that it still makes logical sense in combination with the rest of the code. In addition to adding the new functionality, that is, which in itself is a tall order. Frankly, I for one would disadvise you from going down that road, but it's your life.
  11. Just to re-iterate: from what I've seen while taking the games apart, the menus are pretty hard-coded. The games in general know what widgets are there for each menu, how they hang together, how they behave and what they do, and they only use the GUI files to read the look. I.e. there is no way to get fully working menus from one game into the other, just by exchanging data files. You will need to hook into the binary itself
  12. IANAL, but from a legal standpoint, you won't be able to distribute these converted data files. What you can distribute (probably, I guess, but again, IANAL) is a converter that everyone will have to run for themselves, and everybody who does that will need to have legal, bought copies of both KotOR Android/iOS and KotOR2 PC. This is actually not *that* difficult, at least the Android part. Primarily, what we need is an OpenGL ES renderer. Currently, the renderer subsystem is being rewritten, so adding an OpenGL ES render path would probably clash, but after the rewrite is finished, that would be a very valid thing to do (in fact, that is already on my TODO list). Apart from that, all our libraries, including SDL2, would already work on Android. You can look at ScummVM for a proof of concept of sorts. There's certain things we'll have to keep in mind, like input, but it's not magic. iOS, on the other hand... the problem there is licensing, and the walled garden that is iOS. There's no chance xoreos will be allowed on the official Apple App Store. Not with Apple being Apple, and also because xoreos is GPLv3, a license that's fundamentally at odds with the closed-off nature of an App Store. The GPLv3 grants its users to modify the program, and run the modified program, and you can't do that with App Store things. I don't think there's really any sideloading on iOS, right?
  13. And just as a note, the task to extend the xoreos NWScript disassembler into a decompiler is still open :P. More information on that can be found here: https://github.com/xoreos/xoreos-tools/issues/16 Likewise, the task for an assembler, and then compiler, is also still open, probably. The person who started working on it has been silent for a few months now: https://github.com/xoreos/xoreos-tools/issues/17
  14. Frankly, I'd advise against using assets from the games as logo or promotional material. Likewise, I'd advise against having game assets in the project repository. That's just a legal minefield.
  15. Huh, it didn't come across as harsh, though? It's all good, I'm not slighted in the slightest (), as far as I'm concerned we're good. We're all good . Heck, I don't want any beef or rivalries with anybody, I want us all to help each other along. That's one of the reasons I'm in FLOSS to begin with. And yes, this KotOR JS project is obviously further along, and that's absolutely fine, and great work. I mean, this (and this, a bit later) is the current state of KotOR in xoreos, more or less, still stuck in the first room. There's only a handful of script functions implemented. The intro cinematic doesn't work play, and you can't get Trask to open the door. Similarily, there's a person implementing their own NWN2 engine over on the Neverwinter Vault. The renderer is way further along than xoreos, with proper shaders and everything. I'd of course be happy to have more contributors to xoreos, the xoreos-tools and Phaethon, but I can also understand that people want to do their own thing. Sorry for taking up this much space here, I just wanted to be absolutely clear on that.