Modernizing Tools (as of 2019)

[AmanoJyaku 184]

Hi, all!

I'm writing modern versions of the tools for myself and wondered if there is interest in making them publicly available. Some motivations:

• Performance - K-GFF, DLGEditor, etc... are quite slow
• Dependencies - DeNCS, KSE, etc... require (sometimes deprecated) Java, Perl, QT, and other 3rd party packages
• Workflow - The tools are generally great, but you need to use more than one to do simple things like edit a GIT (e.g. ERFEdit on the SAV files, then K-GFF on the GIT)
• Ease of use - Most of these tools are less than intuitive, with steep learning curves
• Bugs - Some of these tools have still have bugs (e.g. GITEdit wouldn't open a file, or exit)

 

With these issues in mind, my "holy grail" tool would be:

• Fast - Every action should complete in milliseconds, including building structures from GFF files
• Self-contained - No dependencies, just pure C++ code
• One workspace - Files that contain other files should be modifiable along with the files they contain, with valid options for all file types (e.g. add creature to GIT stored in save game file)
• Intuitive interface - Noobs shouldn't have to read a manual
• Error-free - Including required knowledge of file formats, file names, and game mechanics

 

And the tools I'm targeting for replacement/consolidation:

• KOTOR Tool
• KOTOR Save Editor
• ERFEdit
• K-GFF
• DLGEditor
• DeNCS
• GITEdit
• Others?

 

Let me know what you think!

Edited December 17, 2019 by AmanoJyaku

[DarthParametric 3,785]

I would suggest DeNCS is the most in need of a replacement, since it chokes on a number of scripts. You should start by looking at what progress Xoreos has made on their decompiler (along with other tools for the various Odyssey formats) - https://github.com/xoreos/xoreos-tools

[AmanoJyaku 184]

46 minutes ago, DarthParametric said:

I would suggest DeNCS is the most in need of a replacement, since it chokes on a number of scripts. You should start by looking at what progress Xoreos has made on their decompiler (along with other tools for the various Odyssey formats) - https://github.com/xoreos/xoreos-tools

Indeed, DeNCS is the tool that got me started. I can't believe I have Java on my computer in 2019... Also, would like a tool that decompiles all scripts.

Funny coincidence: I just found out DrMcCoy is a member of another site I frequent. Kind of have to help with Xoreos now... 🤣

[bead-v 251]

57 minutes ago, AmanoJyaku said:

I'm writing modern versions of the tools for myself

What have you done so far?

Some time ago I started working on a "dlgedit" (for the same reasons you listed) in C++. ATM it can open/save dialog files, display them in a tree (which I wrote from scratch to maximize speed), it allows for copy/paste/cut/delete/create tree operations and most of the fields work, although not all are fully implemented yet. With it I made a GFF and a TLK library. If you're interested I can show you the code, either to use parts from it in your own work or to continue work on this tool (since I don't have the time right now. It is written using WINAPI, however... I can imagine that's not something you're interested in..).

 

(this topic should probably be moved to the modding tools section?)

[AmanoJyaku 184]

15 minutes ago, bead-v said:

What have you done so far?

Nothing, yet. I've written a basic ERF and GFF reader, but so has everyone else. 😂

15 minutes ago, bead-v said:

Some time ago I started working on a "dlgedit" (for the same reasons you listed) in C++. ATM it can open/save dialog files, display them in a tree (which I wrote from scratch to maximize speed), it allows for copy/paste/cut/delete/create tree operations and most of the fields work, although not all are fully implemented yet. With it I made a GFF and a TLK library. If you're interested I can show you the code, either to use parts from it in your own work or to continue work on this tool (since I don't have the time right now. It is written using WINAPI, however... I can imagine that's not something you're interested in..).

I would totally be interested in that! I'll probably scrap the WINAPI at some point since I want the code to be "portable". I'm developing for Windows, so I can start with the WINAPI and concentrate on the functional code.

17 minutes ago, bead-v said:

(this topic should probably be moved to the modding tools section?)

Maybe? I wasn't sure how to classify this since I wasn't asking about an existing tool, or announcing the release of one.

Also, building a tool is only part of the battle. I'll need help on mechanics, like how enemy levels and/or challenge ratings are calculated. Some of this is detailed in the BioWare Aurora file format docs, but other things I'm finding out through trial-and-error. For example, setting the PCLevelAtSpawn field of a creature in a GIT is what makes them higher-level; changing the ChallengeRating field doesn't seem to have an effect. In fact, it's possible ChallengeRating may be calculated each time the module is loaded, just like Max Vitality and Force points.

Thanks!

[bead-v 251]

7 minutes ago, AmanoJyaku said:

I would totally be interested in that! I'll probably scrap the WINAPI at some point since I want the code to be "portable". I'm developing for Windows, so I can start with the WINAPI and concentrate on the functional code.

I'll get it to you when I have a moment
 

7 minutes ago, AmanoJyaku said:

Maybe? I wasn't sure how to classify this since I wasn't asking about an existing tool, or announcing the release of one.

That was a hint to the mods

 

8 minutes ago, AmanoJyaku said:

Also, building a tool is only part of the battle. I'll need help on mechanics, like how enemy levels and/or challenge ratings are calculated. Some of this is detailed in the BioWare Aurora file format docs, but other things I'm finding out through trial-and-error. For example, setting the PCLevelAtSpawn field of a creature in a GIT is what makes them higher-level; changing the ChallengeRating field doesn't seem to have an effect. In fact, it's possible ChallengeRating may be calculated each time the module is loaded, just like Max Vitality and Force points.

Maybe this will help:

 

[JCarter426 1,215]

18 hours ago, AmanoJyaku said:

I'm writing modern versions of the tools for myself and wondered if there is interest in making them publicly available.

There is interest.

18 hours ago, AmanoJyaku said:

Self-contained - No dependencies, just pure C++ code

Hmm, would it be cross-platform? That's another hurdle for some people, since most modding tools were made before the game was released on various platforms and the only advice we can give to Mac or Linux users is "run it in Wine".

I recently started learning C++ as well but give me a year or so. I would like to contribute but I still have a lot to learn.

[AmanoJyaku 184]

On 12/17/2019 at 8:30 PM, JCarter426 said:

There is interest.

Dope. I haven't played this game since it first came out, so I never knew there was a modding scene.

On 12/17/2019 at 8:30 PM, JCarter426 said:

Hmm, would it be cross-platform? That's another hurdler for some people, since most modding tools were made before the game was released on various platforms and the only advice we can give to Mac or Linux users is "run it in Wine".

In an ideal world, yes. But, I only have a Windows machine, and no interest in installing Linux or buying a Mac again. lol

The initial focus is making a program that works correctly. After that I can make an API so the source can be ported. I prefer as pure a C++ implementation as possible, with only the GUI using non-portable code. For Windows that's DirectX, and then whatever equivalents exist on Mac and Linux (Metal and OpenGL?). Since it will have an API, it can easily have any front-end, even QT.

On 12/17/2019 at 8:30 PM, JCarter426 said:

I recently started learning C++ as well but give me a year or so. I would like to contribute but I still have a lot to learn.

I feel the same, and I've been coding for two years now. 🤣

Eventually, you'll realize you don't know sh#t and will never know sh#t, yet somehow a pot of coffee and a box of donuts gets something decent enough out the door the morning of a release.

Mm, donuts...

[JCarter426 1,215]

Makes sense. I'll be getting to GUI soon, but I'm afraid they may be teaching me MFC rather than anything relevant to start.

In any case, that sounds like a good plan, and as you say the interface can be sorted later if the back end is pure C++.

[DrMcCoy 40]

(Btw, please feel free to tag me in when talking about xoreos or something related. Google takes a while until it picks up new posts :P)

In either case, I'd be happy to have any help willing to work on xoreos and its sub-projects (xoreos-tools and Phaethon). It'd very much like xoreos-tools and Phaethon to be a collection of tools helpful for modders as well.

My general idea was that xoreos-tools was to be a CLI collection and Phaethon a tool incorporating complimentary GUI functionality. Both, I feel, are a good base to add more things that can edit and create, as opposed to just read, the files for the targetted BioWare games.

I'd also be willing to have similar sub-projects under the same xoreos banner as well [1][2][3].

Basically, what I'm saying: I for one think xoreos would be the ideal grounds for modernized KotOR modding tools [4]. We're not quite dependency-free, because there are libraries we depend on. Especially Boost is somewhat of a beast, I'm aware. In the future, I do want to phase it out as stuff gets approved into new the C++ standards and those become mature to be used in userland code.

 

[1] Though it might maybe be a good idea to migrate some of the code into a library first. Right now, xoreos-tools and Phaethon contains (mostly) unmodified copies of the xoreos files and that's a bit eeeeeeh

[2] I do want them to be portable, though and of course license-compatible with the GPLv3'd xoreos code. I can understand if people disagree with me on these points :P

[3] And yes, I am sometimes a stickler for form and following a consistent coding style. Again, I can understand if people feel I'm too strict there and wouldn't want to me near any of their code

[4] And also the other targetted BioWare games, to be honest

[DarthParametric 3,785]

4 minutes ago, DrMcCoy said:

It'd very much like xoreos-tools and Phaethon to be a collection of tools helpful for modders as well.

Get ncsdecomp working properly and you'll be our new best friend.