[WIP] (b1.1.0) KOTORPatcher - TSLPatcher-like app for Android

[LewsTherinTelescope 18]

So far, this app is very basic, and lacks most of TSLPatcher's features. The only thing it currently can do is copy files from tslpatchdata to the proper folders, and patch 2DA files. However, I plan to try to add more features from TSLPatcher.

You can download the latest release version of the app here, or contribute here. The app is licensed under the GPLv3 (not 3+, just 3). For more details, press the licenses button in-app, or look on the source repo.

Bugs:

 • Buttons in menu have no margin 
 • Please report!

Features:

 • Copying files...              ✔
 • Editing 2DAs...               ✔
 • Editing TLKs...               ❌
 • Editing GFFs...               ❌
 • Editing MODs...               ❌
 • File selector...              ✔
 • Namespaces...                 ❌
 • Show info.rtf...              ❌
 • Other TSLPatcher features?... ❌

Changelog:

b1.1.0 {146}

    • Theme UI
    • New launcher icon
    • Changed minimum API level from 14 to 7

  BTS:
    • Switch from MarkdownView to txtmark
    • Moved files from assets to raw resources
    • No longer uses Xdroid.Toaster
    • Now uses AppCompat support library and VectorDrawables
    • Switch from official aFileChooser to Zhuowei Zang's version

[LewsTherinTelescope 18]

Unrelated, but is there a way to disable code block coloring? And why don't newlines stick in code blocks? Do I need to do \n or something?

[Kexikus 994]

Unrelated, but is there a way to disable code block coloring? And why don't newlines stick in code blocks? Do I need to do \n or something?

 

You can set the code type to XML to get rid of the coloring. But I don't know about the newlines.

[LewsTherinTelescope 18]

You can set the code type to XML to get rid of the coloring. But I don't know about the newlines.

Setting it to XML also fixed the newlines. Thanks!

[LewsTherinTelescope 18]

Update a1.1.0

 

Changelog in OP

 

I changed the signing key for releases from the default debug key, so you must uninstall and reinstall. Debug builds are not affected.

 

You can use the new file chooser to select the TSLPatcher.exe file in the same directory as the tslpatchdata (NOT a file in the tslpatchdata itself).

 

Updates are NOT automatic, you must choose the Check for update menu item. If you have downloaded a debug build for this update, checking for updates will fetch the latest debug build, which I upload as soon as I get a functional build, and are signed with the default android debug key. (I wouldn't reccommed them.) Release builds are uploaded once I feel it deserves an update, and are signed with my key. You cannot install one over the other, you must uninstall to switch.

 

The letter before the version number (like a in a1.1.0) stands for alpha, beta or release. Alpha is what I am calling all builds until I complete 2DA support. After that, builds will be called beta. Release will be once all features are added or marked will not add.

 

I was told by DrMcCoy on GitHub that due to a problem with the build script, compiling xoreos-tools statically would mean recompiling Boost, which I can't/have no clue how to do (see the GitHub issue for details), so I am starting work on b1.0.0 hopefully this weekend, and will just use dynamic builds (and ship a bunch of libraries, which is why I hoped for static, but it'll at least work).

[peedeeboy 23]

However, I plan to try to add more features, like 2da editing.

 

 

I've uploaded some Java wrote to parse binary (v2.0b) 2da files here for you: https://bitbucket.org/peedeeboy/2daeditor

 

It's pretty rough and ready., but feel free to pinch any if you want - just credit me.

 

I stopped working on this as I got frustrated using Swing for UI (you have to resize the window after loading the 2da to force a repaint), and intended to come back to it after learning some JavaFX.  However, after learning some JavaFX, I actually think the best thing to do to make a complete desktop toolkit would be to learn how to make plugins for Eclipse / Netbeans.

[LewsTherinTelescope 18]

I've uploaded some Java wrote to parse binary (v2.0b) 2da files here for you: https://bitbucket.org/peedeeboy/2daeditor

 

It's pretty rough and ready., but feel free to pinch any if you want - just credit me.

 

I stopped working on this as I got frustrated using Swing for UI (you have to resize the window after loading the 2da to force a repaint), and intended to come back to it after learning some JavaFX.  However, after learning some JavaFX, I actually think the best thing to do to make a complete desktop toolkit would be to learn how to make plugins for Eclipse / Netbeans.

Thanks, awesome, I'll look at that. The thing is, since my project is GPL licensed, I think the code needs to be under an open license, not just with permission. (Not a lawyer, so not 100% sure).

[LewsTherinTelescope 18]

I've uploaded some Java wrote to parse binary (v2.0b) 2da files here for you: https://bitbucket.org/peedeeboy/2daeditor

 

It's pretty rough and ready., but feel free to pinch any if you want - just credit me.

 

I stopped working on this as I got frustrated using Swing for UI (you have to resize the window after loading the 2da to force a repaint), and intended to come back to it after learning some JavaFX.  However, after learning some JavaFX, I actually think the best thing to do to make a complete desktop toolkit would be to learn how to make plugins for Eclipse / Netbeans.

Do you have any code for writing to the 2DAs or just reading?

[peedeeboy 23]

Do you have any code for writing to the 2DAs or just reading?

 

At the moment, just reading. 

 

I noticed that the binary 2DA has a kind of built-in compression - due to the file format having a 'data section' and each item in the 2da having a 'pointer' to the data section meaning that if six items were to have a data value of say '10', then rather than '10' be stored six times, the six pointers would all point to the same data location containing the value '10'.

 

It appears that some existing tools (KOTORTool has the behavior I think - but I may be forgetting) have a one-to-one relationship between the pointers and data, so in the above example, if one were to open a 2da file in KOTORTool and save it, you will end up with a larger file size than you started with....

 

 

I've been learning the Netbeans platform recently, as it has a lot of really useful built-in things in the framework (like built in functionality to determine a file type by the first n bytes, abstraction layer between files and editor, data loading classes, syntax highlighting etc.) along with the Netbeans module system (meaning an editor solution could be easily expanded one file type at a time) would be great for making a cross-platform editor.

 

When I've learned a bit more, I will probably have a bash at Netbeans plugins for binary 2DA (including) and .nss editing first.  It's really for my own programming satisfaction rather than as a serious project, but I will be happy to share the Java with you if it would be useful for your Android project. 

[LewsTherinTelescope 18]

Pretty sure I have 2DA editing done, once I figure out how to write it to a file, I'll double check, and should release the next update! After that, I'll focus on namespaces, then other stuff. School starts back up tommorow (well, technically it'll be today in an hour... I need sleep) though, so I won't have as much time.

[LewsTherinTelescope 18]

Latest update out! Thanks a ton peedeeboy for the code to read and write the 2da files!

Changelog:
b1.0.0 {13}

  • REQUIRES REINSTALL - Changed signing key (I'm sorry, I forgot the old one)
  • Not doing debug builds any more
   
  + 2DA support (thanks a ton @peedeeboy!)
  + Option to create a 2DA from scratch (test)
  + Do not install music or launcher (not supported by Android KOTOR)

  • Log now updates while in progress
  • App no longer freezes when installing mods

Next, I will be working on a nicer UI, and then namespaces and info.rtf (both of which will require UI changes anyway, so I'm taking advantage of that).

[zsouza13 0]

I'm having troubles getting the app to properly install mods, it doesn't seem to install all of the files? For instance I'm trying to install revan hoodless mod for Kotor 1

[zsouza13 0]

Hmm it seems like it overrides danm files as well instead of skipping them?

[LewsTherinTelescope 18]

Hmm it seems like it overrides danm files as well instead of skipping them?

You mean this mod? Unfortunately, this app doesn't yet support editing GFF files (such as UTI files), or .mod files, so that is likely the issue. Is your issue the model (which I think should work?) or the fabricator on Dantooine?

EDIT: Also, currently, if there are conflicts the app overrides them. I'm planning next update to add an option.

[zsouza13 0]

I believe the issue is what you mentioned about the GFF files and the mod files, the kiosk and fabricator do appear. However nothing happens when I click on it. Thank you for the quick reply. I look forward to supportinf this app.

[LewsTherinTelescope 18]

I believe the issue is what you mentioned about the GFF files and the mod files, the kiosk and fabricator do appear. However nothing happens when I click on it. Thank you for the quick reply. I look forward to supportinf this app.

Actually, looking at it again (and based on your reply about nothing happening), I am pretty certain that the issue is that this app does not compile NSS scripts into NCS. If so, then you will probably still be able to use them on the Star Forge.

[LewsTherinTelescope 18]

Latest update just (badly) themes the UI and has some behind the scenes changes. Nothing major, last week was semester finals, so I didn't have much free time. I'm working on the next update, though.

Changelog:

b1.1.0 {146}

    • Theme UI
    • New launcher icon
    • Changed minimum API level from 14 to 7

  BTS:
    • Switch from MarkdownView to txtmark
    • Moved files from assets to raw resources
    • No longer uses Xdroid.Toaster
    • Now uses AppCompat support library and VectorDrawables
    • Switch from official aFileChooser to Zhuowei Zang's version

[Violation 0]

Props for working on something like this. I am personally drewling over the idea of an android app to help me use mods from the pc style game on my android system. I am curious is there a how to use readme somewhere that I missed? I have the app first app installed that you posted for some reason the update you posted will not download for me. Anyways, I have the pc versions of mods downloaded via Advanced Download Manager (ADM), they are Extracted into a custom Kotor Mods File directory within ADM. I open your program and choose a file location for the tslpatcher.exe (example: Bodysuit Bastila mod and when I click install the app instantly crashes. Not sure what I am doing wrong. I will see if I can add the program to SU since my system is rooted.

[LewsTherinTelescope 18]

Props for working on something like this. I am personally drewling over the idea of an android app to help me use mods from the pc style game on my android system. I am curious is there a how to use readme somewhere that I missed? I have the app first app installed that you posted for some reason the update you posted will not download for me. Anyways, I have the pc versions of mods downloaded via Advanced Download Manager (ADM), they are Extracted into a custom Kotor Mods File directory within ADM. I open your program and choose a file location for the tslpatcher.exe (example: Bodysuit Bastila mod and when I click install the app instantly crashes. Not sure what I am doing wrong. I will see if I can add the program to SU since my system is rooted.

The app does not use root. Could you get me a logcat of the crash? Also, keep in mind it can currently only copy files and patch 2das, not compile scripts, edit .mod and UTC, etc. And it always overwrites files, so be sure to install any mods in the correct order.

You'll have to manually download the update from the post, because I changed the location of the apk file from what the old version expects, because it makes it easier in the code. Also, if you have the very first (pre-updater) version, you have to uninstall and reinstall, due to a signing key difference.

[Violation 0]

yeah i caught the part where it only copies other files main reason i was trying something that i knew would not overwite or change the uti/utc etc. lemme see if i can find this log you speak of.

[Fair Strides 509]

PorygonZRocks: I'm not sure if you know of me or not, but that knowledge would only help in giving you confidence in my offer. I have access to most of the file formats the KotOR games use and I recently just finished converting TSLPatcher from Pascal to Perl so I have the entire source code in two different langauges and can answer questions.

 

Had I even known this project existed (it likely got buried in all of the holiday and Episode 8 stuff), I would have been able to help out much sooner. Things like the .mod, .erf, and .sav files are all in the ERF format, while the .rim files are in the RIM format; both are similar, but have their tricks for writing the files. Also, with writing RIM files, you will never be able to get a 1:1 byte-to-byte ratio between the original game files and edited files... :|

 

I can be contacted on here via PM, on Discord (User name FairStrides, ID# 2148), or on Skype (though I'm rarely on that nowadays).

 

@peedeeboy: I realize that you guys are already past the fun of writing .2da files (yeah, the pointers part was fun, avoiding a 1:1 ratio and doing it the way Bioware did it), but I felt the need to correct you in that KotOR Tool does not do a 1:1 Pointer-to-Data ratio. It still uses a single pointer for each unique piece of data, with duplicate data being referenced by the original pointer.

[LewsTherinTelescope 18]

PorygonZRocks: I'm not sure if you know of me or not, but that knowledge would only help in giving you confidence in my offer. I have access to most of the file formats the KotOR games use and I recently just finished converting TSLPatcher from Pascal to Perl so I have the entire source code in two different langauges and can answer questions.

 

Had I even known this project existed (it likely got buried in all of the holiday and Episode 8 stuff), I would have been able to help out much sooner. Things like the .mod, .erf, and .sav files are all in the ERF format, while the .rim files are in the RIM format; both are similar, but have their tricks for writing the files. Also, with writing RIM files, you will never be able to get a 1:1 byte-to-byte ratio between the original game files and edited files... :|

 

I can be contacted on here via PM, on Discord (User name FairStrides, ID# 2148), or on Skype (though I'm rarely on that nowadays).

 

@peedeeboy: I realize that you guys are already past the fun of writing .2da files (yeah, the pointers part was fun, avoiding a 1:1 ratio and doing it the way Bioware did it), but I felt the need to correct you in that KotOR Tool does not do a 1:1 Pointer-to-Data ratio. It still uses a single pointer for each unique piece of data, with duplicate data being referenced by the original pointer.

Yeah, I know of you. Thanks a bunch for the offer, I'll be sure to ask if I need help!

I have already found the Bioware Aurora File Format docs online, and I think that KOTOR mostly uses those? I know some things, like binary 2DAs, aren't there (perhaps weren't used for NWN?), but there is info on that, and thankfully peedeeboy has dealt with that anyway. Is anything else missing from the docs, or are any of the formats updated from the documented versions that you know of?

[peedeeboy 23]

 

 

@peedeeboy: I realize that you guys are already past the fun of writing .2da files (yeah, the pointers part was fun, avoiding a 1:1 ratio and doing it the way Bioware did it), but I felt the need to correct you in that KotOR Tool does not do a 1:1 Pointer-to-Data ratio. It still uses a single pointer for each unique piece of data, with duplicate data being referenced by the original pointer.

 

I humbly retract my spurious, slanderous allegations against KOTORTool!    In my defense, your honour, I did state that I couldn't exactly remember if it was KOTORTool I was thinking of that did a 1:1 ratio... I started looking into the 2da format 2 years ago, then got distracted... and I'm getting on a bit, and my memory isn't what it used to be... 

 

I would be very interested and grateful if you would be willing to share your knowledge of the file formats.  It would be really useful for my own project - it's really difficult to find information on the Odyssey file formats, especially if there are cases where they digress subtlety from the equivalent Aurora format.  Cheers! 

[LewsTherinTelescope 18]

I humbly retract my spurious, slanderous allegations against KOTORTool! In my defense, your honour, I did state that I couldn't exactly remember if it was KOTORTool I was thinking of that did a 1:1 ratio... I started looking into the 2da format 2 years ago, then got distracted... and I'm getting on a bit, and my memory isn't what it used to be...

 

I would be very interested and grateful if you would be willing to share your knowledge of the file formats. It would be really useful for my own project - it's really difficult to find information on the Odyssey file formats, especially if there are cases where they digress subtlety from the equivalent Aurora format. Cheers!

In a PM, Fair Strides told me that the files in KOTOR are the same as their documented Aurora forms. Obviously that's excluding binary 2DA, which you've figured out, and RIM, which is missing. In the xoreos-tools source, it says that RIM is basically a simplified ERF. Quoting:

A RIM file is a resource archive, used in several Aurora games.
It is a simplified version of the more complex ERF format, lacking
the support for a localized description text as found in early ERF
formats, as well as compression and encryption as found in later
ERF formats.

There is only one single version, V1.0, of the RIM format known and
supported. For each resource file, it stores a path- and extension-less
name (with a maximum of 16 ASCII characters) and a Type ID.

[peedeeboy 23]

In a PM, Fair Strides told me that the files in KOTOR are the same as their documented Aurora forms. Obviously that's excluding binary 2DA, which you've figured out, and RIM, which is missing. In the xoreos-tools source, it says that RIM is basically a simplified ERF. Quoting:

 

Thanks!  That is reassuring!