TLK Merging

[ChrisC26 12]

I did some searching and saw that Stoffe had a TLK file merger but it was linked to some dead site (starwarsknights.com) 

I am trying to merge 3 .tlk files

1. The TLK file in the collaboration, All In One Force Power Mod (AIOFPM)
2. The TLK file in KP's PC Response Mod
3. The TLK file in Salk's Dialogue Fix Mod

Thanks!

[DarthParametric 3,777]

TSLPatcher can do TLK appends. It comes with a TLK editor, TalkEd. I've never done it though. The official link is still on the now defunct SWKnights domain though. There is a mirror on the K1 Nexus site, albeit mostly likely unsanctioned by Stoffe.

[ChrisC26 12]

So use TalkEd that comes with the TSLPatcher. And what would the next step be? Append the second/third .TLK files? I'm trying to find examples on LucasForums but haven't been able to do so yet.

Also, while we're looking at the TSLPatcher... I had another question. How easy is it to make a mod install by editing existing files before replacing them? There are a few TSLPatcher mods I have that just overrite files and if I can edit them simply with a line or two in the TSLPatcher that would save me time when re-installing my massive mod library.

[DarthParametric 3,777]

Using TalkEd would be to create an append TLK file, that TSLPatcher then adds to whatever existing TLK file you want to merge with. You should read the instruction PDF that comes with TSLPatcher.

 

As far as merging files rather than overwriting goes, yes, you can do that to some extent, but it depends on what it is exactly you want to do. If you want to merge/edit 2DAs, GFF files (UTCs, UTIs, etc.), that's possible. It is, after all, TSLPatcher's entire raison d'être. If you want to play around with binary files, like textures and models, then all TSLPatcher can do is overwrite. It's all covered in the instruction PDF.

[Kexikus 994]

Yeah, it's all covered in the instructions pdf.

 

But what you basially do is, you create a file called append.tlk with TalkEd, set TSLPatcher to add all entries of this file to the tlk file you want to be edited during the installation. If that's all you need you're done. If you need StringRefs for those new lines, you can do that too by using the StrRef## number coming with each line when setting them to be added to the tlk file. TSLPatcher will then add all those lines with new StringRefs to the end of whatever tlk file you edit and put corresponding StringRefs wherever you need them.

[ChrisC26 12]

So would that be the same in the case of the PC Response Mod and the Dialogue Fix Mod where the authors released the dialog.tlk file on it's own with the intent to overwrite?

[DarthParametric 3,777]

In the case of direct overwrites, you'll need to find the additional entries and strip them out as an append TLK file. For other types of files TSLPatcher has the compare functionality, that finds the differences between two files, but it doesn't seem it can do that for TLK files.

[redrob41 86]

I'm not sure if you'll find any answers to your specific questions, but this is a thread where Stoffe helped me directly with using TalkED

http://lucasforums.com/showthread.php?t=181550

I'm no expert with the program, so I always have to refer back to that thread (in combination with the program's instructions). Hopefully it acts as a good example for you.

[ChrisC26 12]

This might not be as easy as I thought lol. Because all three mods are installed as Overwrites rather than updates to the .tlk file, it means going through the entirety of the .tlk files to find the changes,  2 or even 3 times.

I'll look through the link RR (and thanks BTW) but I don't think there is gonna be any easy way to do this. 

[DarthParametric 3,777]

Yeah, unless the authors provided/can provide a list of all the lines they changed/added, I would think it would be fairly excruciating.

 

I would think the reason that Stoffe didn't provide a comparison functionality for TLKs was becasue of memory, or lack thereof, back in the day. TalkEd provides a warning about memory use based on number of entries loaded. It probably wouldn't be an issue with a 64bit program on modern systems.

[ChrisC26 12]

Yah. Its close to 50000 entries in ONE TLK file. Going through the three of them to compare and contrast is time that I am not willing to spend on a game I've played God knows how many times lol.

Truth be told I've been wanting to use the 2nd and 3rd mods I've mentioned for quite some time now but haven't been able to since the authors didn't include a TSLpatcher version. I'll try contacting them to see if they're still active.

And with some luck and a bit more time, compatibility might be found between these three mods.

[DarthParametric 3,777]

If you can contact the original authors and find out the specific additions they made, that should make it fairly straightforward. The question for old mods always comes down to, assuming you can contact the authors in the first place, do they still have the source material/records of what they did?

 

Edit: Here's one brute force way you could tackle it. Grab the Xoreos Tools Windows binaries. Put the vanilla TLK and the mod TLKs (renamed appropriately) in the same folder. Via a command prompt, use tlk2xml.exe to convert the TLK files to XML. For example:

 

tlk2xml dialog.tlk k1.xml
tlk2xml salk.tlk salk.xml

Use a comparison tool to find the changes between each mod's TLK and the vanilla one. You could use something like Beyond Compare. For example, comparing Salk's TLK to the vanilla gives the following summary:

 

53259 same line(s)

194 unimportant difference line(s)

1 important right orphan line(s)

301 important difference line(s)

 

433 difference section(s)

 

You can then export the actual differences, but it would be a bunch of manual work to then merge all that into an append TLK for TSL Patcher. A hell of a lot easier than trying to do it manually line-by-line though.

[ChrisC26 12]

Not a bad idea. I might try that, and if it doesn't take too long I'll give it a shot for all three.

[Inawordyes 0]

WinMerge is another good comparing program. It can't read TLK's, but it can read XML's (which you can get as an export through the Kotor Tool dialog editor as well). I know these are KotOR mods, but specifically for TSL users, unless the mods you're looking into are specifically built from TSLRCM (assuming you're using it), it's best to compare with the vanilla Dialog.TLK, as TSLRCM has so many differences it can drown out the actual changes (most of which however, will be added content at the bottom of the file, but still, it's a bit of a pain to go through). It doesn't have an export feature (at least that I've used), so I don't know how it compares (hehe puns), but maybe it'll work

 

This thread is a good resource for me, as I was wondering how I do the same thing in order to combine some mods that built on TSLRCM but weren't TSLPatcher'd and so wouldn't play nice with each other.

[Fair Strides 509]

You could also convert the TLK file to a TXT file using my converter in the downloads section. You could convert all three dialog.tlk files to txt, combine the different lines at the bottom into one of the TXT file and then convert it back to TLK...

 

However, this won't do a thing for which lines the mods' files reference, so you might need some work with K-GFF or something...

[DarthParametric 3,777]

Two of those mods change existing lines rather than add new ones, so you have to use some sort of differencing tool to find the changes.

[DrMcCoy 40]

You can then export the actual differences, but it would be a bunch of manual work to then merge all that into an append TLK for TSL Patcher.

Or alternatively, if you (or anybody else reading) are proficient in C++, consider extending the xoreos tools with an xml2tlk program.

We're a very small team (so small that it's still mostly just me ), and are always looking for more devs to join in the fun.

 

Anybody interested just poke me in #xoreos on Freenode IRC or write to the mailing list, and read over the Developer Central page on our wiki.

 

</shamelessplug>

[ChrisC26 12]

I was thinking something like this.

Take AIOFPM and use that as the "base" TLK since that one probably adds the most rather than edit. Or at least I would imagine it does. Plus convert it to either xml or txt depending on whichever is best for comparing

Take Salk's second since it makes fixes to existing dialogue it would seem logical to convert and compare it first and add the changes to the AIOFPM dialog.tlk.

Finally take KP's and convert and compare last. His mod also edits existing dialogue AFAIK, I don't believe it adds anything. 

So between Salk's and KP's I imagine there would be anywhere from 400-600 actual changes based on DP's comparison and considering that some of those changes might be to the same dialogue.

Since the lines I am changing in the xml or txt file are just changes to existing lines, would I just need to copypaste the lines, save, then convert back. Which I imagine would be about 2-3 hours of work. Or would I still have to look at the references like Fair Strikes mentioned even though it's being done in the order I'm doing it?

[DarthParametric 3,777]

Or alternatively, if you (or anybody else reading) are proficient in C++, consider extending the xoreos tools with an xml2tlk program.

I'm a biologist, so good luck with that. As to anyone else, I call dibs on any C++ gurus coding a replacement for MDLOps first.

 

Take AIOFPM and use that as the "base" TLK

If the point is to create an append TLK for TSLPatcher, you'll need to compare it all back to the vanilla TLK at some point.

[Vriff 21]

Two of those mods change existing lines rather than add new ones, so you have to use some sort of differencing tool to find the changes.

I've written one in C++ for plaintext files, it's been a while but I might be able to convert it if I can understand the file structure.

[Fair Strides 509]

I've written one in C++ for plaintext files, it's been a while but I might be able to convert it if I can understand the file structure.

PM or Skype me and I can help with the file format.

[iMacThere4iAm 0]

I have followed this train of thought to what I believe is a successful conclusion, using the xml2tlk/tlk2xml programmes in the latest xoreos-tools 0.0.3 and ExamXML to diff those two mods against the original KOTOR dialog.tlk.

I still need to carry out some testing but will upload TSLPatcher versions here once I confirm it works correctly.

[DrMcCoy 40]

using the xml2tlk/tlk2xml programmes in the latest xoreos-tools 0.0.3

 

Ah, yes, in case anybody didn't notice: I did end up writing an xml2tlk tool myself, that can recreate a TLK out of an XML file created by tlk2xml.

 

It's able to write the versions 3.0 (NWN, NWN2, KotOR, KotOR2, The Witcher) and 4.0 (Jade Empire) of the TLK format. It does not currently support writing the GFF4'd versions 0.2 and 0.5 (Sonic Chronicles, Dragon Age: Origins, Dragon Age II) of the format, because those need a GFF4 writer first.

 

I only did a cursory testing of the produced files, though, and some simple binary diffing between original and recreated file. They should work, but I can't give a 100% guarantee. If you do find cases that do not correctly work, I'd be very much interested in samples that break, and descriptions how and where, so that I can fix the issues.