Porting Hax

[JCarter426 1,216]

With the change in porting rules now firmly in effect and the interest in porting up to and including an entire game's worth of content obvious, I've started poking around to see how things can be done in the long term. As it is now, we have a smattering of individual mods that include ported content, which is all well and good, but this depends on individual modders with the required knowledge producing and releasing mods at their own leisure. As with a lot of things in the modding world, a lot of people are interested, but far fewer know how to do it. So I'm interested in automating the process in order to port content on a much larger scale  - for example, making every module in K1 available in K2 - and release this content as mod resources that will be as easy to access as anything you can extract from KOTOR Tool currently, or perhaps even allow for the creation of an automated porting tool that the average end user can run. The latter case would apply even to stuff that was allowed under the old rules, like a lightsaber mod that was done for one game but not another. In general, I'm looking to make things easier and require less manual tinkering.

Even before the rules change, I experimented for my own personal use. To make things easier for me, I started extracting files en masse rather than bothering to figure out what I needed for each specific job, given that this was for personal use and not release. Once the rules changed and I was free to distribute these, though, it became clear that this was not ideal. My modified texture packs that included all of K1 and K2's textures amounted to about 800 MB of data. It didn't seem reasonable to include all that in an official upload and put a strain on the servers. Plus, this is the sort of thing I've criticized the Apeiron team over, so I figured I should tread carefully and look for a better method. A more ideal situation would involve a tool that could extract assets from one game and send them to another. I started writing some scripts for this, though I've had mixed results and what I really want involves charting into some unexplored territory. Ideally, this new tool would generate a new BIF or ERF archive - portedmodels.bif or what have you - rather than requiring thousands of files to be placed in Override and all the potential file conflicts that come along with that. Ideally, these will be made as universal resources like the existing models and textures - that way people can still edit them and put their edited files in Override without having to overwrite any files.

I've spoken with @VarsityPuppet about developing something like this in the long term, and I'm also interested in seeing whether certain porting processes can be automated with MDLOps or MDLEdit, so I imagine we'll be talking about those eventually. In the meantime, I'll also be posting my own, more hacky results. I've included two such attachments in this post.

K1toK2 Texture Conflict Guides

K2 has hundreds and hundreds of files that were reused from K1. It's not necessary, and generally not a good idea, to try to copy these over. Not just because they're unneeded, but because files with the same names might not actually be the same textures. There's a very good chance Obsidian altered them and kept the original names. So I've created these guides that list the names of textures that are included in both games. There are three lists: for textures.bif, swpc_tex_gui.erf, and swpc_tex_tpa.erf. (The last list also applies to to _tpb.erf and _tpc.erf.) There are two sets of lists: one that lists conflicts, and one that lists files that are only in K1.

K1toK2 Texture Filters

These are simple batch scripts that will delete all textures in a folder that are known to be in both games. That way you can extract a game's textures, run the batch files, and filter out the textures that are already in the other game.

1. Extract the contents of textures.bif, swpc_tex_gui.erf, and swpc_tex_tpa.erf from K1 with your preferred method and place them in the same location as the included batch scripts, or you can put them each in their own folders if you prefer. You should have thousands of TPC files and a handful of TGA and TXI files.
2. Run the three batch scripts. These will delete all the conflicting texture files, leaving only the ones that are unique to K1. The batch scripts will delete themselves when finished.
3. Copy the remaining textures to K2. You can put them in Override (or a subfolder in Override) or you can repackage the ERF texture packs to include both the original K2 textures and the extracted K1 assets. Note that if you do want to put them in ERFs, you will have to do _gui and _tpa separately, and you will also have to repeat the process for _tpb and _tpc using the _tpa script if you want to include support for the lower quality texture options.

For the first version, I've specifically filtered out every file that has a matching file in K2. Eventually, I want to confirm which (if any) textures Obsidian modified, so we can include both game versions by renaming one copy. If you use my filters and start porting K1 content to K2, you won't run into any missing textures, but they might not necessarily look as they did in K1, at least for now.

K1toK2 Texture Conflict Guides v1.zip

K1toK2 Texture Filters v1.zip

[JediArchivist 70]

This is great work, can't wait to try the script and export some modules from KOTOR 1 to KOTOR II TSL. Many Kudos to you for figuring out the module porting!

I'm sorry if i am posting this question in a wrong place. I am currently trying to port all modules from KOTOR II TSL to KOTOR 1, and convert them to work with the older version of the Aurora Engine and then make them install and work as a mod, with the purpose to be able to play KOTOR II as a total conversion mod for the Mobile (Android / IOS) version of KOTOR 1! This is sort of the oposite thing from what you are doing (instead of porting the modules from KOTOR 1 to work with KOTOR II TSL, i am trying to get them to work vice versa), but it should work similar...

Could You tell me how exactly I need to convert the KOTOR II TSL modules and textures, in order to make them work in KOTOR 1. Since I am new to modding, I would need to know what tools i should use, what settings and what should i do exactly (with a given example), what file do i need to open with what tool and how should i export it, if possible...since there was no Porting tutorial written because it was not allowed at the time the modding tutorials were written. If You could also give me some advice as how i could to archive all the files from KOTOR II TSL in order to make them install like a mod (in the Override folder, etc.) and launch as a total conversion mod for KOTOR 1 it would be awesome for me and the K2M - KOTOR II Mobile project!

I give You my gratitude and many thanks in advance for taking Your valuable time to review my question!

Happy Hollidays!

Edited December 22, 2018 by JediArchivist

[Thor110 502]

Very useful information here, thank you  I have recently started work on porting some maps myself funnily enough and have also been trawling through both games files, it seems that dor_lyv01 the door for yavin is in both games, but I am unable to get it to load for the moment, either that or kotor tool didnt properly update after I deleted the file from the override folder and restarted kotor tool. ( even though I saw it reload the libraries as it does every startup )

Will definitely come in handy ^^ and the batch tools could also prove to be useful for mod development.

Great work!

[JCarter426 1,216]

On 12/22/2018 at 9:46 AM, JediArchivist said:

Could You tell me how exactly I need to convert the KOTOR II TSL modules and textures, in order to make them work in KOTOR 1.

For the textures, you need to replace all the files in the TexturePacks folder. Though I believe you will still get some hangups with the GUI on account of name changes. All I can suggest is you do that first, then eyeball what is still wrong and continue from there.

A module includes the following files:

• all the contents of the .rim and _s.rim files (these can be extracted with ERFEdit)
• the module's layout (.lyt) file (located in layouts.bif)
• the module's visibility (.vis) file (located in lightmaps.bif)
• the map (located in swpc_tex_gui.erf)
• the loadscreen (also located in the _gui.erf)
• the area models (.mdl, .mdx, & .wok, located in models.bif)
• the lightmaps (located in lightmaps.bif)

If you are porting the whole texture archive, that should take care of the map and loadscreen already.

The models must be converted because the games use slightly different formats. My usual procedure for this is to extract everything, convert the binary MDL/MDX to ASCII, then load the ASCII files and convert those back to binary. This isn't foolproof, though, as you can see here . I've put my area porting endeavors on hold until the next update to MDLEdit (plus I've got other projects anyway).

Everything else can be archived in a new .mod with ERFEdit and then placed in the modules folder.You can put modelsand textures in there too, but they are often used for more than one area, so it's not as practical. Like I said in the first post, ultimately I would like to explore a way to create new archives for the games rather than shunting everything into Override. We've had some progress, but there's still more work to be done.

[Thor110 502]

37 minutes ago, JCarter426 said:

The models must be converted because the games use slightly different formats. My usual procedure for this is to extract everything, convert the binary MDL/MDX to ASCII, then load the ASCII files and convert those back to binary. This isn't foolproof, though, as you can see here . I've put my area porting endeavors on hold until the next update to MDLEdit (plus I've got other projects anyway).

I figured this must be the case but it's great to have confirmation!

Any tips for converting back into binary?

[JCarter426 1,216]

Just that with MDLEdit, you have to change the game before importing - basically it does all the work when reading the model. So you should load the binary model in the original game and convert to ASCII, then change the game, then load the ASCII, then convert back to binary.

[Thor110 502]

For a little while I was unsure exactly what you mean, but the steps I need I think are as follows.

1 Extract Binary with KotoR Tool.
2 Load Binary from MDLOps with the kotor game it came from highlighted.
3 convert from binary to ASCII with the same game highlighted.
4 Change the selected kotor game in MDLOps to the one I wish to port the map over to and convert back into binary.

I think I understand now. Converting out of K1 binary into ascii and then back into binary K2 can read.
Thanks for help, I hope I understand / the way I have said it makes sense.

Thor110

[JCarter426 1,216]

Those steps were for MDLEdit. MDLOps doesn't have the same requirement, though the same basic procedure applies.

[Thor110 502]

Ah I see the confusion now, thank you very much once again for your input and great resources / information, I have just downloaded MDLEdit now, I thought I already had it along with all the other tools, but seems I only had MDLOps and ended up a little unsure whether or not they were even different programs, due to seeing MDLEdit / MDLOps referred to everywhere.

Hopefully now I can make a bit of leeway on my project, should definitely make things less frustrating :D

Have you had any luck porting doors? I have seen a tutorial on creating entirely new doors & animations, so it shouldn't be difficult / isn't impossible, unfortunately KotoR tool seems to corrupt the utd file for me and add extra entries.

After editing every entry I could in K-GFF editor, it still doesn't change the door type, opening the utd in kotor tool shows why, there is a drop down / list that allows me to choose the door type from the genericdoors.2da file, however it is set to Peragus Door 01 instead of Yavin Door 01, they both exist in TSL's files, Yavin Door 01 is still there as #76, what was originally the end of the list for the first game.

However I cannot seem to find an entry in K-GFF editor that refers to either #76, #77, related string references or PeragusDoor01/YavinDoor01, so it appears to me the field is unreadable through K-GFF editor but corrupts through Kotor tool, so I am seeking viable alternative.
Any thoughts?

I still have a whole bunch of kotor related tools that I haven't even tried yet, so it could very well be that I am using the wrong program, but I feel like I am taking the right path with the wrong tools, unless it is just a kotor tool bug / problem.

Thor110

[JCarter426 1,216]

The Yavin door appears to use line 64 in genericdoors.2da for K1. If it's 76 in K2 and all the other files are there, then you should only have to change it to GenericType 76.

[Thor110 502]

Ah you have solved my problem just with the mention of GenericType, I kept looking through the file for what might be a reference to the door numbers but couldn't work out which was which unless it was open in kotor tool, came to the conclusion it wasn't even there because of the way I could see it fine in kotor tool but it didnt stand out to me in K-GFF editor for some reason, probably spent too long staring at it.

You've been a great help though if anything I am annoyed at myself for clearly over-looking the solution for the door, being unfamiliar with kotor modding and not knowing what GenericType was for some reason I didn't make the connection to it being GenericDoors.2da key reference for an item within the list.

Big thanks to you the door now works fine, I can go about correcting all the door files / locations and carry on with my project.

Thor110

I am currently porting Manaan as well, have managed to port 26aa, 26ab, 26ac, 26ad and 26ae so far, but a, b, c & e all seem to crash regularly, d, does not, perhaps it is because I did that one first / followed a set of steps but have been making sure to look around / check for any leftover textures or files I might be missing.

Probably another simple thing I am overlooking because I am ridiculously tired, will take another crack at them tomorrow, AD the hanger loads / players fine, all the doors work, animations work etc, skyboxes aren't there and some textures appear transparent but for the most part it's there.

All I can say is wish me luck as I am bound to keep encountering regular problems  half the fun though!