Jump to content


Photo

[TOOL] KotOR Stuff


  • Please log in to reply
5 replies to this topic

#1 ApanLoon

ApanLoon

    Jedi Initiate

  • Registered
  • Pip
  • 8 posts
  • LocationSweden

Posted 17 July 2017 - 07:28 PM

I started to work on a set of small tools for extracting assets from KotOR. These tools are not in any way competing with more complex tools such as "KotOR Tool" or "KotOR Toolset". Those focus on giving a complete experience - a complete solution for all your modding needs. KotOR Stuff does the opposite.

 

KotOR Stuff provides several tiny command-line tools that each perform a small task. With scripting, these tools could be used to simplify batch processing or support other tools that perform more complex tasks.

 

Currently three tools are included:

  • bif - Extract assets from KeyTable/BIF repositories
  • erf - Extract assets from ERF/MOD/SAV and HAK repositories
  • tpc - Convert TPC files to PNG. Use command line options to select which side of a cubemap, what frame of a cycle animation or which mip map to convert. Can also list TXI attributes if any are present.

Future releases of this project may include:

  • Creating new BIF/ERF/MOD/SAV and HAK repositories
  • Adding resources to BIF/ERF/MOD/SAV and HAK repositories
  • Convert PNG to TPC given additional information on the command line
  • Improved TPC support: Even though the tpc tool can correctly convert some TPC types that "KotOR Tool" can't, some image types are still not handled correctly. For example, cube maps with only four sides exist but can't be converted yet.

Future releases of this project will probably not include:

  • Pretty GUIs that makes modding a breeze for anyone (That is what "KotOR Tool" and "KotOR Toolset" are for.)

C# source code can be found here:

 

https://github.com/ApanLoon/KotORStuff

 

Binary version for Windows (might also work with mono on other platforms) can be found here:

 

http://deadlystream....28-kotor-stuff/



#2 VarsityPuppet

VarsityPuppet

    Jedi Master

  • Premium Member
  • 1,986 posts

Posted 18 July 2017 - 12:51 AM

Ahhh a man after my own heart, with all of that C# code. I'll take a closer look later, but I'm happy just to see this :D
  • LDR likes this

Be careful how you think; your life is shaped by your thoughts. - Proverbs 4:23


#3 ApanLoon

ApanLoon

    Jedi Initiate

  • Registered
  • Pip
  • 8 posts
  • LocationSweden

Posted 18 July 2017 - 03:39 PM

UPDATE: v0.3

 

The tpc tool can now convert PNG files to uncompressed TPC files.

 

The following example creates an uncompressed RGB file with two sub images (One full scale and one half-scale mip map) and a couple of TXI attributes:

tpc -o tpc -s2 --outpath PO_PFHA3d.tpc -txi="proceduretype cycle" -txi="channelscale 4 0.1 0.2 0.3 0.4" PO_PFHA3d.png

Note that it might still not be possible to create cyclic procedurals with this tool, I just added those TXI attributes to illustrate how it should be done.

 

To get some more info about what TXI attributes are used in game and how they need to be formatted for this tool, use the following command to examine an existing TPC file:

tpc -o none -t PO_PFHA3d.tpc

Run this on one, some or all the files in the texture pack to see what's in there.



#4 VarsityPuppet

VarsityPuppet

    Jedi Master

  • Premium Member
  • 1,986 posts

Posted 18 July 2017 - 04:25 PM

UPDATE: v0.3

The tpc tool can now convert PNG files to uncompressed TPC files.

The following example creates an uncompressed RGB file with two sub images (One full scale and one half-scale mip map) and a couple of TXI attributes:

tpc -o tpc -s2 --outpath PO_PFHA3d.tpc -txi="proceduretype cycle" -txi="channelscale 4 0.1 0.2 0.3 0.4" PO_PFHA3d.png
Note that it might still not be possible to create cyclic procedurals with this tool, I just added those TXI attributes to illustrate how it should be done.

To get some more info about what TXI attributes are used in game and how they need to be formatted for this tool, use the following command to examine an existing TPC file:
tpc -o none -t PO_PFHA3d.tpc
Run this on one, some or all the files in the texture pack to see what's in there.

Okay, now this stuff is legit. I will be looking very closely at this code. I've been wondering about this for years, but never took the initiative to look into it. Thank you for your hard work, and I'd be honored if you continued to share more of it in the future. If I can learn anything from it I'd love to contribute as well

I hope God blesses every single one of you toolmakers. ( You know who you are ;) )

Be careful how you think; your life is shaped by your thoughts. - Proverbs 4:23


#5 ApanLoon

ApanLoon

    Jedi Initiate

  • Registered
  • Pip
  • 8 posts
  • LocationSweden

Posted 18 July 2017 - 09:37 PM

Cool. I am happy that you like it. :D

 

There are still some types of TPC files that this tool can't read and there are some questionable assumptions in the current code. I would appreciate any and all assistance making the code better and more complete.

 

I was told that TGA support is kind of important, so I might look into ways to add that. The library I use can't do TGA - only BMP, EMF, EXIF, GIF, ICO, JPEG, PNG, TIF and WMF. Adding more of those would be trivial, adding TGA is a bit of work but not too complicated, I think.



#6 ApanLoon

ApanLoon

    Jedi Initiate

  • Registered
  • Pip
  • 8 posts
  • LocationSweden

Posted 19 July 2017 - 01:25 PM

Assistance requested:

 

This will be a long post. It contains some tips about how to use the tpc tool but also questions about further development of the tool.

 

I just added the possibility to extract the TXI to a separate file when reading a TPC and conversely, to read the TXI from a separate file when creating a TPC. However, the command line options are a bit weird at the moment. I would like to get feedback on how these should be changed to make them more intuitive.

 

For the rest of this post, I will be talking about the current development branch on GitHub and NOT the currently released version. It behaves differently in regards to TXI.

 

The current help info looks like this:

$ ./tpc -h
Usage: tpc.exe [OPTIONS]+ [FILE]+

Operate on files in the Bioware TPC format.

Options:
  -n, --nofile               Don't output to a file, instead use stdout.
  -o, --output=VALUE         The format of the output file(s). Possible
                               options are: png, tif, raw, hex, tpc, none
  -i, --image=VALUE          The index of the image to extract.
  -s, --subimage=VALUE       The index of the sub image to extract or the
                               number of sub images to generate in new TPC.
  -t, --txi[=VALUE]          Dump the texture info or add TXI attribute to
                               new TPC. When adding an attribute, use the form -
                               -txi="key value..."
      --txipath=VALUE        Create this separate TXI file when reading a TPC
                               or read this separate TXI file when creating a
                               TPC.
  -c, --compress             Use DXT compression when creating new TPC.
  -f, --format=VALUE         Pixel format for new TPC. Possible options are:
                               Gray, RGB, RGBA, BGRA
  -p, --outpath=VALUE        Path and file name of output. Default is the
                               same as the input with a new extension.
  -h, --help                 Show this message and exit

Unless the nofile option is given, tpc.exe prints information about the file
on stdout. This example explains what is shown:

zc   64   64  6  7 42 RGBA         1 CM_rakata.tpc

  Flags:                      z=DXT compression, c=Cube map

  Width Height:               Of the full-size version of the image(s).
                              Each additional sub image, if any is reduced
                              in size by half width and half height.

  Number of images:           The number of full-size images. Cube maps have
                              six images.

  Sub images per image:       The number of sub images for each image.
                              The first is the full-size image, subsequent
                              sub images are mip maps.

  Total number of sub images: The header of some files indicate more sub
                              images than there is data for in the file.
                              This shows how many sub images there actually
                              are.

  Pixel format:               Gray, RGB, RGBA, SwizzledBGRA

  Unknown1:                   Float value of unknown purpose.

  Full path to the file

I realise that not many of you have even tested this yet, but I would like to have a cleaner way to tell the tool what type of output to generate and where to put that output.

 

The tool basically outputs three things: Info, Image and TXI

 

I think that he most common usage would be to output the image and TXI to files and print the info on stdout. Currently, this is what happens if you, for example, run this:

tpc -o png -t PO_PFHA3d.tpc C_HoloDodonna.tpc

The new files will be named PO_PFHA3d.png, PO_PFHA3d.txi, C_HoloDodonna.png and C_HoloDodonna.txi respectively and the info for the two TPCs are printed on stdout.

 

It is possible that, in some situations, you would want the image data to be sent to stdout instead of a file. An example of this would be if you want to rotate an image with something like ImageMagick before storing the file:

tpc -n -o png CM_HawkBay.tpc | convert - -rotate 90 CM_HawkBay.png

I don't know if this works in Windows "cmd" application, but I do know that it works in "bash" on Linux, OSX and Cygwin under Windows.

 

You also might want to filter stuff out from the info:

tpc -o none *.tpc | grep "RGB "

only lists the info for RGB files. Note the extra space at the end of the grep to make sure that it doesn't also match RGBA.

 

We can also get the TXI on stdout:

tpc -n -o none -t C_HoloDodonna.tpc | grep proceduretype

When you specify that the tool should not send output to a file with -n, the tool won't output the info as that would interfere with the image or TXI output. "convert", for example, would fail to read the png if the data also contained the text of the info.

 

Now we arrive at the more complex situations. What if we want both the info and the TXI on stdout? Like the currently released version does with -t?

It would be nice to see what TXI belongs to what TPC file when listing more than one. We might try:

tpc -n -o none -t *.tpc | grep proceduretype

But this doesn't do exactly what we want. From the result of this we will get a list of all proceduretype attributes, and that could be useful too, especially if you add " | sort | uniq" to the command line. However, we won't see which file has which attribute and that would probably be useful. There is no way to do this yet.

 

Question 1 to you is: What options should we have to make this happen?

 

Note that, when working on a lot of files, you might get an error such as this:

-bash: ./tpc: Argument list too long

If that happens, you should embed your command in a "find":

find AllTPCs/ -exec tpc -n -o none -t "{}" "+" | grep proceduretype | sort | uniq

The "+" will make find call tpc multiple times with an optimal(?) number of files each time.

 

Question 2 to you is: Are there other combinations of outputs that would be useful? Sending TXI to a file while sending the image to stdout or sending the info to a file, or...

 

Question 3 to you is: What additional options do we need? One I know already is that we should have a way to specify the unknown float that is put into new TPCs. Currently, this simply defaults to 1.0.

 

Question 4 to you is: What features/bugs need to be addressed and in what order? I.e. What features are most important to you?






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users