How many new rows can the appearance file have?

[N-DReW25 1,336]

Out of curiosity is there an actual limit as to how many new appearances can be put onto the appearance.2da file or is it unlimited?

[JCarter426 1,215]

Interesting question. My appearance.2da file is currently at 862 lines, so my estimate is greater than or equal to 862.

 

That puts us over 16-bit, so assuming it's 32-bit or greater (and not something weird) then we're dealing with more than 65,000.

[bead-v 251]

That puts us over 16-bit, so assuming it's 32-bit or greater (and not something weird) then we're dealing with more than 65,000.

 

Puts us over 8-bit, right? So it's 16-bit or greater, which is indeed at least 65,000.

[JCarter426 1,215]

Oh, right. Bah. Meant in 16-bit.

[N-DReW25 1,336]

Interesting question. My appearance.2da file is currently at 862 lines, so my estimate is greater than or equal to 862.

 

That puts us over 16-bit, so assuming it's 32-bit or greater (and not something weird) then we're dealing with more than 65,000.

Puts us over 8-bit, right? So it's 16-bit or greater, which is indeed at least 65,000.

Thank you both, I now have no need to worry about reaching a limit. Out of curiosity what happens if someone did magically go over 65,000? Does the game crash or what?

[JCarter426 1,215]

Well, I'm just guessing. The texture variations for the UTI files are stored in 8-bit, so they only recognize that 256 numbers exist - 0 through 255. If you try to input anything higher, it just rolls over back to the beginning, so it thinks 256 is 0, and so on. appearance.2da clearly goes higher than 256, so my guess is 16-bit, which would support rows 0 through 65,535. If you go higher than that? I don't know. Perhaps it would recognize no more entries. Or maybe it would crash the game like everything else does.

[ndix UR 218]

The number of rows is a full 32-bit unsigned int, so ~4 billion. However, AFAICT, that's not the limitation you would reach.

 

2DA files store offsets to each piece of data using 2-byte uint16's, the offsets are from the end of the offsets block to the start of the particular datum (think, one actual cell in the table). This means that you will run out of space when your table data reaches 64KB (and you want to add another thing beyond 64K). Everything is a string, so it completely depends on the content of your table. You could run out with a single row of 4 32K strings, for example. Alternatively, you could store a single row, single cell with 256K or more probably, because it's just one really long string from offset 0.

 

For perspective, my appearance.2da is 693 lines, and its minimum available offset is 19935, which means the file is 30.4% "full". My alienvo.2da is only 44 rows, but 16% full.

[JCarter426 1,215]

It's weird that they would use a 32-bit integer if, practically speaking, you'll never get as high as a 16-bit integer would get you anyway.

[DarthParametric 3,784]

Sounds kind of like an issue in Dragon Age Origins, where even though the toolset could address a theoretical 65,535 rows in a 2DA, a GFF bug/error meant that the ID field in UTIs was only 1 byte instead of 4 bytes, so the game could only address up to 255 rows.

 

http://www.datoolset.net/wiki/Bug:_High_M2DA_ID_ranges_might_work_in_the_toolset,_but_not_in_game

[ndix UR 218]

Another thing I noticed today about the format is that they 'de-dupe' the data values, so one piece of data referred to in many columns only appears once in the data section. So you actually have 64K worth of 'unique' data space, and then you can combine those data into ~4 billion rows max. So that makes a bit more sense. It also explains why they would use uint16's for the offsets, because in all the 2DA's I've looked at, the offsets actually make up the largest portion of data in the file (in bytes).

 

That would be the theoretical limit of the format. As DarthParametric recounted for Dragon Age Origins, I would be relatively shocked if any game's shipped 2DA handler was so correctly implemented as to let you take full advantage of the format.