Fixing the Dialog Skipping Issue in TSL

[lachjames 24]

Hi everyone

I've been working on KLE and some mods, and I've come across an interesting discovery: I think that the infamous dialog skipping issue in TSL is due to an issue with the "Delay" field in entries in the DLG file.

The Problem

It's well known that TSL sometimes skips through dialogs - this problem is historically fixed by restarting the game. Some examples of people reporting this issue include:
 https://steamcommunity.com/app/208580/discussions/0/882966056936924094/
 https://gamefaqs.gamespot.com/pc/920601-star-wars-knights-of-the-old-republic-ii-the-sith-lords/answers/21030-how-do-you-fix-the-dialogue-skipping (originally I thought this issue was related to the Steam version of TSL, but this report is from 11 years ago so clearly it was a problem in the base game too).
 Plenty of Reddit posts, but I don't want to embed them.

To be more specific, here's my train of thought:

1) Dialog entries (which are things NPCs say; contrast this with replies, which are what the PC says) have a Delay field, as well as a VO_ResRef field. My understanding is that, if the delay is set to 0, the entry is supposed to be immediately skipped UNLESS there is also an entry in the VO_ResRef field (or perhaps an entry in the TLK instead corresponding to the sound of the text, I'm not sure on this point yet).

2) For lines which have text and audio, the Delay field is sometimes set to 0, and other times set to 4294967295 (which is 2^32 - 1, or the largest possible 32-bit unsigned integer, aka the longest possible delay).

3) When the Delay field is set to 0, I have found that the dialog skipping occurs. I have found that setting the Dialog field to 4294967295 instead causes the skipping to stop.

This is difficult to verify, because the problem appears to happen at random. I've read speculation that this is due to a memory leak, but I'm not yet convinced of that - I think there is some unfortunate combination of events that cause TSL to no longer use the audio clip's length to determine the length of a dialog entry, but to use the Delay field instead.

The fact that alt-tabbing can cause the issue leads me to believe this might be some sort of "race condition", where two checks are supposed to be performed (one to stop the dialog if the Delay time has elapsed, and another to stop the dialog if the Audio clip's time has elapsed). I conjecture that these checks need to occur in one order, but due to stuttering/other unintended effects from alt-tabbing, etc., sometimes they happen out of order. Again, this is pure conjecture (and if I'm completely wrong, which I might well be, that doesn't invalidate the findings above and below).

The Solution

The only way to fix this problem is to go through every DLG file in the game, and set all appropriate Delay=0 entries as Delay=4294967295 instead. It wouldn't be difficult to automate this process using xoreos-tools, Python, and an hour of scripting. However, the issue is setting all "appropriate" entries, as some dialog entries are likely intended to actually have a delay of 0.

I am considering the following algorithm to solve this problem, but I'm not sure if it's foolproof:

For every DLG file in the game:
    For every Entry in the DLG:
        1) If the delay is non-zero, skip the entry
        2) If the VO_ResRef field is not empty, or the text referenced in the .tlk file has no sound, skip the entry
        3) If this point is reached, the entry has sound but has a Delay of 0 set. Set the Delay to 4294967295 instead.

I'm curious to hear thoughts/suggestions/issues with what I have written. Thanks

[DrMcCoy 40]

I'm not sure about the race condition theory, considering that the game isn't even, AFAIK, multi-threaded. I don't see how the checks could happen out of order. And how that would then persist for the next checks. Likewise, a memory leak also sounds implausible. Not that I have any better idea, mind.

I'm pretty sure there's even a lot of legitimate entries that have delay 0 that should be skipped: ones with only developer commentary. TSL has commentary in basically all of its dialogue files, mostly talking about how the camera is moved, the mental state of a character, etc. I think that's meant for the animators and voice actors? The commentary is enclosed in curly brackets and the game filters those out. Likewise, there's entries that exist solely for camera movement.

Can't say anything about your proposed changes, though. However, 0xFFFFFFFF is often used as a sentinel value of sorts, so it might be there's some other functional difference between delay 0 and delay 0xFFFFFFFF in the game. In either case, the solution should of course be properly tested.

Maybe even cross-checked with a disassembly of the game, or even attaching a debugger like OllyDbg while the game is running (and/or even already in the buggy state). That is, however, time-consuming and complex.

[Thor110 499]

Currently I have changed entries to match the specified delay for the kodin.dlg that is widely known for doing this, I will continue to test it over and over again to see if it occurs however I could not get it to occur beforehand anyway.

As this other issue is dialog related that I have been trying to solve for sometime I thought I would post about it here as well to hopefully get some discussion going on both dialog related issues.

The above image shows the issue which sometimes happens in levels, recently I came across a custom level that was reproducing the bug 100% of the time when entering the level and I compared it against another .dlg file ( kodin's actually ) and discovered that it was missing the NextNodeID, adding this in for the custom level fixed the issue of the top part of the border not coming down.

Removing this entry, would also cause it to happen in the custom level, adding it back in would also fix it and I verified this multiple times.

I have come across a few levels in the main game that do this almost every time, for me at least this is the Harbinger Sion Cutscene and the dialog with atton when you reach the Hanger on Peragus and he complains about it being locked and possibly a few others.

Considering that it is now a reproducable bug as well as a fixable bug, at the least for a custom level, I am almost certain that this can be fixed anywhere else it can be found throughout the game. ( however so far I have not had any luck with this level )

I will also be documenting any dialogs I come across that have the skipping issue and testing the proposed changes for those.

Thor110

Update :

Though I have not been able to fix it with the above level yet, I have fixed it in another main game level. It seems having the wrong NextNodeID set ( which I believe is meant to be 0 or 1 or whatever the starting NodeID is to the .dlg file ) causes this issue.

In this dialog it was set to 25, setting it to 1 which is the NodeID of the first line of dialog Kreia speaks fixed it.

Hopefully this can lead to all of these dialog issues being fixed for the game.

Quote

Can't say anything about your proposed changes, though. However, 0xFFFFFFFF is often used as a sentinel value of sorts, so it might be there's some other functional difference between delay 0 and delay 0xFFFFFFFF in the game. In either case, the solution should of course be properly tested.

Is it not possible this value is already recognised as something in the game?

[lachjames 24]

Thanks @DrMcCoy and @Thor110 for the discussion

@DrMcCoy I should clarify, when I say race condition I mean that the checks are done in a different ordering on the same thread. The fact that alt-tabbing tends to cause the issue is a clue, since some game systems deactivate when alt-tabbed and others don’t (I think). This is all conjecture though.

I’m unsure if 0xFFFFFFFF is a sentinel, the default large value, or even just “-1”, to signify that the delay should be from the audio (making it a sentinel of sorts).

The algorithm I suggested could be modified to check for and ignore dialog entries with camera animations, or camera angles (certainly the former, perhaps the latter). As you say, it’ll need some testing.

@Thor110 Interesting that you’ve found other dialog issues as well with the base game content - some sort of “automatic dialog fixer” might be useful. Perhaps, if it works, these fixes could be integrated into TSLRCM?

I think that 2DA file refers to XP requirements for different levels (so level 51 has the highest possible requirement, making it effectively impossible to reach). Don’t think that in particular is the link, but you and @DrMcCoy are right that it could have a meaning of some kind.

[ZeldaTheSwordsman 40]

Does the bug occur when running the game on a one-core PC?

It's absolutely a fault in the base game, since it also sometimes happens to the Xbox version. Had it happen to me in my current playthrough actually, cheating me of a chance to garner influence with Visas Marr (when you give the fleeing mother on Onderon a starport visa, she reacts to it in a dialogue scene where the right response bolsters influence; the scene started but cut off before Visas' lips even had a chance to move). One thing I'm not sure of is if it happens more often when playing on the 360 (which I was) compared to an original Xbox.

Also had a possibly-related bug when I tried to gain influence with Disciple by giving Xaart a visa. Disciple spoke Mandalore's "I don't like the idea that he's working with them now" line instead of what he's supposed to say, and no Influence was gained (the latter might be because I'm maxed; don't know). Anyone else ever seen that?