Actually, reading through the dialogue, it doesn't say what he did.
It only says he "didn't follow her archaic definition of marriage," which could mean anything ranging from not following some cultural/societal marriage norm to cheating on her.
I read about every line, in both Tanis and Wife dlg files, and there is nothing saying what he did.
He flirts with the PC if she's female, so I can see sort of where you are coming from about him being a womanizer. But there's nothing to suggest physical abuse, unless I missed something, nor is there any way of telling if he cheated on her.
It's implied that he ruined his relationship, but I don't quite understand how that should result in a death sentence. She did try to kill him!
Besides, what did Bastila say about Revan? "Jedi don't kill prisoners, no one deserves death.. " Seems a bit hypocritical of Bastila, doesn't it? Malak was right about her!
And why five options? To give the player a choice, sometimes the line " I'll kill you " can become very stale. People want options, and various ways of doing/saying things, it's a writing strategy that fits very well with kotor and RPGs in general.
Edit: you do have a solid point about a Jedi killing someone. I did add the option to deliberately let her go, and HK-47 comments on this. You might want to check the mod out, and then let me know what you think of it. I'd be happy to hear your feedback!