In a very real way, when you create a mod and release it onto a site, you're putting it up for others to enjoy and share around with their friends and enjoy some more. Some mods are small, some big. Some are stupid and funny, some are smart and amazing. Just as the mods come in all shapes and sizes, so do the modders who think them up, plan them, and put a lot of time and effort into making them happen.
When you release something to the public, you as the author have a right to set restrictions on how this file can be used. You could be perfectly fine with others uploading it to wherever they want, or you could decide that you don't want the possibility that someone will upload one of your mods that you're proud of and put so much time and effort into somewhere else without your knowledge and then saying it's their work.
Another real thing is the Credits section of a mod's read-me. If a modder had to get help from another modder (such as reskinned model, a few scripts, or a lot of dialog and audio), then it's not just the author's work being distributed; the modder that helped them is having their work sent out as well, which is understandably a big issue if the author had to do something special or got very strict restrictions when they received the files themselves.
Personally, I release my mods where I know they'll be appreciated. I know that they won't be misused and I can have a reasonable feeling of security that they won't get ripped off, torn apart, and sewn back together as someone's creation...