New Area Knowledge Thread

[Sithspecter 909]

New area creation is something of an experiment and hasn't been discussed thoroughly. The purpose of this thread is to serve as a knowledge collective of new area knowledge, and discuss what we know and don't know, and what we can and cannot do with areas at this point. I hope this thread will allow us to share our techniques and eventually allow me to write a comprehensive tutorial on new area creation.

For the purposes of this thread, "new area creation" refers to modeling a brand new area, not simply a module reskin. Additionally, this thread does not serve as a tutorial on how to use 3DSMax or GMax, though various techniques and functions may be discussed.

We've been able to create new areas for a while, ever since Magnus II released KAurora, which enabled us to compile new walkmeshes. KAurora has added several functions, including adding emitters and lightmaps.

1. We can create newly modeled areas

This one is fairly obvious, but I felt the need to include it anyways. As several members on here have already done, newly modeled areas are entirely possible. However, a new area isn't something that happens overnight. A new area takes careful planning, consideration, and a lot of hard work. Making a new area on par with existing KotOR areas will take several months of work. Area creation also entails a good amount of troubleshooting, most of which will have to be done on your own.

2. We can create emitters for our areas

KAurora also has the ability to export new emitters, for use in areas and other applications. To add an emitter, you can start by creating one from the helpers tab in Max. I have found that the best way is to base your emitter off one from the game. Pull a model with a similar emitter up in KAurora, and fill in the fields as closely as possible, changing only what you need. Additionally, not all of the fields are visible in Max, so you'll have to work on your emitters in KAurora as well. I have found that creating a model for each emitter and including them in the layout and .vis is the most hassle free. Additionally, I've had trouble getting the orientation fields set correctly. The ones that export from Max seem to screw it up when they are set at angles. For instance, when I create the steam emitters that come from the bottom of the Ebon Hawk, I always copy the orientation from the landing module of an existing planet, and everything is good. Emitters take some fine tuning, but can add a lot of detail to an area.

3. We can add lightmaps to new areas

I'd say that given the difference between lightmapped and non-lightmapped areas, it's almost not even worth it to create an area if you don't intend to lightmap it. In my opinion, lightmapping accounts for about 60-80% of how good an area looks. Quanon's created an excellent tutorial on lightmapping, and it covers most everything pretty well. Quanon and I have exchanged quite a bit of discussion on lightmaps, and we've each found different techniques that work. Quanon has found that it's best for him to do one massive lightmap of his whole area. I have found that ti's best to split my areas up into small sections before lightmapping, and doing multiple lightmaps. I've also found that lightmaps are a tricky and buggy portion of the process. They take a long time to get right, and sometimes you have to start over it it's not applying correctly in the game. When finished, they are spectacular.

 

4. We can add lights to new areas

 

We've known since the beginning that AuroraDLights were possible in KAurora, though there is some confusion surrounding them. Hopefully I can help clear some of that up.

 

In the days before lightmaps, ignorant people such as myself thought that all the lighting came from AuroraDLights and Aurora Lights. It was thought, at least by myself, that AuroraDLights and Aurora Lights were 2 different things. AuroraDLights were possible to create through NWMax, but imported areas from the game through the old MDLOps 0.5 showed mostly blank dummies called Aurora Lights. New evidence from areas run through MDLOps 0.6a shows that all of these are actually AuroraDLights. So yes, AuroraDLights are possible. Aurora Lights aren't real things.

 

Lightmaps provide the lighting for an area. AuroraDLights are meant to create shadows and give light to dynamic objects such as creatures, placeables, doors, etc.

 

Quanon seems to be correct in that only 3 can light up at a time. However, you can have far more than 3 AuroraDLights in any given area. I think a common misconception used to be that there could only be 3 AuroraDLights. This is false. Use as many as you want.

 

A much more in depth explanation of AuroraDLights can be found here.

 

5. We can create minimaps for our areas

This one is a minor addition, and doesn't really have to do with KAurora. But, with some trial and error and a top-down render of the area, it's entirely possible to create a minimap for your area. It's a nice touch to finish up your area, even though it's not entirely necessary.

 

Here is a quick synopsis of my findings with K1 minimaps:

 

I was dreading working on the minimaps. I did one for the main street a while back, and it was a 4-6 hour trial and error process that was annoying as all get out.

 

So, I pulled up another module, rendered the top down view, and took it to Photoshop. Opened up the .are, and got ready to begin the process again. I knew that the fields in the .are were based on knowing the precise pixel coordinates on the 512 by 256 TGA of the map, and referencing actual meter coordinates of the area. However, last time I tried this, something didn't add up.

 

So I have 8 different values I have to figure out in the .are to get the map and the scaling right, based on 2 (x,y) points on the TGA and the coordinates of the area.

 

MapPt1X (TGA reference of X coordinate #1)

MapPt1Y (TGA reference of Y coordinate #1)

MapPt2X (TGA reference of X coordinate #2)

MapPt2Y (TGA reference of Y coordinate #2)

 

And

 

WorldPt1X (Meter value of X coordinate #1)

WorldPt1Y (Meter value of Y coordinate #1)

WorldPt2X (Meter value of X coordinate #2)

WorldPt2Y (Meter value of Y coordinate #2)

 

I picked out two references I could see visually on the TGA of the map, and got their meter coordinates in MAX. Filled in the WorldPt values, which was the easy part.

 

So I had figured out that the MapPt values were decimals based on how far from the top left corner the pixel representing the coordinates I just found was. For example, if the pixel was halfway to the bottom, the MapPtY for that pixel would be .5. That seemed to work fine for the Y-axis. I would figure out how many pixels from the top the pixel was, divide by 256, then I'd have the right MapPtY. The weird thing was the X-axis. This was off for the X-axis for some inexplicable reason.

 

However, I finally discovered that while the textures are all 512 by 256, the game only uses ~435 by 256 of the texture. By diving the X pixel values by 435 instead of 512, the map worked perfectly!! I did several maps right in a row, making up for the sustained time loss fighting the Great Walkmesh War of 2016.

 

6. We can use walkmeshes that are divided into multiple parts

Quanon probably has more to say about this, as he's done it and I haven't. But it appears that when you divide the walkmesh into multiple parts, you can rig it to work by including a doorhook line in the layout file with the coordinates of the seam between the two. This should enable us to create areas that utilize the full functionality of the vis file and multiple AuroraDLights.

7. We can block line of sight and blaster fire with walkmeshes

When I looked at the existing walkmeshes in the game, I discovered that they had "walls". By extruding the edges of the walkmesh, and raising them up, you can create walkmesh "barriers" that block line of sight and blaster fire. The polygons of these walls need to be assigned the Non-Walk material. There's not much else to say about this, but it works well. I guess the main lesson is to use the existing KotOR and TSL models as your guide.

8. We CAN block camera movement by using the walkmesh injector!

This one has plagued us from the beginning, and I'm not sure we can do it with the tools we have. I have determined that the camera is blocked not by the walkmesh, but by the model itself. This makes sense, as things such as characters still block the camera even though they don't have walkmeshes. I'm very open for any discussion into this, as it's one of the last features that existing areas have that we can't duplicate.

 

We can block the camera movement through walls by using Dastardly's Walkmesh Injector!!

9. We cannot create area animations through KAurora

KAurora currently can't handle animations, but there are several ways around it. Placeables can still be animated through MDLOps, and included in the area. Additionally, it's possible that you could use a faux character model in the same manner. I haven't found the need to experiment much with this yet, but others may want to use it in their areas.

[Fair Strides 510]

In addition, I have learned that KAurora apparently does support Lights, which are called Aurora DLights in 3DSMax and GMax.

 

This allows us to actually use light-sources similar to the floor lighting used on Taris' walkways or the Leviathan, as well as the normal lighting from something like a lamp or wall-light...

[Dastardly 31]

The cause of camera movement being blocked (in area models) does technically come from the walkmesh but here is the plot twist; it comes from the walkmesh node INSIDE the .mdl file. It specifically comes from the AABB tree. You can see for yourself: Find a area model in a selected module, remove the AABB tree (have to replace the first AABB with a AABB assigned to a face, otherwise it crashes), then add the .mdl to the override, proceed to load the module and pan the camera through the walls - just make sure you are in the right model of the module. Now we just need to put this information to good use...

[VarsityPuppet 1,085]

The cause of camera movement being blocked (in area models) does technically come from the walkmesh but here is the plot twist; it comes from the walkmesh node INSIDE the .mdl file. It specifically comes from the AABB tree. You can see for yourself: Find a area model in a selected module, remove the AABB tree (have to replace the first AABB with a AABB assigned to a face, otherwise it crashes), then add the .mdl to the override, proceed to load the module and pan the camera through the walls - just make sure you are in the right model of the module. Now we just need to put this information to good use...

If we can figure out the camera collision on walls... I may have to try my hand at new areas

[Sithspecter 909]

The cause of camera movement being blocked (in area models) does technically come from the walkmesh but here is the plot twist; it comes from the walkmesh node INSIDE the .mdl file. It specifically comes from the AABB tree. You can see for yourself: Find a area model in a selected module, remove the AABB tree (have to replace the first AABB with a AABB assigned to a face, otherwise it crashes), then add the .mdl to the override, proceed to load the module and pan the camera through the walls - just make sure you are in the right model of the module. Now we just need to put this information to good use...

 

 

How exactly are you removing the AABB tree? Are you extracting the model and running it through KAurora or MDLOps?

 

I'm extremely interested to hear more on this subject.

[Dastardly 31]

Alright, so a AABB node is 40 bytes in length in the .mdl (compared to 44 bytes in the .wok). The node type for walkmeshs is 545. It turns out you just need to change the 40 bytes of the first AABB into null bytes using a hex editor and the AABB tree won't be formed. The first AABB is from offset 0 into the AABB tree which can be found using MDLOPs 0.6.1. Once all 40 bytes of the first AABB are null you can noclip the camera through the walls.

 

Use KT to extract the area model from the models.bif.

 

This link contains a two models - the Manaan hangar and the Dantooine hangar - both have had their AABB trees removed.

[Sithspecter 909]

This is an excellent piece of work, as you're the first to successfully narrow down exactly what causes this issue. Several people have had their suspicions that this is the case, but this is the first positive proof that I've seen.

 

Now, the real issue is reversing what you've done and adding the AABB node to new areas. Do you have any suggestions on a potential process for adding that to a custom model?

[Fair Strides 510]

This is an excellent piece of work, as you're the first to successfully narrow down exactly what causes this issue. Several people have had their suspicions that this is the case, but this is the first positive proof that I've seen.

 

Now, the real issue is reversing what you've done and adding the AABB node to new areas. Do you have any suggestions on a potential process for adding that to a custom model?

If he can get me all the data format in the mdl's binary, I could try adding it to KAurora.

[Quanon 263]

In addition, I have learned that KAurora apparently does support Lights, which are called Aurora DLights in 3DSMax and GMax.

 

This allows us to actually use light-sources similar to the floor lighting used on Taris' walkways or the Leviathan, as well as the normal lighting from something like a lamp or wall-light...

 

Yes, AuroraDLights work, but only 3 at a time will light up. When Kaurora didn't handle lightmaps I did a few tests with the AuroraDlights.

You can add a lot of them, but as mentioned above. Only 3 will work, depending you walk closer of further away, plus the orientation of the game camera, lights will switch on and off.

 

I would guess the function of these lights is to add a little extra flavour on some parts of your model. Or animate them to blink or pulse, again more as a detail, then say to do the proper lighting of a whole area. 

 

 

On the walkmesh splitting: I could write up a more in depth tutorial or explanation on the subject. The benefit of splitting up the walkmesh is you can say what the game engine has to render, depending on which part the PC character is standing. This is very usefull when working with emitters for example. These can really slow down the framerate of the game.

 

But by using the .vis file in addition with splitting the walkmesh into bits, you can avoid this.

It's not a very difficult thing to do actually. You just need to keep track of which area part is Nr 0, 1, 2, 3 ,4 and how they link up. Yes it starts counting from 0. Not 1.

 

Again; this all depends on how your area looks like ofcourse.

[Sithspecter 909]

We've known since the beginning that AuroraDLights were possible in KAurora, though there is some confusion surrounding them. Hopefully I can help clear some of that up.

 

In the days before lightmaps, ignorant people such as myself thought that all the lighting came from AuroraDLights and Aurora Lights. It was thought, at least by myself, that AuroraDLights and Aurora Lights were 2 different things. AuroraDLights were possible to create through NWMax, but imported areas from the game through the old MDLOps 0.5 showed mostly blank dummies called Aurora Lights. New evidence from areas run through MDLOps 0.6a shows that all of these are actually AuroraDLights. So yes, AuroraDLights are possible. Aurora Lights aren't real things.

 

Lightmaps provide the lighting for an area. AuroraDLights are meant to create shadows and give light to dynamic objects such as creatures, placeables, doors, etc.

 

Quanon seems to be correct in that only 3 can light up at a time. However, you can have far more than 3 AuroraDLights in any given area. I think a common misconception used to be that there could only be 3 AuroraDLights. This is false. Use as many as you want.

 

A much more in depth explanation of AuroraDLights can be found here.

 

 

The radius of the light is the effective radius (in cm) of the light.

The multiplier can be used to enlarge the inner radius of the light, as well as create negative lighting.

Priority The NWN engine has a light manager to prevent the engine from counting too many lights that affect dynamic objects. This relates to the light count setting in the game. The priority order is like this...

1 - Highest - Sun/Moon

2 - High... - Torches / Light spells

3 - Medium. - Spells, general lights

4 - Low.... - High-Priority Tile Lights

5 - Lowest. - Magic Weapon Lights, Tile Lights

 

Fading Light - Enable this to make lights turn on smoothly, instead of instantly turning to full brightness. As lights only get processed when they are on-screen, this helps with the general feel of tilesets for example - you don't want to have flickering ambient lights there.

 

Color - you can set the color of the light here. Either use the color box and choose a color or enter RGB values in the fields. Note that the color is ignored on tile mainlights and sourcelights as this parameter gets controlled by the setting in the toolset.

 

Negative Light - make this a negative light, make sure to up the multiplier value for best effect.

 

Ambient - turn this on to create an ambient light, when off the light will affect diffuse colors.

 

Is Dynamic Light , Affect Dynamic Objects - lights can have one of three dynamic states in NWN. Based on the setting the light will be more or less performance hungry. These two checkboxes control this.

 

Static - off/off - This setting will only affect static meshes, which basically is only TileGeometry. You can use this to shade certain areas on tiles for example.

 

Dynamic - on/on - This will affect all geom, dynamic and static, and should be turned on for animated lights. This is the most hungry setting - use sparingly.

 

Hybrid - off/on - The engine will determine if the light should be considered Static or Dynamic based on the situation. Sounds like the other two are useless, but the engine isn't always sure about your lighting motives

 

Shadow - if the light should make objects cast a shadow based on this light.

 

 

[Malkior 476]

Fascinating. I then hope there might be a script or process to enable or disable D-Lights on existing levels?

 

(I am referring to some of the more notoriously shadowy modules, Like Citadel Station or the Restoration Zone. )

[Fair Strides 510]

Well, if we can pinpoint the settings for the AABB node in the binary mdl, I could look into adding that to KAurora. And I know that some people have been looking into why it won't support animations, so there's that as a possibility as well...

 

If we can find these pieces, then I could look into updating KAurora to support these, which would at that point make MDLOps obsolete except for the "export to ascii" and the Replacer tool. Well, and the data viewer. But I reckon we could add the export to ascii pretty easily...

[Sithspecter 909]

Fascinating. I then hope there might be a script or process to enable or disable D-Lights on existing levels?

 

(I am referring to some of the more notoriously shadowy modules, Like Citadel Station or the Restoration Zone. )

 

It's extremely unlikely that there are un-enabled AuroraDLights just sitting around in modules. The fact is that they exist inside of the models themselves and I think their properties are pretty much set. For the kind of issues you are talking about, perhaps lightmap edits will do the trick?

 

Also, I'm really encouraged by the discussion going on here. Let's get as much information as we can out in the open for all to see.

[Dastardly 31]

KAurora already does have some support for walkmesh nodes, but it doesn't export the data correctly (or maybe it gets the wrong offset for the AABB tree?) Also, I created a module where you spawn in a lone cube which walls blocks the camera. See for yourself.

[Fair Strides 510]

KAurora already does have some support for walkmesh nodes, but it doesn't export the data correctly (or maybe it gets the wrong offset for the AABB tree?) Also, I created a module where you spawn in a lone cube which walls blocks the camera. See for yourself.

Can you send me all the files for that module? If I can compile the ascii with KAurora and compare that with your working version...

[Dastardly 31]

Well, if we can pinpoint the settings for the AABB node in the binary mdl, I could look into adding that to KAurora. And I know that some people have been looking into why it won't support animations, so there's that as a possibility as well...

 

If we can find these pieces, then I could look into updating KAurora to support these, which would at that point make MDLOps obsolete except for the "export to ascii" and the Replacer tool. Well, and the data viewer. But I reckon we could add the export to ascii pretty easily...

There are only two differences with the trimesh and the walkmesh node type. First difference is that the walkmesh node type is 545 while the normal trimesh is 33. The second difference is that the end of the subheader (right underneath mdx and vertices offsets) there is an int32 which contains the offset to the AABB tree. That's it.

 

The AABB node structure is like so:

Bounding Box Min

Bounding Box Max

Left Leaf Offset

Right Leaf Offset

Leaf Index

Most Siginficant Plane

 

The left and right leaves are offsets to another AABB node. The leaf offsets are ignored and set to 0 if the leaf index != -1. The leaf index is an index to a face to which the AABB node is attached to. The amount of AABBs are twice the amount of faces minus one.

 

KAurora already has classes for the AABB Node and the Walkmesh Header.

 

 

Can you send me all the files for that module? If I can compile the ascii with KAurora and compare that with your working version...

Sent you the binary files - don't have the ascii. Check your PMs. Trying to compare files probably won't be feasible since I loaded up .wok data and created a new node in the .mdl file with it.

[Sithspecter 909]

KAurora already does have some support for walkmesh nodes, but it doesn't export the data correctly (or maybe it gets the wrong offset for the AABB tree?) Also, I created a module where you spawn in a lone cube which walls blocks the camera. See for yourself.

 

 Way to go Dastardly! This is huge news! Amazing! I can hardly believe it myself. It's been so many years and nobody has cracked this. Congratulations are definitely in order.

[Sith Holocron 2,480]

I wonder if this discovery could fix some of the places that are already in the game . . .
 
Place 1: The HK Factory

 

 
From what I recall, there's a few spots in the HK Factory where clipping through the walls was possible.

 

 
Place 2: The "secret area" of Telos Citadel Station
 

 
The "secret" area of Telos Citadel Station has a faulty walkmesh and has wall clipping. The walkmeshes in these areas extend a bit over the edges of the walkways, and a bit into the walls.

 

 

They also end abruptly in the middle of what looks like a walkway because of adjacent areas above or below. It looks like Obsidian intended these areas for NPCs to walk about, making the place look larger than it is. (Moza is there for the "we're under attack" message when you leave the Ithorian compound area.) Fixing this area could potentially open even more of Citadel Station for locations for new mods.
 
Fixing the above two would be beneficial and if possible, should probably be implemented in TSLRCM.
 
Place 3: M4-78
 

The "shoot through wall" issues in the Environmental Zone can't be fixed. OE didn't really have time for hit detection, and modders can't add it in. You'll notice the same happening on certain locations in the HK-Factory in TSLRCM. Just something we have to live with, and we can't fix OE's rushing...

 
There's a place in the environmental zone where the droids can actually shoot through the walls. I asked Zbyl2 about this when I pointed out this thread to him but he suggested that you fine folks would be the ones to ask. Fixing those clipping issues would be helpful as well - and if possible - should be included in the next upgrade of the M4-78 Enhancement Project.

[VarsityPuppet 1,085]

So can this all be done in KAurora? I'm just wondering if I should be on standby if we need any tools to perform hex editing to models.

 

Fair Strides, I suppose you would also be able to do this, but I am also personally curious as to how this works, not to mention I would like to have a go at area modeling, now that camera collision and area lights work.

[Black_Rogue 1]

Very interesting article. but a question though, would this technology also allow not only new "map areas" on planets but also alough the creation of new planets itself? "just curious because im interested in building new maps and areas on Kotor 2

[Dastardly 31]

So can this all be done in KAurora? I'm just wondering if I should be on standby if we need any tools to perform hex editing to models.

 

Fair Strides, I suppose you would also be able to do this, but I am also personally curious as to how this works, not to mention I would like to have a go at area modeling, now that camera collision and area lights work.

KAurora doesn't seem to export the necessary AABB tree required to block the camera. However, I am currently writing a tool which will dump .wok data into a .mdl file to create a walkmesh node with the tree.

 

Its only a matter of time. Don't be afraid to start area modelling.

[VarsityPuppet 1,085]

It is a matter of time. I'm too busy

 

 

And there's a certain other project that is calling...

 

Duplisaber.... DUPLISABER!!!!

[Quanon 263]

@SithSpecter: Can you still do emitters with the latest Kaurora? Somehow it crashes whenever I try to save my model files.

I'm working on Win10 though... So perhaps Kaurora isn't behaving "good" because of my OS.

[Sithspecter 909]

Yes, I'm still able to do emitters. I'm still using Windows 8.1 and staying away from 10. I've heard bad things about compatibility with 10.

[Dastardly 31]

Yes, I'm still able to do emitters. I'm still using Windows 8.1 and staying away from 10. I've heard bad things about compatibility with 10.

Not to mention the abundance of privacy concerns.

 

 

Edit: My tool is just about ready.