Optimizing a Huge City Map

Alright, now I got a really big question… This really can’t go in the Megathread. It’s just too long of a question and hopeful answer to go in there.

To start this out, just for reference, my map was basically spawned from playing Hiyougami’s Freecity series of maps. I aspired to make something like that, except not focusing on how the original did on GPK gameplay, or how Plaza focused on light style and interiors, but on the buildings and rooftops. I took heavy inspiration from those and Mirror’s Edge, playing Mirror’s Edge and getting ideas for buildings and then applying those ideas into full buildings.

I’ve talked with Hiyougami numerous times, and as a result, I got to use his models and textures that he’s used, and along with other textures/models I had available, I finally could start. Over time, more people got involved, I got new models, new textures, and lots of support.

After constant work on my map, it’s getting harder and harder to get everything to work. Basically, I haven’t a clue on proper optimization, Propper is making crappy shadows so the model doesn’t look good at all, I get the “ClientRenderable” error, and I’m stuck at what is the final part of this map. It’s a big map that has plenty of tall buildings, and those tall buildings are using a lot of resources that make compile time long and cause errors and reach limits.

Then I look at maps like gm_bigcity and see loads and loads of buildings and areas of detail with good framerate and no errors for too many things being rendered and using up a lot of engine resources. The thing with bigcity is that the skybox is also pretty much a big box too. You can go at any point of the map and find that the highest point of the skybox is uniform. I have no idea how that was optimized, but that’d be the way that my map would need to be optimized. Can someone point me in the right direction?

I mean, I look at maps like this…

http://www.hiyougami.co.uk/highslide/images/large/gpk_freecity_v1_b200001.jpg
http://www.playground.ru/images/files/1/1/44.jpg

… and I go “How on Earth can these be fully rendered and not cause major lag?” This big mapping project that I’ve been working on have been a work in progress for a long time. I finally really got to the point where the layout and detail and geometry is done, and then I need to optimize, and, once again, that’s where I’m stuck.

I see you guys showing off your maps all the time, sometimes showing big maps with high detail, lightmap scales, or just general size, so I’m hoping someone here can really point me in the direction I need to go if I want to get this done right.

I know of the many optimization tools: hints, areaportals, func_detail, propper, etc. but really have no real guidance as to how to really use them, and the way I’d need to isn’t in a simple way. Func_detail only goes so far until there are too may t-junctions, and I’m at that limit, with Propper aiding me a bit, but so far hasn’t yielded a good result, with me getting horrible shadows on the models and such. I’ve tried my best at using optimization methods, and have been hoping that I’ll get that right, but it’s not happening.

It’s not like I need to optimize rooms and break up corridors, it’s a mostly outdoor city full of a bunch of decently detailed buildings (16 to be exact, and in a full grid size (length x width x height) of about 13328x10222x3400 units, and the average height of a building being around 1800 units and length of 1900).

I started a long time ago in what was a small test, and it turned into a huge ambitious project of mine, one that kept going and going, expanding to what it is now. That’s where the problem really is. This project is what basically taught me everything I know about Hammer. I had made stuff before, but it was basic easy stuff that wasn’t very good, and with this project, I applied the skills I had into a bigger scale, and learned ever more and more about Hammer with the help of many.

What I’m doing isn’t going to go to waste, and I’m not just throwing away what I’ve worked so hard on. I worked on my brushwork, made things non-blocky and generally better looking, cleaned up brushwork, drew inspiration and made my ideas into reality, really learned and focused on lighting, but keep failing at optimization. It’s the final stretch for me, and at this point, I’m stuck. I’m not giving up, I’m never giving up, but I need some help to get there.

Can you post your map?

Before you do anything, please add (or recompile the textures with:) Mipmaps.
Its a nice little flag in VTFedit that makes textures not eye-rape from distances.

I don’t think that’s his map, from the OP it seems like he’s using that as an example of highly detailed maps that somehow don’t cause lag. I support working out how people manage to make maps like that without issues, as one of my most recent releases (which still wasn’t recent anyway) suffered from similar problems.

A rudimentry tip is to add horizontal hint brushes. 1 about 128, another higher up, then everything on top of that func detail. Currently a player standing anywhere can see anything since their leaf goes right to the top of the skybox. The horizontal hint brushes will let a player on the ground gain SOME blocking. The Higher hint is for player climbing over stuff, jumping etc.

Use smaller stuff, “props”, walls, lorries etc to block visibility in that bottom 128 zone. Use horizontal hints on corners so people cannot see round them. This will let you add lots of detail to the bottom level of some buildings at least.

Consider recompiling some of your models to have LOD models and to use no shaders at higher LOD models. So you could build a rather intricate set of roof buildings, with fancy geometry. Then as you go further away the LOD model simplifies so a simple box. So from the highest rooftop you look at the roof, see there is still a building there.
LOD detail up close, model still there at a distance (unlike fade) it increases the RAM usage slightly but it will improve performance.

Consider also building details on the facades of buildings as models with LOD. Do this after cutting up ur visleafs. Then cut the model across the visleaf. Otherwise you will get the entire model facade being rendered despite only seeing a tiny bit of it.

Use Func_viscluster in larger areas which won’t have much blocking. (like the top area I mentioned) and large boulvards.

For anything where lighting on it doesn’t really matter try to use models. You have much more control over how they are rendered and the material they are using.

Try to use a small selection of textures. Not 10 different 1024x1024 textures on 1 building. Every new texture adds to the texture budget and unlike geometry, fancy textures will gobble up ur budget.
Reuse bumpmaps, detail and specular maps where necisary to reduce memory usage. If you have a fancy shader on a model make an lod model which doesn’t use said shader.

IF you have interiors put portals in the windows. If you insist on being able to see into the window, put 1 bug portal across the room about 64 units from the window. So you can see people inside the window but stuff further in isn’t rendered. in this case don’t put detail near the window, focus it more to the inside of the room.

Occluders will be useful for the bigger buildings which you can’t hint off. I suggestion is complete the map with everything else, add the occluders and see if it saves you performance.

For some of those colored things in the first image, instead of using different coloured textures consider using a func_brush and changing the colour on it, it will cause other issues like clipping, decals and visibility but it will save you adding the extra textures. This last paragraph is abit dull the rest might be useful though.

Good luck I look forward to seeing this.

EDIT: I put “props” in speech marks since there should be some geometry or an occluder inside the large ones to reduce visibility. Big bins etc

Yeah, although in the current build I didn’t render the entire street (thank you Cordon Tools), as otherwise it wouldn’t compile. It’ll at least show the entire map with all the rooftops and such. That’s just an overview, and for views of individual buildings I tested with Cordon, then go my Steam Screenshots. I’d post them here too, but I have loads, and that would be an extreme page strethcher.

http://cloud.steampowered.com/ugc/578958940126142223/57652458A73A1EF5718C13CB07669180AA8A68E8/

I barely got though compiling that too. Most of the points in the map when played normal or not get the same “ClientRenderable” error. Also, don’t worry, I’m working on editing the textures so that it doesn’t look all weird at farther distances.

See, that’s the thing with hints. I really don’t know how to use them. Pretty much the only thing I did with hints was make a big one the height of the skybox and put a bunch between buildings, and then at one point, i cut all of them into half. With func_viscluster, I put one big one at the top of the skybox, stopping at the height of the tallest building, and then what I did was place one in every spot above the rooftops of each building and then in all spots separating them, not touching any other brushes but hovering over the closest one. Now, I don’t know if I’ve done any of that right, or if I could do it better.

Also, how would I go about using occluders? Maybe that’s what I really need and it’ll get everything to start working again.

Using viscluster and not knowing fully what it does can be a mistake, you can kill any hope of optimization with it.

Exactly. That’s why I was hoping I could get some sort of direction as to how to use all the different optimization methods for this map to work like the two previously mentioned maps. Was the way I used viscluster the right way, and if not, how would I properly use it?

As much as you should be using it in open areas, where there should be no vis-cutting/splitting. This can cause issues, if you use the whole space with a vis-cluster, then you might avoid any of the important bits from reaching areas where it could cut out a whole building that is being rendered behind something; Viscluster should only really be used when you KNOW what you’re doing, and when you can easily compare what it will do when you place it somewhere.

As for hint and skip, use them in places that are in complex areas, such as rooms down a hallway, or even in hallways.

But I recommend to use Vis-portals a lot, I know they can cause performance issues, but thats if you leave them open; if you use the window version, you can easily make it close after X distance, thereby sealing the room (This comes in handy if you have a huge map, where you are so far away from one side that there is no chance you could see inside, yet the game still renders it despite hints / its distance.)

For a lot of these large city maps, visleaf optimisation is almost not possible on a sane level. You’d be looking at perhaps finding the lowest building (avoid the 1-story ones if the average height of a building is something much higher) and use this as sort of the baseline.
Below this baseline you can divide up visleaves on a per block or per street basis, perhaps using diamond hint brushes at intersections and making open areas into large single visleaves.
Above the baseline, it’s quite likely that you’ll be able to see most of the streets in an area anyway, so it’s much easier to treat this like one large single leaf. You could possibly work out an intricate mesh of hint brushes between each building, but there’s a large chance that buildings in this area will be differing heights and it’ll be a real hassle to work out the hints required as it’s sort of a case by case scenario. You may want to chop the buildings at this baseline and func_detail the top halves, then viscluster this zone, which realistically wouldn’t cause too many issues if you do the latter stages of optimisation correctly. Obviously there’s probably a better way you could do this for performance sake, but it’d be quite complex and I really don’t even know how to explain how in words.

Now what I said about the later stages of optimisation, this would be things like model fade distances, LODs, areaportal windows (if you have buildings with interiors), func_brush details on buildings with fade distances and far-z clip planes (which can work for some maps, but not others, depending entirely on the intended use of the map).

When it comes down to it, optimisation is really one of those things that’s hard to grasp, takes time and should be considered during the initial creation stages of the map. Over time you will just generally get better at it and pick up neat ways to make things work best for you.

All I can say now is good luck.

func_brush can’t have fading to my knowledge. func_lod does but you are better off proppering it.

Not necessarily better off. You lose lightmapped shadows with a model, so if the section of building in question is half in the sunlight and half in the shadow of another building, then propper will make it look out of place.
If it’s just a building’s entrance or awning however, then you are normally either in full view of the sun or entirely in the shade, so a model would be fine.

Hm… That’s interesting about how to use hint. Yeah, there’s really no building the same height, save for two which are VERY close, but are about 50 units off. I’d be fine with using intricate ways of optimizing, and taking time with it, because everything is done, save for optimization and just a few things involving funtionality of the map, but brushwork is done.

The other thing is just that I reached the LDR leaf ambient limit of the compile that I posted in the thread, and that’s without the street added in, so dunno how I’m going to resolve that either. I’m also close to the visdata limit too, but that’d be resolved with hint/better viscluster/areaportal optimization, right?

Anyway, I really wouldn’t care if the way I’d need to optimize would take a lot of work and would be on a less than sane level. I’ve been working on this map too long to give it up. The good thing is that not only is optimization the only thing left to do, but it’s summer, and I have loads or free-time to do it.