Anti-Aliasing and You: Information and Tips

One of my biggest pet peeves in images is poor anti-aliasing, even more so than poor jpeg use. I’ve done a few informational/tutorial based threads before, so here’s another one.
Notice: Anti-Aliasing is a really broad, complicated field. There’s a lot more to it, even with what you’ll see in your GPU driver’s control panel. Here I’m going over the most common/important ones, google anti-aliasing to learn more!

When people in this forum talk about “turning up your graphics,” one of the most important, but most often overlooked, is anti-aliasing. Anti-aliasing is a solution to a pretty serious problem with computer graphics. Our current digital displays rely on pixels, little square colored dot things (in simple terms :v:) that, when looked at from a distance, are used to form an image. It’s an extremely intelligent design, and it gets good results. However, it has one drawback. Our eyes pick up light, which consists of photons and waves and little hookers that are, in fact, a hell of a lot smaller and “smoother” than pixels. So, when you look at an image on a digital screen with square, harsh pixels, you pick up on the sharp edges and your brain tells you that something is wrong. Now, displays with very high pixel density (such as the Droid DNA cell phone and its 441 ppi display) get around this by having pixels so small that your eyes cannot make them out. Sadly, our full-sized monitors are nowhere near at this level yet.

Enter Anti-Aliasing, the cheaper solution to this significant problem. Anti-aliasing uses some hardcore mathematics to find harsh edges, and “blur” or “smooth” them intelligently so the edges appear less harsh (think how white->black is harsher to the eyes than white->gray->black). Now, good AA can be very intensive, so up until the 21st century it was not very common in games. Today it’s common place, and there are different kinds to choose from, which we will see below.

1 - No Anti-Aliasing
Check out the image below.

https://dl.dropbox.com/u/17239680/Facepunch/Tutorials/AntiAliasing/Extreme/NoAA.jpg[/t]
The textures, shadows, models, anisotropic filtering, post processing details, etc are all on high, and yet it looks horrible. This image has no anti-aliasing, and so your brain gives you a feeling of “something isn’t right” that you cannot shake, not unlike the uncanny valley. We want to avoid this, because, as you can see, it can ruin an otherwise nice image.

2 - Post Anti-Aliasing
This is our first option. Post-AA, such as FXAA, MLAA and SMAA, works by looking at the finished, fully rendered image, and looking for areas of contrast that take on the appearance of aliasing. It then intelligently smooths the image in just that area. For an example of this, see the two images below. The first is the way that the anti-aliasing filter looks at the image and identifies aliasing, the second is the final product.
[t]https://dl.dropbox.com/u/17239680/Facepunch/Tutorials/AntiAliasing/Extreme/MLAA-Edges.jpg[/t] [t]https://dl.dropbox.com/u/17239680/Facepunch/Tutorials/AntiAliasing/Extreme/MLAA.jpg[/t]
This method is nice because it only requires the program to look at the final image, rather than each object, and as a result renders the fastest. This method works well on computers with poor performance (i.e. laptops), and at the cost of 1-5fps, is almost never worth not having on unless you are using a better method. Also, this type of AA can often be easily “injected” into games with programs such as the FXAA Injector and SweetFX. The downside to post-AA is that it affects the entire image, and can make the entire image seem soft or blurry (Crysis 2 and Battlefield 3 are great examples of where this can get bad). Also, it can miss certain edges or fail entirely when it has too little data to work with (such as small or distant objects).

3 - Non-post Anti-Aliasing (Often seen as MSAA)
This is the option most people are familiar with - the one that comes in 2x, 4x, 16x, 32x, etc. This method is very diverse in the actual way the programming deals with aliasing, but the way it is processed in the engine is not. This method is applied to each object within a rendered scene in a game, based on the edges of the polygon based models and terrain in the scene. An example can be seen below.
[t]https://dl.dropbox.com/u/17239680/Facepunch/Tutorials/AntiAliasing/Extreme/MSAA.jpg[/t]
This method is nice because of it’s accuracy and sharpness. Simply put, this method has a lot of information at it’s disposal, so it can effectively remove much of the aliasing in an image without affecting over-all image quality. That said, because it must look at every object on the screen, it is often very performance intensive. Also, because most methods are based on the models in games, aliasing in sprites/textures (such as fences) is often missed.

**4 - SuperSampling/Downsampling (slightly different, work on same principle)
**This option is the easiest to understand, yields the highest results, works with every game, and is very performance heavy. Supersampling and Downsampling work by rendering the aliased areas (or the entire image, with downsampling) at a much higher resolution, then simply shrinking the image to your monitor’s resolution with a simple filtering method (i.e. bilinear, trilinear, bicubic - that’s another thread for another time). An example can be seen below.
[t]https://dl.dropbox.com/u/17239680/Facepunch/Tutorials/AntiAliasing/Extreme/Downsampled.jpg[/t]
This method looks amazing in screenshots, and looks like a cross between Post-AA and Non-Post-AA in game (most image processors will downsample better than your monitor will in-game). Getting it to work can be tricky, but it can be used for extremely high-quality results in almost any game. The only downside is the performance cost associated with rendering at high resolutions, which can be very significant. If you can use this method, I highly recommend turning it on when taking your screenshots. [del](a tutorial can be found here)[/del] This feature is now supported directly in the control panel for most new GPUs. See “Virtual Super Resolution” for AMD cards and “Dynamic Supersampling” for Nvidia cards.You can also use this method with the poster command. type “poster 3” into your console to render a nice sized image for downsampling. The poster method does not support super DOF, any sort of HUD, and a few other things, so use it cautiously! Because this method uses the whole image and doesn’t have to make any guesses, it effectively removes all aliasing in an accurate, smooth manner. This can best be seen on the grooves in the gun in the image above (which are distorted in every other image).

Now, all that said, you are not limited to using only one AA method at a time. Using them in conjunction is very important in order to get a smooth image. For example, the downsampling image above uses all 3 methods (although downsampling makes the biggest impact). However, using the cheaper two (MSAA and Post-AA) yields amazing results as well, which can be seen below.
[t]https://dl.dropbox.com/u/17239680/Facepunch/Tutorials/AntiAliasing/Extreme/Both.jpg

Here are some images to show what your games can look like with carefully implemented anti-aliasing.
I hope this was informative and useful, I put it in this subforum because I figured you guys would get the most use out of it.

Also: Post-AA can be enabled in almost all games/programs (I have yet to see one not supported) via your GPU Drivers. With these options, you can also turn up the deferred AA to higher levels. Pictured below is the NVIDIA control panel, with FXAA (Nvidia’s post-AA) highlighted. AMD has a similar feature (using a method called MLAA, which is pictured in post-AA feature).

https://dl.dropbox.com/u/17239680/Facepunch/Tutorials/AntiAliasing/PostEnable.png

Great stuff, Brim. Very useful.

Also, something else worth noting about post-AA is that, due to it being a post process effect, you can add it in eentirely after the image was taken using an MLAA filter with Photoshop (one comes with the Pixelbender filter suite for free).

I also want to point out that there’s a cheap way to ‘fake’ Anti-Aliasing using SuperDOF processing.

If you set the blur size to a small number such as 0.1 and the focal distance to 0. It will effectively blend multiple tiny angles into one image. Because the blur size and focal distance are so small, the blur isn’t really noticeable. That’s not to say that it isn’t there, though.
You can play with the values to try to get the best results. You can also apply a slight sharpen filter to make up for the blur.

Without.

http://puu.sh/1HAQc[/t]
With:
[t]http://puu.sh/1HAQI

(The thumbnail resizing makes them look as if they are heavily aliased, open them fullscreen to see.)

http://puu.sh/1HBvQ/a02031df909f565069ae542c9fc89c20

I like it.

Another thing to note about the downsampling method I mentioned is that, due to the entire image being rendered at a higher resolution, it will often force better texture filtering, sharper shadows and more realistic DOF in games that support it.

Using the poster feature pretty much takes care of any anti aliasing problems too. “Poster 3” or above should do the trick.

as lonefirewarrior said, the Poster command then downscaling/cropping the image is often your best bet for antialiasing, as it creates a pixel-perfect .png that’s X sizes bigger than your current game res (aka 4 times, 3 times etc) last time i used it, however, it was broken, but it was probably a problem on my end.

however if you are using in-game light sprites, you will probably have errors with the screenshot (aka the lightsprite appearing disjointed/covering up details) so you’ll probably have to not-use ingame light sprites for poster, edited in sprites look better usually anyway.

[editline]31st December 2012[/editline]

what i want to know is why valve removed .bmp screenshot output, as while .bmp is an incredibly archaic format, it produces absolutely bloody perfect pixel-perfect quality images, take HL1’s screenshot function for example.

Agreed. However, the downsampling method linked above does take pictures at the higher resolution, you need to resize them in post. Of course, your monitor will resize the image as well while you are in-game, but any screenshots you take will be saved as whatever resolution you are forcing the game to run at. Because things like sprites scale well when only the resolution is increased, there usually aren’t any compatibility problems.

Here’s an example image I took in battlefield 3 prior to resizing (warning: it’s huge)
https://dl.dropbox.com/u/17239680/Game Photos/BF3/Downsampled/FullSize/02.jpg

3164x1780 isn’t tremendously large. i’ve worked with 6000x6000 images before.

[sub][sub]and my cpu is still recovering[/sub][/sub]

[editline]31st December 2012[/editline]

also that image made me realize how fucking awful BF3’s textures are. did they seriously sell that game retail?

Yeah, it isn’t all that large, but that’s my monitor’s limit before I hit my pixel clock limit. I’m not talking downsampling by pre-rendering images - that is the best, and you can reach insane resolutions, but it isn’t practical for gaming.

As for BF3, that’s just a poor picture. I have an [image series](https://dl.dropbox.com/u/17239680/Game Photos/BF3/BG/BG49.jpg) showcasing the game, it can look pretty damn spectacular. That link will work with numbers 1-54. The images do NOT use downsampling, but rather a combination of MSAA, FXAA and SMAA that I’ve been tinkering with for almost a year :v:

Also, updated the OP

jesus christ the blur and aliasing at the back. that looks like mirror’s edge being played on a bad PC.

Like I said, not downsampled :v:
I’m trying to keep 60fps, there’s not much room around it without downsampling, or bumping up the deferred to some ungodly number.

i’m not insulting your AA. the game looks frankly like a 2008-era game played on bad settings. the textures are all pixellated and i can tell that even with high-level AA everything gets jaggy as hell at far distances.

Matter of opinion I guess, I always thought the game looked nice. Anywho, updated the OP again with an example of how to add FXAA to a game. While Post-AA gets ugly when it is used alone, the one nice thing about it is that it is incredibly versatile. I’ve even used it with emulators before, running old PS1 games. The cost to FPS is almost nothing as well.