DXTC texture compression

Why do we need compression?

To achieve our target level of detail in Overgrowth, we have to use some very high-resolution textures. We would like the game to run smoothly on older graphics cards, which often have less than 128 MB of video RAM. This is a problem for texture storage: just one uncompressed 2048*2048 RGBA texture uses 16 MB of RAM, and we may have dozens of them. One of the techniques we're using to solve this problem is DXTC texture compression.

What is DXTC?

Most common image compression formats like PNG and JPEG aren't directly supported on the graphics card. We can use them to make smaller files on the hard disk, but as soon as they're loaded into the game they're decompressed to their full size. However, a few formats can actually be loaded and processed by the graphics card in their compressed form. The oldest and most widely supported of these is DXTC (previously known as S3TC). Here's a closeup of a compressed and uncompressed version of a cat concept by Aubrey.

As you can see, the compressed version looks almost the same as the uncompressed version, and uses 128 KB of memory instead of 768 KB. It's a form of lossy compression, so some of the subtle color gradients are simplified. To put this into perspective, here's what an uncompressed 128 KB image looks like.

How do we use DXTC in OpenGL?

I spent a long time looking for the best way to do this, and ended up using the .dds (DirectDraw Surface) file format, loaded with the NVImage class included with their OpenGL SDK. To create these .dds files, we load up the old .tga textures using FreeImage and automatically convert them in-engine using Nvidia's Texture Tools code. We choose the type of DXTC compression based on the suffix of the file. For example, rabbit_head_dxt5.tga would be converted to rabbit_head.dds using DXT5 compression.

What do the different types of DXTC mean?

There are many different kinds of DXTC compression, and it can be overwhelming at first to figure out which ones to use. The only really important ones are DXT1, DXT3, and DXT5. They all handle color compression in the same way, but the key difference is how they handle the alpha channel. Here's the alpha channel of a smoke sprite compressed with each type:

DXT1 has a binary alpha, and is basically unsuitable for anything that will be rendered with alpha blending (it might be ok for simple alpha testing). DXT3 stores an uncompressed 4-bit alpha for each pixel. This means that the colors are all in the right place, but there are only 16 levels to choose from, causing severe banding artifacts on long gradients. DXT5 stores an interpolated 4-bit alpha. This means that there are the same kind of block compression artifacts as in the color channel, but smooth gradients look pretty good. It's safe to say that DXT5 alpha channels look the best for most images.

Does that mean we should always use DXT5? There are two cases where we shouldn't. First, if there's no alpha channel, we should use DXT1, because it's half as big as DXT5. Second, if there are only sharp edges, DXT3 is actually slightly more accurate than DXT5. Here is an example image with sharp alpha edges.

It's impossible to tell the difference between the uncompressed, DXT3 and DXT5 images here just by looking at the alpha, but by subtracting the compressed images from the TGA and amplifying it 16x, we can see which is more accurate (darker is better):

This shows that DXT3 is more accurate for hard anti-aliased edges than DXT5.

The last question is which DXTC format to use for normal maps. This is too complicated to get into here in detail; you can find whole articles on this subject. However, the consensus seems to be that we should use DXT5 and put one of the directional channels in the alpha channel. Unfortunately, that means we can't use the alpha channel for specularity like we do now; we'll need a whole new texture. On the bright side, using two DXT5 textures is still only half as much VRAM as an uncompressed texture!

Finding out how to use DXTC compression effectively in our game took a few days of research on my part, so I wrote this post to save some time for other graphics programmers out there. Please let me know if there are any tricks or best practices that I am missing!