As some of you noticed in the last alpha, the rabbit would cast shadows on the environment, but wouldn't 'catch' shadows from the environment. This meant that he could stand in a shadow and remain brightly lit, like this:
I had been putting this off because casting baked shadows onto dynamic objects seemed like a tricky problem, and I was busy with animation and movement scripting. However, I thought of a quick way to get some basic shadow catching working. Here is the same scene as above, but with shadow catching:
This method is just the opposite of the one used for casting character shadows (as described here). To review, the character shadows are created by rendering the character into a depth map, and then casting that depth map subtractively onto the scene.
To catch environment shadows, I thought I could read back the baked shadow values, and cast those onto the character! This is done by rendering the baked shadow maps into a texture from the same perspective that the shadow is cast from, and then projecting that texture onto the character model. There are a few discontinuity artifacts from shadow map seams, but these are easily blurred away.
This seemed too easy, and it turned out that it was -- there is a problem. The problem is that the shadows are blurred, so they often bleed out past their precise silhouette edges. This meant that the character could sometimes catch shadows that shouldn't have been cast yet, as in the picture below. You can see the caught shadow map on the right, showing how the shadow is bleeding out past the silhouette edge.
Since I have a lot of work to do besides shadows, I decided to use a quick and dirty fix for now. Since this problem is characterized by a band of darkness above a white silhouette, I just added an image processing filter to bleed light upwards and remove the false shadows. This is done in just one pass, but I added some intermediate stages below to clarify how the shadow is removed.
Now the character is unaffected by the false shadow, and is properly lit when standing above silhouette edges. Here is the same scene as before:
Most normal shadows are unaffected because they are dark at the base, so there is no light that can bleed upwards. Here is a typical example of a shadow gradient that is unaffected by the filter.
Shadows are an important cue for our sense of spatial awareness, and this is vital for a game with as much jumping as Overgrowth. I didn't want to use standard depth map shadows for the environment because they have hard edges, and I wanted to use the softness of the baked shadows to contrast with the harder character shadows. You can see this contrast here on the tree -- the hardness of the character shadow and the softness of the building shadow help define the spatial relationships.
This technique won't work well with environments with raised horizontal objects (due to the upwards light bleeding filter), but in typical Overgrowth situations it works very well! I am happy with it because it's intuitive, inexpensive, and preserves the soft, smooth shadow gradients from the baked shadow maps. Do you have any questions about how this works, or ideas on how to improve it?