IRC Banner Live Chat
Welcome to the Wolfire Blog! This is where we keep everyone up to date on our progress on Overgrowth and other new stuff. Be sure to subscribe to get the latest news! Also, be sure to check out the forums for even more up to date news - or get on IRC for up to the second updates.

Linear algebra for game developers ~ part 2

By David on July 3rd, 2009

Now that I’ve introduced vectors in Part 1, we need to look at some of the fundamental tools for working with them. The most important tools to understand are length, normalization, distance, the dot product, and the cross product. Once you wrap your mind around these concepts, and write functions to use them, you can solve most vector problems you might encounter.

Length

If we have a ship with velocity vector V (4,3), we might also want to know how fast it is going, in order to calculate how much the screen should shake or how much fuel it should use. To do that, we need to find the length (or magnitude) of vector V. The length of a vector is often written using || for short, so the length of V is |V|.

We can think of V as a right triangle with sides 4 and 3, and use the Pythagorean theorem to find the hypotenuse: x2 + y2 = h2. That is, the length of a vector H with components (x,y) is sqrt(x2+y2). So, to calculate the speed of our ship, we just use:

|V| = sqrt(42+32) = sqrt(25) = 5

Pythagorean theorem

This works with 3D vectors as well — the length of a vector with components (x,y,z) is sqrt(x2+y2+z2).

Distance

If the player P is at (3,3) and there is an explosion E at (1,2), we need to find the distance between them to see how much damage the player takes. This is easy to find by combining two tools we have already gone over: subtraction and length. We subtract P-E to get the vector between them, and then find the length of this vector to get the distance between them. The order doesn’t matter here, |E-P| will give us the same result.

Distance = |P-E| = |(3,3)-(1,2)| = |(2,1)| = sqrt(22+12) = sqrt(5) = 2.23

Distance

Normalization

When we are dealing with directions (as opposed to positions or velocities), it is important that they have unit length (length of 1). This makes life a lot easier for us. For example, let’s say there is a gun pointing in the direction of (1,0) that shoots a bullet at 20 m/s. What is the velocity of the bullet? Since the direction has length 1, we can just multiply the direction and the bullet speed to get the bullet velocity: (20,0). If the direction vector had any other length, we couldn’t do this — the bullet would be too fast or too slow.

So how do we normalize a vector? Easy, we divide each component by the vector’s length. If we want to normalize vector V with components (3,4), we just divide each component by its length, 5, to get (3/5, 4/5). Now we can use the pythagorean theorem to prove that it has length 1:

(3/5)2 + (4/5)2 = 9/25 + 16/25 = 25/25 = 1

Dot product

What is the dot product (written •)? Let’s hold off on that for a second, and look at how we calculate it. To get the dot product of two vectors, we multiply the components, and then add them together.

(a1,a2)•(b1,b2) = a1b1 + a2b2

For example, (3,2)•(1,4) = 3*1 + 2*4 = 11. This seems kind of useless at first, but lets look at a few examples:

Dot products

Here, we can see that when the vectors are pointing the same direction, the dot product is positive. When they are perpendicular, the dot product is zero, and when they point in opposite directions, it is negative. Basically, it is proportional to how much the vectors are pointing in the same direction. This is a small taste of the power of the dot product, and it’s already pretty useful!

Let’s say we have a guard at position G (1,3) facing in the direction D (1,1), with a 180° field of view. We have a hero sneaking by at position H (3,2). Is he in the guard’s field of view? We can find out by checking the sign of the dotproduct of D and V (the vector from the guard to the hero). This gives us:

V = H-G = (3,2)-(1,3) = (3-1,2-3) = (2,-1)
D•V = (1,1)•(2,-1) = 1*2+1*-1 = 2-1 = 1

Since 1 is positive, the hero is in the guard’s field of view!

Sneaking

We know that the dot product is related to the extent to which the vectors are pointing in the same direction, but what is the exact relation? It turns out that the exact equation for the dot product is:

A•B = |A||B|cosΘ

Where Θ is the angle between A and B. This allows us to solve for Θ if we want to find out the angle:

Θ = acos([A•B]/[|A||B|]).

As I mentioned before, normalizing vectors makes our life easier! If A and B are normalized, then the equation is simply:

Θ = acos(A•B)

Let’s revisit the guard scenario above, except the guard’s field of view is only 120°. First, we get the normalized vectors for the direction the guard is facing (D’), and the direction from the guard to the hero (V’). Then, we check the angle between them. If it is greater than 60° (half of the field of view), then the hero is not seen.

D’ = D/|D| = (1,1)/sqrt(12+12) = (1,1)/sqrt(2) = (0.71,0.71)
V’ = V/|V| = (2,-1)/sqrt(22+(-1)2) = (2,-1)/sqrt(5) = (0.89,-0.45)

Θ = acos(D’•V’) = acos(0.71*0.89 + 0.71*(-0.45)) = acos(0.31) = 72°

The angle between the center of the guard’s vision and the hero is 72°, so the guard does not see him!

Sneaking

I know this looks like a lot of work, and it is, because I’m doing it by hand. However, in a program, this is pretty simple. Here is what this would like in Overgrowth using the C++ vector libraries I wrote (inspired by GLSL syntax).

//Initialize starting vectors
vec2 guard_pos = vec2(1,3);
vec2 guard_facing = vec2(1,1);
vec2 hero_pos = vec2(3,2);

//Prepare normalized vectors
vec2 guard_facing_n = normalize(guard_facing);
vec2 guard_to_hero = normalize(hero_pos - guard_pos);

//Check angle
float angle = acos(dot(guard_facing_n, guard_to_hero));

Cross Product

Let’s say you have a boat that has cannons that fire to the left and right. Given that the boat is facing along the direction vector (2,1), in which directions do the cannons fire? This is easy in 2D: to rotate 90 degrees clockwise, just flip the two vector components, and then switch the sign of the second component. (a,b) becomes (b,-a). So, if the boat is facing along (2,1), the right-facing cannons fire towards (1,-2). The left-facing cannons fire in the opposite direction, so we flip both signs to get: (-1,2).

Boat

So, what if we want to do this in 3D? Let’s revisit our sailing ship. We have a vector for the direction of the mast M, going straight up (0,1,0), and the direction of the north-north-east wind W (1,0,2), and we want to find the direction the sail S should stick out in order to best catch the wind. The sail has to be perpendicular to the mast, and also perpendicular to the wind. To solve this, we can use the cross product: S = M x W.

Boat

The cross product of A(a1,a2,a3)) and B(b1,b2,b3)) is:

(a2b3-a3b2, a3b1-a1b3, a1b2-a2b1)

So now we can plug in our numbers and solve our problem:

S = MxW = (0,1,0)x(1,0,2) = ([1*2-0*0], [0*1-0*2], [0*0-1*1]) = (2,0,-1)

This is pretty ugly to do by hand. For most graphics and game work I would recommend just encapsulating it in a function like the one below, and never thinking about the details again.

vec3 cross(vec3 a, vec3 b) {
    vec3 result;
    result[0] = a[1] * b[2] - a[2] * b[1];
    result[1] = a[2] * b[0] - a[0] * b[2];
    result[2] = a[0] * b[1] - a[1] * b[0];
    return result;
}

Another common use for the cross product in games is to find surface normals — the direction that a surface is facing. For example, let’s take a triangle with vertex vectors A, B and C. How do we find the direction that the triangle is facing? It seems tricky, but we have the tools to do it now. We can use subtraction to get the direction from A to C (C-A) ‘Edge 1′ and A to B (B-A) ‘Edge 2′, and then use the cross product to find a new vector N perpendicular to both of them… the surface normal.

Triangle normal

Here is what that would like in code:

vec3 GetTriangleNormal(vec3 a, vec3 b, vec3 c) {
    vec3 edge1 = b-a;
    vec3 edge2 = c-a;
    vec3 normal = cross(edge1,edge2);
    return normal;
}

Fun fact: the basic expression for lighting in games is just N•L, where N is the surface normal, and L is the normalized light direction. This is what makes surfaces bright when they face towards the light, and dark when they don’t.

Next time

We covered what vectors are in Part 1, and now we have a solid toolbox for working with them. Next, I would like to talk about vector spaces and matrix transformations.

This post was a lot more complicated than the last one — is there anything confusing here that you have questions about?

  • jph
    Great stuff,. nicely explained. For those of us that where not figuring math had much use to us, at the time we should have been learning it,. . then went on to try to build a game. Math teatures in highschool really should teach simple game dev. to get kids interested.
  • TheBigCheese
    Yeah, honestly I don't know why they don't take that approach.

    It really brings a lot of abstract down to earth.
  • Phil (Sumguy720)
    You guys should release better math teachers along with the alphas.
  • M.Carabas
    that's so weird...
    I'm a private tutor and when tutoring Grade 11s and 12s for Algebra I usually ask them if they would like to write a simple computer game like pong (or tetris for the more adventurous)... usually the response is very enthusiastic...
    I've found this is the best way to get kids to take Algebra seriously... it's so much easier to explain vectors in terms of bullets ricocheting off walls instead of canoes being paddled against currents.
  • Excellent information here!
    Thank you for taking the time to write all this, I would lova having all this explained when I started game development 8 years ago, but it's still definitely useful even now.
  • GreenFlame
    Thanks for this post, haven't thinked that vectors are SO useful =D
  • A nice post. I like the addition od code in this post :)
  • Ragdollsoft
    Be sure to check that distance is not 0 when you normalize, or you'll get some hard to find bugs that happen every once in a while.
  • David
    That's a good point, it can be frustrating to track down why your vectors are becoming INF or NaN!
  • RazZziel
    Should't S be WxM instead of MxW?
    According to the Right Hand Rule (http://en.wikipedia.org/wiki/Cross_product) S'=MxW would face the opposite direction.
  • G. Mudskipper
    I think the reason the cross products are backwards is because these examples use a left-handed coordinate system (X east, Y up, Z north).

    If Y was north and Z was up, the Right Hand Rule would apply.
  • Net
    Thanks a lot for this. Looking forward to the next part, it's what I need to learn most.
  • Θ = acos(D’•B’) should read Θ = acos(D’•V’).

    Well, acksherly, it should read Θ = acos(D' · V') or Θ = acos(D′ ∙ V′). Why yes, I am a typography geek.

    Also, your sailing example is utterly wrong. Sails don’t work like that. ;-) Keep it up, though!
  • David was referring to the angle that would maximize the surface area of the sail presented to the wind. He never claimed to be calculating the optimum sail trim angle for maximizing boat speed. Though catching as much wind as possible as David suggested is definitely optimal if you are running dead down wind.
  • David
    You got me, I don't know anything about sails or math typography :) Fixed the error you noticed, good catch!
  • Xeirxes
    I was not aware of how useful the dot and cross products were :) I always thought that they were the harder to use functions. I guess now I can imagine how useful they would be in physics applications as well. A falling object bouncing off of a slanted surface, for instance, could be easily calculated with the cross product.

    Thanks for the interesting read! I'm looking forward to seeing more from you guys in the future.
  • I know most of this from A-level mathematics but you really are great to make posts like this. It encourages people to learn mathematics and shows them the reality of programming basic games.

    If only they did this in schools.
  • timidteacher
    They usually do. You just forgot.
  • AJS
    Great article! I was feeling a bit rusty on these topics until this little gem; can't wait to read the next part!

    Btw, small typo in the post-- I think the code should read:

    //Check angle
    float angle = acos(dot(guard_facing_normalized, guard_to_hero));
  • Whiterabbit
    I would just like to say thank you David.
    I'd like to think my self as a game developer (although only really got my toes in the pool so far)
    And I'm not the greatest with math (shot in the foot right ?)
    this whole look at vectors in games is amazingly understandable for me.
    They totally need to teach math in terms of game design as a staple :D
  • Mashandar
    Thank you so much for that! I always wondered how the normal fit into things and how the cross product worked, and you've explained it fabulously!

    I'm looking forward to future posts
  • A future topic I suggest is optimisations! For example always compare distance squared values instead of distance. Using distance squared saves doing a computationally expensive square root which is quite a saving when you are doing hundreds of thousands a second.
  • David
    That's a good idea: I always use distance squared for comparisons, as well as only normalizing vectors when necessary, and generally avoiding sqrt()s.
  • Excellent. More of this thanks. Most of the articles on this stuff have overly technical explanations. This wasn't hard to follow at all.
  • I must say, I couldn't understand it at first, and reread it many times (got confused by the absence of coordinations in images), but now I thing I can catch on. Keep it up.
  • David
    Sorry about that, I kept moving the origin around to try and keep the images from wasting too much space. Maybe I should label the grid lines next time!
  • David
    I added some labels to the grid lines -- please let me know if that helps!
  • Yes, very helpful. Thank you.
  • Just as motivation to continue with these in case you need it, I was just in need of a way to get diagonal movement to move at the right speed, and then I remembered this post and it made it extra easy for me :D

    keep it up david, my input scripts will forever be a paragraph shorter :D
  • Your explanation of the dot and cross products were really good; I learned more about them from your article than a semester of calculus.
  • Oliver
    Really keen for Vector Spaces and Matrix Transformations.
    You have a great way of explaining things to people.
    Is there somewhere I can learn the basics of Matrices? Recommendations anyone?
  • M
    The library is a good place to start, this article is a rare find in its quality. Books are generally better than most internet resources.
  • Pete
    AWESOME post. This is the best I have seen covering vector math with good examples. More more more plz
  • ukuk
    Fantastically well explained. Thumbs up! Everything suddenly became clear and simple...
    Looking forward to reading the next part.
  • Name
    Hi, this is very helpful! You lost me during the second paragraph of the section on normalizing a vector, though. I feel like there was supposed to be a paragraph in the middle explaining how the latter paragraph relates to the first, or defining the term "normalizing a vector," for that matter. :S
  • Mohamed idris
    Hi everyone plz I'm new here i mean it's my first year in the university so i want to understand this subject cause it's very important for me really cause i didn't understand anything in the university so if anyone can explain it to me from begging i will be thanks full for him or her this is my email:maed_89@hotmail.com
blog comments powered by Disqus
Powered by WordPress ; Entries (RSS) and Comments (RSS).