How to calculate Tangent and Binormal?

Question

Talking about bump mapping, specular highlight and these kind of things in OpenGL Shading Language (GLSL)

I have:

• An array of vertices (e.g. {0.2,0.5,0

Answer 1

Based on the answer from kvark, I would like to add more thoughts.

If you are in need of an orthonormalized tangent space matrix you have to do some work any way. Even if you add tangent and binormal attributes, they will be interpolated during the shader stages and at the end they are neither normalized nor they are normal to each another.

Let's assume that we have a normalized normalvector n, and we have the tangent t and the binormalb or we can calculate them from the derivations as follows:

// derivations of the fragment position
vec3 pos_dx = dFdx( fragPos );
vec3 pos_dy = dFdy( fragPos );
// derivations of the texture coordinate
vec2 texC_dx = dFdx( texCoord );
vec2 texC_dy = dFdy( texCoord );
// tangent vector and binormal vector
vec3 t = texC_dy.y * pos_dx - texC_dx.y * pos_dy;
vec3 b = texC_dx.x * pos_dy - texC_dy.x * pos_dx;

Of course an orthonormalized tangent space matrix can be calcualted by using the cross product, but this would only work for right-hand systems. If a matrix was mirrored (left-hand system) it will turn to a right hand system:

t = cross( cross( n, t ), t ); // orthonormalization of the tangent vector
b = cross( n, t );             // orthonormalization of the binormal vector 
                               //   may invert the binormal vector
mat3 tbn = mat3( normalize(t), normalize(b), n );

In the code snippet above the binormal vector is reversed if the tangent space is a left-handed system. To avoid this, the hard way must be gone:

t = cross( cross( n, t ), t ); // orthonormalization of the tangent vector
b = cross( b, cross( b, n ) ); // orthonormalization of the binormal vectors to the normal vector 
b = cross( cross( t, b ), t ); // orthonormalization of the binormal vectors to the tangent vector
mat3 tbn = mat3( normalize(t), normalize(b), n );

A common way to orthogonalize any matrix is the Gram–Schmidt process:

t = t - n * dot( t, n ); // orthonormalization ot the tangent vectors
b = b - n * dot( b, n ); // orthonormalization of the binormal vectors to the normal vector 
b = b - t * dot( b, t ); // orthonormalization of the binormal vectors to the tangent vector
mat3 tbn = mat3( normalize(t), normalize(b), n );

Another possibility is to use the determinant of the 2*2 matrix, which results from the derivations of the texture coordinates texC_dx, texC_dy, to take the direction of the binormal vector into account. The idea is that the determinant of a orthogonal matrix is 1 and the determined one of a orthogonal mirror matrix -1.

The determinant can eihter be calcualted by the GLSL function determinant( mat2( texC_dx, texC_dy ) or it can be calcualated by it formula texC_dx.x * texC_dy.y - texC_dy.x * texC_dx.y.

For the calculation of the orthonormalized tangent space matrix, the binormal vector is no longer required and the calculation of the unit vector (normalize) of the binormal vector can be evaded.

float texDet = texC_dx.x * texC_dy.y - texC_dy.x * texC_dx.y;
vec3 t = texC_dy.y * pos_dx - texC_dx.y * pos_dy;
t      = normalize( t - n * dot( t, n ) );
vec3 b = cross( n, t );                      // b is normlized because n and t are orthonormalized unit vectors
mat3 tbn = mat3( t, sign( texDet ) * b, n ); // take in account the direction of the binormal vector

Answer 2

Generally, you have 2 ways of generating the TBN matrix: off-line and on-line.

• On-line = right in the fragment shader using derivative instructions. Those derivations give you a flat TBN basis for each point of a polygon. In order to get a smooth one we have to re-orthogonalize it based on a given (smooth) vertex normal. This procedure is even more heavy on GPU than initial TBN extraction.

  // compute derivations of the world position
  vec3 p_dx = dFdx(pw_i);
  vec3 p_dy = dFdy(pw_i);
  // compute derivations of the texture coordinate
  vec2 tc_dx = dFdx(tc_i);
  vec2 tc_dy = dFdy(tc_i);
  // compute initial tangent and bi-tangent
  vec3 t = normalize( tc_dy.y * p_dx - tc_dx.y * p_dy );
  vec3 b = normalize( tc_dy.x * p_dx - tc_dx.x * p_dy ); // sign inversion
  // get new tangent from a given mesh normal
  vec3 n = normalize(n_obj_i);
  vec3 x = cross(n, t);
  t = cross(x, n);
  t = normalize(t);
  // get updated bi-tangent
  x = cross(b, n);
  b = cross(n, x);
  b = normalize(b);
  mat3 tbn = mat3(t, b, n);
  

• Off-line = prepare tangent as a vertex attribute. This is more difficult to get because it will not just add another vertex attrib but also will require to re-compose all other attributes. Moreover, it will not 100% give you a better performance as you'll get an additional cost of storing/passing/animating(!) vector3 vertex attribute.

The math is described in many places (google it), including the @datenwolf post.

The problem here is that 2 vertices may have the same normal and texture coordinate but different tangents. That means you can not just add a vertex attribute to a vertex, you'll need to split the vertex into 2 and specify different tangents for the clones.

The best way to get unique tangent (and other attribs) per vertex is to do it as early as possible = in the exporter. There on the stage of sorting pure vertices by attributes you'll just need to add the tangent vector to the sorting key.

As a radical solution to the problem consider using quaternions. A single quaternion (vec4) can successfully represent tangential space of a pre-defined handiness. It's easy to keep orthonormal (including passing to the fragment shader), store and extract normal if needed. More info on the KRI wiki.

Answer 3

The relevant input data to your problem are the texture coordinates. Tangent and Binormal are vectors locally parallel to the object's surface. And in the case of normal mapping they're describing the local orientation of the normal texture.

So you have to calculate the direction (in the model's space) in which the texturing vectors point. Say you have a triangle ABC, with texture coordinates HKL. This gives us vectors:

D = B-A
E = C-A

F = K-H
G = L-H

Now we want to express D and E in terms of tangent space T, U, i.e.

D = F.s * T + F.t * U
E = G.s * T + G.t * U

This is a system of linear equations with 6 unknowns and 6 equations, it can be written as

| D.x D.y D.z |   | F.s F.t | | T.x T.y T.z |
|             | = |         | |             |
| E.x E.y E.z |   | G.s G.t | | U.x U.y U.z |

Inverting the FG matrix yields

| T.x T.y T.z |           1         |  G.t  -F.t | | D.x D.y D.z |
|             | = ----------------- |            | |             |
| U.x U.y U.z |   F.s G.t - F.t G.s | -G.s   F.s | | E.x E.y E.z |

Together with the vertex normal T and U form a local space basis, called the tangent space, described by the matrix

| T.x U.x N.x |
| T.y U.y N.y |
| T.z U.z N.z |

Transforming from tangent space into object space. To do lighting calculations one needs the inverse of this. With a little bit of exercise one finds:

T' = T - (N·T) N
U' = U - (N·U) N - (T'·U) T'

Normalizing the vectors T' and U', calling them tangent and binormal we obtain the matrix transforming from object into tangent space, where we do the lighting:

| T'.x T'.y T'.z |
| U'.x U'.y U'.z |
| N.x  N.y  N.z  |

We store T' and U' them together with the vertex normal as a part of the model's geometry (as vertex attributes), so that we can use them in the shader for lighting calculations. I repeat: You don't determine tangent and binormal in the shader, you precompute them and store them as part of the model's geometry (just like normals).

(The notation between the vertical bars above are all matrices, never determinants, which normally use vertical bars instead of brackets in their notation.)