Hints for 3D Programming

Programming in 3D is difficult as everyone knows who has tried to do it. It is particularily difficult because it is easy to program something that looks almost but is wrong nevertheless. Here are some hints from my personal experience. The key is to address 3D transformations and coordinate systems in a more formal and rigorous way then just by try and error. I believe sticking to the following procedure greatly helps to find errors.

• First assign and document coordinate systems! Assign a coordinate system to the world and every body which is moving. You can view everything as a transformation of points between coordinate systems.
• If a body is moving, there is a coordinate system at each point of time. Motion is described as a transformation of points from the body coordinate system at one time step to the bodies coordinate system at another point of time.
• My personal recommendation: X-Right, Y-Forward, Z-Up for world and vehicle/body coordinates, because it is easy to visualize with your right hand. X-Right, Y-Down, Z-Front for a camera, because then camera X/Y correspond to image x/y.
• Represent the scene inside your program by 4*4 homogenous matrices (or quarternions if you want to) that encode the transformation of a point from one coordinate system into another.

• Name your point/vector variables  p_A  if they represent the point  p  in coordinates of coordinate system  A ! If you know which coordinate system it refers to and you documented the sytems, you can debug, whether the value of such a variable is correct.

• Name your transformation variables  A2B  if it converts a point p in A-coordinates  p_a  to the same point in  p_B  in B coordinates, so p_B = A2B * p_A. If  A2B  is a matrix, then the colums of  A2B  are the X/Y/Z/origin of  A  in  B  coordinates. If you know that and have documented your coordinate systems, you can debug whether the value of a transformation variable is correct.

• If you name your variables according to the above convention, you can see whether an expression is correct just by reading it from right to left. p_Camera = Camera2World^{-1}*Object2World*p_Object} is correct, because Object2``World transforms p_Object into p_World, {{{ Camera2World^{-1}  is World2Camera and transforms p_World into p_Camera. There are 8 ways to write down this equation! 7 are wrong. This problem is the most common reason for bugs in 3D programs and it's trivial with the appropriate naming convention.

• Use A2World rather than World2A because it's easier to imagine the objects X/Y/Z axis or origin in world coordinates than vice versa.

• If you have a hierarchy of transformations use  Local2Global  not  Global2Local  for the same reason as above.

• If something doesn't work, go through the sequence of computations and debug whether the value in the variables corresponds to their documented definition.

• Don't simply try adding  ^{-1}  or  ^T  or permuting matrix terms! There are many computations that are wrong but look almost right. And even if they look completely right, the value of internal variable may be incompatible to their documented definition and as a result if you add new correct code the result may be wrong.

• With 4*4 homogenous matrices you never need  ^T  only  ^{-1} . For 3*3 orthonormal matrices, both are the same.

• If you are estimating transformations in a calibration or tracking setting (for instance using Levenberg-Marquardt), represent the current estimate as a matrix / quarternion and parametrize only the change in a single step with tree variable, e.g. using matrix exponential.