From OpenGL to Metal – The Projection Matrix Problem

Apple’s Metal API has been designed to make porting your OpenGL/OpenGLES applications easy. In most cases, there is a 1:1 equivalent for each OpenGL call and the only major change is how instead of a state engine one uses RenderState objects and so forth. Even matrices are column-based in both APIs. However, the devil is in the detail as I’ve found out today.

I had some weird problems when implementing an orthogonal projection view. For perspective projection, I had used the same matrix code as my OpenGL renderer and everything worked fine but when using the orthogonal projection matrix, my objects simply disappeared! So if the perspective matrix worked, why should the orthogonal matrix make problems if it works fine for OpenGL? I was stumped and debugged through shader code, frame captures, matrix multiplications and so forth.

As it turns out, I had just blindly used my OpenGL matrix code and it seemed to work so I was under the impression the whole matrix math for Metal is the same as for OpenGL. That is true for the most part except for one tiny detail, found in the Metal Programming Guide, p. 51 “Working with Viewport and Pixel Space Coordinates”:

“Metal defines its Normalized Device Coordinate (NDC) system as a 2x2x1 cube with its center at (0, 0, 0.5). The left and bottom for x and y, respectively, of the NDC system are specified as -1. The right and top for x and y, respectively, of the NDC system are specified as +1.”

So where is the problem? Well, OpenGL defines NDC as a 2x2x2 cube with its center at (0,0,0). If you have no idea what NDC is, the gist of it is that each vertex is run through four types of transformation:

1. Model matrix: The object space to world space transformation
2. View matrix: The world space to camera space transformation
3. Projection matrix: The camera space to NDC space transformation
4. Viewport: The NDC space is then mapped via the viewport to the 2D pixels on your UIView.

By using the OpenGL matrix in Metal, I had basically cut my view NDC cube in half and all the fragments that end up in the first half are automatically discarded. Due to the way perspective shortening works, this isn’t much of a problem for the perspective matrix but with the orthogonal matrix, all my objects were in the first half of the NDC cube and thus disappeared! If one checks Apple’s MetalBasic3D sample code, one can verify that the matrix they construct is in fact not the OpenGL projection matrix! I haven’t found that mentioned in any of the WWDC videos and there are a number of websites that also make the mistake of using the OpenGL projection matrix. So what is the correct matrix?

The Math

For reference, here is an excellent site that explains how a perspective and orthogonal matrix can be derived. For Metal with its specific NDC, we could either derive the matrix in a similar fashion or use the easy way: transform OpenGL’s NDC to Metal’s NDC. This is done by doing a post multiplication which first scales the 2x2x2 cube to 2x2x1 and then shifts it by 0.5 to have the correct center:

\(\begin{array}{lcccccl}M_{{proj}_{metal}} & = & M_{adjust} \cdot M_{{proj}_{OpenGL}} & = & \begin{bmatrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0.5 & 0.5 \\ 0 & 0 & 0 & 1\end{bmatrix}  & \cdot & \begin{bmatrix} \frac{2n}{r-l} & 0 & \frac{r+l}{r-l} & 0 \\ 0 & \frac{2n}{t-b} & \frac{t+b}{t-b} & 0 \\ 0 & 0 & \frac{-(f+n)}{f-n} & \frac{-2fn}{f-n} \\ 0 & 0 & -1 & 0 \end{bmatrix} & = & \begin{bmatrix} \frac{2n}{r-l} & 0 & \frac{r+l}{r-l} & 0 \\ 0 & \frac{2n}{t-b} & \frac{t+b}{t-b} & 0 \\ 0 & 0 & \frac{-f}{f-n} & \frac{-fn}{f-n} \\ 0 & 0 & -1 & 0 \end{bmatrix} \\ M_{{orth}_{metal}} & = & M_{adjust} \cdot M_{{orth}_{OpenGL}} & = & \begin{bmatrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0.5 & 0.5 \\ 0 & 0 & 0 & 1\end{bmatrix}  & \cdot & \begin{bmatrix} \frac{2}{r-l} & 0 & 0 & -\frac{r+l}{r-l} \\ 0 & \frac{2}{t-b} & 0 & -\frac{t+b}{t-b} \\ 0 & 0 & \frac{-2}{f-n} & -\frac{f+n}{f-n} \\ 0 & 0 & 0 & 1 \end{bmatrix} & = & \begin{bmatrix} \frac{2}{r-l} & 0 & 0 & -\frac{r+l}{r-l} \\ 0 & \frac{2}{t-b} & 0 & -\frac{t+b}{t-b} \\ 0 & 0 & \frac{-1}{f-n} & -\frac{n}{f-n} \\ 0 & 0 & 0 & 1 \end{bmatrix}\end{array} \)