Last weekend (26-27 April 2014) I took part in Ludum Dare 29. My game, Frog V Ducks is quite fun (at least I think so) but not hugely original. It looks a bit like this:

While the game isn’t that innovative, it uses quite a cool voxel sprite technique, and this post describes how it works.

This technique uses GLSL shaders – these are small programs that run on the GPU itself and can modify data as it streams through its rendering pipeline. In OpenGL ES and WebGL, there are two types.

• Vertex Shaders – these act on the vertices of your 3D mesh and implement its translation, rotation and projection
• Fragment Shaders – these act on each pixel of your screen and perform operations like texture mapping and per-pixel lighting

The approach I used is entirely implemented in the fragment shader – the vertex shader just does transformation of a plain cube.  Here’s a zoomed-in shot of a duck, compared with a similar image if I replace the fragment shader with code that just makes every pixel red.

As you can see, there is no topology for any of the models in the mesh itself.

Instead, the topology is encoded as voxels in a texture, which looks like this:

The texture (a “voxel map”) consists of 8 layers of 8×8 slices through the model.  Each slice has the X and Y axes oriented the same as the 3D model will be, and the layers are stuck together in the Z axis.  Colored pixels have the same color in the 3D model – transparent pixels are transparent and can be seen through.  (This conveniently means I can build my 3D models in Gimp or any other paint package.)

The algorithm used to render this is per-pixel raycasting.  For each pixel, the shader works out the path of the ray from the camera to the cube, and then steps along this ray through the voxel map until it comes to an opaque voxel.  Once it hits an opaque voxel, it does a simple lighting calculation and then returns the color of the voxel as the color of the pixel.  If it doesn’t find any opaque voxels, the shader “discards” the pixel, causing it to be rendered transparently.

Raycasting is a fairly standard algorithm, but there are two slightly unusual things here.

• Often raycasting does two “render-to-texture” renders to calculate the points at which the ray intersects with the front and back sides of the cube.  See this article for an example.  This article gives an alternative, in which the fragment shader calculates the ray direction based on screen coordinate and knowledge of the position of the camera and then calculates the intersection with the front and back of the cube.  My approach is a hybrid – I use the same approach as the “render-to-texture” method to determine the intersection point with the front face and then use knowledge of the position of the camera to calculate the ray direction.
• Often raycasting uses high-resolution datasets (e.g. MRI scans).  The algorithm then steps a regular step distance along the ray’s path.  In my implementation, the datasets are very low resolution and, to avoid aliasing, it’s important that the shader examines every voxel along the path.  Rather than stepping a regular distance, it instead performs a series of cube intersection tests (each of which is actually surprisingly efficient).  Due to the limited resolution (8x8x8), there is also a tight bound on the number of voxels that a ray could hypothetically pass through.

For the full code, see LD29.Sprite3DRenderer in https://github.com/matt-williams/ld29/blob/master/ld29.js.  Note that a restriction in WebGL means that the depth of each pixel is not calculated – in full OpenGL shaders, it would be possible to fix this.  The only impact of this is aliasing when the cubes defining two objects intersect.

I’d like to experiment more with this.  In particular, I’d like to see whether I could apply a texture map on top of the model, and also encode more data into the voxel map, e.g. to add half-full voxels defined by the plane that divided the full and empty halves (allowing the models to be less blocky).