io7m-r1 0.10.0
io7m-r1 0.10.0 Documentation
Package Information
Orientation
Overview
The io7m-r1 package implements an aggressively minimalist 3D renderer.
Features
  • A deferred rendering core for opaque objects.
  • A forward renderer, supporting a subset of the features of the deferred renderer, for rendering translucent objects.
  • A full dynamic lighting system, including both basic and variance shadow mapping. The use of deferred rendering allows for potentially hundreds of dynamic lights per scene.
  • A detailed material system, allowing for artists and developers to create surfaces with a wide variety of effects such as normal mapping, environment-mapped reflections, generic refraction, surface emission, mapped specular highlights, etc, without writing a single line of shader code. All materials are immutable but are created via simple mutable builders in code, and so are effectively dynamic [0].
  • A variety of postprocessing effects such as box blurring, fast approximate antialiasing (FXAA), color correction, bloom, etc. Effects can be applied in any order.
  • Explicit control over all resource loading and caching. For all transient resources [1], the programmer is required to provide the renderer with explicit caches, and the caches themselves are responsible for allocating and loading resources.
  • Extensive use of immutable objects for the purposes of correctness. One of the driving design concepts is that the programmer passes an immutable "snapshot" of the scene to be rendered to a rendering function, and that rendering function will always return the same image for that snapshot.
  • Simplicity. The implementation consists of a few thousand shaders generated at compile-time from the Java material and light types defined in the renderer's API. All of the usual bugs that plague programmers using 3D renderers that directly expose shaders (such as passing incorrect parameters to shaders, forgetting to pass required parameters, etc), are simply not present. The system knows every possible light and material combination, statically, and knows how to pass the right parameters to the shaders that implement them. The programmer doesn't have to worry about it.
  • Extensive use of static types. As with all io7m packages, there is extreme emphasis on using the type system to make it difficult to use the APIs incorrectly.
  • Portability. The renderer will run on any system supporting either OpenGL >= 3.0 or OpenGL ES >= 3.
Non-features
The intention of the io7m-r1 package is to essentially expose an advanced fixed-function rendering pipeline that just happens to provide all of the advanced rendering techniques expected of a modern computer game. Features specifically not implemented by the package include:
  • A scene graph. The renderer expects the programmer to provide a set of instances (with associated materials) and lights once per frame, and the renderer will obediently draw exactly those instances. This frees the programmer from having to interact with a clumsy and type-unsafe "object-oriented" scene graph as with other 3D engines, and from having to try to crowbar their own program's data structures into an existing graph system.
  • Spatial partitioning. The renderer knows nothing of the world the programmer is trying to render. The programmer is expected to have done the work of deciding which instances and lights contribute to the current image, and to provide only those lights and instances for the current frame. This means that the programmer is free to use any spatial partitioning system desired.
  • Input handling. The renderer knows nothing about keyboards, mice, joysticks. The programmer passes an immutable snapshot of a scene to the renderer, and the renderer returns an image. This means that the programmer is free to use any input system desired without having to painfully integrate their own code with an existing input system as with other 3D engines.
  • Audio. The renderer makes images, not sounds. This allows programmers to use any audio system they want in their programs.
  • Skeletal animation. The input to the renderer is a set of triangle meshes in the form of vertex buffer objects. This means that the programmer is free to use any skeletal animation system desired, providing that the system is capable of producing vertex buffer objects of the correct type as a result.
  • Model loading. The input to the renderer is a set of triangle meshes in the form of vertex buffer objects. This means that the programmer is free to use any model loading system desired, providing that the system is capable of producing vertex buffer objects of the correct type as a result [3].
  • Future proofing. The average lifetime of a rendering system is about five years. Due to the extremely rapid pace of advancement in graphics hardware, the methods use to render graphics today will bear almost no relation to those used five years into the future. The io7m-r1 package is under no illusion that it will still be relevant in a decade's time. It is designed to get work done today, using exactly those techniques that are relevant today. It will not be indefinitely expanded and grown organically, as this would directly contradict the goal of having a minimalist and correct rendering system.
  • OpenGL ES 2 support. The ES 2 standard was written as a reaction to the insane committee politics that plagued the OpenGL 2.* standards. It is crippled to the point that it essentially cannot support almost any of the rendering techniques present in the io7m-r1 package, and is becoming increasingly irrelevant as the much saner ES 3 is adopted by hardware vendors.
Installation
Source compilation
The project can be compiled and installed with Maven:
$ mvn -C clean install
Maven
Regular releases are made to the Central Repository, so it's possible to use the io7m-r1 package in your projects with the following Maven dependency:
<dependency>
  <groupId>com.io7m.r1</groupId>
  <artifactId>io7m-r1-kernel</artifactId>
  <version>0.10.0</version>
</dependency>
All io7m.com packages use Semantic Versioning [4], which implies that it is always safe to use version ranges with an exclusive upper bound equal to the next major version - the API of the package will not change in a backwards-incompatible manner before the next major version.
Platform Specific Issues
There are currently no known platform-specific issues.
License
All files distributed with the io7m-r1 package are placed under the following license:
Copyright © 2014 <code@io7m.com> http://io7m.com

Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
        
Design and Implementation
Conventions
Overview
This section attempts to document the mathematical and typographical conventions used in the rest of the documentation.
Mathematics
Rather than rely on untyped and ambiguous mathematical notation, this documentation expresses all mathematics in strict Haskell 2010 with no extensions. All Haskell sources are included along with the documentation and can therefore be executed from the command line GHCi tool in order to interactively check results and experiment with functions.
When used within prose, functions are referred to using fully qualified notation, such as (Vector3f.cross n t). This is the application of the cross function defined in the Vector3f module, to the arguments n and t.
Formal examples and definitions, however, will typically be defined within their own modules, possibly with import statements used to allow for shorter names. As an example [LightDiffuse.hs]:
module LightDiffuse where

import qualified Color3
import qualified Direction
import qualified Normal
import qualified Spaces
import qualified Vector3f

diffuse ::  Direction.T Spaces.Eye -> Normal.T -> Color3.T -> Float -> Vector3f.T
diffuse stl n light_color light_intensity =
  let 
    factor       = max 0.0 (Vector3f.dot3 stl n)
    light_scaled = Vector3f.scale light_color light_intensity
  in 
    Vector3f.scale light_scaled factor
Concepts
Overview
This section attempts to provide a rough overview of the concepts present in the io7m-r1 package. Specific implementation details, mathematics, and other technical information is given in later sections that focus on each concept in detail.
Renderers
The io7m-r1 package consists of a set of renderers. A renderer consumes an immutable visible set and produces an image, writing the image to a given framebuffer.
The renderers expose an interface of conceptually pure functions from visible sets to images. That is, the renderers should be considered to simply take visible sets as input and return images as output. In reality, because the Java language is not pure and because the code is required to perform I/O in order to speak to the GPU, the renderer functions are not really pure. Nevertheless, for the sake of ease of use, lack of surprising results, and correctness, the renderers at least attempt to adhere to the idea of pure functional rendering! This means that the renderers are very easy to integrate into any existing system: They are simply functions that are evaluated whenever the programmer wants an image. The renderers do not have their own "main loop", they do not have any concept of time, do not remember any images that they have produced previously, do not maintain any state of their own, and simply write their results to a programmer-provided framebuffer. Passing the same immutable visible set to a renderer multiple times should result in the same image each time.
Visible Sets
A visible set is an immutable snapshot of all instances and lights that contribute to an image. The name implies that a renderer expects to receive only those instances and lights that are actually visible from the perspective of the observer. This is a conscious design decision that frees the renderer from the complicated task of trying to decide which objects are visible and which are not. A programmer using the renderer is expected to be using some sort of spatial partitioning data structure [5] to efficiently decide which objects are visible for the current rendering call. This is essentially the same approach taken by the OpenGL API: The API draws what it is told to draw, and does not try (beyond clipping primitives based on the viewing frustum), to intelligently decide what should and should not be drawn.
Framebuffers
A framebuffer is a rectangular region of memory allocated on the GPU that can accept the results of rendering. The programmer typically allocates one framebuffer, passes it to a renderer along with a visible set, and the renderer populates the given framebuffer with an image of the rendered visible set. The programmer can then copy the contents of this framebuffer to the screen for viewing, pass it on to a separate filter for extra visual effects, use it as a texture to be applied to objects in further rendered visible sets, etc.
Meshes
A mesh is a collection of vertices that define a polyhedral object, along with a list of indices that describe how to make triangles out of the given vertices.
Meshes are allocated on the GPU and can be shared between any number of instances (meaning that rendering 100 identical objects does not require storing 100 copies of the mesh data).
Transforms
A transform moves coordinates in one coordinate space to another. Essentially, a transform is used to position and orient a mesh inside a visible set.
Camera
A camera defines both the viewing position and viewing projection for the visible set. The viewing projection describes the orthographic or perspective projection used to render the visible set, and the viewing position is used to transform all instances and lights in the visible set to eye-space during rendering. The camera is represented by the KCamera type.
A visible set always contains a single camera, because, as mentioned earlier, a visible set is supposed to represent the instances and lights that are visible from the observation point that the camera describes.
Materials
A material describes the surface properties of an object. A material may either be opaque or translucent. An object with a opaque material completely occludes the pixels of all other objects that appear behind it. An object with a translucent material is blended with the objects that appear behind it.
Materials consist of a multitude of different properties describing different aspects of the surface. For example, most opaque materials have data describing all of the following:
  • The surface albedo; the basic color of the surface prior to any lighting.
  • The surface depth properties; to give per-pixel control over "transparency" without requiring the overhead of making the material translucent.
  • The surface emissive properties; to present the illusion of surfaces emitting light.
  • The surface environment properties; to provide environment-mapped reflections and other effects.
  • The surface normal properties; to provide per-pixel control of surface normal vectors ("normal mapping").
  • The surface specular properties; to provide per-pixel control of surface specularity.
An extremely trivial material (with a simple red albedo and no other properties) applied to a square, lit by a directional light:
A complex material (with mapped normals, specular highlights, an environment map, and a textured albedo) applied to the same square, lit by a spherical light:
Materials are immutable once created, but are created through the use of mutable builder types. This allows renderers to effectively support "animated" materials without actually needing to know anything about them; the renderers see ordinary immutable materials, but the programmer is supplying new materials each time the renderer is called, giving the illusion that one material is changing over time. As an example, the type of builders for opaque materials is KMaterialOpaqueBuilderType.
Instances
An instance is essentially a reference to a mesh, a transform (to position that mesh in the visible set), and a material to define the appearance of the mesh surface.
Lights
A light describes a light source within a visible set. There are many different types of lights, each with different behaviours. Lights may or may not cast shadows, depending on their type. All lighting in the io7m-r1 package is completely dynamic; there is no support for static lighting in any form. Shadows are exclusively provided via shadow-mapping, resulting in efficient per-pixel shadows.
Types of lights include:
  • KLightDirectional - a directional light that simulates parallel light rays without any origin. Directional lights cannot cast shadows (because directional lights do not have origins).
  • KLightSphereWithoutShadow - a spherical light. A spherical light casts light in all directions from a given position in world-space, up to a given radius. The emitted light is attenuated over distance based on a configurable falloff value. This type of light is often referred to as a point light in other renderers.
  • KLightProjectiveWithoutShadow - a projective light. A projective light effectively projects a given image onto the visible set from a given position in world-space, up to a given radius. The emitted light is attenuated over distance based on a configurable falloff value.
  • KLightProjectiveWithShadowBasic - a projective light that can also cast basic shadows. Basic shadows are very cheap to compute, but can suffer from aliasing issues (resulting in sharp edges to shadows).
  • KLightProjectiveWithShadowVariance - a projective light that can also cast variance shadows. Variance shadows are slightly more expensive to compute than basic shadows, but can be filtered by hardware, resulting in attractive soft shadows.
  • KLightSpherePseudoWithShadowBasic - a pseudo-spherical light. A pseudo-spherical light behaves like a spherical light but is emulated via at most six projective lights arranged such that each light provides a section of the sphere. Individual lights can be enabled and disabled, and the lights can project basic shadows.
  • KLightSpherePseudoWithShadowVariance - is to KLightSpherePseudoWithShadowBasic as KLightProjectiveWithShadowVariance is to KLightProjectiveWithShadowBasic.
  • KLightSphereTexturedCubeWithoutShadow - a spherical light that projects a cube map in all directions.
As with materials (and indeed, all other objects in the io7m-r1 package), all lights are immutable once created, but are created through the use of mutable builder types. This allows renderers to effectively support "animated" lights without actually needing to know anything about them; the renderers see ordinary immutable lights, but the programmer is supplying new lights each time the renderer is called, giving the illusion that lights are changing over time. As an example, the type of builders for spherical lights without shadows is KLightSphereWithoutShadowBuilderType.
Light groups
Light groups are a means to partition a visible set into separate lighting environments. An instance belongs to exactly one light group, but a given light can be placed into any number of light groups.
The io7m-r1 package, due to the current state of graphics hardware, implements purely local illumination [6]. Because of this, when using lights that do not project shadows, it is possible for lights to "bleed" through objects that would normally occlude their radiance had the renderer implemented physically correct lighting. As an example:
The visible set contains three lights: A red spherical light s0 in the left room, a white spherical light s1 in the middle room, and a blue spherical light s2 in the right room. The visible set contains four instances: The left room i0, the middle room i1, a piece of furniture i2 in the middle room, and the right room i3. None of the lights are configured to cast shadows. Note that the red and blue lights bleed into the center room as if the two dividing walls were not even there! Light groups can help to solve this issue (without requiring shadow mapping).
First, three light groups are created: g0, g1, and g2. The light s0 and instance i0, are added to g0. The light s1 and instances i1 and i2, are added to g1. Finally, light s2 and instance i3 are added to g2. With these groups configured, the renderer produces the following image:
Instances i1 and i2 are no longer affected by lights s0 and s2, and so on. The image looks more physically correct, at the expense of having somewhat hard transitions between the rooms, without actually having to calculate any shadows [7].
Light groups can also be used for other miscellaneous visual effects. For example, an object in a visible set could be highlighted by placing it in its own light group, and adding a strong red directional light to that group. No other objects in the visible set would be affected by the light and the object would, as a result, be displayed very conspicuously!
The majority of visible sets will contain only a single light group. They are intended to assist with working around the lack of global illumination, and with implementing specific visual effects.
Image Filters
An image filter is similar to a renderer except that it accepts a framebuffer as input (as opposed to a visible set), processes the image in the framebuffer in some manner, and then writes the results to an output framebuffer (possibly the same as the input framebuffer). It is used to provide visual effects such as full-screen blurring, color-correction, emission, and others.
Typically, an image filter accepts a framebuffer that was populated by a deferred renderer, and therefore has access to much more per-pixel data than a typical image processor. For example, for each pixel in the image, a framebuffer from a deferred renderer will contain at least the linear depth value of that pixel, and the normal vector of the surface at that pixel. If the filter also has access to the viewing projection that was used to produce the image, then it can actually efficiently reconstruct the original eye-space position of the pixel! This allows for interesting effects such as fog and depth-of-field simulation, that rely on knowing the original positions of objects within the visible set - information that would not usually be available to a simple image-based filter.
Image Sources
An image source is analogous to an image filter that does not take a framebuffer as input. They are usually provided as a convenience (such as conveniently populating a framebuffer with a fixed image prior to rendering).
Coordinate systems
Conventions
This section attempts to describe the mathematical conventions that the io7m-r1 package uses with respect to coordinate systems. The io7m-r1 package generally does not deviate from standard OpenGL conventions, and this section does not attempt to give a rigorous formal definition of these existing conventions. It does however attempt to establish the naming conventions that the package uses to refer to the standard coordinate spaces [8].
The io7m-r1 package uses the jtensors package for all mathematical operations on the CPU, and therefore shares its conventions with regards to coordinate system handedness. Important parts are repeated here, but the documentation for the jtensors package should be inspected for details.
Any of the matrix functions that deal with rotations assume a right-handed coordinate system. This matches the system conventionally used by OpenGL (and most mathematics literature). A right-handed coordinate system assumes that if the viewer is standing at the origin and looking towards negative infinity on the Z axis, then the X axis runs horizontally (left towards negative infinity and right towards positive infinity), and the Y axis runs vertically (down towards negative infinity and up towards positive infinity). The following image demonstrates this axis configuration:
The jtensors package adheres to the convention that a positive rotation around an axis represents a counter-clockwise rotation when viewing the system along the negative direction of the axis in question.
The package uses the following matrices to define rotations around each axis:
Which results in the following matrix for rotating r radians around the axis given by (x, y, z), assuming s = sin(r) and c = cos(r) [9]:
Types
The io7m-r1 package uses so-called phantom type parameters to statically indicate the coordinate systems of vectors, and the types of transformations that matrices represent. For example, a value of type RVectorI3F<RSpaceObjectType> represents an immutable three-dimension vectors with coordinate specified in object space. A value of type RMatrixI4x4F<RSpaceTransformViewType> represents an immutable 4x4 matrix that contains a transformation from world space to eye space.
Due to the limited nature of Java's type system, it is obviously possible for the programmer to deliberately construct vectors and matrices that do not represent valid coordinates or transforms in any coordinate space. However, mistakes involving the mixing up of coordinate systems are rampant in graphics programming, and in practice, the system as implemented catches many of the mistakes at compile time.
The package contains the following coordinate system and transform indexed types:
Object space
Object space is the local coordinate system used to describe the positions of vertices in meshes. For example, a unit cube with the origin placed at the center of the cube would have eight vertices with positions expressed as object-space coordinates:
cube = {
    (-0.5, -0.5, -0.5),
    ( 0.5, -0.5, -0.5),
    ( 0.5, -0.5,  0.5),
    (-0.5, -0.5,  0.5),
  
    (-0.5,  0.5, -0.5),
    ( 0.5,  0.5, -0.5),
    ( 0.5,  0.5,  0.5),
    (-0.5,  0.5,  0.5)
  }
In other rendering systems, object space is sometimes referred to as local space, or model space.
In the io7m-r1 package, object space is indicated by the RSpaceObjectType.
World space
In order to position objects in a scene, they must be assigned a transform that can be applied to each of their object space vertices to yield absolute positions in so-called world space.
As an example, if the unit cube described above was assigned a transform that moved its origin to (3, 5, 1), then its object space vertex (-0.5, 0.5, 0.5) would end up at (3 + -0.5, 5 + 0.5, 1 + 0.5) = (2.5, 5.5, 1.5) in world space.
In the io7m-r1 package, a transform applied to an object produces a 4x4 model matrix. Multiplying the model matrix with the positions of the object space vertices yields vertices in world space.
In the io7m-r1 package, world space is indicated by the RSpaceWorldType.
Eye space
Eye space represents the coordinate system of the camera of a given visible set. In eye space, the observer is implicitly fixed at the origin (0.0, 0.0, 0.0) and is looking towards infinity in the negative Z direction.
The main purpose of eye space is to simplify the mathematics required to implement various algorithms such as lighting. The problem with implementing these sorts of algorithms in world space is that one must constantly take into account the position of the observer (typically by subtracting the location of the observer from each set of world space coordinates and accounting for any change in orientation of the observer). By fixing the orientation of the observer towards negative Z, and the position of the observer at (0.0, 0.0, 0.0), and by transforming all vertices of all objects into the same system, the mathematics of lighting are greatly simplified. The majority of the rendering algorithms used in the io7m-r1 package are implemented in eye space.
In the io7m-r1 package, the camera produces a 4x4 view matrix. Multiplying the view matrix with any given world space position yields a position in eye space. In practice, the view matrix v and the current object's model matrix m are concatenated (multiplied) to produce a model-view matrix mv = v * m [10], and mv is then passed directly to the renderer's vertex shaders to transform the current object's vertices [11].
Additionally, as the io7m-r1 package does all lighting in eye space, it's necessary to transform the object space normal vectors given in mesh data to eye space. However, the usual model-view matrix will almost certainly contain some sort of translational component and possibly a scaling component. Normal vectors are not supposed to be translated; they represent directions! A non-uniform scale applied to an object will also deform the normal vectors, making them non-perpendicular to the surface they're associated with:
With the scaled triangle on the right, the normal vector is now not perpendicular to the surface (in addition to no longer being of unit length). The red vector indicates what the surface normal should be.
Therefore it's necessary to derive another 3x3 matrix known as the normal matrix from the model-view matrix that contains just the rotational component of the original matrix. The full derivation of this matrix is given in Mathematics for 3D Game Programming and Computer Graphics, Third Edition [12]. Briefly, the normal matrix is equal to the inverse transpose of the top left 3x3 elements of an arbitrary 4x4 model-view matrix.
In other rendering systems, eye space is sometimes referred to as camera space, or view space.
In the io7m-r1 package, eye space is indicated by the RSpaceEyeType.
Clip space
Clip space is a homogeneous coordinate system in which OpenGL performs clipping of primitives (such as triangles). In OpenGL, clip space is effectively a left-handed coordinate system by default [13]. Intuitively, coordinates in eye space are transformed with a projection (normally either an orthographic or perspective projection) such that all vertices are projected into a homogeneous unit cube placed at the origin - clip space - resulting in four-dimensional (x, y, z, w) positions. Positions that end up outside of the cube are clipped (discarded) by dedicated clipping hardware, typically producing more triangles as a result.
A projection effectively determines how objects in the three-dimensional scene are projected onto the two-dimensional viewing plane (a computer screen, in most cases). A perspective projection transforms vertices such that objects that are further away from the viewing plane appear to be smaller than objects that are close to it, while an orthographic projection preserves the perceived sizes of objects regardless of their distance from the viewing plane.
Because eye space is a right-handed coordinate system by convention, but by default clip space is left-handed, the projection matrix used will invert the sign of the z component of any given point.
In the io7m-r1 package, the camera produces a 4x4 projection matrix. The projection matrix is passed, along with the model-view matrix, to the renderer's vertex shaders. As is normal in OpenGL, the vertex shader produces clip space coordinates which are then used by the hardware rasterizer to produce color fragments onscreen.
In the io7m-r1 package, clip space is indicated by the RSpaceClipType.
Normalized-device space
Normalized-device space is, by default, a left-handed [14] coordinate space in which clip space coordinates have been divided by their own w component (discarding the resulting w = 1 component in the process), yielding three dimensional coordinates. The range of values in the resulting coordinates are effectively normalized by the division to fall within the ranges [(-1, -1, -1), (1, 1, 1)] [15]. The coordinate space represents a simplifying intermediate step between having clip space coordinates and getting something projected into a two-dimensional image (screen space) for viewing.
The io7m-r1 package does not directly use or manipulate values in normalized-device space; it is mentioned here for completeness.
In the io7m-r1 package, normalized-device space is indicated by the RSpaceNDCType.
Screen space
Screen space is, by default, a left-handed coordinate system representing the screen (or window) that is displaying the actual results of rendering. If the screen is of width w and height h, and the current depth range of the window is [n, f], then the range of values in screen space coordinates runs from [(0, 0, n), (w, h, f)]. The origin (0, 0, 0) is assumed to be at the bottom-left corner.
The depth range is actually a configurable value, but the io7m-r1 package keeps the OpenGL default. From the glDepthRange function manual page:
After clipping and division by w, depth coordinates range from -1 to 1, 
corresponding to the near and far clipping planes. glDepthRange specifies a 
linear mapping of the normalized depth coordinates in this range to window 
depth coordinates. Regardless of the actual depth buffer implementation, 
window coordinate depth values are treated as though they range from 0 
through 1 (like color components). Thus, the values accepted by 
glDepthRange are both clamped to this range before they are accepted.
The setting of (0,1) maps the near plane to 0 and the far plane to 1. 
With this mapping, the depth buffer range is fully utilized.
As OpenGL, by default, specifies a depth range of [0, 1], the positive Z axis points away from the observer, making the coordinate system left handed.
In the io7m-r1 package, screen space is indicated by the RSpaceWindowType.
Rendering Process
This section attempts to give a high-level view of the rendering process as it occurs in the io7m-r1 package.
A rough diagram of the rendering process for a typical program is as follows, with red nodes indicating actions taken by the programmer, and blue nodes indicating actions performed by the io7m-r1 package:
During the initialization stage, the programmer is required to create a framebuffer that will contain the results of rendering. Then, the programmer loads all of the meshes that will be used during rendering [16]. Then, the programmer creates instances of any renderers and/or filters that will be used during rendering.
Most applications will enter into some form of rendering loop, where a new image is produced onto the screen at a rate of sixty or so per second. The io7m-r1 package does not have its own rendering loop: The programmer is simply required to call the renderers provided by the package whenever a new image is needed.
Ordinarily, the programmer will be rendering a scene of some description. Most 3D simulations contain some sort of representation of a world containing objects and entities, with at least one observer placed within that world, with images produced of that world from the perspective of the observer. Almost certainly, the world contains some sort of spatial data structure that partitions the world into sections in order to efficiently implement collision detection algorithms, physics simulations, and for determining exactly which objects are potentially visible from the perspective of the observer. The io7m-r1 package remains completely ignorant of these details: It expects the programmer to pass it a list of objects and light sources to render, and does not make any attempt to do any spatial partitioning or visibility calculations of its own. Therefore, the first step taken by the programmer in most rendering loops is to work out exactly what needs to be rendered, and then put together a list of things that need to be rendered in a form that the io7m-r1 package can use. Concretely, the programmer constructs instances (associated with materials) and lights, placing them into an immutable snapshot known as a visible set. Because all objects in the io7m-r1 package are immutable, it may be that instances, materials, and lights are re-used from the previous loop iteration (or submitted again with minor modifications via the use of mutable builders). This is how the illusion of animated materials, lights, and instances are achieved in the io7m-r1 package: If the programmer has created a light L0 in the previous rendering loop iteration, and submits a light L1 in the current loop iteration that is structurally identical to L0 but with a slightly different intensity, then there will appear to be a single light in the scene with an intensity that varies over time. The io7m-r1 package remains completely ignorant of the passage of time and doesn't need to be concerned with keeping any sort of internal state to handle animation - simplifying the implementation drastically.
The io7m-r1 package takes the immutable scene produced by the programmer and efficiently batches all of the instances by material, aiming to reduce the number of state changes required to render all of the given instances. The instances in the scene are also separated into translucent and opaque groups. The programmer then passes this batched visible set to whichever renderer is currently being used. The renderer generates any shadow maps required for the lights in the visible set and then renders all of the opaque instances to the given framebuffer. The programmer can then optionally pass this framebuffer to a filter if desired. Then, the programmer submits the same visible set to the renderer in order to allow it to render the remaining translucent instances. Again, the programmer can optionally pass the framebuffer to another filter, or the contents of the framebuffer can simply be copied to the screen for display.
Of course, the above process is simply one possible way to use the io7m-r1 package. Real applications might use the renderers to produce images that will then be re-used in further visible sets. For example, if a visible set uses materials that use environment mapping, the programmer might pass six drastically simplified versions of a scene to the renderer in order to produce the six faces of a cube map. This cube map may then be used in materials that are used with instances to render a visible set that will actually be displayed to the screen.
Materials
Overview
This section attempts to provide information on the material system used in the io7m-r1 package as well as the rationale for its existence.
In contrast to most other rendering systems, the io7m-r1 package uses a typed material system configured directly from code, rather than having programmers and artists write shaders in a shading language directly. It was a conscious design decision to reduce flexibility in order to increase ease of use and correctness. The material system has the following disadvantages:
  • The material system was designed to accomodate the majority of rendering techniques seen in computer games circa 2014 (such as normal mapping, environment mapped reflections, etc). If the io7m-r1 package doesn't provide direct support for a technique, then the programmer is required to modify the io7m-r1 package to use it.
However, the design also allows for the following advantages:
  • The material system was designed to accomodate the majority of rendering techniques seen in computer games circa 2014 (such as normal mapping, environment mapped reflections, etc). Rather than having to tediously re-implement error-prone techniques such as normal mapping over and over, the programmer simply enables normal mapping for a material.
  • The material system provides enough metadata about a given surface for all renderers in the io7m-r1 package to give consistently correct results regardless of the lighting in a scene. For example, other rendering systems that utilize shadow mapping often give incorrect results when a user-written shader conditionally discards pixels. Shadow mapping is implemented by rendering a depth-only image of an object from a different perspective, and so the same pixels have to be discarded when rendering the depth-only image as when rendering the actual color image. In systems where the user is required to write shaders, the system has no way of knowing that some pixels need to be discarded and often has to fall back to running the full material shader with writes to the color buffer masked. The material system in the io7m-r1 package indicates statically when this can occur, and so the system simply uses a very efficient depth-only shader to render accurate shadow maps.
  • In other rendering systems (particularly those that use forward rendering), the programmer is required to write one shader per combination of material and light. That is, the programmer is expected to re-implement each of the different lighting techniques for every single material so that each can be used in any possible lighting environment. Many systems use a shader generation system to work around these sorts of problems, requiring the development of (and the programmer to learn) even more complex tools and development environments. This issue is simply not present with a static material system.
  • Because the set of possible material and light types in the io7m-r1 package is fixed, the system knows exactly how to correctly send data to all of the possible shaders. The programmer is not required to laboriously configure all of the connections between shader parameters and the data that must be supplied to shaders. All of the usual classes of bugs involving forgetting to set parameters, sending parameters of the wrong type, etc, are simply not possible.
  • The material system allows for extremely rapid development and previewing of materials. Because the components of the material systems are exposed as simple pseudo-algebraic Java types, it is extremely easy to put together a graphical user interface for configuring materials. Rendering systems that require programmers to write shaders typically end up implementing their own complete integrated development environments!
  • Because the GLSL language is fundamentally poorly designed and entirely anti-modular, any rendering system requiring the programmer to write shaders must implement its own custom shading language to give good type errors and to work identically across all of the different possible versions of OpenGL [17]. It is simply not possible, in the io7m-r1 package, for the programmer to create a material that will work on some of the supported OpenGL versions and not others.
  • The material system has predictable performance characteristics. In systems that require programmers to write their own shaders, it is standard practice for programmers to repeatedly re-visit and do tedious optimization work to get their materials to run faster. The io7m-r1 package tries to expose a very minimalist material system and delegates the work of, for example, procedural texture generation to external systems [18].
The io7m-r1 package is generally developed under the assumption that if a programmer is competent enough to write all of their own shading and lighting algorithms, then they might as well have written their own rendering system in the first place [19]!
Types
All materials implement KMaterialType.
All materials have a 3x3 texture matrix which is concatenated with a per-instance 3x3 texture matrix during rendering. The texture matrix affects all textures (such as the albedo map, normal map, specular map, etc) in the material simultaneously. This allows for the orientation, scale and position of a texture to be set on a per-material basis, and then adjusted on a per-instance basis later.
All materials have a unique material code (a simple string) that allows the rendering system to select a shader in O(1) time to render the material.
All materials take the form pseudo-algebraic data types, with each case (or constructor in typed functional languages - not used here because the term already exists in Java and means something else) being represented by a single type. For example, the type of albedo material properties is represented by the KMaterialAlbedoType type, with specific cases represented by the KMaterialAlbedoTextured and KMaterialAlbedoUntextured types. When given a value of type KMaterialAlbedoType, it is necessary to pattern-match on the value to find out which of the specific types the value actually is. This is achieved in Java by the use of generic visitors. As an example:
/*
 * Copyright © 2014 <code@io7m.com> http://io7m.com
 *
 * Permission to use, copy, modify, and/or distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
 * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
 * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
 * SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
 * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
 * IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
 */

package com.io7m.r1.documentation.examples;

import com.io7m.jfunctional.Unit;
import com.io7m.junreachable.UnreachableCodeException;
import com.io7m.r1.kernel.types.KMaterialAlbedoTextured;
import com.io7m.r1.kernel.types.KMaterialAlbedoType;
import com.io7m.r1.kernel.types.KMaterialAlbedoUntextured;
import com.io7m.r1.kernel.types.KMaterialAlbedoVisitorType;
import com.io7m.r1.types.RException;

/**
 * An example of pattern matching on materials.
 */

public final class Match0
{
  private Match0()
  {
    throw new UnreachableCodeException();
  }

  public static void whichAlbedoType(
    final KMaterialAlbedoType m)
    throws RException
  {
    m
      .albedoAccept(new KMaterialAlbedoVisitorType<Unit, UnreachableCodeException>() {
        @Override public Unit textured(
          final KMaterialAlbedoTextured mt)
        {
          System.out.println("Textured");
          return Unit.unit();
        }

        @Override public Unit untextured(
          final KMaterialAlbedoUntextured mu)
        {
          System.out.println("Untextured");
          return Unit.unit();
        }
      });
  }
}
Essentially, the method albedoAccept is passed a value v of type KMaterialAlbedoVisitorType. The textured method of v is called if m is of type KMaterialAlbedoTextured, and the untextured method of v is called if m is of type KMaterialAlbedoUntextured. The methods of v can return values of type A (in this case, because the visitor simply prints a message and doesn't return anything, the visitor returns a value of type Unit), and can raise exceptions of type E (in this case, the visitor does not raise any exceptions and so throws an unchecked UnreachableCodeException).
This method of representing types with a small number of fixed cases is used throughout the io7m-r1 package [20].
Materials are divided into opaque and translucent types, represented by the KMaterialOpaqueType and KMaterialTranslucentType types. Specific cases of the translucent types allow for effects such as specular-only rendering (KMaterialTranslucentSpecularOnly) and generic refraction (KMaterialTranslucentRefractive).
Shaders
In order to render an object on modern graphics hardware, it's necessary to use one or more shaders (shading programs) to actually render surface details and/or apply lighting.
When performing forward rendering, one shader is used per combination of surface material type and light type. When performing deferred rendering, one shader is used to render the surface properties into a buffer, and another shader is used to calculate lighting using the data in the buffer.
It's therefore necessary to be able to, given an arbitrary material and light, efficiently choose shaders with which to perform rendering. Because all possible material and light combinations in the io7m-r1 package are known ahead of time, the io7m-r1 package simply picks one from the complete set of possible shaders that were generated when the package was built. When an object with a given material m is placed into a scene, the io7m-r1 package reads the material code (a simple unique identifier string) for the material and uses that code to select a shader. The shader is then compiled, loaded, and cached. The next time a material (or material and light combination) appears with the same code, the cached shader is re-used.
The advantage of this approach, somewhat ironically, is a reduction in the number of shaders that have to be loaded at any given time. In other rendering systems that equate materials with shaders, a scene containing a thousand different material types requires a thousand different shaders to be loaded (even though many of those shaders may be functionally identical). The same scene using the material system in the io7m-r1 package may only require dozens of shaders, because a shader essentially represents a combination of material properties (such as "normal mapped, environment mapped, textured albedo"), as opposed to representing an entire material. This also reduces the number of rendering state changes that have to be performed during rendering: If two materials differ structurally only in the actual textures that they use, then the same shader will be used for both and therefore the rendering system does not need to switch shaders during rendering to render objects using either material.
Regular Materials
Overview
The io7m-r1 package uses a loose categorization for certain materials, based on observations taken whilst studying the range of surface types used in the average commercial 3D game circa 2014. Basically, a material is regular if it allows for control over the surface albedo, emission, normal, specular, and environment properties. This combination of five properties is sufficient for expressing the vast majority of surface materials implemented in most computer games. The type of regular materials is KMaterialRegularType. The type of opaque regular materials is KMaterialOpaqueRegular. The type of translucent regular materials is KMaterialTranslucentRegular.
Albedo Properties
The albedo of a surface is defined as the coefficient of diffuse reflectivity of a surface. In practical terms, it can be thought of as the color that a given surface will appear when lit with a pure white light, prior to the application of any other surface-modifying effects such as environment mapping. As an example, an object with a pure red albedo will appear to be red when lit with a white light; conceptually, all other parts of the light are absorbed by the surface.
In the io7m-r1 package, the albedo for a surface can be defined as a simple base color (KMaterialAlbedoUntextured), or as a configurable mix (linear interpolation) between a base color and a texture (KMaterialAlbedoTextured).
An example square lit by a white light, with an untextured albedo with a red base color and no other surface properties:
The same square and light, but with a textured albedo mixing in 100% of the texture, effectively replacing the base color:
The same sphere and light with a textured albedo mixing in 100% of the texture, but using a texture that has an alpha channel so that transparent parts of the texture can show the underlying base color:
The same sphere and light with a textured albedo mixing in 50% of the texture (indicating a simple linear interpolation between the texture and base color):
Specifically, the albedo of a given surface is given by albedo [Albedo.hs]:
module Albedo where

import qualified Color4
import qualified Vector4f

albedo :: Color4.T -> Float -> Color4.T -> Color4.T
albedo base mix t =
  Vector4f.interpolate base ((Vector4f.w t) * mix) t
Emission Properties
The emission properties of a surface specify to what degree the surface appears to emit light. The color of the light is equal to the final surface color. Emission is implemented as a filter and therefore emissive surfaces do not represent true light sources. Additionally, emission is not supported for translucent objects.
Normal Properties
The normal properties of a surface define whether the surface normal vectors will be taken directly from the vertex data of the associated mesh (KMaterialNormalVertex), or sampled from a texture to perform normal mapping (KMaterialNormalMapped).
Specular Properties
The specular properties of a surface define how that surface reflects the specular term of any given light source. Surfaces can absorb the specular term and not reflect it at all (KMaterialSpecularNone), or can multiply the incoming light with a constant color (KMaterialSpecularConstant), or can multiply the incoming light with a color sampled from a provided texture (KMaterialSpecularMapped). Each of the constant or mapped options also require the specification of a specular exponent.
The purpose of the specular color coefficients serve two purposes: A pure black coefficient means that the surface does not reflect any specular light (simulating a rough surface). A pure white coefficient means that the surface reflects specular light exactly as received (simulating a shiny surface). When these values are sampled from a texture, it becomes possible to control the "shininess" of a texture on a per-pixel basis. A good example would be that of a bathroom tile texture: The faces of the tiles themselves should be shiny, but the cement between the tiles should appear to be rough. A simple greyscale specular map achieves this effect:
Note how the black pixels in the specular map effectively prevent the areas between the tiles from receiving specular highlights, whereas the faces of tiles receive full highlights.
Using a specular map that contains colors other than shades of grey is often useful for modelling metals. Materials such as plastic and paint consist of layers, the topmost of which will ordinarily reflect light as it is received. This usually means that an orange plastic will reflect a white specular highlight when lit with a pure white light. Metals, however, often have specular reflections similar in color to their diffuse reflection; An orange or gold metal will have orange/gold specular highlights when lit with a pure white light. Specular maps containing bizarre combinations of colors can be used to create materials that are supernatural and alien in appearance:
Environment Properties
The environment properties of a surface specify a cube map and some properties controlling how the resulting environment-mapped reflection is combined with the surface color defined by all of the previously specified material parameters.
Depth Materials
Overview
The depth properties of opaque materials indicate how surfaces with those materials will affect the current depth buffer. Essentially, parts of a surface can elect to effectively be discarded by not being written to the depth buffer, allowing parts of otherwise opaque instances to appear fully transparent (without requiring the overhead of using fully translucent instances).
Depth properties are described by the KMaterialDepthType, with KMaterialDepthConstant indicating that an object is fully opaque, and KMaterialDepthAlpha indicating that parts of the object with an albedo that has an opacity of less than a given threshold will appear to be completely translucent.
Depth information is also taken into account during the application of shadow mapping:
Meshes
Overview
A mesh is a collection of vertices and triangles that define a polyhedral object, allocated on the GPU upon which the renderer is executing. In practical terms, a mesh is a pair (a, i), where a is an OpenGL vertex buffer object consisting of vertices of a standard type, i is an OpenGL element buffer object consisting of indices that describe how to draw the mesh as a series of triangles.
The contents of a are mutable, but mesh references are considered to be immutable as with all other objects in the renderer.
Attributes
A mesh consists of vertices. A vertex can be considered to be a value of a record type, with the fields of the record referred to as the attributes of the vertex. In the io7m-r1 package, an array buffer containing vertex data is specified using the array buffer types from jcanephora. The jcanephora package allows programmers to specify the exact types of array buffers, allows for the full inspection of type information at runtime, including the ability to reference attributes by name, and allows for type-safe modification of the contents of array buffers using an efficient cursor interface. In the io7m-r1 package, meshes must have at least the following attributes, in any order:
NameTypeDescription
v_positionvector_3fThe object-space position of the vertex
v_normalvector_3fThe object-space normal vector of the vertex
v_uvvector_2fThe UV coordinates of the vertex
v_tangentvector_4fThe tangent vector of the vertex
The io7m-r1 package ignores any other attributes that happen to be present [21].
Meshes explicitly store per-vertex tangent vectors. The purpose and format of these vectors is given in the section on normal mapping.
Types
Meshes are represented in the io7m-r1 package by the KMesh type.
Transforms
Overview
The ultimate purpose of a transform is to produce one or more matrices that can be combined with other matrices and then finally passed to a shader. The shader uses these matrices to transform vertices and normal vectors during the rendering of objects.
A transform is effectively responsible for producing a model matrix that transforms positions in object-space to world-space.
In practical terms, a transform is a matrix used to position, scale, and rotate instances in a scene. This is achieved by multiplying the matrix with the object-space positions of all vertices of the mesh that makes up the instance, during rendering. The type of transforms in the io7m-r1 package is the KTransformType which, at the time of writing, is implemented by two specific types that are documented in the following sections.
KTransformMatrix4x4
The KTransformMatrix4x4 type represents a model matrix directly. In other words, the programmer is assumed to be using some other system of producing transforms that directly produces a 4x4 model matrix as a result.
This kind of transform is useful for representing transforms that are the result of many other concatenated transforms. As an example, consider a robot arm with multiple joints: The transform for the tip of the arm is the sum of the rotations of all of the ancestor joints.
KTransformOST
The KTransformOST type represents a transform using a quaternion representing an orientation, a vector representing scaling values on three axes, and a translation vector given in world-space.
This representation is simple and allows for convenient positioning and rotation of objects in the world. Quaternions, scaling vectors, and translations are all easily interpolated, and are therefore convenient for producing simple animations. Assuming that mr is a 4x4 rotation matrix produced from the quaternion, ms is a 4x4 scaling matrix produced from the scaling vector, and mt is a 4x4 translation matrix produced from the translation vector, then the final model matrix mm is given by mm = mt * mr * ms. This has the effect of first scaling the object along the global axes, and then rotating the scaled object, and then moving the scaled and rotated object to the world position given by the translation [22].
Instances
Overview
An instance is a 5-tuple (f, m, k, t, u), where:
  • f is a value of type KFaceSelection, which indicates which faces in the given mesh will be rendered.
  • m is a material.
  • k is a reference to a mesh.
  • t is a transform.
  • u is a texture matrix.
If m is an opaque material, then the instance is said to be opaque. If m is a translucent material, then the instance is said to be translucent. This is actually enforced at the type level: An instance of type KInstanceOpaqueRegular may only be associated with a material of type KMaterialOpaqueRegular, and so on. Opaque instances implement KInstanceOpaqueType, whilst translucent instances implement KInstanceTranslucentType.
The rendering of opaque instances is handled differently from the rendering of translucent instances. The io7m-r1 package uses a deferred renderer to efficiently render large numbers of opaque instances lit by potentially hundreds of light sources. However, the nature of the deferred rendering algorithm makes it impossible for deferred renderers to support translucent objects. Therefore, the io7m-r1 package provides a forward renderer that implements a subset of the features of the deferred renderer for rendering translucent instances. Keeping the different categories of instances as distinct types ensures that the programmer is statically prevented from accidentally passing an instance with an opaque material where one with a translucent material was expected, and vice versa.
Visible Sets
Overview
A visible set is the collection of instances and lights that contribute to the scene from the perspective of a given observer. As stated previously the io7m-r1 package assumes that the programmer is using some sort of spatial data structure to intelligently decide what is and is not visible at any given time: The io7m-r1 package draws exactly what it is told to draw, and does not attempt to work out if a given object is visible or not.
Batching
A visible set is batched in order that the renderer can draw the scene with as few internal state changes as possible. Instances and lights are submitted to a mutable builder which produces an immutable visible set as a result. Opaque instances are associated with lights in light groups, and translucent instances are submitted with groups of lights in draw order. The io7m-r1 package renders translucent instances in the order that they are given to the builder.
Because opaque instances can be drawn in any order due to depth buffering, the io7m-r1 package groups opaque instances by material type in order to come up with a draw order that will result in the fewest internal state changes during rendering. It applies this same grouping methodology to produce sets of shadow casters for producing shadow maps for any shadow-projecting lights in the scene. Visible instances are batched by their material code, so if the materials differ only by, for example, their albedo color, then they will both have the same material code and will be in the same batch during rendering. The same batching logic is applied to shadow casting instances.
Shadow Geometry
Because the system requires the programmer to explicitly and separately state that an opaque instance is visible in the scene, and that an opaque instance is casting a shadow, it becomes possible to effectively specify different shadow geometry for a given instance. As an example, a very complex and high resolution mesh may still have the silhouette of a simple sphere, and therefore the user can separately add the high resolution mesh to a scene as a visible instance, but add a low resolution version of the mesh as an invisible shadow-casting instance with the same transform. As a rather extreme example, assuming a high resolution mesh m0 added to the scene as both a visible instance and a shadow caster:
A low resolution mesh m1 added to the scene as both a visible instance and shadow caster:
Now, with m1 added as only a shadow caster, and m0 added as only a visible instance:
Using lower resolution geometry for shadow casters can lead to efficiency gains on systems where vertex processing is expensive.
Deferred Rendering
Overview
Deferred rendering is a rendering technique where all of the opaque objects in a given scene are rendered into a series of buffers, and then lighting is applied to those buffers in screen-space. This is in contrast to forward rendering, where all lighting is applied to objects as they are rendered.
One major advantage of deferred rendering is a massive reduction in the number of shaders required (traditional forward rendering requires s * l shaders, where s is the number of different object surface types in the scene, and l is the number of different light types). In contrast, deferred rendering requires s + l shaders, because surface and lighting shaders are applied separately.
Traditional forward rendering also suffers severe performance problems as the number of lights in the scene increases, because it is necessary to recompute all of the surface attributes of an object each time a light is applied. In contrast, deferred rendering calculates all surface attributes of all objects once, and then reuses them when lighting is applied.
However, deferred renderers are usually incapable of rendering translucent objects. The deferred renderer in the io7m-r1 package is no exception, and a severely restricted forward renderer is provided to render translucent objects.
Algorithm
An informal description of the deferred rendering algorithm as implemented in the io7m-r1 package is as follows:
  1. Clear the current g-buffer, depth buffer, stencil buffer, and optionally the color buffer. The stencil buffer is cleared to 0 and the depth buffer is cleared to 1.
  2. For each light group g in the scene:
    1. Enable writing to the depth and stencil buffers, and disable stencil testing.
    2. Set all non-zero values in the current stencil buffer to 1. See the section on light group stencils for the meaning behind these values.
    3. For each instance o in g:
      1. Render the surface albedo, eye-space normals, specular color, and emission level of o into the g-buffer. Normal mapping is performed during rendering, and if o does not have specular highlights, then a pure black (zero intensity) specular color is written. Effects such as environment mapping are considered to be part of the surface albedo and so are performed in this step. Depth testing is enabled, and a depth function that only results in pixels being drawn if the depth of the current pixel is less than or equal to the current depth buffer value is used. The corresponding stencil buffer value is set to 2.
    4. Disable depth buffer and stencil buffer writing. Keep depth testing enabled and set the depth function to greater than or equal. Enable the stencil test, and configure it such that only pixels with a corresponding value of 2 in the stencil buffer will be affected.
    5. For each light k in g:
      1. Render a light volume representing k. All pixels that are overlapped by k and that satisfy the depth test have lighting applied. Lighting is applied in eye-space, and the original eye-space position of the current surface is reconstructed using a position reconstruction algorithm.
In the io7m-r1 package, deferred renderers have the type KRendererDeferredOpaqueType, and the primary implementation is given by KRendererDeferredOpaque. Deferred renderers are usually paired with simple forward renderers in order to render any translucent instances in the scene. The type of paired deferred/forward renderers is KRendererDeferredType with the primary implementation given by KRendererDeferred.
G-Buffer
The g-buffer (the abbreviated form of geometry buffer) is the buffer in which the surface attributes of objects are stored prior to having lighting applied to produce a final rendered image.
One of the main implementation issues in any deferred renderer is deciding which surface attributes (such as position, albedo, normals, etc) to store and which to reconstruct. The more attributes that are stored, the less work is required during rendering to reconstruct those values. However, storing more attributes requires a larger g-buffer and more memory bandwidth to actually populate that g-buffer during rendering. The io7m-r1 package leans towards having a more compact g-buffer and doing slightly more reconstruction work during rendering.
The io7m-r1 package explicitly stores the albedo, normals, emission level, and specular color of surfaces. Additionally, the depth buffer is sampled to recover the depth of surfaces. The eye-space positions of surfaces are recovered via an efficient position reconstruction algorithm which uses the current viewing projection and screen-space depth value as input. In order to reduce the amount of storage required, three-dimensional eye-space normal vectors are stored compressed as two 16 half-precision floating point components via a simple mapping. This means that only 32 bits are required to store the vectors, and very little precision is lost. There is support for optionally storing the vectors as two 8 bit components for systems that are memory-starved, with a noticeable loss in the visual quality of specular highlights.
The precise format of the g-buffer when using 16 bit normals is as follows:
The vertical lines indicate byte boundaries. Not including the depth/stencil buffer, the amount of storage required per pixel is 3 * 4 = 12 bytes = 96 bits. When 8 bit normals are used, the layout format is:
The albedo_r, albedo_g, and albedo_b components correspond to the red, green, and blue components of the surface, respectively. The emission component refers to the surface emission level. The normal_x and normal_y components correspond to the two components of the compressed surface normal vector. The specular_r, specular_g, and specular_b components correspond to the red, green, and blue components of the surface specularity. Surfaces that will not receive specular highlights simply have 0 for each component. The specular_e component holds the surface specular exponent divided by 256.
Light Group Stencil
Light groups partition the scene into separate lighting environments. A given light k can be in any number of groups, and will be applied to all instances that are in the same group. To implement this, the io7m-r1 package uses the stencil buffer to control which pixels will receive lighting during rendering of each group. Essentially:
  • A value of 0 in the stencil buffer indicates that the current pixel has never been affected by any light group. This is the initial state of all pixels.
  • A value of 1 in the stencil buffer indicates that the current pixel was previously affected by a light group.
  • A value of 2 in the stencil buffer indicates that the current pixel is in the current light group and will have lighting applied.
As noted by the algorithm given above, pixels belonging to the current light group are marked with a value of 2 in the stencil buffer when all of the surfaces in the light group are rendered into the g-buffer. Only pixels with a corresponding value of 2 in the stencil buffer have lighting applied. This is the step that prevents lights in one group from affecting surfaces in another group. When a light group has completed rendering, all pixels with a non-zero value in the stencil buffer have their stencil values set to 1. When all light groups have been rendered, the stencil buffer will contain a non-zero value for all pixels that were touched during rendering - this fact can then be used in further postprocessing stages if desired.
Normal Compression
The io7m-r1 package uses a Lambert azimuthal equal-area projection to store surface normal vectors in two components instead of three. This makes use of the fact that normalized vectors represent points on the unit sphere. The mapping from normal vectors to two-dimensional spheremap coordinates is given by compress [NormalCompress.hs]:
module NormalCompress where

import qualified Vector3f
import qualified Vector2f
import qualified Normal

compress :: Normal.T -> Vector2f.T
compress n =
  let p = sqrt ((Vector3f.z n * 8.0) + 8.0)
      x = (Vector3f.x n / p) + 0.5
      y = (Vector3f.y n / p) + 0.5
  in Vector2f.V2 x y
The mapping from two-dimensional spheremap coordinates to normal vectors is given by decompress [NormalDecompress.hs]:
module NormalDecompress where

import qualified Vector3f
import qualified Vector2f
import qualified Normal

decompress :: Vector2f.T -> Normal.T
decompress v =
  let fn = Vector2f.V2 ((Vector2f.x v * 4.0) - 2.0) ((Vector2f.y v * 4.0) - 2.0)
      f  = Vector2f.dot2 fn fn
      g  = sqrt (1.0 - (f / 4.0))
      x  = (Vector2f.x fn) * g
      y  = (Vector2f.y fn) * g
      z  = 1.0 - (f / 2.0)
  in Vector3f.V3 x y z
Light Volumes
In order to apply lighting during deferred rendering, it is necessary to render light volumes representing the shape and size of the current light. All pixels that fall within this light volume have lighting applied. Specifically:
  • Spherical lights with radius r are represented by unit spheres with a transform that scales them during rendering to spheres with a resulting radius of r.
  • Directional lights are represented by full-screen quads.
  • Projective lights are represented by frustums that are created and cached on demand to match the size and shape of the light's projection.
Deferred Rendering: Position Reconstruction
Overview
Applying lighting during deferred rendering is primarily a screen space technique. When the visible set has been rendered into the g-buffer, the original eye space positions of all of the surfaces that resulted in visible fragments in the scene are lost (unless explicitly saved into the g-buffer). However, given the knowledge of the projection that was used to render the visible set (such as perspective or orthographic), it's possible to reconstruct the original eye-space position of the surfaces that produced each of the fragments in the g-buffer.
Specifically then, for each fragment f in the g-buffer for which lighting is being applied, a position reconstruction algorithm attempts to reconstruct surface_eye - the eye-space position of the surface that produced f using the screen-space position of the current light volume fragment position = (screen_x, screen_y) and some form of depth value (such as the screen-space depth of f).
Position reconstruction is a fundamental technique in deferred rendering, and there are a practically unlimited number of ways to reconstruct eye-space positions for fragments, each with various advantages and disadvantages. Some rendering systems actually store the eye-space position of each fragment in the g-buffer, meaning that reconstructing positions means simply reading a value directly from a texture. Some systems store only a normalized eye-space depth value in a separate texture: The first step of most position reconstruction algorithms is to compute the original eye-space Z value of a fragment, so having this value computed during the population of the g-buffer reduces the work performed later. Storing an entire eye-space position into the g-buffer is obviously the simplest and requires the least reconstruction work later on, but is costly in terms of memory bandwidth: Storing a full eye-space position requires an extra 4 * 4 = 16 bytes of storage per fragment (four 32-bit floating point values). As screen resolutions increase, the costs can be prohibitive. Storing a normalized depth value requires only a single 32-bit floating point value per fragment but even this can be too much on less capable hardware. Some algorithms take advantage of the fact that most projections used to render scenes are perspective projections. Some naive algorithms use the full inverse of the current projection matrix to reconstruct eye-space positions having already calculated clip space positions.
The algorithm that the io7m-r1 package uses for position reconstruction is generalized to handle both orthographic and perspective projections, and uses only the existing logarithmic depth values that were written to the depth buffer during scene rendering. This keeps the g-buffer compact, and memory bandwidth requirements comparatively low. The algorithm works with symmetric and asymmetric viewing frustums, but will only work with near and far planes that are parallel to the screen.
The algorithm works in two steps: Firstly, the original eye-space Z value of the fragment in question is recovered, and then this Z value is used to recover the full eye-space position.
Recovering Eye-space Z
During rendering of arbitrary scenes, vertices specified in object-space are transformed to eye-space, and the eye-space coordinates are transformed to clip-space with a projection matrix. The resulting 4D clip-space coordinates are divided by their own w components, resulting in normalized-device space coordinates. These normalized-device space coordinates are then transformed to screen-space by multiplying by the current viewport transform. The transitions from clip-space to screen-space are handled automatically by the graphics hardware.
The first step required is to recover the original eye-space Z value of f. This involves sampling a depth value from the current depth buffer. Sampling from the depth buffer is achieved as with any other texture: A particular texel is addressed by using coordinates in the range [(0, 0), (1, 1)]. The io7m-r1 package currently assumes that the size of the viewport is the same as that of the framebuffer (width, height) and that the bottom left corner of the viewport is positioned at (0, 0) in screen space. Given the assumption on the position and size of the viewport, and assuming that the screen-space position of the current light volume fragment being shaded is position = (screen_x, screen_y), the texture coordinates (screen_uv_x, screen_uv_y) used to access the current depth value are given by:
module ScreenToTexture where

import qualified Vector2f

screen_to_texture :: Vector2f.T -> Float -> Float -> Vector2f.T
screen_to_texture position width height =
  let u = (Vector2f.x position) / width
      v = (Vector2f.y position) / height
  in Vector2f.V2 u v
Intuitively, (screen_uv_x, screen_uv_y) = (0, 0) when the current screen-space position is the bottom-left corner of the screen, (screen_uv_x, screen_uv_y) = (1, 1) when the current screen-space position is the top-right corner of the screen, and (screen_uv_x, screen_uv_y) = (0.5, 0.5) when the current screen-space position is the exact center of the screen.
Originally, the io7m-r1 package used a standard depth buffer and so recovering the eye-space Z value required a slightly different method compared to the steps required for the logarithmic depth encoding that the package now uses. For historical reasons and for completeness, the method to reconstruct an eye-space Z value from a traditional screen-space depth value is given in the section on screen-space depth encoding.
Recovering Eye-space Z (Logarithmic depth encoding)
The io7m-r1 package now uses a logarithmic depth buffer. Depth values sampled from any depth buffer produced by the package can be transformed to a negated eye-space Z value by with a simple decoding equation.
Recovering Eye-space Z (Screen-space depth encoding)
Note: This section is for completeness and historical interest. Please skip ahead to the section on eye-space position reconstruction if you are not interested.
Assuming a screen-space depth value screen_depth sampled from the depth buffer at (screen_uv_x, screen_uv_y), it's now necessary to transform the depth value back into normalized-device space. In OpenGL, screen-space depth values are in the range [0, 1] by default, with 0 representing the near plane and 1 representing the far plane. However, in OpenGL, normalized-device space coordinates are in the range [(-1, -1, -1), (1, 1, 1)]. The transformation from screen-space to normalized-device space is given by:
module ScreenDepthToNDC where

screen_depth_to_ndc :: Float -> Float
screen_depth_to_ndc screen_depth =
  (screen_depth * 2.0) - 1.0
In order to understand how to calculate the eye-space depth value from the resulting NDC Z value ndc_z = screen_depth_to_ndc screen_depth, it's necessary to understand how the normalized-device coordinates of f were derived in the first place. Given a standard 4x4 projection matrix m and an eye-space position eye, clip-space coordinates are calculated by Matrix4x4f.mult_v m eye. This means that the z component of the resulting clip-space coordinates is given by:
module ClipSpaceZLong where

import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;

clip_z_long :: M4x4.T -> V4.T -> Float
clip_z_long m eye =
  let
    m20 = M4x4.row_column m (2, 0)
    m21 = M4x4.row_column m (2, 1)
    m22 = M4x4.row_column m (2, 2)
    m23 = M4x4.row_column m (2, 3)

    k0 = (V4.x eye) * m20
    k1 = (V4.y eye) * m21
    k2 = (V4.z eye) * m22
    k3 = (V4.w eye) * m23
  in
    k0 + k1 + k2 + k3
Similarly, the w component of the resulting clip-space coordinates is given by:
module ClipSpaceWLong where

import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;

clip_w_long :: M4x4.T -> V4.T -> Float
clip_w_long m eye =
  let
    m30 = M4x4.row_column m (3, 0)
    m31 = M4x4.row_column m (3, 1)
    m32 = M4x4.row_column m (3, 2)
    m33 = M4x4.row_column m (3, 3)

    k0 = (V4.x eye) * m30
    k1 = (V4.y eye) * m31
    k2 = (V4.z eye) * m32
    k3 = (V4.w eye) * m33
  in
    k0 + k1 + k2 + k3
However, in the perspective and orthographic projections provided by the io7m-r1 package, Matrix4x4f.row_column m (2, 0) == 0, Matrix4x4f.row_column m (2, 1) == 0, Matrix4x4f.row_column m (3, 0) == 0, and Matrix4x4f.row_column m (3, 1) == 0. Additionally, the w component of all eye-space coordinates is 1. With these assumptions, the previous definitions simplify to:
module ClipSpaceZSimple where

import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;

clip_z_simple :: M4x4.T -> V4.T -> Float
clip_z_simple m eye =
  let
    m22 = M4x4.row_column m (2, 2)
    m23 = M4x4.row_column m (2, 3)
  in
    ((V4.z eye) * m22) + m23
module ClipSpaceWSimple where

import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;

clip_w_simple :: M4x4.T -> V4.T -> Float
clip_w_simple m eye =
  let
    m32 = M4x4.row_column m (3, 2)
    m33 = M4x4.row_column m (3, 3)
  in
    ((V4.z eye) * m32) + m33
It should be noted that for perspective matrices in the io7m-r1 package, Matrix4x4f.row_column m (3, 2) == -1 and Matrix4x4f.row_column m (3, 3) == 0:
This means that the w component of the resulting clip-space coordinates is equal to the negated (and therefore positive) eye-space z of the original coordinates.
For orthographic projections in the io7m-r1 package, Matrix4x4f.row_column m (3, 2) == 0 and Matrix4x4f.row_column m (3, 3) == 1:
This means that the w component of the resulting clip-space coordinates is always equal to 1.
As stated previously, normalized-device space coordinates are calculated by dividing a set of clip-space coordinates by their own w component. So, given clip_z = ClipSpaceZSimple.clip_z_simple m eye and clip_w = ClipSpaceWSimple.clip_w_simple m eye for some arbitrary projection matrix m and eye-space position eye, the normalized-device space Z coordinate is given by ndc_z = clip_z / clip_w. Rearranging the definitions of clip_z and clip_w algebraically yields an equation that takes an arbitrary projection matrix m and a normalized-device space Z value ndc_z and returns an eye-space Z value:
module EyeSpaceZ where

import qualified Matrix4f as M4x4;

eye_z :: M4x4.T -> Float -> Float
eye_z m ndc_z =
  let
    m22 = M4x4.row_column m (2, 2)
    m23 = M4x4.row_column m (2, 3)
    m32 = M4x4.row_column m (3, 2)
    m33 = M4x4.row_column m (3, 3)
    
    a = (ndc_z * m33) - m32
    b = (ndc_z * m23) - m22
  in
    - (a / b)
Recovering Eye-space Position
Given that the eye-space Z value is known, it's now necessary to reconstruct the full eye-space position surface_eye of the surface that resulted in f.
When the current projection is a perspective projection, there is conceptually a ray passing through the near clipping plane (near) from the origin, oriented towards the eye-space position (eye) of f:
When the current projection is an orthographic projection, the ray is always perpendicular to the clipping planes and is offset by a certain amount (q) on the X and Y axes:
Assuming ray = Vector3f.V3 ray_x ray_y 1.0, the eye-space position of f is given by surface_eye = Vector3f.add3 q (Vector3f.scale ray eye_z). In the case of perspective projections, q = Vector3f.V3 0.0 0.0 0.0. The q term is sometimes referred to as the origin (because q is the origin of the view ray), but that terminology is not used here in order to avoid confusion between the ray origin and the eye-space coordinate system origin. It's therefore necessary to calculate q and ray in order to reconstruct the full eye-space position of the fragment. The way this is achieved in the io7m-r1 package is to calculate q and ray for each of the viewing frustum corners [24] and then bilinearly interpolate between the calculated values during rendering based on screen_uv_x and screen_uv_y.
As stated previously, normalized-device space coordinates are in the range [(-1, -1, -1), (1, 1, 1)]. Stating each of the eight corners of the cube that defines normalized-device space as 4D homogeneous coordinates [25] yields the following values:
module NDCCorners where

import qualified Vector4f as V4

near_x0y0 :: V4.T
near_x0y0 = V4.V4 (-1.0) (-1.0) (-1.0) 1.0

near_x1y0 :: V4.T
near_x1y0 = V4.V4 1.0 (-1.0) (-1.0) 1.0

near_x0y1 :: V4.T
near_x0y1 = V4.V4 (-1.0) 1.0 (-1.0) 1.0

near_x1y1 :: V4.T
near_x1y1 = V4.V4 1.0 1.0 (-1.0) 1.0

far_x0y0 :: V4.T
far_x0y0 = V4.V4 (-1.0) (-1.0) 1.0 1.0

far_x1y0 :: V4.T
far_x1y0 = V4.V4 1.0 (-1.0) 1.0 1.0

far_x0y1 :: V4.T
far_x0y1 = V4.V4 (-1.0) 1.0 1.0 1.0

far_x1y1 :: V4.T
far_x1y1 = V4.V4 1.0 1.0 1.0 1.0

Then, for the four pairs of near/far corners ((near_x0y0, far_x0y0), (near_x1y0, far_x1y0), (near_x0y1, far_x0y1), (near_x1y1, far_x1y1)), a q and ray value is calculated. The ray_and_q function describes the calculation for a given pair of near/far corners:
module RayAndQ where

import qualified Matrix4f as M4x4
import qualified Vector4f as V4

-- | Calculate @(ray, q)@ for the given inverse projection matrix and frustum corners
ray_and_q :: M4x4.T -> (V4.T, V4.T) -> (V4.T, V4.T)
ray_and_q inverse_m (near, far) =
  let
    -- Unproject the NDC coordinates to eye-space
    near_hom    = M4x4.mult_v inverse_m near
    near_eye    = V4.div_s near_hom (V4.w near_hom)
    far_hom     = M4x4.mult_v inverse_m far
    far_eye     = V4.div_s far_hom (V4.w far_hom)
    
    -- Calculate a ray with ray.z == 1.0
    ray_initial = V4.sub4 far_eye near_eye
    ray = V4.div_s ray_initial (V4.z ray_initial)
    
    -- Subtract the scaled ray from the near corner to calculate q
    q = V4.sub4 near_eye (V4.scale ray (V4.z near_eye))
  in
    (ray, q)
The function takes a matrix representing the inverse of the current projection matrix, and "unprojects" the given near and far frustum corners from normalized-device space to eye-space. The desired ray value for the pair of corners is simply the vector that results from subtracting the near corner from the far corner, divided by its own z component. The desired q value is the vector that results from subtracting ray scaled by the z component of the near corner, from the near corner.
Note: The function calculates ray in eye-space, but the resulting value will have a non-negative z component. The reason for this is that the resulting ray will be multiplied by the calculated eye-space Z value [26] to produce an eye-space position. If the z component of ray was negative, the resulting position would have a positive z component.
Calculating the ray and q value for each of the pairs of corners is straightforward:
module RayAndQAll where

import qualified NDCCorners
import qualified RayAndQ
import qualified Matrix4f as M4x4
import qualified Vector4f as V4

data T = T {
  q_x0y0 :: V4.T,
  q_x1y0 :: V4.T,
  q_x0y1 :: V4.T,
  q_x1y1 :: V4.T,
  ray_x0y0 :: V4.T,
  ray_x1y0 :: V4.T,
  ray_x0y1 :: V4.T,
  ray_x1y1 :: V4.T
} deriving (Eq, Ord, Show)

-- | Calculate all rays and qs for the four pairs of near/far frustum corners
calculate :: M4x4.T -> T
calculate inverse_m =
  let
    (x0y0_ray, x0y0_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x0y0, NDCCorners.far_x0y0)
    (x1y0_ray, x1y0_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x1y0, NDCCorners.far_x1y0)
    (x0y1_ray, x0y1_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x0y1, NDCCorners.far_x0y1)
    (x1y1_ray, x1y1_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x1y1, NDCCorners.far_x1y1)
  in
    T {
      q_x0y0 = x0y0_q,
      q_x1y0 = x1y0_q,
      q_x0y1 = x0y1_q,
      q_x1y1 = x1y1_q,
      ray_x0y0 = x0y0_ray,
      ray_x1y0 = x1y0_ray,
      ray_x0y1 = x0y1_ray,
      ray_x1y1 = x1y1_ray
    }
Then, by reusing the position = (screen_uv_x, screen_uv_y) values calculated during the initial eye-space Z calculation, determining ray and q for the current fragment involves simply bilinearly interpolating between the precalculated values above. Bilinear interpolation between four vectors is defined as:
module Bilinear4 where

import qualified Vector2f as V2
import qualified Vector4f as V4

interpolate :: (V4.T, V4.T, V4.T, V4.T) -> V2.T -> V4.T
interpolate (x0y0, x1y0, x0y1, x1y1) position =
  let u0 = V4.interpolate x0y0 (V2.x position) x1y0
      u1 = V4.interpolate x0y1 (V2.x position) x1y1
  in V4.interpolate u0 (V2.y position) u1
Finally, now that all of the required components are known, the eye-space position surface_eye of f is calculated as surface_eye = Vector3f.add3 q (Vector3f.scale ray eye_z).
Implementation
In the io7m-r1 package, the KViewRays class precalculates the rays and q values for each of the current frustum corners, and the results of which are cached and re-used based on the current projection each time the scene is rendered.
The actual position reconstruction is performed in a fragment shader, producing an eye-space Z value using the Parasol functions in [LogDepth.p] and the final position in [Reconstruction.p]:
--
-- Copyright © 2014 <code@io7m.com> http://io7m.com
-- 
-- Permission to use, copy, modify, and/or distribute this software for any
-- purpose with or without fee is hereby granted, provided that the above
-- copyright notice and this permission notice appear in all copies.
-- 
-- THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
-- WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
-- MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-- SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
-- WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
-- ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
-- IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
--

package com.io7m.r1.core;

--
-- Functions for handling logarithmic depth buffers.
--

module LogDepth is

  import com.io7m.parasol.Float as F;

  function prepare_eye_z (z : float) : float =
    F.add (F.negate (z), 1.0);

  function encode_partial (
    z                 : float,
    depth_coefficient : float  
  ) : float =
    let 
      value half_co = F.multiply (depth_coefficient, 0.5);
      value clamp_z = F.maximum (0.000001, z);
    in
      F.multiply (F.log2 (clamp_z), half_co)
    end;

  function encode_full (
    z                 : float,
    depth_coefficient : float  
  ) : float =
    let 
      value half_co = F.multiply (depth_coefficient, 0.5);
      value clamp_z = F.maximum (0.000001, F.add (z, 1.0));
    in
      F.multiply (F.log2 (clamp_z), half_co)
    end;

  function decode (
    z                 : float,
    depth_coefficient : float  
  ) : float =
    let value half_co = F.multiply (depth_coefficient, 0.5); in
      F.subtract (F.power (2.0, F.divide (z, half_co)), 1.0)
    end;

end;
--
-- Copyright © 2014 <code@io7m.com> http://io7m.com
--
-- Permission to use, copy, modify, and/or distribute this software for any
-- purpose with or without fee is hereby granted, provided that the above
-- copyright notice and this permission notice appear in all copies.
--
-- THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
-- WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
-- MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-- SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
-- WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
-- ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
-- IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
--

package com.io7m.r1.core;

--
-- Position reconstruction for deferred rendering.
--

module Reconstruction is

  import com.io7m.parasol.Float as F;
  import com.io7m.parasol.Vector2f as V2;
  import com.io7m.parasol.Vector3f as V3;
  import com.io7m.parasol.Vector4f as V4;

  import com.io7m.r1.core.Bilinear;
  import com.io7m.r1.core.Transform;
  import com.io7m.r1.core.Viewport;
  import com.io7m.r1.core.ViewRays;

  function reconstruct_eye (
    screen_depth : float,
    screen_uv    : vector_2f,
    m_projection : matrix_4x4f,
    view_rays    : ViewRays.t
  ) : vector_4f =
    let
      value eye_depth =
        Transform.ndc_to_eye_z (
          m_projection,
          Transform.screen_depth_to_ndc (screen_depth)
        );
    in
      reconstruct_eye_with_eye_z (
        eye_depth, 
        screen_uv, 
        m_projection, 
        view_rays
      )
    end;

  function reconstruct_eye_with_eye_z (
    eye_depth    : float,
    screen_uv    : vector_2f,
    m_projection : matrix_4x4f,
    view_rays    : ViewRays.t
  ) : vector_4f =
    let
      value origin =
        Bilinear.interpolate_3f (
          view_rays.origin_x0y0,
          view_rays.origin_x1y0,
          view_rays.origin_x0y1,
          view_rays.origin_x1y1,
          screen_uv
        );

      value ray_normal =
        Bilinear.interpolate_3f (
          view_rays.ray_x0y0,
          view_rays.ray_x1y0,
          view_rays.ray_x0y1,
          view_rays.ray_x1y1,
          screen_uv
        );
        
      value ray =
        V3.multiply_scalar (
          ray_normal,
          eye_depth
        );
    in
      new vector_4f (V3.add (origin, ray), 1.0)
    end;

end;
The precalculated view ray vectors are passed to the fragment shader in a value of type ViewRays.t:
--
-- Copyright © 2014 <code@io7m.com> http://io7m.com
--
-- Permission to use, copy, modify, and/or distribute this software for any
-- purpose with or without fee is hereby granted, provided that the above
-- copyright notice and this permission notice appear in all copies.
--
-- THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
-- WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
-- MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-- SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
-- WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
-- ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
-- IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
--

package com.io7m.r1.core;

module ViewRays is

  type t is record
    origin_x0y0 : vector_3f,
    origin_x1y0 : vector_3f,
    origin_x0y1 : vector_3f,
    origin_x1y1 : vector_3f,
    ray_x0y0    : vector_3f,
    ray_x1y0    : vector_3f,
    ray_x0y1    : vector_3f,
    ray_x1y1    : vector_3f
  end;

end;
Logarithmic Depth
Overview
The io7m-r1 package exclusively utilizes a so-called logarithmic depth buffer for all rendering operations.
OpenGL Depth Issues
By default, OpenGL (effectively) stores a depth value proportional to the reciprocal of the z component of the clip-space coordinates of each vertex projected onto the screen [27]. Informally, the perspective projection matrix used to transform eye-space coordinates to clip-space will place the negated z component of the original eye-space coordinates into the w component of the resulting clip-space coordinates. When the hardware performs the division by w to produce normalized-device-space coordinates, the resulting z component falls within the range [-1.0, 1.0] (although any point with a z component less than 0 will be clipped away by the clipping hardware). This final value is linearly mapped to a configurable range (typically [0.0, 1.0]) to produce a screen-space depth value.
Unfortunately, the encoding scheme above means that most of the depth buffer is essentially wasted. The above scheme will give excessive precision for objects close to the viewing plane, and almost none for objects further away. Fortunately, a better encoding scheme known as logarithmic depth [28] can be implemented that provides vastly greater precision and coexists happily with the standard projection matrices used in OpenGL-based renderers.
Logarithmic Encoding
A logarithmic depth value is produced by encoding a negated (and therefore positive) eye-space z value in the manner specified by encode [LogDepth.hs]:
module LogDepth where

newtype LogDepth =
  LogDepth Float
    deriving (Eq, Ord, Show)

type Depth = Float

log2 :: Float -> Float
log2 = logBase 2.0

depth_coefficient :: Float -> Float
depth_coefficient far = 2.0 / log2 (far + 1.0)

encode :: Float -> Depth -> LogDepth
encode depth_co depth =
  let hco = depth_co * 0.5 in
    LogDepth $ log2 (depth + 1.0) * hco

decode :: Float -> LogDepth -> Depth
decode depth_co (LogDepth depth) =
  let hco = depth_co * 0.5 in
    (2.0 ** (depth / hco)) - 1
The function is parameterized by a so-called depth coefficient that is derived from the far plane distance as shown by depth_coefficient.
The inverse of encode is decode, such that for a given negated eye-space z, z = decode d (encode d z).
A graph of the functions is as follows:
An interactive GeoGebra construction is provided in [log_depth.ggb]
The io7m-r1 package uses a slightly modified version of the encoding function that clamps the original z value to the range [0.000001, ∞]. The reason for this is that log2 (0) is undefined, and so attempting to derive a depth value in this manner tends to cause issues with triangle clipping. The encoding function is also separated into two parts as a simple optimization: The encoding function contains a term z + 1.0, and this term can be calculated by a vertex shader and interpolated. The actual functions as implemented are given by [LogDepth.p]:
--
-- Copyright © 2014 <code@io7m.com> http://io7m.com
-- 
-- Permission to use, copy, modify, and/or distribute this software for any
-- purpose with or without fee is hereby granted, provided that the above
-- copyright notice and this permission notice appear in all copies.
-- 
-- THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
-- WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
-- MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
-- SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
-- WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
-- ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
-- IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
--

package com.io7m.r1.core;

--
-- Functions for handling logarithmic depth buffers.
--

module LogDepth is

  import com.io7m.parasol.Float as F;

  function prepare_eye_z (z : float) : float =
    F.add (F.negate (z), 1.0);

  function encode_partial (
    z                 : float,
    depth_coefficient : float  
  ) : float =
    let 
      value half_co = F.multiply (depth_coefficient, 0.5);
      value clamp_z = F.maximum (0.000001, z);
    in
      F.multiply (F.log2 (clamp_z), half_co)
    end;

  function encode_full (
    z                 : float,
    depth_coefficient : float  
  ) : float =
    let 
      value half_co = F.multiply (depth_coefficient, 0.5);
      value clamp_z = F.maximum (0.000001, F.add (z, 1.0));
    in
      F.multiply (F.log2 (clamp_z), half_co)
    end;

  function decode (
    z                 : float,
    depth_coefficient : float  
  ) : float =
    let value half_co = F.multiply (depth_coefficient, 0.5); in
      F.subtract (F.power (2.0, F.divide (z, half_co)), 1.0)
    end;

end;
A fragment shader can use encode_full to compute a logarithmic depth value from a given positive eye-space z value. Alternatively, a vertex shader can compute the z + 1.0 term r from a non-negated eye-space z value, and pass r to a cooperating fragment shader which then finishes the computation by applying encode_partial to r. When performing position reconstruction during deferred rendering, the original eye-space z value of a fragment is retrieved by negating the result of decode applied to a given logarithmic depth sample.
The original derivation of the encoding and decoding functions as described by Brano Kemen used the w component of the resulting clip-space coordinates. Unfortunately, this does not work correctly with orthographic projections, as the typical orthographic projection matrix will produce clip-space coordinates with a w component always equal to 1. Aside from the effects that this will have on depth testing (essentially mapping the depth of all fragments to the far plane), it also makes position reconstruction impossible as the original eye-space z value cannot be recovered. Instead, the io7m-r1 package uses the negated eye-space z value directly in all cases.
Forward Rendering (Translucents)
Overview
Because the deferred renderer in the io7m-r1 package is incapable of rendering translucent instances, a separate forward renderer is provided. The forward renderer in the package can only work with a subset of the available light types, in order to prevent the combinatorial explosion of shaders required to support a large number of light and surface types.
Translucent instances are divided into lit and unlit categories. As stated previously, translucent instances are drawn in the order that they are submitted to the visible set builder. A lit instance is submitted to the builder with the set of lights that affect it. Only lights of type KLightTranslucentType can be applied to translucent instances. Rendering of translucent instances typically occurs after the rendering of opaque instances and so rendering works with a depth buffer that has already been populated. Depth testing is enabled to ensure that instances overlap and are overlapped by existing opaque instances, but translucent instances do not have their depths written into the depth buffer, and so care should be taken when rendering translucent instances that intersect with each other [29].
Lit
Lit translucent instances can have regular and specular-only materials. Specular-only materials result in only a specular term being calculated for a given object, which is useful for implementing glass-like objects:
Rendering of lit instances proceeds as follows:
  1. For each light k affecting the instance o:
    1. If k is the first light in the set, set the blending functions to (BlendFunction.BLEND_ONE, BlendFunction.BLEND_ONE_MINUS_SOURCE_ALPHA). That is, the source color will be multiplied by 1 and the destination color will be multiplied by 1 - alpha. This has the effect of setting the overall "opacity" of the object for subsequent light contributions. If k is not the first light in the set, set the blending functions to (BlendFunction.BLEND_ONE, BlendFunction.BLEND_ONE). This is the standard additive blending used to sum light contributions.
    2. Render o, lit by k.
Unlit
Unlit translucent instances can have regular and refractive materials. Refractive materials are implemented using the generic refraction effect.
Normal Mapping
Overview
The io7m-r1 package supports the use of tangent-space normal mapping to allow for per-pixel control over the surface normal of rendered triangles. This allows for meshes to appear to have very complex surface details without requiring those details to actually be rendered as triangles within the scene.
Tangent Space
Conceptually, there is a three-dimensional coordinate system based at each vertex, formed by three orthonormal basis vectors: The vertex normal, tangent and bitangent vectors. The normal vector is a the vector perpendicular to the surface at that vertex. The tangent and bitangent vectors are parallel to the surface, and each vector is obviously perpendicular to the other two vectors. This coordinate space is often referred to as tangent space. The normal vector actually forms the Z axis of the coordinate space, and this fact is central to the process of normal mapping. The coordinate system at each vertex may be left or right-handed depending on the arrangement of UV coordinates at that vertex.
Tangent/Bitangent Generation
Tangent and bitangent vectors can be generated by the modelling programs that artists use to create polygon meshes, but, additionally, the RMeshTangents class can take an arbitrary mesh with only normal vectors and UV coordinates and produce tangent and bitangent vectors. The full description of the algorithm used is given in Mathematics for 3D Game Programming and Computer Graphics, Third Edition [30], and also in an article by the same author. The actual aim of tangent/bitangent generation is to produce a pair of orthogonal vectors that are oriented to the x and y axes of an arbitrary texture. In order to do achieve this, the generated vectors are oriented according to the UV coordinates in the mesh.
In the io7m-r1 package, the bitangent vector is not actually stored in the mesh data, and the tangent vector for any given vertex is instead stored as a four-component vector. The reasons for this are as follows: Because the normal, tangent, and bitangent vectors are known to be orthonormal, it should be possible to reconstruct any one of the three vectors given the other two at run-time. This would eliminate the need to store one of the vectors and would reduce the size of mesh data (including the on-disk size, and the size of mesh data allocated on the GPU) by a significant amount. Given any two orthogonal vectors V0 and V1, a vector orthogonal to both can be calculated by taking the cross product of both, denoted (cross V0 V1). The problem here is that if V0 is assumed to be the original normal vector N, and V1 is assumed to be the original tangent vector T, there is no guarantee that (cross N T) will produce a vector equal to the original bitangent vector B: There are two possible choices of value for B that differ only in the sign of their coordinate values.
As an example, a triangle that will produce T and B vectors that form a right-handed coordinate system with the normal vector N (with UV coordinates indicated at each vertex):
The same triangle will produce vectors that form a left-handed system when generating vectors for another vertex (note that the result of (Vector3f.cross N T) = (Vector3f.negation B)):
However, if the original tangent vector T was augmented with a piece of extra information that indicated whether or not the result of (cross N T) needed to be inverted, then reconstructing B would be trivial. Therefore, the fourth component of the tangent vector T contains 1.0 if (cross N T) = B, and -1.0 if (cross N T) = -B. The bitangent vector can therefore be reconstructed by calculating cross (N, T.xyz) * T.w.
With the three vectors (T, B, N), it's now possible construct a 3x3 matrix that can transform arbitrary vectors in tangent space to object space:
With this matrix, it's now obviously possible to take an arbitrary vector in tangent space and transform it to object space. Then, with the current normal matrix (object → eye), transform the object space vector all the way to eye space in the same manner as ordinary per-vertex object space normal vectors.
Normal Maps
A normal map is an ordinary RGB texture where each texel represents a tangent space normal vector. The x coordinate is stored in the red channel, the y coordinate is stored in the green channel, and the z coordinate is stored in the blue channel. The original coordinate values are assumed to fall within the inclusive range [-1.0, 1.0], and these values are mapped to the range [0.0, 1.0] before being encoded to a specific pixel format.
As an example, the vector (0.0, 0.0, 1.0) is first mapped to (0.5, 0.5, 1.0) and then, assuming an image format with 8-bits of precision per color channel, encoded to (0x7f, 0x7f, 0xff). This results in a pale blue color that is characteristic of tangent space normal maps:
Typically, tangent space normal maps are generated from a simple height maps: Greyscale images where 0.0 denotes the lowest possible height, and 1.0 indicates the highest possible height. There are multiple algorithms that are capable of generating normal vectors from height maps, but the majority of them work from the same basic principle: For a given pixel with value h at location (x, y) in an image, the neighbouring pixel values at (x - 1, y), (x - 1, y - 1), (x + 1, y), (x + 1, y + 1) are compared with h in order to determine the slope between the height values. As an example, the Prewitt (3x3) operator when used from the gimp-normalmap plugin will produce the following map from a given greyscale height map:
It is reasonably easy to infer the general directions of vectors from a visual inspection of a tangent space normal map alone. In the above image, the flat faces of the bricks are mostly pale blue. This is because the tangent space normal for that surface is pointing straight towards the viewer - mostly towards the positive z direction. The right edges of the bricks in the image are tinted with a pinkish hue - this indicates that the normal vectors at that pixel point mostly towards the positive x direction.
Rendering With Normal Maps
As stated, the purpose of a normal map is to give per-pixel control over the surface normal for a given triangle during rendering. The process is as follows:
  1. Calculate the bitangent vector B from the N and T vectors. This step is performed on a per-vertex basis (in the vertex shader).
  2. Construct a 3x3 tangent → object matrix M from the (T, B, N) vectors. This step is performed on a per-fragment basis (in the fragment shader) using the interpolated vectors calculated in the previous step.
  3. Sample a tangent space normal vector P from the current normal map.
  4. Transform the vector P with the matrix M by calculating M * P, resulting in an object space normal vector Q.
  5. Transform the vector Q to eye space, in the same manner that an ordinary per-vertex normal vector would be (using the 3x3 normal matrix).
Effectively, a "replacement" normal vector is sampled from the map, and transformed to object space using the existing (T, B, N) vectors. When the replacement normal vector is used when applying lighting, the effect is dramatic. Given a simple two-polygon square textured with the following albedo texture and normal map:
The square when textured and normal mapped, with three spherical lights:
The same square with the same lights but missing the normal map:
Environment Mapping
Overview
Environment mapping is conceptually the process of constructing an artificial environment around an object in order to provide, for example, effects such as reflective surfaces or refractive objects. In the io7m-r1 package, the artificial environment is represented by cube maps, and the only supported effect is reflection. Effects such as refraction are instead provided via generic refraction, which doesn't use environment mapping.
Cube Maps
A cube map is a texture with six faces. When used for environment mapping, each face represents a 90° image of the environment visible in the direction (in world space) of that face. Cube maps are normally constructed by placing an observer in a scene and then orienting the observer in the direction of each cube face in turn and rendering an image. As an example:
Given the above scene, with the observer placed exactly in the center of the indicated magenta circle and assuming a 90° field of view, the six images visible from that location corresponding to the -x, +x, -y, -z, -z, +z cube faces are:
While sampling from ordinary two-dimensional textures involves looking up texels by their two-dimensional coordinates, sampling from cube maps requires three-dimensional coordinates. The three-dimensional coordinates are interpreted as a direction vector or ray emanating from the center of the cube, and the point of intersection between the ray and the corresponding cube face is used to select a texel from that face. Note that in OpenGL there are issues with coordinate system handedness that the io7m-r1 package corrects.
Reflections
So-called environment-mapped reflections are trivially provided by cube maps. For a given surface with a normal vector n, and given the view direction v (from the observer to the surface), a reflection vector is given by r = Reflection.reflection v n:
module Reflection where

import qualified Vector3f as V3

reflection :: V3.T -> V3.T -> V3.T
reflection v0 v1 = V3.sub3 v0 (V3.scale v1 (2.0 * (V3.dot3 v1 v0)))
The reflection vector r is then used to look up a texel in the current cube map directly. This gives a convincing illusion of reflection that will change as the observer moves relative to the surface. Combining normal mapping and environment mapped reflections gives a striking effect:
Note that in the actual io7m-r1 implementation, the vectors n and v will be in eye-space and therefore so will r. The vector r is transformed back to world space by the inverse of the current view matrix for use with the cube map.
Handedness
For reasons lost to time, cube maps in OpenGL use a left-handed coordinate system in contrast to the usual right-handed coordinate system. Because of this, calculated reflection vectors actually have to be inverted to prevent sampling from the wrong cube face. The io7m-r1 package enforces a consistent right-handed coordinate system everywhere. The direction of each cube face corresponds to the same direction in world space, without exception.
Lighting
Overview
The following sections of documentation attempt to describe the theory and implementation of lighting in the io7m-r1 package. All lighting in the package is dynamic - there is no support for precomputed lighting and all contributions from lights are recalculated every time a scene is rendered. Lighting is configured by adding instances of KLightType to a scene.
Diffuse/Specular Terms
The light applied to a surface by a given light is divided into diffuse and specular terms [31]. The actual light applied to a surface is dependent upon the properties of the surface. Conceptually, the diffuse and specular terms are multiplied by the final color of the surface and summed. In practice, the materials applied to surfaces have control over how light is actually applied to the surface. For example, materials may include a specular map which is used to manipulate the specular term as it is applied to the surface. Additionally, if a light supports attenuation, then the diffuse and specular terms are scaled by the attenuation factor prior to being applied.
The diffuse term is modelled by Lambertian reflectance. Specifically, the amount of diffuse light reflected from a surface is given by diffuse [LightDiffuse.hs]:
module LightDiffuse where

import qualified Color3
import qualified Direction
import qualified Normal
import qualified Spaces
import qualified Vector3f

diffuse ::  Direction.T Spaces.Eye -> Normal.T -> Color3.T -> Float -> Vector3f.T
diffuse stl n light_color light_intensity =
  let 
    factor       = max 0.0 (Vector3f.dot3 stl n)
    light_scaled = Vector3f.scale light_color light_intensity
  in 
    Vector3f.scale light_scaled factor
Where stl is a unit length direction vector from the surface to the light source, n is the surface normal vector, light_color is the light color, and light_intensity is the light intensity. Informally, the algorithm determines how much diffuse light should be reflected from a surface based on how directly that surface points towards the light. When stl == n, Vector3f.dot3 stl n == 1.0, and therefore the light is reflected exactly as received. When stl is perpendicular to n (such that Vector3f.dot3 stl n == 0.0), no light is reflected at all. If the two directions are greater than 90° perpendicular, the dot product is negative, but the algorithm clamps negative values to 0.0 so the effect is the same.
The specular term is modelled by Phong reflection [32]. Specifically, the amount of specular light reflected from a surface is given by specular [LightSpecular.hs]:
module LightSpecular where

import qualified Color3
import qualified Direction
import qualified Normal
import qualified Reflection
import qualified Spaces
import qualified Specular
import qualified Vector3f

specular :: Direction.T Spaces.Eye -> Direction.T Spaces.Eye -> Normal.T -> Color3.T -> Float -> Specular.T -> Vector3f.T
specular stl view n light_color light_intensity (Specular.S surface_spec surface_exponent) =
  let 
    reflection   = Reflection.reflection view n
    factor       = (max 0.0 (Vector3f.dot3 reflection stl)) ** surface_exponent
    light_raw    = Vector3f.scale light_color light_intensity
    light_scaled = Vector3f.scale light_raw factor
  in 
    Vector3f.mult3 light_scaled surface_spec
Where stl is a unit length direction vector from the surface to the light source, view is a unit length direction vector from the observer to the surface, n is the surface normal vector, light_color is the light color, light_intensity is the light intensity, surface_exponent is the specular exponent defined by the surface, and surface_spec is the surface specularity factor.
The specular exponent is a value, ordinarily in the range [0, 255], that controls how sharp the specular highlights appear on the surface. The exponent is a property of the surface, as opposed to being a property of the light. Low specular exponents result in soft and widely dispersed specular highlights (giving the appearance of a rough surface), while high specular exponents result in hard and focused highlights (giving the appearance of a polished surface). As an example, three models lit with progressively lower specular exponents from left to right (128, 32, and 8, respectively):
Diffuse-Only Lights
Some lights have diffuse-only variants. Little explanation is required: The specular term is simply not calculated and only the diffuse term is used.
Attenuation
Attenuation is the property of the influence of a given light on a surface in inverse proportion to the distance from the light to the surface. In other words, for lights that support attenuation, the further a surface is from a light source, the less that surface will appear to be lit by the light. For light types that support attenuation, an attenuation factor is calculated based on a given inverse_maximum_range (where the maximum_range is a light-type specific positive value that represents the maximum possible range of influence for the light), a configurable inverse falloff value, and the current distance between the surface being lit and the light source. The attenuation factor is a value in the range [0.0, 1.0], with 1.0 meaning "no attenuation" and 0.0 meaning "maximum attenuation". The resulting attenuation factor is multiplied by the raw unattenuated light values produced for the light in order to produce the illusion of distance attenuation. Specifically:
module Attenuation where

attenuation_from_inverses :: Float -> Float -> Float -> Float
attenuation_from_inverses inverse_maximum_range inverse_falloff distance =
  max 0.0 (1.0 - (distance * inverse_maximum_range) ** inverse_falloff)

attenuation :: Float -> Float -> Float -> Float
attenuation maximum_range falloff distance =
  attenuation_from_inverses (1.0 / maximum_range) (1.0 / falloff) distance
Given the above definitions, a number of observations can be made.
If falloff == 1, then the attenuation is linear over distance [33]:
If maximum_range == 0, then the inverse range is undefined, and therefore the results of lighting are undefined. The io7m-r1 package handles this case by raising an exception when the light is created.
If falloff == 0, then the inverse falloff is undefined, and therefore the results of lighting are undefined. The io7m-r1 package handles this case by raising an exception when the light is created.
As falloff decreases towards 0.0, then the attenuation curve remains at 1.0 for increasingly higher distance values before falling sharply to 0.0:
As falloff increases away from 0.0, then the attenuation curve decreases more for lower distance values:
Directional Lighting
Overview
A directional light in the io7m-r1 package is a light that emits parallel rays of light in a given eye space direction. It has a color and an intensity, but does not have an origin and therefore is not attenuated over distance. It does not cause objects to cast shadows.
Types
Directional lights are represented in the io7m-r1 package by the following types:
Attenuation
Directional lights do not have origins and cannot therefore be attenuated over distance.
Application
The final light applied to the surface is given by directional [Directional.hs], where sr, sg, sb are the red, green, and blue channels, respectively, of the surface being lit. Note that the surface-to-light vector stl is simply the negation of the light direction.
module Directional where

import qualified Color4
import qualified Direction
import qualified LightDirectional
import qualified LightDiffuse
import qualified LightSpecular
import qualified Normal
import qualified Position3
import qualified Spaces
import qualified Specular
import qualified Vector3f
import qualified Vector4f

directional :: Direction.T Spaces.Eye -> Normal.T -> Position3.T Spaces.Eye -> LightDirectional.T -> Specular.T -> Color4.T -> Vector3f.T
directional view n position light specular (Vector4f.V4 sr sg sb _) =
  let
    stl             = Vector3f.normalize (Vector3f.negation position)
    light_color     = LightDirectional.color light
    light_intensity = LightDirectional.intensity light
    light_d         = LightDiffuse.diffuse stl n light_color light_intensity
    light_s         = LightSpecular.specular stl view n light_color light_intensity specular
    lit_d           = Vector3f.mult3 (Vector3f.V3 sr sg sb) light_d
    lit_s           = Vector3f.add3 lit_d light_s
  in
    lit_s
Spherical Lighting
Overview
A spherical light in the io7m-r1 package is a light that emits rays of light in all directions from a given origin specified in eye space up to a given maximum radius.
The term spherical comes from the fact that the light has a defined radius. Most rendering systems instead use point lights that specify multiple attenuation constants to control how light is attenuated over distance. The problem with this approach is that it requires solving a quadratic equation to determine a minimum bounding sphere that can contain the light. Essentially, the programmer/artist is forced to determine "at which radius does the contribution from this light effectively reach zero?". With spherical lights, the maximum radius is declared up front, and a single falloff value is used to determine the attenuation curve within that radius. This makes spherical lights more intuitive to use: The programmer/artist simply places a sphere within the scene and knows exactly from the radius which objects are lit by it. It also means that bounding light volumes can be trivially constructed from unit spheres by simply scaling those spheres by the light radius, when performing deferred rendering.
Types
Spherical lights are represented in the io7m-r1 package by the following types:
Attenuation
The light supports attenuation using the radius as the maximum range.
Application
The final light applied to the surface is given by spherical [Spherical.hs], where sr, sg, sb are the red, green, and blue channels, respectively, of the surface being lit. The surface-to-light vector stl is calculated by normalizing the negation of the difference between the the current eye space surface_position and the eye space origin of the light.
module Spherical where

import qualified Attenuation
import qualified Color4
import qualified Direction
import qualified LightDiffuse
import qualified LightSpecular
import qualified LightSpherical
import qualified Normal
import qualified Position3
import qualified Specular
import qualified Spaces
import qualified Vector3f
import qualified Vector4f

spherical :: Direction.T Spaces.Eye -> Normal.T -> Position3.T Spaces.Eye -> LightSpherical.T -> Specular.T -> Color4.T -> Vector3f.T
spherical view n surface_position light specular (Vector4f.V4 sr sg sb _) =
  let
    position_diff   = Position3.sub3 surface_position (LightSpherical.origin light)
    stl             = Vector3f.normalize (Vector3f.negation position_diff)
    distance        = Vector3f.magnitude (position_diff)
    attenuation     = Attenuation.attenuation (LightSpherical.radius light) (LightSpherical.falloff light) distance
    light_color     = LightSpherical.color light
    light_intensity = LightSpherical.intensity light
    light_d         = LightDiffuse.diffuse stl n light_color light_intensity
    light_s         = LightSpecular.specular stl view n light_color light_intensity specular
    light_da        = Vector3f.scale light_d attenuation
    light_sa        = Vector3f.scale light_s attenuation
    lit_d           = Vector3f.mult3 (Vector3f.V3 sr sg sb) light_da
    lit_s           = Vector3f.add3 lit_d light_sa
  in 
    lit_s
Shadows
Spherical lights cannot project shadows. However, the io7m-r1 provides a so-called pseudo-spherical lights implemented with six projective lights, each of which can project shadows.
Projective Lighting
Overview
A projective light in the io7m-r1 package is a light that projects a texture onto the visible set from a given origin specified in eye space up to a given maximum radius. Projective lights are the only types of lights in the io7m-r1 package that are able to project shadows.
Algorithm
At a basic level, a projective light performs the same operations that occur when an ordinary 3D position is projected onto the screen during rendering. During normal rendering, a point p given in world space is transformed to eye space given the current camera's view matrix, and is then transformed to clip space using the current camera's projection matrix. During rendering of a scene lit by a projective light, a given point q in the scene is transformed back to world space given the current camera's inverse view matrix, and is then transformed to eye space from the point of view of the light (subsequently referred to as light-eye space) using the light's view matrix. Finally, q is transformed to clip space from the point of view of the light (subsequently referred to as light-clip space) using the light's projection matrix. It should be noted (in order to indicate that there is nothing unusual about the light's view or projection matrices) that if the camera and light have the same position, orientation, scale, and projection, then the resulting transformed values of q and p are identical. The resulting transformed value of q is mapped from the range [(-1, -1, -1), (1, 1, 1)] to [(0, 0, 0), (1, 1, 1)], and the resulting coordinates are used to retrieve a texel from the 2D texture associated with the light.
Intuitively, an ordinary perspective projection will cause the light to appear to take the shape of a frustum:
There are two issues with the projective lighting algorithm that also have to be solved: back projection and clamping.
Back projection
The algorithm described above will produce a so-called dual or back projection. In other words, the texture will be projected along the view direction of the camera, but will also be projected along the negative view direction [34]. The visual result is that it appears that there are two projective lights in the scene, oriented in opposite directions. As mentioned previously, given the typical projection matrix, the w component of a given clip-space position is the negation of the eye-space z component. Because it is assumed that the observer is looking towards the negative z direction, all positions that are in front of the observer must have positive w components. Therefore, if w is negative, then the position is behind the observer. The standard fix for this problem is to check to see if the w component of the light-clip space coordinate is negative, and simply return a pure black color (indicating no light contribution) rather than sampling from the projected texture.
The io7m-r1 package takes an arguably simpler approach to the problem. Because projective lights are only applied by the deferred renderer, and because the deferred renderer uses accurate light volumes, pixels that fall outside of the light volume are simply not shaded (meaning that back-projection is free to occur, but pixels that would receive light contributions due to it are simply outside of the light volume). The package essentially depends on the rasterization process and depth testing to ensure that no pixels that would have received a back-projection will be shaded during rendering of the light.
Clamping
The algorithm described above takes an arbitrary point in the scene and projects it from the point of view of the light. There is no guarantee that the point actually falls within the light's view frustum (although this is mitigated slightly by the io7m-r1 package's use of light volumes for deferred rendering), and therefore the calculated texture coordinates used to sample from the projected texture are not guaranteed to be in the range [(0, 0), (1, 1)]. In order to get the intended visual effect, the texture used must be set to clamp-to-edge and have black pixels on all of the edges of the texture image, or clamp-to-border with a black border color. Failing to do this can result in strange visual anomalies, as the texture will be unexpectedly repeated or smeared across the area outside of the intersection between the light volume and the receiving surface:
The io7m-r1 package will raise an exception if a non-clamped texture is assigned to a projective light.
Types
Projective lights are represented in the io7m-r1 package by the following types:
Attenuation
The light supports attenuation using the maximum range taken from the projection.
Application
The final light applied to the surface is given by projective [Projective.hs], where sr, sg, sb are the red, green, and blue channels, respectively, of the surface being lit. The surface-to-light vector stl is calculated by normalizing the negation of the difference between the the current eye space surface_position and the eye space origin of the light.
module Projective where

import qualified Attenuation
import qualified Color3
import qualified Color4
import qualified Direction
import qualified LightDiffuse
import qualified LightSpecular
import qualified LightProjective
import qualified Normal
import qualified Position3
import qualified Specular
import qualified Spaces
import qualified Vector3f
import qualified Vector4f

projective :: Direction.T Spaces.Eye -> Normal.T -> Position3.T Spaces.Eye -> LightProjective.T -> Specular.T -> Float -> Color3.T -> Color4.T -> Vector3f.T
projective view n surface_position light specular shadow texture (Vector4f.V4 sr sg sb _) =
  let
    position_diff   = Position3.sub3 surface_position (LightProjective.origin light)
    stl             = Vector3f.normalize (Vector3f.negation position_diff)
    distance        = Vector3f.magnitude (position_diff)
    attenuation_raw = Attenuation.attenuation (LightProjective.radius light) (LightProjective.falloff light) distance
    attenuation     = attenuation_raw * shadow
    light_color     = Vector3f.mult3 (LightProjective.color light) texture
    light_intensity = LightProjective.intensity light
    light_d         = LightDiffuse.diffuse stl n light_color light_intensity
    light_s         = LightSpecular.specular stl view n light_color light_intensity specular
    light_da        = Vector3f.scale light_d attenuation
    light_sa        = Vector3f.scale light_s attenuation
    lit_d           = Vector3f.mult3 (Vector3f.V3 sr sg sb) light_da
    lit_s           = Vector3f.add3 lit_d light_sa
  in 
    lit_s
The given shadow factor is a value in the range [0, 1], where 0 indicates that the lit point is fully in shadow for the current light, and 1 indicates that the lit point is not in shadow. This is calculated for basic and variance shadows and is assumed to be 1 for lights without shadows. As can be seen, a value of 0 has the effect of fully attenuating the light.
The color denoted by texture is assumed to have been sampled from the projected texture. Assuming the eye-space position being shaded p, the matrix to get from eye-space to light-clip space is given by The final light applied to the surface is given by projective_matrix [ProjectiveMatrix.hs]:
module ProjectiveMatrix where

import qualified Matrix4f

projective_matrix :: Matrix4f.T -> Matrix4f.T -> Matrix4f.T -> Matrix4f.T
projective_matrix camera_view light_view light_projection =
  case Matrix4f.inverse camera_view of
    Just cv -> Matrix4f.mult (Matrix4f.mult light_projection light_view) cv
    Nothing -> undefined -- A view matrix is always invertible

Pseudo-Spherical Lights
As mentioned, the io7m-r1 package does not support shadow projection for spherical lights. However, spherical lights can be emulated with six projective lights, each of which can be configured to project shadows.
A pseudo-spherical light is six projective sub-lights, each of which are oriented along the major axis directions with projections that have a 90° field of view. Shadow casters are assigned to each sub-light as with any ordinary projective light.
Shadows
Overview
Because the io7m-r1 package implements local illumination, it is necessary to associate shadows with those light sources capable of projecting them (currently only projective lights). The io7m-r1 package currently supports basic and variance shadow mapping. So-called mapped shadows allow efficient per-pixel shadows to be calculated with varying degrees of visual quality.
Caching
The io7m-r1 package creates new shadow maps on demand during rendering, and the maps are returned to a soft-bounded [35] cache after use. This implies that in a visible set with n shadow-projecting lights, there will be at least n shadow maps allocated and in use at any one time. Shadow maps are requested from the cache based on their map description, and unused maps are discarded from the cache after a configurable time period. The intention is to avoid having to frequently allocate new shadow maps without requiring that all maps be allocated up-front, and without exhausting all available memory on shadow maps.
The functions responsible for creating a new shadow map based on a map description are given in KShadowMapCacheLoader, with the actual map implementation for basic shadow mapping given in KShadowMapBasic.
The actual production of all shadow maps is managed by the KShadowMapRenderer type.
Shadow mapping - Basic
Overview
Basic shadow mapping is a technique that results in simple per-pixel hard-edged shadows. Using the same view and projection matrices used to apply projective lights, a depth-only image of the current scene is rendered, and those stored depth values are compared with those in the rendered scene to determine if a given point is in shadow with respect to the current light.
Algorithm
Prior to actually rendering a visible set, shadow maps are generated for all shadow-projecting lights in the set. A shadow map for basic shadow mapping for a light k is an image of all of the shadow casters associated with k in the visible set, rendered from the point of view of k. Each pixel in the image represents the logarithmic depth of the closest surface at that pixel. For example:
Darker pixels indicate a lower depth value than light pixels, which indicate that the surface was closer to the observer than in the lighter case.
Then, when actually applying lighting during rendering of the scene, a given eye space position p is transformed to light-clip space and mapped to the range [(0, 0, 0), (1, 1, 1)], producing a position pos_light_clip. The same p is also transformed to light-eye space, producing a position pos_light_eye. The pos_light_clip position is used directly in order to sample a value d from the shadow map (as with sampling from a projected texture with projective lighting). The negated z component of pos_light_eye is encoded as a logarithmic depth value using the same depth coefficient as was used when populating the shadow map, producing a value k. Then, k is compared against d. If the k is less than d, then this means that p is closer to the light than whatever surface resulted in d during the population of the shadow map, and therefore p is not in shadow with respect to the light.
Issues
Unfortunately, the basic shadow mapping algorithm is subject to a number of issues related to numerical imprecision, and the io7m-r1 package applies a number of user-adjustable workarounds for the problems. Firstly, the algorithm is prone to a problem known as shadow acne, which generally manifests as strange moiré patterns and dots on surfaces. This is caused by self-shadowing which is caused by, amongst other things, quantization in the storage of depth values:
One widely used workaround to prevent self-shadowing is to bias stored depth values by a user-configurable amount, effectively increasing the likelihood that a given point will not be considered to be in shadow. Unfortunately, adding a bias value that is too large tends to result in a visual effect where shadows appear to become "detached" from their casting objects, because the the bias value is causing the depth test to pass at positions where the shadow touches the caster.
Another workaround to help prevent self-shadowing is to only render the back-faces of geometry into the shadow map. Unfortunately, this can result in very thin geometry failing to cast shadows.
Finally, it is generally beneficial to increase the precision of stored values in the shadow map by using a projection for the light that has the near and far planes as close together as the light will allow. In other words, if the light has a radius of 10 units, then it is beneficial to use a far plane at 10 units, in order to allow for the best distribution of depth values in that range.
The io7m-r1 package implements both back-face-only rendering, and a configurable per-shadow bias value. Unfortunately, neither of these workarounds can ever fully solve the problems, so the package also provides variance shadows which have far fewer artifacts and better visual quality at a slightly higher computational cost.
Types
Basic mapped shadows are represented by the KShadowMappedBasic type, and can be associated with projective lights.
Rendering of depth-only images is handled by the KDepthRendererType type.
Shadow mapping - Variance
Overview
Variance shadow mapping is a technique that can give attractive soft-edged shadows. Using the same view and projection matrices used to apply projective lights, a depth-variance image of the current scene is rendered, and those stored depth distribution values are used to determine the probability that a given point in the scene is in shadow with respect to the current light.
The algorithm implemented in the io7m-r1 package is described in GPU Gems 3, which is a set of improvements to the original variance shadow mapping algorithm by William Donnelly and Andrew Lauritzen. The io7m-r1 package implements all of the improvements to the algorithm except summed area tables. The package also provides optional box blurring of shadows as described in the chapter.
Algorithm
Prior to actually rendering a visible set, shadow maps are generated for all shadow-projecting lights in the set. A shadow map for variance shadow mapping, for a light k, is a two-component red/green image of all of the shadow casters associated with k in the visible set. The image is produced by rendering the instances from the point of view of k. The red channel of each pixel in the image represents the logarithmic depth of the closest surface at that pixel, and the green channel represents the depth squared (literally depth * depth). For example:
Then, when actually applying lighting during rendering of the scene, a given eye space position p is transformed to light-clip space and then mapped to the range [(0, 0, 0), (1, 1, 1)] in order to sample the depth and depth squared values (d, ds) from the shadow map (as with sampling from a projected texture with projective lighting).
As stated previously, the intent of variance shadow mapping is to essentially calculate the probability that a given point is in shadow, rather than the binary is/is not of basic shadow mapping. A one-tailed variant of Chebyshev's inequality is used to calculate the upper bound u on the probability that, given (d, ds), a given point with depth t is in shadow:
module ShadowVarianceChebyshev0 where

chebyshev :: (Float, Float) -> Float -> Float
chebyshev (d, ds) t =
  let p        = if t <= d then 1.0 else 0.0
      variance = ds - (d * d)
      du       = t - d
      p_max    = variance / (variance + (du * du))
  in max p p_max

factor :: (Float, Float) -> Float -> Float
factor = chebyshev
One of the improvements suggested to the original variance shadow algorithm is to clamp the minimum variance to some small value (the io7m-r1 package uses 0.00002 by default, but this is configurable on a per-shadow basis). The equation above becomes:
module ShadowVarianceChebyshev1 where

data T = T {
  minimum_variance :: Float
} deriving (Eq, Show)

chebyshev :: (Float, Float) -> Float -> Float -> Float
chebyshev (d, ds) min_variance t =
  let p        = if t <= d then 1.0 else 0.0
      variance = max (ds - (d * d)) min_variance
      du       = t - d
      p_max    = variance / (variance + (du * du))
  in max p p_max

factor :: T -> (Float, Float) -> Float -> Float
factor shadow (d, ds) t =
  chebyshev (d, ds) (minimum_variance shadow) t
The above is sufficient to give shadows that are roughly equivalent in visual quality to basic shadow mapping with the added benefit of being generally better behaved and with far fewer artifacts. However, the algorithm can suffer from light bleeding, where the penumbrae of overlapping shadows can be unexpectedly bright despite the fact that the entire area should be in shadow. One of the suggested improvements to reduce light bleeding is to modify the upper bound u such that all values below a configurable threshold are mapped to zero, and values above the threshold are rescaled to map them to the range [0, 1]. The original article suggests a linear step function applied to u:
module ShadowVarianceChebyshev2 where

data T = T {
  minimum_variance :: Float,
  bleed_reduction  :: Float
} deriving (Eq, Show)

chebyshev :: (Float, Float) -> Float -> Float -> Float
chebyshev (d, ds) min_variance t =
  let p        = if t <= d then 1.0 else 0.0
      variance = max (ds - (d * d)) min_variance
      du       = t - d
      p_max    = variance / (variance + (du * du))
  in max p p_max

clamp :: Float -> (Float, Float) -> Float
clamp x (lower, upper) = max (min x upper) lower

linear_step :: Float -> Float -> Float -> Float
linear_step lower upper x = clamp ((x - lower) / (upper - lower)) (0.0, 1.0)

factor :: T -> (Float, Float) -> Float -> Float
factor shadow (d, ds) t =
  let u = chebyshev (d, ds) (minimum_variance shadow) t in
    linear_step (bleed_reduction shadow) 1.0 u
The amount of light bleed reduction is adjustable on a per-shadow basis.
To reduce problems involving numeric inaccuracy, the original article suggests the use of 32-bit floating point textures in depth variance maps. The io7m-r1 package allows 16-bit or 32-bit textures, configurable on a per-shadow basis.
Finally, as mentioned previously, the io7m-r1 package allows both optional box blurring and mipmap generation for shadow maps. Both blurring and mipmapping can reduce aliasing artifacts, with the former also allowing the edges of shadows to be significantly softened as a visual effect:
Types
Variance mapped shadows are represented by the KShadowMappedVariance type, and can be associated with projective lights.
Rendering of depth-variance images is handled by the KDepthVarianceRendererType type.
Generic Refraction
Overview
The io7m-r1 package implements the generic refraction effect described in GPU Gems 2. The technique lends itself to a huge range of effects such as lenses, glass, heat haze, and water - simply by varying the meshes and textures used when performing refraction.
Algorithm
For a given instance with a refractive material applied, the process to render the instance is as follows:
  • Make a temporary copy b of the current scene's color buffer.
  • If masking is enabled for the material, render a mask for the instance into a temporary mask image m.
  • Render the instance, using b as the refraction source, material-dependent refraction vectors, a refraction color, and optionally m for masking.
The actual rendering technique is very simple: Given a screen-space position (x, y), sample the color from a saved image of the scene at (x + s, y + t), where (s, t) are signed per-pixel offset values - the refraction vectors - that are sampled from textures or derived from existing normal vectors.
Masking
Because refractive instances are translucent, they are normally rendered after having already rendered all of the opaque objects in the scene. Because rendering of translucent instances occurs with depth testing enabled, it is therefore possible for opaque instances to occlude refractive instances. This poses a problem for the implementation of refraction described above, because the pixels of an occluding object may be sampled when performing the refraction, as shown in the following image:
Note how the pixels of the opaque instances are bleeding into the refracting object, despite being conceptually "in front of" it. This is because the refraction effect is implemented in screen space and is just sampling pixels from the surrounding area to simulate the bending of light rays. Using a mask prevents this:
A mask is produced by rendering a black and white silhouette of the refracting object, and then using the values of this mask to linearly interpolate between the colors at (x, y) and (x + s, y + t). This has the effect of preventing the refraction simulation from using pixels that fall outside of the mask area.
Vectors
Refraction vectors may either be sampled from the current instance's (possibly mapped) normals, or from the red and green components of a delta texture. The sampled values are scaled by the material's scale factor and used directly to calculate (x + s, y + t). For example, a simple noisy red/green delta texture applied to a quad results in the following effect:
Color
The sampled scene colors used to perform the refraction effect are multiplied by a constant color, specified by each material. This allows for simple colored glass effects (shown here with a specular-only instance rendered over the top of the refractive instance to provide specular highlights):
Using pure RGBA white (1.0, 1.0, 1.0, 1.0) results in a clear glass material:
Filter: Blur
Overview
The io7m-r1 offers a set of simple box blurs for use on both color and depth-variance data, with the effect being used on the latter to soften the edges of shadows.
Algorithm
The implemented algorithm is a simple box blur separated into horizontal and vertical passes. The package also allows for scaling of the image prior to blurring, in order to use bilinear filtering during scaling to accentuate the blur effect. The following image shows a blur of size 1.0 but with the image scaled to 0.5 times its original size, the blur applied, and then the image scaled back up to the original size again with bilinear filtering:
The blur effect for RGBA data is provided by the KImageFilterBlurRGBA filter. The blur effect for depth-variance data is provided by the KImageFilterBlurDepthVariance filter.
Filter: Emission
Overview
An emissive surface is a surface that appears to emit light. The io7m-r1 package offers emission as a visual effect implemented as a filter. An optional glow effect is provided to allow emissive surfaces to appear to have a configurable aura.
The emission effect is obviously not physically accurate - surfaces do not really emit light. The user is expected to make intelligent use of the standard light types to provide lighting, and to use the emission effect to complement them.
Algorithm
The plain emission effect without glow is implemented as trivially as possible by sampling the emission value from a rendered scene's g-buffer, multiplying it by the albedo color and then simply adding the result to the current pixel color.
The emission effect with glow is implemented similarly, except that the albedo * emission term is stored in a separate image, and that image is blurred with a configurable box blur before being additively blended over the original scene. Higher levels of blurring can give the impression of a dusty atmosphere.
The emission effect without glow is provided by the KImageFilterEmission filter. The emission effect with glow is provided by the KImageFilterEmissionGlow filter.
Filter: FXAA
Overview
Fast Approximate Anti-Aliasing is a simple algorithm that attempts to detect and smooth aliasing in a color image. The algorithm works with only the color components of the image in question; no other per-pixel information or knowledge of the scene is required.
Implementation
Unfortunately, information on the FXAA algorithm is sparse, and much of it has been lost to time. The original FXAA algorithm was published in a whitepaper by NVIDIA [36] and was severely optimized by the author on the suggestions of many mostly anonymous contributors. The latest published version of the algorithm (version 3.11) bears little resemblance to the original and no documentation exists on the changes. The 3.11 version of the algorithm is constructed from a maze of C preprocessor macros, and many different variations of the algorithm are possible based on how the parameter macros are defined.
The implementation of FXAA in the io7m-r1 package is a set of GLSL expansions of the public domain [37] Fxaa3_11.h header with a few minor modifications (unused parameter removals). Specifically, the PC algorithm is used, with quality presets (10, 15, 20, 25, 29, 39).
The algorithm is applied via the use of the KImageFilterFXAA filter.
Filter: Z Fog
Overview
The io7m-r1 offers a filter that provides simple distance fog effects for roughly simulating the scattering of light due to atmosphere. It can also be used for special effects such as smoke and dust.
The fog filter applies a configurable fog color to all pixels in the scene, based on given near and far planes for the fog. The fog is applied based on the current view direction (the local negative Z axis).
Algorithm
The implemented algorithm takes a color c, an eye-space near fog distance near, and an eye-space far fog distance far. It samples the depth of each pixel in the current scene, and reconstructs the eye-space Z component of the surface that produced the pixel in question and takes the absolute value of that component to yield a positive distance z from the observer. Then, a fog factor r is calculated by fog_factor [FogFactorZ.hs]:
module FogFactorZ where

clamp :: Float -> (Float, Float) -> Float
clamp x (lower, upper) = max (min x upper) lower

fog_factor :: Float -> (Float, Float) -> Float
fog_factor z (near, far) =
  let r = (z - near) / (far - near) in
    clamp r (0.0, 1.0)
The value of r is then used to interpolate between the original color of the pixel and c. Surfaces with z less than near are unaffected by fog. Surfaces with z greater than or equal to far are maximally affected by fog and will actually have their color values completely replaced by c. Surfaces with z values that fall between near and far will have differing amounts of fog applied based on the type of fog progression selected. The filter offers linear, exponential, and logarithmic fog progressions. With linear fog, the r value linearly interpolates between the original surface color and c. With exponential fog, the value of r * r is used to linearly interpolate between the original surface color and c. Finally, with logarithmic fog, the value of sqrt(r) is used to linearly interpolate between the original surface color and c.
The fog effect is provided by the KImageFilterFogZ filter.
Filter: Y Fog
Overview
The io7m-r1 offers a filter that provides simple vertical fog effects for simulating various atmospheric effects such as ground fog and "house fire" smoke.
The fog filter applies a configurable fog color to all pixels in the scene, based on given upper and lower planes for the fog.
Algorithm
The implemented algorithm takes a color c, a world-space upper Y value upper_y, and a world-space lower Y value lower_y. It samples the depth of each pixel in the current scene, and reconstructs the full eye-space position of the surface that produced the pixel in question. It then transforms this position back to world-space using the scene's inverse view matrix, yielding a Y component y. Then, a fog factor r is calculated by fog_factor [FogFactorY.hs]:
module FogFactorY where

clamp :: Float -> (Float, Float) -> Float
clamp x (lower, upper) = max (min x upper) lower

fog_factor :: Float -> (Float, Float) -> Float
fog_factor y (lower, upper) =
  let r = (y - lower) / (upper - lower) in
    clamp r (0.0, 1.0)
The value of r is then used to interpolate between the original color of the pixel and c. Surfaces with y less than lower_y are unaffected by fog. Surfaces with y greater than or equal to upper_y are maximally affected by fog and will actually have their color values completely replaced by c. Surfaces with y values that fall between lower_y and upper_y will have differing amounts of fog applied based on the type of fog progression selected. The filter offers linear, exponential, and logarithmic fog progressions. With linear fog, the r value linearly interpolates between the original surface color and c. With exponential fog, the value of r * r is used to linearly interpolate between the original surface color and c. Finally, with logarithmic fog, the value of sqrt(r) is used to linearly interpolate between the original surface color and c.
If the upper_y value is greater than the lower_y value, the fog will appear to get thicker as the Y position decreases. This can be used to give the effect of fog on the ground. Unfortunately, the illusion is somewhat ruined if the viewer goes below the fog plane:
If the lower_y value is greater than the upper_y value, the fog will appear to get thicker as the Y position increases. This can be used to give the effect of smoke accumulating at the top of an enclosed space. Unfortunately, the illusion of both types of fog is ruined if the viewer travels above the fog plane:
The simple-minded nature of the effect means that it should be used to supplement some other simulation of fog. Rendering a simple opaque flat plane the same color as the fog and then applying the fog effect would be sufficient.
The fog effect is provided by the KImageFilterFogY filter.
API Reference
Javadoc
API documentation for the package is provided via the included Javadoc.

[0]
All materials are immutable once created, but if a material is recreated every frame with varying parameters, the material becomes effectively dynamic.
[1]
Such as shaders, temporary framebuffers, shadow maps, etc.
[2]
Such as the mesh shapes required to represent light volumes when performing deferred rendering.
[3]
The renderer does define two brutally simple on-disk static mesh formats for the sake of convenience, the test suite, and to store some of its own rendering resources [2], and provides tools to convert to those formats from COLLADA documents, but the programmer is absolutely not required to use them.
[5]
The jspatial package is intended to provide exactly these sorts of data structures.
[6]
That is, each surface is rendered as if it was the only surface in the visible set. There are no light bounces between surfaces, and shadows are created by explicit shadow mapping, rather than occuring naturally as part of a physically accurate global illumination algorithm.
[7]
The believability of this effect is obviously very scene-specific. Shadow mapping gives results that look much more physically accurate, at the cost of being much more computationally expensive.
[8]
Almost all rendering systems use different names to refer to the same concepts, without ever bothering to document their conventions. This harms comprehension and generally wastes everybody's time.
[9]
See Mathematics for 3D Game Programming and Computer Graphics 3rd Edition, section 4.3.1 for the derivation.
[10]
Note that matrix multiplication is not commutative.
[11]
The reason for producing the concatenated matrix on the CPU and then passing it to the shader is efficiency; if a mesh had 1000 vertices, and the shader was passed m and v separately, the shader would repeatedly perform the same mv = v * m multiplication to produce mv for each vertex - yielding the exact same mv each time!
[12]
See section 4.5, "Transforming normal vectors".
[13]
Because normalized device space is a left-handed system by default, with the viewer looking towards positive Z, and because the transformation from clip space to normalized device space for a given point is the division of the components of that point by the point's own w component.
[14]
The handedness of the coordinate space is dependent on the depth range configured for screen space.
[15]
It is actually the division by w that produces the scaling effect necessary to produce the illusion of perspective in perspective projections.
[16]
Of course, it may be that the programmer actually loads many more meshes during the lifetime of the application in question. This step is just included for the purposes of the process description.
[17]
The io7m-r1 package actually uses the Parasol language internally to avoid the mentioned issues, but does not expose this to programmers using the package.
[18]
This has the advantage that, if a texture is procedurally generated by an external system, the texture can be cached and re-used by many different surfaces. This is obviously not possible in systems that do all of that work during the actual rendering of an object.
[19]
There is a tension in rendering systems between providing something that is restrictive but easy to use, and providing something that is so generalized that it is capable of anything but barely any easier to use than using the OpenGL API directly.
[20]
Please see A Crash Course in Algebraic Types, and more or less any software written in a typed functional languages for a description of the immense correctness and maintenance benefits that algebraic data types provide.
[21]
The reason for this is that, for example, many skeletal animation systems store extra information in each vertex about the bones that influence said vertex. If the package did not ignore this extra data, then programmers would be forced to allocate whole new meshes and copy out only the attributes given above.
[22]
An intuitive way to think about transforms concatenated in this manner is that a transform always occurs about the origin. So, if a translation is applied first, and then a rotation is applied, the object will appear to orbit around the origin (because the rotation happens around the origin, and the translation moved it away from the origin first).
[23]
Which, for many applications, may be once for the entire lifetime of the program.
[24]
This step is performed once on the CPU and is only repeated when the projection matrix changes [23].
[25]
By simply setting the w component to 1.
[26]
Which is guaranteed to be negative, as only a negative Z value could have resulted in a visible fragment in the g-buffer.
[28]
Apparently first discovered by Brano Kemen.
[29]
It is fairly widely acknowledged that there is no completely satisfactory solution to the problem of rendering intersecting translucent objects. The io7m-r1 package makes no attempt to solve the problem.
[30]
See section 7.8.3, "Calculating tangent vectors".
[31]
The io7m-r1 package does not use ambient terms.
[32]
Note: Specifically Phong reflection and not the more commonly used Blinn-Phong reflection.
[33]
The attenuation function development is available for experimentation in the included GeoGebra file [attenuation.ggb].
[34]
The same issue occurs when performing ordinary rendering of points in a scene. The issue is solved there by clipping primitives based on their w component so that primitives that are "behind" the observer are not rendered.
[35]
In other words, the cache tries to keep itself to a given maximum size, but is permitted to overstep these bounds if asked: A scene that has 100 shadow casting lights requires 100 pre-populated shadow maps, regardless of whether or not this would exceed the stated maximum cache size.
[37]
The included Fxaa3_11.h file bears an NVIDIA copyright, but was placed into the public domain by the original author.