com.io7m.r2 0.3.0-SNAPSHOT Documentation
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Contents
─────────────────────────────────────────────────────────────────────────────
1 Package Information ................................................... pkg
1.1 Orientation ........................................... pkg.orientation
1.2 Installation .............................................. pkg.install
1.3 Platform Specific Issues ................................. pkg.platform
1.4 License ................................................... pkg.license
2 Design And Implementation .............................................. di
2.1 Conventions ............................................ di.conventions
2.2 Concepts .................................................. di.concepts
2.3 Coordinate Systems .......................................... di.coords
2.4 Meshes ...................................................... di.meshes
2.5 Transforms .............................................. di.transforms
2.6 Instances ................................................ di.instances
2.7 Render Targets ....................................... di.render-target
2.8 Shaders .................................................... di.shaders
2.9 Shaders: Instance ................................. di.shaders.instance
2.10 Shaders: Light ...................................... di.shaders.light
2.11 Stencils .................................................. di.stencil
2.12 Lighting ................................................. di.lighting
2.13 Lighting: Directional ........................ di.lighting.directional
2.14 Lighting: Spherical ............................ di.lighting.spherical
2.15 Lighting: Projective .......................... di.lighting.projective
2.16 Shadows ................................................... di.shadows
2.17 Shadows: Variance Mapping ........................ di.shadows.variance
2.18 Deferred Rendering ....................................... di.deferred
2.19 Deferred Rendering: Geometry ........................ di.deferred.geom
2.20 Deferred Rendering: Lighting ....................... di.deferred.light
2.21 Deferred Rendering: Position Reconstruction di.deferred-position-recon
2.22 Forward rendering (Translucency) .......................... di.forward
2.23 Normal Mapping ..................................... di.normal-mapping
2.24 Logarithmic Depth ....................................... di.log_depth
2.25 Environment Mapping ........................... di.environment-mapping
2.26 Stippling ............................................... di.stippling
2.27 Generic Refraction ............................. di.generic-refraction
2.28 Filter: Fog ................................................... di.fog
2.29 Filter: Screen Space Ambient Occlusion ....................... di.ssao
2.30 Filter: Emission ......................................... di.emission
2.31 Filter: FXAA ................................................. di.fxaa
3 API Documentation .........................................................
3.1 API Documentation ................................................. api
1 Package Information [id: pkg]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Contents
─────────────────────────────────────────────────────────────────────────────
1.1 Orientation ............................................. pkg.orientation
1.1.1 Overview ............................................................
1.1.2 Features ............................................................
1.2 Installation ................................................ pkg.install
1.2.1 Source compilation ..................................................
1.2.2 Maven ...............................................................
1.3 Platform Specific Issues ................................... pkg.platform
1.4 License ..................................................... pkg.license
1.1 Orientation [id: pkg.orientation]
1.1.1 Overview
1 The r2 package provides a minimalist deferred rendering system.
1.1.2 Features
1.1.2.1 Features
────────────────
• A deferred rendering core for opaque objects.
• A forward renderer, supporting a subset of the features of the deferred
renderer, for rendering translucent objects.
• A full dynamic lighting system, including variance shadow mapping. The
use of deferred rendering allows for potentially hundreds of dynamic
lights per scene.
• Ready-to-use shaders providing surfaces with a wide variety of effects
such as normal mapping, environment-mapped reflections, generic
refraction, surface emission, mapped specular highlights, etc.
• A variety of postprocessing effects such as box blurring, screen-space
ambient occlusion (SSAO), fast approximate antialiasing (FXAA), color
correction, bloom, etc. Effects can be applied in any order.
• Explicit control over all resource loading and caching. For all
transient resources, the programmer is required to provide the renderer
with explicit pools, and the pools themselves are responsible for
allocating and loading resources.
• Extensive use of static types. As with all io7m [url: http://io7m.com]
packages, there is extreme emphasis on using the type system to make it
difficult to use the APIs incorrectly.
• Portability. The renderer will run on any system supporting OpenGL 3.3
and Java 8.
1.1.2.2 Non-features
────────────────────
• A scene graph. The renderer expects the programmer to provide a set of
instances (with associated shaders) and lights once per frame, and the
renderer will obediently draw exactly those instances. This frees the
programmer from having to interact with a clumsy and type-unsafe
object-oriented scene graph as with other 3D engines, and from having
to try to crowbar their own program's data structures into an existing
graph system.
• Spatial partitioning. The renderer knows nothing of the world the
programmer is trying to render. The programmer is expected to have done
the work of deciding which instances and lights contribute to the
current image, and to provide only those lights and instances for the
current frame. This means that the programmer is free to use any
spatial partitioning system desired.
• Input handling. The renderer knows nothing about keyboards, mice,
joysticks. The programmer passes an immutable snapshot of a scene to
the renderer, and the renderer returns an image. This means that the
programmer is free to use any input system desired without having to
painfully integrate their own code with an existing input system as
with other 3D engines.
• Audio. The renderer makes images, not sounds. This allows programmers
to use any audio system they want in their programs.
• Skeletal animation. The input to the renderer is a set of triangle
meshes in the form of vertex buffer objects. This means that the
programmer is free to use any skeletal animation system desired,
providing that the system is capable of producing vertex buffer objects
of the correct type as a result.
• Model loading. The input to the renderer is a set of triangle meshes in
the form of vertex buffer objects. This means that the programmer is
free to use any model loading system desired, providing that the system
is capable of producing vertex buffer objects of the correct type as a
result.
• Future proofing. The average lifetime of a rendering system is about
five years. Due to the extremely rapid pace of advancement in graphics
hardware, the methods use to render graphics today will bear almost no
relation to those used five years into the future. The r2 package is
under no illusion that it will still be relevant in a decade's time. It
is designed to get work done today, using exactly those techniques that
are relevant today. It will not be indefinitely expanded and grown
organically, as this would directly contradict the goal of having a
minimalist and correct rendering system.
• OpenGL ES 2 support. The ES 2 standard was written as a reaction to the
insane committee politics that plagued the OpenGL 2.* standards. It is
crippled to the point that it essentially cannot support almost any of
the rendering techniques present in the r2 package, and is becoming
increasingly irrelevant as the much saner ES 3 is adopted by hardware
vendors.
1.2 Installation [id: pkg.install]
1.2.1 Source compilation
1 The project can be compiled and installed with Maven [url:
http://maven.apache.org]:
2 $ mvn -C clean install
1.2.2 Maven
1 Regular releases are made to the Central Repository [url:
http://search.maven.org/#search%7Cga%7C1%7Ccom.io7m.r2], so it's possible
to use the com.io7m.r2 package in your projects with the following Maven
dependencies:
2
com.io7m.r2
io7m-r2-main
0.3.0-SNAPSHOT
3 All io7m.com [url: http://io7m.com] packages use Semantic Versioning [0],
which implies that it is always safe to use version ranges with an
exclusive upper bound equal to the next major version - the API of the
package will not change in a backwards-incompatible manner before the
next major version.
1.3 Platform Specific Issues [id: pkg.platform]
1 There are currently no known platform-specific issues.
1.4 License [id: pkg.license]
1 All files distributed with the com.io7m.r2 package are placed under the
following license:
Copyright © 2016 http://io7m.com
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
2 Design And Implementation [id: di]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Contents
─────────────────────────────────────────────────────────────────────────────
2.1 Conventions .............................................. di.conventions
2.1.1 Overview ............................................................
2.1.2 Mathematics ..................................... di.conventions.math
2.2 Concepts .................................................... di.concepts
2.2.1 Overview ............................................................
2.2.2 Renderer ....................................... di.concepts.renderer
2.2.3 Render Target ............................. di.concepts.render_target
2.2.4 Geometry Buffer ................................. di.concepts.gbuffer
2.2.5 Light Buffer .................................... di.concepts.lbuffer
2.2.6 Mesh ............................................... di.concepts.mesh
2.2.7 Transform ..................................... di.concepts.transform
2.2.8 Instance ....................................... di.concepts.instance
2.2.9 Light ............................................. di.concepts.light
2.2.10 Light Clip Group ...................... di.concepts.light_clip_group
2.2.11 Light Group ................................ di.concepts.light_group
2.2.12 Shader .......................................... di.concepts.shader
2.2.13 Material ...................................... di.concepts.material
2.3 Coordinate Systems ............................................ di.coords
2.3.1 Conventions ................................... di.coords.conventions
2.3.2 Object Space ....................................... di.coords.object
2.3.3 World Space ......................................... di.coords.world
2.3.4 Eye Space ............................................. di.coords.eye
2.3.5 Clip Space ........................................... di.coords.clip
2.3.6 Normalized-Device Space ........................... di.coords.ndevice
2.3.7 Screen Space ....................................... di.coords.screen
2.4 Meshes ........................................................ di.meshes
2.4.1 Overview ......................................... di.meshes.overview
2.4.2 Attributes ..................................... di.meshes.attributes
2.4.3 Types ............................................... di.meshes.types
2.5 Transforms ................................................ di.transforms
2.5.1 Overview ..................................... di.transforms.overview
2.5.2 Types ........................................... di.transforms.types
2.6 Instances .................................................. di.instances
2.6.1 Overview ...................................... di.instances.overview
2.6.2 Single .......................................... di.instances.single
2.6.3 Batched ........................................ di.instances.batched
2.6.4 Billboarded ................................ di.instances.billboarded
2.6.5 Types ............................................ di.instances.types
2.7 Render Targets ......................................... di.render-target
2.7.1 Overview .................................. di.render-target.overview
2.7.2 Types ........................................ di.render-target.types
2.8 Shaders ...................................................... di.shaders
2.8.1 Overview ........................................ di.shaders.overview
2.8.2 Interface And Calling Protocol ................. di.shaders.interface
2.8.3 Shader Modules ................................... di.shaders.modules
2.8.4 Types .............................................. di.shaders.types
2.9 Shaders: Instance ................................... di.shaders.instance
2.9.1 Overview ............................... di.shaders.instance.overview
2.9.2 Materials .............................. di.shaders.instance.material
2.9.3 Provided Shaders ....................... di.shaders.instance.provided
2.9.4 Types ..................................... di.shaders.instance.types
2.10 Shaders: Light ........................................ di.shaders.light
2.10.1 Overview ................................. di.shaders.light.overview
2.10.2 Types ....................................... di.shaders.light.types
2.11 Stencils .................................................... di.stencil
2.11.1 Overview ....................................... di.stencil.overview
2.11.2 Reserved Bits .................................. di.stencil.reserved
2.11.3 Allow Bit ..................................... di.stencil.allow_bit
2.12 Lighting ................................................... di.lighting
2.12.1 Overview ...................................... di.lighting.overview
2.12.2 Diffuse/Specular Terms ................ di.lighting.diffuse-specular
2.12.3 Diffuse-Only Lights ....................... di.lighting.diffuse-only
2.12.4 Attenuation ................................ di.lighting.attenuation
2.13 Lighting: Directional .......................... di.lighting.directional
2.13.1 Overview .......................... di.lighting.directional.overview
2.13.2 Attenuation .................... di.lighting.directional.attenuation
2.13.3 Application .................... di.lighting.directional.application
2.13.4 Types ................................ di.lighting.directional.types
2.14 Lighting: Spherical .............................. di.lighting.spherical
2.14.1 Overview ............................ di.lighting.spherical.overview
2.14.2 Attenuation ...................... di.lighting.spherical.attenuation
2.14.3 Application ...................... di.lighting.spherical.application
2.14.4 Types .................................. di.lighting.spherical.types
2.15 Lighting: Projective ............................ di.lighting.projective
2.15.1 Overview ........................... di.lighting.projective.overview
2.15.2 Algorithm ......................... di.lighting.projective.algorithm
2.15.3 Back projection ............. di.lighting.projective.back-projection
2.15.4 Clamping ........................... di.lighting.projective.clamping
2.15.5 Attenuation ..................... di.lighting.projective.attenuation
2.15.6 Application ..................... di.lighting.projective.application
2.16 Shadows ..................................................... di.shadows
2.16.1 Overview ....................................... di.shadows.overview
2.16.2 Shadow Geometry ......................... di.shadows.shadow-geometry
2.17 Shadows: Variance Mapping .......................... di.shadows.variance
2.17.1 Overview .............................. di.shadows.variance.overview
2.17.2 Algorithm ............................ di.shadows.variance.algorithm
2.17.3 Advantages .......................... di.shadows.variance.advantages
2.17.4 Disadvantages .................... di.shadows.variance.disadvantages
2.17.5 Types .................................... di.shadows.variance.types
2.18 Deferred Rendering ......................................... di.deferred
2.18.1 Overview ...................................... di.deferred.overview
2.19 Deferred Rendering: Geometry .......................... di.deferred.geom
2.19.1 Overview ................................. di.deferred.geom.overview
2.19.2 Groups ...................................... di.deferred.geom.group
2.19.3 Geometry Buffer ........................... di.deferred.geom.gbuffer
2.19.4 Algorithm ..........................................................
2.19.5 Ordering/Batching ........................ di.deferred.geom.ordering
2.19.6 Normal Compression ............. di.deferred.geom.normal-compression
2.20 Deferred Rendering: Lighting ......................... di.deferred.light
2.20.1 Overview ................................ di.deferred.light.overview
2.20.2 Light Buffer ............................. di.deferred.light.lbuffer
2.20.3 Light Clip Volumes .................. di.deferred.light.clip_volumes
2.20.4 No L-Buffer ........................... di.deferred.light.no_lbuffer
2.20.5 Types ...................................... di.deferred.light.types
2.21 Deferred Rendering: Position Reconstruction . di.deferred-position-recon
2.21.1 Overview ....................... di.deferred-position-recon.overview
2.21.2 Recovering Eye space Z ...... di.deferred-position-recon.eye-space-z
2.21.3 Recovering Eye space Z (Logarithmic depth encoding) di.deferred-position-recon.eye-space-z.log-depth-encoding
2.21.4 Recovering Eye space Z (Screen space depth encoding) di.deferred-position-recon.eye-space-z.screen-space-encoding
2.21.5 Recovering Eye space Position . di.deferred-position-recon.eye-space
2.21.6 Implementation ........... di.deferred-position-recon.implementation
2.22 Forward rendering (Translucency) ............................ di.forward
2.22.1 Overview ....................................... di.forward.overview
2.22.2 Instances ..................................... di.forward.instances
2.22.3 Blending ....................................... di.forward.blending
2.22.4 Culling ......................................... di.forward.culling
2.22.5 Ordering ....................................... di.forward.ordering
2.22.6 Types ............................................. di.forward.types
2.22.7 Provided Shaders ............................... di.forward.provided
2.23 Normal Mapping ....................................... di.normal-mapping
2.23.1 Overview ................................ di.normal-mapping.overview
2.23.2 Tangent Space ...................... di.normal-mapping.tangent-space
2.23.3 Tangent/Bitangent Generation di.normal-mapping.tangent-bitangent-generation
2.23.4 Normal Maps .................................. di.normal-mapping.map
2.23.5 Rendering With Normal Maps ............. di.normal-mapping.rendering
2.24 Logarithmic Depth ......................................... di.log_depth
2.24.1 Overview ..................................... di.log_depth.overview
2.24.2 OpenGL Depth Issues ................... di.log_depth.issues_existing
2.24.3 Logarithmic Encoding ......................... di.log_depth.encoding
2.25 Environment Mapping ............................. di.environment-mapping
2.25.1 Overview ........................... di.environment-mapping.overview
2.25.2 Cube Maps ......................... di.environment-mapping.cube-maps
2.25.3 Reflections ...................... di.environment-mapping.reflection
2.25.4 Handedness ....................... di.environment-mapping.handedness
2.26 Stippling ................................................. di.stippling
2.26.1 Overview ..................................... di.stippling.overview
2.26.2 Algorithm ................................... di.stippling.algorithm
2.26.3 Types ........................................... di.stippling.types
2.27 Generic Refraction ............................... di.generic-refraction
2.27.1 Overview ............................ di.generic-refraction.overview
2.27.2 Algorithm .......................... di.generic-refraction.algorithm
2.27.3 Sources ............................... di.generic-refraction.source
2.27.4 Vectors .............................. di.generic-refraction.vectors
2.27.5 Colors ................................. di.generic-refraction.color
2.27.6 Masking .............................. di.generic-refraction.masking
2.27.7 Types .................................. di.generic-refraction.types
2.28 Filter: Fog ..................................................... di.fog
2.28.1 Overview ........................................... di.fog.overview
2.28.2 Algorithm ......................................... di.fog.algorithm
2.28.3 Types ................................................. di.fog.types
2.29 Filter: Screen Space Ambient Occlusion ......................... di.ssao
2.29.1 Overview .......................................... di.ssao.overview
2.29.2 Ambient Occlusion Buffer ........................... di.ssao.abuffer
2.29.3 Algorithm ........................................ di.ssao.algorithm
2.29.4 Noise Texture ................................ di.ssao.noise_texture
2.29.5 Sample Kernel ....................................... di.ssao.kernel
2.29.6 Halo Removal .................................. di.ssao.halo_removal
2.29.7 Performance .................................... di.ssao.performance
2.29.8 Types ................................................ di.ssao.types
2.29.9 Shaders ............................................ di.ssao.shaders
2.30 Filter: Emission ........................................... di.emission
2.30.1 Overview ...................................... di.emission.overview
2.30.2 Algorithm .................................... di.emission.algorithm
2.30.3 Types ............................................ di.emission.types
2.31 Filter: FXAA ................................................... di.fxaa
2.31.1 Overview .......................................... di.fxaa.overview
2.31.2 Implementation .............................. di.fxaa.implementation
2.31.3 Types ................................................ di.fxaa.types
2.1 Conventions [id: di.conventions]
2.1.1 Overview
1 This section attempts to document the mathematical and typographical
conventions used in the rest of the documentation.
2.1.2 Mathematics [id: di.conventions.math]
1 Rather than rely on untyped and ambiguous mathematical notation, this
documentation expresses all mathematics and type definitions in strict
Haskell 2010 [url: http://www.haskell.org/onlinereport/haskell2010/] with
no extensions. All Haskell sources are included along with the
documentation and can therefore be executed from the command line GHCi
[url: http://www.haskell.org/haskellwiki/GHC/GHCi] tool in order to
interactively check results and experiment with functions.
2 When used within prose, functions are usually referred to using fully
qualified notation, such as (Vector3f.cross n t). This is the application
of the cross function defined in the Vector3f module, to the arguments n
and t.
2.2 Concepts [id: di.concepts]
2.2.1 Overview
1 This section attempts to provide a rough overview of the concepts present
in the r2 package. Specific implementation details, mathematics, and
other technical information is given in later sections that focus on each
concept in detail.
2.2.2 Renderer [id: di.concepts.renderer]
1 A renderer is a function that takes an input of some type and produces an
output to a render target [ref: di.concepts.render_target].
2 The renderers expose an interface of stateless functions from inputs to
outputs. That is, the renderers should be considered to simply take input
and produce return images as output. In reality, because the Java
language is not pure and because the code is required to perform I/O in
order to speak to the GPU, the renderer functions are not really pure.
Nevertheless, for the sake of ease of use, lack of surprising results,
and correctness, the renderers at least attempt to adhere to the idea of
pure functional rendering! This means that the renderers are very easy to
integrate into any existing system: They are simply functions that are
evaluated whenever the programmer wants an image. The renderers do not
have their own main loop, they do not have any concept of time, do not
remember any images that they have produced previously, do not maintain
any state of their own, and simply write their results to a
programmer-provided render target. Passing the same input to a renderer
multiple times should result in the same image each time.
2.2.3 Render Target [id: di.concepts.render_target]
1 A render target is a rectangular region of memory allocated on the GPU
that can accept the results of a rendering operation. The programmer
typically allocates one render target, passes it to a renderer along with
a renderer-specific input value, and the renderer populates the given
render target with the results. The programmer can then copy the contents
of this render target to the screen for viewing, pass it on to a separate
filter for extra visual effects, use it as a texture to be applied to
objects in further rendered images, etc.
2.2.4 Geometry Buffer [id: di.concepts.gbuffer]
1 A geometry buffer is a specific type of render target [ref:
di.concepts.render_target] that contains the surface attributes of a set
of rendered instances [ref: di.concepts.instance]. It is a fundamental
part of deferred rendering that allows lighting to be efficiently
calculated in screen space [ref: di.coords.screen], touching only those
pixels that will actually contribute to the final rendered image.
2.2.5 Light Buffer [id: di.concepts.lbuffer]
1 A light buffer is a specific type of render target [ref:
di.concepts.render_target] that contains the summed light contributions
for each pixel in the currently rendered scene.
2.2.6 Mesh [id: di.concepts.mesh]
1 A mesh is a collection of vertices that define a polyhedral object, along
with a list of indices that describe how to make triangles out of the
given vertices.
2 Meshes are allocated on the GPU and can be shared between any number of
instances [ref: di.concepts.instance] (meaning that rendering 100
identical objects does not require storing 100 copies of the mesh data).
2.2.7 Transform [id: di.concepts.transform]
1 A transform moves coordinates in one coordinate space [ref: di.coords] to
another. Typically, a transform is used to position and orient a mesh
[ref: di.concepts.mesh] inside a visible set.
2.2.8 Instance [id: di.concepts.instance]
1 An instance is essentially an object or group of objects that can be
rendered. Instances come in several forms: single, batched, and
billboarded.
2 A single instance consists of a reference to a mesh [ref:
di.concepts.mesh] and a transform [ref: di.concepts.transform] for
positioning the instance within a scene.
3 A batched instance consists of a reference to a mesh and an array of
transforms. The results of rendering a batched instance are the same as
if a single instance had been created and rendered for each transform in
the array. The advantage of batched instances is efficiency: Batched
instances are submitted to the GPU for rendering in a single draw call.
Reducing the total number of draw calls per scene is an important
optimization on modern graphics hardware, and batched instances provide a
means to achieve this.
4 A billboarded instance is a further specialization of a batched instance
intended for rendering large numbers of objects that always face towards
the observer. Billboarding is a technique that is often used to render
large numbers of distant objects in a scene: Rather than incur the
overhead of rendering lots of barely-visible objects at full detail, the
objects are replaced with billboarded sprites at a fraction of the cost.
There is also a significant saving in the memory used to store
transforms, because a billboarded sprite need only store a position and
scale as opposed to a full transform matrix per rendered object.
2.2.9 Light [id: di.concepts.light]
1 A light describes a light source within a scene. There are many different
types of lights, each with different behaviours. Lights may or may not
cast shadows, depending on their type. All lighting in the r2 package is
completely dynamic; there is no support for static lighting in any form.
Shadows are exclusively provided via shadow mapping, resulting in
efficient per-pixel shadows.
2.2.10 Light Clip Group [id: di.concepts.light_clip_group]
1 A light clip group is a means of constraining the contributions of groups
of lights [ref: di.concepts.light] to a provided volume.
2 Because, like most renderers, the r2 package implements so-called local
illumination, lights that do not have explicit shadow mapping enabled are
able to bleed through solid objects:
2.2.10.3 Local Light Bleed
──────────────────────────
[image: images/lightbleed_noclip.png] (Local light bleeding.)
4 Enabling shadow mapping for every single light source would be
prohibitively expensive [1], but for some scenes, acceptable results can
be achieved by simply preventing the light source from affecting pixels
outside of a given clip volume.
2.2.10.5 Local Light Clipped
────────────────────────────
[image: images/lightbleed_clip.png] (Local light clipped to a volume.)
2.2.11 Light Group [id: di.concepts.light_group]
1 A light group is similar to a light clip group [ref:
di.concepts.light_clip_group] in that is intended to constrain the
contributions of a set of lights. A light group instead requires the
cooperation of a renderer that can mark groups of instances [ref:
di.concepts.instance] using the stencil component of the current geometry
buffer [ref: di.concepts.gbuffer]. At most 15 light groups can be present
in a given scene, and for a given light group n, only instances in group
n will be affected by lights in group n. By default, if a group is not
otherwise specified, all lights and instances are rendered in group 1.
2.2.12 Shader [id: di.concepts.shader]
1 A shader is a small program that executes on the GPU and is used to
produce images. The r2 package provides a wide array of general-purpose
shaders, and the intention is that users of the package will not
typically have to write their own [2].
2 The package roughly divides shaders into categories. Single instance
shaders are typically used to calculate and render the surface attributes
of single instances [ref: di.concepts.instance.single] into a geometry
buffer [ref: di.concepts.gbuffer]. Batched instance shaders do the same
for batched instances [ref: di.concepts.instance.batched]. Light shaders
render the contributions of light sources into a light buffer [ref:
di.concepts.lbuffer]. There are many other types of shader in the r2
package but users are generally not exposed to them directly.
3 Shaders are intended to be effectively stateless. A given shader S is an
opaque function that takes a single parameter value M, and the user
actually supplies M by configuring a material [ref: di.concepts.material]
for S and then using it each frame.
2.2.13 Material [id: di.concepts.material]
1 A material is a pair consisting of a shader [ref: di.concepts.shader] and
a set of parameters for that shader [3].
2.3 Coordinate Systems [id: di.coords]
2.3.1 Conventions [id: di.coords.conventions]
1 This section attempts to describe the mathematical conventions that the
r2 package uses with respect to coordinate systems. The r2 package
generally does not deviate from standard OpenGL conventions, and this
section does not attempt to give a rigorous formal definition of these
existing conventions. It does however attempt to establish the naming
conventions that the package uses to refer to the standard coordinate
spaces [4].
2 The r2 package uses the jtensors [url: http://io7m.github.io/jtensors]
package for all mathematical operations on the CPU, and therefore shares
its conventions with regards to coordinate system handedness. Important
parts are repeated here, but the documentation for the jtensors package
should be inspected for details.
3 Any of the matrix functions that deal with rotations assume a
right-handed coordinate system. This matches the system conventionally
used by OpenGL [url: http://opengl.org] (and most mathematics literature)
. A right-handed coordinate system assumes that if the viewer is standing
at the origin and looking towards negative infinity on the Z axis, then
the X axis runs horizontally (left towards negative infinity and right
towards positive infinity) , and the Y axis runs vertically (down towards
negative infinity and up towards positive infinity). The following image
demonstrates this axis configuration:
2.3.1.4 Right Handed Coordinate System
──────────────────────────────────────
[image: images/axes2.png] (A right handed coordinate system diagram.)
5 The jtensors package adheres to the convention that a positive rotation
around an axis represents a counter-clockwise rotation when viewing the
system along the negative direction of the axis in question.
2.3.1.6 Rotations
─────────────────
[image: images/rotations.png] (A diagram of right-handed rotations.)
7 The package uses the following matrices to define rotations around each
axis:
2.3.1.8 Rotation of r radians around the X axis
───────────────────────────────────────────────
[image: images/matrix_rx.png] (Rotation of r radians around the X axis.)
2.3.1.9 Rotation of r radians around the Y axis
───────────────────────────────────────────────
[image: images/matrix_ry.png] (Rotation of r radians around the Y axis.)
2.3.1.10 Rotation of r radians around the Z axis
────────────────────────────────────────────────
[image: images/matrix_rz.png] (Rotation of r radians around the Z axis.)
11 Which results in the following matrix for rotating r radians around the
axis given by (x, y, z), assuming s = sin(r) and c = cos(r):
2.3.1.12 Rotation of r radians around an arbitrary axis
───────────────────────────────────────────────────────
[image: images/rot_matrix.png] (Rotation of r radians around an arbitrary
axis.)
2.3.2 Object Space [id: di.coords.object]
1 Object space is the local coordinate system used to describe the
positions of vertices in meshes. For example, a unit cube with the origin
placed at the center of the cube would have eight vertices with positions
expressed as object-space coordinates:
2.3.2.2 Unit cube vertices
──────────────────────────
cube = {
(-0.5, -0.5, -0.5),
( 0.5, -0.5, -0.5),
( 0.5, -0.5, 0.5),
(-0.5, -0.5, 0.5),
(-0.5, 0.5, -0.5),
( 0.5, 0.5, -0.5),
( 0.5, 0.5, 0.5),
(-0.5, 0.5, 0.5)
}
3 In other rendering systems, object space is sometimes referred to as
local space, or model space.
4 In the r2 package, object space is represented by the R2SpaceObjectType
[url: apidocs/com/io7m/r2/spaces/R2SpaceObjectType.html].
2.3.3 World Space [id: di.coords.world]
1 In order to position objects in a scene, they must be assigned a
transform [ref: di.concepts.transform] that can be applied to each of
their object space [ref: di.coords.object] vertices to yield absolute
positions in so-called world space.
2 As an example, if the unit cube [ref: di.coords.object.cube] described
above was assigned a transform that moved its origin to (3, 5, 1), then
its object space vertex (-0.5, 0.5, 0.5) would end up at
(3 + -0.5, 5 + 0.5, 1 + 0.5) = (2.5, 5.5, 1.5) in world space.
3 In the r2 package, a transform applied to an object produces a 4x4 model
matrix. Multiplying the model matrix with the positions of the object
space vertices yields vertices in world space.
4 Note that, despite the name, world space does not imply that users have
to store their actual world representation in this coordinate space. For
example, flight simulators often have to transform their planet-scale
world representation to an aircraft relative representation for rendering
to work around the issues inherent in rendering extremely large scenes.
The basic issue is that the relatively low level of floating point
precision available on current graphics hardware means that if the
coordinates of objects within the flight simulator's world were to be
used directly, the values would tend to be drastically larger than those
that could be expressed by the available limited-precision floating point
types on the GPU. Instead, simulators often transform the locations of
objects in their worlds such that the aircraft is placed at the origin
(0, 0, 0) and the objects are positioned relative to the aircraft before
being passed to the GPU for rendering. As a concrete example, within the
simulator's world, the aircraft may be at (1882838.3, 450.0, 5892309.0),
and a control tower nearby may be at (1883838.5, 0.0, 5892809.0). These
coordinate values would be far too large to pass to the GPU if a
reasonable level of precision is required, but if the current aircraft
location is subtracted from all positions, the coordinates in aircraft
relative space of the aircraft become (0, 0, 0) and the coordinates of
the tower become
(1883838.5 - 1882838.3, 0.0 - 450.0, 5892809.0 - 5892309.0) = (1000.19, -450.0, 500.0).
The aircraft relative space coordinates are certainly small enough to be
given to the GPU directly without risking imprecision issues, and
therefore the simulator would essentially treat aircraft relative space
and r2 world space as equivalent [5].
5 In the r2 package, world space is represented by the R2SpaceWorldType
[url: apidocs/com/io7m/r2/spaces/R2SpaceWorldType.html].
2.3.4 Eye Space [id: di.coords.eye]
1 Eye space represents a coordinate system with the observer implicitly
fixed at the origin (0.0, 0.0, 0.0) and looking towards infinity in the
negative Z direction.
2 The main purpose of eye space is to simplify the mathematics required to
implement various algorithms such as lighting. The problem with
implementing these sorts of algorithms in world space [ref:
di.coords.world] is that one must constantly take into account the
position of the observer (typically by subtracting the location of the
observer from each set of world space coordinates and accounting for any
change in orientation of the observer). By fixing the orientation of the
observer towards negative Z, and the position of the observer at
(0.0, 0.0, 0.0), and by transforming all vertices of all objects into the
same system, the mathematics of lighting are greatly simplified. The
majority of the rendering algorithms used in the r2 package are
implemented in eye space.
3 In the r2 package, the observer produces a 4x4 view matrix. Multiplying
the view matrix with any given world space position yields a position in
eye space. In practice, the view matrix v and the current object's model
matrix m are concatenated (multiplied) to produce a model-view matrix
mv = v * m [6], and mv is then passed directly to the renderer's vertex
shaders to transform the current object's vertices [7].
4 Additionally, as the r2 package does all lighting in eye space, it's
necessary to transform the object space normal vectors given in mesh data
to eye space. However, the usual model-view matrix will almost certainly
contain some sort of translational component and possibly a scaling
component. Normal vectors are not supposed to be translated; they
represent directions! A non-uniform scale applied to an object will also
deform the normal vectors, making them non-perpendicular to the surface
they're associated with:
2.3.4.5 Unit cube vertices
──────────────────────────
[image: images/normal_deform.png] (Deformed normal vectors.)
6 With the scaled triangle on the right, the normal vector is now not
perpendicular to the surface (in addition to no longer being of unit
length). The red vector indicates what the surface normal should be.
7 Therefore it's necessary to derive another 3x3 matrix known as the normal
matrix from the model-view matrix that contains just the rotational
component of the original matrix. The full derivation of this matrix is
given in Mathematics for 3D Game Programming and Computer Graphics, Third
Edition [url: http://www.mathfor3dgameprogramming.com/] [8]. Briefly, the
normal matrix is equal to the inverse transpose of the top left 3x3
elements of an arbitrary 4x4 model-view matrix.
8 In other rendering systems, eye space is sometimes referred to as camera
space, or view space.
9 In the r2 package, eye space is represented by the R2SpaceEyeType [url:
apidocs/com/io7m/r2/spaces/R2SpaceEyeType.html].
2.3.5 Clip Space [id: di.coords.clip]
1 Clip space is a homogeneous coordinate system in which OpenGL performs
clipping of primitives (such as triangles). In OpenGL, clip space is
effectively a left-handed coordinate system by default [9]. Intuitively,
coordinates in eye space are transformed with a projection (normally
either an orthographic or perspective projection) such that all vertices
are projected into a homogeneous unit cube placed at the origin - clip
space - resulting in four-dimensional (x, y, z, w) positions. Positions
that end up outside of the cube are clipped (discarded) by dedicated
clipping hardware, usually producing more triangles as a result.
2.3.5.2 Primitive Clipping
──────────────────────────
[image: images/clipping.png] (A diagram of primitive clipping.)
3 A projection effectively determines how objects in the three-dimensional
scene are projected onto the two-dimensional viewing plane (a computer
screen, in most cases) . A perspective projection transforms vertices
such that objects that are further away from the viewing plane appear to
be smaller than objects that are close to it, while an orthographic
projection preserves the perceived sizes of objects regardless of their
distance from the viewing plane.
2.3.5.4 Perspective projection
──────────────────────────────
[image: images/proj_perspective.png] (A diagram of perspective
projection.)
2.3.5.5 Orthographic projection
───────────────────────────────
[image: images/proj_ortho.png] (A diagram of orthographic projection.)
6 Because eye space [ref: di.coords.eye] is a right-handed coordinate
system by convention, but by default clip space is left-handed, the
projection matrix used will invert the sign of the z component of any
given point.
7 In the r2 package, the observer produces a 4x4 projection matrix. The
projection matrix is passed, along with the model-view [ref:
di.coords.eye.modelview] matrix, to the renderer's vertex shaders. As is
normal in OpenGL, the vertex shader produces clip space coordinates which
are then used by the hardware rasterizer to produce color fragments
onscreen.
8 In the r2 package, clip space is represented by the R2SpaceClipType [url:
apidocs/com/io7m/r2/spaces/R2SpaceClipType.html].
2.3.6 Normalized-Device Space [id: di.coords.ndevice]
1 Normalized-device space is, by default, a left-handed [10] coordinate
space in which clip space [ref: di.coords.clip] coordinates have been
divided by their own w component (discarding the resulting w = 1
component in the process), yielding three dimensional coordinates. The
range of values in the resulting coordinates are effectively normalized
by the division to fall within the ranges [(-1, -1, -1), (1, 1, 1)] [11].
The coordinate space represents a simplifying intermediate step between
having clip space coordinates and getting something projected into a
two-dimensional image (screen space) [ref: di.coords.screen] for viewing.
2 The r2 package does not directly use or manipulate values in
normalized-device space; it is mentioned here for completeness.
2.3.7 Screen Space [id: di.coords.screen]
1 Screen space is, by default, a left-handed coordinate system representing
the screen (or window) that is displaying the actual results of
rendering. If the screen is of width w and height h, and the current
depth range of the window is [n, f], then the range of values in screen
space coordinates runs from [(0, 0, n), (w, h, f)]. The origin (0, 0, 0)
is assumed to be at the bottom-left corner.
2 The depth range is actually a configurable value, but the r2 package
keeps the OpenGL default. From the glDepthRange function manual page:
2.3.7.3 glDepthRange
────────────────────
After clipping and division by w, depth coordinates range from -1 to 1,
corresponding to the near and far clipping planes. glDepthRange specifies a
linear mapping of the normalized depth coordinates in this range to window
depth coordinates. Regardless of the actual depth buffer implementation,
window coordinate depth values are treated as though they range from 0
through 1 (like color components). Thus, the values accepted by
glDepthRange are both clamped to this range before they are accepted.
The setting of (0,1) maps the near plane to 0 and the far plane to 1.
With this mapping, the depth buffer range is fully utilized.
4 As OpenGL, by default, specifies a depth range of [0, 1], the positive Z
axis points away from the observer and so the coordinate system is left
handed.
2.4 Meshes [id: di.meshes]
2.4.1 Overview [id: di.meshes.overview]
1 A mesh is a collection of vertices that make up the triangles that define
a polyhedral object, allocated on the GPU upon which the renderer is
executing. In practical terms, a mesh is a pair (a, i), where a is an
OpenGL vertex buffer object consisting of vertices, an i is an OpenGL
element buffer object consisting of indices that describe how to draw the
mesh as a series of triangles.
2 The contents of a are mutable, but mesh references are considered to be
immutable.
2.4.2 Attributes [id: di.meshes.attributes]
1 A mesh consists of vertices. A vertex can be considered to be a value of
a record type, with the fields of the record referred to as the
attributes of the vertex. In the r2 package, an array buffer containing
vertex data is specified using the array buffer types from jcanephora
[url: http://io7m.github.io/jcanephora]. The jcanephora package allows
programmers to specify the exact types of array buffers, allows for the
full inspection of type information at runtime, including the ability to
reference attributes by name, and allows for type-safe modification of
the contents of array buffers using an efficient cursor interface.
2 Each attribute within an array buffer is assigned a numeric attribute
index. A numeric index is an arbitrary number between (including) 0 and
some OpenGL implementation-defined upper limit. On modern graphics
hardware, OpenGL allows for at least 16 numeric attributes. The indices
are used to create an association between fields in the array buffer and
shader inputs. For the sake of sanity and consistency, it is the
responsibility of rendering systems using OpenGL to establish conventions
for the assignment of numeric attribute indices in shaders and array
buffers [12]. For example, many systems state that attribute 0 should be
of type vec4 and should represent vertex positions. Shaders simply assume
that data arriving on attribute input 0 represents position data, and
programmers are expected to create meshes where attribute 0 points to the
field within the array that contains position data.
3 The r2 package uses the following conventions everywhere:
2.4.2.4 Mesh attribute conventions
──────────────────────────────────
┌─────────────────────┬────────────────────┬───────────────────────────┐
│ Index │ Type │ Description │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 0 │ vec3 │ The object-space position │
│ │ │ of the vertex │
│ │ │ │
│ │ │ │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 1 │ vec2 │ The UV coordinates of the │
│ │ │ vertex │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 2 │ vec3 │ The object-space normal │
│ │ │ vector of the vertex │
│ │ │ │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 3 │ vec4 │ The tangent vector of the │
│ │ │ vertex │
└─────────────────────┴────────────────────┴───────────────────────────┘
5 Batched instances are expected to use the following additional
conventions:
2.4.2.6 Batched instance attribute conventions
──────────────────────────────────────────────
┌─────────────────────┬────────────────────┬───────────────────────────┐
│ Index │ Type │ Description │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 4 │ vec4 │ Column 0 of the │
│ │ │ per-instance model matrix │
│ │ │ for batched instances. │
│ │ │ │
│ │ │ │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 5 │ vec4 │ Column 1 of the │
│ │ │ per-instance model matrix │
│ │ │ for batched instances. │
│ │ │ │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 6 │ vec4 │ Column 2 of the │
│ │ │ per-instance model matrix │
│ │ │ for batched instances. │
│ │ │ │
├─────────────────────┼────────────────────┼───────────────────────────┤
│ 7 │ vec4 │ Column 3 of the │
│ │ │ per-instance model matrix │
│ │ │ for batched instances. │
│ │ │ │
└─────────────────────┴────────────────────┴───────────────────────────┘
2.4.3 Types [id: di.meshes.types]
1 In the r2 package, the given attribute conventions are specified by the
R2AttributeConventions [url:
apidocs/com/io7m/r2/core/R2AttributeConventions.html] type.
2.5 Transforms [id: di.transforms]
2.5.1 Overview [id: di.transforms.overview]
1 The ultimate purpose of a transform is to produce one or more matrices
that can be combined with other matrices and then finally passed to a
shader. The shader uses these matrices to transform vertices and normal
vectors during the rendering of objects.
2 A transform is effectively responsible for producing a model matrix that
transforms positions in object space [ref: di.coords.object] to world
space [ref: di.coords.world].
3 In practical terms, a transform is a matrix used to position, scale, and
rotate instances [ref: di.instances] in a scene. This is achieved by
multiplying the matrix with the object space positions of all vertices of
the mesh that makes up the instance during rendering.
2.5.2 Types [id: di.transforms.types]
1 In the r2 package, transforms are instances of R2TransformType [url:
apidocs/com/io7m/r2/core/R2TransformType.html].
2.6 Instances [id: di.instances]
2.6.1 Overview [id: di.instances.overview]
1 An instance is a renderable object. There are several types of instances
available in the r2 package: single [ref: di.instances.single], batched
[ref: di.instances.batched], and billboarded [ref:
di.instances.billboarded].
2.6.2 Single [id: di.instances.single]
1 A single instance is the simplest type of instance available in the r2
package. A single instance is simply a pair (m, t), where m is a mesh
[ref: di.meshes], and t is a transform [ref: di.transforms] capable of
transforming the object space [ref: di.coords.object] coordinates of the
vertices contained within m to world space [ref: di.coords.world].
2.6.3 Batched [id: di.instances.batched]
1 A batched instance represents a group of (identical) renderable objects.
The reason for the existence of batched instances is simple efficiency:
On modern rendering hardware, rendering n single instances means
submitting n draw calls to the GPU. As n becomes increasingly large, the
overhead of the large number of draw calls becomes a bottleneck for
rendering performance. A batched instance of size m allows for rendering
a given mesh m times in a single draw call.
2 A batched instance of size n is a 3-tuple (m, b, t), where m is a mesh
[ref: di.meshes], b is a buffer of n 4x4 matrices [ref:
di.meshes.attributes.batched] allocated on the GPU, and t is an array of
n transforms [ref: di.transforms] allocated on the CPU. For each i where
0 <= i < n, b[i] is the 4x4 model matrix produced from t[i]. The contents
of b are typically recalculated and uploaded to the GPU once per
rendering frame.
2.6.4 Billboarded [id: di.instances.billboarded]
1 A billboarded instance is a further specialization of batched instances.
Billboarding is the name given to a rendering technique where instead of
rendering full 3D objects, simple 2D images of those objects are rendered
instead using flat rectangles that are arranged such that they are always
facing directly towards the observer.
2.6.4.2 Billboarded Render
──────────────────────────
[image: images/billboard_render.png] (Billboarding
2.6.4.3 Billboarded Wireframe
─────────────────────────────
[image: images/billboard_wire.png] (Billboarding in wireframe view)
2.6.4.4 Billboarded Wireframe (Side)
────────────────────────────────────
[image: images/billboard_wire_side.png] (Billboarding in wireframe side
view)
5 A billboarded instance of size n is a pair (m, p), where m is a mesh
[ref: di.meshes] [13], and p is a buffer of n world space [ref:
di.coords.world] positions allocated on the GPU.
2.6.5 Types [id: di.instances.types]
1 In the r2 package, instances are instances of R2InstanceType [url:
apidocs/com/io7m/r2/core/R2InstanceType.html].
2.7 Render Targets [id: di.render-target]
2.7.1 Overview [id: di.render-target.overview]
1 A render target is a rectangular region of memory allocated on the GPU
that can accept the results of a rendering operation.
2.7.2 Types [id: di.render-target.types]
1 In the r2 package, render targets are instances of R2RenderTargetType
[url: apidocs/com/io7m/r2/core/R2RenderTargetType.html].
2.8 Shaders [id: di.shaders]
2.8.1 Overview [id: di.shaders.overview]
1 A shader is a small program that executes on the GPU and is used to
produce images. In the r2 package, shaders perform a variety of tasks and
the programmer is not always exposed to them directly. The primary shader
types to which the programmer is directly exposed are instance [ref:
di.shaders.instance] and light [ref: di.shaders.light] shaders.
2.8.2 Interface And Calling Protocol [id: di.shaders.interface]
1 Every shader in the r2 package has an associated Java class. Each class
may implement one of the interfaces that are themselves subtypes of the
R2ShaderType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderType.html] interface. Each
class is responsible for uploading parameters to the actual compiled GLSL
shader on the GPU. Certain parameters, such as view matrices, the current
size of the screen, etc, are only calculated during each rendering pass
and therefore will be supplied to the shader classes at more or less the
last possible moment. The calculated parameters are supplied via methods
defined on the R2ShaderType subinterfaces, and implementations of the
subinterfaces can rely on the methods being called in a very strict
predefined order. For example, instances of type
R2ShaderInstanceSingleUsableType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderInstanceSingleUsableType.html]
will receive calls in exactly this order:
2.8.2.2 R2ShaderInstanceSingleUsableType call order
───────────────────────────────────────────────────
0. First, onActivate will be called. It is the class's responsibility to
activate the GLSL shader at this point.
1. Then onReceiveViewValues will be called when the current
view-specific values have been calculated.
2. Now, for each material m that uses the current shader:
0. onReceiveMaterialValues will be called once.
1. For each instance i using that uses a material that uses the
current shader, onReceiveInstanceTransformValues will be called,
followed by onValidate.
3 The final onValidate call allows the shader to check that all of the
required method calls have actually been made by the caller, and the
method is permitted to throw R2ExceptionShaderValidationFailed [url:
apidocs/com/io7m/r2/core/R2ExceptionShaderValidationFailed.html] if the
caller makes a mistake at any point. The implicit promise is that callers
will call all of the methods in the correct order and the correct number
of times, and shaders are allowed to loudly complain if and when this
does not happen.
4 Of course, actually requiring the programmer to manually implement all of
the above for each new shader would be unreasonable and would just become
a new source of bugs. The r2 provides abstract shader implementations to
perform the run-time checks listed above without forcing the programmer
to implement them all manually. The R2AbstractInstanceShaderSingle [url:
apidocs/com/io7m/r2/core/shaders/abstracts/R2AbstractInstanceShaderSingle.html]
type, for example, implements the R2ShaderInstanceSingleUsableType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderInstanceSingleUsableType.html]
interface and provides a few abstract methods that the programmer
implements in order to upload parameters to the GPU. The abstract
implementation enforces the calling protocol.
5 The calling protocol described both ensures that all shader parameters
will be set and that the renderers themselves are insulated from the
interfaces of actual GLSL shaders. Failing to set parameters, attempting
to set parameters that no longer exist, or passing values of the wrong
types to GLSL shaders is a common source of bugs in OpenGL programs and
almost always results in either silent failure or corrupted visuals. The
r2 package takes care to ensure that mistakes of that type are difficult
to make.
2.8.3 Shader Modules [id: di.shaders.modules]
1 Although the GLSL shading language is anti-modular in the sense that it
has one large namespace, the r2 package attempts to relieve some of the
pain of shader management by delegating to the sombrero [url:
http://io7m.github.io/sombrero] package. The sombrero package provides a
preprocessor for shader code, allowing shader code to make use of
#include directives. It also provides a system for publishing and
importing modules full of shaders based internally on the standard Java
ServiceLoader [url:
https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html]
API. This allows users that want to write their own shaders to import
much of the re-usable shader code from the r2 package into their own
shaders without needing to do anything more than have the correct shader
jar on the Java classpath [14].
2 As a simple example, if the user writing custom shaders wants to take
advantage of the bilinear interpolation functions used in many r2
shaders, the following #include is sufficient:
2.8.3.3 Include
───────────────
#include
vec3 x = R2_bilinearInterpolate3(...);
4 The text com.io7m.r2.shaders.core is considered to be the module name,
and the R2Bilinear.h name refers to that file within the module. The
sombrero resolver maps the request to a concrete resource on the
filesystem or in a jar file and returns the content for inclusion.
5 The r2 package also provides an interface, the
R2ShaderPreprocessingEnvironmentType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderPreprocessingEnvironmentType.html]
type, that allows constants to be set that will be exposed to shaders
upon being preprocessed. Each shader stores an immutable snapshot of the
environment used to preprocess it after successful compilation.
2.8.4 Types [id: di.shaders.types]
1 In the r2 package, shaders are instances of R2ShaderType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderType.html].
2.9 Shaders: Instance [id: di.shaders.instance]
2.9.1 Overview [id: di.shaders.instance.overview]
1 An instance shader is a shader used to render the surfaces of instances
[ref: di.instances]. Depending on the context, this may mean rendering
the surface attributes of the instances into a geometry buffer [ref:
di.deferred.geom.gbuffer], forward rendering the instance directly to the
screen (or
other image) , rendering only the depth [ref:
di.shadows.variance] of the surface, or perhaps not producing any output
at all as shaders used simply for stencilling [ref: di.stencil] are
permitted to do. Instance shaders are most often exposed to the
programmer via materials [ref: di.shaders.instance.material].
2.9.2 Materials [id: di.shaders.instance.material]
1 A material is a pair (s, i, p) where p is a value of type m that
represents a set of shader parameters, s is a shader that takes
parameters of type m, and i is a unique identifier for the material.
Materials primarily exist to facilitate batching [ref:
di.deferred.geom.ordering]: By assigning each material a unique
identifier, the system can assume that two materials are the same if they
have the same identifier, without needing to perform a relatively
expensive structural equality comparison between the shaders and shader
parameters.
2.9.3 Provided Shaders [id: di.shaders.instance.provided]
1 Writing shaders is difficult. The programmer must be aware of an endless
series of pitfalls inherent in the OpenGL API and the shading language.
While the r2 package does allow users to write their own shaders, the
intention has always been to provide a small set of general purpose
shaders that cover the majority of the use cases in modern games and
simulations. The instance shaders provided by default are:
2.9.3.2 Provided instance shaders
─────────────────────────────────
┌──────────────────────────────────────────────────────────────────────────────────────┬─────────────────────┐
│ Shader │ Description │
├──────────────────────────────────────────────────────────────────────────────────────┼─────────────────────┤
│ R2SurfaceShaderBasicSingle [url: │ Basic textured │
│ apidocs/com/io7m/r2/core/shaders/provided/R2SurfaceShaderBasicSingle.html] │ surface with normal │
│ │ mapping, specular │
│ │ mapping, emission │
│ │ mapping, and │
│ │ conditional │
│ │ discarding based on │
│ │ alpha. │
│ │ │
│ │ │
├──────────────────────────────────────────────────────────────────────────────────────┼─────────────────────┤
│ R2SurfaceShaderBasicReflectiveSingle [url: │ Basic textured │
│ apidocs/com/io7m/r2/core/shaders/provided/R2SurfaceShaderBasicReflectiveSingle.html] │ surface with pseudo │
│ │ reflections from a │
│ │ cube map, normal │
│ │ mapping, specular │
│ │ mapping, emission │
│ │ mapping, and │
│ │ conditional │
│ │ discarding based on │
│ │ alpha. │
└──────────────────────────────────────────────────────────────────────────────────────┴─────────────────────┘
2.9.4 Types [id: di.shaders.instance.types]
1 In the r2 package, materials are instances of R2MaterialType [url:
apidocs/com/io7m/r2/core/R2MaterialType.html]. Geometry renderers
primarily consume instances that are associated with values of the
R2MaterialOpaqueSingleType [url:
apidocs/com/io7m/r2/core/R2MaterialOpaqueSingleType.html] and
R2MaterialOpaqueBatchedType [url:
apidocs/com/io7m/r2/core/R2MaterialOpaqueBatchedType.html] types.
Instance shaders are instances of the R2ShaderInstanceSingleType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderInstanceSingle.html] and
R2ShaderInstanceBatchedType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderInstanceBatched.html]
types.
2.10 Shaders: Light [id: di.shaders.light]
2.10.1 Overview [id: di.shaders.light.overview]
1 A light shader is a shader used to render the contributions of a light
source [ref: di.deferred.light]. Light shaders in the r2 package are only
used within the context of deferred rendering [ref: di.deferred].
2.10.2 Types [id: di.shaders.light.types]
1 Light shaders are instances of the R2ShaderLightSingleType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderLightSingleType.html]
type.
2.11 Stencils [id: di.stencil]
2.11.1 Overview [id: di.stencil.overview]
1 The stencil buffer enables per-pixel control over rendering. The r2
package uses the stencil buffer to implement several rendering techniques
internally, and also exposes limited control of the stencil buffer to
users of the renderer via the allow bit [ref: di.stencil.allow_bit].
2.11.2 Reserved Bits [id: di.stencil.reserved]
1 The current stencil buffer layout used by the r2 package is as follows:
2.11.2.2 Reserved bits
──────────────────────
[image: images/stencil_bits.png] (Reserved stencil bits.)
3 Bit 0 is used for light clip volumes [ref:
di.deferred.light.clip_volumes].
4 Bits 1-2 are reserved for future use.
5 Bits 3-6 are used for groups [ref: di.deferred.geom.group].
6 Bit 7 is the allow bit [ref: di.stencil.allow_bit].
2.11.3 Allow Bit [id: di.stencil.allow_bit]
1 The r2 package reserves a single bit in the current stencil buffer, known
as the allow bit. In all subsequent rendering operations, a pixel may
only be written if the corresponding allow bit in the stencil buffer is
true.
2 The stencil buffer allow bits are populated via the use of a stencil
renderer [url: apidocs/com/io7m/r2/core/R2StencilRendererType.html]. The
user specifies a series of instances [ref: di.instances] whose only
purpose is to either enable or disable the allow bit for each rendered
pixel. Users may specify whether instances are positive or negative.
Positive instances set the allow bit to true for each overlapped pixel,
and negative instances set the allow bit to false for each overlapped
pixel.
2.12 Lighting [id: di.lighting]
2.12.1 Overview [id: di.lighting.overview]
1 The following sections of documentation attempt to describe the theory
and implementation of lighting in the r2 package. All lighting in the
package is dynamic- there is no support for precomputed lighting and all
contributions from lights are recalculated every time a scene is
rendered. Lighting is configured by adding instances of R2LightType [url:
apidocs/com/io7m/r2/core/R2LightType.html] to a scene.
2.12.2 Diffuse/Specular Terms [id: di.lighting.diffuse-specular]
1 The light applied to a surface by a given light is divided into diffuse
and specular terms [15]. The actual light applied to a surface is
dependent upon the properties of the surface. Conceptually, the diffuse
and specular terms are multiplied by the final color of the surface and
summed. In practice, the materials applied to surfaces have control over
how light is actually applied to the surface. For example, materials may
include a specular map which is used to manipulate the specular term as
it is applied to the surface. Additionally, if a light supports
attenuation, then the diffuse and specular terms are scaled by the
attenuation factor prior to being applied.
2 The diffuse term is modelled by Lambertian reflectance [url:
http://en.wikipedia.org/wiki/Lambertian_reflectance]. Specifically, the
amount of diffuse light reflected from a surface is given by diffuse in
LightDiffuse.hs [url: haskell/LightDiffuse.hs]:
2.12.2.3 Diffuse term
─────────────────────
module LightDiffuse where
import qualified Color3
import qualified Direction
import qualified Normal
import qualified Spaces
import qualified Vector3f
diffuse :: Direction.T Spaces.Eye -> Normal.T -> Color3.T -> Float -> Vector3f.T
diffuse stl n light_color light_intensity =
let
factor = max 0.0 (Vector3f.dot3 stl n)
light_scaled = Vector3f.scale light_color light_intensity
in
Vector3f.scale light_scaled factor
4 Where stl is a unit length direction vector from the surface to the light
source, n is the surface normal vector, light_color is the light color,
and light_intensity is the light intensity. Informally, the algorithm
determines how much diffuse light should be reflected from a surface
based on how directly that surface points towards the light. When stl ==
n, Vector3f.dot3 stl n == 1.0, and therefore the light is reflected
exactly as received. When stl is perpendicular to n (such that
Vector3f.dot3 stl n == 0.0 ), no light is reflected at all. If the two
directions are greater than 90° perpendicular, the dot product is
negative, but the algorithm clamps negative values to 0.0 so the effect
is the same.
2.12.2.5 Diffuse light
──────────────────────
[image: images/directional_diffuse.png] (Diffuse light)
6 The specular term is modelled either by Phong [url:
http://en.wikipedia.org/wiki/Phong_reflection_model] or Blinn-Phong [url:
https://en.wikipedia.org/wiki/Blinn%E2%80%93Phong_shading_model]
reflection. The r2 package provides light shaders that provide both Phong
and Blinn-Phong specular lighting and the user may freely pick between
implementations. For the sake of simplicity, the rest of this
documentation assumes that Blinn-Phong shading is being used.
Specifically, the amount of specular light reflected from a surface is
given by specularBlinnPhong in LightSpecular.hs [url:
haskell/LightSpecular.hs]:
2.12.2.7 Specular Term
──────────────────────
module LightSpecular where
import qualified Color3
import qualified Direction
import qualified Normal
import qualified Reflection
import qualified Spaces
import qualified Specular
import qualified Vector3f
specularPhong :: Direction.T Spaces.Eye -> Direction.T Spaces.Eye -> Normal.T -> Color3.T -> Float -> Specular.T -> Vector3f.T
specularPhong stl view n light_color light_intensity (Specular.S surface_spec surface_exponent) =
let
reflection = Reflection.reflection view n
factor = (max 0.0 (Vector3f.dot3 reflection stl)) ** surface_exponent
light_raw = Vector3f.scale light_color light_intensity
light_scaled = Vector3f.scale light_raw factor
in
Vector3f.mult3 light_scaled surface_spec
specularBlinnPhong :: Direction.T Spaces.Eye -> Direction.T Spaces.Eye -> Normal.T -> Color3.T -> Float -> Specular.T -> Vector3f.T
specularBlinnPhong stl view n light_color light_intensity (Specular.S surface_spec surface_exponent) =
let
reflection = Reflection.reflection view n
factor = (max 0.0 (Vector3f.dot3 reflection stl)) ** surface_exponent
light_raw = Vector3f.scale light_color light_intensity
light_scaled = Vector3f.scale light_raw factor
in
Vector3f.mult3 light_scaled surface_spec
8 Where stl is a unit length direction vector from the surface to the light
source, view is a unit length direction vector from the observer to the
surface, n is the surface normal vector, light_color is the light color,
light_intensity is the light intensity, surface_exponent is the specular
exponent defined by the surface, and surface_spec is the surface
specularity factor.
9 The specular exponent is a value, ordinarily in the range [0, 255], that
controls how sharp the specular highlights appear on the surface. The
exponent is a property of the surface, as opposed to being a property of
the light. Low specular exponents result in soft and widely dispersed
specular highlights (giving the appearance of a rough surface), while
high specular exponents result in hard and focused highlights (giving the
appearance of a polished surface). As an example, three models lit with
progressively lower specular exponents from left to right ( 128, 32, and
8, respectively):
2.12.2.10 Specular exponents
────────────────────────────
[image: images/directional_specular_exponents.png] (Specular exponents)
2.12.3 Diffuse-Only Lights [id: di.lighting.diffuse-only]
1 Some lights have diffuse-only variants. Little explanation is required:
The specular [ref: di.lighting.specular] term is simply not calculated
and only the diffuse term is used.
2.12.4 Attenuation [id: di.lighting.attenuation]
1 Attenuation is the property of the influence of a given light on a
surface in inverse proportion to the distance from the light to the
surface. In other words, for lights that support attenuation, the further
a surface is from a light source, the less that surface will appear to be
lit by the light. For light types that support attenuation, an
attenuation factor is calculated based on a given inverse_maximum_range
(where the maximum_range is a light-type specific positive value that
represents the maximum possible range of influence for the light), a
configurable inverse falloff value, and the current distance between the
surface being lit and the light source. The attenuation factor is a value
in the range [0.0, 1.0], with 1.0 meaning "no attenuation" and 0.0
meaning "maximum attenuation". The resulting attenuation factor is
multiplied by the raw unattenuated light values produced for the light in
order to produce the illusion of distance attenuation. Specifically:
2.12.4.2 Attenuation
────────────────────
module Attenuation where
attenuation_from_inverses :: Float -> Float -> Float -> Float
attenuation_from_inverses inverse_maximum_range inverse_falloff distance =
max 0.0 (1.0 - (distance * inverse_maximum_range) ** inverse_falloff)
attenuation :: Float -> Float -> Float -> Float
attenuation maximum_range falloff distance =
attenuation_from_inverses (1.0 / maximum_range) (1.0 / falloff) distance
3 Given the above definitions, a number of observations can be made.
4 If falloff == 1, then the attenuation is linear over distance [16]:
2.12.4.5 Linear attenuation
───────────────────────────
[image: images/attenuation_linear.png] (Linear attenuation)
6 If maximum_range == 0, then the inverse range is undefined, and therefore
the results of lighting are undefined. The r2 package handles this case
by raising an exception when the light is created.
7 If falloff == 0, then the inverse falloff is undefined, and therefore the
results of lighting are undefined. The r2 package handles this case by
raising an exception when the light is created.
8 As falloff decreases towards 0.0, then the attenuation curve remains at
1.0 for increasingly higher distance values before falling sharply to
0.0:
2.12.4.9 Low falloff attenuation
────────────────────────────────
[image: images/attenuation_low_falloff.png] (Low falloff attenuation)
10 As falloff increases away from 0.0, then the attenuation curve decreases
more for lower distance values:
2.12.4.11 High falloff attenuation
──────────────────────────────────
[image: images/attenuation_high_falloff.png] (High falloff attenuation)
2.13 Lighting: Directional [id: di.lighting.directional]
2.13.1 Overview [id: di.lighting.directional.overview]
1 Directional lighting is the most trivial form of lighting provided by the
r2 package. A directional light is a light that emits parallel rays of
light in a given eye space direction. It has a color and an intensity,
but does not have an origin and therefore is not attenuated over
distance. It does not cause objects to cast shadows.
2.13.1.2 Directional lighting
─────────────────────────────
[image: images/directional_diagram.png] (Directional lighting)
2.13.2 Attenuation [id: di.lighting.directional.attenuation]
1 Directional lights do not have origins and cannot therefore be attenuated
over distance.
2.13.3 Application [id: di.lighting.directional.application]
1 The final light applied to the surface is given by directional
(Directional.hs) [url: haskell/Directional.hs], where sr, sg, sb are the
red, green, and blue channels, respectively, of the surface being lit.
Note that the surface-to-light vector stl is simply the negation of the
light direction.
2.13.3.2 Directional lighting (Application)
───────────────────────────────────────────
module Directional where
import qualified Color4
import qualified Direction
import qualified LightDirectional
import qualified LightDiffuse
import qualified LightSpecular
import qualified Normal
import qualified Position3
import qualified Spaces
import qualified Specular
import qualified Vector3f
import qualified Vector4f
directional :: Direction.T Spaces.Eye -> Normal.T -> Position3.T Spaces.Eye -> LightDirectional.T -> Specular.T -> Color4.T -> Vector3f.T
directional view n position light specular (Vector4f.V4 sr sg sb _) =
let
stl = Vector3f.normalize (Vector3f.negation position)
light_color = LightDirectional.color light
light_intensity = LightDirectional.intensity light
light_d = LightDiffuse.diffuse stl n light_color light_intensity
light_s = LightSpecular.specularBlinnPhong stl view n light_color light_intensity specular
lit_d = Vector3f.mult3 (Vector3f.V3 sr sg sb) light_d
lit_s = Vector3f.add3 lit_d light_s
in
lit_s
2.13.4 Types [id: di.lighting.directional.types]
1 Directional lights are represented in the r2 package by the
R2LightDirectionalScreenSingle [url:
apidocs/com/io7m/r2/core/R2LightDirectionalScreenSingle.html] type.
2.14 Lighting: Spherical [id: di.lighting.spherical]
2.14.1 Overview [id: di.lighting.spherical.overview]
1 A spherical light in the r2 package is a light that emits rays of light
in all directions from a given origin specified in eye space [ref:
di.coords.eye] up to a given maximum radius.
2.14.1.2 Spherical lighting
───────────────────────────
[image: images/spherical_diagram.png] (Spherical lighting)
3 The term spherical comes from the fact that the light has a defined
radius. Most rendering systems instead use point lights that specify
multiple attenuation constants to control how light is attenuated over
distance. The problem with this approach is that it requires solving a
quadratic equation to determine a minimum bounding sphere that can
contain the light. Essentially, the programmer/artist is forced to
determine "at which radius does the contribution from this light
effectively reach zero?". With spherical lights, the maximum radius is
declared up front, and a single falloff value is used to determine the
attenuation curve within that radius. This makes spherical lights more
intuitive to use: The programmer/artist simply places a sphere within the
scene and knows exactly from the radius which objects are lit by it. It
also means that bounding light volumes can be trivially constructed from
unit spheres by simply scaling those spheres by the light radius, when
performing deferred rendering [ref: di.deferred].
2.14.2 Attenuation [id: di.lighting.spherical.attenuation]
1 The light supports attenuation [ref: di.lighting.attenuation] using the
radius as the maximum range.
2.14.3 Application [id: di.lighting.spherical.application]
1 The final light applied to the surface is given by spherical
(Spherical.hs) [url: haskell/Spherical.hs], where sr, sg, sb are the red,
green, and blue channels, respectively, of the surface being lit. The
surface-to-light vector stl is calculated by normalizing the negation of
the difference between the the current eye space surface_position and the
eye space origin of the light.
2.14.3.2 Spherical lighting (Application)
─────────────────────────────────────────
module Spherical where
import qualified Attenuation
import qualified Color4
import qualified Direction
import qualified LightDiffuse
import qualified LightSpecular
import qualified LightSpherical
import qualified Normal
import qualified Position3
import qualified Specular
import qualified Spaces
import qualified Vector3f
import qualified Vector4f
spherical :: Direction.T Spaces.Eye -> Normal.T -> Position3.T Spaces.Eye -> LightSpherical.T -> Specular.T -> Color4.T -> Vector3f.T
spherical view n surface_position light specular (Vector4f.V4 sr sg sb _) =
let
position_diff = Position3.sub3 surface_position (LightSpherical.origin light)
stl = Vector3f.normalize (Vector3f.negation position_diff)
distance = Vector3f.magnitude (position_diff)
attenuation = Attenuation.attenuation (LightSpherical.radius light) (LightSpherical.falloff light) distance
light_color = LightSpherical.color light
light_intensity = LightSpherical.intensity light
light_d = LightDiffuse.diffuse stl n light_color light_intensity
light_s = LightSpecular.specularBlinnPhong stl view n light_color light_intensity specular
light_da = Vector3f.scale light_d attenuation
light_sa = Vector3f.scale light_s attenuation
lit_d = Vector3f.mult3 (Vector3f.V3 sr sg sb) light_da
lit_s = Vector3f.add3 lit_d light_sa
in
lit_s
2.14.4 Types [id: di.lighting.spherical.types]
1 Spherical lights are represented in the r2 package by the
R2LightSphericalSingle [url:
apidocs/com/io7m/r2/core/R2LightSphericalSingle.html] type.
2.15 Lighting: Projective [id: di.lighting.projective]
2.15.1 Overview [id: di.lighting.projective.overview]
1 A projective light in the r2 package is a light that projects a texture
onto the scene from a given origin specified in eye space [ref:
di.coords.eye] up to a given maximum radius. Projective lights are the
only types of lights in the r2 package that are able to project shadows.
2.15.1.2 Projective lighting
────────────────────────────
[image: images/projective.png] (Projective lighting)
2.15.1.3 Projective lighting (Texture)
──────────────────────────────────────
[image: images/sunflower.png] (Projective lighting (Texture)
2.15.2 Algorithm [id: di.lighting.projective.algorithm]
1 At a basic level, a projective light performs the same operations that
occur when an ordinary 3D position is projected onto the screen during
rendering. During normal rendering, a point p given in world space [ref:
di.coords.world] is transformed to eye space [ref: di.coords.eye] given
the current camera's view matrix, and is then transformed to clip space
[ref: di.coords.clip] using the current camera's projection matrix.
During rendering of a scene lit by a projective light, a given point q in
the scene is transformed back to world space given the current camera's
inverse view matrix, and is then transformed to eye space from the point
of view of the light (subsequently referred to as light eye space) using
the light's view matrix. Finally, q is transformed to clip space from the
point of view of the light (subsequently referred to as light clip space)
using the light's projection matrix. It should be noted (in order to
indicate that there is nothing unusual about the light's view or
projection matrices) that if the camera and light have the same position,
orientation, scale, and projection, then the resulting transformed values
of q and p are identical. The resulting transformed value of q is mapped
from the range [(-1, -1, -1), (1, 1, 1)] to [(0, 0, 0), (1, 1, 1)], and
the resulting coordinates are used to retrieve a texel from the 2D
texture associated with the light.
2 Intuitively, an ordinary perspective projection will cause the light to
appear to take the shape of a frustum:
2.15.2.3 Projective lighting (Frustum)
──────────────────────────────────────
[image: images/projective_frustum.png] (Projective lighting (Frustum)
4 There are two issues with the projective lighting algorithm that also
have to be solved: back projection [ref:
di.lighting.projective.back-projection] and clamping [ref:
di.lighting.projective.clamping].
2.15.3 Back projection [id: di.lighting.projective.back-projection]
1 The algorithm [ref: di.lighting.projective.algorithm] described above
will produce a so-called dual or back projection. In other words, the
texture will be projected along the view direction of the camera, but
will also be projected along the negative view direction [17]. The visual
result is that it appears that there are two projective lights in the
scene, oriented in opposite directions. As mentioned previously [ref:
di.coords.clip], given the typical projection matrix, the w component of
a given clip space position is the negation of the eye space z component.
Because it is assumed that the observer is looking towards the negative z
direction, all positions that are in front of the observer must have
positive w components. Therefore, if w is negative, then the position is
behind the observer. The standard fix for this problem is to check to see
if the w component of the light-clip space coordinate is negative, and
simply return a pure black color (indicating no light contribution)
rather than sampling from the projected texture.
2.15.4 Clamping [id: di.lighting.projective.clamping]
1 The algorithm [ref: di.lighting.projective.algorithm] described above
takes an arbitrary point in the scene and projects it from the point of
view of the light. There is no guarantee that the point actually falls
within the light's view frustum (although this is mitigated slightly by
the r2 package's use of light volumes for deferred rendering), and
therefore the calculated texture coordinates used to sample from the
projected texture are not guaranteed to be in the range [(0, 0), (1, 1)].
In order to get the intended visual effect, the texture used must be set
to clamp-to-edge and have black pixels on all of the edges of the texture
image, or clamp-to-border with a black border color. Failing to do this
can result in strange visual anomalies, as the texture will be
unexpectedly repeated or smeared across the area outside of the
intersection between the light volume and the receiving surface:
2.15.4.2 Projective lighting (Correct, clamped)
───────────────────────────────────────────────
[image: images/projective_clamped.png]
(Projective lighting (Correct, clamped)
2.15.4.3 Projective lighting (Incorrect, not clamped)
─────────────────────────────────────────────────────
[image: images/projective_not_clamped.png]
(Projective lighting (Incorrect, not clamped)
4 The r2 package will raise an exception if a non-clamped texture is
assigned to a projective light.
2.15.5 Attenuation [id: di.lighting.projective.attenuation]
1 The light supports attenuation [ref: di.lighting.attenuation] using the
maximum range taken from the projection.
2.15.6 Application [id: di.lighting.projective.application]
1 The final light applied to the surface is given by projective in
Projective.hs [url: haskell/Projective.hs], where sr, sg, sb are the red,
green, and blue channels, respectively, of the surface being lit. The
surface-to-light vector stl is calculated by normalizing the negation of
the difference between the the current eye space surface_position and the
eye space origin of the light.
2.15.6.2 Projective lighting (Application)
──────────────────────────────────────────
module Projective where
import qualified Attenuation
import qualified Color3
import qualified Color4
import qualified Direction
import qualified LightDiffuse
import qualified LightSpecular
import qualified LightProjective
import qualified Normal
import qualified Position3
import qualified Specular
import qualified Spaces
import qualified Vector3f
import qualified Vector4f
projective :: Direction.T Spaces.Eye -> Normal.T -> Position3.T Spaces.Eye -> LightProjective.T -> Specular.T -> Float -> Color3.T -> Color4.T -> Vector3f.T
projective view n surface_position light specular shadow texture (Vector4f.V4 sr sg sb _) =
let
position_diff = Position3.sub3 surface_position (LightProjective.origin light)
stl = Vector3f.normalize (Vector3f.negation position_diff)
distance = Vector3f.magnitude (position_diff)
attenuation_raw = Attenuation.attenuation (LightProjective.radius light) (LightProjective.falloff light) distance
attenuation = attenuation_raw * shadow
light_color = Vector3f.mult3 (LightProjective.color light) texture
light_intensity = LightProjective.intensity light
light_d = LightDiffuse.diffuse stl n light_color light_intensity
light_s = LightSpecular.specularBlinnPhong stl view n light_color light_intensity specular
light_da = Vector3f.scale light_d attenuation
light_sa = Vector3f.scale light_s attenuation
lit_d = Vector3f.mult3 (Vector3f.V3 sr sg sb) light_da
lit_s = Vector3f.add3 lit_d light_sa
in
lit_s
3 The given shadow factor is a value in the range [0, 1], where 0 indicates
that the lit point is fully in shadow for the current light, and 1
indicates that the lit point is not in shadow. This is calculated for
variance [ref: di.shadows.variance] shadows and is assumed to be 1 for
lights without shadows. As can be seen, a value of 0 has the effect of
fully attenuating the light.
4 The color denoted by texture is assumed to have been sampled from the
projected texture. Assuming the eye space position being shaded p, the
matrix to get from eye space to light-clip space is given by The final
light applied to the surface is given by projective_matrix in
ProjectiveMatrix.hs [url: haskell/ProjectiveMatrix.hs]:
2.15.6.5 Projective matrix
──────────────────────────
module ProjectiveMatrix where
import qualified Matrix4f
projective_matrix :: Matrix4f.T -> Matrix4f.T -> Matrix4f.T -> Matrix4f.T
projective_matrix camera_view light_view light_projection =
case Matrix4f.inverse camera_view of
Just cv -> Matrix4f.mult (Matrix4f.mult light_projection light_view) cv
Nothing -> undefined -- A view matrix is always invertible
2.16 Shadows [id: di.shadows]
2.16.1 Overview [id: di.shadows.overview]
1 Because the r2 package implements local illumination, it is necessary to
associate shadows with those light sources capable of projecting them
(currently only projective [ref: di.lighting.projective] lights). The r2
package currently only supports variance [ref: di.shadows.variance]
shadow mapping. So-called mapped shadows allow efficient per-pixel
shadows to be calculated with varying degrees of visual quality.
2.16.2 Shadow Geometry [id: di.shadows.shadow-geometry]
1 Because the system requires the programmer to explicitly and separately
state that an opaque instance is visible in the scene, and that an opaque
instance is casting a shadow, it becomes possible to effectively specify
different shadow geometry for a given instance. As an example, a very
complex and high resolution mesh may still have the silhouette of a
simple sphere, and therefore the user can separately add the high
resolution mesh to a scene as a visible instance, but add a low
resolution version of the mesh as an invisible shadow-casting instance
with the same transform [ref: di.transforms]. As a rather extreme
example, assuming a high resolution mesh m0 added to the scene as both a
visible instance and a shadow caster:
2.16.2.2 Visible and shadow casting (High)
──────────────────────────────────────────
[image: images/shadow_geo_0.png] (Visible and shadow casting (High)
3 A low resolution mesh m1 added to the scene as both a visible instance
and shadow caster:
2.16.2.4 Visible and shadow casting (Low)
─────────────────────────────────────────
[image: images/shadow_geo_1.png] (Visible and shadow casting (Low)
5 Now, with m1 added as only a shadow caster, and m0 added as only a
visible instance:
2.16.2.6 Visible and shadow casting (Low shadow, high visible)
──────────────────────────────────────────────────────────────
[image: images/shadow_geo_2.png]
(Visible and shadow casting (Low shadow, high visible)
7 Using lower resolution geometry for shadow casters can lead to efficiency
gains on systems where vertex processing is expensive.
2.17 Shadows: Variance Mapping [id: di.shadows.variance]
2.17.1 Overview [id: di.shadows.variance.overview]
1 Variance shadow mapping is a technique that can give attractive
soft-edged shadows. Using the same view and projection matrices used to
apply projective lights [ref: di.lighting.projective], a depth-variance
image of the current scene is rendered, and those stored depth
distribution values are used to determine the probability that a given
point in the scene is in shadow with respect to the current light.
2 The algorithm implemented in the r2 package is described in GPU Gems 3
[url: http://http.developer.nvidia.com/GPUGems3/gpugems3_ch08.html],
which is a set of improvements to the original variance shadow mapping
algorithm by William Donnelly and Andrew Lauritzen. The r2 package
implements all of the improvements to the algorithm except summed area
tables. The package also provides optional box blurring of shadows as
described in the chapter.
2.17.2 Algorithm [id: di.shadows.variance.algorithm]
1 Prior to actually rendering [ref: di.deferred] a scene, shadow maps are
generated for all shadow-projecting lights in the scene. A shadow map for
variance shadow mapping, for a light k, is a two-component red/green
image of all of the shadow casters [ref: di.shadows.shadow-geometry]
associated with k in the visible set. The image is produced by rendering
the instances from the point of view of k. The red channel of each pixel
in the image represents the logarithmic depth [ref: di.log_depth] of the
closest surface at that pixel, and the green channel represents the depth
squared (literally depth * depth ). For example:
2.17.2.2 Depth-variance image
─────────────────────────────
[image: images/depth_variance.png] (Depth-variance image)
3 Then, when actually applying lighting during rendering of the scene, a
given eye space [ref: di.coords.eye] position p is transformed to
light-clip space [ref: di.lighting.projective.algorithm] and then mapped
to the range [(0, 0, 0), (1, 1, 1)] in order to sample the depth and
depth squared values (d, ds) from the shadow map (as with sampling from a
projected texture with projective lighting).
4 As stated previously, the intent of variance shadow mapping is to
essentially calculate the probability that a given point is in shadow. A
one-tailed variant of Chebyshev's inequality [url:
https://en.wikipedia.org/wiki/Chebyshev%27s_inequality] is used to
calculate the upper bound u on the probability that, given (d, ds), a
given point with depth t is in shadow:
2.17.2.5 Chebyshev 0
────────────────────
module ShadowVarianceChebyshev0 where
chebyshev :: (Float, Float) -> Float -> Float
chebyshev (d, ds) t =
let p = if t <= d then 1.0 else 0.0
variance = ds - (d * d)
du = t - d
p_max = variance / (variance + (du * du))
in max p p_max
factor :: (Float, Float) -> Float -> Float
factor = chebyshev
6 One of the improvements suggested to the original variance shadow
algorithm is to clamp the minimum variance to some small value (the r2
package uses 0.00002 by default, but this is configurable on a per-shadow
basis). The equation above becomes:
2.17.2.7 Chebyshev 1
────────────────────
module ShadowVarianceChebyshev1 where
data T = T {
minimum_variance :: Float
} deriving (Eq, Show)
chebyshev :: (Float, Float) -> Float -> Float -> Float
chebyshev (d, ds) min_variance t =
let p = if t <= d then 1.0 else 0.0
variance = max (ds - (d * d)) min_variance
du = t - d
p_max = variance / (variance + (du * du))
in max p p_max
factor :: T -> (Float, Float) -> Float -> Float
factor shadow (d, ds) t =
chebyshev (d, ds) (minimum_variance shadow) t
8 The above is sufficient to give shadows that are roughly equivalent in
visual quality to basic shadow mapping with the added benefit of being
generally better behaved and with far fewer artifacts. However, the
algorithm can suffer from light bleeding, where the penumbrae of
overlapping shadows can be unexpectedly bright despite the fact that the
entire area should be in shadow. One of the suggested improvements to
reduce light bleeding is to modify the upper bound u such that all values
below a configurable threshold are mapped to zero, and values above the
threshold are rescaled to map them to the range [0, 1]. The original
article suggests a linear step function applied to u:
2.17.2.9 Chebyshev 2
────────────────────
module ShadowVarianceChebyshev2 where
data T = T {
minimum_variance :: Float,
bleed_reduction :: Float
} deriving (Eq, Show)
chebyshev :: (Float, Float) -> Float -> Float -> Float
chebyshev (d, ds) min_variance t =
let p = if t <= d then 1.0 else 0.0
variance = max (ds - (d * d)) min_variance
du = t - d
p_max = variance / (variance + (du * du))
in max p p_max
clamp :: Float -> (Float, Float) -> Float
clamp x (lower, upper) = max (min x upper) lower
linear_step :: Float -> Float -> Float -> Float
linear_step lower upper x = clamp ((x - lower) / (upper - lower)) (0.0, 1.0)
factor :: T -> (Float, Float) -> Float -> Float
factor shadow (d, ds) t =
let u = chebyshev (d, ds) (minimum_variance shadow) t in
linear_step (bleed_reduction shadow) 1.0 u
10 The amount of light bleed reduction is adjustable on a per-shadow basis.
11 To reduce problems involving numeric inaccuracy, the original article
suggests the use of 32-bit floating point textures in depth variance
maps. The r2 package allows 16-bit or 32-bit textures, configurable on a
per-shadow basis.
12 Finally, as mentioned previously, the r2 package allows both optional box
blurring and mipmap generation for shadow maps. Both blurring and
mipmapping can reduce aliasing artifacts, with the former also allowing
the edges of shadows to be significantly softened as a visual effect:
2.17.2.13 Depth-variance shadows (Minimal blur)
───────────────────────────────────────────────
[image: images/variance_0.png] (Depth-variance shadows (Minimal blur)
2.17.2.14 Depth-variance shadows (High blur)
────────────────────────────────────────────
[image: images/variance_1.png] (Depth-variance shadows (High blur)
2.17.3 Advantages [id: di.shadows.variance.advantages]
1 The main advantage of variance shadow mapping is that they can
essentially be thought of as much better behaved version of basic shadow
mapping that just happen to have built-in softening and filtering.
Variance shadows typically require far less in the way of scene-specific
tuning to get good results.
2.17.4 Disadvantages [id: di.shadows.variance.disadvantages]
1 One disadvantage of variance shadows is that for large shadow maps,
filtering quickly becomes a major bottleneck. On reasonably old hardware
such as the Radeon 4670 [url:
https://en.wikipedia.org/wiki/Radeon_HD_4670], one 8192x8192 shadow map
with two 16-bit components takes too long to filter to give a reliable 60
frames per second rendering rate. Shadow maps of this size are usually
used to simulate the influence of the sun over large outdoor scenes.
2.17.5 Types [id: di.shadows.variance.types]
1 Variance mapped shadows are represented by the R2ShadowDepthVarianceType
[url: apidocs/com/io7m/r1/core/R2ShadowDepthVarianceType.html] type, and
can be associated with projective lights [ref: di.lighting.projective].
2 Rendering of depth-variance images is handled by implementations of the
R2ShadowMapRendererType [url:
apidocs/com/io7m/r1/core/R2ShadowMapRendererType.html] type.
2.18 Deferred Rendering [id: di.deferred]
2.18.1 Overview [id: di.deferred.overview]
1 Deferred rendering is a rendering technique where all of the opaque
objects in a given scene are rendered into a series of buffers, and then
lighting is applied to those buffers in screen space [ref:
di.coords.screen]. This is in contrast to forward rendering, where all
lighting is applied to objects as they are rendered.
2 One major advantage of deferred rendering is a massive reduction in the
number of shaders required (traditional forward rendering requires s * l
shaders, where s is the number of different object surface types in the
scene, and l is the number of different light types). In contrast,
deferred rendering requires s + l shaders, because surface and lighting
shaders are applied separately.
3 Traditional forward rendering also suffers severe performance problems as
the number of lights in the scene increases, because it is necessary to
recompute all of the surface attributes of an object each time a light is
applied. In contrast, deferred rendering calculates all surface
attributes of all objects once, and then reuses them when lighting is
applied.
4 However, deferred renderers are usually incapable of rendering
translucent objects. The deferred renderer in the r2 package is no
exception, and a separate set of renderers are provided to render
translucent objects.
5 Due to the size of the subject, the deferred rendering infrastructure in
the r2 package is described in several sections. The rendering of opaque
geometry is described in the Geometry [ref: di.deferred.geom] section,
the subsequent lighting of that geometry is described in the Lighting
[ref: di.deferred.light] section. The details of the position
reconstruction algorithm, an algorithm utterly fundamental to deferred
rendering, is described in Position Reconstruction [ref:
di.deferred-position-recon].
2.19 Deferred Rendering: Geometry [id: di.deferred.geom]
2.19.1 Overview [id: di.deferred.geom.overview]
1 The first step in deferred rendering involves rendering all opaque
instances in the current scene to a geometry buffer [ref:
di.deferred.geom.gbuffer]. This populated geometry buffer is then
primarily used in later stages to calculate lighting [ref:
di.deferred.light], but can also be used to implement effects such as
screen-space ambient occlusion [ref: di.ssao] and emission [ref:
di.emission].
2 In the r2 package, the primary implementation of the deferred geometry
rendering algorithm is the R2GeometryRenderer [url:
apidocs/com/io7m/r2/core/R2GeometryRenderer.html] type.
2.19.2 Groups [id: di.deferred.geom.group]
1 Groups are a simple means to constrain the contributions of sets of
specific light sources to sets of specific rendered instances. Instances
and lights are assigned a group number in the range [1, 15]. If the
programmer does not explicitly assign a number, the number 1 is assigned
automatically. During rendering, the group number of each rendered
instance is written to the stencil buffer [ref: di.stencil]. Then, when
the light contribution is calculated for a light with group number n,
only those pixels that have a corresponding value of n in the stencil
buffer are allowed to be modified.
2.19.3 Geometry Buffer [id: di.deferred.geom.gbuffer]
1 A geometry buffer is a render target [ref: di.render-target] in which the
surface attributes of objects are stored prior to being combined with the
contents of a light buffer [ref: di.deferred.light.lbuffer] to produce a
lit image.
2 One of the main implementation issues in any deferred renderer is
deciding which surface attributes (such as position, albedo, normals,
etc) to store and which to reconstruct. The more attributes that are
stored, the less work is required during rendering to reconstruct those
values. However, storing more attributes requires a larger geometry
buffer and more memory bandwidth to actually populate that geometry
buffer during rendering. The r2 package leans towards having a more
compact geometry buffer and doing slightly more reconstruction work
during rendering.
2.19.3.3 Geometry Buffer
────────────────────────
[image: images/gbuffer.png] (Geometry Buffer)
4 The r2 package explicitly stores the albedo, normals, emission level, and
specular color of surfaces. Additionally, the depth buffer is sampled to
recover the depth of surfaces. The eye-space positions of surfaces are
recovered via an efficient position reconstruction [ref:
di.deferred-position-recon] algorithm which uses the current viewing
projection and logarithmic depth [ref: di.log_depth] value as input. In
order to reduce the amount of storage required, three-dimensional
eye-space normal vectors are stored compressed as two 16 half-precision
floating point components via a simple mapping [ref:
di.deferred.geom.normal-compression]. This means that only 32 bits are
required to store the vectors, and very little precision is lost. The
precise format of the geometry buffer is as follows:
2.19.3.5 Geometry Buffer Format
───────────────────────────────
[image: images/gbuffer_format_0.png] (Geometry Buffer Format)
6 The albedo_r, albedo_g, and albedo_b components correspond to the red,
green, and blue components of the surface, respectively. The emission
component refers to the surface emission level. The normal_x and normal_y
components correspond to the two components of the compressed surface
normal [ref: di.deferred.geom.normal-compression] vector. The specular_r,
specular_g, and specular_b components correspond to the red, green, and
blue components of the surface specularity. Surfaces that will not
receive specular highlights simply have 0 for each component. The
specular_e component holds the surface specular exponent divided by 256.
7 In the r2 package, geometry buffers are instances of R2GeometryBufferType
[url: apidocs/com/io7m/r2/core/R2GeometryBufferType.html].
2.19.4 Algorithm
1 An informal description of the geometry rendering algorithm as
implemented in the r2 package is as follows:
2.19.4.2 Geometry Rendering Overview
────────────────────────────────────
0. Set the current render target [ref: di.render-target] to a geometry
buffer b.
1. Enable writing to the depth and stencil buffers, and enable stencil
testing. Enable depth testing such that only pixels with a depth less
than or equal to the current depth are touched.
2. For each group [ref: di.deferred.geom.group] g:
0. Configure stencil testing such that only pixels with the allow
bit [ref: di.stencil.allow_bit] enabled are touched, and
configure stencil writing such that the index of g is recorded in
the stencil buffer.
1. For each instance o in g:
0. Render the surface albedo, eye space normals, specular color,
and emission level of o into b. Normal mapping [ref:
di.normal-mapping] is performed during rendering, and if o
does not have specular highlights, then a pure black
(zero intensity) specular color is written. Effects such as
environment mapping [ref: di.environment-mapping] are
considered to be part of the surface albedo and so are
performed in this step.
2.19.5 Ordering/Batching [id: di.deferred.geom.ordering]
1 Due to the use of depth testing, the geometry rendering algorithm is
effectively order independent: Instances can be rendered in any order and
the final image will always be the same. However, there are efficiency
advantages in rendering instances in a particular order. The most
efficient order of rendering is the one that minimizes internal OpenGL
state changes. NVIDIA's Beyond Porting [url:
http://media.steampowered.com/apps/steamdevdays/slides/beyondporting.pdf]
presentation gives the relative cost of OpenGL state changes, from most
expensive to least expensive, as [18]:
2.19.5.2 State changes
──────────────────────
0. Render target changes: 60,000/second
1. Program bindings: 300,000/second
2. Texture bindings: 1,500,000/second
3. Vertex format (exact cost unspecified)
4. UBO bindings (exact cost unspecified)
5. Vertex Bindings (exact cost unspecified)
6. Uniform Updates: 10,000,000/second
3 Therefore, it is beneficial to order rendering operations such that the
most expensive state changes happen the least frequently.
4 The R2SceneOpaquesType [url:
apidocs/com/io7m/r2/core/R2SceneOpaquesType.html] type provides a simple
interface that allows the programmer to specify instances without
worrying about ordering concerns. When all instances have been submitted,
they will be delivered to a given consumer
(typically a geometry renderer) via the opaquesExecute method in the
order that would be most efficient for rendering. Typically, this means
that instances are first batched by shader [ref: di.shaders.instance],
because switching programs is the second most expensive type of render
state change. The shader-batched instances are then batched by material
[ref: di.shaders.instance.material], in order to reduce the number of
uniform updates that need to occur per shader.
2.19.6 Normal Compression [id: di.deferred.geom.normal-compression]
1 The r2 package uses a Lambert azimuthal equal-area projection [url:
http://en.wikipedia.org/wiki/Lambert_azimuthal_equal-area_projection] to
store surface normal vectors in two components instead of three. This
makes use of the fact that normalized vectors represent points on the
unit sphere. The mapping from normal vectors to two-dimensional spheremap
coordinates is given by compress NormalCompress.hs [url:
haskell/NormalCompress.hs]:
2.19.6.2 Normal Compression
───────────────────────────
module NormalCompress where
import qualified Vector3f
import qualified Vector2f
import qualified Normal
compress :: Normal.T -> Vector2f.T
compress n =
let p = sqrt ((Vector3f.z n * 8.0) + 8.0)
x = (Vector3f.x n / p) + 0.5
y = (Vector3f.y n / p) + 0.5
in Vector2f.V2 x y
3 The mapping from two-dimensional spheremap coordinates to normal vectors
is given by decompress NormalDecompress.hs [url:
haskell/NormalDecompress.hs]:
2.19.6.4 Normal Decompression
─────────────────────────────
module NormalDecompress where
import qualified Vector3f
import qualified Vector2f
import qualified Normal
decompress :: Vector2f.T -> Normal.T
decompress v =
let fn = Vector2f.V2 ((Vector2f.x v * 4.0) - 2.0) ((Vector2f.y v * 4.0) - 2.0)
f = Vector2f.dot2 fn fn
g = sqrt (1.0 - (f / 4.0))
x = (Vector2f.x fn) * g
y = (Vector2f.y fn) * g
z = 1.0 - (f / 2.0)
in Vector3f.V3 x y z
2.20 Deferred Rendering: Lighting [id: di.deferred.light]
2.20.1 Overview [id: di.deferred.light.overview]
1 The second step in deferred rendering involves rendering the light
contributions of all light sources within a scene to a light buffer [ref:
di.deferred.light.lbuffer]. The rendering algorithm requires sampling
from a populated geometry buffer [ref: di.deferred.geom.gbuffer].
2.20.2 Light Buffer [id: di.deferred.light.lbuffer]
1 A light buffer is a render target [ref: di.render-target] in which the
light contributions of all light sources are summed in preparation for
being combined with the surface albedo of a geometry buffer [ref:
di.deferred.geom.gbuffer] to produce a lit image.
2 A light buffer consists of a 32-bit RGBA diffuse image and a 32-bit RGBA
specular image. Currently, the alpha channels of both images are unused
and exist solely because OpenGL 3.3 does not provide a color-renderable
24-bit RGB format.
3 The r2 package offers the ability to disable specular lighting [ref:
di.lighting.specular] entirely if it is not needed, and so light buffer
implementations provide the ability to avoid allocating an image for
specular contributions if they will not be calculated.
4 In the r2 package, light buffers are instances of R2LightBufferType [url:
apidocs/com/io7m/r2/core/R2LightBufferType.html].
2.20.3 Light Clip Volumes [id: di.deferred.light.clip_volumes]
1 A light clip volume is a means of constraining the contributions of
groups of light sources to a provided volume.
2 Because, like most renderers, the r2 package implements so-called local
illumination, lights that do not have explicit shadow mapping enabled are
able to bleed through solid objects:
2.20.3.3 Local Light Bleed
──────────────────────────
[image: images/lightbleed_noclip.png] (Local light bleeding.)
4 Enabling shadow mapping for every single light source would be
prohibitively expensive [19], but for some scenes, acceptable results can
be achieved by simply preventing the light source from affecting pixels
outside of a given clip volume.
2.20.3.5 Local Light Clipped
────────────────────────────
[image: images/lightbleed_clip.png] (Local light clipped to a volume.)
6 The technique is implemented using the stencil buffer [ref: di.stencil],
using a single light clip volume bit.
2.20.3.7 Algorithm
──────────────────
0. Disable depth writing, and enable depth testing using the standard
less-than-or-equal-to depth function is used.
1. For each light clip volume v:
0. Clear the light clip volume bit in the stencil buffer.
1. Configure stencil testing such that the stencil test always
passes.
2. Configure stencil writing such that:
• Only the light clip volume bit can be written.
• Pixels that fail the depth test will invert the value of the
light clip volume bit (GL_INVERT).
• Pixels that pass the depth test leave the value of the light
clip volume bit untouched.
• Pixels that pass the stencil test leave the value of the light
clip volume bit untouched.
3. Render both the front and back faces of v.
4. Configure stencil testing such that only those pixels with both
the allow bit [ref: di.stencil.allow_bit] and light clip volume
bit set will be touched.
5. Render all of the light sources associated with v.
8 The reason the algorithm works can be inferred from the following
diagram:
2.20.3.9 Stencil Test Diagram
─────────────────────────────
[image: images/light_clip_volume_diagram.png] (Stencil test diagram)
10 In the diagram, the grey polygons represent the already-rendered depths
of the scene geometry [20]. If a point is inside or behind (from the
perspective of the observer) one of the polygons, then the depth of the
point is considered to be greater than the scene geometry.
11 In the diagram, when rendering the front face of the light volume at
point P0, the depth of the light volume face at P0 is less than the
current scene depth, and so the depth test succeeds and the light clip
volume bit is not touched. When rendering the back face of the light
volume at point P1, the depth of the light volume face at P1 is greater
than the current scene depth so the depth test fails, and the light clip
volume bit is inverted, setting it to true. This means that the scene
geometry along that view ray is inside the light clip volume.
12 In the diagram, when rendering the front face of the light volume at
point P2, the depth of the light volume face at P2 is greater than the
current scene depth, and so the depth test fails and the light clip
volume bit is inverted, setting it to true. When rendering the back face
of the light volume at point P3, the depth of the light volume face at P3
is greater than the current scene depth, so the depth test fails and the
light clip volume bit is inverted again, setting it to false. This means
that the scene geometry along that view ray is outside the light clip
volume.
13 In the diagram, when rendering the front face of the light volume at
point P4, the depth of the light volume face at P4 is less than the
current scene depth, and so the depth test succeeds and the light clip
volume bit is not touched. When rendering the back face of the light
volume at point P5, the depth of the light volume face at P5 is less than
the current scene depth, and so the depth test succeeds and the light
clip volume bit is not touched. Because the light clip volume bit is
false by default and is not modified, this results in the scene geometry
along that view ray being considered to be outside the light clip volume.
14 Given the initial depth buffer from an example scene:
2.20.3.15 Depth Buffer
──────────────────────
[image: images/light_clip_volume_depth.png] (Scene depth buffer.)
16 The stencil buffer for the initial scene has all of the geometry with the
allow bit [ref: di.stencil.allow_bit] set:
2.20.3.17 Stencil Buffer (Initial)
──────────────────────────────────
[image: images/light_clip_volume_stencil_before.png] (Scene stencil
buffer (initial) .)
18 After rendering a cuboid-shaped light volume that is intended to
constrain the contributions of a light source to a single area, all
pixels that fell within the clip volume have the light clip volume bit
set:
2.20.3.19 Stencil Buffer (Result)
─────────────────────────────────
[image: images/light_clip_volume_stencil_after.png] (Scene stencil buffer
(result) .)
20 Then, after rendering the light contribution of the constrainted light,
the light contribution becomes:
2.20.3.21 Light Buffer (Result)
───────────────────────────────
[image: images/light_clip_volume_diffuse.png] (Scene light buffer
(result) .)
2.20.4 No L-Buffer [id: di.deferred.light.no_lbuffer]
1 The r2 package also provides basic support for rendering lit images
directly without the use of an intermediate light buffer. This can save
greatly on memory bandwidth if no intermediate processing of light
buffers is required. In order to achieve this, light shaders must be
preprocessed [ref: di.shaders.modules] such that the output of the
generated code is a lit image rather than simply the light contribution.
Doing this is simple: Simply set R2_LIGHT_SHADER_OUTPUT_TARGET_DEFINE
[url:
apidocs/com/io7m/r2/shaders/core/R2LightShaderDefines.html#R2_LIGHT_SHADER_OUTPUT_TARGET_DEFINE]
to R2_LIGHT_SHADER_OUTPUT_TARGET_IBUFFER [url:
apidocs/com/io7m/r2/shaders/core/R2LightShaderDefines.html#R2_LIGHT_SHADER_OUTPUT_TARGET_IBUFFER]
in the shading environment [ref: di.shaders.modules] prior to compiling
any light shaders. The r2 renderer implementations perform simple
run-time checks to ensure that light shaders have been compiled to
support the current output type, so the programmer will be notified if
they try to render directly to an image but fail to make the above
configuration change.
2.20.5 Types [id: di.deferred.light.types]
1 In the r2 package, the primary implementation of the deferred light
rendering algorithm is the R2LightRenderer [url:
apidocs/com/io7m/r2/core/R2LightRenderer.html] type.
2.21 Deferred Rendering: Position Reconstruction [id: di.deferred-position-recon]
2.21.1 Overview [id: di.deferred-position-recon.overview]
1 Applying lighting during deferred rendering is primarily a screen space
[ref: di.coords.screen] technique. When the visible opaque objects have
been rendered into the geometry buffer [ref: di.deferred.geom.gbuffer],
the original eye space [ref: di.coords.eye] positions of all of the
surfaces that resulted in visible fragments in the scene are lost (unless
explicitly saved into the geometry buffer). However, given the knowledge
of the projection that was used to render the scene (such as perspective
or orthographic), it's possible to reconstruct the original eye space
position of the surfaces that produced each of the fragments in the
geometry buffer.
2 Specifically then, for each fragment f in the geometry buffer for which
lighting is being applied, a position reconstruction algorithm attempts
to reconstruct surface_eye - the eye space position of the surface that
produced f - using the screen space position of the current light volume
fragment position = (screen_x, screen_y) and some form of depth value
(such as the screen space depth of f ).
3 Position reconstruction is a fundamental technique in deferred rendering,
and there are a practically unlimited number of ways to reconstruct eye
space positions for fragments, each with various advantages and
disadvantages. Some rendering systems actually store the eye space
position of each fragment in the geometry buffer, meaning that
reconstructing positions means simply reading a value directly from a
texture. Some systems store only a normalized eye space depth value in a
separate texture: The first step of most position reconstruction
algorithms is to compute the original eye space Z value of a fragment, so
having this value computed during the population of the geometry buffer
reduces the work performed later. Storing an entire eye space position
into the geometry buffer is obviously the simplest and requires the least
reconstruction work later on, but is costly in terms of memory bandwidth:
Storing a full eye space position requires an extra 4 * 4 = 16 bytes of
storage per fragment (four 32-bit floating point values). As screen
resolutions increase, the costs can be prohibitive. Storing a normalized
depth value requires only a single 32-bit floating point value per
fragment but even this can be too much on less capable hardware. Some
algorithms take advantage of the fact that most projections used to
render scenes are perspective projections. Some naive algorithms use the
full inverse of the current projection matrix to reconstruct eye space
positions having already calculated clip space [ref: di.coords.clip]
positions.
4 The algorithm that the r2 package uses for position reconstruction is
generalized to handle both orthographic and perspective projections, and
uses only the existing logarithmic depth values [ref: di.log_depth] that
were written to the depth buffer during scene rendering. This keeps the
geometry buffer compact, and memory bandwidth requirements comparatively
low. The algorithm works with symmetric and asymmetric viewing frustums,
but will only work with near and far planes that are parallel to the
screen.
5 The algorithm works in two steps: Firstly, the original eye space Z [ref:
di.deferred-position-recon.eye-space-z] value of the fragment in question
is recovered, and then this Z value is used to recover the full eye space
position [ref: di.deferred-position-recon.eye-space].
2.21.2 Recovering Eye space Z [id: di.deferred-position-recon.eye-space-z]
1 During rendering of arbitrary scenes, vertices specified in object space
[ref: di.coords.object] are transformed to eye space, and the eye space
coordinates are transformed to clip space [ref: di.coords.clip] with a
projection matrix. The resulting 4D clip space coordinates are divided by
their own w components, resulting in normalized-device space [ref:
di.coords.ndevice] coordinates. These normalized-device space coordinates
are then transformed to screen space [ref: di.coords.screen] by
multiplying by the current viewport transform. The transitions from clip
space to screen space are handled automatically by the graphics hardware.
2 The first step required is to recover the original eye space Z value of
f. This involves sampling a depth value from the current depth buffer.
Sampling from the depth buffer is achieved as with any other texture: A
particular texel is addressed by using coordinates in the range
[(0, 0), (1, 1)]. The r2 package currently assumes that the size of the
viewport is the same as that of the framebuffer (width, height) and that
the bottom left corner of the viewport is positioned at (0, 0) in screen
space. Given the assumption on the position and size of the viewport, and
assuming that the screen space position of the current light volume
fragment being shaded is position = (screen_x, screen_y), the texture
coordinates (screen_uv_x, screen_uv_y) used to access the current depth
value are given by:
2.21.2.3 Screen to texture
──────────────────────────
module ScreenToTexture where
import qualified Vector2f
screen_to_texture :: Vector2f.T -> Float -> Float -> Vector2f.T
screen_to_texture position width height =
let u = (Vector2f.x position) / width
v = (Vector2f.y position) / height
in Vector2f.V2 u v
4 Intuitively, (screen_uv_x, screen_uv_y) = (0, 0) when the current screen
space position is the bottom-left corner of the screen,
(screen_uv_x, screen_uv_y) = (1, 1) when the current screen space
position is the top-right corner of the screen, and
(screen_uv_x, screen_uv_y) = (0.5, 0.5) when the current screen space
position is the exact center of the screen.
5 Originally, the spiritual ancestor of the r2 package, r1, used a standard
depth buffer and so recovering the eye space Z value required a slightly
different method compared to the steps required for the logarithmic depth
encoding [ref: di.log_depth] that the r2 package uses. For historical
reasons and for completeness, the method to reconstruct an eye space Z
value from a traditional screen space depth value is given in the section
on screen space depth encoding [ref:
di.deferred-position-recon.eye-space-z.screen-space-encoding].
2.21.3 Recovering Eye space Z (Logarithmic depth encoding) [id: di.deferred-position-recon.eye-space-z.log-depth-encoding]
1 The r2 package uses a logarithmic depth buffer [ref: di.log_depth]. Depth
values sampled from any depth buffer produced by the package can be
transformed to a negated eye space Z value by with a simple decoding
equation [ref: di.log_depth.encoding].
2.21.4 Recovering Eye space Z (Screen space depth encoding) [id: di.deferred-position-recon.eye-space-z.screen-space-encoding]
1 Note: This section is for completeness and historical interest. Please
skip ahead to the section on eye space position reconstruction [ref:
di.deferred-position-recon.eye-space] if you are not interested.
2 Assuming a screen space depth value screen_depth sampled from the depth
buffer at (screen_uv_x, screen_uv_y), it's now necessary to transform the
depth value back into normalized-device space. In OpenGL, screen space
depth values are in the range [0, 1] by default, with 0 representing the
near plane and 1 representing the far plane. However, in OpenGL,
normalized-device space coordinates are in the range
[(-1, -1, -1), (1, 1, 1)]. The transformation from screen space to
normalized-device space is given by:
2.21.4.3 Screen space depth to NDC Z
────────────────────────────────────
module ScreenDepthToNDC where
screen_depth_to_ndc :: Float -> Float
screen_depth_to_ndc screen_depth =
(screen_depth * 2.0) - 1.0
4 In order to understand how to calculate the eye space depth value from
the resulting NDC Z value ndc_z = screen_depth_to_ndc screen_depth, it's
necessary to understand how the normalized-device coordinates of f were
derived in the first place. Given a standard 4x4 projection matrix m and
an eye space position eye, clip space coordinates are calculated by
Matrix4x4f.mult_v m eye. This means that the z component of the resulting
clip space coordinates is given by:
2.21.4.5 Clip space Z Long (Diagram)
────────────────────────────────────
[image: images/matrix_clip_z_long.png] (Clip space Z Long (Diagram)
2.21.4.6 Clip space Z Long
──────────────────────────
module ClipSpaceZLong where
import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;
clip_z_long :: M4x4.T -> V4.T -> Float
clip_z_long m eye =
let
m20 = M4x4.row_column m (2, 0)
m21 = M4x4.row_column m (2, 1)
m22 = M4x4.row_column m (2, 2)
m23 = M4x4.row_column m (2, 3)
k0 = (V4.x eye) * m20
k1 = (V4.y eye) * m21
k2 = (V4.z eye) * m22
k3 = (V4.w eye) * m23
in
k0 + k1 + k2 + k3
7 Similarly, the w component of the resulting clip space coordinates is
given by:
2.21.4.8 Clip space W Long (Diagram)
────────────────────────────────────
[image: images/matrix_clip_w_long.png] (Clip space W Long (Diagram)
2.21.4.9 Clip space W Long
──────────────────────────
module ClipSpaceWLong where
import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;
clip_w_long :: M4x4.T -> V4.T -> Float
clip_w_long m eye =
let
m30 = M4x4.row_column m (3, 0)
m31 = M4x4.row_column m (3, 1)
m32 = M4x4.row_column m (3, 2)
m33 = M4x4.row_column m (3, 3)
k0 = (V4.x eye) * m30
k1 = (V4.y eye) * m31
k2 = (V4.z eye) * m32
k3 = (V4.w eye) * m33
in
k0 + k1 + k2 + k3
10 However, in the perspective and orthographic projections provided by the
r2 package, Matrix4x4f.row_column m (2, 0) == 0,
Matrix4x4f.row_column m (2, 1) == 0, Matrix4x4f.row_column m (3, 0) == 0,
and Matrix4x4f.row_column m (3, 1) == 0. Additionally, the w component of
all eye space coordinates is 1. With these assumptions, the previous
definitions simplify to:
2.21.4.11 Clip space Z Simple (Diagram)
───────────────────────────────────────
[image: images/matrix_clip_z_simple.png] (Clip space Z Simple (Diagram)
2.21.4.12 Clip space Z Simple
─────────────────────────────
module ClipSpaceZSimple where
import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;
clip_z_simple :: M4x4.T -> V4.T -> Float
clip_z_simple m eye =
let
m22 = M4x4.row_column m (2, 2)
m23 = M4x4.row_column m (2, 3)
in
((V4.z eye) * m22) + m23
2.21.4.13 Clip space W Simple (Diagram)
───────────────────────────────────────
[image: images/matrix_clip_w_simple.png] (Clip space W Simple (Diagram)
2.21.4.14 Clip space W Simple
─────────────────────────────
module ClipSpaceWSimple where
import qualified Matrix4f as M4x4;
import qualified Vector4f as V4;
clip_w_simple :: M4x4.T -> V4.T -> Float
clip_w_simple m eye =
let
m32 = M4x4.row_column m (3, 2)
m33 = M4x4.row_column m (3, 3)
in
((V4.z eye) * m32) + m33
15 It should be noted that for perspective matrices in the r2 package,
Matrix4x4f.row_column m (3, 2) == -1 and
Matrix4x4f.row_column m (3, 3) == 0:
2.21.4.16 Clip space W Simple (Perspective, Diagram)
────────────────────────────────────────────────────
[image: images/matrix_clip_w_simple_perspective.png]
(Clip space W Simple (Perspective, Diagram)
17 This means that the w component of the resulting clip space coordinates
is equal to the negated (and therefore positive) eye space z of the
original coordinates.
18 For orthographic projections in the r2 package,
Matrix4x4f.row_column m (3, 2) == 0 and
Matrix4x4f.row_column m (3, 3) == 1:
2.21.4.19 Clip space W Simple (Orthographic, Diagram)
─────────────────────────────────────────────────────
[image: images/matrix_clip_w_simple_orthographic.png]
(Clip space W Simple (Orthographic, Diagram)
20 This means that the w component of the resulting clip space coordinates
is always equal to 1.
21 As stated previously, normalized-device space coordinates are calculated
by dividing a set of clip space coordinates by their own w component. So,
given clip_z = ClipSpaceZSimple.clip_z_simple m eye and clip_w =
ClipSpaceWSimple.clip_w_simple m eye for some arbitrary projection matrix
m and eye space position eye, the normalized-device space Z coordinate is
given by ndc_z = clip_z / clip_w. Rearranging the definitions of clip_z
and clip_w algebraically yields an equation that takes an arbitrary
projection matrix m and a normalized-device space Z value ndc_z and
returns an eye space Z value:
2.21.4.22 Eye space Z
─────────────────────
module EyeSpaceZ where
import qualified Matrix4f as M4x4;
eye_z :: M4x4.T -> Float -> Float
eye_z m ndc_z =
let
m22 = M4x4.row_column m (2, 2)
m23 = M4x4.row_column m (2, 3)
m32 = M4x4.row_column m (3, 2)
m33 = M4x4.row_column m (3, 3)
a = (ndc_z * m33) - m32
b = (ndc_z * m23) - m22
in
- (a / b)
2.21.5 Recovering Eye space Position [id: di.deferred-position-recon.eye-space]
1 Given that the eye space Z value is known, it's now necessary to
reconstruct the full eye space position surface_eye of the surface that
resulted in f.
2 When the current projection is a perspective projection, there is
conceptually a ray passing through the near clipping plane ( near) from
the origin, oriented towards the eye space position ( eye) of f:
2.21.5.3 Perspective projection (Diagram)
─────────────────────────────────────────
[image: images/reconstruction_view_perspective.png]
(Perspective projection (Diagram)
4 When the current projection is an orthographic projection, the ray is
always perpendicular to the clipping planes and is offset by a certain
amount ( q) on the X and Y axes:
2.21.5.5 Orthographic projection (Diagram)
──────────────────────────────────────────
[image: images/reconstruction_view_ortho.png]
(Orthographic projection (Diagram)
6 Assuming ray = Vector3f.V3 ray_x ray_y 1.0, the eye space position of f
is given by surface_eye = Vector3f.add3 q (Vector3f.scale ray eye_z). In
the case of perspective projections, q = Vector3f.V3 0.0 0.0 0.0. The q
term is sometimes referred to as the origin (because q is the origin of
the view ray), but that terminology is not used here in order to avoid
confusion between the ray origin and the eye space coordinate system
origin. It's therefore necessary to calculate q and ray in order to
reconstruct the full eye space position of the fragment. The way this is
achieved in the r2 package is to calculate q and ray for each of the
viewing frustum corners [21] and then bilinearly interpolate between the
calculated values during rendering based on screen_uv_x and screen_uv_y.
7 As stated previously, normalized-device space coordinates are in the
range [(-1, -1, -1), (1, 1, 1)]. Stating each of the eight corners of the
cube that defines normalized-device space as 4D homogeneous coordinates
[22] yields the following values:
2.21.5.8 Normalized-device space corners
────────────────────────────────────────
module NDCCorners where
import qualified Vector4f as V4
near_x0y0 :: V4.T
near_x0y0 = V4.V4 (-1.0) (-1.0) (-1.0) 1.0
near_x1y0 :: V4.T
near_x1y0 = V4.V4 1.0 (-1.0) (-1.0) 1.0
near_x0y1 :: V4.T
near_x0y1 = V4.V4 (-1.0) 1.0 (-1.0) 1.0
near_x1y1 :: V4.T
near_x1y1 = V4.V4 1.0 1.0 (-1.0) 1.0
far_x0y0 :: V4.T
far_x0y0 = V4.V4 (-1.0) (-1.0) 1.0 1.0
far_x1y0 :: V4.T
far_x1y0 = V4.V4 1.0 (-1.0) 1.0 1.0
far_x0y1 :: V4.T
far_x0y1 = V4.V4 (-1.0) 1.0 1.0 1.0
far_x1y1 :: V4.T
far_x1y1 = V4.V4 1.0 1.0 1.0 1.0
9 Then, for the four pairs of near/far corners ((near_x0y0, far_x0y0),
(near_x1y0, far_x1y0), (near_x0y1, far_x0y1), (near_x1y1, far_x1y1)), a q
and ray value is calculated. The ray_and_q function describes the
calculation for a given pair of near/far corners:
2.21.5.10 Ray and Q calculation (Single)
────────────────────────────────────────
module RayAndQ where
import qualified Matrix4f as M4x4
import qualified Vector4f as V4
-- | Calculate @(ray, q)@ for the given inverse projection matrix and frustum corners
ray_and_q :: M4x4.T -> (V4.T, V4.T) -> (V4.T, V4.T)
ray_and_q inverse_m (near, far) =
let
-- Unproject the NDC coordinates to eye-space
near_hom = M4x4.mult_v inverse_m near
near_eye = V4.div_s near_hom (V4.w near_hom)
far_hom = M4x4.mult_v inverse_m far
far_eye = V4.div_s far_hom (V4.w far_hom)
-- Calculate a ray with ray.z == 1.0
ray_initial = V4.sub4 far_eye near_eye
ray = V4.div_s ray_initial (V4.z ray_initial)
-- Subtract the scaled ray from the near corner to calculate q
q = V4.sub4 near_eye (V4.scale ray (V4.z near_eye))
in
(ray, q)
11 The function takes a matrix representing the inverse of the current
projection matrix, and "unprojects" the given near and far frustum
corners from normalized-device space to eye space. The desired ray value
for the pair of corners is simply the vector that results from
subtracting the near corner from the far corner, divided by its own z
component. The desired q value is the vector that results from
subtracting ray scaled by the z component of the near corner, from the
near corner.
12 Note: The function calculates ray in eye space, but the resulting value
will have a non-negative z component. The reason for this is that the
resulting ray will be multiplied by the calculated eye space Z value
[ref: di.deferred-position-recon.eye-space-z] [23] to produce an eye
space position. If the z component of ray was negative, the resulting
position would have a positive z component.
13 Calculating the ray and q value for each of the pairs of corners is
straightforward:
2.21.5.14 Ray and Q calculation (All)
─────────────────────────────────────
module RayAndQAll where
import qualified NDCCorners
import qualified RayAndQ
import qualified Matrix4f as M4x4
import qualified Vector4f as V4
data T = T {
q_x0y0 :: V4.T,
q_x1y0 :: V4.T,
q_x0y1 :: V4.T,
q_x1y1 :: V4.T,
ray_x0y0 :: V4.T,
ray_x1y0 :: V4.T,
ray_x0y1 :: V4.T,
ray_x1y1 :: V4.T
} deriving (Eq, Ord, Show)
-- | Calculate all rays and qs for the four pairs of near/far frustum corners
calculate :: M4x4.T -> T
calculate inverse_m =
let
(x0y0_ray, x0y0_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x0y0, NDCCorners.far_x0y0)
(x1y0_ray, x1y0_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x1y0, NDCCorners.far_x1y0)
(x0y1_ray, x0y1_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x0y1, NDCCorners.far_x0y1)
(x1y1_ray, x1y1_q) = RayAndQ.ray_and_q inverse_m (NDCCorners.near_x1y1, NDCCorners.far_x1y1)
in
T {
q_x0y0 = x0y0_q,
q_x1y0 = x1y0_q,
q_x0y1 = x0y1_q,
q_x1y1 = x1y1_q,
ray_x0y0 = x0y0_ray,
ray_x1y0 = x1y0_ray,
ray_x0y1 = x0y1_ray,
ray_x1y1 = x1y1_ray
}
15 Then, by reusing the position = (screen_uv_x, screen_uv_y) values
calculated during the initial eye space Z [ref:
di.deferred-position-recon.eye-space-z.initial] calculation, determining
ray and q for the current fragment involves simply bilinearly
interpolating between the precalculated values above. Bilinear
interpolation between four vectors is defined as:
2.21.5.16 Bilinear interpolation (Vector4f)
───────────────────────────────────────────
module Bilinear4 where
import qualified Vector2f as V2
import qualified Vector4f as V4
interpolate :: (V4.T, V4.T, V4.T, V4.T) -> V2.T -> V4.T
interpolate (x0y0, x1y0, x0y1, x1y1) position =
let u0 = V4.interpolate x0y0 (V2.x position) x1y0
u1 = V4.interpolate x0y1 (V2.x position) x1y1
in V4.interpolate u0 (V2.y position) u1
17 Finally, now that all of the required components are known, the eye space
position surface_eye of f is calculated as
surface_eye = Vector3f.add3 q (Vector3f.scale ray eye_z).
2.21.6 Implementation [id: di.deferred-position-recon.implementation]
1 In the r2 package, the R2ViewRays [url:
apidocs/com/io7m/r2/core/R2ViewRays.html] class precalculates the rays
and q values [ref: di.deferred-position-recon.eye-space.rays_and_qs] for
each of the current frustum corners, and the results of which are cached
and re-used based on the current projection each time the scene is
rendered.
2 The actual position reconstruction is performed in a fragment shader,
producing an eye space Z value using the GLSL functions in R2LogDepth.h
[url: glsl/com/io7m/r2/shaders/core/R2LogDepth.h] and the final position
in R2PositionReconstruction.h [url:
glsl/com/io7m/r2/shaders/core/R2PositionReconstruction.h]:
2.21.6.3 Position reconstruction (LogDepth)
───────────────────────────────────────────
#ifndef R2_LOG_DEPTH_H
#define R2_LOG_DEPTH_H
/// \file R2LogDepth.h
/// \brief Logarithmic depth functions.
///
/// Prepare an eye-space Z value for encoding. See R2_logDepthEncodePartial.
///
/// @param z An eye-space Z value
/// @return The prepared value
///
float
R2_logDepthPrepareEyeZ(
const float z)
{
return 1.0 + (-z);
}
///
/// Partially encode the given _positive_ eye-space Z value. This partial encoding
/// can be used when performing part of the encoding in a vertex shader
/// and the rest in a fragment shader (for efficiency reasons) - See R2_logDepthPrepareEyeZ.
///
/// @param z An eye-space Z value
/// @param depth_coefficient The depth coefficient used to encode \a z
///
/// @return The encoded depth
///
float
R2_logDepthEncodePartial(
const float z,
const float depth_coefficient)
{
float half_co = depth_coefficient * 0.5;
float clamp_z = max (0.000001, z);
return log2 (clamp_z) * half_co;
}
///
/// Fully encode the given eye-space Z value.
///
/// @param z An eye-space Z value
/// @param depth_coefficient The depth coefficient used to encode \a z
/// @return The fully encoded depth
///
float
R2_logDepthEncodeFull(
const float z,
const float depth_coefficient)
{
float half_co = depth_coefficient * 0.5;
float clamp_z = max (0.000001, z + 1.0);
return log2 (clamp_z) * half_co;
}
///
/// Decode a depth value that was encoded with the given depth coefficient.
/// Note that in most cases, this will yield a _positive_ eye-space Z value,
/// and must be negated to yield a conventional negative eye-space Z value.
///
/// @param z The depth value
/// @param depth_coefficient The coefficient used during encoding
///
/// @return The original (positive) eye-space Z value
///
float
R2_logDepthDecode(
const float z,
const float depth_coefficient)
{
float half_co = depth_coefficient * 0.5;
float exponent = z / half_co;
return pow (2.0, exponent) - 1.0;
}
#endif // R2_LOG_DEPTH_H
2.21.6.4 Position reconstruction (GLSL)
───────────────────────────────────────
#ifndef R2_POSITION_RECONSTRUCTION_H
#define R2_POSITION_RECONSTRUCTION_H
/// \file R2PositionReconstruction.h
/// \brief Functions for performing position reconstruction during deferred rendering.
#include "R2Bilinear.h"
#include "R2ViewRays.h"
///
/// Reconstruct an eye-space position from the given parameters.
///
/// @param eye_z The eye-space Z value of the position
/// @param uv The current position on the screen in UV coordinates
/// @param view_rays The current set of view rays
///
vec4
R2_positionReconstructFromEyeZ(
const float eye_z,
const vec2 uv,
const R2_view_rays_t view_rays)
{
vec3 origin =
R2_bilinearInterpolate3(
view_rays.origin_x0y0,
view_rays.origin_x1y0,
view_rays.origin_x0y1,
view_rays.origin_x1y1,
uv
);
vec3 ray_normal =
R2_bilinearInterpolate3(
view_rays.ray_x0y0,
view_rays.ray_x1y0,
view_rays.ray_x0y1,
view_rays.ray_x1y1,
uv
);
vec3 ray =
(ray_normal * eye_z) + origin;
return vec4 (ray, 1.0);
}
#endif // R2_POSITION_RECONSTRUCTION_H
5 The precalculated view ray vectors are passed to the fragment shader in a
value of type R2_view_rays_t:
2.21.6.6 View Rays (GLSL)
─────────────────────────
#ifndef R2_VIEW_RAYS_H
#define R2_VIEW_RAYS_H
/// \file R2ViewRays.h
/// \brief View ray types
/// The type of view rays used to reconstruct positions during deferred rendering.
struct R2_view_rays_t {
/// The bottom left origin
vec3 origin_x0y0;
/// The bottom right origin
vec3 origin_x1y0;
/// The top left origin
vec3 origin_x0y1;
/// The top right origin
vec3 origin_x1y1;
/// The view ray pointing out of the bottom left origin
vec3 ray_x0y0;
/// The view ray pointing out of the bottom right origin
vec3 ray_x1y0;
/// The view ray pointing out of the top left origin
vec3 ray_x0y1;
/// The view ray pointing out of the top right origin
vec3 ray_x1y1;
};
#endif // R2_VIEW_RAYS_H
2.22 Forward rendering (Translucency) [id: di.forward]
2.22.1 Overview [id: di.forward.overview]
1 Because the deferred renderer [ref: di.deferred] in the r2 package is
incapable of rendering translucent instances, a separate forward renderer
is provided. A translucent instance is an instance that, when rendered,
is simply blended with the current image. This is used to implement
visual effects such as glass, water, smoke, fire, etc.
2.22.2 Instances [id: di.forward.instances]
1 The r2 package provides a slightly different abstraction for translucent
instances. Because of the strict ordering [ref: di.forward.ordering]
requirements when rendering translucent instances, it is simply not
possible to batch translucent instances by material and shaders for
performance reasons as is done with opaque instances [ref:
di.deferred.geom.ordering]. The r2 package therefore simply has users
submit a list of values of type R2TranslucentType [url:
apidocs/com/io7m/r2/core/R2TranslucentType.html] in draw order. Each
translucent value contains a blending [ref: di.forward.blending] and
culling [ref: di.forward.culling] configuration, along with an instance
[ref: di.instances.overview], and a shader [ref: di.shaders.overview] for
rendering the instance.
2.22.3 Blending [id: di.forward.blending]
1 Each translucent instance provides a blending configuration that states
how the rendered instance is blended with the contents of the current
framebuffer.
2.22.4 Culling [id: di.forward.culling]
1 Typically, it is not desirable to render the back faces of opaque
instances as they are by definition invisible. However, translucent
instances are by definition translucent and therefore the back faces of
the instances may be visible. A translucent instance therefore contains a
value that specifies whether front faces, back faces, or both should be
rendered.
2.22.5 Ordering [id: di.forward.ordering]
1 Unlike opaque instances which can be rendered in any order due to depth
testing, translucent instances must be rendered in strict
furthest-to-nearest order. The r2 package simply delegates the
responsibility of submitting instances in the correct order to the user.
This frees the package from having to know anything about the spatial
properties of the scene being rendered.
2.22.6 Types [id: di.forward.types]
1 In the r2 package, translucent instances are rendered via implementations
of the R2TranslucentRendererType [url:
apidocs/com/io7m/r2/core/R2TranslucentRendererType.html] interface.
2 Shaders for rendering translucent instances are of type
R2ShaderTranslucentType [url:
apidocs/com/io7m/r2/core/shaders/types/R2ShaderTranslucentType.html].
2.22.7 Provided Shaders [id: di.forward.provided]
1 Because translucent surface can have a massive range of appearances, the
r2 package makes no attempt to provide a wide range of shaders for
translucent surfaces.
2.22.7.2 Provided instance shaders
──────────────────────────────────
┌─────────────────────────────────────────────────────────────────────────────────────────────┬──────────────────┐
│ Shader │ Description │
├─────────────────────────────────────────────────────────────────────────────────────────────┼──────────────────┤
│ R2TranslucentShaderBasicPremultipliedSingle [url: │ Basic textured │
│ apidocs/com/io7m/r2/core/shaders/provided/R2TranslucentShaderBasicPremultipliedSingle.html] │ surface without │
│ │ lighting, with │
│ │ distance fading, │
│ │ producing │
│ │ premultiplied │
│ │ alpha output. │
│ │ │
└─────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────┘
2.23 Normal Mapping [id: di.normal-mapping]
2.23.1 Overview [id: di.normal-mapping.overview]
1 The r2 package supports the use of tangent-space normal mapping to allow
for per-pixel control over the surface normal of rendered triangles. This
allows for meshes to appear to have very complex surface details without
requiring those details to actually be rendered as triangles within the
scene.
2.23.2 Tangent Space [id: di.normal-mapping.tangent-space]
1 Conceptually, there is a three-dimensional coordinate system based at
each vertex, formed by three orthonormal basis vectors: The vertex
normal, tangent and bitangent vectors. The normal vector is a the vector
perpendicular to the surface at that vertex. The tangent and bitangent
vectors are parallel to the surface, and each vector is obviously
perpendicular to the other two vectors. This coordinate space is often
referred to as tangent space. The normal vector actually forms the Z axis
of the coordinate space, and this fact is central to the process of
normal mapping. The coordinate system at each vertex may be left or
right-handed depending on the arrangement of UV coordinates at that
vertex.
2.23.2.2 Vertex coordinate system
─────────────────────────────────
[image: images/normal.png] (Vertex coordinate system)
2.23.3 Tangent/Bitangent Generation [id: di.normal-mapping.tangent-bitangent-generation]
1 Tangent and bitangent vectors can be generated by the modelling programs
that artists use to create polygon meshes, but, additionally, the
R2MeshTangents [url: apidocs/com/io7m/r2/meshes/R2MeshTangents.html]
class can take an arbitrary mesh with only normal vectors and UV
coordinates and produce tangent and bitangent vectors. The full
description of the algorithm used is given in Mathematics for 3D Game
Programming and Computer Graphics, Third Edition [url:
http://www.mathfor3dgameprogramming.com/] [25], and also in an article
[url: http://www.terathon.com/code/tangent.html] by the same author. The
actual aim of tangent/bitangent generation is to produce a pair of
orthogonal vectors that are oriented to the x and y axes of an arbitrary
texture. In order to do achieve this, the generated vectors are oriented
according to the UV coordinates in the mesh.
2 In the r2 package, the bitangent vector is not actually stored in the
mesh data, and the tangent vector for any given vertex is instead stored
as a four-component vector. The reasons for this are as follows: Because
the normal, tangent, and bitangent vectors are known to be orthonormal,
it should be possible to reconstruct any one of the three vectors given
the other two at run-time. This would eliminate the need to store one of
the vectors and would reduce the size of mesh data (including the on-disk
size, and the size of mesh data allocated on the GPU) by a significant
amount. Given any two orthogonal vectors V0 and V1, a vector orthogonal
to both can be calculated by taking the cross product of both, denoted
(cross V0 V1). The problem here is that if V0 is assumed to be the
original normal vector N, and V1 is assumed to be the original tangent
vector T, there is no guarantee that (cross N T) will produce a vector
equal to the original bitangent vector B: There are two possible choices
of value for B that differ only in the sign of their coordinate values.
3 As an example, a triangle that will produce T and B vectors that form a
right-handed coordinate system with the normal vector N (with UV
coordinates indicated at each vertex):
2.23.3.4 Tangent generation (RHC)
─────────────────────────────────
[image: images/tangent_gen_RHC.png]
(Tangent generation (Resulting in a right-handed coordinate system)
5 The same triangle will produce vectors that form a left-handed system
when generating vectors for another vertex (note that the result of
(Vector3f.cross N T) = (Vector3f.negation B) ):
2.23.3.6 Tangent generation (LHC)
─────────────────────────────────
[image: images/tangent_gen_LHC.png]
(Tangent generation (Resulting in a left-handed coordinate system)
7 However, if the original tangent vector T was augmented with a piece of
extra information that indicated whether or not the result of (cross N T)
needed to be inverted, then reconstructing B would be trivial. Therefore,
the fourth component of the tangent vector T contains 1.0 if
(cross N T) = B, and -1.0 if (cross N T) = -B. The bitangent vector can
therefore be reconstructed by calculating cross (N, T.xyz) * T.w.
8 With the three vectors (T, B, N), it's now possible construct a 3x3
matrix that can transform arbitrary vectors in tangent space [ref:
di.normal-mapping.tangent-space] to object space [ref: di.coords.object]:
2.23.3.9 Tangent → Object matrix
────────────────────────────────
[image: images/tangent_object_matrix.png] (Tangent → Object matrix)
10 With this matrix, it's now obviously possible to take an arbitrary vector
in tangent space and transform it to object space. Then, with the current
normal matrix (object → eye), transform the object space vector all the
way to eye space [ref: di.coords.eye] in the same manner as ordinary
per-vertex object space normal vectors.
2.23.4 Normal Maps [id: di.normal-mapping.map]
1 A normal map is an ordinary RGB texture where each texel represents a
tangent space normal vector. The x coordinate is stored in the red
channel, the y coordinate is stored in the green channel, and the z
coordinate is stored in the blue channel. The original coordinate values
are assumed to fall within the inclusive range [-1.0, 1.0], and these
values are mapped to the range [0.0, 1.0] before being encoded to a
specific pixel format.
2 As an example, the vector (0.0, 0.0, 1.0) is first mapped to
(0.5, 0.5, 1.0) and then, assuming an image format with 8-bits of
precision per color channel, encoded to (0x7f, 0x7f, 0xff). This results
in a pale blue color that is characteristic of tangent space normal maps:
2.23.4.3 (0.0, 0.0, 1.0)
────────────────────────
[image: images/normalmap_zerozeroone.png] ((0.0, 0.0, 1.0)
4 Typically, tangent space normal maps are generated from a simple height
maps: Greyscale images where 0.0 denotes the lowest possible height, and
1.0 indicates the highest possible height. There are multiple algorithms
that are capable of generating normal vectors from height maps, but the
majority of them work from the same basic principle: For a given pixel
with value h at location (x, y) in an image, the neighbouring pixel
values at (x - 1, y), (x - 1, y - 1), (x + 1, y), (x + 1, y + 1) are
compared with h in order to determine the slope between the height
values. As an example, the Prewitt (3x3) operator [url:
https://en.wikipedia.org/wiki/Prewitt_operator] when used from the
gimp-normalmap [url: https://code.google.com/p/gimp-normalmap/] plugin
will produce the following map from a given greyscale height map:
2.23.4.5 Prewitt 3x3 normal map
───────────────────────────────
[image: images/normalmap_fromheight.png] (Prewitt 3x3 normal map)
6 It is reasonably easy to infer the general directions of vectors from a
visual inspection of a tangent space normal map alone. In the above
image, the flat faces of the bricks are mostly pale blue. This is because
the tangent space normal for that surface is pointing straight towards
the viewer - mostly towards the positive z direction. The right edges of
the bricks in the image are tinted with a pinkish hue - this indicates
that the normal vectors at that pixel point mostly towards the positive x
direction.
2.23.5 Rendering With Normal Maps [id: di.normal-mapping.rendering]
1 As stated, the purpose of a normal map is to give per-pixel control over
the surface normal for a given triangle during rendering. The process is
as follows:
2.23.5.2 Rendering process
──────────────────────────
0. Calculate the bitangent vector B from the N and T vectors. This step
is performed on a per-vertex basis (in the vertex shader ).
1. Construct a 3x3 tangent → object matrix M from the (T, B, N) vectors.
This step is performed on a per-fragment basis (in the fragment
shader) using the interpolated vectors calculated in the previous
step.
2. Sample a tangent space normal vector P from the current normal map.
3. Transform the vector P with the matrix M by calculating M * P,
resulting in an object space normal vector Q.
4. Transform the vector Q to eye space, in the same manner that an
ordinary per-vertex normal vector would be (using the 3x3 normal
matrix [ref: di.coords.eye.normal-matrix] ).
3 Effectively, a "replacement" normal vector is sampled from the map, and
transformed to object space using the existing (T, B, N) vectors. When
the replacement normal vector is used when applying lighting, the effect
is dramatic. Given a simple two-polygon square textured with the
following albedo texture and normal map:
2.23.5.4 Example albedo and normal maps
───────────────────────────────────────
[image: images/normalmap_metalpanels.png] (Example albedo and normal
maps)
5 The square when textured and normal mapped, with three spherical lights:
2.23.5.6 Lit and normal mapped
──────────────────────────────
[image: images/normalmap_applied.png] (Lit and normal mapped)
7 The same square with the same lights but missing the normal map:
2.23.5.8 Lit and not normal mapped
──────────────────────────────────
[image: images/normalmap_none.png] (Lit and not normal mapped)
2.24 Logarithmic Depth [id: di.log_depth]
2.24.1 Overview [id: di.log_depth.overview]
1 The r2 package exclusively utilizes a so-called logarithmic depth buffer
for all rendering operations.
2.24.2 OpenGL Depth Issues [id: di.log_depth.issues_existing]
1 By default, OpenGL (effectively) stores a depth value proportional to the
reciprocal of the z component of the clip space [ref: di.coords.clip]
coordinates of each vertex projected onto the screen [26]. Informally,
the perspective projection matrix used to transform eye space [ref:
di.coords.eye] coordinates to clip space will place the negated z
component of the original eye space coordinates into the w component of
the resulting clip space coordinates. When the hardware performs the
division by w [ref: di.coords.ndevice] to produce normalized-device space
coordinates, the resulting z component falls within the range [-1.0, 1.0]
(although any point with a z component less than 0 will be clipped away
by the clipping hardware). This final value is linearly mapped to a
configurable range (typically [0.0, 1.0]) to produce a screen space [ref:
di.coords.screen] depth value.
2 Unfortunately, the encoding scheme above means that most of the depth
buffer is essentially wasted. The above scheme will give excessive
precision for objects close to the viewing plane, and almost none for
objects further away. Fortunately, a better encoding scheme known as
logarithmic depth [27] can be implemented that provides vastly greater
precision and coexists happily with the standard projection matrices used
in OpenGL-based renderers.
2.24.3 Logarithmic Encoding [id: di.log_depth.encoding]
1 A logarithmic depth value is produced by encoding a negated (and
therefore positive) eye space z value in the manner specified by encode
LogDepth.hs [url: haskell/LogDepth.hs]:
2.24.3.2 Logarithmic Depth (Encoding)
─────────────────────────────────────
module LogDepth where
newtype LogDepth =
LogDepth Float
deriving (Eq, Ord, Show)
type Depth = Float
log2 :: Float -> Float
log2 = logBase 2.0
depth_coefficient :: Float -> Float
depth_coefficient far = 2.0 / log2 (far + 1.0)
encode :: Float -> Depth -> LogDepth
encode depth_co depth =
let hco = depth_co * 0.5 in
LogDepth $ log2 (depth + 1.0) * hco
decode :: Float -> LogDepth -> Depth
decode depth_co (LogDepth depth) =
let hco = depth_co * 0.5 in
(2.0 ** (depth / hco)) - 1
3 The function is parameterized by a so-called depth coefficient that is
derived from the far plane distance as shown by depth_coefficient.
4 The inverse of encode is decode, such that for a given negated eye space
z, z = decode d (encode d z).
5 A graph of the functions is as follows:
2.24.3.6 Logarithmic Depth (Graph)
──────────────────────────────────
[image: images/log_depth.png] (Logarithmic Depth (Graph)
7 An interactive GeoGebra [url: http://geogebra.org] construction is
provided in log_depth.ggb [url: log_depth.ggb].
8 The r2 package uses a slightly modified version of the encoding function
that clamps the original z value to the range [0.000001, ∞]. The reason
for this is that log2 (0) is undefined, and so attempting to derive a
depth value in this manner tends to cause issues with triangle clipping.
The encoding function is also separated into two parts as a simple
optimization: The encoding function contains a term z + 1.0, and this
term can be calculated by a vertex shader and interpolated. The actual
functions as implemented are given by R2LogDepth.h [url:
glsl/com/io7m/r2/shaders/core/R2LogDepth.h]:
2.24.3.9 Logarithmic Depth (GLSL)
─────────────────────────────────
#ifndef R2_LOG_DEPTH_H
#define R2_LOG_DEPTH_H
/// \file R2LogDepth.h
/// \brief Logarithmic depth functions.
///
/// Prepare an eye-space Z value for encoding. See R2_logDepthEncodePartial.
///
/// @param z An eye-space Z value
/// @return The prepared value
///
float
R2_logDepthPrepareEyeZ(
const float z)
{
return 1.0 + (-z);
}
///
/// Partially encode the given _positive_ eye-space Z value. This partial encoding
/// can be used when performing part of the encoding in a vertex shader
/// and the rest in a fragment shader (for efficiency reasons) - See R2_logDepthPrepareEyeZ.
///
/// @param z An eye-space Z value
/// @param depth_coefficient The depth coefficient used to encode \a z
///
/// @return The encoded depth
///
float
R2_logDepthEncodePartial(
const float z,
const float depth_coefficient)
{
float half_co = depth_coefficient * 0.5;
float clamp_z = max (0.000001, z);
return log2 (clamp_z) * half_co;
}
///
/// Fully encode the given eye-space Z value.
///
/// @param z An eye-space Z value
/// @param depth_coefficient The depth coefficient used to encode \a z
/// @return The fully encoded depth
///
float
R2_logDepthEncodeFull(
const float z,
const float depth_coefficient)
{
float half_co = depth_coefficient * 0.5;
float clamp_z = max (0.000001, z + 1.0);
return log2 (clamp_z) * half_co;
}
///
/// Decode a depth value that was encoded with the given depth coefficient.
/// Note that in most cases, this will yield a _positive_ eye-space Z value,
/// and must be negated to yield a conventional negative eye-space Z value.
///
/// @param z The depth value
/// @param depth_coefficient The coefficient used during encoding
///
/// @return The original (positive) eye-space Z value
///
float
R2_logDepthDecode(
const float z,
const float depth_coefficient)
{
float half_co = depth_coefficient * 0.5;
float exponent = z / half_co;
return pow (2.0, exponent) - 1.0;
}
#endif // R2_LOG_DEPTH_H
10 A fragment shader can use encode_full to compute a logarithmic depth
value from a given positive eye space z value. Alternatively, a vertex
shader can compute the z + 1.0 term r from a non-negated eye space z
value, and pass r to a cooperating fragment shader which then finishes
the computation by applying encode_partial to r. When performing position
reconstruction [ref: di.deferred-position-recon] during deferred
rendering, the original eye space z value of a fragment is retrieved by
negating the result of decode applied to a given logarithmic depth
sample.
11 The original derivation of the encoding and decoding functions as
described by Brano Kemen used the w component of the resulting clip space
coordinates. Unfortunately, this does not work correctly with
orthographic projections, as the typical orthographic projection matrix
will produce clip space coordinates with a w component always equal to 1.
Aside from the effects that this will have on depth testing (essentially
mapping the depth of all fragments to the far plane), it also makes
position reconstruction impossible as the original eye space z value
cannot be recovered. Instead, the r2 package uses the negated eye space z
value directly in all cases.
2.25 Environment Mapping [id: di.environment-mapping]
2.25.1 Overview [id: di.environment-mapping.overview]
1 Environment mapping is conceptually the process of constructing an
artificial environment around an object in order to provide, for example,
effects such as reflective surfaces or refractive objects. In the r2
package, the artificial environment is represented by cube maps [ref:
di.environment-mapping.cube-maps], and the only provided effect is a
simulation of reflection. Effects such as refraction are instead provided
via generic refraction [ref: di.generic-refraction], which doesn't use
environment mapping.
2.25.2 Cube Maps [id: di.environment-mapping.cube-maps]
1 A cube map is a texture with six faces. When used for environment
mapping, each face represents a 90° image of the environment visible in
the direction (in world space [ref: di.coords.world]) of that face. Cube
maps are normally constructed by placing an observer in a scene and then
orienting the observer in the direction of each cube face in turn and
rendering an image. As an example:
2.25.2.2 Cube map scene
───────────────────────
[image: images/envmap_cube_scene.png] (Cube map scene)
3 Given the above scene, with the observer placed exactly in the center of
the indicated magenta circle and assuming a 90° field of view, the six
images visible from that location corresponding to the -x, +x, -y, -z,
-z, +z cube faces are:
2.25.2.4 Cube map example
─────────────────────────
[image: images/envmap_cubemap.png] (Cube map example)
5 While sampling from ordinary two-dimensional textures involves looking up
texels by their two-dimensional coordinates, sampling from cube maps
requires three-dimensional coordinates. The three-dimensional coordinates
are interpreted as a direction vector or ray emanating from the center of
the cube, and the point of intersection between the ray and the
corresponding cube face is used to select a texel from that face. Note
that in OpenGL there are issues with coordinate system handedness [ref:
di.environment-mapping.handedness] that the r2 package corrects.
2.25.3 Reflections [id: di.environment-mapping.reflection]
1 So-called environment-mapped reflections are trivially provided by cube
maps. For a given surface with a normal vector n, and given the view
direction v (from the observer to the surface), a reflection vector is
given by r = Reflection.reflection v n:
2.25.3.2 Reflection vector
──────────────────────────
module Reflection where
import qualified Vector3f as V3
reflection :: V3.T -> V3.T -> V3.T
reflection v0 v1 = V3.sub3 v0 (V3.scale v1 (2.0 * (V3.dot3 v1 v0)))
3 The reflection vector r is then used to look up a texel in the current
cube map directly. This gives a convincing illusion of reflection that
will change as the observer moves relative to the surface. Combining
normal mapping and environment mapped reflections gives a striking
effect:
2.25.3.4 Normal/Environment mapping
───────────────────────────────────
[image: images/envmap_tiles.png] (Normal/Environment mapping)
2.25.3.5 Normal/Environment mapping (Cube map)
──────────────────────────────────────────────
[image: images/envmap_toronto.png] (Normal/Environment mapping (Cube map)
6 Note that in the actual r2 implementation, the vectors n and v will be in
eye-space and therefore so will r. The vector r is transformed back to
world space [ref: di.coords.world] by the inverse of the current view
matrix for use with the cube map.
2.25.4 Handedness [id: di.environment-mapping.handedness]
1 For reasons lost to time, cube maps in OpenGL use a left-handed
coordinate system in contrast to the usual right-handed coordinate
system. Because of this, calculated reflection vectors actually have to
be inverted to prevent sampling from the wrong cube face. The r2 package
enforces a consistent right-handed coordinate system everywhere. The
direction of each cube face corresponds to the same direction in world
space [ref: di.coords.world], without exception.
2.26 Stippling [id: di.stippling]
2.26.1 Overview [id: di.stippling.overview]
1 One major issue with deferred rendering [ref: di.deferred] is that it
does not allow for translucency; the scene is placed into a single flat
image and there is no way to express the fact that an object is to be
overlaid on top of another object. This can be a problem when
implementing systems such as level-of-detail (or LOD) switching. A basic
requirement for an LOD system is that when the viewer moves a certain
distance away from an object, that object should be replaced with a
lower-polygon version in order to reduce rendering costs. Switching from
a high polygon version to a low polygon version in one frame can be
visually jarring, however, so it is usually desirable to fade out one
version of the object whilst fading in another version over the course of
a few frames. This presents an immediate problem: It is not possible to
implement traditional alpha translucency fading in a deferred rendering
system, as described above.
2 The stippling technique attempts to provide an alternative to alpha
translucency. The technique is simple: Simply discard some pixels of an
object when the object is rendered into the geometry buffer [ref:
di.deferred.geom]. By progressively discarding more pixels over the
course of a few frames, the object can be made to fade . If the pattern
of discarded pixels is randomized and the fading time is short, the
result is visually acceptable for implementing LOD systems.
2.26.1.3 Stippling
──────────────────
[image: images/stipple.png] (Progressive stippling)
2.26.2 Algorithm [id: di.stippling.algorithm]
1 The stippling algorithm is very simple:
2.26.2.2 Algorithm
──────────────────
0. Tile a pattern texture t over the entire screen.
1. For each pixel with screen coordinates p in the object currently
being rendered, sample a value x from t at p.
2. If x is less than the defined stippling threshold s, discard the
pixel.
3 In practice, for good visual results when fading between two objects, the
programmer should use two complementary stippling patterns for the
objects. For example, using a checkerboard stippling pattern for the
first object and an inverted copy of the pattern for the other. This
guarantees that at no point are the same pixels from both objects
discarded.
2.26.3 Types [id: di.stippling.types]
1 In the r2 package, the stippling effect is provided by shaders such as
R2SurfaceShaderBasicStippledSingle [url:
apidocs/com/io7m/r2/core/shaders/provided/R2SurfaceShaderBasicStippledSingle.html].
2.27 Generic Refraction [id: di.generic-refraction]
2.27.1 Overview [id: di.generic-refraction.overview]
1 The r2 package implements the generic refraction effect described in GPU
Gems 2 [url:
http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter19.html]. The
technique lends itself to a huge range of effects such as lenses, glass,
heat haze, and water - simply by varying the meshes and textures used
when performing refraction.
2.27.2 Algorithm [id: di.generic-refraction.algorithm]
1 For a given instance, the process to render the instance is as follows:
2.27.2.2 Algorithm
──────────────────
0. Produce a mask [ref: di.generic-refraction.masking], if necessary.
1. Render the instance using a given source image [ref:
di.generic-refraction.source], vector texture [ref:
di.generic-refraction.vectors], color [ref:
di.generic-refraction.color], and mask image [ref:
di.generic-refraction.masking].
3 The actual rendering technique is very simple: Given a screen-space
position (x, y), sample the color from a source [ref:
di.generic-refraction.source] image at (x + s, y + t), where (s, t) are
signed per-pixel offset values that are sampled from textures or derived
from an associated vector [ref: di.generic-refraction.vectors] texture.
2.27.3 Sources [id: di.generic-refraction.source]
1 The refraction effect typically uses a (possibly downsized) image of the
scene as a source image. The r2 allows for use of an arbitrary image.
2.27.4 Vectors [id: di.generic-refraction.vectors]
1 Refraction vectors are sampled from the red and green components of a
delta texture. The sampled values are scaled by the material's scale
factor and used directly to calculate (x + s, y + t). For example, a
simple noisy red/green delta texture applied to a quad results in the
following effect:
2.27.4.2 Noise quad
───────────────────
[image: images/refract_noise_quad.png] (Noise quad)
2.27.4.3 Noise quad (texture)
─────────────────────────────
[image: images/refract_noise_quad_texture.png] (Noise quad texture)
2.27.5 Colors [id: di.generic-refraction.color]
1 The sampled scene colors used to perform the refraction effect are
multiplied by a constant color, specified by each material. This allows
for simple colored glass effects (shown here with a specular-only
instance rendered over the top of the refractive instance to provide
specular highlights):
2.27.5.2 Color 0
────────────────
[image: images/refract_color_0.png] (Color 0)
3 Using pure RGBA white (1.0, 1.0, 1.0, 1.0) results in a clear glass
material:
2.27.5.4 Color 1
────────────────
[image: images/refract_color_1.png] (Color 1)
2.27.6 Masking [id: di.generic-refraction.masking]
1 Because refractive instances are translucent, they are normally rendered
after having already rendered all of the opaque objects in the scene.
Because rendering of translucent instances occurs with depth testing
enabled, it is therefore possible for opaque instances to occlude
refractive instances. This poses a problem for the implementation of
refraction described above, because the pixels of an occluding object may
be sampled when performing the refraction, as shown in the following
image:
2.27.6.2 Refraction bleeding
────────────────────────────
[image: images/refract_bleed.png] (Refraction bleeding)
3 Note how the pixels of the opaque instances are bleeding into the
refracting object, despite being conceptually in front of it. This is
because the refraction effect is implemented in screen space and is just
sampling pixels from the surrounding area to simulate the bending of
light rays. Using a mask prevents this:
2.27.6.4 Refraction without bleeding
────────────────────────────────────
[image: images/refract_nobleed.png] (Refraction without bleeding)
5 A mask is produced by rendering a monochrome silhouette of the refracting
object, and then using the values of this mask to linearly interpolate
between the colors c at (x, y) and the colors r at (x + s, y + t). That
is, a value of m = 0 sampled from the mask yields
mix c r m = mix c r 0 = c, and a value of m = 1 sampled from the mask
yields mix c r m = mix c r 1 = r. This has the effect of preventing the
refraction simulation from using pixels that fall outside of the mask
area.
2.27.6.6 Mask
─────────────
[image: images/refract_mask.png] (Mask
7 The mask image can also be softened with a simple box blur to reduce
artifacts in the refracted image.
2.27.7 Types [id: di.generic-refraction.types]
1 In the r2 package, the refraction effect is provided by rendering a
translucent instance with a refraction shader such as
R2RefractionMaskedDeltaShaderSingle [url:
apidocs/com/io7m/r2/core/shaders/provided/R2RefractionMaskedDeltaShaderSingle.html].
2 Masks can be produced via implementations of the R2MaskRendererType [url:
apidocs/com/io7m/r2/core/R2MaskRendererType.html] interface.
2.28 Filter: Fog [id: di.fog]
2.28.1 Overview [id: di.fog.overview]
1 The fog effect is a simple effect that is intended to simulate
atmospheric fog within a scene.
2.28.1.2 Fog
────────────
[image: images/fog_linear.png] (Fog
2.28.2 Algorithm [id: di.fog.algorithm]
1 The algorithm is trivial:
2.28.2.2 Algorithm
──────────────────
0. For each pixel p at (x, y)
0. Sample the scene's depth d at (x, y)
1. Determine the positive eye-space Z value z of p
2. Mix between the global fog color d and p using a mix function
fog(z)
3 The mix function fog(z) is selectable. The r2 package provides linear,
quadratic, and inverse quadratic fog. The definitions of the available
mix functions are as follows:
2.28.2.4 Fog functions
──────────────────────
module FogFactorZ where
clamp :: Float -> (Float, Float) -> Float
clamp x (lower, upper) = max (min x upper) lower
fogLinear :: Float -> (Float, Float) -> Float
fogLinear z (near, far) =
let r = (z - near) / (far - near) in
clamp r (0.0, 1.0)
fogQuadratic :: Float -> (Float, Float) -> Float
fogQuadratic z (near, far) =
let q = fogLinear z (near, far) in q * q
fogQuadraticInverse :: Float -> (Float, Float) -> Float
fogQuadraticInverse z (near, far) =
let q = fogLinear z (near, far) in sqrt(q)
2.28.2.5 Linear Fog
───────────────────
[image: images/fog_linear.png] (Linear Fog)
2.28.2.6 Quadratic Fog
──────────────────────
[image: images/fog_quadratic.png] (Quadratic Fog)
2.28.2.7 Inverse Quadratic Fog
──────────────────────────────
[image: images/fog_quadratic_inverse.png] (Inverse Quadratic Fog)
2.28.3 Types [id: di.fog.types]
1 In the r2 package, the fog effect is provided by the R2FilterFogDepth
[url: apidocs/com/io7m/r2/filters/R2FilterFogDepth.html] type.
2.29 Filter: Screen Space Ambient Occlusion [id: di.ssao]
2.29.1 Overview [id: di.ssao.overview]
1 Screen space ambient occlusion is, unsurprisingly, an approximate
algorithm for calculating ambient occlusion in screen space [ref:
di.coords.screen]. Informally, ambient occlusion is a measure of how
exposed a given point is to the environment's ambient light. The r2
package does not directly support ambient lighting, so instead the
diffuse [ref: di.lighting.diffuse] light term is typically modulated by
an ambient occlusion term [28] to produce the same overall effect.
2.29.1.2 SSAO
─────────────
[image: images/ssao.png] (SSAO
2.29.2 Ambient Occlusion Buffer [id: di.ssao.abuffer]
1 An ambient occlusion buffer is a render target [ref: di.render-target] in
which an occlusion term is stored. In the r2 package, ambient occlusion
buffers are simple single-channel 8-bit images, where 0 means fully
occluded and 1 means not occluded.
2.29.3 Algorithm [id: di.ssao.algorithm]
1 The algorithm works by consuming the depth and normal values from
populated geometry buffer [ref: di.deferred.geom.gbuffer]. For the sake
of simplicity, the algorithm will be described as if the ambient
occlusion buffer [ref: di.ssao.abuffer] that will contain the calculated
occlusion terms will be the same size as the geometry buffer. This is not
necessarily the case in practice, for performance reasons. For each pixel
at (x, y) in the geometry buffer, the eye space Z [ref: di.coords.eye]
value z is reconstructed [ref: di.deferred-position-recon.eye-space-z]
for the pixel, and the eye space normal vector n is sampled at the same
location.
2 Then, a sampling hemisphere is placed on the surface at z, oriented along
n. A list of points, known as the sample kernel [ref: di.ssao.kernel],
are used to sample from random positions that fall inside the hemisphere.
If a sample point appears to be inside the scene geometry, then the scene
geometry is occluding that point.
2.29.3.3 Sampling Hemispheres
─────────────────────────────
[image: images/ssao_hemi.png] (Sampling hemispheres)
4 Informally, the algorithm for a point at (x, y):
2.29.3.5 Algorithm
──────────────────
0. Reconstruct the eye space position e of the screen space position
(x, y).
1. Sample the normal vector n at (x, y).
2. Peturb the normal vector n using values sampled from a random noise
texture [ref: di.ssao.noise_texture] that is tiled across the screen.
3. Produce a normal matrix [ref: di.coords.eye.normal-matrix] from n
that will transform the inherently tangent space [ref:
di.normal-mapping.tangent-space] sampling kernel vector to eye space.
The peturbed normal vector has the effect of rotating the sampling
hemisphere.
4. For a sampling kernel k of m points, of radius r, for each
i | 0 <= i < m:
0. Calculate the eye space position q of the sampling point k[i].
This is calculated as q = e + (k[i] * r).
1. Project q to screen space, use it to sample the depth buffer, and
reconstruct the resulting eye space Z value sz. The value sz then
represents the eye space Z value of the closest position of the
surface in the geometry buffer to q.
2. If abs (e.z - sz) > r then the point is automatically assumed not
to be occluded. See halo removal [ref: di.ssao.halo_removal] for
details.
3. If sz >= e.z, then it means that the sampling point in the
hemisphere has ended up underneath the rendered surface and is
therefore being occluded by it.
5. Calculate the final occlusion value o by summing the occlusion values
of each sample point, where 1.0 means the point was occluded, and 0.0
means that it was not. Return 1.0 - (o / m).
2.29.4 Noise Texture [id: di.ssao.noise_texture]
1 The noise texture used by the algorithm is a simple RGB texture with each
texel being given by the expression
normalize ((random() * 2.0) - 1.0, (random() * 2.0) - 1.0, 0.0). The
sampling kernel [ref: di.ssao.kernel] used by the algorithm is
conceptually oriented along the tangent space [ref:
di.normal-mapping.tangent-space] Z axis, and therefore each texel in the
noise texture effectively represents a rotation around the Z axis.
2 In the implementation of the algorithm, the texture is simply tiled
across the screen and sampled using the current screen space coordinates.
2.29.5 Sample Kernel [id: di.ssao.kernel]
1 A sample kernel is a fixed-size list of random sampling points, arranged
in a hemispherical pattern. For better visual results, the random points
are not evenly distributed within the hemisphere but are instead
clustered more densely nearer the origin.
2.29.5.2 Point Distribution
───────────────────────────
[image: images/ssao_hemi_points.png] (Distribution of sampling points)
3 By using a distribution of sample points nearer the origin, samples
closer to the origin have the effect of occluding more than points that
are further away.
2.29.6 Halo Removal [id: di.ssao.halo_removal]
1 A common problem with SSAO implementations is haloing. In practical
terms, this is an issue caused by two objects being very close when
considered in screen space, but that were actually far apart when
considered in eye space.
2.29.6.2 Halo Artifacts
───────────────────────
[image: images/ssao_halo.png] (Halo artifacts)
3 The simple solution to this problem is to ignore any surface points that
are at a distance greater than the sampling radius from the origin. In
the actual implementation, a simple comparison of the eye-space Z values
is used.
2.29.7 Performance [id: di.ssao.performance]
1 The SSAO algorithm is extremely expensive; by far the most expensive
algorithm implemented in the r2 package. The package provides numerous
means to control the performance of the algorithm.
2 For a kernel of size n, an occlusion map of size w * h will incur at
least w * h * n texture reads when sampling from the geometry buffer
[ref: di.deferred.geom.gbuffer] to calculate the occlusion term.
Therefore, reducing the resolution of the ambient occlusion buffer [ref:
di.ssao.abuffer] is an effective way to improve the performance of the
algorithm at a noticeable reduction in visual quality. The r2 package
does not provide any specific support for this; the programmer simply
needs to allocate a smaller ambient occlusion buffer. For the same
reason, using a smaller kernel (a smaller value of n) will also improve
performance but reduce visual quality.
3 To reduce high frequency noise introduced by the random sampling pattern
used, a bilateral blur filter is often used. In the r2 package, the blur
is separate from the SSAO effect and can therefore be omitted to improve
performance at the cost of producing a noisier image:
2.29.7.4 SSAO (Without blur)
────────────────────────────
[image: images/ssao_noise.png] (SSAO without blur)
5 The image displayed at the start of this section uses an ambient
occlusion buffer that is exactly half the size of the screen, a kernel of
size 64, and a maximum sampling distance of 0.25 eye-space units. A
single bilateral blur pass was used.
2.29.8 Types [id: di.ssao.types]
1 In the r2 package, the SSAO effect is provided by the R2FilterSSAO [url:
apidocs/com/io7m/r2/filters/R2FilterSSAO.html] type.
2 Occlusion maps can be conveniently applied to light maps with the
R2FilterOcclusionApplicator [url:
apidocs/com/io7m/r2/filters/R2FilterOcclusionApplicator.html] filter.
3 The provided implementation of the sampling kernel is given by the
R2SSAOKernel [url: apidocs/com/io7m/r2/filters/R2SSAOKernel.html] type.
4 The provided implementation of the noise texture is given by the
R2SSAONoiseTexture [url:
apidocs/com/io7m/r2/filters/R2SSAONoiseTexture.html] type.
2.29.9 Shaders [id: di.ssao.shaders]
1 The shader implementation of the SSAO algorithm is the R2SSAO [url:
doxygen/io7m-r2-shaders-0.3.0-SNAPSHOT/R2SSAO_8frag_source.html] shader.
2.30 Filter: Emission [id: di.emission]
2.30.1 Overview [id: di.emission.overview]
1 An emissive surface is a surface that appears to emit light. The r2
package offers emission as a visual effect implemented as a filter. An
optional glow effect is provided to allow emissive surfaces to appear to
have a configurable aura [url:
http://en.wikipedia.org/wiki/Halo_%28optical_phenomenon%29].
2 The emission effect is obviously not physically accurate - surfaces do
not really emit light. The user is expected to make intelligent use of
the standard light sources [ref: di.deferred.light] to provide lighting,
and to use the emission effect to complement them.
2.30.1.3 Emission
─────────────────
[image: images/emission.png] (Emission
2.30.2 Algorithm [id: di.emission.algorithm]
1 The plain emission effect without glow is implemented as trivially as
possible by sampling the emission [ref: di.deferred.geom.gbuffer] value
from a rendered scene's geometry buffer, multiplying it by the albedo
color and then simply adding the result to the current pixel color.
2 The emission effect with glow is implemented similarly, except that the
albedo * emission term is stored in a separate image, and that image is
blurred with a configurable box blur before being additively blended over
the original scene. Higher levels of blurring can give the impression of
a dusty atmosphere.
2.30.2.3 Emission (Glow)
────────────────────────
[image: images/emission_glow.png] (Emission (Glow)
2.30.3 Types [id: di.emission.types]
1 In the r2 package, the emission effect is provided by the
R2FilterEmission [url: apidocs/com/io7m/r2/filters/R2FilterEmission.html]
type.
2.31 Filter: FXAA [id: di.fxaa]
2.31.1 Overview [id: di.fxaa.overview]
1 Fast Approximate Anti-Aliasing is a simple algorithm that attempts to
detect and smooth aliasing in a color image. The algorithm works with
only the color components of the image in question; no other per-pixel
information or knowledge of the scene is required.
2.31.1.2 Without FXAA
─────────────────────
[image: images/fxaa_without.png] (Without FXAA)
2.31.1.3 With FXAA
──────────────────
[image: images/fxaa_with.png] (With FXAA)
2.31.2 Implementation [id: di.fxaa.implementation]
1 Unfortunately, information on the FXAA algorithm is sparse, and much of
it has been lost to time. The original FXAA algorithm was published in a
whitepaper by NVIDIA [29] and was severely optimized by the author on the
suggestions of many mostly anonymous contributors. The latest published
version of the algorithm (version 3.11) bears little resemblance to the
original and no documentation exists on the changes. The 3.11 version of
the algorithm is constructed from a maze of C preprocessor macros, and
many different variations of the algorithm are possible based on how the
parameter macros are defined.
2 The implementation of FXAA in the r2 package is a set of GLSL expansions
of the public domain [30] Fxaa3_11.h header with a few minor
modifications (unused parameter removals). Specifically, the PC algorithm
is used, with quality presets (10, 15, 20, 25, 29, 39).
2.31.3 Types [id: di.fxaa.types]
1 In the r2 package, the FXAA effect is provided by the R2FilterFXAA [url:
apidocs/com/io7m/r2/filters/R2FilterFXAA.html] type.
3 API Documentation
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Contents
─────────────────────────────────────────────────────────────────────────────
3.1 API Documentation ................................................... api
3.1.1 Javadoc .............................................................
3.1.2 GLSL Doxygen ........................................................
3.1 API Documentation [id: api]
3.1.1 Javadoc
1 API documentation for the package is provided via the included Javadoc
[url: apidocs].
3.1.2 GLSL Doxygen
1 API documentation for the GLSL sources are provided via the included
Doxygen [url: doxygen/html].
Footnotes
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[0] http://semver.org [url: http://semver.org]
[1] The spiritual ancestor of r2, the r1 [url: http://io7m.github.io/r1]
renderer, exposed only immutable materials. While these made it easier to
demonstrate the correctness of the programs using the renderer, it also
increased pressure on the garbage collector. Materials in the r2 may
optionally be mutable or immutable, and the user is expected understand
the difference and the consequences of using one over the other.
[2] The spiritual ancestor of r2, the r1 [url: http://io7m.github.io/r1]
renderer, exposed a fixed material system and did not expose shaders to
the user at all. While this made it easier to demonstrate the correctness
of the renderer implementation, it turned out to be needlessly inflexible
and made it more difficult to experiment with new renderer features.
[3] However, the r2 package places no limits on the number of lights that
have shadow maps, so enabling them for all light sources is possible, if
not actually advisable.
[4] See section 4.5, Transforming normal vectors .
[5] Note that matrix multiplication is not commutative.
[6] The reason for producing the concatenated matrix on the CPU and then
passing it to the shader is efficiency; if a mesh had 1000 vertices, and
the shader was passed m and v separately, the shader would repeatedly
perform the same mv = v * m multiplication to produce mv for each vertex
- yielding the exact same mv each time!
[7] Because normalized device space is a left-handed system by default, with
the viewer looking towards positive Z, and because the transformation
from clip space to normalized device space for a given point is the
division of the components of that point by the point's own w component.
[8] The handedness of the coordinate space is dependent on the depth range
[ref: di.coords.screen.depth] configured for screen space.
[9] It is actually the division by w that produces the scaling effect
necessary to produce the illusion of perspective in perspective
projections.
[10] Almost all rendering systems use different names to refer to the same
concepts, without ever bothering to document their conventions. This
harms comprehension and generally wastes everybody's time.
[11] A classic example of a modern game title that failed to anticipate
precision issues is Minecraft [url:
http://minecraft.gamepedia.com/Far_Lands].
[12] Naturally, as is standard with OpenGL, failing to associate the correct
shader attributes with the correct vertex attributes results in silent
failure and/or bizarre visual results.
[13] Typically a simple two-polygon unit quad.
[14] The core of the r2 package depends directly on the shader package, so the
correct jars will inevitably be on the classpath already.
[15] The r2 package does not use ambient terms.
[16] The attenuation function development is available for experimentation in
the included GeoGebra [url: http://geogebra.org] file attenuation.ggb
[url: attenuation.ggb].
[17] The same issue occurs when performing ordinary rendering of points in a
scene. The issue is solved there by clipping primitives based on their w
component so that primitives that are "behind" the observer are not
rendered.
[18] For some reason, the presentation does not specify a publication date.
However, inspection of the presentation's metadata suggests that it was
written in October 2014, so the numbers given are likely for reasonably
high-end 2014-era hardware.
[19] This is slightly misleading because the depth buffer is a simple
heightmap and so of course only the nearest faces of each shape would be
preserved by the depth buffer. Nevertheless, for the purposes of
comprehension, the full shapes are shown.
[20] This step is performed once on the CPU and is only repeated when the
projection matrix changes [24].
[21] Which, for many applications, may be once for the entire lifetime of the
program.
[22] By simply setting the w component to 1.
[23] Which is guaranteed to be negative, as only a negative Z value could have
resulted in a visible fragment in the geometry buffer.
[24] See section 7.8.3, "Calculating tangent vectors".
[25] See Learning To Love Your Depth Buffer [url:
http://www.sjbaker.org/steve/omniv/love_your_z_buffer.html].
[26] Apparently first discovered by Brano Kemen [url:
http://outerra.blogspot.co.uk/2012/11/maximizing-depth-buffer-range-and.html].
[27] The r2 package provides convenient methods to apply ambient occlusion to
lighting, but does not require the programmer to use any particular
method.
[28] http://developer.download.nvidia.com/assets/gamedev/files/sdk/11/FXAA_WhitePaper.pdf
[url:
http://developer.download.nvidia.com/assets/gamedev/files/sdk/11/FXAA_WhitePaper.pdf]
[29] The included Fxaa3_11.h file bears an NVIDIA copyright, but was placed
into the public domain by the original author.