Masking
Because refractive instances are translucent, they are normally rendered after having already rendered all of the opaque objects in the scene. Because rendering of translucent instances occurs with depth testing enabled, it is therefore possible for opaque instances to occlude refractive instances. This poses a problem for the implementation of refraction described above, because the pixels of an occluding object may be sampled when performing the refraction, as shown in the following image:
Note how the pixels of the opaque instances are bleeding into the refracting object, despite being conceptually in front of it. This is because the refraction effect is implemented in screen space and is just sampling pixels from the surrounding area to simulate the bending of light rays. Using a mask prevents this:
A mask is produced by rendering a monochrome silhouette of the refracting object, and then using the values of this mask to linearly interpolate between the colors c at (x, y) and the colors r at (x + s, y + t). That is, a value of m = 0 sampled from the mask yields mix c r m = mix c r 0 = c, and a value of m = 1 sampled from the mask yields mix c r m = mix c r 1 = r. This has the effect of preventing the refraction simulation from using pixels that fall outside of the mask area.
The mask image can also be softened with a simple box blur to reduce artifacts in the refracted image.