GLSL Shader Live-Reloading

July 28, 2016

Live-reloading shaders can improve productivity in rendering projects by an order of magnitude or two, so definitely worth your time to set up. In this article, I’ll describe a design for a live-reloading system that is unintrusive to your code, and simplifies a lot of resource binding.

The Interface

The usage is as follows:

---

ShaderSet shaders;

// prepended to shaders as "#version 450"
shaders.SetVersion("450");

// File prepended to shaders (after #version)
shaders.SetPreambleFile("preamble.glsl");

// detects shader stage from extensions
// returns pointer to a program in the set
GLuint* program = shaders.AddProgramFromExts({"foo.vert", "foo.frag"});

while (running)
{
    // polls timestamps and reloads any shaders
    // (programs auto-updated through pointers)
    shaders.UpdatePrograms();

    glUseProgram(*program);
    render();
}

---

Separation of Concerns

Since the ShaderSet returns a plain GLuint* (which points to to the program), the code making use of the program is separated from the ShaderSet system, since it doesn’t have to know about any ShaderSet specific interface details. This is in contrast to creating a boilerplate “Shader” class and passing that around, which makes your everyday OpenGL code strongly tied to your custom shader system.

Shader Stage Detection

Since OpenGL needs to know the shader type (vertex, fragment, geometry, etc) to create a shader object, I rely on a convention for the shader extension. The convention is as follows:

• Vertex shader: .vert
• Fragment shader: .frag
• Geometry shader: .geom
• Tessellation Control shader: .tesc
• Tessellation Evaluation shader: .tese
• Compute shader: .comp

Update Method

In my current design, I poll timestamps at every frame. This might become a performance problem if you have hundreds or thousands of shaders, but my average research codebases don’t, so polling keeps things simple. In contrast, getting file change callbacks on Windows is very complicated, and hard to make robust. I poll the timestamp using Win32 GetFileTime, and stat can be used for Unixes.

Compile/Link Errors

If a shader fails to compile, I output the error to the console and just keep on trucking. The user can then fix the compile errors through live-reloading.

If the user keeps running code with a shader that failed to compile, it’s likely that they’ll get spammed with errors from every call that uses that bad shader. For this reason, the ShaderSet automatically sets bad shaders to 0 until the compilation is fixed. This makes it possible to  detect and handle failed compilation from user code. For example:

if (*program) {
    glUseProgram(*program);
    render();
}

Resource Binding

In order to link resources like vertex attributes, uniforms, textures, and buffers to the shader, I rely exclusively on in-shader specification of locations using layout directives. This removes the need to query or bind all the attribute/uniform locations every time the programs are re-linked, which removes a lot of tedious binding code. Also, it makes it easier to swap between shaders that use the same inputs.

Synchronizing Bindings

One major drawback of in-shader specification of locations is the risk of C++ and GLSL going out of sync, and the risk of introducing “magic numbers” for locations. In order to counter this, I define a “preamble” file, which contains code that is prepended to all compiled shaders. This file contains all binding locations. A preamble file might be as follows:

---

#ifndef PREAMBLE_GLSL
#define PREAMBLE_GLSL

#define POSITION_ATTRIB_LOCATION 0
#define TEXCOORD_ATTRIB_LOCATION 1
#define NORMAL_ATTRIB_LOCATION 2

#define MVP_UNIFORM_LOCATION 0

#define DIFFUSE_TEXTURE_BINDING 0

#endif // PREAMBLE_GLSL

---

The nice thing is that this preamble is both valid GLSL and C++. Since my GLSL shaders have access to it, they can set their bindings using them, for example as follows:

---

layout(location = POSITION_ATTRIB_LOCATION)
    in vec4 Position;
layout(location = TEXCOORD_ATTRIB_LOCATION)
    in vec2 TexCoord;
layout(location = NORMAL_ATTRIB_LOCATION)
    in vec3 Normal;

layout(location = MVP_UNIFORM_LOCATION)
    uniform mat4 MVP;

layout(binding = DIFFUSE_TEXTURE_BINDING)
    uniform sampler2D Diffuse;

---

From there, my C++ code can also include preamble.glsl, and can thus also access the bindings. For example:

---

#include "preamble.glsl"

// binding attributes
glEnableVertexArrayAttrib(vao, POSITION_ATTRIB_LOCATION);

// binding uniforms
glProgramUniformMatrix4fv(*program, MVP_UNIFORM_LOCATION, 1, 0, mvp);

// binding textures
glBindTextures(DIFFUSE_TEXTURE_BINDING, 1, &diffuseTex);

---

If you’re careful, you can even put simple structs inside the preamble, which will also compile as C++ if you’ve defined types like vec4 and mat4 on the C++-side. If you’re even more careful, you can also put simple functions to share code between C++ and GLSL. This works especially well with libraries designed to work like GLSL, like glm.

I’d like to emphasize that the static nature of these bindings is not a weakness. By establishing conventions about the input locations, it’s easy to plug in new shaders by using only the inputs you want, and the C++ code you have to write to bind resources is both simplified (no need to query and re-bind constantly) and more efficient (less overhead of switching bindings.)

Implementation

You can find a sample implementation here:
https://github.com/nlguillemot/ShaderSet

Written by nlguillemot Posted in Uncategorized