Lightform effects are based on the Interactive Shader Format.

Shaders

At their core, effects are GLSL fragment shaders. Fragment shaders are sometimes called "pixel shaders," which might be more appropriate considering how they work.

Every frame, the fragment shader runs for each pixel in the output buffer. Each pixel runs the exact same shader script. The only things that change from pixel to pixel are the pixel's coordinates (referenced as gl_FragCoord.xy).

For each pixel, the shader outputs a color. Pixel coordinates come in, colors come out. That's the basic idea. An excellent introduction to shader programming is The Book of Shaders.

Lightform Effect Specifications

Lightform's effects are based on the Interactive Shader Format. Before getting started, you should review the ISF spec.

Lightform effects come in two pieces:

• A metadata file (formatted as JSON) outlines input variables that can be used/referenced in the shader
• A shader file contains the shader script itself.

Metadata file: [effect_name].liteffect

• The purpose of this file is to define non-const variables (uniforms) that you wish to use in your effect.
• This differs from the ISF spec only because it is a separate file, and not bundled into the same file with the fragment shader.
• It can contain the same kinds of fields as an ISF shader.

Metadata file example:

{
 "DESCRIPTION": "color effect",
 "INPUTS": [
 {
 "NAME": "scanImage",
 "TYPE": "image",
 "DEFAULT": "lightform:scan_rgb"
 },
 {
 "NAME": "invGamma",
 "TYPE": "float",
 "DEFAULT": 0.5,
 "MIN": 0.0,
 "MAX": 1.0
 },
 {
 "NAME": "center",
 "TYPE": "point2D",
 "DEFAULT": [ 0.5, 0.5 ],
 "MIN": [ 0, 0 ],
 "MAX": [ 1, 1 ]
 },
 {
 "NAME": "foregroundColor",
 "TYPE": "color",
 "DEFAULT": [
 1.0,
 1.0,
 1.0,
 1.0
 ]
 }
 ]
}

Shader file: [effect_name].fs

• This file is just like an ISF fragment shader file.
• Some keywords from the ISF spec are unimplemented (list below)

Shader file example:

void main()
{
 vec2 uv = gl_FragCoord.xy / RENDERSIZE.xy;
 vec4 img = IMG_NORM_PIXEL(scanImage, fract(uv + center));
 gl_FragColor = pow(img * foregroundColor, vec4(invGamma));
}

Parts of the ISF spec that are not implemented

• • isf_FragNormCoord (use gl_FragCoord.xy / RENDERSIZE.xy instead)
  • IMG_THIS_NORM_PIXEL (use IMG_NORM_PIXEL(texture, uv) instead)
  • The INPUT types audio and audioFFT. Instead use our Audio Reactivity inputs, described below
  • FRAMEINDEX
  • PASSINDEX
  • DATE
• We do not have plans to implement these features:
  • IMG_PIXEL
  • IMG_THIS_PIXEL
  • IMG_SIZE
  • event type for a INPUT

Additions to the ISF spec

Audio Reactivity

• • Audio reactivity can be integrated into custom effects by accessing the LF_AUDIOand LF_AUDIO_ACCUMvec4 uniforms from a live effect shader.
    
    • LF_AUDIO: The current audio volume/intensity. When the audio reactive gain is set to default, the range will mostly stay within [0,1], with 0 being silence. The upper limit of this range will increase when the gain is increased.
    • LF_AUDIO_ACCUM: Stores the accumulated audio intensity. Starts at 0, and every frame the value of LF_AUDIO/60.0 is added to it.
  • Each component of the audio vec4's correspond to the value for a specific audio frequency range:
    
    • .x: Low 50hz-300hz
    • .y: Mid-Low 300hz-1200hz
    • .z: Mid-High 1200hz-5000hz
    • .w: High 5000hz-20000hz
  • Note: Custom effects will only be reactive if published as a live effect, and will not be listed in the list of reactive effects on the device page. They cannot be selectively enabled or disabled, only by completely disabling audio reactivity.

Scan Images

Scan images can be used in effects by defining an IMAGE input with a corresponding URL as default value. The URL format is lightform:<scan image name>, where the image name is one of the following:

• • • scan_rgb (3 channels): the RGB data of the scan
    • scan_disparity (1 channel): the disparity data from the scan
    • scan_rgb_cartoonized (3 channels): the RGB data of the scan, processed through a "cartoonize" filter
    • scan_edge_distance (1 channel): A value in the range between [0..1] that increases the further you get from the image edges in the RGB scan data.
    • scan_edge_trace (1 channel): increasing values along the detected edges in the range (0..1], with the value 0 outside of the edges.
    • scan_normals (3 channels): scan surface normals, approximated using the disparity scan data. To use these, you need to include an image input in your metadata (.liteffect) file, just like the above example uses "lightform:scan_rgb":

"INPUTS": [
 {
 "NAME": "scanImage",
 "TYPE": "image",
 "DEFAULT": "lightform:scan_rgb"
 }
]

Extension of the metadata file

• • point3D type: it is encoded as a vec3
  • CONTROL_TYPE set to "hue_saturation" on a point2D type input to use a color picker widget instead of the default 2d widget.

{
 "NAME": "hueSaturation",
 "TYPE": "point2D",
 "MIN": [ 0, 0 ],
 "MAX": [ 1, 1 ],
 "DEFAULT": [ 0.5, 0.5 ],
 "CONTROL_TYPE": "hue_saturation"
}

To convert to RGB in the shader use:

  vec3 color = hsvToRgb(vec3(hueSaturation, 1.0));

• PRESETS array: Each element in this array defines a preset with a NAME string and a VALUES dictionary containing the preset values for each input. All non-image ****INPUTS MUST be specified in PRESETS.

"PRESETS": [
 {
 "NAME": "Red Color",
 "VALUES": {
 "colorRGBA": [ 1.0, 0.0, 0.0, 1.0 ]
 }
]

Included GLSL functions and constants

// Constants:
float EPSILON = 1.0e-10;

vec4 BLACK = vec4(0.0, 0.0, 0.0, 1.0);
vec4 WHITE = vec4(1.0, 1.0, 1.0, 1.0);
vec4 TRANSPARENT = vec4(0.0, 0.0, 0.0, 0.0);
vec4 RED = vec4(1.0, 0.0, 0.0, 1.0);
vec4 GREEN = vec4(0.0, 1.0, 0.0, 1.0);
vec4 BLUE = vec4(0.0, 0.0, 1.0, 1.0);

float PI = 3.14159265358979323846;
float TWO_PI = 6.28318530717958647692;

// Functions:

// Clamp value between 0.0 and 1.0
float saturate(float value)
vec2 saturate(vec2 value)
vec3 saturate(vec3 value)
vec4 saturate(vec4 value)

// Remap value from range [start1, stop1] to [start2, stop2]
float map(float value, float start1, float stop1, float start2, float stop2)
vec2 map(vec2 value, float start1, float stop1, float start2, float stop2)
vec3 map(vec3 value, float start1, float stop1, float start2, float stop2)
vec4 map(vec4 value, float start1, float stop1, float start2, float stop2)

// Remap value and clamp to output range
float map_clamp(float value, float start1, float stop1, float start2, float stop2)
vec2 map_clamp(vec2 value, float start1, float stop1, float start2, float stop2)
vec3 map_clamp(vec3 value, float start1, float stop1, float start2, float stop2)
vec4 map_clamp(vec4 value, float start1, float stop1, float start2, float stop2)

// Get the max/min component of a vector
float max3(vec3 value)
float min3(vec3 value)

// Color space conversions and utilities
vec4 colorFromIntRgba(uint iColor)
vec4 colorFromIntAbgr(uint iColor)
vec3 linearize(vec3 gColor)
vec4 linearize(vec4 gColor)
vec3 gamma(vec3 lColor)
vec4 gamma(vec4 lColor)
vec4 premultiply(vec4 color)
vec4 depremultiply(vec4 color)
float grayscaleValue(vec3 lColor)
vec4 grayscale(vec4 lColor)
vec3 grayscale(vec3 lColor)
vec3 rgbToHcv(vec3 c)
vec3 rgbToHsv(vec3 c)
vec3 rgbToHsl(vec3 c)
vec3 hueToRgb(float H)
vec3 hsvToRgb(vec3 hsv)
vec3 hslToRgb(vec3 hsl)
float adobeLum(vec3 C)
vec3 adobeClipColor(vec3 C)
vec3 adobeSetLum(vec3 C, float l)
float adobeSat(vec3 C)
vec3 adobeSetSat(vec3 C, float s)

// Perlin noise functions

// Returns a 2D/3D/4D perlin noise value at position P
// Value returned is in the range [-1, 1]
float perlin_noise4(vec4 P)
float perlin_noise3(vec3 P)
float perlin_noise2(vec2 P)

// Repeated/tiled 2D noise, with the tiling boundary R
float perlin_noise2(vec2 P, vec2 R)

Converting Conventional ISF Effects

ISF files typically come in one or two files, .fs file, and a .vs file. We do not currently support the .vs vertex shader portion of ISF, but for most effects, the .vs file can be ignored.

The fragments shader file has the JSON input specification inside of a /* ... */ comment block at the beginning of the file. The JSON needs to be removed from the comment block and placed in a .liteffect file with the same base name as the fragment shader.

If you experience any errors, check to see if the effect is using any feature listed above.