Shader Pipeline Reference

The observation pipeline consists of 5 fragment shader passes executed sequentially via framebuffer ping-pong. Each pass reads from the previous pass's output texture and writes to the next.

Encode→Partition→Interference→Entropy→Display

Pass 1: Encode

encode_microscopy.glsl

Converts the raw image into partition feature space. Computes luminance, local mean and variance over a 3x3 neighbourhood, gradient direction, and Shannon entropy from a 4-bin local histogram. Outputs are quantised to the range [0, n_max].

Inputsu_image (raw image texture), u_nmax

Outputsvec4(n, l, m, s) — partition features per pixel

float I = luminance(texture(u_image, v_uv).rgb);
float mean, variance;
neighbourhood(u_image, v_uv, texelSize, mean, variance);
float H = localEntropy(u_image, v_uv, texelSize);
float n = floor(clamp(I * u_nmax, 1.0, u_nmax));
float l = clamp(sqrt(variance) * u_nmax, 0.0, n - 1.0);
fragColor = vec4(n / u_nmax, l / u_nmax, m_norm, H);

Pass 2: Partition

partition.glsl

Computes the invisible pixel via the oxygen model. Uses Beer-Lambert to estimate O2 concentration from image intensity. Solves the Einstein rate equations at steady state to compute the three ternary probabilities: P0 (ground/reference), P1 (absorbing/detector), P2 (emitting/source). Combines with the visible partition coordinates from pass 1.

Inputsu_image (raw), u_pass1 (encode output), u_Aeg, u_nmax

Outputsvec4(n_vis, P0, P1, P2) — visible partition + ternary O2 states

float OD = -log(max(I, 0.001) / I0);
float O2conc = clamp(OD / 2.0, 0.0, 1.0);
float u_nu = I * 0.5;
float Bge = u_Aeg * 2.0;
float Beg = u_Aeg * 1.5;
float denom = u_Aeg + (Beg + Bge) * u_nu;
float P2 = (Bge * u_nu) / max(denom, 1e-6);
float P0 = u_Aeg / max(denom, 1e-6);
float P1 = 1.0 - P0 - P2;

Pass 3: Interference

interference.glsl

Cross-modal consistency check between visible and invisible modalities. Computes inter-modality correlation over a 5x5 neighbourhood, dual-pixel consistency (whether the two paths agree within threshold epsilon), coupling strength via Boltzmann-weighted energy, and local phase coherence from gradient alignment.

Inputsu_pass1 (visible), u_pass2 (invisible), u_epsilon, u_J, u_beta, u_nmax

Outputsvec4(correlation, consistency, coupling, coherence)

float corr = interModalCorr(v_uv, texelSize);
float vis_n = texture(u_pass1, v_uv).r;
float inv_n = texture(u_pass2, v_uv).a;
float diff = abs(vis_n - inv_n);
float consistent = step(diff, u_epsilon) ? 1.0 : 0.0;
float E = -u_J * corr;
float coupling = exp(-u_beta * E);
fragColor = vec4(corr, consistent, coupling, coherence);

Pass 4: Entropy

entropy.glsl

Computes the three entropy coordinates and enforces the conservation law. S_k (kinetic) from gradient magnitude measures translational disorder. S_t (thermal) from local 5x5 variance measures configurational disorder. S_e (emission) from the ternary state probabilities measures information in the invisible channel. The three are normalised so S_k + S_t + S_e = 1.

Inputsu_image, u_pass2 (ternary), u_pass3 (consistency), u_alpha, u_nmax

Outputsvec4(S_k, S_t, S_e, conservation)

// Kinetic entropy: gradient magnitude
float gx = I_right - I_left;
float gy = I_up - I_down;
float grad = sqrt(gx*gx + gy*gy);
float Sk = clamp(grad * 4.0, 0.0, 1.0);

// Thermal entropy: local variance
float var = localVariance(v_uv, ts);
float St = clamp(sqrt(var) * 3.0, 0.0, 1.0);

// Emission entropy: ternary information
float Se = clamp(ternaryEntropy * u_alpha, 0.0, 1.0);

// Normalise to conservation law
float total = Sk + St + Se;
fragColor = vec4(Sk/total, St/total, Se/total, abs(total - 1.0));

Pass 5: Display

display.glsl

Renders any intermediate or final pass as a false-colour image for human inspection. Uses the inferno colourmap for scalar quantities and custom colour mapping for vector quantities. Switchable via u_pass uniform: 0 = partition depth (cyan-magenta), 1 = ternary states (RGB), 2 = consistency (green/red), 3 = entropy triple (RGB mapped to S_k/S_t/S_e).

Inputsu_tex (any pass output), u_pass (which pass to visualise)

Outputsvec4(r, g, b, 1.0) — false-colour visualisation

vec3 inferno(float t) {
  // Perceptually uniform colourmap
  const vec3 c0 = vec3(0.0002, 0.0016, 0.0140);
  const vec3 c5 = vec3(0.9882, 0.9922, 0.7490);
  // ... 6-segment piecewise linear interpolation
}

if (u_pass == 0) {
  fragColor = vec4(mix(cyan, magenta, tex.r), 1.0);
} else if (u_pass == 3) {
  fragColor = vec4(tex.r, tex.g, tex.b, 1.0); // S_k=R, S_t=G, S_e=B
}

Uniforms Reference

All uniforms used across the pipeline. Each shader only binds the uniforms it needs — unused uniforms are silently ignored by WebGL2.

Name	Type	Passes	Description
u_image	sampler2D	1, 2, 4	The original input image texture
u_pass1	sampler2D	2, 3	Output of encode pass (partition features)
u_pass2	sampler2D	3, 4	Output of partition pass (ternary states)
u_pass3	sampler2D	4	Output of interference pass (consistency data)
u_tex	sampler2D	5	Any pass output for display visualisation
u_nmax	float	1-4	Maximum principal quantum number (default: 8)
u_epsilon	float	3	Consistency threshold (default: 0.15)
u_J	float	3	Inter-modal coupling constant (default: 1.0)
u_beta	float	3	Inverse temperature for Boltzmann weighting (default: 2.0)
u_Aeg	float	2	Einstein A coefficient for spontaneous emission (default: 2.58)
u_alpha	float	4	Information transfer efficiency (default: 0.5)
u_pass	int	5	Which pass to visualise in display shader (0-3)

Framebuffer Layout

The pipeline uses RGBA32F textures for all intermediate framebuffers. Each pass writes to a dedicated FBO, and the output becomes the input texture for the next pass. The final display pass renders to the default framebuffer (canvas).

FBO 0 (encode)       → tex0  [RGBA32F, W x H]
FBO 1 (partition)    → tex1  [RGBA32F, W x H]
FBO 2 (interference) → tex2  [RGBA32F, W x H]
FBO 3 (entropy)      → tex3  [RGBA32F, W x H]
Default FBO (display) → canvas

← Writing Encoders Documentation home