HOW I BUILT SONUS: AUDIO-REACTIVE WEBGPU SHADERS & 3D CINEMATOGRAPHY

1.1The vgpu.sh Foundation & My Focus on the Orb

When I discovered the Glass Fractal example on vgpu.sh, I was blown away by how clean WebGPU’s direct buffer access was compared to older WebGL boilerplate. But the original example was static.

I wanted to take that foundation and ask: can I transform the core of this into a living, breathing ferrofluid organism? A liquid body that sleeps peacefully in idle state, but erupts into hundreds of viscous magnetic spikes when the bass drops.

1.2The 256-Pole Magnetic Tensor Optimization (WGSL)

My first prototype attempted to evaluate attraction to 1,024 dynamic particle centers. With 32,768 vertices on the sphere, that meant over 33 million distance calculations per frame. Frame rates plummeted instantly.

To solve this, I pre-computed a deterministic Fibonacci sphere containing exactly 256 magnetic pole vectors embedded directly into the WGSL vertex shader memory:

const MAGNET_COUNT = 256u;
const MAGNETS = array<vec3f, 256>(
  vec3f(0.0883020, 0.9960938, 0.0000000),
  vec3f(-0.1125549, 0.9882812, 0.1031095),
  vec3f(0.0171944, 0.9804688, -0.1959219),
  ... // 256 pre-computed Fibonacci pole centers
);

fn ferrofluidSpikeHeight(
  direction: vec3f,
  time: f32,
  audio: vec4f
) -> f32 {
  let bass = saturate(audio.x);
  let mids = saturate(audio.y);
  let energy = saturate(audio.w);

  // Dynamic audio drive combining low-end kicks and harmonic density
  let drive = saturate(bass * 0.88 + mids * 0.70 + energy * 0.32);
  let magnetization = smoothstep(0.02, 0.22, drive) * smoothstep(0.72, 1.0, params.sphereMix);

  // Crucial GPU performance optimization: exit immediately if idle
  if (magnetization <= 0.0001) {
    return 0.0;
  }

  let fieldDirection = magneticFieldDirection(direction, time, audio);
  let poleField = magneticPolesIntensity(fieldDirection, time, mids, bass, energy);

  if (poleField <= 0.005) {
    return 0.0;
  }

  return pow(poleField, 2.2) * magnetization * 0.42;
}

By projecting vertex normals against this magnetic tensor and modulating the pole field with audio drive, hundreds of razor-sharp spikes protrude organically along dynamic magnetic flux lines, maintaining a rock-solid 60+ FPS even during chaotic peaks.

1.3Volumetric Light Rays & The Blue-Noise De-banding Trick

The visual anchor of NODAL is the overhead theatrical spotlight (god rays). Instead of running a costly raymarch loop (which requires 64+ texture fetches per pixel), I designed an analytical two-layer harmonic counter-drifting shaft generator:

// Layer 1: Clockwise macro shafts
let p1 = clamp(
  (0.48 + 0.18 * sin(angle * 32.0 + rayTime * 0.85)) +
  (0.32 + 0.22 * cos(-angle * 19.0 + rayTime * 0.70)),
  0.0, 1.0
);

// Layer 2: Counter-clockwise micro shafts
let p2 = clamp(
  (0.45 + 0.20 * sin(-angle * 48.0 + rayTime * 0.45)) +
  (0.30 + 0.18 * cos(angle * 26.0 - rayTime * 0.55)),
  0.0, 1.0
);

// Interference creates organic smoke-like particulate turbulence
let shafts = pow(p1 * 0.54 + p2 * 0.46, 2.2);

// 128x128 Blue-Noise temporal jitter prevents 8-bit dark color banding
let frameOffset = vec2u(u32(params.time * 60.0) * 13u, u32(params.time * 60.0) * 7u);
let noiseCoord = (vec2u(in.position.xy) + frameOffset) % 128u;
let blueNoise = textureLoad(blueNoiseTexture, noiseCoord, 0).r;
let ditherMultiplier = mix(0.82, 1.28, blueNoise);

Dark gradients on 8-bit web displays suffer from hideous stepping artifacts (banding). Sampling a 128×128 blue noise texture with temporal jitter smooths out the luminance decay seamlessly, producing a velvety falloff into pure black.

1.4Audio Reactivity & The Hardcore Drop in Bellatores

When using Vladimir Cauchemar & Apashe’s track Bellatores, the song builds up to an enormous drop at timestamp 0:58. I wanted the graphics to tell this dramatic story:

// 0:55 to 0:58: Pre-drop tension blackout (all lights fade to pitch black)
if (time >= 55.0 && time < 58.0) {
  blackout = Math.min(1.0, (time - 55.0) / 1.5);
} else if (time >= 58.0 && time < 125.0) {
  // 0:58: The Drop! Blinding strobe impact + 2.4s crimson laser envelope
  const dropElapsed = time - 58.0;
  if (dropElapsed < 0.6) {
    hardcoreFlash = Math.pow(1.0 - dropElapsed / 0.6, 2.0); // Intense white strobe
  }
  hardcoreMode = Math.min(1.0, dropElapsed / 0.8); // Transitions into multi-tonal red
}

At 0:55, everything plunges into darkness. At 0:58, the drop strikes with a blinding white strobe that tears through the god rays, followed by an incandescent crimson laser shockwave that turns the entire environment into a pulsing blood-red cathedral.

2.1Virtual Cinematography as an Instrument

For the second experiment, I wanted to step away from abstract fluid shaders and explore figurative 3D form and automated cinematography.

I chose a classical marble sculpture of an angel. When rendered under static lighting, 3D models can easily feel lifeless or museum-like. But when you treat the virtual camera like a music video director—cutting on beat transients, diving into extreme macro close-ups, and sweeping around wing contours—the cold marble sculpture suddenly pulses with dramatic tension.

2.214 Camera Presets & MathUtils Physical Dampening

Instead of simple spherical orbital math, I composed 14 deliberate cinematic framing presets:

export const CAMERA_PRESETS = {
  hero: {
    position: [-1.75, -1.05, -3.35],
    target: [-0.35, 0.52, -0.22],
    fov: 42,
  },
  dramaticLow: {
    position: [-0.85, -2.40, -2.10],
    target: [-0.20, 0.80, -0.10],
    fov: 48,
  },
  extremeCloseUp: {
    position: [-0.72, 0.48, -1.25],
    target: [-0.38, 0.58, -0.25],
    fov: 28, // Telephoto lens compresses facial depth
  },
  zenithOrbit: {
    position: [0.0, 3.80, -1.50],
    target: [0.0, 0.20, 0.0],
    fov: 52,
  },
};

// Physical spring dampening inside useFrame avoids robotic linear lerps:
camera.position.x = THREE.MathUtils.damp(camera.position.x, targetPosX, 3.5, delta);
camera.position.y = THREE.MathUtils.damp(camera.position.y, targetPosY, 3.5, delta);
camera.position.z = THREE.MathUtils.damp(camera.position.z, targetPosZ, 3.5, delta);

Using THREE.MathUtils.damp provides a spring-like exponential arrival that creates an organic camera deceleration instead of a stiff linear glide.

2.3Baroque Chiaroscuro 3-Point Lighting Rig

To capture the mood of Baroque sculptors like Bernini, I kept the ambient light extremely low (0.16) and placed high-intensity directional lights to sculpt the contours:

• Key Directional Light: Placed at [0.5, 3.2, -3.8] with intensity 2.2, casting deep natural shadows under the angel’s brow and folded robes.
• Rim Light: Positioned at [3.0, 4.0, 3.0] with intensity 1.8 to trace the feathered edges of the wings against the pitch-black backdrop.
• Cyan Fill: A subtle cold tone at [-3.0, 0.8, -1.5] (#90b0d8) giving an ethereal cinematic glow to the shaded marble.

2.4Timeline & Lyric Synchronization

I connected the camera sequencer directly to the timecodes of Vladimir Cauchemar’s Laboratoires. As lyric phrases hit, the timeline fires keyframe triggers that cut the camera to extreme close-ups, activate rhythmic strobe lights, or initiate Dutch tilts.

Meanwhile, the user retains subtle mouse parallax control: you can subtly steer the camera around the angel to inspect the sculpture without interrupting the musical choreography.

3.1Chladni Nodal Resonances: Physical Visible Sound

The name of this repository (chladni-web) honors Ernst Chladni, the 18th-century physicist who discovered that sand scattered over vibrating brass plates migrates away from antinodes and settles into stationary nodal lines of zero displacement.

3.2The WebGPU Compute Particle Roadmap

My ongoing development for the third experiment is a WebGPU compute shader simulation supporting up to 1,000,000 independent physical sand particles. As audio frequencies shift live through a microphone or synthesizer, the particles will dynamically collide, scatter, and settle into intricate geometric mandalas in real time.