Gesture visuals that remember the note

Do not collapse the performance too soon

An MPE performance can contain several simultaneous expressive gestures. If all incoming pressure messages are immediately reduced to one number, the last moving finger may control the entire image. That may be a useful artistic choice, but it is different from showing independent voices. Decide which relationship you want before designing the visual patch.

Roger Linn’s developer guidance describes the channel-based organisation of MPE expression. TouchDesigner’s MIDI documentation explains how channel prefixes and selected channel ranges affect the incoming CHOP channels. Preserve the distinctions your visual design needs instead of merging channels simply to make the network look smaller.

Start with two notes and two plain shapes. Hold one note while moving the other. Confirm that the intended shape responds and the other remains stable. This small test is more informative than a colourful field driven by many gestures at once, where accidental coupling can hide inside the activity.

Give a note a visible life

Define the visual states of one note: arrival, sustained gesture and departure. Velocity might influence the entrance; pressure might shape the sustained form; release might let it fade. These are design suggestions, not obligations imposed by MPE. Choose mappings that fit the instrument’s sound and the performer’s movement.

Decide what happens when a channel is reused for another note. The old visual object may still be fading when a new gesture arrives. A simple implementation can accidentally make the old object jump to the new note’s position or colour. Keep the identity of the visual event separate from the current channel state when overlapping releases require it.

Now define what shared controls do. A pedal or overall intensity control might influence every visible object, while a per-note slide affects only one. Make that hierarchy legible in the image. Independent movement should remain visible enough that the audience can perceive the relationship between a hand gesture and its result.

Test note release while pressure is high, a new note during a long fade and several simultaneous releases. Also test the deliberate reset action. A useful visual instrument needs a way to clear abandoned state, just as a synth needs a way to release stuck notes.

Choose a meaningful reduction

Sometimes a single shared image is preferable. In that case, choose how several gestures combine. Maximum pressure emphasises the strongest touch; average pressure describes a different relationship; a count of active notes describes density. These are different compositions of the same performance data. Compare them rather than choosing whichever value is easiest to obtain.

Add smoothing only after the combination rule is clear. Smoothing can make the image feel more physical, but it cannot repair an inappropriate rule. A last-message mapping remains last-message mapping even when its jumps are softened. Keep a view of the unsmoothed signal while adjusting the response.

Record a short passage with one independent visual per note, then a version using your chosen shared reduction. Watch each alongside the hands if possible. Ask what a viewer can infer about the performance. The aim is not to display every MIDI detail; it is to make a musically useful relationship visible.

Save the mapping with a short explanation of note identity, shared controls and release behaviour. These decisions are the substance of the visual instrument. A sophisticated shader can be added later, once the basic correspondence between touch and image is clear enough to survive without decoration.