Exploiting Latency in the Design of a Networked Music Performance System for Percussive Collective Improvisation

Musicians on a two-way audio link can tolerate about 75 ms of latency, but over the Internet delay and jitter grow with distance and are far above that. Most networked music performance (NMP) systems either pick styles that do not need tight timing or try to hide the delay. A known trick for cyclic music, which the paper calls the "fake time" approach and which NINJAM uses, goes the other way: impose a tempo, synchronize the clocks, estimate the worst-case end-to-end latency (WCL), and schedule every event to play at least WCL later, rounded to a whole musical cycle. Everyone hears a coherent piece, just shifted in time.

This NIME 2025 paper, by Ari Liloia and Roger Dannenberg at Carnegie Mellon University, builds a percussion jam on that idea and pushes it further. Latency is not only padded, it becomes part of the music and of the picture on screen: the distance between players on a shared grid, measured in beats, is derived from their round-trip times. The system grew out of the same authors' Global Drum Circle (2022), and the code is on GitHub (arililoia-cmu/elbs_nime). For anyone who wants to keep a metronome in sync among remote musicians, the useful part is the timing design: one master clock on a server, a beat timeline every client shares, and delays that are always whole multiples of a beat.

• Three roles on a grid. Composers (red), Performers (blue) and Listeners (green) are nodes on a two-dimensional grid. A key press plays a conga sample locally at once and sends a "drum hit" to the server. A Composer draws connections to Performers; its hit markers travel along them, and the travel time in beats (the procession time) equals the connection's length in grid lines. Performers play along as markers arrive, like a scrolling score, and their hits ripple out as a diamond-shaped wavefront that reaches every Listener after as many beats as the grid distance. A Listener drags its node to hear a different timing mix of all Performers.
• Latency as musical distance. The server estimates each client's round-trip time and places the nodes with a spring layout, so latency between users becomes a delay measured in beats on a "novel musical timescale".
• A beat-aligned hidden delay. On top of the procession time, every action reaches other clients after a delay H of at least WCL that is a whole number of beats, which keeps overlapping output coherent even when several clients play in the same room.
• Client-side prediction borrowed from games. The user's own hits are shown and heard immediately, and actions that need a response are delayed by 2H for the observers, so a Performer's reaction looks instantaneous everywhere.
• Records for later composition. Each Listener's perspective is saved as a MIDI file, with one track per Performer, meant to be opened in a DAW. The first author composed a piece from one test session.

The paper is a design and implementation report followed by one informal test.

Stack and topology. The back end is Java with Maven, built on the Webbit WebSocket and HTTP server. The front end uses p5.js for drawing and the Web Audio API for playback. Messages go over WebSockets using O2lite, Dannenberg's lightweight version of the O2 protocol. The topology is hub-and-spoke: every message passes through a central server, which keeps the master clock, holds the ensemble state, logs the performance and rejects invalid actions. To run a session, one user compiles the application on a virtual machine with a public IP address and shares the link to the browser client.

Clock and beats. A tempo is imposed and maps time to beats. During setup each client synchronizes its local clock and its "timemap" (the time-to-beat mapping) with the server. The paper does not detail the synchronization algorithm itself; it relies on O2lite. Round-trip time is "crudely" estimated with N O2lite messages, the first spaced S ms apart and each gap growing by a factor F. The defaults N = 10, S = 20 and F = 2 take roughly 20 seconds.

Worst-case latency. WCL is configurable and defaults to 3500 ms. The authors justify it from Google Cloud's performance dashboard, whose highest median round trip between a cloud region and an external endpoint was 350 ms, so the default is an order of magnitude above it for safety.

Scheduling a hit. A hit sent at beat b_S is played on a receiver at beat b_S + C, where C beats last at least WCL. The hidden delay is H = xB, with B the beat duration and x an integer (default 4). With H at least 3.5 s spread over 4 beats, the tempo is capped at about 68 BPM. The paper prints the formula as 60 * x / (H * 1000) and then computes 60 * 4 / 3.5 = 68.57; the number is right when H is in seconds. Figure 4 of the paper shows the timeline: played at t_T, received at t_R, displayed at t_V with t_V minus t_T at least WCL, and arriving at the target node at t_E after the procession time.

Grid placement. On the server, clients are masses joined by springs whose rest lengths are proportional to the sum of their RTTs. After M = 1000 iterations, positions are quantized to a G by G grid with G = 16, so a marker crossing the grid in a straight line takes four measures. G also caps a session at G * G users.

Session control. Users log in and pick a role. Once at least one client of each class is validated, they vote to start; a new or dropped client restarts the vote. Every client shows a tempo indicator, a started/stopped indicator, a volume mixer and lights that blink with the metronome. Only Composers can start play or change the tempo. Drum hits, tempo changes, Listener moves and new connections are followed by cooldowns longer than WCL, so each change has time to reach the whole ensemble. A disconnect covers the grid with an interruption window. After the session, the server turns its logs into CSV and one MIDI file per Listener, downloaded over HTTP POST.

Test. One session with three distributed participants who gave informed consent: a video call with an introduction, then three rounds (free play, swapped roles, and a round where some people ran two clients in two browser tabs).

These are the authors' reported, qualitative results. There are no latency or synchronization measurements.

• During tests with distributed users, "no issues suggesting a higher WCL was necessary presented themselves", so 3.5 s was enough in practice.
• The crude RTT estimate "generated a reasonable set of values" for placing the nodes, and every realistic node and RTT configuration tried during development converged within 1000 layout iterations.
• The MIDI files from all three rounds opened without problems "in a range of DAWs". The logs and MIDI files are published under extras/logs in the repository.
• Feedback from the session led to two changes already in the prototype: clients' own nodes are easier to spot on the grid, and the setup instructions are clearer.
• Average setup took slightly longer than the roughly 20 seconds of the RTT probing.
• A complete piece (extras/composition.wav) was composed from the third session's MIDI, voiced with marimba plus atmospheric layers. Two Listeners at different distances from the same Performer produced a "call and response" effect, which is the latency-as-distance idea heard in practice.

The evaluation is informal, and the authors say so in the abstract, the test section and the conclusion. One session with three people supports "it works and people could use it", not claims about usability, musical quality or timing accuracy.

• No timing data. The paper never measures clock offset, jitter, actual end-to-end delay or how often events missed their scheduled beat. The 3.5 s WCL is a generous constant, and "no issues" during a few rounds does not show it holds on worse networks or mobile links.
• The sync mechanism is delegated. The client clock and timemap are synchronized with the server through O2lite, but the paper gives no algorithm or accuracy. A reader building a shared metronome learns the architecture (server master clock, shared tempo map, events scheduled on future beats, beat-aligned delays, cooldowns on state changes) but must look up O2 for the clock sync itself.
• The cost of the trick is tempo and responsiveness. A large beat-aligned delay caps the tempo at about 68 BPM with default settings, and every change, including tempo, waits a cooldown longer than 3.5 s. That fits a slow, cyclic, percussive jam and rules out tight call and response at normal speeds.
• Crude latency estimation. RTT is measured once, at setup, with ten probes, and the authors call the method "crude and not thorough". Conditions that change during a session do not move the nodes.
• Central server. Hub-and-spoke makes logging, the master clock and conflict handling simple, but every message pays the trip through one VM, and the server is a single point of failure.
• Small details. The maximum tempo formula is printed with mixed units, and the third test round says "two users" managed two clients each while listing three people.

The authors suggest:

• More accurate inter-user latency estimation, possibly with an adaptive algorithm or more measurements spread over a longer time.
• A more legible scrolling score for Performers than the grid, based on user feedback.
• More post-session editing features, or modest performance features such as uploading custom drum samples, which would also help users tell each other's parts apart, since today everyone shares the same two conga sounds.

Natural next steps for the systems side would be measuring the actual synchronization error and delivery delay under different network conditions, and adapting WCL and H during a session instead of fixing them at setup.