Klangraum
A map of music you can play.
- Role
- Exploratory tool research, interaction & UI design, prototyping, AI-assisted development
- Team
- Solo
- Year
- 2025–26
- Context
- Bachelor thesis + continuation, HfG Schwäbisch Gmünd
- Outcome
- A working web app: the music is generated live as you move.
- Tools
- Google Lyria RealTime
- Gemini
- TypeScript
- Lit
- Web Audio
- Canvas + Web Worker
Images 8
A phone-first web app where you drag a map of 19 genres and 190 subgenres under a fixed needle, and Google's Lyria model generates the music live as you move. An exploratory design project; no formal user study.
Klangraum is a map of music. 19 genres are countries, 190 subgenres are regions inside them. An orange needle sits in the middle of the screen; you drag the map underneath it and the music follows. Nothing is pre-recorded: Google’s Lyria model generates it live. In the middle of Jazz you hear jazz. On the border with Soul you hear an exact 50/50 blend, and the screen says “in the borderland of Jazz and Soul”. Double-tap a country to explore its subgenres, such as Bebop or Cool Jazz.
One drag from Jazz across Soul into Funk. Recorded from the current build (interface in German, audio off), played at 2× speed.
Exploratory design project. No formal user study was conducted; the outcomes below reflect my own exploration, not validated user effects.
The problem
Music can feel immediate without instrumental training, but expressing a musical intention usually takes a production workflow. My question: could you play a whole musical experience as an instrument, shaping AI-generated music by listening and experimenting?
An exploratory comparison of twelve AI music tools (not a benchmark) pointed to the timing of input: changing a description before generation is different from steering music while you listen. Real-time systems existed before, such as MusicFX DJ (Google DeepMind, 2024). My contribution explores spatial interaction and the move between listening and shaping.
What I built
I prototyped eight interaction approaches. Text → music meant every change was a new track: you compare results instead of shaping sound. Letters → drum patterns were transparent but too small a space. Body data → music (“Auditory Self”) was dynamic, but there was nothing to steer. A live layer mixer was the first time music felt shapeable in the moment. Music as positions in space was direct, but you got lost without landmarks.
Three principles came out of this, still to be evaluated with users: continuity (sound never stops while you change it), consistency (changes morph instead of cutting) and orientation (an open space needs landmarks).
The thesis version (winter 2025/26) arranged genres as a ring around a fixed crosshair; dragging the ring changed the mix by distance.
After submission I rebuilt Klangraum phone-first in five days in September 2026, testing on my own phone after every step. I worked with an AI coding agent (Claude): I wrote the briefs and made every design call; the agent implemented. That came to 121 commits across more than 30 documented iterations. The ring didn’t scale (19 genres × 10 subgenres don’t fit on a circle, and a ring has no neighbourhood), so the map became the product.
Key decisions
01
Borders instead of distances.
- Problem
- blending by distance to genre centres felt random; some genres dominated.
- Decision
- every genre owns a territory. The mix depends on closeness to a border: 100% deep inside, 50/50 on a border, ⅓ each where three meet. The blend zone keeps the same on-screen size at any zoom.
- Result
- what you see is what you hear.
02
The sound follows slower than your finger.
- Problem
- a fast swipe made the music jump like a skipping record.
- Decision
- a second point, the sound needle, trails the finger at limited speed. Crossing a border takes 8 seconds.
- Result
- a fast swipe sounds like a DJ transition. A status line shows what you hear now (“Sound: Jazz 70% · Soul 30%”) versus where you’re heading.
On the border: Jazz 58% · Soul 42%. The white ring is the sound needle, trailing the orange finger position.
03
Neighbouring genres share words.
- Problem
- switching between genre prompts produced audible breaks.
- Decision
- every genre is described in one grammar (instruments, moods, era, tempo), split into weighted blocks. Neighbours share vocabulary (Jazz and Soul both contain “Rhodes piano” and “warm”), and shared blocks stay active across the border. A script enforces this.
- Result
- crossing a border morphs the music instead of switching it.
04
Six map styles until orientation worked.
Islands, a contour map (my note: “no orientation at all, chaotic”), a political map, a halftone grid, and finally a colour field of dots whose size shows how pure a genre is at that point. Rendering moved to a background thread so dragging stays fluid on a phone.
The final style, zoomed out: 19 genres as colour fields of dots.
05
Entering a genre is a deliberate step.
- Problem
- subgenres appearing while zooming was disorienting.
- Decision
- zoom only zooms. Double-tap enters a territory, the rest dims, and a breadcrumb ”‹ Jazz” leads back out.
Inside Jazz: ten subgenres, one step back out.
06
Dragging should feel heavy.
- Problem
- dragging felt twitchy.
- Decision
- a small dead zone, smoothing, inertia and a soft snap onto genre cores, so the map should “either glide or slide over quickly, never jerk.”
What I cut
- A vector pad and a two-deck DJ mode, both fully built: “pad and decks work badly, the map concept works best.” 2,200 lines deleted.
- A two-parallel-sessions trick against buffering. Every switch sounded like a new track.
- An AI-computed genre layout from embeddings. It improved neighbour similarity by only 8% and tore Folk and Country apart.
Outcome
- A working phone-first web app that turns genre exploration into movement through a space. On a buffer underrun the last chunk loops quietly instead of cutting to silence; sessions hand over with a crossfade before Lyria’s 10-minute limit; the last 60 seconds can become a full song with vocals.
- A 73-page thesis exploring timing and understandable musical interaction. It does not establish that these factors matter more than model quality.
- A process where every change traces back to a problem, a decision and a result.
A planned evaluation would test understandable control, intentional changes, and switching between listening and shaping. No results exist yet.