An instrument that starts with the music

Try the interactive sketch

This is a simplified 6-style Web Audio sketch of MusicLab's core loop running directly on this page (the full 78-arrangement standalone studio adds for the complete circle, live MusicXML 4.0 sheet notation and 32 MB sampled instrument bank). Pick a starting card or press Listen to hear the band within one beat, use the persistent Style, Blend and Spectrum bar on any step of the six-step studio wizard to morph the groove, play a tune on the eight scale keys (A through K, with home chord tones marked and Z / X to shift octaves), jump to 6. Chorus & Arrange to shape a returning chorus without overwriting your notes, or drag directly on the 16-step timeline to scrub playback and move your notes in pitch and time.

This sketch does not save or export songs. Your notes do not transfer to the full studio. Keep this tab open to keep playing your sketch; reloading or leaving the page loses it.

What if the music came first, and the instrument second?

Watch a child hear a song they like. They move to it before anyone explains a beat. Then they meet the instrument, or the app, and it asks for things they do not have yet: years of finger practice, note names, a screen of controls. Many stop there. That does not mean they are not musical. It means the tool does not fit them.

I saw a version of this years ago working with people with special needs. They loved music, often through dance, and wanted a sound to respond to them. I have my own version too: more than a year of guitar and piano lessons, and I still cannot read music. Fruity Loops was the first tool that let me put sounds together and feel that the result came from me.

MusicLab is my attempt to give more people that way in. It starts with music that is already playing, and anything you do joins it.

How the full instrument works

In the full MusicLab application (inspected at commit d0a8d7a, November 3, 2026), the genre circle and song builder form one continuous route (Find, Play, Shape, Detail, Share) so the music keeps playing between stages:

1. Find across 78 arrangements. Press Listen and a band begins in 303 to 351 ms (319 ms median across 12 quiet runs on the built release package), using native Web Audio stand-in voices that hand over cleanly at the bar line once the 32 MB sampled bank finishes loading. Drag around the circle, use the arrow keys, or open Choose without dragging to steer across 74 named style interpretations in ten families plus four foundational arrangements, sized so the scale keys stay above the fold. The Ska, Rocksteady and Reggae sector supports a continuous scored blend; a separate six-region field connects Reggae, Rocksteady, Offbeat, Pop, Funk and Dance.
2. Play across eight scale keys and see what the band hears. Eight scale keys (A through K, with Z and X to shift octaves) climb one octave of the active mode with chord tones visibly marked as home notes, alongside four drum pads and pictured instruments including Muted guitar and Upright bass. Pressing a key while stopped starts the band automatically. When I first played the instrument myself, the band was reading my note density, held durations and pitch lean in code, but I could not feel the connection because the read was invisible. Now the instrument shows both sides as they happen: a live heard meter under the keys, a hold ring on held pads, and a direction glow on the circle show what the band is reading while you play, and a picture receipt, lit adopted notes with a bar count, and a melody connector show how the band answered at the bar line.
3. Make my song, Deal Adjust cards, or tap Make it nicer. When you hear something you like, Make my song captures up to 24 contiguous bars (eight four-bar parts) of realized live playing across rests longer than four bars and across Stop and restart (including held notes and the unfinished bar, while excluding future scheduled attacks). If you played an even longer session, earlier notes are saved as a separate song rather than dropped, and Your songs appears directly on the starting screen when you return. On every level after Find, Make it nicer nudges your notes to the nearest chord tone, aligns off-grid slips while keeping deliberate swing, resolves overlaps, brings the final note home, and gives the arrangement a calmer start and real ending, always with one-tap Undo. Three applicable Adjust idea cards are also dealt on arrival so you can audition an idea in place.
4. Shape, open the 3-tile Detail hub, read the sheet, scrub the timeline and export. A companion sheet of music writes itself on treble, bass and percussion staves alongside a four-band part visualizer (with a Bars / Line / Off switch, a What is this? plain-word colour and shape legend, and a single Help control that reads the screen aloud). Pressing Detail opens three clear tiles (Beat, Tune and Backing) while the song keeps playing. You can shape parts, drag anywhere on the whole-song timeline to play from there, drag notes directly in pitch, time and length while hearing them live, download or reopen a standard MusicXML 4.0 score (Download sheet) or portable .json project file, and export a stereo .WAV file directly in the browser.

Everything runs locally in the browser on your own computer. There is no account, and nothing is uploaded.

The band follows you

The part I care most about is the transitions. In Banjo-Kazooie (Rare, 1998), Grant Kirkhope's music changes instrumentation as you move between areas, so the tune feels continuous while the world around it changes. MusicLab applies the same idea to a person's own playing: the band can move from bluegrass toward ska, reggae or soul while the notes you contributed stay yours. The idea is borrowed; no music from the game is used.

What stays yours

• No generated audio. MusicLab does not use AI to generate music or sound. The band is built from written musical rules, and every sound responds to something you chose. I built MusicLab by directing AI coding agents; the instrument itself makes no model calls.
• Your notes are labelled. Each note records where it came from (note_origins): played or placed by you, shaped by you, or supplied by the band. Changes transform your notes rather than replacing them, the original stays recoverable, and saving never depends on how many notes you wrote. Anything you make with MusicLab is yours.
• Safe, deliberate sound. Sound starts only when you press something, a −3 dBFS volume ceiling and soft starts keep it comfortable, Escape stops everything, and a Calm setting removes moving lights.
• Checked invariants (commit d0a8d7a, November 3, 2026). The release-one candidate runs 804 JavaScript tests and 130 Python tests (including 22 subtests) plus a 29-gate release cycle on a clean git archive and the built release package behind a nonce-verified server. The gates cover Python and JavaScript compiler parity, −3 dBFS peak ceilings, 3,168 Ska–Rocksteady–Reggae resource allocation cases, MusicXML 4.0 round-trip score engraving, whole-song timeline scrubbing, live note dragging, one-tap Make it nicer optimization, the three-tile Detail hub, 18 visible band-answer checks paired with 7 negative-control probes (each proving the gate fails if a receipt, note-lit rule or Calm suppression is broken), a 78-of-78 style sweep (play, Make my song, return, play a second long take, Make my song again, reopen, and draw the sheet without losing notes or overwriting an earlier song), three consecutive 29-check two-switch scanning runs, and Chrome and Edge desktop and phone-viewport browser journeys with zero page errors. In the separate verifier walk across 29 end-to-end routes, every route kept all played notes once Make my song was pressed, and the one unkept-jam finding (reloading during Jam before pressing Make my song dropped unkept notes without warning) was repaired in d0a8d7a with a browser leave warning verified across live and stopped Jam takes. These check technical contracts, not human enjoyment.

What comes next: one camera over every plane, and the bass room

The interactive sketch at the top of this page previews two post-release milestones planned for Release Two in the standalone application:

• One camera over every plane (Phase 16, planned). Today the full studio's style wheel steers the whole-song jam. Phase 16 builds a shared scope token and reversible section transaction (prototyped on branch spectrum-chorus/phase-a) so you can zoom from the whole song into a single part without losing personal notes. It adds Make it come back (marking a repeating chorus card with a versioned per-card band gain in both compilers, rather than an automatic pitch lift), a compact style circle inside the selected part tile, and optional melody variations on a copy with the original kept beside it.
• The bass room (Phase 17, planned). Inspired by hearing a fat, funky bass thumping through enclosed corridors before a door opens, Phase 17 plans upper-harmonic bass voicing that survives small laptop and phone speakers, hearing the next style's bass and kick muffled through the wall as you move toward a new style on the circle before the bar-line handover, and letting your own played rhythm become the bass groove, all within the −3 dBFS ceiling.

Release status

The standalone application is in development and is not yet released for public use. Until then, you can try the 6-style sketch at the top of this page.

Source code is already available in the public repository (MIT licence).

If you would like to try it with a child, a student, or anyone who has been told they are not musical, I would like to hear what worked and what got in the way: jon@simoneresearch.com.

Limits

• It has not yet been observed with the beginners it is meant to serve. Children, disabled people and screen-reader users are design targets, not validated audiences.
• The supported routes are keyboard, pointer and two-switch scanning in the browser, verified in automated desktop Chrome and Edge runs (including a phone-sized viewport). Safari, Firefox, physical phones, physical switch hardware, eye-gaze, screen readers and speaker output have not yet been tested with participants.
• Past 24 bars of live playing, empty four-bar rest blocks drop out so no played note is lost, which compresses long silences between phrases; earlier notes beyond the 24-bar window are saved as a separate song.
• While the circle reaches 78 arrangements, only the Ska–Rocksteady–Reggae arc and the separate six-region field use continuous scored interpolation; the remaining circle sectors use bar-boundary handovers. Also, while Band answers me echoes recent phrases and next-bar lead anchoring is active, persistent multi-bar melody development across style changes is not yet complete.
• The styles are scoped musical interpretations written for the app, not complete simulations of each genre or of acoustic ensembles. It is not therapy and makes no claim about learning or health outcomes.

Credits

I designed MusicLab and directed the team (the AI coding agents I direct) that wrote and tested the code. The optional sampled instruments use SpessaSynth (Apache-2.0) and the GeneralUser GS sound bank by S. Christian Collins under its own licence. The transitions are inspired by Grant Kirkhope's adaptive music for Banjo-Kazooie, and other features by Chrome Music Lab, BeepBox and Mutable Instruments' open designs. From those four, only ideas were borrowed, not code, music or recordings. MusicLab is not affiliated with or endorsed by any of the projects named here.