An AI agent that reverse-engineers keyboard sounds from songs and applies them to hardware synthesizers via MIDI. Powered by Vercel AI SDK's ToolLoopAgent and the Model Context Protocol.
The agent runs an agentic loop over the MCP servers — fetching audio, analyzing it, predicting parameters, and dialing them into your synth.
You name a song; the agent fetches the audio, separates stems, and isolates the keyboard parts.
It profiles the spectral characteristics to spot the sound engine — subtractive, FM, organ, or sample-based.
Synth parameters are predicted via ML-based inverse synthesis or manual spectral analysis.
It applies the settings to your connected MIDI keyboard through keyboards-mcp.
The server holds no state — conversation history lives in the client REPL.
Drives keyboards-mcp and audio-analysis-mcp as MCP servers over stdio.
A stateless HTTP server runs the ToolLoopAgent against an AI Gateway, spawning both MCP servers over stdio.
The ToolLoopAgent handles the agentic loop — tool calls, results, and multi-step reasoning — while the MCP servers provide keyboard control and audio analysis. Bring it up with npm run dev:full and run npm run repl.
The agent ties analysis and MIDI control together into one end-to-end loop.