Research · Audio ML
Guitar Pedal AI
An audio-to-control system that listens to processed guitar and works toward reconstructing an editable effect chain instead of generating an opaque replacement waveform.
Project overview
This final-year Berklee project investigates the inverse problem behind guitar tone: given a wet guitar recording, can a model infer effect structure and controls that a musician can inspect, edit, and reuse? The research evolved through several architectures as I studied where musical content, dry-signal variation, and effect identity become entangled.
The research includes a DDSP direction for reconstructing delay-line-based effects and a separate audio-to-program training pipeline. The broader aim is to combine machine learning with explicit DSP modules so that the output remains useful inside a producer’s workflow.
V4: audio to DSP program tokens
The V4 pipeline adapts Whisper Base to stereo guitar features and a 3,206-token DSP vocabulary. It learns to predict program tokens from processed audio, with program-group data splits, feature caching, checkpoint recovery, and model export.
The bilingual source guide follows all 15 stages, explaining the logic before showing the corresponding implementation. It also distinguishes teacher-forced token accuracy from free-generation performance. V4 is a training framework; the guide does not run model inference in the browser.
Explore the V4 training guide → 中文指南 →
Project demonstration
Guitar Pedal AI Demo
Watch the recorded project demonstration.