I Built An ESP32 MIDI Controller Into My Guitar — Play Guitar And MIDI At The Same Time

I wanted to build a MIDI controller that did not require keys, pads, knobs, or a separate device sitting next to me. The idea was to attach an ESP32 MIDI controller directly to my guitar and use hand movement as the input. Four ultrasonic sensors measure how close my palm is to them, the ESP32 converts those distances into MIDI notes, and Bluetooth Low Energy sends the notes wirelessly to Ableton Live.
The guitar itself still works exactly like a guitar. Its audio signal can enter Ableton through my audio interface while the ESP32 independently sends MIDI notes. That means I can play a guitar part and, during the same performance, move my hand near the sensors to trigger a synthesizer, piano, pad, bass, or another virtual instrument.
After getting the prototype working, I soldered the complete circuit and mounted the electronics and sensors onto the guitar. The result is one physical performance setup with two independent musical inputs: guitar audio and wireless MIDI.
Key Takeaways
I built an ESP32 MIDI controller with four HC-SR04 ultrasonic sensors and physically attached it to my guitar.
The guitar continues producing its normal audio signal while hand movements near the sensors independently trigger MIDI notes in Ableton Live.
Four sensors provide eight MIDI notes using two distance zones per sensor, while BLE MIDI eliminates the need for a MIDI cable.
The finished build is soldered and battery powered, but getting there required solving unstable ESP32 uploads, ultrasonic sensor wiring, voltage-level conversion, and an unusual ground issue in the final assembly.
What I Actually Built
It is important to separate the two parts of this project.
I did not convert the guitar strings into MIDI. The ESP32 does not listen to the guitar, detect its pitch, or translate guitar notes into MIDI. Instead, I attached an independent gesture-controlled MIDI controller to the guitar.
The complete signal flow is:
GUITAR AUDIO
Guitar
│
▼
Audio Interface
│
▼
Ableton Audio Track
MIDI CONTROLLER
Palm Movement
│
▼
HC-SR04 Sensors
│
▼
ESP32
│
▼
BLE MIDI
│
▼
Ableton MIDI TrackBoth paths operate at the same time.
I can therefore strum or pick the guitar while moving my hand toward one of the ultrasonic sensors to trigger another instrument.
That was the main goal of the project: not to replace the guitar, but to add another playable layer to it.
How The MIDI Controller Works
The controller has four HC-SR04 ultrasonic sensors mounted where I can reach their detection zones with my hand while using the guitar.
Each sensor provides two MIDI notes.
Sensor | Palm At 2–10 cm | Palm At 10–20 cm | Above 20 cm |
Sensor 1 | C4 | D4 | Note Off |
Sensor 2 | E4 | F4 | Note Off |
Sensor 3 | G4 | A4 | Note Off |
Sensor 4 | B4 | C5 | Note Off |
That gives me eight notes from four sensors.
For example, moving my hand close to Sensor 1 sends C4. Moving farther away changes it to D4. Moving outside the active area stops the note.
The same principle applies to the other sensors.
This gives the guitar an additional performance interface without changing the guitar's normal sound.
Why I Chose ESP32
ESP32 is particularly well suited to this project because Bluetooth Low Energy is built into the platform.
A basic microcontroller could read the HC-SR04 sensors, but ESP32 can handle all of the following on one board:
Four ultrasonic sensors
OLED display
Distance processing
MIDI note logic



