Piano Capture is our term for our audio recognition and transcription technology, designed to transcribe real, acoustic piano recordings, opening up a whole new way of putting your musical ideas into StaffPad.
Note
Piano Capture requires at least StaffPad version 1.6 for iOS or 3.7 for Windows. Make sure you are fully up-to-date to use these features!
To get started with Piano Capture, add an instrument staff into your score. Open the staff control menu by tapping the staff name on the left side of the screen, then record-enable the staff by tapping the Record toggle at the bottom of the menu. This will inform StaffPad which instrument (if you have multiple in the score) is ready for transcription.
If you haven't already, now's a good time to set your tempo and activate the metronome. By long-pressing the metronome icon, you'll get some options, including a new click sound - the Digital Click. You can set the click volume as well as how many bars of count-in you'd like before the recording starts.
Now we're ready to transcribe our playing into StaffPad. Once your instrument is ready to record, tap the Record icon in the Tool Bar. After your selected count-in, StaffPad will begin to record your playing.
When recording onto a staff, you'll see that a second or so later some notes pop on to the staff. The duration of these notes will refine as you continue to record, and may continue to adjust themselves after playing. The AI listens as you go, but then has another pass at the full recording once you've pressed stop. On older devices, you may find that the staff lines pulse red for a little while after recording; this is to indicate the AI is still working.
Much like any audio recording, the space you record in matters as well. If the room is noisy, or if you're playing quietly, or if the device mic isn't positioned close enough to the sound source, that may affect recognition. Piano Capture in StaffPad is designed to use your device's internal microphone, so external audio devices and interfaces may not work as well.
Recording Piano Audio
Our transcription AI is modeled on an acoustic piano, and works best with that. Due to the way electric pianos and keyboards already synthesize the piano sound, the recognition may not be as precise. Most electric pianos do have MIDI out, however, so you can also try plugging it in and using it as a MIDI controller instead.
This transcription technology uses your device's microphones and on-device machine learning to listen to your piano performance and transcribe it into notation. You can do this in near real-time by enabling record on any instrument staff, pressing record, and then simply playing your (real) piano. If you're overdubbing parts (that is, recording on top of existing music), then make sure you connect headphones - otherwise the sound from the speakers will interfere with the transcription!