AI turns songs into sheet music and MIDI
Science & Technology
Introduction
In a groundbreaking development, researchers at Stanford have released an open-source neural network capable of transcribing songs into MIDI files and sheet music. This remarkable tool showcases how automation and AI continue to infiltrate creative industries, including music.
The Technology
The software, named Sheet Sage, allows users to fetch audio directly from platforms like YouTube or from local MP3 files. It operates with impressive efficiency, capable of running even on low-spec hardware, such as a potato or a basic calculator. However, a more demanding version exists that requires a powerful Nvidia GPU with a minimum of 10 to 12 GB of VRAM for optimal performance.
Upon entering a song's URL—as demonstrated with the song "I Miss You"—the program begins by retrieving the audio and detecting the beats. Once transcribing is complete, users receive outputs in the form of a PDF containing the sheet music and a MIDI file representing the musical piece.
Performance Insights
Initial tests have shown that Sheet Sage delivers commendable results. It accurately captures song harmonies and displays substantial improvements compared to prior models. However, it does face challenges, particularly with songs that have complex features such as key changes and varying tempos.
In practical examples, the program produced accurate harmonies for “I Miss You” but struggled slightly with the melody. Users can experiment with various genres beyond just pop music, although previous implementations leaned heavily toward pop songs.
Future Potential
Despite some limitations, the program is viewed as a significant advancement in music transcription technology. Researchers note that it represents a considerable leap, being around 20 to 30 percent more accurate than earlier models. The accessibility of such technology has sparked excitement about future developments and possibilities in automating musical transcription, hinting at even more sophisticated tools to come.
Keywords
- AI
- Music transcription
- Sheet music
- MIDI files
- Open-source
- Sheet Sage
- Stanford researchers
- Automation
- Neural network
- Key changes
- Tempo changes
FAQ
Q: What is Sheet Sage?
A: Sheet Sage is an open-source neural network developed by Stanford researchers that can transcribe music into sheet music and MIDI files.
Q: How does it work?
A: Users can provide a song URL (from platforms like YouTube) or an MP3 file. The program fetches the audio, detects beats, and transcribes the song.
Q: What types of music can it transcribe?
A: While primarily trained on pop songs, it can be used for various genres, though its performance may vary.
Q: Does it require special hardware?
A: It can run on low-spec hardware, but a more demanding version is available that functions best with a powerful Nvidia GPU.
Q: Is this technology accurate?
A: Initial results have shown that it is about 20 to 30 percent more accurate than previous models, but it may struggle with songs featuring complex musical elements.