Introduction:
In this guide, we'll explore some of my favorite AI applications that leverage the power of artificial intelligence to do insanely cool things with MIDI – from instantly generating original AI music, to converting text and audio into MIDI with the click of a button.
Whether you're a seasoned musician looking to spice up your workflow, or just getting started with MIDI music creation, using AI can provide you with endless inspiration and seriously enhance your process. We'll cover everything from MIDI basics to real-world use cases so you can fully understand how to harness AI's potential.
Table of Contents
1. Background on AI and MIDI
What is MIDI?
What is AI?
How can I create music with AI and MIDI?
2. AI MIDI Generator Tools
Musenet
Sony’s Flow Machines
Google Magenta Studio Plugin
Los Angeles Music Composer
Staccato's AI Instrument
3. Text-to-MIDI Tools
ChatGPT
Audiocipher
Staccato's Text-to-MIDI
4. Audio to MIDI File Tools
Samplab
AI-MIDI.com
Spotify Basic Pitch
Background on AI and MIDI
What is MIDI?
Chances are you're already familiar with the magic of MIDI (Musical Instrument Digital Interface). This technology has been revolutionizing the creation of digital music since it first came onto the scene in the 1980s.
In a nutshell, MIDI is a super convenient digital format that allows computers, synthesizers, samplers, and all kinds of other equipment to seamlessly communicate with each other. This made it possible to control and arrange all the elements of digital music creation.
MIDI is commonly used with digital audio workstations (DAWs) like Ableton or Logic Pro to produce music. DAWs are software platforms that allow for audio recording, MIDI sequencing, and music production.
With MIDI, you can program things like notes, tempo, rhythm, dynamics and more. In many ways MIDI laid the foundation for modern digital music production.
What is AI?
Now, another technology is taking MIDI to unbelievable new heights – artificial intelligence! Artificial intelligence (or AI) refers to computer systems that can perform tasks comparable to human intelligence.
AI is trained using machine learning algorithms that process massive amounts of data. This allows them to recognize patterns, interpret complex information, make predictions – and even generate original music and lyrics!
When combined with MIDI, AI introduces game-changing new ways to make music. If you’re interested in how AI will shape the future of music, check out our article on the future of music composition.
How can I create music with AI and MIDI?
There are some seriously futuristic ways AI can enhance MIDI technology:
AI MIDI music generators: With these tools, you can get AI to generate completely new MIDI music for you with just a click! Some of these tools were trained on thousands of song samples to be able to create original tunes in any style you want.
Text-to-MIDI music generators: This tool is perfect if you have some stylistic ideas but need some melodic inspiration. Simply type in some text and it will turn it into a MIDI composition.
Audio-to-MIDI converters: Upload any audio file, and the tool will convert it to MIDI by detecting notes, instruments etc. This comes in super handy if you have existing audio that you want to edit in MIDI form.
As you can see, the possibilities are truly endless when AI and MIDI join forces. Let's check out some of these unbelievable AI tools for music creation!
AI MIDI Generator Tools
Musenet
While Musenet is no longer available, we believe it deserves a mention in this article because of its widespread popularity during its availability.
Created by OpenAI, the same company that made ChatGPT, Musenet was a popular AI tool to generate musical compositions. The platform had the capability to generate 4-minute compositions with the freedom to customise and combine various instruments and musical styles.
Musenet allowed you to import a MIDI file into the software and then it would generate continuations of the MIDI in the style you chose. The options included Jazz, Pop, African, Indian, and even musicians like Mozart, The Beatles, and Queen.
Once you played around with the options and heard something you liked, you could download the composition to your computer.
Though you can’t use it anymore, you can still check out some of the tool’s creations on the Musenet homepage.
There is some hope for Musenet’s return as Christine McLeavey, the creator of Musenet, replied to a tweet on January 2, 2023 saying that "Musenet will be back soon, but I don't have the exact date yet."
However, it has now been 8 months since Musenet stopped working in December 2022 and it seems less and less likely for it to come back. People speculated that it was taken down to fix a bug, but there are no concrete answers as of yet.
Price: Musenet is no longer available to the public
What people did with it: OpenAI’s Soundcloud with Songs from Musenet, Megalovania Extended by Musenet
Sony’s Flow Machines
Despite the popularity of Sony as a company, ‘Flow Machines’ has stayed rather under the radar as an AI MIDI maker. One of the reasons for this may be because it doesn’t work like the other AI MIDI generators:
Instead of generating continuations of music that you give it, you can make music with Flow Machines by training the AI. The generator then gives you several variations of melodies according to the training.
Here’s how it works:
Set parameters like key, tempo, and scale
Choose the guiding chord progression
Capture melodies (play them on a keyboard) that you want Flow Machines to use as a guide
Flow Machines creates several varieties of AI generated MIDI compositions based on your training
If you have the standalone app (computer or mobile), you can download the MIDI file then load it up in your DAW. If you have the DAW plugin, you can upload it straight into your DAW.
Each of the training sets you make will be saved into “palettes”. You can think of palettes as directions for the AI music generator.
If all of this sounds too complicated for you, Flow Machines also has preset palettes that you can use. There are over 100 presets in genres that include Pop, EDM, Jazz, and more.
Once you have the palette that you like, you can adjust the parameters and even select bars to regenerate.
Flow Machines is a great tool for seasoned musicians who have a good understanding of music and digital music production. This MIDI generator is different from the others because it requires you to have some idea of the music that you want to generate. Despite all of this, Flow Machines is incredibly useful if you can use it right. If you want to see how it is done, check out Ayatake Ezaki’s tutorial.
Price: Free
Level: Advanced
Best for: Experienced musicians
Format: DAW plugin, Standalone for Mac, iPad app, iPhone app
What people did with it: Pop Music Made by Flow Machine’s AI , Ayatake Ezaki’s Tutorial of the Platform
Google Magenta Studio Plugin
Launched in 2016, the Magenta Studio Plugin was designed to allow music creators to harness the power of AI directly within a Digital Audio Workstation (DAW). Magenta Studio was also a completely self-contained plugin, negating the need for an internet connection and a web browser, unlike most other AI-based tools. At the time of writing, the plugin is only available for Ableton Live.
The plugin works by inserting a MIDI part in the Session View of the program. There are five different options that the user can choose from: Continue (which generates music based on an existing MIDI clip), Interpolate (which keys off of two MIDI clips to generate music based on attributes of both), Generate (which creates a part from scratch), Groove (made for drum parts, taking the hits and playing with quantization to make them sound more human), and Drumify (which creates drum grooves based on a melody).
Using these options, you can let the plugin generate multiple instances of music as clips that can be further manipulated in Ableton Live’s MIDI editor.
The plugin specializes in monophonic lines and drum grooves, so generating chords is impossible for now. The developers also include a great user guide that’s easy to follow to get you going quickly.
Price: Free
Level: Intermediate
Best for: People who want AI integrated into their DAW
Format: DAW Plugin for Ableton Live only
What people did with it: Five Beats with Magenta Studio
Los Angeles Music Composer
Los Angeles Music Composer (LAMC) is a GitHub project whose code can be accessed and run in Google Colab. According to the description, LAMC is a: “Local windowed attention multi-instrumental music transformer for supervised music generation.”
LAMC uses seed data from a MIDI file to generate compositions. You can create your own in most DAWs, such as a melody or chord progression you’re working on, and load that into the program. Alternatively, LAMC comes pre-loaded with seed files for you to get started with.
The program is all contained within your web browser after loading the necessary cells in Colab (and there are a few). From there, you simply enter some information into the generator (number of instruments, temperature, etc.). The program generates MIDI based on the information you entered, which can then be downloaded to your computer for further editing and manipulation.
It is important to point out that the Collab interface is quite daunting. For those who have no experience in using a Collab notebook (loading cells and looking at code), one can be easily scared away from trying this. Thankfully, YouTuber Timzart7 created a step-by-step tutorial video on how to use this tool.
https://www.youtube.com/watch?v=YW_hHeSwNG8
Price: Free
Level: Intermediate to Advanced
Best for: People comfortable using Colab
Format: Web App
What people did with it: Samples of music created with the tool on SoundCloud, AI Visions by Timzart7
Staccato
Staccato offers two main AI features that musicians can use to enhance their creative process. One of them is the AI MIDI generator, otherwise known as the AI Instrument™. Staccato’s AI MIDI generator is able to create a seamless continuation of your tracks that matches your original key, style, and mood.
To start, you can either upload a MIDI file that you want to build upon OR you can create your own track directly in Staccato’s online MIDI editor.
Once you've uploaded or created your track, you can click “Generate” and Staccato's AI will instantly compose an appropriate continuation from scratch.
Staccato’s MIDI generator is unique because the tool is centered around a powerful MIDI editor that lets you effortlessly experiment and adjust the songs. This means that you can play some chords, click generate, edit the generation, and generate some more. Rather than just generating finished tracks, Staccato gives you more control and input over your AI-powered music creation.
With the Staccato platform, you aren't passively receiving computer-made music. Instead, you have an interactive partner that helps you bring your own musical ideas to life. It provides an accessible way to guide the AI collaboration and see your vision take shape.
For even easier access, Staccato will soon be integrating directly with top DAWs like Ableton and Logic Pro as a plugin. Staccato also has an AI lyric generator that’s sure to come in handy when you're creating your next song.
Price: 3 Day Free Trial – starting at $9.99 per month for premium access
Level: Intermediate
Best for: Those with at least a little musical knowledge
What people did with it: Staccato’s Soundcloud and Compositions that used the AI MIDI Feature
Format: DAW plugin (coming soon), Web app
Text-to-MIDI Tools
ChatGPT
That’s right, the application that put AI in the public consciousness can be used to generate MIDI data.
The process itself is multi-tiered, but YouTuber Nobody and the Computer managed to train ChatGPT to create a text representation of MIDI data that can be copied into a special Text-to-MIDI conversion program in Google Colab, resulting in a downloadable MIDI file.
To watch the video on the process of doing this, it seems quite the task. Thankfully, Nobody and the Computer does provide step-by-step instructions on the Colab page, including the text to copy and paste into ChatGPT to get it ready to generate MIDI information.
The MIDI files generated using Nobody and the Computer’s methods are downloadable, meaning that you can further work with them in your DAW once you’re happy with your AI generated music.
Price: Free
Level: Intermediate to Advanced
Best for: People comfortable using Google Colab
Format: Web App
What people did with it: BachGPT, JazzGPT
Audiocipher
AudioCipher has come out with a pretty cool tool that is unlike anything I’ve seen. Although not technically AI, their text-to-MIDI generator is made to turn any text you input into customizable melodies and chord progressions. All you need to do is to set the parameters which are key signature, chord type and rhythm.
The process gives you an active role in crafting the song. Once AudioCipher outputs the AI-generated MIDI, simply drag it into your DAW's piano roll for editing. In the end, you create a track that stems directly from the core meaning and essence of the words you chose, while still retaining creative control.
The team is currently developing Version 4.0 for release in Fall 2023, with lifetime free upgrades for customers.
One thing that wasn’t clear to me was whether the generated chord progressions were supposed to match the vibes and feelings of the words that I inputted, or something else. To give you an example, if I put in text about feeling happy, it didn’t always feel like the output had a “happy” vibe to it.
Regardless, as far as I know, Audiocipher stands as the lone text-to-MIDI software. This tool is awesome for creating ideas that you can build off to break through writer’s block. The interface is easy to navigate and the MIDI is made almost instantly. And having a DAW plugin is always a major plus.
Price: $29.99 one-time payment (future versions free)
Level: Beginner–Intermediate
What people did with it: AudioCipher Demo Videos
Format: DAW Plugin, Standalone forMac and PC
Staccato
Staccato’s Text-to-MIDI tool is coming soon. If there’s specific features you’d like to see, you can email us at [email protected] and we’ll do our best to include them!
While you’re waiting, why not sign up for a free account so that you can be notified when Text-to-MIDI comes out? We promise this will be a revolutionary tool, unlike anything you’ve ever seen.
Audio to MIDI File Tools
Samplab
Samplab is an innovative audio-to-MIDI transcription tool that makes it easy for music producers to extract melodies, chords, and rhythms from audio samples.
The user-friendly drag-and-drop interface lets you get started quickly by loading in any audio file. Samplab analyzes the audio and generates a MIDI version, separating out the drums, bass, vocals, and instruments into different tracks.
One standout feature is the transcription speed. Within seconds of uploading your file, accurate MIDI results are displayed and ready for you to edit or export. The generated notes are mapped to the key signature you specify, or you can disable this to enable chromatic editing freedom.
The free version of Samplab offers these core transcription features for samples up to 10 seconds.
For those able to pay, Samplab's paid premium version ($9.99/mo subscription) unlocks additional capabilities:
Unlimited audio track length
Polyphonic editing tools to refine multi-instrument parts
Smart warping algorithms to easily match samples
Fine-tune pitch, pan position, and more
In my experience, Samplab’s audio-to-MIDI is pretty accurate. Even with the free version, you can turn longer audio samples into MIDI by converting 10 seconds at a time. The DAW integration is also extremely convenient because it takes away that extra step of having to download the MIDI file.
Price: Free – $9.99 for premium (14 day free trial)
Level: Beginner-Intermediate
What people did with it: Samplab Demo Page
Format: DAW Plugin, Standalone forMac and PC
AI-MIDI.com
One of the simplest applications so far, AI-MIDI takes your audio files, asks you to fill in a couple of parameters, and, seconds later, a MIDI file is automatically downloaded into your computer.
There is no preview option, so you will need to have a DAW handy to load the files and see how the program did in translating the audio to MIDI.
Don’t let its simplicity fool you, though. After trying a couple of files, it does succeed at getting things right, if not pretty close (though vocal takes with lots of vibrato yielded some interesting results). Nothing is daunting or intimidating here either, making it very easy for newer music makers.
AI-MIDI was created by AudAI, which also makes another application called Vox Factory. Not much is known about the company save for their Discord server.
Price: Free
Level: All levels
Best for: Those who want a MIDI version of a sample, a melody or other harmonic content
Format: Web App
What people did with it: We can't find any use cases online so far, though some users may eventually start posting on AudAI’s Discord server.
Spotify Basic Pitch
Spotify's Basic Pitch is an online web app allows you to record or upload any mono-instrumental audio recording like piano, guitar, or voice. Basic Pitch then instantly analyzes the audio using AI note detection developed by Spotify and generates a MIDI file download.
A major advantage is that Basic Pitch works with instruments playing multiple notes like chords or harmonies. Many audio-to-MIDI tools struggle with multiple simultaneous notes, but Basic Pitch handles them with accuracy.
The AI model also detects subtle pitch bends and expressions, so the MIDI conveys the nuance of a live performance. You can also adjust the MIDI before downloading it so the file sounds exactly how you want it.
After downloading the MIDI, you can import it into your digital audio workstation for editing and refinement.
Basic Pitch is also a mobile-friendly web app, so it brings accessible audio-to-MIDI conversion to iPhone and Android users.
Basic Pitch provides basically the same capabilities as the premium Samplab version for free, so it is definitely worth checking out. The major downside to Basic Pitch is that you have to download the MIDI before bringing it into your DAW which is a step that Samplab eliminates with DAW integration.
Price: Free
Level: Beginner-Intermediate
What people did with it: YouTuber ‘Bad Snacks’ Uses Basic Pitch
Format: Web App
Conclusion
And there you have it – some of the most powerful AI out there totally transforming MIDI! From instantly generating original compositions, to converting text and audio into MIDI with just a click, the possibilities are endless.
Whether you're looking for an AI system to turn your ideas into full songs, convert old recordings into editable MIDI files, or just provide unlimited new musical ideas – there's a tool out there for you. And they all have the power to save you time while unlocking new dimensions of creativity.
Now that you've seen what AI can bring to the table, I'd encourage you to just pick one application that appeals to your style and give it a try on your next track. See for yourself how AI can enhance your musical workflow.
The future is now when it comes to AI and MIDI! Thanks for reading – now go make some magic!
MIDI AI FAQs:
Can AI generate MIDI?
Yes! Staccato’s AI Instrument is a prime example of the opportunities with AI and MIDI. Over time the tool will only continue to improve as AI models get better.
Is MIDI still being used?
Absolutely! Almost every professional song you’ve heard on Spotify and Apple Music used MIDI as part of the creation process. You can read more about the history and importance of MIDI here. In many cases, you can make music using MIDI that would not have been possible to record using an acoustic or electric instrument. For example, Dua Lipa’s producer for ‘Don’t Start Now’, said they did not use a bass guitar when making the song; it was all MIDI!
Is there a program that coverts audio to MIDI?
You can convert audio to MIDI using tools like Spotify Basic Pitch. You can learn more here.
What is MIDI in music?
MIDI is a digital language that allows musical devices to communicate with each other. MIDI is used to create most of the professional music that you hear on streaming services like Apple Music and Spotify.
Artists can play any type of MIDI device to generate MIDI music, including keyboards, controllers, and synthesizers. One of the biggest advantages of MIDI is that it allows artists to change parts of their music without having to re-record.
Can AI create MIDI files?
Yes! Staccato’s AI Instrument showcases the opportunities with AI and MIDI. Over time the tool will only continue to improve as AI models get better.
You can read more about existing MIDI AI tools here.
Can AI generate melodies?
Absolutely! Simply play your existing melody and watch and listen as Staccato suggests what should some next!
Why do people use MIDI?
MIDI allows for digital music devices to ‘talk’ to each other and work together to make music. You can generate sounds from MIDI using a digital audio workstation (DAW) and/or MIDI controller. This allows a user to suddenly have a virtual symphony or rock band in their room, without having to purchase several different instruments.
Dr. Jeffrey Lupker - Co-founder, Staccato
International Speaker & Published Author on Deep Learning & Music.
Dr. Lupker has published peer-reviewed journal articles and book chapters and has given lectures internationally in the fields of deep learning, machine learning and music. Beyond Staccato and his own research, Dr. Lupker is an active performer on guitar and keyboards and has played across Canada and USA with award winning artists.