Introduction:
In this guide, we'll explore some of my favorite AI applications that leverage the power of artificial intelligence to do insanely cool things with MIDI – from instantly generating original AI music, to converting text and audio into MIDI with the click of a button.
Whether you're a seasoned musician looking to spice up your workflow, or just getting started with MIDI music creation, using AI can provide you with endless inspiration and seriously enhance your process. We'll cover everything from basics to real-world use cases so you can fully understand how to harness AI's potential.
Table of Contents
1. Background on AI and MIDI
What is MIDI?
What is AI?
How can I create music with AI and MIDI?
2. AI MIDI Generator Tools
AI MIDI Generator: Musenet
AI MIDI Generator: Sony’s Flow Machines
AI MIDI Generator: Google Magenta Studio Plugin
AI MIDI Generator: Los Angeles Music Composer
AI MIDI Generator: AIVA
AI MIDI Generator: Lemonaide
AI MIDI Generator: Hookpad
AI MIDI Generator: Unison MIDI Wizard 2.0
AI MIDI Generator: Staccato
3. Text-to-MIDI Tools: a Subset of AI MIDI Generators
Text-to-MIDI: ChatGPT
Text-to-MIDI: Audiocipher
Text-to-MIDI: Wavtool
Text-to-MIDI: Staccato
4. AI-MIDI Converter Tools
AI-MIDI Converter: Samplab
AI-MIDI Converter: AI-MIDI.com
AI-MIDI Converter: Spotify Basic Pitch
Background on AI and MIDI
What is MIDI?
Chances are you're already familiar with the magic of MIDI (Musical Instrument Digital Interface). This technology has been revolutionizing the creation of digital music since it first came onto the scene in the 1980s.
In a nutshell, MIDI is a super convenient digital format that allows computers, synthesizers, samplers, and all kinds of other equipment to seamlessly communicate with each other. This made it possible to control and arrange all the elements of digital music creation.
MIDI is commonly used with digital audio workstations (DAWs) like Ableton or Logic Pro to produce music. DAWs are software platforms that allow for audio recording, MIDI sequencing, and music production.
With MIDI, you can program things like notes, tempo, rhythm, dynamics and more. In many ways MIDI laid the foundation for modern digital music production.
What is AI?
Now, another technology is taking MIDI to unbelievable new heights – artificial intelligence! Artificial intelligence (or AI) refers to computer systems that can perform tasks comparable to human intelligence.
AI is trained using machine learning algorithms that process massive amounts of data. This allows them to recognize patterns, interpret complex information, make predictions – and even generate original music and lyrics!
When combined with MIDI, AI introduces game-changing new ways to make music. If you’re interested in how AI will shape the future of music, check out our article on the future of music composition.
How can I create music with AI and MIDI?
There are some seriously futuristic ways AI can enhance this technology:
AI MIDI generator: With these tools, you can get AI to generate completely new music for you with just a click! Some of these tools were trained on thousands of song samples to be able to create original tunes in any style you want.
Text to MIDI music generator: Text-to-MIDI music generators are a special subset of AI-MIDI generators. This tool is perfect if you have some stylistic ideas but need some melodic inspiration. Simply type in some text and it will turn it into a MIDI composition.
AI MIDI converter: Upload any audio file, and the tool will convert it to MIDI by detecting notes, instruments etc. This comes in super handy if you have existing audio that you want to edit in MIDI form.
Let's check out some of these unbelievable AI tools for music creation!
AI MIDI Generator Tools
OpenAI Musenet
While Musenet is no longer available, we believe it deserves a mention in this article because of its widespread popularity during its availability.
Created by OpenAI, the same company that made ChatGPT, Musenet was a popular AI tool to generate musical compositions. The platform had the capability to generate 4-minute compositions with the freedom to customize and combine various instruments and musical styles.
Musenet allowed you to import a MIDI file into the software and then it would generate continuations of the music in the style you chose. The options included Jazz, Pop, African, Indian, and even musicians like Mozart, The Beatles, and Queen.
Once you played around with the options and heard something you liked, you could download the composition to your computer.
Though you can’t use it anymore, you can still check out some of the tool’s creations on the Musenet homepage.
There is some hope for Musenet’s return as Christine McLeavey, the creator of Musenet, replied to a tweet on January 2, 2023 saying that "Musenet will be back soon, but I don't have the exact date yet."
However, it has now been more than 2 years since Musenet stopped working in December 2022 and it seems less and less likely for it to come back.
Price: Musenet is no longer available to the public
Sony’s Flow Machines
Instead of generating continuations of music that you give it, you can make music with Flow Machines by training the AI. The generator then gives you several variations of melodies according to the training.
Here’s how it works:
Set parameters like key, tempo, and scale
Choose the guiding chord progression
Capture melodies (play them on a keyboard) that you want Flow Machines to use as a guide
Flow Machines creates several varieties of AI generated compositions based on your training
If you have the standalone app (computer or mobile), you can download the MIDI file then load it up in your DAW. If you have the DAW plugin, you can upload it straight into your DAW.
Each of the training sets you make will be saved into “palettes”. You can think of palettes as directions for the AI music generator.
If all of this sounds too complicated for you, Flow Machines also has preset palettes that you can use. There are over 100 presets in genres that include Pop, EDM, Jazz, and more.
Once you have the palette that you like, you can adjust the parameters and even select bars to regenerate.
Flow Machines is a great tool for seasoned musicians who have a good understanding of music and digital music production. It is different from the others because it requires you to have some idea of the music that you want to generate. Despite all of this, Flow Machines is incredibly useful if you can use it right. If you want to see how it is done, check out Ayatake Ezaki’s tutorial.
Price: Free
Level: Advanced
Best for: Experienced musicians
Format: DAW plugin, Standalone for Mac, iPad app, iPhone app
Google Magenta Studio Plugin
Launched in 2016, the Magenta Studio Plugin was designed to allow music creators to harness the power of AI directly within a Digital Audio Workstation (DAW). Magenta Studio was also a completely self-contained plugin, negating the need for an internet connection and a web browser, unlike most other AI-based tools. At the time of writing, the plugin is only available for Ableton Live.
The plugin works by inserting a MIDI part in the Session View of the program. There are five different options that the user can choose from: Continue (which generates music based on an existing clip), Interpolate (which keys off of two clips to generate music based on attributes of both), Generate (which creates a part from scratch), Groove (made for drum parts, taking the hits and playing with quantization to make them sound more human), and Drumify (which creates drum grooves based on a melody).
Using these options, you can let the plugin generate multiple instances of music as clips that can be further manipulated in Ableton Live’s editor.
The plugin specializes in monophonic lines and drum grooves, so generating chords is impossible for now. The developers also include a great user guide that’s easy to follow to get you going quickly.
Price: Free
Level: Intermediate
Best for: People who want AI integrated into their DAW
Format: DAW Plugin for Ableton Live only
Los Angeles Music Composer
Los Angeles Music Composer (LAMC) is a GitHub project whose code can be accessed and run in Google Colab. According to the description, LAMC is a: “Local windowed attention multi-instrumental music transformer for supervised music generation.”
LAMC uses seed data from a MIDI file to generate compositions. You can create your own in most DAWs, such as a melody or chord progression you’re working on, and load that into the program. Alternatively, LAMC comes pre-loaded with seed files for you to get started with.
The program is all contained within your web browser after loading the necessary cells in Colab (and there are a few). From there, you simply enter some information into the generator (number of instruments, temperature, etc.). The program generates MIDI based on the information you entered, which can then be downloaded to your computer for further editing and manipulation.
It is important to point out that the Collab interface is quite daunting. For those who have no experience in using a Collab notebook (loading cells and looking at code), one can be easily scared away from trying this. Thankfully, YouTuber Timzart7 created a step-by-step tutorial video on how to use this tool.
Price: Free
Level: Intermediate to Advanced
Best for: People comfortable using Colab
Format: Web App
AIVA: Early AI MIDI Generator
AIVA is a browser-based AI music composition tool that specializes in creating orchestral and cinematic scores. Launched in 2016, it became the first virtual composer registered with SACEM, showcasing its pioneering approach to music technology.
AIVA uses musical parameters and existing pieces to influence its compositions. Users can select genres, tweak parameters, or upload MIDI and audio files to guide the creation process. This approach offers precision and control but may feel restrictive compared to the flexibility of text-prompt systems.
AIVA’s interface, while powerful, has a steep learning curve that can feel like mastering a new DAW. Its licensing model varies, with Free and Standard plans retaining copyright ownership under AIVA, while Pro Plan subscribers can gain full ownership if they meet specific criteria, such as being an individual or small business with less than $300k in annual revenue.
Despite these challenges, AIVA stands out as one of the robust tools for AI composition, available both as a web application and a standalone program for Mac and Windows.
Price: Standard Plan $15 per month, ownership under AIVA
Level: Intermediate to Advanced
Best for: People who enjoy experimenting with parameters
Format: Web App, Standalone App
Hookpad Aria: AI MIDI Generator for HookPad
HookPad’s Aria, introduced in 2024, brings AI-assisted music creation to HookTheory’s popular browser-based MIDI platform. Designed as an add-on, Aria enhances and builds on existing musical content rather than creating entirely new compositions.
Using Aria is straightforward: users create or load a project in HookPad, highlight specific sections, and request AI suggestions for chord progressions or melodies. This integration keeps Aria deeply connected to HookPad, acting as a collaborative tool rather than an independent composer.
Aria’s underlying technology blends Google Deepmind’s open-source Anticipatory Music Transformer with HookTheory’s vast database of over 50,000 MIDI-formatted transcriptions. However, its use of community-generated data for fine-tuning raises potential copyright concerns, an area where some competitors take more precautionary steps.
Despite some limitations, Aria adds a creative edge to HookPad’s ecosystem. For users who thrive within HookPad’s structured workflow, it offers a valuable way to explore new ideas and refine compositions with the help of AI.
Price: $14.99 per month
Level: Beginner to Intermediate
Best for: People who already use HookPad
Format: Add-on to HookPad
Unison MIDI Wizard 2.0: Offline AI MIDI Generator
Unison MIDI Wizard 2.0, launched in late 2023, offers musicians a way to quickly generate chord progressions and melodies. Designed with beginners in mind, it lets users choose key signature, genre, and tempo, making it an easy entry point for streamlining the creative process.
However, the tool has faced criticism. Unison’s well-known chord packs have drawn skepticism from the production community, with many questioning their originality and overall value.
One advantage of Wizard 2.0 is that it works offline, but this raises questions about its technology. Is it driven by advanced AI like Magenta Studio, or is it simply pulling from a database of pre-made sequences?
The price is another sticking point. At $397, it’s significantly more expensive than many alternatives, and numerous online complaints highlight dissatisfaction with the product’s performance and Unison’s strict no-refund policy.
While it might appeal to beginners, the high cost and mixed reviews make Wizard 2.0 a controversial choice for AI-assisted music creation.
Price: $397
Level: Beginner
Best for: Music producers who value offline functionality
Format: DAW plugin
Lemonaide: AI MIDI Generators for Hip-hop Producers
Lemonaide, introduced in 2023, is an AI-powered tool designed to help beginner hip-hop producers create music with ease. It streamlines the process by generating beats and melodies through a straightforward, step-by-step interface.
The platform’s dropdown menu system allows users to select key musical elements, eliminating the need for extensive knowledge of music theory. This makes it especially appealing to new producers exploring genres like trap and rap.
That said, more experienced users may find Lemonaide’s simplicity limiting. The beats and melodies it produces often leans toward generic, relying on pre-set options that can lack originality. As a result, creating professional and unique beats might require additional customization.
Price: $9.99 per month
Level: Beginner
Best for: hip-hop producers
Format: Standalone App
Staccato: Best AI MIDI Generator
Staccato offers two main AI features that musicians can use to enhance their creative process. One of them is the AI MIDI generator, otherwise known as the AI Instrument™. Staccato’s AI MIDI generator is able to create a seamless continuation of your tracks that matches your original key, style, and mood.
To start, you can either upload a MIDI file that you want to build upon OR you can create your own track directly in Staccato’s online MIDI editor.
Once you've uploaded or created your track, you can click “Generate” and Staccato's AI will instantly compose an appropriate continuation from scratch.
Staccato’s MIDI generator is unique because the tool is centered around a powerful MIDI editor that lets you effortlessly experiment and adjust the songs. This means that you can play some chords, click generate, edit the generation, and generate some more. Rather than just generating finished tracks, Staccato gives you more control and input over your AI-powered music creation.
With the Staccato platform, you aren't passively receiving computer-made music. Instead, you have an interactive partner that helps you bring your own musical ideas to life. It provides an accessible way to guide the AI collaboration and see your vision take shape.
For even easier access, Staccato will soon be integrating directly with top DAWs like Ableton and Logic Pro as a plugin. Staccato also has an AI lyric generator that’s sure to come in handy when you're creating your next song.
Price: 3 Day Free Trial – starting at $9.99 per month for premium access
Level: Intermediate
Best for: Those with at least a little musical knowledge
What people did with it: Staccato’s Soundcloud and Compositions that used the AI MIDI Feature
Format: DAW plugin (coming soon), Web app
Text-to-MIDI Tools
ChatGPT as Text to MIDI AI Generator
That’s right, the application that put AI in the public consciousness can be used to generate MIDI data.
The process itself is multi-tiered, but YouTuber Nobody and the Computer managed to train ChatGPT to create a text representation of MIDI data that can be copied into a special Text-to-MIDI conversion program in Google Colab, resulting in a downloadable MIDI file.
To watch the video on the process of doing this, it seems quite the task. Thankfully, Nobody and the Computer does provide step-by-step instructions on the Colab page, including the text to copy and paste into ChatGPT to get it ready to generate MIDI information.
The MIDI files generated using Nobody and the Computer’s methods are downloadable, meaning that you can further work with them in your DAW once you’re happy with your AI generated music.
Price: Free
Level: Intermediate to Advanced
Best for: People comfortable using Google Colab
Format: Web App
What people did with it: BachGPT, JazzGPT
Audiocipher
AudioCipher has come out with a pretty cool tool that is unlike anything I’ve seen. Although not technically AI MIDI generator, their text-to-MIDI generator is made to turn any text you input into customizable melodies and chord progressions. All you need to do is to set the parameters which are key signature, chord type and rhythm.
The process gives you an active role in crafting the song. Once AudioCipher outputs the generated MIDI, simply drag it into your DAW's piano roll for editing. In the end, you create a track that stems directly from the core meaning and essence of the words you chose, while still retaining creative control.
One thing that wasn’t clear to me was whether the generated chord progressions were supposed to match the vibes and feelings of the words that I inputted, or something else. To give you an example, if I put in text about feeling happy, it didn’t always feel like the output had a “happy” vibe to it.
Regardless, as far as I know, Audiocipher stands as the lone text-to-MIDI software. This tool is awesome for creating ideas that you can build off to break through writer’s block. The interface is easy to navigate and the MIDI is made almost instantly. And having a DAW plugin is always a major plus.
Price: $29.99 one-time payment (future versions free)
Level: Beginner–Intermediate
Format: DAW Plugin, Standalone forMac and PC
Wavtool DAW
WavTool was a browser-based DAW (Digital Audio Workstation) that aimed to revolutionize music production with AI-assisted tools like chord progressions, timbre transfer, advanced synthesis, stem separation, and audio-to-MIDI conversion. It also included an AI bot designed to inspire creativity and assist users in the production process.
That said, the platform wasn’t without its challenges. Its music generation often required several attempts to produce usable results, and being exclusively browser-based limited its compatibility with traditional desktop workflows.
While WavTool attracted some interest within the music production community, it has since shut down. Whether it will return under a new guise or is gone for good remains unclear.
Price: Previously offered at $20 per month before going offline
Level: Beginner–Intermediate
Format: Browser-based DAW
Staccato Text-to-MIDI
Staccato’s text-to-MIDI feature is a game-changer in the realm of AI MIDI generator. This innovative tool lets you transform simple text prompts into detailed compositions, bridging the gap between abstract ideas and tangible music creation.
By translating text-based ideas into structured MIDI-formatted files, Staccato’s text-to-MIDI feature makes it easier than ever to bring musical concepts to life, catering to both beginners and seasoned producers seeking inspiration or a starting point for their tracks.
To use the text to MIDI AI generator, you just describe the musical elements you want — such as a "melancholic melody in A minor at 100 BPM" or "a driving rhythm with jazzy chords." Staccato’s AI processes the input and creates a MIDI-formatted file that matches the requested style, key, and tempo. This flexibility allows you to quickly experiment with new ideas and explore creative possibilities without starting from scratch.
Price: 7 Day Free Trial – starting at $9.99 per month for premium access
Best for: Those with at least a little musical knowledge
What people did with it: AI-powered soundscapes
Format: DAW plugin, Web app
AI MIDI Converter Tools
Samplab
Samplab is an innovative audio-to-MIDI transcription tool that makes it easy for music producers to extract melodies, chords, and rhythms from audio samples.
The user-friendly drag-and-drop interface lets you get started quickly by loading in any audio file. Samplab analyzes the audio and generates a MIDI-formatted version, separating out the drums, bass, vocals, and instruments into different tracks.
One standout feature is the transcription speed. Within seconds of uploading your file, accurate results are displayed and ready for you to edit or export. The generated notes are mapped to the key signature you specify, or you can disable this to enable chromatic editing freedom.
The free version of Samplab offers these core transcription features for samples up to 10 seconds.
For those able to pay, Samplab's paid premium version ($9.99/mo subscription) unlocks additional capabilities:
Unlimited audio track length
editing tools to refine multi-instrument parts
Smart warping algorithms to easily match samples
Fine-tune pitch, pan position, and more
In my experience, Samplab is pretty accurate. Even with the free version, you can turn longer audio samples into MIDI by converting 10 seconds at a time. The DAW integration is also extremely convenient because it takes away that extra step of having to download the file.
Price: Free – $9.99 for premium (14 day free trial)
Level: Beginner-Intermediate
AI-MIDI.com
One of the simplest applications so far, AI-MIDI takes your audio files, asks you to fill in a couple of parameters, and, seconds later, a MIDI-formatted file is automatically downloaded into your computer.
There is no preview option, so you will need to have a DAW handy to load the files and see how the program did in translating the audio to MIDI.
Don’t let its simplicity fool you, though. After trying a couple of files, it does succeed at getting things right, if not pretty close (though vocal takes with lots of vibrato yielded some interesting results). Nothing is daunting or intimidating here either, making it very easy for newer music makers.
AI-MIDI was created by AudAI, which also makes another application called Vox Factory. Not much is known about the company save for their Discord server.
Level: All levels
Best for: Those who want a MIDI-formatted version of a sample, a melody or other harmonic content
Spotify Basic Pitch
Spotify's Basic Pitch is an online web app allows you to record or upload any mono-instrumental audio recording like piano, guitar, or voice. Basic Pitch then instantly analyzes the audio using AI note detection developed by Spotify and generates a MIDI-formatted file download.
A major advantage is that Basic Pitch works with instruments playing multiple notes like chords or harmonies. Many audio-to-MIDI tools struggle with multiple simultaneous notes, but Basic Pitch handles them with accuracy.
The AI model also detects subtle pitch bends and expressions, so the output conveys the nuance of a live performance. You can also adjust the MIDI before downloading it so the file sounds exactly how you want it.
After downloading the output, you can import it into your digital audio workstation for editing and refinement.
Basic Pitch is also a mobile-friendly web app, so it brings accessible audio-to-MIDI conversion to iPhone and Android users.
Basic Pitch provides basically the same capabilities as the premium Samplab version for free, so it is definitely worth checking out. The major downside to Basic Pitch is that you have to download the MIDI-formatted file before bringing it into your DAW which is a step that Samplab eliminates with DAW integration.
Level: Beginner-Intermediate
Conclusion
And there you have it – some of the most powerful AI out there totally transforming MIDI! From instantly generating original compositions, to converting text and audio into MIDI with just a click, the possibilities are endless.
Whether you're looking for an AI system to turn your ideas into full songs, convert old recordings into editable files, or just provide unlimited new musical ideas – there's a tool out there for you. And they all have the power to save you time while unlocking new dimensions of creativity.
Now that you've seen what AI can bring to the table, I'd encourage you to just pick one application that appeals to your style and give it a try on your next track. See for yourself how AI can enhance your musical workflow.
The future is now when it comes to AI and MIDI! Thanks for reading – now go make some magic!
MIDI AI FAQs
Can AI generate MIDI?
Yes! Staccato’s AI Instrument is a prime example of the opportunities with AI and MIDI. Over time the tool will only continue to improve as AI models get better.
Is MIDI still being used?
Absolutely! Almost every professional song you’ve heard on Spotify and Apple Music used MIDI as part of the creation process. You can read more about the history and importance of MIDI here. In many cases, you can make music using MIDI that would not have been possible to record using an acoustic or electric instrument. For example, Dua Lipa’s producer for ‘Don’t Start Now’, said they did not use a bass guitar when making the song; it was all MIDI!
Is there a program that coverts audio to MIDI?
You can convert audio to MIDI using tools like Spotify Basic Pitch. You can learn more here.
What is MIDI in music?
MIDI is a digital language that allows musical devices to communicate with each other. MIDI is used to create most of the professional music that you hear on streaming services like Apple Music and Spotify.
Artists can play any type of MIDI device to generate MIDI music, including keyboards, controllers, and synthesizers. One of the biggest advantages of MIDI is that it allows artists to change parts of their music without having to re-record.
Can AI create MIDI files?
Yes! Staccato’s AI Instrument showcases the opportunities with AI and MIDI. Over time the tool will only continue to improve as AI models get better.
You can read more about existing MIDI AI tools here, including AI MIDI generator and AI MIDI converter.
Can AI generate melodies?
Absolutely! Simply play your existing melody and watch and listen as Staccato suggests what should some next!
Why do people use MIDI?
MIDI allows for digital music devices to ‘talk’ to each other and work together to make music. You can generate sounds from MIDI using a digital audio workstation (DAW) and/or MIDI controller. This allows a user to suddenly have a virtual symphony or rock band in their room, without having to purchase several different instruments.
Dr. Jeffrey Lupker - Co-founder, Staccato
International Speaker & Published Author on Deep Learning & Music.
Dr. Lupker has published peer-reviewed journal articles and book chapters and has given lectures internationally in the fields of deep learning, machine learning and music. Beyond Staccato and his own research, Dr. Lupker is an active performer on guitar and keyboards and has played across Canada and USA with award winning artists.