Project Music GenAI Control: Adobe’s amazing AI tool for music creation and editing 2024

Music is a powerful form of expression and communication, but creating and editing music can be difficult and time-consuming without professional audio experience. That’s why Adobe is working on a new generative AI experiment that aims to make music creation and editing more accessible and intuitive for everyone. The experiment is called Project Music GenAI Control, and it was announced during the Hot Pod Summit in Brooklyn on Wednesday.

What is Project Music GenAI Control?

Project Music GenAI Control is a new prototype tool that leverages generative AI to help users create and customize music using text prompts and editing controls. Users can simply input a text description of the desired music style, such as “happy dance” or “sad jazz”, and the tool will generate music that matches the description.

Users can then use the integrated editing controls to fine-tune the generated music, adjusting aspects such as repeating patterns, tempo, intensity, and structure. Users can also remix sections of music, generate audio as a repeating loop, and extend the length of audio clips to suit their needs. For example, users can use the tool to create backing tracks or background music for content creation or to make the track long enough for fixed animations or podcast segments.

How does Project Music GenAI Control work?

Project Music GenAI Control uses a deep neural network to generate music based on text prompts and reference melodies. The neural network is trained on a large corpus of music data, and it learns to capture the musical features and patterns that correspond to different styles and genres.

The tool can also adjust the generated music based on a reference melody that the user provides, such as a song or a tune that they like. The tool can then modify the generated music to match the reference melody in terms of pitch, rhythm, harmony, and timbre.

The tool also provides a user interface that allows users to edit the generated music without switching to dedicated editing software. The user interface is designed to be simple and intuitive, and it offers various editing controls that can manipulate the music at different levels of granularity.

Users can edit the music at the global level, such as changing the tempo or intensity, or at the local level, such as changing the notes or instruments. Users can also edit the music at the structural level, such as adding, deleting, or rearranging sections of music. The user interface also allows users to preview and play back the music as they edit it, and to export the music as an audio file.

What are the benefits of Project Music GenAI Control?

Project Music GenAI Control is a novel and innovative tool that can offer several benefits for users who want to create and customize music. Some of the benefits are:

• Accessibility: The tool can lower the barrier to entry for music creation and editing, as users do not need any professional audio experience or expensive equipment to use it. Users can simply use text prompts and editing controls to generate and customize music, without having to deal with complex audio software or hardware.
• Creativity: The tool can inspire and enhance users’ creativity, as users can explore different music styles and genres, and experiment with different musical elements and effects. Users can also use the tool to generate music that matches their mood, emotion, or intention, and to express themselves through music.
• Productivity: The tool can improve users’ productivity, as users can create and edit music faster and easier than before. Users can also use the tool to create music for various purposes and contexts, such as content creation, entertainment, education, or therapy.

What is the status of Project Music GenAI Control?

Project Music GenAI Control is an early-stage experiment that is being developed by Adobe Research in collaboration with the University of California and the School of Computer Science at Carnegie Mellon University. The tool is not available to the public yet, and no release date has been announced.

However, Adobe has released a public demo of the tool, which uses public-domain content as reference material. The public demo can be accessed at the Adobe Labs website, where users can also track the development of Project Music GenAI Control and other experiments that Adobe is working on.

Project Music GenAI Control is part of Adobe’s vision to empower creatives with generative AI tools that can help them create and edit various types of media, such as images, videos, and audio. Adobe’s senior research scientist Nicholas Bryan said in a press release: “One of the most exciting things about these new tools is that they aren’t just about generating audio.

They’re taking it to the level of Photoshop by giving creatives the same kind of deep control to shape, tweak, and edit their audio. It’s a kind of pixel-level control for music.”