As part of CEO Mark Zuckerberg's larger push towards generative AI, Meta launched the new tool called AudioCraft on Wednesday. This AI application enables users to create music and sounds from scratch with text-based inputs. This programme has three models: MusicGen, AudioGen and EnCodec, each with its own set of features. MusicGen is trained on Meta-owned and expressly licenced music, whereas AudioGen is taught on public sound effects.
Meta said in June that it trained MusicGen with 20,000 hours of licenced music, including 10,000 "high-quality" licenced music recordings. At the time, Meta's researchers published a paper outlining the ethical problems they faced while developing generative AI models like MusicGen. Meta is not the first to attempt to combine music with AI. MusicLM is a big language model developed by Google that generates minutes of sounds based on text inputs and is only available to academics.
Introducing Meta AudioCraft
In October 2022, Meta released the first version of EnCodec, an AI tool for compressing and decompressing audio files without sacrificing quality for rapid and easy sharing. Its goal was to improve the quality of all sounds, not only music. Since then, the model has grown and is now presented alongside AudioGen and SoundGen as a tool for making synthetic sounds and music appear natural when they are not.
In order to be more inventive, some artists have welcomed AI-generated tools, yet others have criticised copyright violations. Drake and the Weeknd were not pleased about their voices being imitated in the TikTok viral hit "Heart on My Sleeve," which claimed to employ AI copies of their vocals to produce a passable deepfake. Paul McCartney is one of the more ardent, on the other hand. He has revealed that artificial intelligence would assist in finishing a Beatles song from 1969.
This entails that you can quickly create several scenes with parts that are each independently focused and that sync up in the finished product. For instance, if you use the command "Jazz music from the 1980s with a dog barking in the background," AudioCraft will use its MusicGen to play your jazz portion while AudioGen will smoothly insert and merge the dog's background barking. And the EnCodec's sophisticated decoding skills will be used to convey all of this to you.
Although you may believe that AudioCraft's generative AI skills are its strongest suit, this is untrue. Additionally open-source, AudioCraft. In order to better comprehend this technology and develop their own datasets to aid in its improvement, researchers can see the AudioCraft model's source code. GitHub provides access to AudioCraft's source code.
You can quickly produce compression and generation using AudioCraft, as well as music and sound. Because users may improve upon the existing code base to produce better sound generators and compression techniques, AudioCraft is flexible. You do not have to start from scratch, to put it briefly. Your foundation will be based on the dataset's current ceiling.