OpenAI appears to be clearly concerned about the use of its new voice cloning tool for scams
OpenAI announced a new AI-based audio cloning tool called Voice Engine on Friday. While the company is obviously proud of the potential of this technology—promoting how it could be used to provide reading assistance to children and give a voice to those who have lost theirs—OpenAI is clearly very nervous about how it could be abused. And with good reason.
“OpenAI is committed to developing safe and broadly beneficial AI,” the company said in a statement on Friday, making its concerns very clear.
Voice Engine Technology
Voice Engine essentially uses the same technology behind its text-to-speech API and ChatGPT Voice, but this application of the technology is all about cloning a voice rather than reading something aloud with the tone and inflection of a stranger. OpenAI points out that its technology is exceptional in that it only requires a 15-second sample to “create emotive and realistic voices.”
“Today we share ideas and preliminary results from a small-scale preview of a model called Voice Engine, which uses text input and a single 15-second audio sample to generate natural-sounding speech that closely resembles the original speaker,” the company wrote.
Concerns and Future Implications
It is unclear what kind of training data was used to build Voice Engine, a sensitive point for AI companies that have been accused of violating copyright laws by training their models on protected works. Companies like OpenAI argue that their training methods qualify as “fair use” under U.S. copyright law, but several rights holders have filed complaints, claiming they were not compensated for their work.
OpenAI’s website has example audio clips that have been fed through Voice Engine and sound nice. The ability to change the language someone speaks is also cool, but you can’t try it out for yourself just yet.
There are already several voice cloning tools available like [mentioned tools], and translators like [mentioned tool]. But OpenAI has become a giant since it first publicly released ChatGPT in late 2022. And as soon as it turns Voice Engine into a publicly available product (there is still no information on the release date) it could open the floodgates for all kinds of new abuses that we have never even dreamed of.
