GERMANY – 2007/01/04: An original Gutenberg printing press on display in the Gutenberg Museum in the … [+]
In 2022, audiobook usage increased by 70% in the U.S., and audiobook publishers had $1.8 billion in 2022. By 2032, the global audiobook industry will reach around $39.1 billion.
Project Gutenberg is a nonprofit organization that wants to democratize literature and make it easy for anyone worldwide to access open-access literature with their phone or computer. Project Gutenberg searches the public domain and collects works of literature in a centralized place accessible by everyone.
The all-volunteer-based group worked with the Massachusetts Institute of Technology (MIT) and Microsoft to produce more than 35,000 hours of audiobooks from Gutenberg’s collection. Using artificial intelligence (AI), researchers used neural text-to-speech, producing lifelike voices from e-books in record time.
Researchers said that with only five seconds of the user’s voice sample, the technology could create personalized audiobooks in the user’s voice.
According to Mark T. Hamilton, a Computer Science Ph.D. Student at MIT and Senior Software Engineer at Microsoft, the initiative aims to democratize literary access for the visually impaired, language learners, children, and audiobook lovers.
“Project Gutenberg has made 60k+ books available for free and open sharing. However, these books are only available as e-books (text only), so it’s difficult for the low-vision community to engage with this great content, and reading visible text can be a challenge,” said Hamilton. “To this end, we used the best AI speech system we could get our hands on to read the books aloud so that more of the world’s literature is available to the low-vision community.”
One of the challenges in automated audiobook production is moving past robotic narration. Hamilton says that AI has turned this challenge has turned this around.
“New speech synthesis systems are trained to sound as much like humans as possible. The modern systems utilize large deep networks, transformers like those used in GPT and trained on millions of hours of speech,” said Hamilton. “They not only learn to speak words clearly and carefully but also learn how to pronounce things like humans do.”
For example, they learn to say “w” “w” “w” “dot” “the name of the website” instead of “www” [Pause] “thenameofthewebsite,” notes Hamilton.
“Other examples include recognizing phone numbers and reading them as a human would, where numbers are grouped for ease of understanding,” said Hamilton. “There are a million of these tiny things we do when speaking that we don’t give any thought to. However, for algorithms, these are nontrivial context-dependent changes to their speech. Teaching these algorithms with millions of hours of real human speech helps them learn all these little tricks.”
“If you had to listen to the algorithm read the table of contents before it read the book, you would be mad; if it read all the page numbers, you would be very confused; if it read the legalese, you might be wondering if you accidentally picked the wrong book,” added Hamilton.
Project Gutenberg works to detect that kind of content and filter it out of the recording.
The Role Of AI
Project Gutenberg has successfully parsed and voiced over 5,000 books using AI.
AI comes up in two key places here. Every e-book in the collection has an idiosyncratic format, some with long tables of contents and illustrations and some that start immediately. So AI is used to tell Project Gutenberg which content should and should not be read aloud in a professional audiobook.
Then, AI has to read the text out loud. According to Hamilton, it can take tens to hundreds of hours to read aloud, edit, master and assemble a full eight-hour audiobook. “If you are paying a human to do this for 5k books, it quickly becomes infeasible,” adds Hamilton.
Hamilton says Project Gutenberg wanted to show they could automatically create reasonable audiobooks using these new types of neural text-to-speech algorithms. “These algorithms can read an eight-hour book clearly and professionally in two minutes, which is a real game-changer for a nonprofit like Project Gutenberg that doesn’t have the time or resources to read the books out loud themselves,” said Hamilton.
With this time savings for audiobook recordings, Hamilton points to the potential for personalized audio content creation. “For children on the spectrum, there might be a value in having a book read in a familiar voice.”
“We hope technology like ours can help the field investigate questions like these. We don’t want to automate away parent-child bonding over reading; we want to enable new ways for parents and children to connect,” said Hamilton.
“We hope this project can further this effort by providing 5k audiobooks with the same open and permissible licensing, and we hope that by making these works listenable, it will remove barriers to literature,” said Hamilton.
