AI language model hopes to preserve metropolis' local dialect
2024-11-22
A team of researchers at Shanghai University has unveiled an artificial intelligence language model designed to help preserve and promote the native Shanghai dialect. Named Xiao Hu, the model is capable of speech recognition, text-to-speech conversion, and dialogue between the Shanghai dialect and Mandarin Chinese.
"Our goal extends far beyond merely teaching AI to speak Shanghainese," says Zeng Jun, director of Shanghai University's publicity department. "Our true objective is to leverage this language model as a means to preserve and innovate the development of the Shanghai dialect, enabling more people, especially Shanghai, to become familiar with and use it."
The impetus behind the project stems from the university's deep-rooted excellence in Chinese language studies and its vision of empowering Chinese through AI technology. Renowned scholars, such as Professor Qian Nairong, a leading authority on the research of the Shanghai dialect, and Associate Professor Ding Dimeng, a Shanghai dialect expert, have laid the foundation for this project.
Ding underscored the cultural significance of the Shanghai dialect, describing it as the root of the city's identity, which contains ancient pronunciations and modern loanwords. "The Shanghai dialect is indispensable in this metropolis," Ding says. "I believe this language model can play an important role in language and dialect preservation efforts, accomplishing tasks that individuals cannot."
The urgency of the project cannot be overstated.
According to doctorate candidate Fan Tianyu, a key member of the development team, a recent statistic shows that only 22 percent of Shanghai residents aged 6 to 20 still use the dialect. The dialect's precarious state highlights the challenge of creating and purifying high-quality language data sets.
The development process was an interdisciplinary effort, according to Fan. While linguists like Qian and Ding provided crucial resources, the Chinese Language Department of the university spearheaded the model training and integration. The university's art academy handled 3D modeling and visual design, and the Ziqiang 5000 supercomputing system furnished the necessary computational muscle.
"The creation of Xiao Hu represents the collective efforts of multiple generations of scholars at Shanghai University," Fan remarked. Hu is a short form of "Shanghai "in Mandarin. "The project is advancing thanks to the pioneering work of professors Qian and Ding and contributions from our data processing experts, computational linguists and design team."
Looking ahead, the team plans to expand Xiao Hu's capabilities with features like additional voice models for different genders and ages, enabling functions like singing in the Shanghai dialect. Enhancements to language understanding and generation are also planned.
Zeng highlights the model's vast potential, with applications ranging from elderly voice services and language-learning platforms to livestreams hosted entirely in the distinctive local vernacular.
Perhaps most promising, Xiao Hu could serve as a translation tool and emotional companion for the city's aging Shanghainese speakers. Official data shows Shanghai had over 5.6 million residents aged 60 and above as of late 2023, accounting for more than a third of the total registered population in the city.
"By enabling seamless communication in the Shanghai dialect, Xiao Hu could provide companionship, translation services and emotional support for the elderly, many of whom are more comfortable conversing in their native dialect," Zeng adds.
Other applications include integrating the model into schools to facilitate Shanghainese learning through interactive products and courses and utilizing it as an AI host for engaging audiences in livestreams and broadcasts in the Shanghai dialect, he adds.