[1]Collage of Odia Book Covers. Image via Flickr by Erin Mclaughlin [2]. From the Flickr set Odia Book Covers. CC-BY 2.0 [3].
While open-source initiatives are still underway, Google and Microsoft have both added the Odia language [4] of India to their respective machine translation engines this year — Google Translate [5] in February and Microsoft [6] more recently on August 13.
Odia [4] is the official language of the Indian state of Odisha [7] and the second official language in the state of Jharkhand [8]. It's spoken by about 35 million native speakers and as a second language by about 4 million people. It is also classified by the Indian government as one of the country's classical languages [9] based on a set of requirements that includes a literary tradition of more than 1,500 years.
However, Odia's digital presence is limited [10]. For instance, the Odia Wikipedia, which is one of the largest repository of textual content, currently has only 15,858 articles [11] after being revived in 2011 following a nine-year-long hiatus [12]. In contrast, Malayalm, which is spoken by about almost the same number of speakers as Odia, has about 70K articles [13] on Wikipedia. Content in Odia used to be available online in the form of image and PDF for a very long time — some including the Odisha state government-run magazine Utkal Prasanga [14] still continue to publish in a combination of image and PDF. Late adoption of Unicode [15] has made content less searchable.
Machine translation significantly contributes to increasing the digital presence of languages by making content more searchable and more accessible to non-speakers.
Microsoft-operated cloud services, including Microsoft Translator app [16], Office, Translator for Bing [17], and through the Azure Cognitive Services Translator [18], will now all support translations from Odia. Both Microsoft Translator and Google Translate (available both on the web [19] and as an app [20]) allow translation of text copied directly into the input field.
Additionally, these platforms also support the translation of text documents, websites, and of live-chats. The Google Translate mobile app has additional features, including offline translation, handwriting recognition, scanning, translating and reading text from images [21], and using voice command to speak to a foreign-language speaker [22]. A feature called “tap to translate” [23] allows a user to directly translate a typed text inside any app. One can also hear how a text in a supported language is pronounced using Google’s speech synthesis.
The addition of Odia was thus well-received by the Odisha state government. The Office of the Chief Minister of Odisha tweeted:
#OdiaTranslation [24] has now been added by @Microsoft [25] to its @mstranslator [26], becoming the 12th commonly used Indian language to be added. This will facilitate access of global information in #Odia [27] and promote inter-language interactions. https://t.co/O4dZgZhbrs [28]
— CMO Odisha (@CMO_Odisha) August 17, 2020 [29]
The Electronics and Information Technology Department of the Government of Odisha also reacted:
Used by millions across the world, @Google [30] Translate has now added #Odia [27] to its list of supported languages. A major step towards promoting digital literacy in our native language & to help millions of non-speakers embrace it. #OdiaOnGoogle [31] @CMO_Odisha [32] https://t.co/lfSskvxSjR [33]
— E&IT Department Odisha (@EIT_Odisha) February 28, 2020 [34]
Machine Translation
Machine translation is used to translate the text or speech of the source language to a target language. The translation that Google uses relies on the Neural Machine Translation [35], a computational system that uses a technique called artificial neural network [36] where large datasets consisting of translation of phrases (from source to target language [37]) are used for training.
With the inclusion of Odia, Google Translate and Microsoft Translator now support 11 Indian languages each. In total, Google supports 109 world languages while Microsoft supports 73.
Meanwhile, open-source initiatives are yet to create successful machine translation projects in Odia.
At least one open-source community-led project is in development — MTEnglish2Odia [38] is training a machine translation engine by collecting translation pairs from existing sources such as Odia-language Wikipedia [39] and crowdsourcing [40] from user contributions on Twitter [41].
In addition, there are some research [42] and resources [43] that can be used for building machine translation engines by other organizations.
The politics of machine translation
The technology used by Google Translate or Microsoft Translator is complex from social, legal, ethical, and rights aspects.
A machine translation platform can be of great use for many people, including journalists to quickly access news from multiple languages, or students desiring to learn from multilingual resources.
Similarly, voice synthesis support helps persons with a disability, especially blindness, to access and share information more easily.
Education, media, and the entertainment industry also benefit from the potential of Google Translate to translate vast amounts of content in a short period of time.
On the other hand, machine translation can contribute to spreading misinformation, while voice synthesis facilitates fraudsters who look to prey on people by communicating with them in their language.
There are over 6,000 documented languages [44] around the world, and only a minority of them have established writing systems. Those are the languages that get to be included in machine translation projects such as Google Translate and Microsoft Translator.
The availability of online content, as well as the number of internet users who speak a given language, are major factors for-profit corporations consider when deciding which languages to include in their systems. The more languages a corporation supports, the more targeted content it can deliver to users — and the more revenue it generates from ads [45].
In addition, there are ethical questions of attribution and remuneration in projects such as Google Translate, which has a contributor community structure [46] to review existing translations (that helps engineers regularly improve the tool).
While Google is a for-profit company with many paid products — including a cloud translation service [47] — neither the individual volunteers nor the numerous public sources from where the machine learns are attributed or remunerated.
The use of private communications for improving machine learning and artificial intelligence are also controversial from a privacy standpoint — though Google has been working towards anonymizing such data [48].
In a country like India, where the creation of multilingual content faces bottle-necks due to costs, products such as Google Translate and Microsoft Translator can revolutionize the Indian content economy. They can make a difference for projects such as Wikipedia, which currently exist in 23 Indian languages [49], or StoryWeaver [50], a multilingual online children’s literature platform that heavily relies on volunteer work.
With many Indian languages disappearing rapidly [51], and with the added challenge of illiteracy and digital accessibility, the communications pathway needs more innovation in voice and visual technology. Machine translation can be a viable tool to stop language extinction — but in India, it still has a long way to go.
Disclaimer: The author has been involved with Odia Wikipedia [52] as a volunteer since 2011 and with MTEnglish2Odia [38] since its early stages.