Wijeratne, Y., de Silva, N., Shanmugarajah, Y. (2019). Natural Language Processing for Government: Problems and Potential. LIRNEasia.
Last updated: January 24, 2020
A whitepaper distilling LIRNEasia‘s current thoughts on the possibilities and issues with the computation extraction of syntactic and semantic language from digital text.
General takeaways: for many languages in the Global South, the current development in corpuses, tokenizers, lemmatizers and other foundational algorithmic minutae are far behind the the current state of the art in English. Were these to be resolved, differences in linguistic structure would cause many existing higher-level algorithms to function far below their current efficiency levels. Efforts will have to be invested in building these resources from scratch, or waiting for machine translation to be sufficiently adequate.
Comments are closed.