Automatic language identification in online multilingual conversations

Abstract

With the abundance of multilingual content on the Web Automatic Language Identification ALI is an important pre requisite for different Natural Language Processing applications While ALI of well edited text over a fairly distinct collection of languages may be regarded as a trivial problem ALI in social media text is considered to be a non trivial task due to the presence of slang words misspellings creative spellings and special elements such as hashtags user mentions etc Additionall...