Benchmarking NLP Toolkits for Enterprise Application

150 150 MIMOS Berhad


Kok Wei Ying, Duc Nghia Pham, Yasaman Eftekharypour and Ang Jia Pheng



Natural Language Processing (NLP) is an important technology that motivatesthe form of AI applicationstoday.ManyNLP librariesareavailable for researchers and developers to perform standardNLP tasks (such as segmen-tation, tokenization, lemmatization, POS tagging, and NER) without the need to develop from scratch. However, there are some challenges inselecting the most suitable librarysuch asdatatype, performance, and the compatibility.In this pa-per, we assessed five popular NLP libraries forperforming the standard pro-cessingtasksondatasetscrawled from differentonline news sourcesin Malaysia. Theobtainedresults are analysed anddifferences of those librariesarelisted. The goal of this study isto provide a clear viewfor usersto selectthe suitable NLP libraryfor their text analysistask.



The 16th Pacific Rim International Conference on Artificial Intelligence PRICAI 2019, Yanuca Island, Cuvu, Fiji