AN INTELLECTUAL HYBRID MODEL AND FEATURE SPACE FOR NAMED ENTITY RECOGNITION IN KARAKALPAK TEXTS
DOI:
https://doi.org/10.5281/zenodo.23307865Abstract
An intelligent hybrid model and a multi-source feature space are presented for Named Entity Recognition (NER) in Karakalpak electronic texts. The agglutinative morphological structure of the Karakalpak language, multilayer affixation, inconsistencies between Latin and Cyrillic writing systems, and limited availability of electronic language resources may lead to entity boundary detection errors during subword tokenization in multilingual Transformer models (mBERT and XLM-RoBERTa). To address these challenges, an architecture is proposed that integrates contextual Transformer representations, morphological affix features, and domain-specific gazetteers through a Confidence-Guided Gated Fusion mechanism. A Conditional Random Fields (CRF) layer is employed to constrain invalid transitions between sequence labels. The mathematical formulations of the model, the multidimensional feature space, and the software implementation process in the PyTorch environment are described in detail.Keywords
Karakalpak language, Named Entity Recognition, intelligent hybrid model, gated fusion mechanism, morphological features, Transformer, gazetteers, Conditional Random FieldsReferences
Jurafsky, D. and Martin, J. H. (2020) Speech and Language Processing. Pearson, pp. 9–10. Available at: https://web.stanford. edu/~jurafsky/slp3/
Bird, S., Klein, E. and Loper, E. (2009) Natural Language Processing with Python. Sebastopol: O’Reilly Media, pp. 179–202. Available at: https://www.nltk.org/book/
Goldberg, Y. (2017) Neural Network Methods for Natural Language Processing. Morgan & Claypool Publishers, pp. 65–67. Available at: https://doi.org/10.2200/S00762ED1V01Y201703HLT037
Conneau, A. et al. (2020) ‘Unsupervised Cross-lingual Representation Learning at Scale’, in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440–8451. Available at: https://aclanthology.org/2020. acl-main.747/
Devlin, J. et al. (2019) ‘BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding’, in Proceedings of NAACL-HLT 2019, pp. 4171–4186. Available at: https://aclanthology.org/N19-1423/
Liu, Y. et al. (2019) ‘RoBERTa: A Robustly Optimized BERT Pretraining Approach’, arXiv preprint arXiv:1907.11692. Available at: https://arxiv.org/abs/1907.11692
Küçük, D. and Eryiğit, G. (2014) ‘Named Entity Recognition in Turkish’, Turkish Journal of Electrical Engineering and 5 Note: Compiled by the authors based on the developed algorithmic model. 131 SOCIAL, ECONOMIC, SCIENTIFIC, AND TECHNICAL ACADEMIC JOURNAL 10/2026 Computer Sciences, 22(5), pp. 1251–1267.
Şeker, E. A. and Eryiğit, G. (2023) ‘Extending a CRF-based NER System for Turkish’, Journal of Information Science, 49(3), pp. 621–636.
Dai, X. et al. (2021) ‘Gated Multimodal Fusion for Low-Resource NER’, in Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 3218–3228.
Lafferty, J., McCallum, A. and Pereira, F. C. (2001) ‘Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data’, in Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001), pp. 282–289. Available at: https://repository.upenn.edu/handle/20.500.14332/1578
Lample, G. et al. (2016) ‘Neural Architectures for Named Entity Recognition’, in Proceedings of NAACL-HLT 2016, pp. 260–270. Available at: https://aclanthology.org/N16-1030/
Mengliev, D., Ayimbetov, M. and Kojobayev, A. (2023) ‘Automated Morphological Analysis for Karakalpak’, Science and Innovative Development, (3), pp. 78–86.
NLTK Project (2023) Natural Language Toolkit. Available at: https://www.nltk.org/
Downloads
Published
How to Cite
Issue
Section
