A WEIGHTED FINITE-STATE TRANSDUCER FOR CONTEXT-AWARE SYNONYM SELECTION: A HYBRID SYMBOLIC–NEURAL APPROACH

Authors

DOI:

https://doi.org/10.5281/zenodo.24141282

Abstract

This paper treats context-aware synonym selection as a ranking problem over a constrained candidate space. A synonym dictionary is compiled into a two-layer weighted finite-state transducer (WFST): input arcs map a word to its sense states, and candidate arcs map each sense state to its substitutes. Candidate-arc weights are not fixed at compilation time but are computed at decoding time from sentence embeddings of the context. The FST-BUILD and FST-DE- CODE algorithms are presented with complexity bounds. On a 1,104-entry Uzbek synonym resource, the transducer has 1,104 sense states and 3,171 candidate arcs, builds in 7.7 ms, and requires at most 10 encoder calls per query. A protocol for evaluating semantic accuracy is specified for the next stage of the study.

Keywords

weighted finite-state transducer, lexical substitution, synonym selection, neuro-symbolic approach, computational complexity, sentence embeddings, low-resource languages, Uzbek

References

1. Mohri, M. (1997). Finite-state transducers in language and speech processing. Computational Linguistics, 23.

2. – P. 269–311.

3. Mohri, M., Pereira, F., & Riley, M. (2002). Weighted finite-state transducers in speech recognition. Computer Speech & Language, 16(1), 69–88.

4. Rastogi, P., Cotterell, R., & Eisner, J. (2016). Weighting finite-state transductions with neural context. Proceedings of NAACL-HLT, (16), 623–633. https://doi.org/10.18653/v1/N16-1076

5. Lin, C. (2019). -C., Zhu H., Gormley M. R., Eisner J. Neural finite-state transducers: Beyond rational relations. Proceedings of NAACL-HLT, (19), 272–283. https://doi.org/10.18653/v1/N19-1024

6. Aleksic, P., Ghodsi, M., Michaely, A., Allauzen, C., Hall, K., Roark, B., Rybach, D., & Moreno, P. (2015). Bringing contextual information to Google speech recognition. Proceedings of Interspeech, 468–472. https://doi.org/10.21437/Interspeech.2015-177

7. McCarthy, D., & Navigli, R. (2007). SemEval-2007 Task 10: English Lexical Substitution Task. Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), 48–53.

8. Kremer, G., Erk, K., Padó, S., & Thater, S. (2014). What substitutes tell us – analysis of an “all-words” lexical substitution corpus. Proceedings of EACL, 540–549. https://doi.org/10.3115/v1/E14-1057

9. Lee, M., Donahue, C., Jia, R., Iyabor, A., & Liang, P. (2021). Swords: A benchmark for lexical substitution with improved data coverage and quality. Proceedings of NAACL-HLT, 4362–4379. https://doi.org/10.18653/v1/2021.naacl-main.345

10. Zhou, W., Ge, T., Xu, K., Wei, F., & Zhou, M. (2019). BERT-based lexical substitution. Proceedings of the 57th Annual Meeting of the ACL, 3368–3373. https://doi.org/10.18653/v1/P19-1328

11. Michalopoulos, G., McKillop, I., Wong, A., & Chen, H. (2022). LexSubCon: Integrating knowledge from lexical resources into contextual embeddings for lexical substitution. Proceedings of the 60th Annual Meeting of the ACL, 1226–1236. https://doi.org/10.18653/v1/2022.acl-long.87

12. Qiang, J., Liu, K., Li, Y., Yuan, Y., & Zhu, Y. (2023). ParaLS: Lexical substitution via pretrained paraphraser. Proceedings of the 61st Annual Meeting of the ACL, 3731–3746. https://doi.org/10.18653/v1/2023.acl-long.206

13. Vladika, J., Meisenbacher, S., & Matthes, F. (2025). Lexical substitution is not synonym substitution: On the importance of producing contextually relevant word substitutes. Proceedings of ICAART, 3, 1481–1488. https://doi.org/10.5220/0013389000003890

14. Panchendrarajan, R., & Zubiaga, A. (2024). Synergizing machine learning & symbolic methods: A survey on hybrid approaches to natural language processing. Expert Systems with Applications, 251. https://doi.org/10.1016/j

15. Mao, Z., & Nakagawa, T. (2023). LEALLA: Learning lightweight language-agnostic sentence embeddings with knowledge distillation. Proceedings of EACL, 1886–1894. https://doi.org/10.18653/v1/2023.eacl-main.138

16. Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., & Zhou, M. (2020). MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Advances in Neural Information Processing Systems 33 (NeurIPS, 5776–5788.

17. Agostini, A., Usmanov, T., Khamdamov, U., Abdurakhmonova, N., & Mamasaidov, M. (2021). UZWORDNET: A lexical-semantic database for the Uzbek language. Proceedings of the 11th Global Wordnet Conference (GWC, 8–19. https://doi.org/10.18653/v1/2021.gwc-1.2

Downloads

Published

2026-11-01

How to Cite

Abdunasimov, S. (2026). A WEIGHTED FINITE-STATE TRANSDUCER FOR CONTEXT-AWARE SYNONYM SELECTION: A HYBRID SYMBOLIC–NEURAL APPROACH. Innovation Science and Technology, 4(11), 46–50. https://doi.org/10.5281/zenodo.24141282
Vol. 4 No. 11 (2026): Innovation Science and Technology