On Multilingual Text Summarization: A Review of Model Architectures, Techniques, and Research Gaps
DOI:
https://doi.org/10.59461/ijdiic.v5i4.311Keywords:
Text Summarization, Natural Language Processing, Systematic Literature Review, Low-Resource Languages, Large Language ModelsAbstract
The rapid expansion of multilingual digital content has intensified demand for automated text summarization systems that can operate across diverse languages. Yet the architectural and methodological foundations of this domain remain dispersed across individual studies. This paper presents a systematic literature review of multilingual text summarization research in Natural Language Processing, with particular attention to model architectures, multilingual enabling techniques, and existing research gaps. Following a PRISMA-guided review protocol, 47 studies published between 2017 and 2026 were identified, screened, and synthesized from major scholarly databases using predefined eligibility and quality assessment criteria. The analysis shows that the literature is dominated by four dominant architectural paradigms: multilingual sequence-to-sequence models, large language models, adapter-based models, and hybrid architectures, with a recent shift toward large language model-based approaches. Across these studies, multilingual capability is primarily achieved through pre-training and transfer learning, with support from parameter-efficient adaptation and data augmentation. The review further identified four persistent limitation categories: low-resource language underrepresentation; dataset and benchmark constraints; evaluation limitations dominated by ROUGE overreliance; and generalization and scalability challenges, manifested as the curse of multilinguality. These findings indicate that current progress in the field is largely driven by scaling existing models rather than resolving structural multilingual limitations, particularly those related to data availability and model architecture. This paper provides a consolidated view of architectural and methodological trends and offers a structured, informed basis for future research on equitable multilingual summarization, particularly for low-resource languages.
References
[1] D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural language processing: State of the art, current trends and challenges,” Multimedia Tools Appl., vol. 82, no. 3, pp. 3713–3744, Jan. 2023, doi: 10.1007/s11042-022-13428-4.
[2] M. Munaf, H. Afzal, K. Mahmood, and N. Iltaf, “Low resource summarization using pre-trained language models,” ACM Trans. Asian Low-Resour. Lang. Inf. Process., vol. 23, no. 10, Art. no. 141, pp. 1–19, Oct. 2024, doi: 10.1145/3675780.
[3] H. Zhang, P. S. Yu, and J. Zhang, “A systematic survey of text summarization: From statistical methods to large language models,” ACM Comput. Surv., vol. 57, no. 11, Art. no. 277, 2025, doi: 10.1145/3731445.
[4] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” in Proc. 58th Annu. Meeting Assoc. Comput. Linguistics (ACL), 2020, pp. 7871–7880, doi: 10.18653/v1/2020.acl-main.703.
[5] L. Xue et al., “mT5: A massively multilingual pre-trained text-to-text transformer,” in Proc. 2021 Conf. North Amer. Chapter Assoc. Comput. Linguistics: Human Lang. Technol. (NAACL-HLT), 2021, pp. 483–498, doi: 10.18653/v1/2021.naacl-main.41.
[6] T. Hasan et al., “XL-Sum: Large-scale multilingual abstractive summarization for 44 languages,” in Findings Assoc. Comput. Linguistics: ACL-IJCNLP 2021, 2021, pp. 4693–4703, doi: 10.18653/v1/2021.findings-acl.413.
[7] K. Huang et al., “A survey on large language models with multilingualism: Recent advances and new frontiers,” Artif. Intell. Rev., vol. 59, Art. no. 146, 2026, doi: 10.1007/s10462-026-11534-5.
[8] H. Shakil, A. Farooq, and J. Kalita, “Abstractive text summarization: State of the art, challenges, and improvements,” Neurocomputing, vol. 603, Art. no. 128255, Oct. 2024, doi: 10.1016/j.neucom.2024.128255.
[9] X. Zhang, “The path of multilingual neural machine translation: Cross-language transfer and domain adaptation,” Procedia Comput. Sci., vol. 279, pp. 327–336, 2026, doi: 10.1016/j.procs.2026.03.241.
[10] B. Nath and Y. Gulzar, “Zero-shot English–Assamese neural machine translation via pivot-based cross-lingual embedding alignment and transfer learning,” Sci. Rep., vol. 16, Art. no. 13732, 2026, doi: 10.1038/s41598-026-44209-w.
[11] G. Park, J. Park, and H. Lee, “Cross-lingual summarization for low-resource languages using multilingual retrieval-based in-context learning,” Appl. Sci., vol. 15, no. 14, Art. no. 7800, Jul. 2025, doi: 10.3390/app15147800.
[12] A. Ghabi and H. Hamam, “Architectural deep dive into large language models,” in Generative AI and Large Language Models: Opportunities, Challenges, and Applications, vol. 1, A. Koubaa, A. Ammar, L. Ghouti, W. Boulila, and B. Benjdira, Eds. Cham, Switzerland: Springer Nature Switzerland, 2025, pp. 65–105, doi: 10.1007/978-3-031-90573-5_3.
[13] J. Wang et al., “A survey on cross-lingual summarization,” Trans. Assoc. Comput. Linguistics, vol. 10, pp. 1304–1323, 2022, doi: 10.1162/tacl_a_00520.
[14] K. P. Sharma et al., “A systematic review on text summarization: Techniques, challenges, opportunities,” Expert Syst., vol. 42, no. 4, Art. no. e13833, 2025, doi: 10.1111/exsy.13833.
[15] M. Parikh, A. Gandhi, and R. Tripathi, “Approaches for summarization of articles across multiple languages: A review,” in Proc. 2025 Int. Conf. Mach. Learn. Auton. Syst. (ICMLAS), 2025, pp. 1150–1154, doi: 10.1109/ICMLAS64557.2025.10968896.
[16] A. C. Tricco et al., “Updating the PRISMA reporting guideline for scoping reviews: A scoping review,” J. Clin. Epidemiol., vol. 196, Art. no. 112314, 2026, doi: 10.1016/j.jclinepi.2026.112314.
[17] I. G. A. I. U. Putri, I. N. P. Trisna, and N. K. D. Rusjayanthi, “Abstractive text summarization to generate Indonesian news highlight using transformers model,” J. Inf. Syst. Informatics, vol. 7, no. 2, pp. 1248–1263, Jun. 2025, doi: 10.51519/journalisi.v7i2.1082.
[18] S. Doddapaneni, M. S. U. R. Khan, D. Venkatesh, R. Dabre, A. Kunchukuttan, and M. M. Khapra, “Cross-lingual auto evaluation for assessing multilingual LLMs,” in Proc. 63rd Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1: Long Papers, Vienna, Austria, 2025, pp. 29297–29329, doi: 10.18653/v1/2025.acl-long.1419.
[19] S. Ajay Mukund and K. S. Easwarakumar, “Optimizing legal text summarization through dynamic retrieval-augmented generation and domain-specific adaptation,” Symmetry, vol. 17, no. 5, Art. no. 633, May 2025, doi: 10.3390/sym17050633.
[20] R. Pasunuru, M. Liu, M. Bansal, S. Ravi, and M. Dreyer, “Efficiently summarizing text and graph encodings of multi-document clusters,” in Proc. 2021 Conf. North Amer. Chapter Assoc. Comput. Linguistics: Human Lang. Technol. (NAACL-HLT), 2021, pp. 4768–4779, doi: 10.18653/v1/2021.naacl-main.380.
[21] B. Dong, C. Bu, Y. Zhu, S. Ji, and X. Wu, “Simplified multi-view graph neural network for multilingual knowledge graph completion,” Front. Comput. Sci., vol. 19, no. 7, Art. no. 197324, 2024, doi: 10.1007/s11704-024-3577-3.
[22] G. Balde, S. Roy, M. Mondal, and N. Ganguly, “Learning faster with better tokens: Parameter-efficient vocabulary adaptation for specialized text summarization,” in Proc. 64th Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1: Long Papers, San Diego, CA, USA, 2026, pp. 45071–45086, doi: 10.18653/v1/2026.acl-long.2088.
[23] I. Mondshine, T. Paz-Argaman, and R. Tsarfaty, “Beyond n-grams: Rethinking evaluation metrics and strategies for multilingual abstractive summarization,” in Proc. 63rd Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1: Long Papers, 2025, pp. 19019–19035, doi: 10.18653/v1/2025.acl-long.932.
[24] A. Conneau et al., “Unsupervised cross-lingual representation learning at scale,” in Proc. 58th Annu. Meeting Assoc. Comput. Linguistics (ACL), 2020, pp. 8440–8451, doi: 10.18653/v1/2020.acl-main.747.
[25] S. Rothe, J. Maynez, and S. Narayan, “A thorough evaluation of task-specific pretraining for summarization,” in Proc. 2021 Conf. Empirical Methods Natural Lang. Process. (EMNLP), 2021, pp. 140–145, doi: 10.18653/v1/2021.emnlp-main.12.
[26] Y. Ye et al., “CC-Tuning: A cross-lingual connection mechanism for improving joint multilingual supervised fine-tuning,” in Proc. 63rd Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1: Long Papers, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, Eds., Vienna, Austria, 2025, pp. 19036–19051, doi: 10.18653/v1/2025.acl-long.933.
[27] J. Daðason and H. Loftsson, “Subword tokenization for low- and medium-resource languages: A systematic evaluation,” in Proc. 35th Int. Joint Conf. Artif. Intell. (IJCAI), 2026, pp. 5684–5692, doi: 10.24963/ijcai.2026/633.
[28] F. Feng, Y. Yang, D. Cer, N. Arivazhagan, and W. Wang, “Language-agnostic BERT sentence embedding,” in Proc. 60th Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1: Long Papers, Dublin, Ireland, 2022, pp. 878–891, doi: 10.18653/v1/2022.acl-long.62.
[29] L. Wang et al., “Parameter-efficient fine-tuning in large language models: A survey of methodologies,” Artif. Intell. Rev., vol. 58, no. 8, Art. no. 227, 2025, doi: 10.1007/s10462-025-11236-4.
[30] J. Nicolas, V. Gunawan, A. Sutanto, and Hidayaturrahman, “Evaluation of the effect of back-translation direction on machine translation quality for low-resource Sundanese-English language,” Procedia Comput. Sci., vol. 269, pp. 784–796, 2025, doi: 10.1016/j.procs.2025.09.021.
[31] M. Nadǎş, L. Dioşan, and A. Tomescu, “Synthetic data generation using large language models: Advances in text and code,” IEEE Access, vol. 13, pp. 134615–134633, 2025, doi: 10.1109/ACCESS.2025.3589503.
[32] J. Karthikeyan, M. Y. Sayed, Y. Waykar, V. Sathya, K. P. R, and M. Suseela Grace Padma, “Natural language processing for multilingual and low-resource languages,” in Proc. 2025 IEEE 7th Int. Conf. Comput., Commun. Autom. (ICCCA), 2025, pp. 1–7, doi: 10.1109/ICCCA66364.2025.11325195.
[33] T. Zhang, F. Ladhak, E. Durmus, P. Liang, K. McKeown, and T. B. Hashimoto, “Benchmarking large language models for news summarization,” Trans. Assoc. Comput. Linguistics, vol. 12, pp. 39–57, 2024, doi: 10.1162/tacl_a_00632.
[34] M. Bhandari, P. N. Gour, A. Ashfaq, P. Liu, and G. Neubig, “Re-evaluating evaluation in text summarization,” in Proc. 2020 Conf. Empirical Methods Natural Lang. Process. (EMNLP), B. Webber, T. Cohn, Y. He, and Y. Liu, Eds., 2020, pp. 9347–9359, doi: 10.18653/v1/2020.emnlp-main.751.
[35] A. R. Fabbri, W. Kryściński, B. McCann, C. Xiong, R. Socher, and D. Radev, “SummEval: Re-evaluating summarization evaluation,” Trans. Assoc. Comput. Linguistics, vol. 9, pp. 391–409, 2021, doi: 10.1162/tacl_a_00373.
[36] Y. Liu, A. Fabbri, P. Liu, Y. Zhao, L. Nan, R. Han, S. Han, S. Joty, C.-S. Wu, C. Xiong, and D. Radev, “Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation,” in Proc. 61st Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1: Long Papers, Toronto, Canada, 2023, pp. 4140–4170, doi: 10.18653/v1/2023.acl-long.228.
[37] U. Shaham, M. Elbayad, V. Goswami, O. Levy, and S. Bhosale, “Causes and cures for interference in multilingual translation,” in Proc. 61st Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1: Long Papers, Toronto, Canada, 2023, pp. 15849–15863, doi: 10.18653/v1/2023.acl-long.883.
[38] M. Artetxe, S. Ruder, and D. Yogatama, “On the cross-lingual transferability of monolingual representations,” in Proc. 58th Annu. Meeting Assoc. Comput. Linguistics (ACL), 2020, pp. 4623–4637, doi: 10.18653/v1/2020.acl-main.421.
[39] N. Muennighoff et al., “Scaling data-constrained language models,” in Advances in Neural Information Processing Systems, vol. 36, 2023, doi: 10.52202/075280-2191.
[40] A. D. Kotkar, R. S. Mahadik, P. G. More, and S. A. Thorat, “Comparative analysis of transformer-based large language models (LLMs) for text summarization,” in Proc. 2024 1st Int. Conf. Adv. Comput. Emerg. Technol. (ACET), 2024, pp. 1–7, doi: 10.1109/ACET61898.2024.10730348.
[41] M. La Quatra and L. Cagliero, “BART-IT: An efficient sequence-to-sequence model for Italian text summarization,” Future Internet, vol. 15, no. 1, Art. no. 15, 2023, doi: 10.3390/fi15010015.
[42] R. Aharoni, S. Narayan, J. Maynez, J. Herzig, E. Clark, and L. Lapata, “Multilingual summarization with factual consistency evaluation,” in Findings Assoc. Comput. Linguistics: ACL 2023, A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds., Toronto, Canada, 2023, pp. 3562–3591, doi: 10.18653/v1/2023.findings-acl.220.
[43] O. Razuvayevskaya, B. Wu, J. A. Leite, F. Heppell, I. Srba, C. Scarton, K. Bontcheva, and X. Song, “Comparison between parameter-efficient techniques and full fine-tuning: A case study on multilingual news article classification,” PLOS ONE, vol. 19, no. 5, Art. no. e0301738, 2024, doi: 10.1371/journal.pone.0301738.
[44] Y. Bai, Y. Gao, and H. Huang, “Cross-lingual abstractive summarization with limited parallel resources,” in Proc. 59th Annu. Meeting Assoc. Comput. Linguistics (ACL) and 11th Int. Joint Conf. Natural Lang. Process. (IJCNLP), vol. 1: Long Papers, 2021, pp. 6910–6924, doi: 10.18653/v1/2021.acl-long.538.
[45] R. Hada et al., “Are large language model-based evaluators the solution to scaling up multilingual evaluation?” in Findings Assoc. Comput. Linguistics: EACL 2024, St. Julian’s, Malta, 2024, pp. 1051–1070, doi: 10.18653/v1/2024.findings-eacl.71.
[46] R. Wijayanti, M. L. Khodra, K. Surendro, and D. H. Widyantoro, “Learning bilingual word embedding for automatic text summarization in low-resource language,” J. King Saud Univ. Comput. Inf. Sci., vol. 35, no. 4, pp. 224–235, 2023, doi: 10.1016/j.jksuci.2023.03.015.
[47] A. Kaushik, S. H. Attri, and R. S. Jha, “Exploring text summarization techniques: A review of current challenges and future directions,” in Proc. 2024 2nd Int. Conf. Disruptive Technol. (ICDT), 2024, pp. 289–295, doi: 10.1109/ICDT61202.2024.10489243.
[48] D. Das, S. Guha, and B. Semaan, “How do datasets, developers, and models affect biases in a low-resourced language? The case of the Bengali language,” in Proc. 2026 ACM SIGCAS/SIGCHI Conf. Comput. Sustain. Soc. (COMPASS), 2026, pp. 340–357, doi: 10.1145/3811242.3819100.
[49] H. Schroeder, D. Roy, and J. Kabbara, “Just put a human in the loop? Investigating LLM-assisted annotation for subjective tasks,” in Findings Assoc. Comput. Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, Eds., Vienna, Austria, 2025, pp. 25771–25795, doi: 10.18653/v1/2025.findings-acl.1323.
[50] F. Franzon and C. Zanini, “The entropy of morphological systems in natural languages is modulated by functional and semantic properties,” J. Quant. Linguistics, vol. 30, no. 1, pp. 42–66, 2023, doi: 10.1080/09296174.2022.2063501.
[51] J. Z. Forde, R. Zhang, L. Sutawika, A. F. Aji, S. Cahyawijaya, G. I. Winata, M. Wu, C. Eickhoff, S. Biderman, and E. Pavlick, “Re-evaluating evaluation for multilingual summarization,” in Proc. 2024 Conf. Empirical Methods Natural Lang. Process. (EMNLP), Miami, FL, USA, 2024, pp. 19476–19493, doi: 10.18653/v1/2024.emnlp-main.1085.
[52] T. A. Chang, C. Arnett, Z. Tu, and B. K. Bergen, “When is multilinguality a curse? Language modeling for 250 high- and low-resource languages,” in Proc. 2024 Conf. Empirical Methods Natural Lang. Process. (EMNLP), Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, Eds., Miami, FL, USA, 2024, pp. 4074–4096, doi: 10.18653/v1/2024.emnlp-main.236.
[53] J. Pfeiffer et al., “Lifting the curse of multilinguality by pre-training modular transformers,” in Proc. 2022 Conf. North Amer. Chapter Assoc. Comput. Linguistics: Human Lang. Technol. (NAACL-HLT), M. Carpuat, M.-C. de Marneffe, and I. V. Meza Ruiz, Eds., Seattle, WA, USA, 2022, pp. 3479–3495, doi: 10.18653/v1/2022.naacl-main.255.
[54] V. Yogarajan, G. Dobbie, and T. T. Keegan, “Debiasing large language models: Research opportunities,” J. Roy. Soc. New Zealand, vol. 55, no. 2, pp. 372–395, 2025, doi: 10.1080/03036758.2024.2398567.
[55] H. A. Pandya and B. S. Bhatt, “Does learning from language family help? A case study on a low-resource question-answering task,” Natural Lang. Process., vol. 31, no. 2, pp. 375–392, 2025, doi: 10.1017/nlp.2024.13.
[56] L. Xu, H. Xie, S. J. Qin, X. Tao, and F. L. Wang, “Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 48, no. 6, pp. 6107–6126, Jun. 2026, doi: 10.1109/TPAMI.2026.3657354.
[57] T. R. McIntosh et al., “Inadequacies of large language model benchmarks in the era of generative artificial intelligence,” IEEE Trans. Artif. Intell., vol. 7, no. 1, pp. 22–39, 2026, doi: 10.1109/TAI.2025.3569516.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Charles Munyao Muia, Aaron Mogeni Oirere, Stephen Kahara Wanjau
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
