Multilingual Tokenization Efficiency in Large Language Models: A Study on Indian Languages
Authors: Mohamed Azharudeen M, Balaji Dhamodharan
Authors: Mohamed Azharudeen M, Balaji Dhamodharan
Authors: Sriram Gudimella, Rohit Zajaria, Jagmeet Sarna
Authors: Shubhradeep Nandi, Kalpita Roy
Author(s): Varen Gupta
Author(s): Ullas M S Rao, Mattias Jönsson
Author(s): Taaniya Arora, Shashank Srivastava, Neha Prabhugaonkar
Author(s): Shekar Ramachandran, Aakshi Mittal, Rudra Nath Palit, Anmol Bhasin, Aruna Kumari V, Rupali Agrahari,
Author(s): Saurabh Singh Thakur
Author(s): Raghvendra Tiwari, Rishabh Gupta
Author(s): Praveen Manoharan, Nilesh Nayan, Aaditya Sharma, Aravindakumar Venugopalan
Author(s): Paritosh Sinha, Mohan Krishna Askani
Author(s): Deewakar Thakyal, Adwait Kelkar, Biswajit Biswas
Author(s): Anik Chakraborty, Sayantani Ghosh, Raktim Chakraborty,Dr. Indranil Mitra, Prasun Nandy