Multilingual Tokenization Efficiency in Large Language Models: A Study on Indian Languages
Authors: Mohamed Azharudeen M, Balaji Dhamodharan
Authors: Mohamed Azharudeen M, Balaji Dhamodharan
Authors: Sriram Gudimella, Rohit Zajaria, Jagmeet Sarna
Authors: Shubhradeep Nandi, Kalpita Roy
Author(s):Anand Jha
Author(s):Anusha Bhagam,Naveen Kumar Kokku,Monali Barbate
Author(s):Manikanta Allanki,Tanya Kaintura,Rajeev Ranjan
Author(s):Namrata Hanspal,Rashmi Agarwal,Sanjeev Chidambaram
Author(s):Ria Nag,Padmakumar Nambiar
Author(s):Moyan Mei,Rohit Sroch
Author(s):Saruchi,Sanjay Shukla
Author(s):Avinash Patel,Mrinal Haloi,Asif Salim,Shashank Shekhar
Author(s):Ankush Chopra,Mahima Arora,Shubham Pandey
Author(s):Udit Shrivastava,Ray Uwe Hidaka