Multilingual Tokenization Efficiency in Large Language Models: A Study on Indian Languages
Authors: Mohamed Azharudeen M, Balaji Dhamodharan
Authors: Mohamed Azharudeen M, Balaji Dhamodharan
Authors: Sriram Gudimella, Rohit Zajaria, Jagmeet Sarna
Authors: Shubhradeep Nandi, Kalpita Roy
Author(s):Varun Aggarwal,Anurag Mukherjee,Shikhar Baid,Preeti Aggarwal
Author(s):Anil Prasad,Vibhav Patil
Author(s): Vishal Pandey, Pravar Kulbhushan, Sakshi Tyagi, Sudipta Mukhopadhyay, Jacob Vishal, Rik Das, Indrajit Kar