Generative AI Crash Course for Non-Tech Professionals. Register Now >

A Hands-On Guide to RecurrentGemma With Hugging Face

RecurrentGemma: Google DeepMind's efficient AI model revolutionizes text generation with innovative hybrid architecture.

In the rapidly developing world of artificial intelligence, the development of efficient and high-performing language models is paramount. In this article, we will go through a groundbreaking innovation from Google DeepMind that redefines the landscape of text generation. RecurrentGemma is built on novel Griffin architecture. It combines linear recurrences with local attention mechanisms to deliver superior performance with significantly lower memory usage compared to traditional transformer models. This hybrid architecture enables RecurrentGemma to handle long sequences more efficiently. In this article, we will delve into the workings of RecurrentGemma, its key features, and its integration with Hugging Face.

Table of Contents

  1. Understanding RecurrentGemma
  2. Key Features of RecurrentGemma
  3. Technical Advantages
  4. Performance of RecurrentGemma
  5. Integration of RecurrentGemma with Hugging Face

Now, let us deep dive into RecurrentGemma and use it to create a response to a given prompt.

Understanding RecurrentGemma

Google DeepMind has developed an innovative language model named RecurrentGemma. It is designed to improve efficiency and performance over traditional transformer-based models. The model leverages the novel Griffin architecture, which combines linear recurrences with local attention mechanisms. This hybrid approach addresses some of the limitations of transformers, particularly in handling long text sequences.

Key Features of RecurrentGemma

Efficiency and Memory Usage

RecurrentGemma’s architecture is optimized for lower memory consumption, making it feasible to run on devices with limited computational resources, such as single GPUs or even CPUS. This efficiency is due to the model’s fixed-sized state, which remains constant regardless of the sequence length, unlike transformer models, where memory requirements grow with sequence length.

High Throughput

The model achieves higher throughput in generating text sequences, translating to more tokens generated per second. This capability is particularly beneficial for applications requiring the generation of long text sequences, as it allows for faster and more efficient processing. 


Despite its lower memory footprint, RecurrentGemma performs competitively with larger transformer models. For example, the RecurrentGemma-2B model matches the performance of the Gemma-2B model, which is based on transformer architecture while being more resource-efficient.


RecurrentGemma is versatile and can be used for various text generation tasks such as question answering summarization, and reasoning. Its instruction-tuned variants further enhance its ability to follow specific tasks and safety protocols, making it suitable for applications requiring high levels of accuracy and safety.

Open-Source Availability Fostering

Another significant aspect of RecurrentGemma is its open-source nature. Google DeepMind has made the code publicly available on platforms like GitHub and Kaggle, enabling researchers and developers to delve into the model’s inner workings and explore its potential. This transparency fosters collaboration and innovation within the AI community.

Technical Advantages

The Griffin architecture underlying RecurrentGemma is a significant advancement. It combines the strengths of both linear recurrences and local attention mechanisms. Linear recurrences allow the model to handle long-term dependencies more effectively, while local attention ensures the model can focus on the most relevant parts of the input. This combination not only improves performance but also reduces latency during inference.

Performance of RecurrentGemma

RecurrentGemma has demonstrated impressive performance in various benchmarks. For instance, in safety-oriented evaluations, the RecurrentGemma-2B-IT variant achieved a 59.8% win rate against the Mistral 7B v0.2 Instruct model, highlighting its robustness and reliability in following instructions and maintaining safety standards. 

Integration of RecurrentGemma with Hugging Face

In this section, we will integrate the RecurrentGemma model with Hugging Face. We will use the “RecurrentGemma-2B-IT” model to achieve the given tasks. 

To begin we have to install the git-huggingface repository and accelerate the smooth working of the code.

!pip install --upgrade git+
!pip install accelerate

Next, log in to the HuggingFace platform by using the HuggingFace token:

from huggingface_hub import notebook_login

We will now import AutoTokenizer and AutoModelCasualLM of HuggingFace to read the pre-trained recurrengemma model.

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("google/recurrentgemma-2b-it")
model = AutoModelForCausalLM.from_pretrained("google/recurrentgemma-2b-it", device_map="auto")

Let us assign the model a task and ask it to produce a proper output. We will set max_length to 1000, or else the output will be restricted to the default length.

input_text = "Create an Indian style story on Harry Potter."
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids, max_length= 1000)

“<bos>Create an Indian-style story on Harry Potter. 

In the heart of the bustling city of Mumbai, where the echoes of honking rickshaws mingled with the aromas of street food, lived a young wizard named Harry Potter. Born to a Muggle father and a magical mother, Harry had always felt an outsider in his own world.

One day, as Harry walked home from school, he stumbled upon a hidden doorway in the alleyway behind his house. Curiosity got the better of him, and he cautiously stepped through. On the other side, he found a lush green garden, filled with exotic plants and a sparkling fountain.

As Harry explored the garden, he stumbled upon a small, dusty book hidden under a pile of leaves. He picked it up and opened it, revealing a faded inscription: “The Secrets of the Ancients.”

Intrigued, Harry began to read, and to his amazement, he discovered that the book was filled with ancient spells and rituals. He realized that he had stumbled upon a hidden world, a world that had been forgotten by most.

With newfound excitement, Harry spent hours studying the book and learning the secrets of the Ancients. He practiced the spells, and to his surprise, they worked! He felt a sense of power and connection to the magic that had always been a part of his life, but which he had never fully understood.

From that day on, Harry became a regular visitor to the hidden garden. He spent his days learning, practicing, and connecting with the magic that was in his heart. He found a community of like-minded wizards and witches, and together, they formed a secret society that sought to protect the magic of the world from those who would misuse it.

And so, Harry Potter, the young wizard from Mumbai, became a legend, a symbol of hope and magic in a world that desperately needed it.<eos>”

Thus, by using the RecurrentGemma model, we were able to get a good response based on the query or request given. 


RecurrentGemma represents a significant step forward in the development of efficient and high-performing language models. Its ability to deliver transformer-level performance with reduced memory usage and higher throughput makes it a valuable tool for a wide range of applications, from academic research to practical deployments in resource-constrained environments.


  1. Link to the above Code
  2. HuggingFace Documentation
  3. RecurrentGemma Documentation

Join the following courses to learn about Google Vertex AI, Generative AI, and LangChain.

Picture of Shreepradha Hegde

Shreepradha Hegde

Shreepradha is an accomplished Associate Lead Consultant at AIM, showcasing expertise in AI and data science, specifically Generative AI. With a wealth of experience, she has consistently demonstrated exceptional skills in leveraging advanced technologies to drive innovation and insightful solutions. Shreepradha's dedication and strategic mindset have made her a valuable asset in the ever-evolving landscape of artificial intelligence and data science.

The Chartered Data Scientist Designation

Achieve the highest distinction in the data science profession.

Elevate Your Team's AI Skills with our Proven Training Programs

Strengthen Critical AI Skills with Trusted Generative AI Training by Association of Data Scientists.

Our Accreditations

Get global recognition for AI skills

Chartered Data Scientist (CDS™)

The highest distinction in the data science profession. Not just earn a charter, but use it as a designation.

Certified Data Scientist - Associate Level

Global recognition of data science skills at the beginner level.

Certified Generative AI Engineer

An upskilling-linked certification initiative designed to recognize talent in generative AI and large language models

Join thousands of members and receive all benefits.

Become Our Member

We offer both Individual & Institutional Membership.