A Practitioners Guide to Running Ollama models in Colab - Collama

What makes it truly remarkable is its ability to provide a streamlined, user-friendly interface for managing various AI models. Whether you're a student, a researcher, or a hobbyist, Ollama offers a gateway to explore advanced AI capabilities without the need for extensive infrastructure or deep technical expertise. By leveraging Google Colab's free cloud computing resources, you can now run sophisticated AI models directly in your web browser, making AI experimentation more accessible than ever before.

Hands-On Implementation

Step 1: Installing Dependencies

The first stage involves preparing your Colab environment. You'll need to install two key components:

pciutils: Helps to detect GPU configurations

!sudo apt update
!sudo apt install -y pciutils
!curl -fsSL https://ollama.com/install.sh | sh

Step 2: Starting the Service

Since Jupyter Notebooks run code sequentially, we'll use Python's threading to run the Ollama service in the background:

import threading
import subprocess
import time

def run_ollama_serve():
    subprocess.Popen(["ollama", "serve"])
thread = threading.Thread(target=run_ollama_serve)

thread.start()
time.sleep(5)  # Allows service to initialize

Step 3: Pulling a Language Model

Ollama offers a wide range of models. In this example, we'll pull Llama 3.2:

ollama pull llama3.2

Step 4: Integrating with LangChain

To interact with the model, we'll use LangChain's Ollama integration:

!pip install langchain-ollama

from langchain_core.prompts import ChatPromptTemplate
from langchain_ollama.llms import OllamaLLM
from IPython.display import Markdown

template = """Question: {question}

Answer: Let's think step by step."""

prompt = ChatPromptTemplate.from_template(template)
model = OllamaLLM(model="llama3.2")
chain = prompt | model

display(Markdown(chain.invoke({"question": "What's the square root of 81"})))

Output:

To find the square root of 81, let's break it down:

We know that a square root is a number that, when multiplied by itself, gives us the original value (in this case, 81).

So, we're looking for two numbers whose product equals 81.

Let's think about perfect squares close to 81: 64 and 100 are both squares of 8 and 10 respectively, but they are not our target number

Now let's try some numbers from 1-9:
If we square a 5 then we get 25 which is less than 81.
If we square an 8 then we get 64 which is also less than 81.
Let’s check what happens if we square the number 9. 9 x 9 = 81
Therefore, the square root of 81 is 9.

Model Selection and Exploration

Ollama offers a vast library of models at ollama.com/library. Some popular models include:

• Llama
• Mistral
• CodeLlama
• Phi
• Gemma
• Stable LM
• QwQ
• Qwen2.5-Coder
• Nomic-Embed-Text
• LLaVA
• CodeLlama
• Mxbai-Embed-Large
• TinyLlama
• StarCoder2
• DeepSeek-Coder
• Dolphin-Mixtral
• CodeGemma
• WizardLM2
• Orca-Mini

Each model has unique strengths, so experimenting is key to finding the right fit for your specific use case.

Final Words

The ability to run sophisticated AI models with just a few lines of code represents a significant democratization of artificial intelligence. Platforms like Ollama, combined with cloud computing resources like Google Colab, are dismantling the traditional barriers to AI experimentation. For enthusiasts, researchers, and developers, this approach opens up endless possibilities. You can now prototype AI applications, explore model capabilities, and conduct advanced research without significant upfront infrastructure investments.

References

• Ollama's Official Site