Hands-on Guide to Langfuse for LLM-Based Applications

In the fast-evolving field of artificial intelligence, particularly in developing and deploying Large Language Models (LLMs), Langfuse has emerged as a groundbreaking platform. Designed to address the unique challenges of building and maintaining LLM applications, Langfuse provides a comprehensive suite of tools for observability, prompt management, evaluation, and more. This article dives into understanding the workings of Langfuse and its key features. We will also see how we can use Langfuse to build an LLM project using Python. 

Table of Contents

1. What is Langfuse?
2. Key Features of Langfuse
3. Langfuse Integrations
4. Importance of Langfuse
5. Integration of Langfuse with OpenAI

Let us deep dive into understanding Langfuse, its key features, and its importance. We will also understand how to integrate it with OpenAI using Python.

What is Langfuse?

Langfuse is an open-source platform that enables teams to build production-grade LLM applications efficiently. Langfuse aims to simplify the complexities involved in developing LLM applications by providing robust tools for monitoring, debugging, and optimizing LLM workflows.

Key Features of Langfuse

Observability

This is an important feature of Langfuse functionality. The platform allows developers to trace and monitor LLM applications comprehensively. By capturing all relevant data points and interactions, Langfuse helps in understanding the behavior of LLMs in production environments. This feature is crucial for debugging complex applications and ensuring they perform as expected.

Prompt Management

Effective prompt management is essential for refining and optimizing LLM responses. Langfuse offers tools to manage, version, and deploy prompts seamlessly. This feature allows developers to test and iterate on their prompts within the platform, ensuring that they can achieve the desired outcomes more efficiently.

Evaluation and Metrics

Evaluating the performance and quality of LLM outputs is a critical aspect of maintaining high standards. Langfuse supports a range of evaluation mechanisms, including user feedback, model-based evaluations, and manual scoring. These tools help developers assess various aspects such as accuracy, relevance, and style of the LLM responses, providing a comprehensive view of the application’s performance.

Experimentation and Testing

Langfuse allows developers to run experiments and track application behavior before deploying new versions. This feature is invaluable for ensuring that updates do not introduce regressions or negatively impact the application’s performance. By supporting datasets and benchmarking, Langfuse helps teams validate their changes rigorously before going live.

Langfuse Integrations

Langfuse is designed to be highly adaptable and integrates seamlessly with various frameworks and SDKs. Some of them are:

• Langchain
• LlamaIndex
• OpenAI SDK
• LiteLLM
• Flowise
• Langflow

Importance of Langfuse

The development of LLM applications poses unique challenges that differ significantly from traditional software engineering. Langfuse addresses these challenges by providing tools specifically designed for the non-deterministic nature of LLMs. Key benefits of using Langfuse include:

• Open-source: Langfuse is open-source, allowing for community contributions and transparency.
• Model and framework agnostic: It supports various models and frameworks, making it versatile.
• Build for production: Langfuse is designed to handle the complexities of production environments.
• Incrementally adoptable: Teams can start small and gradually expand their use of Langfuse’s features.

Integration of Langfuse with OpenAI

We will integrate OpenAI with Langfuse and develop an LLM application. By doing so, we will explore Langfuse and understand how easy it is to use Langfuse.

To begin with, we have to install Langfuse and OpenAI in our environment.

%pip install langfuse openai

Now, we will import the libraries required for the development.

from langfuse import Langfuse
from langfuse.decorators import observe
from langfuse.openai import openai
import os

We need to call all the API keys to the environment. To call Langfuse keys, we must log in to Langfuse web and create a project. 

API keys will create two important Langfuse keys: 

1. Langfuse public key
2. Langfuse secret key

LLM API keys will power the playground and evaluation features. Here, we need to provide our OpenAI API Key.

Now that we have all our keys, we can return to our Python notebook and call the keys.

langfuse = Langfuse(
 secret_key="sk-lf-******",
 public_key="pk-lf-******",
 host="https://us.cloud.langfuse.com"
)
os.environ["OPENAI_API_KEY"]='sk-*******'

Next, let us build an application. This application will have two agents, who will collectively write a summary about a topic. 

@observe()
def summary_generator(hero):
 HP = openai.chat.completions.create(
   model="gpt-3.5-turbo",
   messages=[
       {"role": "system", "content": "Who is the best friend of the hero in Harry Potter movie?"},
       {"role": "user", "content": hero}],
   name="get-capital",
 ).choices[0].message.content
 poem = openai.chat.completions.create(
   model="gpt-3.5-turbo",
   messages=[
       {"role": "system", "content": "You are a blogger. Create a summary about this best friend."},
       {"role": "user", "content": HP}],
   name="generate-poem",
 ).choices[0].message.content
 return poem
summary_generator("Harry Potter")

This will generate a summary something like this:

“Ron Weasley is one of Harry Potter’s best friends and loyal companions throughout the iconic Harry Potter series. He is known for his humorous personality, bravery, and unwavering loyalty to his friends. Despite coming from a large, poor wizarding family, Ron is fiercely protective of those he cares about and always stands by their side in times of need. He is a skilled wizard, particularly in the realm of chess, and often provides comic relief in tense situations. Ron’s character is defined by his strong moral compass, sense of justice, and deep sense of friendship, making him a beloved and essential member of the series’ core trio.”

We can see the trace of this application and the generation of this in our Langfuse account. For that, go back to the Langfuse account and click on traces.

This will have all the traces of our applications.

We can also see the output generated by clicking on Generations or ID in traces.

Langfuse projects will have their playground as well. We can give the tasks to agents and get the work done.

Thus, Langfuse helps in developing LLM applications very easily. The best part of it is that we can integrate it with many frameworks and utilize it very easily.

Conclusion

Langfuse is poised to revolutionize the way teams build, manage, and optimize LLM applications. By offering a robust suite of tools for observability, prompt management, evaluation, and more, Langfuse simplifies the complexities involved in LLM engineering. Its open-source nature and seamless integrations make it a valuable addition to any LLM developer’s toolkit.

References

1. Link to Code
2. Langfuse-Python Documentation
3. Langfuse – Github Repository

Learn More about Large Language Models by joining following courses.