Generative AI Crash Course for Non-Tech Professionals. Register Now >

Hands-on Guide to Langfuse for LLM-Based Applications

Explore Langfuse's powerful tools for building and managing LLM applications in Python, focusing on key features.
Langfuse

In the fast-evolving field of artificial intelligence, particularly in developing and deploying Large Language Models (LLMs), Langfuse has emerged as a groundbreaking platform. Designed to address the unique challenges of building and maintaining LLM applications, Langfuse provides a comprehensive suite of tools for observability, prompt management, evaluation, and more. This article dives into understanding the workings of Langfuse and its key features. We will also see how we can use Langfuse to build an LLM project using Python. 

Table of Contents

  1. What is Langfuse?
  2. Key Features of Langfuse
  3. Langfuse Integrations
  4. Importance of Langfuse
  5. Integration of Langfuse with OpenAI

Let us deep dive into understanding Langfuse, its key features, and its importance. We will also understand how to integrate it with OpenAI using Python.

What is Langfuse?

Langfuse is an open-source platform that enables teams to build production-grade LLM applications efficiently. Langfuse aims to simplify the complexities involved in developing LLM applications by providing robust tools for monitoring, debugging, and optimizing LLM workflows.

Key Features of Langfuse

Observability

This is an important feature of Langfuse functionality. The platform allows developers to trace and monitor LLM applications comprehensively. By capturing all relevant data points and interactions, Langfuse helps in understanding the behavior of LLMs in production environments. This feature is crucial for debugging complex applications and ensuring they perform as expected.

Prompt Management

Effective prompt management is essential for refining and optimizing LLM responses. Langfuse offers tools to manage, version, and deploy prompts seamlessly. This feature allows developers to test and iterate on their prompts within the platform, ensuring that they can achieve the desired outcomes more efficiently.

Evaluation and Metrics

Evaluating the performance and quality of LLM outputs is a critical aspect of maintaining high standards. Langfuse supports a range of evaluation mechanisms, including user feedback, model-based evaluations, and manual scoring. These tools help developers assess various aspects such as accuracy, relevance, and style of the LLM responses, providing a comprehensive view of the application’s performance.

Experimentation and Testing

Langfuse allows developers to run experiments and track application behavior before deploying new versions. This feature is invaluable for ensuring that updates do not introduce regressions or negatively impact the application’s performance. By supporting datasets and benchmarking, Langfuse helps teams validate their changes rigorously before going live.

Langfuse Integrations

Langfuse is designed to be highly adaptable and integrates seamlessly with various frameworks and SDKs. Some of them are:

Importance of Langfuse

The development of LLM applications poses unique challenges that differ significantly from traditional software engineering. Langfuse addresses these challenges by providing tools specifically designed for the non-deterministic nature of LLMs. Key benefits of using Langfuse include:

  • Open-source: Langfuse is open-source, allowing for community contributions and transparency.
  • Model and framework agnostic: It supports various models and frameworks, making it versatile.
  • Build for production: Langfuse is designed to handle the complexities of production environments.
  • Incrementally adoptable: Teams can start small and gradually expand their use of Langfuse’s features.

Integration of Langfuse with OpenAI

We will integrate OpenAI with Langfuse and develop an LLM application. By doing so, we will explore Langfuse and understand how easy it is to use Langfuse.

To begin with, we have to install Langfuse and OpenAI in our environment.

%pip install langfuse openai

Now, we will import the libraries required for the development.

from langfuse import Langfuse
from langfuse.decorators import observe
from langfuse.openai import openai
import os

We need to call all the API keys to the environment. To call Langfuse keys, we must log in to Langfuse web and create a project. 

API keys will create two important Langfuse keys: 

  1. Langfuse public key
  2. Langfuse secret key

LLM API keys will power the playground and evaluation features. Here, we need to provide our OpenAI API Key.

Now that we have all our keys, we can return to our Python notebook and call the keys.

langfuse = Langfuse(
 secret_key="sk-lf-******",
 public_key="pk-lf-******",
 host="https://us.cloud.langfuse.com"
)
os.environ["OPENAI_API_KEY"]='sk-*******'

Next, let us build an application. This application will have two agents, who will collectively write a summary about a topic. 

@observe()
def summary_generator(hero):
 HP = openai.chat.completions.create(
   model="gpt-3.5-turbo",
   messages=[
       {"role": "system", "content": "Who is the best friend of the hero in Harry Potter movie?"},
       {"role": "user", "content": hero}],
   name="get-capital",
 ).choices[0].message.content
 poem = openai.chat.completions.create(
   model="gpt-3.5-turbo",
   messages=[
       {"role": "system", "content": "You are a blogger. Create a summary about this best friend."},
       {"role": "user", "content": HP}],
   name="generate-poem",
 ).choices[0].message.content
 return poem
summary_generator("Harry Potter")

This will generate a summary something like this:

“Ron Weasley is one of Harry Potter’s best friends and loyal companions throughout the iconic Harry Potter series. He is known for his humorous personality, bravery, and unwavering loyalty to his friends. Despite coming from a large, poor wizarding family, Ron is fiercely protective of those he cares about and always stands by their side in times of need. He is a skilled wizard, particularly in the realm of chess, and often provides comic relief in tense situations. Ron’s character is defined by his strong moral compass, sense of justice, and deep sense of friendship, making him a beloved and essential member of the series’ core trio.”

We can see the trace of this application and the generation of this in our Langfuse account. For that, go back to the Langfuse account and click on traces.

This will have all the traces of our applications.

We can also see the output generated by clicking on Generations or ID in traces.

Langfuse projects will have their playground as well. We can give the tasks to agents and get the work done.

Thus, Langfuse helps in developing LLM applications very easily. The best part of it is that we can integrate it with many frameworks and utilize it very easily.

Conclusion

Langfuse is poised to revolutionize the way teams build, manage, and optimize LLM applications. By offering a robust suite of tools for observability, prompt management, evaluation, and more, Langfuse simplifies the complexities involved in LLM engineering. Its open-source nature and seamless integrations make it a valuable addition to any LLM developer’s toolkit.

References

  1. Link to Code
  2. Langfuse-Python Documentation
  3. Langfuse – Github Repository

Learn More about Large Language Models by joining following courses.

Picture of Shreepradha Hegde

Shreepradha Hegde

Shreepradha is an accomplished Associate Lead Consultant at AIM, showcasing expertise in AI and data science, specifically Generative AI. With a wealth of experience, she has consistently demonstrated exceptional skills in leveraging advanced technologies to drive innovation and insightful solutions. Shreepradha's dedication and strategic mindset have made her a valuable asset in the ever-evolving landscape of artificial intelligence and data science.

The Chartered Data Scientist Designation

Achieve the highest distinction in the data science profession.

Elevate Your Team's AI Skills with our Proven Training Programs

Strengthen Critical AI Skills with Trusted Generative AI Training by Association of Data Scientists.

Our Accreditations

Get global recognition for AI skills

Chartered Data Scientist (CDS™)

The highest distinction in the data science profession. Not just earn a charter, but use it as a designation.

Certified Data Scientist - Associate Level

Global recognition of data science skills at the beginner level.

Certified Generative AI Engineer

An upskilling-linked certification initiative designed to recognize talent in generative AI and large language models

Join thousands of members and receive all benefits.

Become Our Member

We offer both Individual & Institutional Membership.