LiveKit Agent

Integrate a Tavus Face into LiveKit as the conversational video avatar.

We recommend using Tavus’s Full Pipeline in its entirety for the lowest latency and most optimized multimodal experience. Third party integrations only provide rendering, while our Full Pipeline includes perception, turn-taking, and rendering for complete conversational intelligence.

With the LiveKit Agents integration, LiveKit runs your voice assistant in the room while Tavus renders the avatar. Start a tavus.AvatarSession with your face_id as in Step 2.

Tavus enables AI developers to create realistic video avatars powered by state-of-the-art speech synthesis, perception, and rendering pipelines. Through its integration with the LiveKit Agents application, you can seamlessly add conversational avatars to real-time voice AI systems.

Prerequisites

Make sure you have the following before starting:

• Tavus face_id
  • You can use Tavus’s stock faces or your own custom face.

• LiveKit Voice Assistant App (Python or Node.js)
  • Your own existing application.
  • Or follow the LiveKit Voice AI quickstart to create one (same guide for Python and Node.js).

We recommend using Phoenix-4.5 Pro faces, which are optimized for low-latency, real-time applications.

Integration Guide

1. Step 1: Setup and Authentication

   1. Install the plugin:

   Python

   pip install "livekit-agents[tavus]~=1.0"
   

   Node.js

   npm install @livekit/agents @livekit/agents-plugin-tavus
   

   2. Set TAVUS_API_KEY in your .env file. Use the same value as your Tavus API key from Authentication.
2. Step 2: Add an Avatar to your Agent

   In your LiveKit app, create an avatar session alongside your AgentSession:

   Python

   from livekit import agents
   from livekit.agents import AgentSession, RoomOutputOptions
   from livekit.plugins import tavus
   
   async def entrypoint(ctx: agents.JobContext):
       await ctx.connect()
   
       session = AgentSession(
           # Add STT, LLM, TTS, and other components here
       )
   
       avatar = tavus.AvatarSession(
           face_id="rc9cff32ceba",
           # Optional: avatar_participant_name="Tavus-avatar-agent"
       )
   
       await avatar.start(session, room=ctx.room)
   
       await session.start(
           room=ctx.room,
           room_output_options=RoomOutputOptions(
               audio_enabled=False  # Tavus handles audio separately
           )
       )
   

   Node.js / TypeScript

   import { type JobContext, WorkerOptions, cli, defineAgent, voice } from '@livekit/agents';
   import * as tavus from '@livekit/agents-plugin-tavus';
   import { fileURLToPath } from 'node:url';
   
   export default defineAgent({
     entry: async (ctx: JobContext) => {
       await ctx.connect();
   
       const session = new voice.AgentSession({
         // Add STT, LLM, TTS, and other components here
       });
   
       const avatar = new tavus.AvatarSession({
         faceId: 'rc9cff32ceba',
         // Optional: avatarParticipantName: 'Tavus-avatar-agent'
       });
   
       // Start avatar first so it joins the room before the session pipes audio out
       await avatar.start(session, ctx.room);
   
       await session.start({
         agent: new voice.Agent({ instructions: 'You are a helpful assistant.' }),
         room: ctx.room,
       });
     },
   });
   
   cli.runApp(new WorkerOptions({ agent: fileURLToPath(import.meta.url) }));
   

   Parameter	Description
   face_id / faceId (string)	ID of the Tavus face to render and speak through.
   pal_id / palId (string, optional)	ID of a custom PAL. Defaults to a stock LiveKit PAL when omitted.
   avatar_participant_name / avatarParticipantName (string, optional)	Display name for the avatar participant in the room. Defaults to Tavus-avatar-agent.

   replica_id / replicaId and persona_id / personaId are still accepted as deprecated aliases for face_id / pal_id.

Try out the integration using this sample code.

Advanced: Avatar Emotions

This section is optional. The integration above works without it.

The Tavus PAL can visually express an emotion while it listens and speaks. Call set_emotion / setEmotion on the avatar session to set it. The PAL will show this emotion until the next call of set_emotion / setEmotion.

Supported emotions: neutral, happiness, sadness, anger, fear, surprise, disgust, contempt.

start() returns before the avatar joins the room, and an emotion sent before the avatar joins is dropped. Wait for the avatar with wait_for_join() / waitForJoin() first.

Python

await avatar.start(session, room=ctx.room)
await avatar.wait_for_join()

await avatar.set_emotion("happiness")

To let the LLM choose the emotion, wrap it in a function tool and add the tool to your Agent:

from livekit.agents import RunContext, function_tool

@function_tool
async def set_emotion(context: RunContext, emotion: tavus.Emotion) -> None:
    """Set the facial emotion of your avatar."""
    await avatar.set_emotion(emotion)

Node.js / TypeScript

await avatar.start(session, ctx.room);
await avatar.waitForJoin();

await avatar.setEmotion('happiness');

To let the LLM choose the emotion, wrap it in a tool and add the tool to your voice.Agent:

import { llm } from '@livekit/agents';
import { z } from 'zod';

const setEmotion = llm.tool({
  description: 'Set the facial emotion of your avatar.',
  parameters: z.object({ emotion: z.enum(tavus.EMOTIONS) }),
  execute: async ({ emotion }) => avatar.setEmotion(emotion),
});

Requires livekit-plugins-tavus 1.8.9 or later (Python) or @livekit/agents-plugin-tavus 1.9.4 or later (Node.js). The plugin sends a conversation.set_emotion interaction to the avatar.