Interactions Protocol

The Interactions Protocol lets you control and customize live conversations with a Replica in real time. You can send interaction events to the Conversational Video Interface (CVI) and listen to events the Replica sends back during the call.

Interaction Types

Observable Events

Event Ordering and Turn Tracking

All events broadcasted by Tavus include two fields for ordering and grouping:

seq (integer) — A globally monotonic sequence number. Every event gets the next value in the sequence, so a higher seq always means the event was sent later. Use this to reconcile events that may arrive out of order over the data channel.
turn_idx (integer, optional) — The conversation turn index. This value increments each time a conversation.respond interaction is received, and groups all events that belong to the same conversational turn. Use it to correlate related events — for example, an utterance, its tool calls, and the replica speaking state changes that all stem from the same user input. This field is present on conversation-related events (utterances, tool calls, speaking state changes, perception events, etc.) and omitted on events that are not tied to a specific turn.

Call Client Example

The interactions protocol uses a WebRTC data channel for communication. In Tavus’s case, this is powered by Daily, which makes setting up the call client quick and simple.

Daily JS
Daily Python
Daily React

Here’s an example of using DailyJS to create a call client in JavaScript:

The Daily app-message event is used to send and receive events and interactions between your server and CVI.

<html>
  <script crossorigin src="https://unpkg.com/@daily-co/daily-js"></script>
  <body>
    <!-- Add input field and send button -->
    <input type="text" id="messageInput" placeholder="Enter your message">
    <button onclick="sendAppMessage()">Send Message</button>

    <script>
      call = window.Daily.createFrame();
      call.on('app-message', (event) => {
        console.log('app-message', event);
      });
      
      call.join({ url: 'YOUR_CONVERSATION_URL' });

      function sendAppMessage() {
        const messageInput = document.getElementById('messageInput');
        const message = messageInput.value;
        if (message) {
          const interaction = {
            "message_type": "conversation",
            "event_type": "conversation.echo",
            "conversation_id": "YOUR_CONVERSATION_ID",
            "properties": {
              "text": `${message}`
            }
          }
          const hi = call.sendAppMessage(interaction, '*');
          console.log('Sending message: ', hi);
          console.log('Sent message: ', interaction);
          messageInput.value = '';
        }
      }
    </script>
  </body>
</html>

Here’s an example of using Daily Python to create a call client in Python:

The Daily app-message event is used to send and receive events and interactions between your server and CVI.

call_client = None

class RoomHandler(EventHandler):
    def __init__(self):
        super().__init__()
    
    def on_app_message(self, message, sender: str) -> None:
        print(f"Incoming app message from {sender}: {message}")

def join_room(url):
    global call_client
    try:
        Daily.init()
        output_handler = RoomHandler()
        call_client = CallClient(event_handler=output_handler)
        call_client.join(url)
    except Exception as e:
        print(f"Error joining room: {e}")
        raise

def send_message(message):
    global call_client
    call_client.send_app_message(message)

Here’s an example of using Daily React to create a call client in React:

The Daily app-message event is used to send and receive events and interactions between your server and CVI.

"use client"

import React, { useEffect, useRef, useState } from 'react';


const TavusConversation = () => {
  const [message, setMessage] = useState('');
  const callRef = useRef(null);
  const containerRef = useRef(null);


  useEffect(() => {
    const loadDaily = async () => {
      const DailyIframe = (await import('@daily-co/daily-js')).default;


      callRef.current = DailyIframe.createFrame({
        iframeStyle: {
          width: '100%',
          height: '500px',
          border: '0',
        }
      });


      if (containerRef.current) {
        containerRef.current.appendChild(callRef.current.iframe());
      }


      callRef.current.on('app-message', (event) => {
        console.log('app-message received:', event);
      });


      callRef.current.join({
        url: 'YOUR_CONVERSATION_URL',
      });
    };


    loadDaily();


    return () => {
      if (callRef.current) {
        callRef.current.leave();
        callRef.current.destroy();
      }
    };
  }, []);


  const sendAppMessage = () => {
    if (!message || !callRef.current) return;


    const interaction = {
      message_type: 'conversation',
      event_type: 'conversation.echo',
      conversation_id: 'YOUR_CONVERSATION_ID',
      properties: { text: message }
    };


    callRef.current.sendAppMessage(interaction, '*');
    setMessage('');
  };


  return (
    <div className="w-full h-full flex flex-col items-center">
      <div ref={containerRef} className="w-full mb-4" />
      <div>
        <input
          type="text"
          className="border p-2 mr-2"
          value={message}
          onChange={(e) => setMessage(e.target.value)}
          placeholder="Type a message"
        />
        <button onClick={sendAppMessage} className="bg-blue-500 text-white px-4 py-2 rounded">
          Send Message
        </button>
      </div>
    </div>
  );
};


export default TavusConversation;

Getting Started

Onboarding Guide

Conversational Video Interface

Replica

Video Generation

Resources

Interactions Protocol

Interaction Types

Observable Events

Event Ordering and Turn Tracking

Call Client Example

Getting Started

Onboarding Guide

Conversational Video Interface

Replica

Video Generation

Resources

​Interaction Types

​Observable Events

​Event Ordering and Turn Tracking

​Call Client Example

Interaction Types

Observable Events

Event Ordering and Turn Tracking

Call Client Example