Conversation overview

Dapr’s conversation API reduces the complexity of securely and reliably interacting with Large Language Models (LLM) at scale. Whether you’re a developer who doesn’t have the necessary native SDKs or a polyglot shop who just wants to focus on the prompt aspects of LLM interactions, the conversation API provides one consistent API entry point to talk to underlying LLM providers.

In additon to enabling critical performance and security functionality (like prompt caching and PII scrubbing), you can also pair the conversation API with Dapr functionalities, like:

• Resiliency circuit breakers and retries to circumvent limit and token errors, or
• Middleware to authenticate requests coming to and from the LLM

Dapr provides observability by issuing metrics for your LLM interactions.

Features

The following features are out-of-the-box for all the supported conversation components.

Prompt caching

Prompt caching optimizes performance by storing and reusing prompts that are often repeated across multiple API calls. To significantly reduce latency and cost, Dapr stores frequent prompts in a local cache to be reused by your cluster, pod, or other, instead of reprocessing the information for every new request.

Personally identifiable information (PII) obfuscation

The PII obfuscation feature identifies and removes any form of sensitve user information from a conversation response. Simply enable PII obfuscation on input and output data to protect your privacy and scrub sensitive details that could be used to identify an individual.

Demo

Watch the demo presented during Diagrid’s Dapr v1.15 celebration to see how the conversation API works using the .NET SDK.

Try out conversation

Quickstarts and tutorials

Want to put the Dapr conversation API to the test? Walk through the following quickstart and tutorials to see it in action:

Quickstart/tutorial	Description
Conversation quickstart	TODO

Start using the conversation API directly in your app

Want to skip the quickstarts? Not a problem. You can try out the conversation building block directly in your application. After Dapr is installed, you can begin using the conversation API starting with the how-to guide.

Next steps

• How-To: Converse with an LLM using the conversation API
• Conversation API components