Streaming

Overview

Streaming lets you receive chat completion responses in real time as they are generated. Instead of waiting for the full response, you receive chunks of text as soon as they are available. This improves perceived latency and enables progressive rendering in your UI.

Unii uses Server-Sent Events (SSE) for streaming. Each event contains a partial completion that you can append to the displayed text.

Enabling streaming

Set stream: true in your chat completion request:

stream.ts

import { UniiClient } from "@unii/sdk";

const client = new UniiClient({
apiKey: process.env.UNII_API_KEY,
});

const stream = await client.createChatCompletion({
model: "unii-latest",
messages: [
{ role: "user", content: "Explain RAG in detail." },
],
stream: true,
});

for await (const chunk of stream) {
process.stdout.write(chunk.choices[0].delta.content || "");
}
// Output: RAG (Retrieval-Augmented Generation) combines...

stream.ts

import { UniiClient } from "@unii/sdk";const client = new UniiClient({apiKey: process.env.UNII_API_KEY,});const stream = await client.createChatCompletion({model: "unii-latest",messages: [{ role: "user", content: "Explain RAG in detail." },],stream: true,});for await (const chunk of stream) {process.stdout.write(chunk.choices[0].delta.content || "");}// Output: RAG (Retrieval-Augmented Generation) combines...

Processing stream chunks

Each chunk contains a delta object with the new content:

chunk.json

{
"id": "chatcmpl-abc123",
"object": "chat.completion.chunk",
"choices": [
  {
    "index": 0,
    "delta": {
      "content": "RAG"
    },
    "finish_reason": null
  }
]
}

chunk.json

{"id": "chatcmpl-abc123","object": "chat.completion.chunk","choices": [  {    "index": 0,    "delta": {      "content": "RAG"    },    "finish_reason": null  }]}

Accumulate the chunks to build the full response:

accumulate.ts

let fullResponse = "";

for await (const chunk of stream) {
const content = chunk.choices[0].delta.content || "";
fullResponse += content;
process.stdout.write(content);
}

console.log("\n\nFull response:", fullResponse);

accumulate.ts

let fullResponse = "";for await (const chunk of stream) {const content = chunk.choices[0].delta.content || "";fullResponse += content;process.stdout.write(content);}console.log("\n\nFull response:", fullResponse);

Streaming with a knowledge base

Streaming works with knowledge bases the same way — add knowledgeBaseId to the request:

stream-kb.ts

const stream = await client.createChatCompletion({
model: "unii-latest",
knowledgeBaseId: "kb_abc123",
messages: [
  { role: "user", content: "What is our return policy?" },
],
stream: true,
});

stream-kb.ts

const stream = await client.createChatCompletion({model: "unii-latest",knowledgeBaseId: "kb_abc123",messages: [  { role: "user", content: "What is our return policy?" },],stream: true,});

Handling the end of a stream

The last chunk contains finish_reason: "stop" (or another reason like "length"):

finish.json

{
"id": "chatcmpl-abc123",
"object": "chat.completion.chunk",
"choices": [
  {
    "index": 0,
    "delta": {},
    "finish_reason": "stop"
  }
]
}

finish.json

{"id": "chatcmpl-abc123","object": "chat.completion.chunk","choices": [  {    "index": 0,    "delta": {},    "finish_reason": "stop"  }]}

Note

The final chunk may have an empty delta.content. Check finish_reason to detect the end of the stream.

Error handling

If an error occurs during streaming, the SDK throws an exception. Wrap the loop in a try/catch block:

error-handling.ts

try {
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0].delta.content || "");
}
} catch (error) {
console.error("Stream error:", error.message);
}

error-handling.ts

try {for await (const chunk of stream) {  process.stdout.write(chunk.choices[0].delta.content || "");}} catch (error) {console.error("Stream error:", error.message);}

Next steps

Read the Stream response API reference for the full streaming specification.
Learn about chat completions for non-streaming requests.
Explore knowledge bases to ground streamed responses.

Streaming

OverviewCopy Anchor Link

Enabling streamingCopy Anchor Link

Processing stream chunksCopy Anchor Link

Streaming with a knowledge baseCopy Anchor Link

Handling the end of a streamCopy Anchor Link

Error handlingCopy Anchor Link

Next stepsCopy Anchor Link