Streaming

Why stream

A non-streaming request blocks until the model has finished generating the entire reply, then returns it all at once. For a short answer that's rarely noticeable. For a longer one, it means waiting several seconds (sometimes longer) staring at nothing before any text appears. Streaming fixes that by sending the reply piece by piece, as the model produces it, so a client can start rendering text within a fraction of a second of the request going out rather than waiting for the whole thing. This is the difference between a chat UI that feels instantaneous and one that feels like it's hanging on every message. It's opt-in and costs nothing extra to use.

Turning it on

{
  "model": "tai-4.1",
  "messages": [{"role": "user", "content": "1'den 5'e kadar say"}],
  "stream": true
}

The response's Content-Type becomes text/event-stream instead of application/json. Every event line is prefixed data: , and the connection stays open, sending one event per chunk of new content, until the stream ends with a literal data: [DONE] line.

The shape of a chunk

Most chunks carry a small piece of newly generated text in choices[0].delta.content:

data: {"id": "chatcmpl-b94...", "object": "chat.completion.chunk", "created": 1788960276, "model": "tai-4.0.6P", "choices": [{"index": 0, "delta": {"content": "1"}, "finish_reason": null}]}

data: {"id": "chatcmpl-b94...", "object": "chat.completion.chunk", "created": 1788960276, "model": "tai-4.0.6P", "choices": [{"index": 0, "delta": {"content": ", 2"}, "finish_reason": null}]}

Concatenating every delta.content value in order, across every chunk, reconstructs the full reply text. Building this concatenation is the client's job; the API does not send the accumulated text at any point, only the incremental piece each chunk adds.

The finish chunk

The last content-bearing chunk carries an empty delta and a real finish_reason ("stop", or "tool_calls" if the model is asking to call a tool, see Tool calling):

data: {"id": "chatcmpl-b94...", "object": "chat.completion.chunk", "created": 1788960276, "model": "tai-4.0.6P", "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}]}

The usage chunk

After the finish chunk, one more chunk is sent with an empty choices array and a populated usage object, the same shape as the non-streaming response's usage field:

data: {"id": "chatcmpl-b94...", "object": "chat.completion.chunk", "created": 1788960276, "model": "tai-4.0.6P", "choices": [], "usage": {"prompt_tokens": 2, "completion_tokens": 3, "total_tokens": 5}}

data: [DONE]

A client that only cares about the visible text can safely ignore any chunk with an empty choices array; a client that wants to display token usage or estimated cost after the fact should look specifically for this chunk rather than assuming the finish chunk carries it.

Reading a stream in code

# Python, using the official openai package
stream = client.chat.completions.create(
    model="tai-4.1",
    messages=[{"role": "user", "content": "Kısa bir şiir yaz"}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

The official OpenAI SDKs (Python and Node) already know how to parse this event shape, since it's the same one their own API uses; nothing tAI-specific is required in the client beyond pointing base_url at https://api.artfical.com/v1, as covered on the Authentication page.

A stream always ends cleanly

Even if something goes wrong partway through, generation fails, a request times out internally, whatever the cause, the stream still ends with a visible signal rather than the connection simply dropping with no explanation. In that case you'll see a content chunk containing a short bracketed notice, a finish chunk with "finish_reason": "stop", and then [DONE], the same well-formed ending as a normal successful stream. A client should always treat the arrival of [DONE] as the definitive end of the turn, and shouldn't assume a stream that ends without it succeeded either; a dropped connection with no [DONE] means treating the request as failed and retrying, the same as you would for a non-streaming request that timed out.