Structured output stops breaking when Claude retries

2 hours ago

Version 4.4.0 of the Claude Code provider for the Vercel AI SDK fixes a failure you may have hit with streamText and a structured output: when Claude retried its answer, the stream broke. It also fixes truncated and oversized tool call inputs, and it changes one default.

Ask about this presentation

Chapters

  1. 0:00Structured output stops breaking when Claude retries
  2. 0:19A retried answer arrived as two objects
  3. 0:44partialOutputStream now delivers one object
  4. 1:08Opt back in with streamStructuredOutput
  5. 1:31Only the root answer reaches your object
  6. 1:53Tool calls finalize from the complete input
  7. 2:14Oversized tool inputs raise an error
  8. 2:32Upgrade to 4.4.0
Show transcript
  1. Version 4.4.0 of the Claude Code provider for the Vercel AI SDK fixes a failure you may have hit with streamText and a structured output: when Claude retried its answer, the stream broke. It also fixes truncated and oversized tool call inputs, and it changes one default.

    Slide text
    npm v4.4.0 AI SDK v7

    ai-sdk-provider-claude-code 4.4.0 fixes streamText structured output and tool-call input

    ben-vargas ben-vargas/ai-sdk-provider-claude-code
  2. With streamText and an output schema, the provider forwarded the model's JSON as text the moment it arrived. Say the first attempt gives the answer as a string. The CLI rejects it for a schema mismatch, and the model retries with a number. Both attempts were already streamed, so the AI SDK parsed them as one string and threw NoObjectGeneratedError, even though the CLI's result held a valid object.

    Slide text
    fixed #175

    A retried answer reached the AI SDK as two objects glued together

    {"answer":"forty-two"} attempt 1: rejected, schema mismatch
    {"answer":42} attempt 2: valid
    {"answer":"forty-two"}{"answer":42} NoObjectGeneratedError
  3. The fix changes how structured output streams. Before, partialOutputStream updated field by field. Now the provider buffers each attempt and emits the CLI-validated object once, when the run finishes, the way doGenerate already did. Tool progress, reasoning and raw events still stream live. If your UI draws a live preview of the object, this is the change you'll notice.

    Slide text
    changed default

    partialOutputStream now delivers the finished object once

    4.3.3
    4.4.0 validated structured_output
    partial object tool progress, reasoning, raw events: still live
  4. Want the old behavior? Pass streamStructuredOutput true in the model settings. It defaults to false because streamed text can't be taken back. A run where the model answers twice still fails, and a preview can show fields from a rejected attempt. For a safer preview, keep the default and read the raw SDK events through onSdkMessage or includeRawChunks.

    Slide text
    added streamStructuredOutput

    Want field-by-field partials back? Turn on streamStructuredOutput

    const { partialOutputStream, output } = streamText({
      model: claudeCode('sonnet', { streamStructuredOutput: true }),
      output: Output.object({ schema: UserProfileSchema }),
      prompt: 'Generate a user profile for a software developer',
    });
    Defaults to false. With it on, a retried answer still fails with NoObjectGeneratedError.
    Preview without the risk: keep the default and read raw events via onSdkMessage or includeRawChunks.
  5. Three smaller structured output fixes come with it. A subagent's own StructuredOutput call can't end up in your root object anymore. The CLI's accept and reject results for that internal tool no longer show up as unknown tool calls, in doStream or doGenerate. And recovery skips any attempt the CLI rejected.

    Slide text
    fixed StructuredOutput

    Only the root answer reaches your object now

    Subagents A subagent's own StructuredOutput call stays out of the root object
    Tool results The CLI's accept and reject frames for the internal tool are hidden in doStream and doGenerate
    Recovery Takes the latest attempt the CLI didn't reject; a truncated stream no longer flushes half an attempt
  6. Ordinary tool calls get a fix too. When a streamed fragment was dropped or arrived out of order, doStream could finalize the call from incomplete input, like this half-written file path. Now the complete assistant input wins, unless the streamed bytes encode the same value. And input that already streamed isn't emitted a second time.

    Slide text
    fixed #179
    {"file_path": streamed fragments, one dropped
    →
    complete assistant input used when the streamed value differs
    Ordinary tool input deltas are emitted once, even when the full tool block arrives before its stop event.
  7. Next, oversized tool inputs. The size check ran inside the try block meant for JSON stringify failures, so an object over the one megabyte limit came through as the string object Object. Now the error propagates. doGenerate rejects, and doStream emits an error part.

    Slide text
    fixed #178

    Object tool inputs over 1 MB now raise the size-limit error

    4.3.3 emitted "[object Object]"
    4.4.0 raises Tool input exceeds maximum size doGenerate rejects · doStream emits an error part
  8. To upgrade, install the provider with npm. Version 4.x is the latest tag, for AI SDK version 7. Decide one thing first. If your app shows partial objects as they stream, accept one complete object at the end, or opt back in and live with the occasional failed run. Everyone else gets structured output that holds up when Claude retries.

    Slide text
    npm v4.4.0

    Upgrade on the 4.x line

    npm install ai-sdk-provider-claude-code ai
    Live object previews? Accept one object at the end, or set streamStructuredOutput: true
    Huge tool inputs? Expect the size-limit error where "[object Object]" used to appear
    ben-vargas github.com/ben-vargas/ai-sdk-provider-claude-code

What happened in AI today, in your inbox

One short email each evening. Read it and you’re caught up.

Built in New York with ❤️