feat(ai): portable image shape with aspect and resolution

1 hour ago

This change replaces five fields on the shared image request in the AI package with two that travel between providers, an aspect ratio and a rough resolution tier. You ask for 16:9 at 2K, and each provider lowers it to its own parameters, approximates it with a notice, or fails with a typed error.

Ask about this presentation

Chapters

  1. 0:00Image requests now ask for an aspect and a resolution
  2. 0:20Swapping the image model usually meant a typed failure
  3. 0:44The shared request is now small
  4. 1:08Each provider lowers aspect and resolution to its own fields
  5. 1:32One helper does the size math for 8 protocols
  6. 1:58When a size can't match, the main routes return a notice and the rest fail
  7. 2:24Callers passing size, seed, mask, format or aspectRatio break
  8. 2:48The other fields moved to typed provider options
  9. 3:10Live runs at 16:9 land on the computed sizes
  10. 3:37What to change if you call packages/ai
Show transcript

Image requests now ask for an aspect and a resolution

opencodepackages/ai
feat(ai): portable image shape
Image.generate({ model, prompt,
  aspect: "16:9", resolution: "2K" })
Image.generate · Image.request · Video.generate

This change replaces five fields on the shared image request in the AI package with two that travel between providers, an aspect ratio and a rough resolution tier. You ask for 16:9 at 2K, and each provider lowers it to its own parameters, approximates it with a notice, or fails with a typed error.

Swapping the image model usually meant a typed failure

opencodepackages/ai
before
old fieldsizeaspectRatioseedmaskformat
OpenAI■··■■
Google·■■··
xAI·■···
Metahint···■
Fig. Support table, packages/ai/README.md, before this change. ■ lowers natively, · fails when set

The old request carried size, aspect ratio, seed, mask and format, and most providers took one or two of them. Here's the support table the README carried for the main four. OpenAI took a size but no aspect ratio, and Google the reverse. So swapping the model usually meant a typed failure. An aspect ratio and a rough size tier are what travel between them.

The shared request is now small

opencodepackages/ai
packages/ai/src/image.ts

The shared request is now small, modeled on OpenRouter's image API

Image.generate({
  model, prompt,
  images?,      // unchanged
  n?,           // unchanged
  aspect?,      // was aspectRatio
  resolution?,  // "1K" | "2K" | "4K"
  providerOptions?, http?,
})

1K ≈ the pixels of 1024×1024

at the requested aspect. Leave both out and the provider default applies.

Fig. The request shape, as written in the pull request

The new request is small, modeled on OpenRouter's image API. Images and the image count are unchanged. Aspect ratio is now just aspect. Resolution is new, an open string with 1K, 2K and 4K as tiers. 1K means about as many pixels as 1024 by 1024, at your aspect. Leave both out, and the provider default applies.

Each provider lowers aspect and resolution to its own fields

opencodepackages/ai
src/protocols/*-images.ts
provideraspectresolutionwhen it can't match
GoogleimageConfig.aspectRatioimageConfig.imageSizepassed through as is
xAIaspect_ratioresolution 1k / 2k4K uses 2k, with a notice
OpenAI + AI Gatewaygpt-image-2+: computed size within its limits; gpt-image-1, DALL·E: nearest listed sizesamea notice
Metasize hint, W×H at that rationot sentignored, with a notice
Fig. How each of the main providers lowers the shared fields

Here's how the main four lower it. Google passes both into its image config. xAI renders 1k or 2k, so a 4K request uses 2k, with a notice. OpenAI and the Vercel gateway compute a size within the model's limits, or take the nearest listed size on older models. Meta always renders about two and a half megapixels, so it ignores resolution and says so.

One helper does the size math for 8 protocols

opencodepackages/ai
src/protocols/utils/image-sizing.ts · new, 124 lines
fit(shape, limits)
aspect + tier → width × height inside step, edge, pixel and ratio limits; rounds away from a limit it lands on
nearest(shape, sizes)
the closest listed size: aspect first, then pixel count
notice(name, shape, size)
a notice when the size misses the aspect by >2% or the tier by >√2 per edge
exact(route, shape, limits)
fit, or UnsupportedOperation naming the closest size it would take
Fig. 2K at 16:9 under a 2048² pixel cap is 2720x1536

The size math lives in one helper, used by 8 protocols. Fit turns an aspect and a tier into a width and height inside a provider's limits, rounding away from any limit it lands on. Nearest picks the closest listed size, aspect first, then pixel count. Notice flags a size more than 2 percent off the aspect, or more than root two off the tier per edge. Exact fits, or fails naming the closest size it would take.

When a size can't match, the main routes return a notice and the rest fail

opencodepackages/ai
response.notices · AIError
renders, then says so
response.notices
OpenAI, Vercel AI Gateway, xAI, Meta
"OpenAI Images doesn't offer 16:9; used 1536x1024 (3:2)"
fails before the request is sent
UnsupportedOperation
BFL, fal, Z.ai, Alibaba, Cloudflare, Stability
"<route> can't render <aspect at resolution>; the closest size it takes is <W>x<H>"
Fig. Strings from image-sizing.test.ts and ImageSizing.exact

When a size can't match, OpenAI, xAI and Meta still render. They put what they used in the response notices. BFL, fal, Z.ai, Alibaba, Cloudflare and Stability fail with UnsupportedOperation instead, naming the closest size they take. For the queued routes, Black Forest Labs and fal, that is the only option, because the result is decoded from the resume token alone, without the request.

Callers passing size, seed, mask, format or aspectRatio break

opencodepackages/ai
breaking
before
Image.generate({
  model: google.image(id),
  prompt,
  aspectRatio: "16:9",
  seed: 42,
  providerOptions: {
    imageSize: "2K",
  },
})
after
Image.generate({
  model: google.image(id),
  prompt,
  aspect: "16:9",
  resolution: "2K",
  providerOptions: {
    seed: 42,
  },
})
Video: aspectRatio → aspect · exports: ImageSize, ImageFormat removed · ImageResolution added

This breaks callers that pass size, seed, mask, format or aspect ratio. Here's the README's Google example before, and after. Aspect ratio becomes aspect, 2K moves up to the shared resolution field, and seed moves down into provider options. Video gets the same rename, and the ImageSize and ImageFormat exports give way to a new ImageResolution type.

The other fields moved to typed provider options

opencodepackages/ai
breaking
routenew providerOptions
OpenAI / Vercel AI Gatewaysize (replaces aspect and resolution), outputFormat, mask
Googleseed (imageSize removed, use resolution)
xAIresolution option removed, use the shared field
Metasize (with aspect fails), outputFormat
Black Forest Labswidth, height, seed, output_format, mask
falimage_size (now also { width, height }), seed, output_format, mask
Stability (image, upscale)seed, output_format
Z.aisize
Alibabasize (W*H), seed
Cloudflare Workers AIwidth, height, seed
Fig. Where size, seed, mask and format went, per route

Everything else moved to typed provider options, on the routes that actually supported it. The whole map is on screen. On OpenAI and the gateway, a size option replaces aspect and resolution, so setting both fails. Meta's size works the same way against aspect. And xAI's own resolution option is gone, in favor of the shared field.

Live runs at 16:9 land on the computed sizes

opencodepackages/ai
evidence
modelrequestedgot
gpt-image-1-mini16:9, 1K1536x1024 + notice
gpt-image-216:9, 1K1360x768
gemini-3.1-flash-image16:9, 1K1376x768
grok-imagine-image16:9, 4K2816x1584 + notice (used 2k)
muse-image-1.016:9, 2K2096x1184 + notice (resolution ignored)
AI Gateway bfl/flux-3-image-klein16:9, 1K1360x768
Alibaba qwen-image-2.016:9, 2K2720x1536 (2736x1536 rejected, over 2048²)
Fig. Live runs from the pull request · recorded request bodies unchanged, cassettes not re-recorded

The recorded request bodies didn't change, so the test cassettes stayed as they were. Run live at 16:9, GPT Image 2 at 1K got 1360 by 768, the size fit computes. The older OpenAI model, Grok and Muse came back with notices, as designed. And Alibaba at 2K got 2720 by 1536, because 2736 wide is over its pixel cap.

What to change if you call packages/ai

opencodepackages/ai
limits · next

Replicate still rejects aspect and resolution. Queued routes (BFL, fal) fail rather than approximate.

Image requests now ask for an aspect and a resolution

aspectRatio → aspect; size, seed, mask, format → providerOptions

packages/ai/src/protocols/utils/image-sizing.tsgithub.com/anomalyco/opencode/pull/58528

Two limits remain. Replicate still rejects aspect and resolution, because its inputs are defined per model, and the queued routes fail rather than approximate. If you call the AI package, rename aspect ratio to aspect, and move size, seed, mask and format into provider options. The size logic is in image-sizing.ts. Image requests now ask for an aspect and a resolution.

What happened in AI today, in your inbox

One short email each evening. Read it and you’re caught up.

Built in New York with ❤️