feat: complete OpenAI browser tools and add Bash

2 hours ago

The OpenAI driver in Browser Use now exposes all 28 browser tools, and it adds Bash. Before this change, it left out four of them: scoped page reads, console and network diagnostics, file uploads, and file downloads, and it had no shell. All of those work now, each with explicit limits on what it can read and move.

Ask about this presentation

Chapters

  1. 0:00All 28 browser tools, and Bash
  2. 0:21Where the change lands
  3. 0:40Bash
  4. 1:03Scoped reads
  5. 1:27Diagnostics
  6. 1:49File grants
  7. 2:15Uploads and downloads
  8. 2:40Form state in read_page
  9. 2:59Evidence
  10. 3:24Open review findings
  11. 3:47Limits and next steps
Show transcript

All 28 browser tools, and Bash

The OpenAI driver now exposes all 28 browser tools, and adds Bash

browser_use/integrations/openai

new in this change
1read_page_scoped
2read_diagnostics
3upload_file
4download_file
+Bash

The OpenAI driver in Browser Use now exposes all 28 browser tools, and it adds Bash. Before this change, it left out four of them: scoped page reads, console and network diagnostics, file uploads, and file downloads, and it had no shell. All of those work now, each with explicit limits on what it can read and move.

Where the change lands

Browser Use
integrations/openai/browser_use.py

BrowserUse gains three kinds of state, and close() clears them

# BrowserUse.__init__
self._scopes: dict[str, Reference] = {}
self._grants: dict[str, FileGrant] = {}
self._diagnostic_client: CDPClient | None = None
self._diagnostic_sessions: dict[str, str] = {}
self._diagnostics: dict[str, dict[str, deque[str]]] = {}

# BrowserUse.close
if self._diagnostic_client:
    await self._diagnostic_client.stop()
    self._diagnostic_client = None
self._grants.clear()
self._scopes.clear()
4 files changed
openai/browser_use.py+241
scoped reads, diagnostics, file grants
openai/bash.py+40
new: the Bash tool
openai/__init__.py+2
exports Bash
test_openai_browser_use.py+137
Chromium integration tests

Most of the change lands in browser_use.py, which grows by about 240 lines. The BrowserUse class picks up three kinds of state. Read scopes and file grants, and a second CDP client with its sessions and log buffers. Close now stops that client and clears the rest.

Bash

Browser Use
integrations/openai/bash.py

Bash runs on the same executor as the Anthropic integration

from browser_use.integrations.anthropic.bash import run_bash

def Bash(
    *,
    output_dir: str | Path = 'outputs',
    timeout_seconds: float = 120,
    max_output_bytes: int = 50_000,
) -> AsyncRegisteredFunction:
    async def execute(args: BashInput) -> str:
        return await run_bash(args.command, ...)
test, with OPENAI_API_KEY set
command printf result > report.txt; printenv OPENAI_API_KEY || true exit_code 0 output '' report.txt 'result'

Bash is a new 40-line file with almost no logic of its own. It imports run_bash from the Anthropic integration, so both providers share one bounded executor. A command gets 120 seconds and 50,000 bytes of output by default. The test puts an OpenAI key in the environment and asks the shell to print it. The output comes back empty.

Scoped reads

Browser Use
BrowserUse.read_page_scoped

A scoped read returns one region of the page and leaves out the rest

test fixture
mainscope_…
Inside
navigation
Outside
root=None → scopes: main, navigation, … root=<main scope> → [ref_…] button Inside
200 lines2,000 nodes100 scopesdepth you set

After a navigation: Unknown or stale read scope.

A scoped read covers one region of a page. Called without a root, read_page_scoped lists landmark regions like main and navigation, with a scope reference for each. Called with the main scope, it walks only that subtree, so the test gets the Inside button and leaves out the Outside one. Reads stop at 200 lines or 2,000 nodes, and a scope goes stale after a navigation.

Diagnostics

Browser Use
BrowserUse.read_diagnostics

Diagnostics keep the last 100 console and network lines for each tab

Browser tab
Target.attachToTarget
flatten: true
→
Second CDP client
Runtime.consoleAPICalled
Network.responseReceived
→
deque(maxlen=100)
console · network
visited http://127.0.0.1:<port>/diagnostic?secret=hidden recorded 200 http://127.0.0.1/diagnostic

Diagnostics run on a second CDP connection. The first time the driver touches a tab, it attaches a client that listens for console calls and network responses, and keeps the last 100 of each. Network lines keep the status, host and path, and drop the query string, so the secret in this test address stays out of what the model reads.

File grants

Browser Use
FileGrant · _consume_grant

Your code approves each file transfer, and the model can spend it once

Your application
grant_upload(path,
  target=ref)
grant_download(url, path)
returns a grant ID
→
The model
upload_file
download_file
{"grant_id": …}
sees only the ID
→
Before any bytes move
  • grant is popped: single use
  • operation matches
  • same tab, same document

File transfers are split between your code and the model. Your application calls grant_upload or grant_download in Python and gets back a grant ID. The model only sees that ID, and spends it through upload_file or download_file. Before anything moves, the driver pops the grant, checks the operation, and checks that the tab still holds the same document. A second use, or a page that navigated away, is rejected.

Uploads and downloads

Browser Use
upload_file · download_file

Uploads send the bytes you approved, and downloads only create new files

↑Upload

  • Read at grant time, up to 10 MB
  • Set on the file input in the page, then input and change fire
Test: the file is rewritten after approval, and the input still gets approved bytes

↓Download

  • In-page fetch: redirect: 'error', 30 s timeout, stops past 10 MB
  • Written with open(…, 'xb'): an existing destination is refused
Test: a navigation before the download rejects the grant, and the file is never created

Uploads send the bytes you approved. The file is read when the grant is made, up to 10 megabytes, and set on the input inside the page. The test rewrites the file after approval, and the page still receives the original bytes. Downloads run as a fetch inside the page that rejects redirects and stops past 10 megabytes. The result is always a new file, so an existing destination is refused.

Form state in read_page

Browser Use
BrowserUse.read_page

read_page now says whether a box is checked

checkbox "Ready" checked="false"
after the driver changes them
checkbox "Ready" checked="true"
option "Beta" selected=true

The plain page read now reports form state. A checkbox line carries checked equals false, and after the driver checks it, the same line reads checked equals true. Select options carry selected the same way. The branch also brings a screenshot fix that lines up screenshot pixels with pointer coordinates.

Evidence

Browser Use
tests/ci/integrations

17 Chromium tests and a Browser Use Cloud round trip passed

17

real Chromium integration tests, plus pre-commit and type checks

201Created a standalone browser V4
200Requested a browser upload grant V2
201Uploaded to storage, read exact bytes in the file input DOM.setFileInputFiles
200Cleanup, after the native download bytes matched V4 downloads endpoint
422200 MB upload grant refused backend caps uploads at 10 MiB

The change passed 17 integration tests against a real Chromium, plus pre-commit and type checks. A smoke test on Browser Use Cloud created a browser, got an upload grant, uploaded to storage, and read back the exact bytes through the file input. A native download matched too. A 200 megabyte upload grant came back 422, because the backend caps uploads at about 10 megabytes.

Open review findings

Browser Use
review · Cursor Bugbot

Cursor Bugbot flagged four medium-severity issues in review

MEDIUM
New tabs miss load diagnostics
new_tab never starts the observer, so the first document response and early console output are lost
MEDIUM
A mismatched grant is discarded
_consume_grant pops the grant before it checks the operation
MEDIUM
A failed diagnostic stop skips cleanup
close awaits stop() before tearing down the owned browser
MEDIUM
Scoped reads drop roles and state
options, tabs and switches get no refs; no checked or selected

Cursor's Bugbot flagged four medium-severity issues in review. New tabs miss their first page load in diagnostics. A mismatched grant is thrown away before the operation check. A failed diagnostic stop skips browser cleanup. And scoped reads skip options, tabs and switches, along with their checked state. Three of the four describe code that is still in this diff.

Limits and next steps

All 28 browser tools and Bash, with files capped at 10 MB

not in this change
  • 200 MB transfers or new Cloud limits
  • The 200-task benchmark on this driver
  • Docs past preview
recommended for large Cloud files
  • Native Cloud upload and download endpoints

This change caps every transfer at 10 megabytes a file. It doesn't claim 200 megabyte support, the 200-task benchmark still runs the earlier driver, and the docs stay in preview. For large files on Cloud, the author recommends the native Cloud upload and download endpoints, once the backend limits go up. If you rely on diagnostics for new tabs, start with the Bugbot findings. As merged, the OpenAI driver exposes all 28 browser tools, and adds Bash.

What happened in AI today, in your inbox

One short email each evening. Read it and you’re caught up.

Built in New York with ❤️