Omnibot

Omnibot is Browser Infrastructure for AI Agents.

It connects Hermes, Claude Code, Codex, OpenCode, and other agent systems to a real Chromium browser through the local omnibot daemon and CLI.

This skill is an execution specification for agents. It is not a human command manual.

When to Use

• Use Omnibot when browser runtime state matters: login, cookies, storage, client-side rendering, extensions, or visible user tabs.
• Use Omnibot when an agent must read rendered pages, click controls, fill forms, navigate, extract content, or collect evidence.
• Do not use Omnibot when static files or plain HTTP content already answer the task.

Core Rules

• Reliability > Convenience.
• Explicit State > Implicit State.
• Pattern > Command.
• Observe -> Act -> Verify.
• Fallback is allowed, but never first.
• Native Omnibot commands before scripts. Do not use execute-js first.
• Run omnibot <command> --help before using uncommon or newly introduced commands, especially batch, network, cdp, record, trace, and skills.
• If a command from memory fails with parser errors, stop using that remembered syntax and re-check omnibot --help plus the subcommand help.
• Prefer semantic locators and snapshot -i refs before selectors.
• When the user has explicitly requested a native Omnibot command and the terminal tool is available, execute it immediately; do not ask for confirmation or claim that it was not run. If dispatch genuinely fails, report the concrete tool error after one bounded retry and do not fabricate browser results.
• A multi-step browser task is incomplete until its requested final verification command has actually run. After the last mutation, always issue the concrete get/is/wait verification, inspect its result, and only then report completion; never stop after reporting that the mutation itself succeeded.
• For workflows with more than five meaningful browser actions, split the work into multiple bounded turns. Each turn should finish its own observe -> act -> verify cycle and report the retained tab id before the next turn; do not put a long mutation chain and all final checks into one model request.
• Raw cdp calls are one-shot. Do not assume Network.enable alone creates readable logs. For API/request evidence, use network clear -> network start -> one action -> network stop -> network logs/summary.
• After a failed click by ref, re-observe or use a higher-evidence fallback. Do not keep using stale @eN refs from a previous snapshot.
• For shopping, checkout, payment, banking, or irreversible workflows: Do not click final submit/pay/place-order controls unless the user explicitly confirms that final action in the current turn.
• Prefer condition-based wait over shell sleep.
• Timeout values for commands that expose a timeout option, such as wait, are seconds, not milliseconds. Use --timeout 5 for a five-second wait; never convert milliseconds into a raw CLI timeout value. dialog logs and dialog clear do not accept --timeout.
• Native dialog workflows are session-token scoped: use the exact same OMNIBOT_SESSION_TOKEN for the triggering click, dialog logs, and dialog handle. A different token cannot observe or handle the original dialog.
• Parse JSON output from commands that return JSON.
• Do not claim success without verification evidence.

Runtime Provenance

Before an agent-driven test in a source checkout, verify the command provenance:

command -v omnibot
omnibot --version

If the current workspace contains pyproject.toml and the global command is not the workspace version, run the test commands as uv run omnibot ... (or use the workspace .venv/bin/omnibot) so the agent exercises the code under test. Do not infer command support from memory or from a different packaged version; run the relevant subcommand --help after selecting the runtime.

For a source-checkout test, make this concrete before the first browser action:
define the command prefix once:

export OMNIBOT="uv run omnibot --no-start"

stop any existing daemon, start the workspace daemon with
uv run python -m omnibot --api-port 18764 --ws-port 18765 daemon run, and
invoke every subsequent CLI command as $OMNIBOT ... (or the
workspace .venv/bin/omnibot --no-start ...). Never use bare omnibot in that
workflow, because it can auto-start the globally installed packaged daemon and
silently test different code.

Native Command Router

Before any page-state operation, choose the narrowest native command that expresses the intent. JavaScript is not a convenience shortcut; it is a fallback that requires evidence that native commands could not reach the state.

If you are about to write document.querySelector, document.body.innerText, .click(), .value =, innerHTML, scrollTo, or scrollIntoView, stop and identify the native command you will try first. Only use execute-js after documenting which native tiers failed and how you will verify the result.

Text entry rule: when the user asks to enter, type, fill, or replace text, use fill or type. keyboard and press dispatch key events for shortcuts and key semantics; they are not text insertion commands and must not be substituted for text entry.

Form commit rule: after fill, type, select, check, uncheck, or upload, verify the control itself with get value, get attr, or is before judging the workflow. Many pages keep a separate saved/rendered status that does not change until a visible Save, Apply, Submit, or equivalent control is activated. If such a commit control exists and the task expects business-state verification, observe it, activate it with a native command, then verify the resulting status. Do not report a form action as failed merely because an unrelated status element is still stale, and do not report it as complete without verifying both the control value and any required commit result.

Form terminal-timeout rule: keep form workflows as short observe → one control action → one verification steps. If a terminal call times out after fill, select, check, or Save, query the browser state first; do not repeat the action blindly because the browser may already have applied it. Continue from the first unverified control and finish with one commit click plus one final status read.

Mutation retry rule: issue at most one mutating browser command (fill, type, select, check, uncheck, click, dblclick, drag, upload, or navigate) per agent request. If its terminal output is empty, delayed, or times out, the next request must be read-only (get, is, snapshot, read, tabs, or wait) to determine whether the mutation already happened. Never replay the mutation merely because the tool response was missing; duplicate fills can append text and duplicate clicks can change business state.

Drag performance rule: selector-based drag may take longer than a terminal tool's short default timeout because it dispatches a human-like event sequence. For agent workflows where exact trajectory is not required, prefer snapshot -i plus coordinate mouse drag --fast after verifying source/target boxes; then verify the target state. If semantic drag is required, allow a longer command timeout and do not retry blindly while the first drag may still be running.

Clipboard rule: both clipboard read and clipboard write require the explicit
--tab-id <TAB_ID> on every invocation, even though the clipboard is shared by
the browser extension. If no tab id is known, discover a safe target tab first;
never omit the flag or infer a default tab.

Cross-origin frame rule: snapshot visibility does not guarantee write access. After targeting a cross-origin iframe, verify fill/type focus and the resulting value; if focus fails, report the limitation explicitly and do not use JavaScript as a bypass.

Workflow Context + Tab Target

Every page-state workflow must use both controls:

• OMNIBOT_SESSION_TOKEN=<workflow-name> for workflow isolation.
• --tab-id <TAB_ID> for page targeting.

OMNIBOT_SESSION_TOKEN is a workflow/context token, not a browser session and not a tab target. It isolates agent state such as refs, trace, recording, aliases, and temporary resources. Tab ID controls which page receives the command.

Do not rely on default tab, active tab, current tab, or prior targeting state. Every page-state command requires explicit --tab-id except when the command is only discovering or creating a tab.

Use the same token inside one workflow. Use different tokens for independent workflows.

Standard form:

OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>

@eN refs are tab-scoped. Refs rule: never reuse @eN refs across tabs.

The agent dispatch pattern is token + tab-id:

1. Set a stable OMNIBOT_SESSION_TOKEN.
2. Discover or create the target tab.
3. Save the returned tab id.
4. Pass --tab-id <TAB_ID> on every command that touches page state.
5. Run observe -> act -> verify on that same tab.

Tab-id identity rule: use the exact id field returned by Omnibot tabs/tab list or by a tab-creation command. Browser-control surfaces may expose a shorter numeric tab_id; that value is not necessarily the Omnibot session id. When the discovery output contains both tab_id and id, pass the full namespaced id (for example, edge-...:12345) to --tab-id. Never reconstruct or strip the namespace.

Phased workflow rule: do not bundle navigation and several page actions into one unverified sequence. After creating or navigating a tab, first run get url (and snapshot -i when controls are involved); then perform one meaningful action; then verify its concrete state before issuing the next action. For a form or multi-control task, repeat this observe -> one action -> verify cycle for each control.

Examples of tab-locked page-state commands:

OMNIBOT_SESSION_TOKEN=checkout omnibot get value "input[name=email]" --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=checkout omnibot find placeholder "Search" --action type --action-value "omnibot" --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=checkout omnibot dom dblclick n1 --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=checkout omnibot clipboard read --tab-id <TAB_ID>

Targeting categories:

Navigation safety invariant:

• When the task says "open", "open a new tab", or "create a test page", use open <URL> or tab new <URL> and use the returned tab id. Do not substitute goto, plain navigate, or the currently active tab.
• navigate <URL> also opens a new tab by default; navigate <URL> --new-tab is an explicit equivalent accepted for agent compatibility. Use navigate <URL> --same-tab --tab-id <TAB_ID> only for deliberate same-tab reuse.
• Use goto --tab-id <TAB_ID> or navigate --same-tab --tab-id <TAB_ID> only when the task explicitly asks to reuse an existing tab.
• After creating a tab, verify get url --tab-id <TAB_ID> before any clipboard, upload, form, or other page-state operation. This prevents a failed or misinterpreted tab-creation result from mutating a user-owned page.
• Tool-created tabs are temporary workflow resources and may be auto-closed by cleanup after inactivity. Complete the intended tab workflow in the same token/context, keep the returned tab id, and do not assume it will still exist after a long model pause or a separate one-shot agent process. If a later close reports not found, re-list tabs before retrying; the cleanup may already have closed it.
• If open or tab new times out, loses its final response, or is retried after an agent/tool interruption, first run tabs or tab list and match the requested URL. Reuse one matching tab id instead of issuing another create command. Only create again after confirming that no matching tab from the interrupted attempt exists.

Observe -> Act -> Verify

All page operations must follow this loop:

1. Observe the current page state.
2. Act once using the highest reliable pattern available.
3. Verify the expected state change.

Click example:

OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=research omnibot click --tab-id <TAB_ID> @e4
OMNIBOT_SESSION_TOKEN=research omnibot snapshot -i --tab-id <TAB_ID>

Read/verify example:

OMNIBOT_SESSION_TOKEN=checkout omnibot is enabled "button[type=submit]" --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=checkout omnibot find role button --name "Submit" --action click --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=checkout omnibot wait --url "/dashboard" --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=checkout omnibot get url --tab-id <TAB_ID>

If verification fails, do not repeat blindly. Re-observe, choose the next fallback tier, and verify again.

Pattern > Command

Start from the task, not the command name. Use operation patterns for read, click, fill, select/check, navigation, wait, extraction, and batch.

Use fallback tiers only after standard patterns fail. Collect evidence separately from fallback execution. The command reference is only a lookup table; it must not decide behavior.

Read vs Snapshot Routing

Choose by intent before choosing by command name:

Do not use read as the first step for click/fill workflows unless the user explicitly asked for page content first.

Screenshots And Visual Regions

screenshot --tab-id <TAB_ID> without --ref remains a full current-viewport screenshot.
For a specific visual region, run snapshot -i and choose a ref marked [visual=true], such as
article, region, dialog, listitem, image, or video:

OMNIBOT_SESSION_TOKEN=visual omnibot snapshot -i --tab-id <TAB_ID>
OMNIBOT_SESSION_TOKEN=visual omnibot screenshot --ref @e146 --tab-id <TAB_ID> -o /tmp/region.png

Visual-region refs are intended for complete content containers, cards, posts, dialogs, media, and similar areas.
Do not screenshot a button, link, or textbox as a standalone target unless the user explicitly asks for that control.
The ref screenshot path re-resolves the live DOM node, scrolls it into view, waits for layout to settle, and then captures its region.
After read or scrolling, refresh the snapshot before using a stale ref. Raw get box plus cdp Page.captureScreenshot is not the normal screenshot workflow.

Dropdowns And Custom Comboboxes

snapshot -i may auto-probe visible custom comboboxes and append their options as refs. If the output lists @eN [option] "...", click that option ref directly; Omnibot will reopen the owning combobox through openerSelector if the dropdown closed during snapshot.

Use select @combobox "value" for native selects or simple controls. Use click @option when snapshot -i exposes auto-probed combobox option refs. Do not manually reopen every dropdown just to discover options unless snapshot -i did not expose option refs.

Rich Text Article Editors

ContentEditable rich text editors (ProseMirror, Quill, Draft.js, Slate, ByteDance/Toutiao article body, etc.) are often missing from the AX tree. snapshot -i appends them under # DOM Rich Text Editors as @eN [richtext] "..." [contenteditable=true] refs.

• Detect the body editor: snapshot -i → look for @eN [richtext] under # DOM Rich Text Editors. Do not fill the article body through a [textbox] title ref.
• Replace the whole body: fill @richtext "paragraph 1\n\nparagraph 2". Empty lines split paragraphs; the previous content is cleared and input/change events are dispatched.
• Append to the body: type @richtext " additional text".
• Focus before keyboard input: click @richtext first, then press/keyboard.
• Raw selector fallback: omnibot fill '[contenteditable="true"]' "..." also routes through the rich text path when the selector looks like a rich editor.

Do not use execute-js first for rich text writes; fill/type on the [richtext] ref is the supported path.

Quick Routing

Runtime First

Before troubleshooting browser behavior, check runtime state:

omnibot doctor
omnibot tabs
omnibot visibility status
omnibot browser current
omnibot license status

If the extension is not connected, open Chrome or Edge with the omnibot extension loaded and keep an HTTP/HTTPS tab open.

Fallback Discipline

Fallback tiers are for completing operations after better patterns fail. They are not debugging shortcuts.

Tier order:

1. Semantic find.
2. snapshot -i refs.
3. Selectors.
4. dom fallback.
5. mouse fallback.
6. execute-js fallback.
7. Raw cdp fallback.

When entering a lower tier, state why the higher tier failed. After any fallback action, verify with snapshot, get, is, wait, or equivalent evidence.

Before execute-js, the agent must have native-attempt evidence: the exact find/snapshot/click/fill/scroll/get/read commands tried, their result, why they were insufficient, and the post-JS verification command to run. Without that evidence, use a native command instead.

Debug Evidence

Debugging is for evidence. Fallback is for completing operations.

Use screenshots, annotated screenshots, console logs, network logs, trace, record/replay, and CDP inspection to explain failures or prove state. Do not treat execute-js as ordinary debug output.

Absolute Prohibitions

• Missing OMNIBOT_SESSION_TOKEN or missing --tab-id on page-state commands.
• Acting without verify or claiming success without evidence.
• Using execute-js first, raw CSS before semantic find, @eN across tabs, implicit tab targeting, or shell sleep instead of omnibot wait.
• Using JavaScript read/click/scroll shortcuts when native get/read/find/snapshot/click/scroll can express the task.
• Using removed commands: switch-tab, focus-tab, tab switch, tab focus. These are no longer available.
• Using tabs[0] or the first tab as a target. The first tab is often a user tab or transport tab.
• Closing user tabs discovered during cleanup. Only close tabs created by the current workflow.

See references/anti-patterns.md before using shortcuts.