interact() — Control the Browser

The interact tool gives the AI control over the browser — navigate, click, type, read elements, run JavaScript, manage state, and display narration. Requires AI Web Pilot to be enabled in the extension popup.

Quick Reference

interact({action: "navigate", url: "https://example.com"})
interact({action: "click", selector: "text=Submit"})
interact({action: "type", selector: "label=Email", text: "user@example.com"})
interact({action: "list_interactive"})
interact({action: "screenshot"})  // use observe({what: "screenshot"}) instead
interact({action: "subtitle", text: "Welcome to the demo"})
interact({action: "execute_js", script: "document.title"})
interact({action: "save_state", snapshot_name: "logged-in"})

Composable Parameters

These parameters can be added to any action:

Parameter	Type	Description
`subtitle`	string	Narration text displayed at bottom of viewport. Empty string clears.
`reason`	string	Why this action is being performed — shown in the action toast
`correlation_id`	string	Link this action to a specific error or investigation
`analyze`	boolean	Enable performance profiling (returns perf_diff in result)
`tab_id`	number	Target a specific tab (omit for active tab)

Subtitles

Subtitles are persistent narration text displayed at the bottom of the viewport, like closed captions. They stay visible until replaced or cleared.

// Standalone
interact({action: "subtitle", text: "Creating a new project"})

// Composable — narration + action in one call
interact({action: "click", selector: "text=Create",
          subtitle: "One click to start a new project"})

// Clear
interact({action: "subtitle", text: ""})

Action Toasts

When enabled, action toasts show brief notifications at the top of the viewport for each action. The reason parameter customizes the toast label:

// Without reason: toast shows "Click: #submit-btn"
interact({action: "click", selector: "#submit-btn"})

// With reason: toast shows "Submit the registration form"
interact({action: "click", selector: "#submit-btn", reason: "Submit the registration form"})

Toasts can be toggled off in the extension popup — useful during demos.

Semantic Selectors

All actions that accept a selector parameter support semantic selectors in addition to CSS:

Syntax	Example	How it works
`text=`	`text=Submit`	Finds element whose text content contains “Submit”
`role=`	`role=button`	Finds `[role="button"]`
`placeholder=`	`placeholder=Email`	Finds `[placeholder="Email"]`
`label=`	`label=Username`	Finds label containing text, follows `for` attribute to target
`aria-label=`	`aria-label=Close`	Finds `[aria-label="Close"]` with starts-with matching
CSS	`#submit-btn`	Standard `document.querySelector()`

Semantic selectors are resilient to UI changes — they target meaning, not structure. text=Submit still works after a CSS redesign, a framework migration, or a component library upgrade.

The aria-label= selector uses starts-with matching, so aria-label=Send matches Gmail’s "Send ‪(⌘Enter)‬".

navigate

interact({action: "navigate", url: "https://example.com"})

Navigates the active tab to the specified URL. Automatically includes perf_diff in the async result (before/after performance comparison).

Parameter	Type	Description
`url`	string (required)	URL to navigate to

refresh

interact({action: "refresh"})

Refreshes the current page. Includes perf_diff automatically.

back / forward

interact({action: "back"})
interact({action: "forward"})

Browser history navigation.

new_tab

interact({action: "new_tab", url: "https://example.com"})

Opens a new tab with the specified URL.

Parameter	Type	Description
`url`	string (required)	URL to open in the new tab

DOM Primitives

click

interact({action: "click", selector: "text=Submit"})
interact({action: "click", selector: "#confirm-btn", reason: "Confirm the order"})

Parameter	Type	Description
`selector`	string	CSS or semantic selector

type

interact({action: "type", selector: "label=Email", text: "user@example.com"})
interact({action: "type", selector: "placeholder=Search", text: "wireless headphones", clear: true})

Parameter	Type	Description
`selector`	string	CSS or semantic selector
`text`	string	Text to type
`clear`	boolean	Clear existing value before typing

select

interact({action: "select", selector: "#country", value: "US"})

Parameter	Type	Description
`selector`	string	CSS or semantic selector for `<select>` element
`value`	string	Option value to select

check

interact({action: "check", selector: "text=I agree to the Terms"})
interact({action: "check", selector: "#newsletter", checked: false})

Parameter	Type	Default	Description
`selector`	string	—	CSS or semantic selector
`checked`	boolean	true	Whether to check or uncheck

key_press

interact({action: "key_press", selector: "label=Search", text: "Enter"})
interact({action: "key_press", text: "Escape"})

Parameter	Type	Description
`selector`	string	Element to target (optional — omit for page-level key press)
`text`	string	Key name: `Enter`, `Tab`, `Escape`, `Backspace`, `ArrowDown`, `ArrowUp`, `Space`

focus

interact({action: "focus", selector: "label=Email"})

scroll_to

interact({action: "scroll_to", selector: "#pricing-section"})

wait_for

Wait for an element to appear on the page. Useful for async content.

interact({action: "wait_for", selector: "text=Dashboard", timeout_ms: 10000})

Parameter	Type	Default	Description
`selector`	string	—	CSS or semantic selector
`timeout_ms`	number	5000	Maximum wait time in milliseconds

Element Reading

get_text

interact({action: "get_text", selector: ".order-total"})

Returns the text content of the matched element.

get_value

interact({action: "get_value", selector: "input[name='email']"})

Returns the current value of a form element.

get_attribute

interact({action: "get_attribute", selector: "#submit", name: "disabled"})

Parameter	Type	Description
`name`	string	Attribute name to read

set_attribute

interact({action: "set_attribute", selector: "#submit", name: "disabled", value: "false"})

Parameter	Type	Description
`name`	string	Attribute name
`value`	string	Value to set

list_interactive

Discovers all interactive elements on the page — buttons, links, inputs, selects, etc. Returns up to 100 elements with suggested selectors.

interact({action: "list_interactive"})

No parameters. Returns an array of elements, each with:

Field	Description
`tag`	Element tag name (`button`, `input`, `a`, etc.)
`type`	Input type (`email`, `password`, `text`) — only for `<input>`
`selector`	Suggested selector (semantic preferred: `text=`, `aria-label=`, `placeholder=`)
`label`	Human-readable label (aria-label > title > placeholder > text content)
`role`	ARIA role
`visible`	Whether the element is visible on screen

This is the best way to discover what’s clickable on an unfamiliar page. The AI can read the list and choose the right selector.

JavaScript Execution

execute_js

Run arbitrary JavaScript in the page context. See How Gasoline Executes Page Scripts for the full deep dive.

interact({action: "execute_js", script: "document.title"})
interact({action: "execute_js", script: "window.location.href"})
interact({action: "execute_js",
          script: "document.querySelectorAll('li').length",
          world: "main"})

Parameter	Type	Default	Description
`script`	string	—	JavaScript code to execute
`world`	string	`auto`	`auto` (try main, fallback to isolated), `main` (page globals), `isolated` (CSP-proof)
`timeout_ms`	number	5000	Maximum execution time

Visual

highlight

Highlights an element on the page with a visual overlay. Useful for demos and debugging.

interact({action: "highlight", selector: "#error-message"})
interact({action: "highlight", selector: "text=Submit", duration_ms: 3000})

Parameter	Type	Default	Description
`selector`	string	—	Element to highlight
`duration_ms`	number	5000	How long to show the highlight

subtitle

Display persistent narration text at the bottom of the viewport.

interact({action: "subtitle", text: "Welcome to the product tour"})
interact({action: "subtitle", text: ""})  // Clear

Parameter	Type	Description
`text`	string (required)	Narration text. Empty string clears.

State Management

save_state

Save the current browser state (URL, title, tab) as a named checkpoint.

interact({action: "save_state", snapshot_name: "logged-in"})

load_state

Restore a saved state checkpoint.

interact({action: "load_state", snapshot_name: "logged-in"})
interact({action: "load_state", snapshot_name: "logged-in", include_url: true})

Parameter	Type	Default	Description
`snapshot_name`	string	—	Name of the saved state
`include_url`	boolean	false	Navigate to the saved URL when restoring

list_states

interact({action: "list_states"})

Returns all saved states with metadata (name, URL, title, saved_at).

delete_state

interact({action: "delete_state", snapshot_name: "old-checkpoint"})

Performance Profiling

Add analyze: true to any DOM action to get a perf_diff in the result — before/after timing comparison with Web Vitals ratings and a verdict.

interact({action: "click", selector: "text=Load Dashboard", analyze: true})

The navigate and refresh actions include perf_diff automatically.