Skip to main content

Manual Chrome extension scenarios

Manual recording runs as an Admin capture job: Desktop → Web Extension. Its internal environment is WebExtension; capture metadata and public MCP use web-extension. The agent downloads the selected Chrome Web Store extension and opens a disposable Chromium profile with it and the Reconstruct capture extension. The operator navigates; Reconstruct saves screens and observed transitions as reconstruct.scene.v1.

Create and run a job​

  1. In Admin → Jobs → Create, select Desktop → Web Extension. Unselect Web if you only want the extension. Mobile environments remain Web, iOS and Android.
  2. In Environment Settings → Web Extension, paste the selected Chrome Web Store link. Local extension directories and personal browser profiles are not inputs.
  3. Queue the job. An enabled Capture Agent assigned Desktop / Web Extension claims it. Web and Web Extension workers cannot claim each other's jobs.
  4. The agent downloads a signed CRX3 through Google's update service and verifies the selected extension ID and publisher signature before extracting it into a private job directory. The package retains its store ID when loaded unpacked. Capture version metadata includes browserExtension with the verified ID, localized name, package version and SHA-256; the catalog display name comes from that package. Archive paths, file counts and expanded size are checked before use.
  5. Chromium opens the extension's own onboarding tab. Otherwise its declared options page or action popup is used. An extension without a UI entry fails.
  6. Open Reconstruct Agent Browser Bridge from the browser's extensions menu. Its popup shows the active job ID and saved screen count. After each screen, use Capture screen. Use Pause, Resume and Finish recording there.

There is no standalone recorder CLI or additional recording MCP to launch. Controls are bound to the active job and a private local session token. They do not accept field values, arbitrary labels or source URLs. Admins can also watch and control the job browser from Admin → Jobs → job → Live browser; input there is accepted only while the job waits for the operator. Another extension or another job is rejected. If the extension opens another tab, activate it before capturing; its identity is checked before switching targets. Passwords, wallet creation, agreements and registration remain manual actions.

Sign-in jobs and automatic exploration​

Extensions that need onboarding (for example MetaMask's wallet creation) can be recorded semi-automatically. In Environment Settings → Web Extension, set Sign-in to Operator registers (or Operator signs in).

  1. The job waits for the operator. Complete onboarding in Admin → Jobs → job → Live browser, or in the job browser itself. Save screens with Capture screen; password and recovery-phrase screens are never saved.
  2. Select Finish sign-in (console), Continue (Admin) or Finish sign-in and explore (Reconstruct popup).
  3. Reconstruct then visits the extension's popup, options and side panel pages and the routes its navigation controls lead to, saving one screen per route. It never clicks controls that could send, sign, confirm, reveal, export, lock, delete or create anything.
  4. If the extension locks itself, the job waits as Signed out until the operator unlocks it and selects Finish sign-in again. The recording finishes and uploads for review when no routes remain or the page limit is reached.

Web jobs have the same Sign-in setting: sign-in screens are saved as stages, the crawl skips sign-out links and pauses if the session ends. CAPTCHA, SMS codes and acceptance of terms always stay with the operator. See backend/docs/operator-sign-in.md for details.

Agent setup​

Build the matching browser adapter with yarn nx run chrome:build in backend/agent-browser-extensions. The host needs Node.js 24+, Python 3 and an extension-capable Chromium build. Configure the existing Capture Agent using ignored local settings or environment variables:

{
  "Agent": {
    "Id": "extension-worker",
    "ApiUrl": "https://internal-admin-api.example",
    "ChromiumPath": "/Applications/Chromium.app/Contents/MacOS/Chromium",
    "ExtensionDir": "/checkout/backend/agent-browser-extensions/chrome"
  },
  "WebExtension": {
    "RecorderDirectory": "/checkout/backend/tools/manual-recorder",
    "NodePath": "node",
    "PythonPath": "python3"
  }
}

Supply the existing ApiTokens:Agent credential through secret configuration. The existing Kestrel Web listener must be loopback. Start the Capture Agent binary with extension serve, then assign its agent record to Desktop / Web Extension in Admin. Each host mode owns its listener and work lock; do not start two workers on one endpoint. Remote workers require a managed isolated browser host and the explicit WebExtension:ManagedBrowserIsolated declaration, as extensions execute their own network and background code. This flag does not create isolation. The existing explicit Agent:AllowHttpApi setting applies to a loopback development API.

The worker owns claims, heartbeats, cancellation and upload retries. Its internal Node child uses private process pipes and no debugging TCP port. Finishing closes that job's browser and deletes its temporary profile. Use an empty test wallet: a scenario does not preserve credentials or a usable wallet.

Secrets and image review​

Pause before entering passwords, recovery phrases, keys or codes. Resume after leaving sensitive screens. Protected text screens fail closed and create an observation gap. Editable controls and descendants are masked before screenshots; field values are not read into Scene. Pre/post checks reject captures when the protected state, masking, navigation or document content changes during capture.

Canvas and other opaque pixels stay visible in private review media. DOM checks cannot establish that they contain no secrets. Inspection and masking run synchronously in one isolated-world evaluation; JavaScript stays enabled for the screenshot so native canvas content can render.

Finish recording packages and uploads the result automatically as an unpublished capture. Each checkpoint owns a page, Scene and screenshot. Its scenario index references pages without copying their text. In Admin, inspect every screenshot in Hide personal data, blur sensitive regions or delete unsafe media, and publish after review. Redaction also clears covered Scene text, accessibility names, repeated values and media references. Separately stored original media requires its own review. Recording controls cannot publish or return raw images.

Reconstruction through MCP​

After review and publication, agents discover the version through the normal Capture MCP catalog. Use get_scenario_scene to load frames, profiles and transitions, assembled from current page-owned data including Admin redactions. Use normal published page tools for approved images. Verify the reconstruction against every required captured screen and profile; successful tools alone do not prove visual fidelity.

Evidence and limits​

Checkpoints are static observations. Capture after navigation, menus, modals and other disclosures, and at each required viewport. Adjacent observations measure changed/unchanged outcomes; they do not prove an automatically executed click. Paused and blocked intervals create explicit gaps. This mode does not silently capture every click, replay registration, capture native OS permission dialogs, or infer animation from still images. Secret steps remain manual and absent from the published scenario. The job page limit bounds checkpoints (automatic selection defaults to 200); the Scene node budget is 100,000 per job. The job time limit and element limit also apply.

Request availability​

In Admin → Requests → Job defaults → Request submissions, configure the allowed product type/environment pairs. Defaults are Desktop/Web, Desktop/Web Extension and Mobile/Web. Allow new requests closes or reopens all new submissions; saving preserves the selected pairs while submissions are closed. Existing requests and published captures remain available. The same persisted policy applies to the platform API and authenticated MCP, including clients with stale form options. The request-options endpoint returns no platforms while submissions are closed. Catalog product type/environment selectors show only environments captured for that product; request availability does not hide published catalog evidence.