Why You Can't Test a Chrome Extension the Normal Way

I've been building browser extensions for a few years now: PageSaver, which turns a web page into a PDF or an image, and a handful of others. Every one of them shipped without a single automated test. Not because I didn't want tests, but because the tools I already knew couldn't reach the parts that actually break.

The loop was always the same. Change a file, open chrome://extensions, hit reload, click the toolbar icon, poke the options page, then open three separate DevTools windows because the popup, the page and the service worker each keep their own console. Miss one and the bug ships.

I finally sat down to fix this properly, and the first thing I had to accept is that the problem isn't laziness in the ecosystem. Testing an extension is genuinely different from testing a web app, in three specific ways.

An extension is not a page

Playwright and Puppeteer are excellent, and they drive pages. An extension is mostly not a page.

Start with the popup. When you click an extension's toolbar icon, the thing that opens isn't a tab — it's browser chrome, rendered by the browser outside the page hierarchy that automation knows how to address. There's no tab to attach to and no handle to grab. You can't click it from a script, and no amount of clever selector work changes that.

The background is worse. Under Manifest V3 it's a service worker, which means no tab, no DOM, and no window. page.evaluate() has nothing to attach to, because there is no page. The worker also shuts down when it goes idle and starts again on the next event, so even "is it running" is a question with a timestamp attached.

Content scripts look like the easy case and aren't. They run in an isolated world: same DOM as the page, completely separate JavaScript context. If your content script sets a variable and you evaluate an expression in the page to read it, you get undefined. The two contexts share markup and nothing else.

So the popup can't be clicked, the worker can't be evaluated, and the content script can't be inspected from the page. That's most of an extension.

Then Chrome took away the front door

The old answer to all of this was to launch a browser with --load-extension and drive it. That stopped working. Google removed the flag in Chrome 137, for a reasonable stated reason: "it was commonly abused to load malicious and unwanted software into the browser."

The part that matters for testing is in the announcement on the chromium-extensions group:

Please note that this change only applies to Chrome branded builds. --load-extension will continue to function as before in non Chrome brands, such as Chromium and Chrome For Testing.

That single sentence is the whole strategy. The flag isn't gone, it's gone from branded Chrome. Chrome for Testing — the build Google publishes specifically for automation — still honours it, and Google names it directly as where to go.

There are blog posts floating around suggesting you can force branded Chrome back into line with --disable-features=DisableLoadExtensionCommandLineSwitch. Don't build on that. I tested it against Chrome 150.0.7871.187 while working on this, and an unpacked extension doesn't load with or without the flag. --enable-unsafe-extension-debugging doesn't bring it back either. That escape hatch is closed, and it was never sanctioned in the first place.

My advice is to stop thinking of the Chrome in your dock as your test browser. Use Chrome for Testing. It's the supported path, it's the one Google points at, and it's the only one that will still work next quarter.

The failure you can't see

Here's the one that convinced me this needed real tooling rather than a pile of scripts.

An MV3 service worker's console only exists while its inspector is open. Chrome doesn't buffer that output for you. So if your background script throws on its first line — a typo, a missing import, an API you forgot to declare a permission for — you open the popup, see an empty rectangle, check the page console, find nothing, and have no idea anything threw at all. The worker died before you were watching.

The fix is to attach over the DevTools protocol before the worker starts, so the buffered output is already being collected when it throws. Do that, and the same failure reports itself:

✗ service worker starts without errors
  assertNoConsoleErrors: 2 console error(s):
    [worker] ReferenceError: initialise is not defined
      at chrome-extension://abc…/background.js:4:1
    [worker] Error: Unhandled rejection: storage quota exceeded
      at chrome-extension://abc…/background.js:31:16

File and line, from a script that never ran long enough to show you a stack trace by hand.

Most of a first test suite is already in your manifest

This is the part that surprised me. manifest.json already declares your popup, your options page, your content script match patterns, and whether you have a service worker. That's enough to derive a real smoke test without writing anything.

So I built pikabo around that idea. Point it at an unpacked extension:

$ npx pikabo explore --ext ./my-extension

my-extension v2.1.0 · MV3
popup: popup.html · options: options.html · service worker · 2 content script blocks · 5 permissions

Generated 5 smoke test(s) → tests/smoke.generated.yaml
my-extension smoke tests
  ✓ service worker starts without errors      12ms
  ✓ popup renders                            431ms
  ✓ options page renders                     318ms
  ✓ content script injects on github.com     772ms
  ✓ content script injects on gitlab.com     684ms

  5 passed · 3.8s
  html      pikabo-results/report.html

No hand-written test was involved in that. It read the manifest, opened each declared surface in a real Chromium, checked that the popup rendered actual content rather than an empty shell, navigated to a URL matching each content script pattern to confirm injection, and asserted the worker started clean. The generated suite is written to disk as YAML so you can keep editing it instead of treating it as a black box.

From there you write assertions that mean something, and the useful part is that each surface is addressable by name:

name: PageSaver

tests:
  - name: popup saves a PDF
    steps:
      - navigate: https://example.com
      - openPopup
      - click: "#save-pdf"
      - waitForDownload: "*.pdf"
      - assertStorage:
          key: lastSave
          exists: true
      - assertNoConsoleErrors

in: worker runs a step inside the service worker, where chrome.storage and chrome.runtime live. in: popup addresses the popup as a real document. There's also sidepanel, devtools, offscreen, and background for MV2. Being able to say which part of the extension a step acts on is the thing that was missing.

For an extension that exports a file, downloads get captured wherever they come from, including chrome.downloads.download() called from the worker with no page involved. No page ever sees it, so page-level automation can't. assertPdf then reads the file: page count, geometry, byte size, producer.

One setup note, because it bit me immediately. If your extension has manifest.json at the repo root, don't npm install into it; node_modules lands next to your manifest and gets swept into the ZIP you upload to the Web Store. Keep the tests in their own folder and point at the extension:

mkdir extension-tests && cd extension-tests
npm init -y && npm install --save-dev pikabo
npx playwright-core install chromium
npx pikabo run tests --ext ../my-extension

What it won't do

Being clear about the edges, because the gap between "tests pass" and "it works" is where trust gets lost.

Packaged .crx installs from the Web Store aren't supported, only unpacked directories. Browser chrome outside a document still can't be clicked — the toolbar icon menu, the puzzle-piece overflow, native permission prompts — though everything those surfaces trigger is reachable through the popup document, the options page, or the worker. Keyboard shortcuts declared in manifest.commands can't be delivered as real key events, so you dispatch the handler through the worker instead. And branded Chrome is out, per everything above.

There's also a permission audit that runs on every session, comparing the chrome.* namespaces your code actually touched against what the manifest declares. It reports findings as not observed rather than unused, which matters: a permission exercised only on a path your tests never take looks identical to a dead one. A smoke suite that opens the popup proves nothing about permissions. The report says so rather than letting you delete something you needed.

pikabo is MIT, on npm as pikabo, and needs Node 22 or newer. If you maintain an extension and have been putting tests off because the tooling didn't seem to exist, this is the specific gap it fills. I'd genuinely like to hear what breaks on extensions that aren't mine.