Userscript Testing

You have an amazing U​serscript...but how can you ensure it works?

While Userscripts aren't as popular as web extensions, not only are they prefered by the more privacy-conscious users, but - assuming they meet your needs - are a simpler thing to develop.

While there are a few established approaches to testing web extensions, there are few for Userscripts, and none that I've found that are as simple as this one for modern userscripts.

The Userscript

If you're actually writing a userscript complex enough to need E2E testing, I'll assume you're already using a bundler like Webpack or Rollup to do the needed combining of all your differing files into one.

I personally am using Webpack:

// ...

const mode = process.env.NODE_ENV || 'development';
const isProd = mode === 'production';

module.exports = {
  // ...
  devtool: isProd ? undefined : 'inline-source-map',
  module: {
    rules: [{
      test: /\.ts?$/,
      use: isProd ? 'ts-loader' : ['@theintern/istanbul-loader', 'ts-loader'],
      exclude: /node_modules/,
    }],
  },
  // ...
  plugins: isProd ? [] : [new webpack.SourceMapDevToolPlugin({
    sourceRoot: path.resolve(__dirname, 'src')
  })]
};

combined with Typescript:

{
  "include": ["src/*.ts"],
  "compilerOptions": {
    "target": "esnext",
    "module": "ES6",
    "esModuleInterop": true,
    "inlineSourceMap": true,
    "inlineSources": true,
  }
}

The observent reader will notice that the above configuration is already set up to work with the @theintern/istanbul-loader - which is what we'll be using to instrument our code.

Testing

My choice of tools to E2E test is Playwright, while you can technically use an browser-controlled automation tool like Selenium, or even Cypress, using Chromium via Playwright makes it much easier to collect code coverage data.

Firstly you'll want to load your Userscript as it would be in the browser, which via Playwright is as simple as:

await page.addInitScript({ path: './dist/<a href="https://en.wikipedia.org/wiki/Userscript">userscript</a>.js' });

After which you'll want to start JS coverage collecting before visiting the page in question:

await page.coverage.startJSCoverage();

and from that point on you can begin the E2E test, asserting that the page is in the state you expect it to be in as you interact with it.

Once your test is completed though, you'll need to stop the coverage, collect, and save the results for C8 to process:

// Find the coverage entry for just the file we're testing
const entry = (await page.coverage.stopJSCoverage()).find(entry => entry.url === './dist/<a href="https://en.wikipedia.org/wiki/Userscript">userscript</a>.js');
if (!entry) throw new Error('Entry point not found');

// Create the './coverage/tmp' folder if it doesn't exist
await fs.promises.mkdir('./coverage/tmp/', { recursive: true });
// Write the coverage data to a file
await fs.promises.writeFile(`./coverage/tmp/coverage-<a href="https://en.wikipedia.org/wiki/Userscript">userscript</a>.json`, <abbr title="JavaScript Object Notation"><a href="https://developer.mozilla.org/en-US/docs/Glossary/JSON">JSON</a></abbr>.stringify({
  result: [{
    ...entry,
    url: 'file://' + 'absolute/path/to/your/dist/<a href="https://en.wikipedia.org/wiki/Userscript">userscript</a>.js'
  }],
  timestamp: Date.now()
}, undefined, '  '));

From this point the testing is actually all complete and you have generated the coverage data for C8 to process into reports, etc.

But also for Web Extensions

This approach can actually be used for testing web extensions as well, as long as said web extension doesn't require communication with any extension-only APIs - for those web extensions there are dozens of online guides on how to do this.