Skip to main content

Playwright

Submit and collect your Playwright tests in Testiny to track results over time, identify frequently failing tests, efficiently debug and fix CI failures or flaky tests. You can use the CLI to import your automated results or use our custom reporter to directly report results to Testiny.

note

This section describes how to set up Playwright Node.js with Testiny. If you're using Playwright with another language, please refer to the generic Quickstart or Automation Guide.

info

See the example code in our GitHub repository.

Playwright Reporter​

You can install and use the custom Playwright Testiny Reporter instead of using the CLI:

  1. Install the Playwright Testiny Reporter:

    npm install @testiny/automation

  2. Add the reporter to your playwright.config.js file:

    playwright.config.js
    import { defineConfig, devices } from '@playwright/test';

    export default defineConfig({
        ...
        reporter: [
            ["@testiny/automation/reporters/playwright", { 
                enable: true,           // enable or disable reporting to Testiny (e.g. only report in CI env)
                project: "key",         // the name, key or id of the Testiny target project,
                sourceName: "ui-test",
                token: "tny_12...",     // optional: Testiny API key, defaults to environment variable TESTINY_API_KEY
                ...
            }],
        ],
        ...
    });

    Find all available configuration options in the README.

  3. Make sure you've generated an API key, set it to env variable TESTINY_API_KEY (or specify in reporter config) and run your Playwright tests with playwright test. The Playwright Testiny Reporter will report all results to Testiny.

Playwright JSON Reports​

  1. Configure your Playwright tests with the junit reporter:

    playwright.config.js
    import { defineConfig, devices } from '@playwright/test';

    export default defineConfig({
        ...
        reporter: [
            ["json", { outputFile: "results/results.json" }],
        ],
        ...
    });

    Run your Playwright tests with playwright test and the reporter will create a JSON report file in the results folder.

  2. Make sure you've generated an API key and use the Testiny CLI to import the results to Testiny:

    export TESTINY_API_KEY=****
npx @testiny/cli automation --project ID_OR_KEY --source "e2e" --playwright results/*.json

    In the above example, the environment variable TESTINY_API_KEY is set to authenticate the request. The --project flag specifies the target Testiny project using its ID or key and the --source flag categorizes the test results (e.g., frontend, backend, integration tests, ...). The --playwright flag specifies the format of the specified test result file(s).

    Once executed, the Testiny CLI will automatically create a new automated test run in Testiny and import all results from the specified report(s).

Playwright JUnit Reports​

  1. Configure your Playwright tests with the junit reporter:

    playwright.config.js
    import { defineConfig, devices } from '@playwright/test';

    export default defineConfig({
        ...
        reporter: [
            ['junit', { outputFile: 'results/results.xml' }]
        ],
        ...
    });

    Run your Playwright tests with playwright test and the junit reporter will create reports for each spec file in the results folder.

  2. Make sure you've generated an API key and use the Testiny CLI to import the results to Testiny:

    export TESTINY_API_KEY=****
npx @testiny/cli automation --project ID_OR_KEY --source "e2e" --junit results/*.xml

    The CLI will create an automated test run in Testiny and import all results from the report.

Playwright Annotations​

With Playwright Annotations you can add additional information to your automated tests.

Test Keys​

The testid or testkey are optional properties that can be used to uniquely identify a test across automation runs. If not specified, a combination of title and folder path will be used to identify a test.
Set test keys with annotations:

test('sample test', {
  annotation: { type: 'testkey', description: 'my-testid-1' },
}, async ({ page }) => {
  // ...
});

Custom Result Fields​

Custom fields are supported via annotations and can be included as fields via the CLI and reporter:

// annotate your tests with custom fields:
test('sample test', {
  annotation: { type: 'type', description: 'smoke' },
  annotation: { type: 'some_link', description: 'https://www.example.com' },
}, async ({ page }) => {
  // ...
});

Import these properties with the option --custom-result-fields (or for the reporter, set property customResultFields):

--custom-result-fields=type,some_link
info

See the example code in our GitHub repository.

More resources​

Here are some further resources to help you get started with integrating Playwright with Testiny, including CI/CD integration examples, a full automation guide, and the Testiny CLI.