Skip to main content

pytest

Submit and collect your pytest results in Testiny to track automated test runs. By integrating pytest with Testiny, you can gain valuable insights into your testing process. Track automation results over time, identify frequently failing tests, and detect patterns in test failures to fix CI failures and find flaky tests.

pytest
info

See the example code for a Pytest integration in our GitHub repository.

Generate pytest XML Reports​

Pytest provides built-in support for generating JUnit-style XML test reports, making it easy to integrate with CI/CD pipelines and test reporting tools. To generate a JUnit-style XML report, simply run Pytest with the --junit option, specifying the desired output file path:

pytest --junit=results/report.xml

When you execute this command, Pytest will run all available tests and generate a structured XML report in the specified file. You can also specify another output directory.

Import Results to Testiny​

Make sure you've generated an API key and use the Testiny CLI to import the results to Testiny:

export TESTINY_API_KEY=****
npx @testiny/cli automation --project ID_OR_KEY --source "pytest-tests" --junit results/report.xml

In the above example, the environment variable TESTINY_API_KEY is set to authenticate the request. The --project flag specifies the target Testiny project using its ID or key and the --source flag categorizes the test results (e.g., frontend, backend, integration tests, ...). The --junit flag specifies the format of the specified test result file(s).

Once executed, the Testiny CLI will automatically create a new automated test run in Testiny and import all results from the specified report(s).

Custom Properties​

Use Pytest's record_property fixture to log additional information for a test.

Test Keys​

testid or testkey are optional properties that can be used to uniquely identify a test across automation runs:

def test_function(record_property):
    record_property("testkey", "my-test-id-1")
    # ...

If not specified, a combination of title and folder path will be used to identify a test.

Custom Result Fields​

Custom fields are supported via custom properties and can be included as fields via the CLI:

def test_function(record_property):
    record_property("type", "regression")
    record_property("sample-link", "https://www.example.com")
    # ...

Import these properties with the option --custom-result-fields:

npx @testiny/cli automation --project <ID_OR_KEY> --source "pytest-tests" --custom-result-fields=type,another_property --junit results/report.xml
info

See the example code for a Pytest integration in our GitHub repository.

See an example of a JUnit-format result file.

More resources​

Here are some further resources to help you get started with integrating pytest with Testiny, including CI/CD integration examples, a full automation guide, and the Testiny CLI.