Playwright day 3 - Integrations

The last remaining articles in Playwright's "Getting Started" section are

• running and debugging

• trace viewer

• CI integration

• VS Code integration

This is everything but writing tests. How to run and review tests, how to integrate with an IDE, how to embed in a CI/CD pipeline.

After this block is done, I can start ramping up on generating a whole glut of tiny practice tests. I'll want to swap "vs code integration" for webstorm or something, but I can probably spin up a jenkins install on ocelotcodesystems.com to embed an arbitrarily simple playwright test for the CI section.

Running and Debugging

Running is very simple

npx playwright test
            

Playwright can also run tests in a "ui" mode, which sounds extremely promising.

you can easily walk through each step of the test and visually see what was happening before, during and after each step. UI mode also comes with many other features such as the locator picker, watch mode and more.

Where have you been all my life?

npx playwright test --ui
            

This is fantastic. You can adjust locators on the fly, see what worked and didnt and why, you can watch tests as they're run, edit and rewatch, this is going to save a phenomenal amount of time over hacking in some kind of css highlighting and screenshot fixture for failed tests.

I think I'll revisit the test from day 2 and see what I can find by analyzing the test in ui mode.

Error: expect(locator).toBeVisible() failed
            
            Locator: getByText('Images')
            Expected: visible
            Error: strict mode violation: getByText('Images') resolved to 3 elements:
                1) <a class="gb_6" data-pid="2" target="_top" aria-label="Search for Images " href="https://www.google.com/imghp?hl=en&ogbl">Images</a> aka getByRole('link', { name: 'Search for Images' })
                2) <span>Create images</span> aka getByText('Create images', { exact: true })
                3) <span id="sb-accessible-label-4">Remove Create Images</span> aka getByText('Remove Create Images')
            
            Call log:
              - Expect "toBeVisible" with timeout 5000ms
              - waiting for getByText('Images')
            

Definitely would have sped up diagnosing the last test.

I guess the stack trace isn't necessar buecase there's a "source" section that just highlights the exact line you were at, and a playwright test probably isn't going to have an interesting stack. It's not like the database repository file chokes but you want to see the service layer, api layer, etc - the goal with playwright is almost definitely to just have a few files with a few tests sitting in each.

Having a timeline is nice, that could be good for identifying laggy tests.

There's a popout tool that you can use to analyze the state of the dom at any point during the test.

Other options for running tests:

headless is the default but you can run headed like so:

npx playwright test --headed
            

You can specify which browser(s) to run tests in

npx playwright test --project webkit --project firefox
            

You can rerun only the tests which failed the last run:

npx playwright test --last-failed
            

The "last failed" list is automatically managed.

Default location is here: <outputDir>/.last-run.json

But you can specify the path to a "last failed" file

npx playwright test --last-failed --last-failed-file=.cache/last-run-shard-1.json
            

Test Grouping and Tagging

Specific tests or groups of tests can be run like so:

# run one test file
            npx playwright test landing-page.spec.ts
            
            # run directories of test files
            npx playwright test tests/todo-page/ tests/landing-page/
            

There appears to be a makeshift tag system; you can run the tests that have search terms in the titles

npx playwright test landing login
            

Actually, there appears to be a tag system as well. This is the counterpart for pytest markers.

Tagging individual tests:

test('test login page', {
              tag: '@fast',
            }, async ({ page }) => {
              // ...
            });
            
            test('test full report @slow', async ({ page }) => {
              // ...
            });
            

Tagging a test group:

import { test, expect } from '@playwright/test';
            
            test.describe('group', {
              tag: '@report',
            }, () => {
              test('test report header', async ({ page }) => {
                // ...
              });
            
              test('test full report', {
                tag: ['@slow', '@vrt'],
              }, async ({ page }) => {
                // ...
              });
            });
            

You can grep for tags when running

npx playwright test --grep @fast
            

You can skip tests based on tags

npx playwright test --grep-invert @fast
            

Locical operators:

# AND operator
            npx playwright test --grep "@fast|@slow"
            
            # OR operator
            npx playwright test --grep "(?=.*@fast)(?=.*@slow)"
            

you can likely flex grep and get real complex with how you decide what to run.