Playwright Day 6 - CI/CD Integration

Preamble

Second to last part of the playwright intro docs is about github actions integraiton.

I typically use jenkins for CI/CD, and have integrated tests into that before, but the playwright tutorials point to github actions, so this will be a nice opportunity to build a quick github integration.

Plan

I'll be using this page for testing: https://ocelotcodesystems.com/index.html

That's where I typically publish these blog posts and some dev stuff.

I have another project, a fullstack webapp, that I'll be using for more in-depth testing later, but for now I can just put together a quick smoke test

Parameterized URL

Before actually configuring github actions I wondered about how to parameterize the URL that the test would run against and decided to figure that out first.

The playwright config file playwright.config.ts has this:

  use: {
                /* Base URL to use in actions like `await page.goto('')`. */
                // baseURL: 'http://localhost:3000',
            
            

You can just pull in an environment variable near the top of the file, right below the dotenv configuration like so:

/**
             * Read environment variables from file.
             * https://github.com/motdotla/dotenv
             */
            // import dotenv from 'dotenv';
            // import path from 'path';
            // dotenv.config({ path: path.resolve(__dirname, '.env') });
            
            const baseURL = process.env.BASE_URL;
            
            if (!baseURL) {
              throw new Error('BASE_URL environment variable is required');
            }
            

I think it's right to do it up there, outside of the defineConfig function, so that you can throw an exception immediately if the variable isn't defined instead of waiting for a test to break.

So then in the defineConfig function, instead of this:

  use: {
                /* Base URL to use in actions like `await page.goto('')`. */
                // baseURL: 'http://localhost:3000',
            
            

We have this:

  use: {
                /* Base URL to use in actions like `await page.goto('')`. */
                baseURL,
            

And now all the URLs in our tests are relative:

    await page.goto('/');
                await page.goto('/blog');
                //etc
            

Big fan of that.

Just as a confirmation, I ran without setting the environment variable, and got a nice easy exception:

npx playwright test tests/smoke_test.spec.ts
            npm notice run 004@1.0.0 npx
            npm notice run 'playwright' test tests/smoke_test.spec.ts
            Error: BASE_URL environment variable is required
            
            

That may not be the right way to set things up though. I'll want the CI/CD server to spin up a webserver with the build candidate and test against that, instead of deploying and then testing.

So instead of configuring it like the above, another option would be to do this up top of playwright.config.ts :

const baseURL = process.env.BASE_URL || 'http://127.0.0.1:8000';

And then down at the bottom of the file there's a commented out web server section, where it wants to spin up node

  /* Run your local dev server before starting the tests */
              // webServer: {
              //   command: 'npm run start',
              //   url: 'http://localhost:3000',
              //   reuseExistingServer: !process.env.CI,
              // },
            

But since this project is just a little html and css site, I can get away with just using a minimal python http server

  // basic python http server instead of node
              webServer: {
                command: 'python3 -m http.server 8000 -d ..',
                url: 'http://127.0.0.1:8000',
                reuseExistingServer: !process.env.CI,
              },
            

I did a push with it set up like this, and it failed, which is great! An opportunity for me to look through some logs and see the results from github actions.

The failure is clear:

Error: locator.click: Error: strict mode violation: getByRole('link', { name: 'blog' }) resolved to 2 elements:
            

And the github action summary makes the playwright html report available for download

So now I can just run the tests on my desktop in UI mode, find a better locator, and push again.

Actually, everything passes locally

$ npx playwright test tests/smoke_test.spec.ts
            npm notice run 004@1.0.0 npx
            npm notice run 'playwright' test tests/smoke_test.spec.ts
            
            Running 9 tests using 4 workers
              9 passed (33.0s)
            
            

Ah. It runs locally because the locator ('link', name: 'blog') pulls one element on the old version of the site, but the new version of the site actually has a second link named 'show all blog posts', so that's the discrepancy.

The fix is simple: a more specific locator:

await page.getByRole('navigation').getByRole('link', { name: 'Blog' }).click()

The link I want to click is in the nav bar, so we narrow it down to the nav tag first.

But that's a nice simple exploration of github actions. Tomorrow I can look into jenkins, or just write more tests.