Playwright Day 2 : Test Writing Basics

Writing Tests

Locators

Turns out the "GetByRole" function I saw in the example test is part of a whole API of element-getters:

• page.getByRole() to locate by explicit and implicit accessibility attributes.
• page.getByText() to locate by text content.
• page.getByLabel() to locate a form control by associated label's text.
• page.getByPlaceholder() to locate an input by placeholder.
• page.getByAltText() to locate an element, usually image, by its text alternative.
• page.getByTitle() to locate an element by its title attribute.
• page.getByTestId() to locate an element based on its data-testid attribute (other attributes can be configured).

I keep going to back to selenium, but having "get by testid" just be a single function is awesome. It feels weird having to add a wrapper function for that functionality.

It looks like these are chainable so you can narrow searches as well, that's good.

const locator = page
                .frameLocator('#my-frame')
                .getByRole('button', { name: 'Sign in' });
            
            await locator.click();
            

Look at how flexible this is

<h3>Sign up</h3>
            <label>
              <input type="checkbox" /> Subscribe
            </label>
            <br/>
            <button>Submit</button>
            

await expect(page.getByRole('heading', { name: 'Sign up' })).toBeVisible();
            
            await page.getByRole('checkbox', { name: 'Subscribe' }).check();
            
            await page.getByRole('button', { name: /submit/i }).click();
            
            
            
            

It honestly reads as kind of brittle, like you should probably get your devs to insert a data-testid element. I've heard some people say "no it's better to have the test identify things the way the user does", but the user is not an automated machine with literally no intuition; if someone changes the text from "Sign up" to something syntactically different but semantically the same, the test may break.

Of course testid is just as, if not more, accessible via playwright, so it's not really a big deal unless it's culturally decided that data-testid attributes are bad.

I say that, and then immediately find this guidance on when to use role locators:

We recommend prioritizing role locators to locate elements, as it is the closest way to how users and assistive technology perceive the page.

Honestly I disagree with this, but I think the right path is to use the language standard tools and keep my suspected drawbacks pocketted; if tests really do end up being brittle because of this, then this is all just a bargaining chip to get data-testid attributes implemented.

Anyway, it's neat that they say when a given locator type should be used.

label locator:

Use this locator when locating form fields.

placeholder locator:

Use this locator when locating form elements that do not have labels but do have placeholder texts.

text locator:

We recommend using text locators to find non interactive elements like div, span, p, etc. For interactive elements like button, a, input, etc. use role locators.

testid locator:

Testing by test ids is the most resilient way of testing as even if your text or role of the attribute changes, the test will still pass. QA's and developers should define explicit test ids and query them with page.getByTestId(). However testing by test ids is not user facing. If the role or text value is important to you then consider using user facing locators such as role and text locators.

I think it would be best to locate by testid and then just add an assertion for the text, allowing the test to cleanly fail if it changes, but this also puts a maintenance burden on whoever is embedding those data-testid attributes.

You can also use test ids when you choose to use the test id methodology or when you can't locate by role or text.

Oh this is tremendous, look at this, you can change it to not strictly search for data-testid but another attribute entirely:

import { defineConfig } from '@playwright/test';
            
            export default defineConfig({
              use: {
                testIdAttribute: 'data-pw'
              }
            });
            

Some leftovers that may be useful:

alt text locator:

Use this locator when your element supports alt text such as img and area elements.

title locator:

Use this locator when your element has the title attribute.

Actions

The actions we can take on elements after locating them:

Assertions

Some convenience functions / syntactic sugar on assertions:

The locators, actions, and assertions are the syntactic sugar that keeps this framework reading like plain english.

Fixtures and Hooks

Fixtures are context packages like browser or page, hooks are triggers like "before every test" or "after all tests complete".

Familiar from selenium; a lot of this is going to be stuff I used to have to write myself but now it's just baked into the framework.

Writing a hook to create a test user, log in, yield for the test, and then delete the user may be useful, but the core "open a browser" kind of fixture and the concept of page classes is certainly baked in.

Generating Tests

CodeGen is a cool little tool that allows you to generate code by navigating around a website.

Not to keep harping on "get things how the user gets them, not by data-testid", but check this test out:

import { test, expect } from '@playwright/test';
            
            test('test', async ({ page }) => {
              await page.goto('https://www.google.com/');
              await page.getByRole('link', { name: 'Search for Images' }).click();
              await expect(page.getByText('Images')).toBeVisible();
            });
            

The idea here was "go to google, click the images link, confirm we're at the image search page"

But the "images" link that takes you to images.google.com is titled "Images", so both pages have something that matches getByText('Images')

The proper method here is probably to assert we're at the url images.google.com , and I'm not saying every link should have a data-testid attribute. I think what itches is being told to "write code that interacts with the site the way a user does", because that's just not going to happen unless you use OCR for everything.

PRANKD

Based on the above I thought I could comment out the second action (clicking the images link) and get a false positive out of the test. Not so!

      5 |   await page.goto('https://www.google.com/');
                  6 |   // await page.getByRole('link', { name: 'Search for Images' }).click();
                > 7 |   await expect(page.getByText('Images')).toBeVisible();
                    |                                          ^
                  8 | });
                  9 |
                    at /home/fundesk/Code/playwright_training/100_plays/002/tests/google-navigation.spec.ts:7:42
            
                Error Context: test-results/google-navigation-test-chromium/error-context.md
            
              2) [webkit] › tests/google-navigation.spec.ts:4:5 › test ─────────────────────────────────────────
            
                Error: expect(locator).toBeVisible() failed
            
            

So what happened?

I tried adding some logging before the click:

  let images_text_count = await page.getByText('Images').count();
              console.log(`"get by text 'Images': " ${images_text_count}`)
            
            
              await page.getByRole('link', { name: 'Search for Images' }).click();
            

and things are certainly being found that have "images" in the text

Running 3 tests using 3 workers
            [chromium] › tests/google-navigation.spec.ts:4:5 › test
            diagnostic time!
            "get by text 'Images': " 3
            [webkit] › tests/google-navigation.spec.ts:4:5 › test
            diagnostic time!
            "get by text 'Images': " 4
            [firefox] › tests/google-navigation.spec.ts:4:5 › test
            diagnostic time!
            "get by text 'Images': " 4
              3 passed (15.9s)
            
            

And with a bit more diagnostic code we can see that the first element found by our getter is exactly what I expected: the link

<a class="gb_6" aria-label="Search for Images " data-pid="2" href="https://www.google.com/imghp?hl=en&amp;ogbl" target="_top">Images</a>

After running the test many times with minimal changes, I started to inconsistency in when the test would pass or fail. Once I even got it to fail in one browser but not another:

      21 |
                  22 |
                > 23 |   await expect(page.getByText('Images')).toBeVisible();
                     |                                          ^
                  24 |
                  25 |
                  26 |   await page.getByRole('link', { name: 'Search for Images' }).click();
                    at /home/fundesk/Code/playwright_training/100_plays/002/tests/google-navigation.spec.ts:23:42
            
                Error Context: test-results/google-navigation-test-firefox/error-context.md
            
              2 failed
                [chromium] › tests/google-navigation.spec.ts:4:5 › test ────────────────────────────────────────
                [firefox] › tests/google-navigation.spec.ts:4:5 › test ─────────────────────────────────────────
              1 passed (15.9s)
            
            

Given that all browsers have typically passed or failed together, I take this as an instance of "the same test passed/failed on the same run". So I thought it was a race condition.

Well, if that were the case, adding implicit or explicit waits would have fixed it:

  await page.waitForTimeout(1000);
            
              await expect(page.getByText('Images')).toBeVisible({
                timeout: 10000
              });
            

But it didn't.

DOUBLE PRANKD

I completely missed this part of the error output:

    Locator: getByText('Images')
                Expected: visible
                Error: strict mode violation: getByText('Images') resolved to 3 elements:
            
            

So real reason the test failed is that you can't assert "toBeVisible" against multiple elements. And the second lesson is that playwright errors provide an "error" field that you should check out before climbing up the stack trace.