Overview

What is this thing?

This is a UI Automation framework to test the new Lifesize Cloud clients (React Web and Electron Desktop). It uses webdriver.io as our driver to control the browsers. It follows the page object design pattern for organization as well as implements the concept of controllers and clients for abstracting code away from the tests themselves. Tests are written in the mocha framework.

Webdriver.io allows us to write our tests in a synchronous fashion without having to worry about dealing with promises. It uses a robust configuration file that allows us to control many factors of our tests and framework, such as which browser to instantiate (and how many), Sauce Labs integration, test suite organization, logging verbosity and much more.

How to use this framework

In general, the tests should use the controllers. We shouldn't have to call the browser directly within the tests. If you do, abstract it out into a controller or page object. The flow of commands should go Tests use clients >> Clients use controllers >> Controllers use Page Objects >> Page Objects interact with browser

Technical Overview

• Javascript, using ES6 syntax.
• AVA test runner
• Webdriver.io (Selenium wrapper) for driving browser.
• Page Object design pattern.
• Docker (using Zalenium) for local Selenium Grid.
• HTTP calls to MMS API (REST API).
• ESDOC for documentation.
• Tested and working on Mac OSX, untested on Windows machines.

Getting Started

Setup

I want to start now!

1. Setup Docker and Zalenium as per the steps below.
2. Run npm install to install dependencies.
3. Copy the contents of the .env_sample file into a new .env file. Obtain the user/pass for the missing fields.
4. Run FILE=test/specs/single_browser/login.test.js HOST=localhost npm test (runs a single login test file).

ESLint and Prettier

This repo uses ESLint and Prettier. It should run them pre-commit, but I also recommend to have both of these tools installed/integrated in your editor of choice. For example, with VS Code I have prettier set to run after save so whenever I save a file it will automatically get formatted and I shouldn't have to worry about it anymore.

.env

We use dotenv to pull in environment variable to configure our tests and framework.

In the root there is a file named .env_sample, simply copy the contents to a new file named .env. Ask one of the maintainers about any missing fields or credentials. Without them, MMS API calls will more than likely fail.

.y4m Files (DEPRECATED)

DEPRECATED: WE DON'T NEED .Y4M FILES AT THE MOMENT

We use some chrome options to be able to send local media in place of your webcam. .y4m files are required for this. Download these files:

• https://s3.amazonaws.com/y4mfiles.lifesizecloudbeta.com/pingpong.y4m
• https://s3.amazonaws.com/y4mfiles.lifesizecloudbeta.com/salesman.y4m
• https://s3.amazonaws.com/y4mfiles.lifesizecloudbeta.com/students.y4m

These are stored in this bucket: https://s3.amazonaws.com/y4mfiles.lifesizecloudbeta.com/.

Place them in the tmp/mounted directory named as they are and the framework will use them.

TODO: Need to link ./tmp/mounted to /tmp/mounted.

Docker Selenium

NOTE: Zalenium is best suited for a *nix platform. It's not fully tested on Windows yet.

To test locally, we are using Docker Selenium, in particular, Zalenium.

To setup, follow these steps:

1. Install Docker https://docs.docker.com/docker-for-mac/install/
2. On the command line, verify Docker is running. docker version should suffice.
3. docker pull elgalu/selenium
4. docker pull dosel/zalenium
5. Make sure you are in this directory and run docker-compose up --force-recreate (it reads the docker-compose.yaml file in this directory)
6. The above command will consume a tab/window of your terminal, so in another one run docker ps to show the VMs. You should see a handful of VMs listed.

From here you effectivily have you local selenium grid up and running. By default, which we set in the docker-compose.yaml, it will run a max of 2 Chrome containers. If box can handle more, or if you don't mind the performance hit, feel free to increase the amount of containers.

With the grid and running, you can view the selenium dashboard at http://localhost:4444/grid/console

For live video of the containers running, see http://localhost:4444/grid/admin/live

You can now run tests against your own local Zalenium/Selenium grid! When running your tests just make sure to set the HOST environment variable to ``localhost.

After your tests run, see http://localhost:4444/dashboard for the video archive of your tests.

Creating Page Objects

Creating page objects is the one of the core fundamentals of this framework. The page objects paired with controllers create the engine that drives the applications.

In it's simplest form, a page object is a class that has 2 things:

• the elements we want to interact with
• methods that use these elements to take actions

Basic Shell

So let's start and stub out our new page object, then we'll talk about what we are looking at:

import BasePage from '../../base.page.js'

/** @class MyNewPage */
class MyNewPage extends BasePage {
  constructor(browser) {
    super(browser)
  }

  // Elements

  // Actions

}

export default MyNewPage

With that we have the shell of a new page object. On the first line we are importing our BasePage (which itself inherits from the main PageObjectCore). It effectively gives us methods that all pages/elements need.

The one other thing I want to call out is /** @class MyNewPage */. This is our inline documentation. Any class you create you should immediately start documenting it. When we get ot methods soon we will cover documenting those.

Integrating sub-pages

Let's say your new page actually has a generic sidebar menu on it that you may want to access at times. Simply import it, then assign it in the constructor like this:

import BasePage from '../../base.page.js'
import MySideBar from '~/src/page_objects/mySideBar.page.js' // <-- Our import

/** @class MyNewPage */
class MyNewPage extends BasePage {
  constructor(browser) {
    super(browser)
    this.mySideBar = new MySideBar(browser) // <-- Our assignment
  }

  // Elements

  // Actions

}

export default MyNewPage

Elements

Our page needs elements we want to interact with, this can be done by creating a function for each element we need, like so:

elementName() {
    return this.element(ELEMENT_SELECTOR);
  }

So, say we want to add a button to our page, we could declare our element like:

myButton() {
  return this.element("button.awesomeClass")
}

One of the benefits of using elements as functions is that we can give them arguments. For example, say we want a div of that's text is the name of a contact we dynamically generated.

contactNamed(name) {
  return this.element(`div=${name}`)
}

When declaring elements like this, we can call actions on the elements themselves like

• this.myButton().click()
• this.myField().setValue('hey there!')
• this.myDiv().getAttribute('class')
• this.myComplexThing().waitForVisible()

Let's go and add some more elements to our page below.

import BasePage from '../../base.page.js'
import MySideBar from '~/src/page_objects/mySideBar.page.js'

/** @class MyNewPage */
class MyNewPage extends BasePage {
  constructor(browser) {
    super(browser)
    this.mySideBar = new MySideBar(browser)
  }

  // Elements
  mySearchField() {
    return this.element("input#myElementId")
  }

  myButton() {
    return this.element("button.sweetClassName")
  }

  theResults() {
    return this.element("div[class^=awkwardClass__Name-that-starts_WithThis]")
  }

  // Actions

}

export default MyNewPage

Actions

Now that we have our elements, we want to actually use these in some capacity. To do so we are simply creating functions on our class. These functions could be simple wrapper just to cause less typing in a tests, or it could a longer complex chain of commands where we parse multiple elements divs, or map through an array of list items.

Generally speaking though, a basic function will look like this, and note the two words async and await.

async clickMyButton() {
  await this.myButton().click()
}

Async/await is critical. All of our commands are effectively asynchronous when it make the request to the browser. As a result, we need to ensure that our funcitons are async, and that we await for the inner function calls to be completed before continuing.

Let's add a few more methods to our page...

import BasePage from '../../base.page.js'
import MySideBar from '~/src/page_objects/mySideBar.page.js'

/** @class MyNewPage */
class MyNewPage extends BasePage {
  constructor(browser) {
    super(browser)
    this.mySideBar = new MySideBar(browser)
  }

  // Elements
  mySearchField() {
    return this.element("input#myElementId")
  }

  myButton() {
    return this.element("button.sweetClassName")
  }

  theResults() {
    return this.element("div[class^=awkwardClass__Name-that-starts_WithThis]")
  }

  // Actions
  async searchForSomething(value) { // <-- new action!
    await this.mySearchField().setValue(value)
    await this.myButton().click()
  }

  async getResultsText() { // <-- new action!
    return await this.theResults().getText()
  }

  async openSearchtab() { // <-- action using a sub-component
    await this.mySideBar.searchTab().click()
  }

}

export default MyNewPage

Adding to WebPageObjects Module

When you are ready to user you page object, don't forget to add it to our WebPageObjects module by adding lines like the following

 import MyNewPage from './path/to/MyNewPage.page.js'

 // ... more imports and pages...

 /**
  * MyNewPage
  * @see MyNewPage
  */
  MyNewPage

We can then get access to that page from WebPageObjects like

myNewPage = new WebPageObjects.MyNewPage(browser)

Next steps

With that our new page object has enough stuff for our controllers. Lets continue this in the next section about Controllers.

Creating Controllers

To fully create a controller, you will need whatever page objects you intend to interact with. You can plan out the controller, but with out the actual page objects, nothing will work. If you are coming from the Creating a Page Object tutorial, you should be all good to go.

Basic Shell

Much like the page objects lesson, here is a basic shell of a controller we will start with.

import BaseController from './base.controller.js'

/** 
 * @extends BaseController
 */
class MyController extends BaseController {
  constructor(browser) {
    super(browser)
  }

  // Controller functions...

}

export default MyController

Adding Page Objects

Adding the page objects is easy. We first have to import the WebPageObjects module (which we should have added our new page object to!) at the top of the file, and then we can instantiate the page objects we need for the controller within the controllers constructor.

import BaseController from './base.controller.js'
import * as WebPageObjects from '~/src/page_objects/web/webPageObjects.js' // <-- Import WebPageObjects module to get all page objects

/** 
 * @extends BaseController
 */
class MyController extends BaseController {
  constructor(browser) {
    super(browser)
    this.mySweetPage = new WebPageObjects.MySweetPage(browser) // <-- adding a page!
    this.anotherPage = new WebPageObjects.AnotherPage(browser) // <-- adding a page!
    this.myNewPage = new WebPageObjects.MyNewPage(browser) // <-- adding a page!
  }

  // Controller functions...

}

export default MyController

Functions

With the page objects in place, you can now create functions that chain together various actions in to longer commands that the controller would provide. Note that much like our page objects, we need to write our functions in the async/await style. Rule of thumb is if we are going to interact with the browser, it will most likely need to be an async/await function.

import BaseController from './base.controller.js'
import * as WebPageObjects from '~/src/page_objects/web/webPageObjects.js'

/** 
 * @extends BaseController
 */
class MyController extends BaseController {
  constructor(browser) {
    super(browser)
    this.mySweetPage = new WebPageObjects.MySweetPage(browser)
    this.anotherPage = new WebPageObjects.AnotherPage(browser)
    this.myNewPage = new WebPageObjects.MyNewPage(browser)
  }

  async loginAndDoSomething(name, password) { // <-- adding a function!
    await this.mySweetPage.visit()
    await this.mySweetPage.login(name, password)
    await this.mySweetPage.verifyHeader()
    await this.myNewPage.visit()
    await this.myNewPage.doSomething()
  }

  async getTextFromWeirdDiv() { // <-- adding a function!
    await this.anotherPage.waitForAvatar()
    return await this.anotherPage.weirdDiv.getText()
  }

}

export default MyController

Adding to the Client

We now have our controller. The next step is to add it to the Client.

At the top of src/clients/web/client.js import your controller like so:

import MyController from '~/src/controllers/path/to/myController.controller.js'

Then use it in the constructior of the client:

this.myController = new MyController(browser)

And there we go! The client now has our new controller integrated with it.

Writing Tests

Test Setup/Teardown

Setup is performed via tests/config/setup.js. We require it with AVA's config, so that code should get run before test.

Teardown is perforved via test/config/teardown.js.

Client(s)

Each test file will need a before and after block for setting up clients and shutting them down.

In essence, our shell of a test looks like this:

import Client from '../../../src/clients/web/client.js'

let client

beforeAll(async () => {
  client = new Client(BROWSER_NAME, 'Example Test Name')
  await client.start()
})

afterEach(async () => {
  await client.clearCookies()
})

afterAll(async () => {
  await client.stop()
})

// Tests go here

So let's break it down into what our requirements for each test are:

1. Before ALL tests in a file are run - create and start the browser(s)
2. After EACH test in the file is run - clear out the cookies
3. After ALL tess in a file are run - stop the browser

Additionally, there are a few things to note here, first: BROWSER_NAME. If your test requires a specific browser, specify it here. If not, the test will use the default browser we specify. Second, again notice that we are using async/await for our tests. Lastly, new Client takes a second argument, which is the test name. This is to identify the browser in either Zalenium or Sauce labs so we know what each session was doing.

Multiple clients

If your test requires multiple clients, you can create multiple clients as you see fit like so:

import Client from '../../../src/clients/web/client.js'

let clientA, clientB

beforeAll(async () => {
  clientA = new Client('chrome', 'BrowserA for Example Test')
  clientB = new Client('ie', 'BrowserB for Example Test)
  await clientA.start()
  await clientB.start()
})

afterEach(async () => {
  await clientA.clearCookies()
  await clientB.clearCookies()
})

afterAll(async () => {
  await clientA.stop()
  await clientB.stop()
})

// Tests go here

With these before and after sections done, our browser will start and then end for our test.

The Actual Test

Below is an example test. Notice that there is now a beforeEach block. We can use those for additional setup that needs to happen before each test in the file is run, such as creating users, logging in, or starting calls.

import Client from '../../../src/clients/web/client.js'

beforeAll(async () => {
  client = new Client(BROWSER_NAME, 'Getting Text from the Thing')
  await client.start()
})

beforeEach(async () => {
  t.context.secretPassword = await SomePasswordApiService.getPassword()
})

afterEach(async () => {
  await client.clearCookies()
})

afterAll(async () => {
  await client.stop()
})

test('get text from the thing', async () => {
  await client.myContreller.login('user@email.com', t.context.secretPassword)
  await client.myController.clickAround()
  const text = await client.myController.getSomeText()
  expect(text).toBe('expected text')
})

Expect vs Should vs Assert

Use the expect syntax for your expectations in the test.

Testing Journeys

At some point it may not be worth it to write small atomic tests if the setup for these tests is expensive or timely to use. This is where the concept of "testing journeys" comes in! A journey will be considered a full user flow where we can make assertions throughout the entire flow. The main difference between how we normally write tests is that the tests within the journey are entirely dependant on the test order. For an example, take a look at the p2pCall.journey.js test.

Commiting Code

Setup

1. git clone https://github.com/lifesize/automation.client.ui.git (Note: No need to fork the repo, just pull directly from the repo)
2. Verify setup as per the setup section.

Making Changes

1. Pull master
2. Checkout a new feature branch. Use yourname.JIRA-##.description as a template. Omit JIRA-## if there is no associated ticket.
3. Make your changes
4. Test your changes
5. Commit your changes and push up to GitHub
6. Create a PR and send off for review

Once review passes we will merge in your changes.

Documentation

ESDoc

Our code should be properly documented. We use ESDoc for these purposes.

Here are examples of how various things should be documented:

/** 
 * Description for class.
 * @extends ParentClass
*/
class MyClass extends ParentClass {
  // MyClass Code
}

 /** 
   * Function description.
   * @param {String} param1 - description of parameter.
   * @example Example of usage.
   * @returns {String} What it returns, if anything.
   */
myFunction = (param1) => {
  // myFunction Code
}

Building docs

To build and preview docs, run npm run docs to build them. They will build into a /docs folder in the root. Open up the index.html to preview your changes. To publish the docs, we need to run the automation.client.ui.publish.docs jenkins job (http://automationdocs.lifesizecloudbeta.com). This pulls master and builds the latest version of the docs.

Test Structure

Our test files run in parallel, but the files themselves should run from top to bottom. Knowing this we can organize our tests with describes, tests, beforeAll, afterAll, beforeEach, and afterEach to hopefully give us good readability and reporting on what we care about from the tests.

As an example, lets look at the following test file, which is in a pretty bad state (Note: code may be pseudo-code):

test("Can adopt a cat", () => {
  dbSetupController.populateDbWithCats(5)
  client.loginController.login()
  client.adoptionController.viewCatList()
  const cats = client.catListController.getCats()
  expect(cats.length).toBe(5)
  client.catListController.selectCatNamed("Garfield")
  const currentCatName = client.catDetailsController.getCurrentCatName()
  expect(currentCatName).toBe("Garfield")
  client.catDetailsController.clickAdopt()
  client.checkoutController.addLitterBox()
  client.checkoutsController.addCatnip()
  client.checkoutController.addCatFood()
  const totalPrice = client.checkoutController.getTotalPrice()
  expect(totalPrice).toBe("$39.15")
  client.checkoutController.finishAdoption()
  const checkoutComplete = client.checkoutController.isCheckoutComplete()
  expect(checkoutComplete).toBe.true
  dbSetupController.resetDb()
})

This test works, as in, it all passes. But, it's doing more than what it's title may indicate. Sure, it's checking out if you can adopt a cat but it's checking a few more things along the way. You have to find out what your test is asking to determine what you need to expect/assert inside the test. So, maybe we don't care about all of the assertions except for the final isCheckoutComplete() one? Or are all of our other assertions still completely valid?

For this scenario, lets pretend that we want to know about all of our other assertions. But first, lets start tearing apart the test. Can you see which parts are setup/teardown and not really a part of the test? Let's refactor this a bit to leverage before/after blocks...

beforeAll(() => {
  dbSetupController.populateDbWithCats(5)
  client.loginController.login()
})

afterAll(() => {
  dbSetupController.resetDb()
})

test("Can adopt a cat", () => {
  client.adoptionController.viewCatList()
  const cats = client.catListController.getCats()
  expect(cats.length).toBe(5)
  client.catListController.selectCatNamed("Garfield")
  const currentCatName = client.catDetailsController.getCurrentCatName()
  expect(currentCatName).toBe("Garfield")
  client.catDetailsController.clickAdopt()
  client.checkoutController.addLitterBox()
  client.checkoutsController.addCatnip()
  client.checkoutController.addCatFood()
  const totalPrice = client.checkoutController.getTotalPrice()
  expect(totalPrice).toBe("$39.15")
  client.checkoutController.finishAdoption()
  const checkoutComplete = client.checkoutController.isCheckoutComplete()
  expect(checkoutComplete).toBe.true 
})

This structure seperates our tests from our test setup/teardown. The next bit is that we are going to split up our test in multiple tests, but wrap it all with a describe block.

describe('Cat adoption process', () => {
  beforeAll(() => {
    dbSetupController.populateDbWithCats(5)
    client.loginController.login()
  })

  afterAll(() => {
    dbSetupController.resetDb()
  })

  test("displays 5 cats in list", () => {
    client.adoptionController.viewCatList()
    const cats = client.catListController.getCats()
    expect(cats.length).toBe(5)
  })

  test("name matches selected cat", () => {
    client.catListController.selectCatNamed("Garfield")
    const currentCatName = client.catDetailsController.getCurrentCatName()
    expect(currentCatName).toBe("Garfield")
  })

  test("price total updates properly", () => {
    client.catDetailsController.clickAdopt()
    client.checkoutController.addLitterBox()
    client.checkoutsController.addCatnip()
    client.checkoutController.addCatFood()
    const totalPrice = client.checkoutController.getTotalPrice()
    expect(totalPrice).toBe("$39.15")
  })

  test("can checkout completely", () => {
    client.checkoutController.finishAdoption()
    const checkoutComplete = client.checkoutController.isCheckoutComplete()
    expect(checkoutComplete).toBe.true 
  })
})

This is, generally, what our finished tests shoud look like. When run with jest we get a nice output for each test we have. In the code, we have clean seperation of concerns. This setup also gives up flexibility in the future as we can nest additional describe blocks to further organize our suites.

Naming Things

Controllers

Controllers filenames should be named fooBar.controller.js, where fooBar is a descriptive name for what it does.

Similarly, the class name for he controller should be FooBarController.

Actions

Actions/function names can be generally whatever you like as long as it is clear and descriptive to what it does. Don't be afraid to fully describe something versus using abbreviations.

Page Objects

By default, all page objects should be named like fooBar.page.js, where fooBar is a descriptive name for what it is.

Optionally, instead of page, you could use a name like modal or panel to better describe what the "page object" is. For example, you could have fooBar.modal.js or fooBar.panel.js. If you go this route, make sure to name the class appropriately. page's should be FooBarPage while modals should be FooBarModal, for example.

Elements

Elements should typically end with what they are if it makes sense. The general idea is that we don't want to clash with our actions vs the elements the actions use.

For example, say we had an element (a button) we named login. Then what if we had an action on this same page object that we named login as well (let's ignore the fact that we have an actual name collision).

In our login function it looks like this:

login() {
  this.doStuff()
  this.login().click()
  this.waitForThings()
}

To me, it's not super clear what the login function call inside our login function IS. So, if it's a button it should be named loginButton.

Actions

Actions/function names can be generally whatever you like as long as it is clear and descriptive to what it does. Don't be afraid to fully describe something versus using abbreviations.

Tests

Test should always end in .test.js. For example, fooBar.test.js. If the test is considered a "journey", it should be fooBar.journey.test.js.

Local Configuration

Concurrency

The amount of browsers and tests you can run locally depends on your machine. But note, the amount of tests you want to run does not equal the amount of browsers you need. Some of our tests are considered "multi browser", where a single test file may spawn up to 3 or more individual browsers. In general, running 2 test files at a time is fairly safe as at one time (with the current tests we have) it would generate 6 total browsers if the right 2 tests were ran at the same time.

The max you can run is highly dependant on your machine, but again, 2 is pretty safe. The concurrency is set via the command line flag for jest named "maxWorkers".

Environment Variables

BASE_URL - base url that will be used (defaults to https://webdemo.lifesizecloudbeta.com) HOST - host to point selenium requests to. Use localhost if you are using your local Zalenium install. FILE - path to single file to test

Use these as such:

BASE_URL=https://myspecial.testenvironment.com npm run test:all

Sauce Labs

We are using Sauce Labs as the platform to run our tests on from Jenkins. If you need to run some test there locally there are a few things that you need to setup to do this.

In your .bashrc/.bash_profile/.zshrc (or whatever you use) add the following:

export SAUCE_USERNAME='my_sauce_user_name'
export SAUCE_ACCESS_KEY='my_sauce_access_key'

Replace the username and access key with the ones provided to you. With those set, you can run tests in Sauce Labs from your local machine.

When running tests, you will need to set USE_SAUCE=true on the command line when you run them. The framework will then create the clients in Sauce Labs versus locally.

Sauce + Zalenium

If you need to setup our Sauce Labs account with your Zalenium installation, change the sauceLabsEnabled setting in the docker-compose.yaml before starting up Zalenium and it should then send requests for browsers that you don't have capabilites for to Sauce Labs (such as IE).

Advanced Techniques

Tricky element selection

Elements that take parameters

Using collections of elements

When you return a collection of elements like this.elements("li#myItems") the resulting "elements" object has some hand methods attached to it: first, last, and all. When you call those methods, the driver will finally query the page for the elements and return what you wanted.

Dropping down to the browser

While in a controller or page object you can drop directly to the browser using this.browser. You have any method available to the browser (found here http://webdriver.io/api.html)

Page Object Inheritance (via extends)

You may have a situation where you have a base page of some kind that will extend to other pages. Let's say we have to create pages for adopting some cute animals. Let's start with our base page.

import BasePage from "path/to/base.page.js"

class BaseAdoptionPage extends BasePage {
  constructor(browser) {
    super(browser)
  }

  availablePets() {
    return this.elements("li.availablePets")
  }

  adoptPetButton() {
    return this.element("button#adoptPet")
  }
}

In the above code you see that our BaseAdoptionPage extends BasePage. Our next page objects will extend BaseAdoptionPage as such:

class KittenAdoptionPage extends BaseAdoptionPage {
  constructor(browser) {
    super(browser)
  }

  addLitterBox() {
    return this.element("button#addLitterBox")
  }

  addCatnip() {
    return this.element("button#addCatnip")
  }
}

class DoggyAdoptionPage extends BaseAdoptionPage {
  constructor(browser) {
    super(browser)
  }

  addBone() {
    return this.element("button#addBone")
  }

  addLeash() {
    return this.element("button#addLeash")
  }
}

The doggy and kitten pages have access to the elements and actions on their inherited pages, in this example that means they'd have access to the availablePets and adoptPetButton elements.3

Skip, only, and other Jest things

If you want to skip a test with Jest, append .skip to the test like so:

test.skip(...)

Additionally, there are a few other handy things you can do with tests. If you want to run only a single test in a particular file you can append .only exactly like above.

Another one is .failing. This will "pass" the test if it is failing, but fail it if it were to start passing. This is a way for us to identify tests that may have a bug associated with them but still keep our builds green. If the bug were to get fixed without our knowledge, the test would then "pass" thereby failing it because we used .failing.

If you want to run only a single file that you are working on you can use npm test path/to/test/file. Combined with .only you can narrow into a single test if you are debugging something.

References