Hello, I'm Maja.

Excerpt from https://xkcd.com/1629/

Excerpt from https://xkcd.com/1629/

Notes:

Example of Remote Control

running tests

Notes:

Web Compatibility

Notes:

Pain

"lol web development" -@notwaldorf

"lol web development" -@notwaldorf

Notes:

Web Predictability

Notes:

Web Standards ➔ Interoperable

Notes:

Interoperability

open instead of fragmented

Notes:

Tools for a healthy web

xkcd comic:tools

Notes:

Coming up

  1. Why browser automation?
  2. What is WebDriver? Selenium?
  3. How does WebDriver work in Firefox?
  4. Hey, let's add a WebDriver command to Firefox!

Notes:

Welcome to browser automation

Notes:

Example

session.get("https://www.mozilla.org/")
link = session.find_element_by_id("participate")
link.click()

Notes:

Turned off by default

Visual cue in URL bar

Visual cue in URL bar

Notes:

Why automate?

Testing...

Notes:

Beyond Testing

Notes:

WebDriver could be used:

How? WebDriver!

WebDriver Overview

Notes:

WebDriver Client Sends Commands to Browser

session.get("https://www.mozilla.org/")
link = session.find_element_by_id("participate")
link.click()

Notes:

Example: Talking to Firefox

pip install selenium
export PATH=$PATH:/path/to/geckodriver

Automation script:

from selenium import webdriver

# e.g. check the web page title
session = webdriver.Firefox()
session.get("http://www.python.org")
assert "Python" in session.title
session.close()

Notes

✨open source project to web standard!✨

Selenium WebDriver ➔ W3C WebDriver Standard

Notes:

What does this imply?

Each WebDriver command is an HTTP request with a JSON payload

Notes:

WebDriver Protocol

Request: please navigate to mozilla.org

POST /session/0152f...a9e8/url
{"url": "https://www.mozilla.org/"}

Response: command was successful!

Status: 200
Data: {}

Notes:

Spec Implementation Status

How many WebDriver commands work in each browser?

Notes:

Aside: Web Platform Tests

testthewebforward.org

testthewebforward.org

Notes:

WebDriver in Firefox

Notes:

Now we'll take a deeper dive into the components that make the WebDriver protocol work in Firefox.

geckodriver + Marionette

WebDriver in Firefox

Notes:

Firefox: Why not JSON over HTTP?

Based on existing remote debugging protocol instead.

Notes:

You've started a WebDriver session. What happens when you send the "Get Title" command?

print session.title

Processing a Command (1)

Client

Send HTTP request to geckodriver

GET     /session/<session id>/title

Processing a Command (2)

geckodriver

Translate and send to Marionette

The result is a message in the format :

30:[0, 21, "getTitle", null]

Notes:

Quick peek inside geckodriver

(Get, "/session/{sessionId}/title",
 Route::GetTitle)
GetTitle => (Some("getTitle"), None)
impl ToJson for MarionetteCommand {
    fn to_json(&self) -> Json {
        Json::Array(vec![Json::U64(0),
                         self.id.to_json(),
                         self.name.to_json(),
                         self.params.to_json()])
    }
}

Notes: these are simplified excerpts from geckodriver and a supporting library, webdriver-rust

  1. Map the title endpoint to GetTitle
  2. Translate to a Marionette command: getTitle
  3. Construct a 4-part request as expected by Marionette: [0, id, name, params]

Processing a Command (3)

Marionette

Parse stream of incoming packets (e.g. from geckodriver)

Request parsed into JSON: type, id, method, params

[0,21,"getTitle",{}]

Processing a Command (4)

Marionette

Dispatch cmd ("getTitle") to the right driver method

execute(cmd) {
  let fn = this.driver.commands[cmd.name];
  if (typeof fn == "undefined") {
    throw new UnknownCommandError(cmd.name);
  }
  let rv = yield fn.bind(this.driver)(cmd, resp);
  // send resp
}

Notes: this is a simplified excerpt from marionette

Processing a Command (5)

Marionette

Perform browser interaction to compute response.

function getTitle() {
  return curContainer.frame.top.document.title;
}

Example response: type, id, error, result

[1,21,null,{"value":"Cats in Boxes"}]

Notes: excerpt from listener.js

Processing a Command (6)

Finally, the response is sent back, translated by geckodriver to JSON-over-HTTP:

Status: 200
Data: {"value":"Cats in Boxes"}

Implementing a Command

Notes:

Most WebDriver commands are functional in Firefox -- many were implemented years ago -- but not all of them are spec-compliant yet. This is also true of other browsers: the commands work, but not consistently. By "implementing a command", I mean modifying or rewriting parts of Marionette to make it follow the WebDriver spec. It's an interesting task because you end up interacting with spec authors, digging into other web standards (like UI events), comparing with other browsers and trying to reconcile what the browser should actually do in response to a command.

Example: The "Actions" Command

POST /session/<id>/actions

{"actions": [{
  "parameters": {"pointerType": "mouse"},
  "type": "pointer",
  "id": "myMouse",
  "actions": [
    {"type": "pointerMove", "y": 10, "x": 80,
     "duration": 50},
    {"type": "pointerDown", "button": 0},
    {"type": "pointerUp", "button": 0}
  ]
}]}

Notes:

Read the spec

https://www.w3.org/TR/webdriver/

https://www.w3.org/TR/webdriver/

Notes:

Read the spec

Notes:

Write web-platform tests

Examples:

Notes:

Add to marionette + geckodriver

First, parse and validate the command payload

Notes:

Control the order of DOM events

action.dispatch = function (chain, ...) {
  let chainEvents = Task.spawn(function* () {
    for (let tickActs of chain) {
      yield action.dispatchTickActions(
          tickActs,
          action.computeTickDuration(tickActs),
          ...);
    }
  });
  return chainEvents;
};

Notes:

What next?

Notes:

Now you have an overview of WebDriver, browser automation, why it's important for the health of the web, how it works in Firefox, and how to approach adding a new command. What can do you do with this knowledge? Work with us on fun browser bugs, help improve WebDriver, and help with web predictability!

Ways to contribute

  1. Use Selenium >3.3.1 with geckodriver + Firefox Nightly then file new issues on Github
  2. Test-drive documentation, help us clarify/update

Notes:

Ways to contribute

  1. Write spec tests (Python, pytest, JS)
  2. Work on bugs in spec implementation (JS, Rust)
  3. Tackle some Mozilla test-harness bugs (Python)

Notes:

Remotely control this browser to make a better Web ❤️🤖

Notes:

Thanks!

I'm Maja Frydrychowicz. Contact:

mozilla

Resources

#ateam channel on irc.mozilla.org