Writing Tests
Using the preferred CSS selector model to locate elements on a page, Nightwatch makes it very easy to write automated End-to-End tests.
Create a separate folder for tests in your project, e.g.: tests
. Each file inside it will be loaded as a test by the Nightwatch test runner. A basic test will look like this:
module.exports = {
'Demo test Google' : function (browser) {
browser
.url('http://www.google.com')
.waitForElementVisible('body', 1000)
.setValue('input[type=text]', 'nightwatch')
.waitForElementVisible('button[name=btnG]', 1000)
.click('button[name=btnG]')
.pause(1000)
.assert.containsText('#main', 'Night Watch')
.end();
}
};
Remember always to call the .end()
method when you want to close your test, in order for the Selenium session to be properly closed.
module.exports = {
'step one' : function (browser) {
browser
.url('http://www.google.com')
.waitForElementVisible('body', 1000)
.setValue('input[type=text]', 'nightwatch')
.waitForElementVisible('button[name=btnG]', 1000)
},
'step two' : function (browser) {
browser
.click('button[name=btnG]')
.pause(1000)
.assert.containsText('#main', 'Night Watch')
.end();
}
};
Tests can also be written in this format:
this.demoTestGoogle = function (browser) {
browser
.url('http://www.google.com')
.waitForElementVisible('body', 1000)
.setValue('input[type=text]', 'nightwatch')
.waitForElementVisible('button[name=btnG]', 1000)
.click('button[name=btnG]')
.pause(1000)
.assert.containsText('#main', 'The Night Watch')
.end();
};
Using XPath selectors
Nightwatch supports xpath selectors also. To switch to xpath instead of css selectors as the locate strategy, in your test call the method useXpath()
, as seen in the example below. To switch back to CSS, call useCss()
.
To always use xpath by default set the property "use_xpath": true
in your test settings.
this.demoTestGoogle = function (browser) {
browser
.useXpath() // every selector now must be xpath
.click("//tr[@data-recordid]/span[text()='Search Text']")
.useCss() // we're back to CSS now
.setValue('input[type=text]', 'nightwatch')
};
BDD Expect Assertions
Nightwatch introduces starting with version v0.7
a new BDD-style assertion library which greatly improves the flexibility as well as readability of the assertions.
The expect
assertions use a subset of the Expect
api from the Chai framework and are available for elements only at this point. Here's an example:
module.exports = {
'Demo test Google' : function (client) {
client
.url('http://google.no')
.pause(1000);
// expect element to be present in 1000ms
client.expect.element('body').to.be.present.before(1000);
// expect element <#lst-ib> to have css property 'display'
client.expect.element('#lst-ib').to.have.css('display');
// expect element to have attribute 'class' which contains text 'vasq'
client.expect.element('body').to.have.attribute('class').which.contains('vasq');
// expect element <#lst-ib> to be an input tag
client.expect.element('#lst-ib').to.be.an('input');
// expect element <#lst-ib> to be visible
client.expect.element('#lst-ib').to.be.visible;
client.end();
}
};
The expect
interface provides a much more flexible and fluid language for defining assertions, significantly improved over the existing assert
interface. The only downside is that it's not possible to chain assertions anymore and at this point custom messages aren't yet supported.
For a complete list of available expect
assertions, refer to the API docs.
Using before[Each] and after[Each] hooks
Nightwatch provides the standard before
/after
and beforeEach
/afterEach
hooks to be used in the tests.
The before
and after
will run before and after the execution of the test suite respectively, while beforeEach
and afterEach
are ran before and after each testcase (test step).
All methods have the Nightwatch instance passed as argument.
Example:
module.exports = {
before : function(browser) {
console.log('Setting up...');
},
after : function(browser) {
console.log('Closing down...');
},
beforeEach : function(browser) {
},
afterEach : function() {
},
'step one' : function (browser) {
browser
// ...
},
'step two' : function (browser) {
browser
// ...
.end();
}
};
In the example above the sequence of method calls will be as follows: before(), beforeEach(), "step one", afterEach(), beforeEach(), "step two", afterEach(), after()
.
For backwards compatibility reasons, the afterEach
hook can receive the browser
object only in its async form - afterEach(browser, done) { .. }
Asynchronous before[Each] and after[Each]
All the before[Each]
and after[Each]
methods can also perform asynchronous operations, in which case they will require the callback
passed as the second argument.
done
function must be called as the last step when the async operation completes. Not calling it will result in a timeout error.
Example with beforeEach & afterEach:
module.exports = {
beforeEach: function(browser, done) {
// performing an async operation
setTimeout(function() {
// finished async duties
done();
}, 100);
},
afterEach: function(browser, done) {
// performing an async operation
setTimeout(function() {
// finished async duties
done();
}, 200);
}
};
Controlling the done
invocation timeout
By default the done
invocation timeout is set to 10 seconds (2 seconds for unit tests). In some cases this might not be sufficient and to avoid a timeout error, you can increase this timeout by defining an asyncHookTimeout
property (in milliseconds) in your external globals file (see below for details on external globals).
For an example, refer to the provided globalsModule example.
Explicitly failing the test
Failing the test intentionally in a test hook is achievable by simply calling done
with an Error
argument:
module.exports = {
afterEach: function(browser, done) {
// performing an async operation
performAsync(function(err) {
if (err) {
done(err);
}
// ...
});
}
};
External Globals
Most of the time it's more useful to have your globals defined in an external file, specified in the globals_path
property, instead of having them defined in nightwatch.json
.
You can overwrite globals per environment as needed. Say you have your tests running locally and also against a remote staging server. Most of the times you will need some different setting up.
Global Hooks
The same set of hooks as per test suite is also available globally, outside the scope of the test. See the below example for more details.
In the case of global hooks, the beforeEach
and afterEach
refers to a test suite (i.e. test file), and are ran before and after a test suite.
Global Settings
There are a number of globals which are holding test settings and can control test execution. These are detailed in the provided globalsModule sample.
Example:
module.exports = {
'default' : {
isLocal : true,
},
'integration' : {
isLocal : false
},
// External before hook is ran at the beginning of the tests run, before creating the Selenium session
before: function(done) {
// run this only for the local-env
if (this.isLocal) {
// start the local server
App.startServer(function() {
// server listening
done();
});
} else {
done();
}
},
// External after hook is ran at the very end of the tests run, after closing the Selenium session
after: function(done) {
// run this only for the local-env
if (this.isLocal) {
// start the local server
App.stopServer(function() {
// shutting down
done();
});
} else {
done();
}
},
// This will be run before each test suite is started
beforeEach: function(browser, done) {
// getting the session info
browser.status(function(result) {
console.log(result.value);
done();
});
},
// This will be run after each test suite is finished
afterEach: function(browser, done) {
console.log(browser.currentTest);
done();
}
};
Test Runner
Nightwatch includes a command-line test runner which makes it easy to run tests and generate useful output. There are a few different options on how to use the test runner, depending on your installation type.
Global
If you have installed Nightwatch globally (with -g
option), the binary nightwatch
will be available anywhere:
$ nightwatch [source] [options]
Project specific
If you have Nightwatch installed as a dependency of your project, you can refer the binary from the node_modules/.bin
folder:
Linux and MacOSX:
$ ./node_modules/.bin/nightwatch [source] [options]
Windows:
Create a file nightwatch.js
and add the following line:
require('nightwatch/bin/runner.js');
Then run as follows:
C:\workspace\project> node nightwatch.js [source] [options]
Tests source
The optional source
argument can be either one or more files or an entire folder. This can be located irrespectively of the src_folders
setting.
Example - single test:
$ nightwatch tests/one/firstTest.js
Example - 2 individual tests:
$ nightwatch tests/one/firstTest.js tests/secondTest.js
Example - 1 individual test and 1 folder:
$ nightwatch tests/one/test.js tests/utils
Command-line Options
The test runner supports a number of run-time options to be passed to. To view all, run the following:
$ nightwatch --help
Name | Shortname | default | description |
---|---|---|---|
--config |
-c |
./nightwatch.json |
The location of the nightwatch.json file - the configuration file which the runner uses and which also includes the Selenium WebDriver options. |
--output |
-o |
tests_output |
The location where the JUnit XML reports will be saved. |
--reporter |
-r |
junit |
Name of a predefined reporter (e.g. junit) or path to a custom reporter file to use. |
--env |
-e |
default |
Which testing environment to use - defined in nightwatch.json |
--verbose |
|
Shows extended selenium command logging during the session | |
--version |
-v |
Shows the version number | |
--test |
-t |
Runs only the specified test suite/module. By default the runner will attempt to run all tests in the src_folders settings folder(s) and their subfolders. |
|
--testcase |
Used only together with --test . Runs the specified testcase from the current suite/module. |
||
--group |
-g |
Runs only the specified group of tests (subfolder). Tests are grouped by being placed in the same subfolder. | |
--skipgroup |
-s |
Skip one or several (comma separated) group of tests. | |
--filter |
-f |
Specify a filter (glob expression) as the file name format to use when loading the test files. | |
--tag |
-a |
Filter test modules by tags. Only tests that have the specified tags will be loaded. | |
--skiptags |
Skips tests that have the specified tag or tags (comma separated). | ||
--retries |
Retries failed or errored testcases up to the specified number of times. Retrying a testcase will also retry the beforeEach and afterEach hooks, if any. |
||
--suiteRetries |
Retries failed or errored testsuites (test modules) up to the specified number of times. Retrying a testsuite will also retry the before and after hooks (in addition to the global beforeEach and afterEach respectively), if any are defined on the testsuite. |
Test Groups
Nightwatch makes it possible to organize your test scripts into groups and run them as needed. To group tests together just place them in the same sub-folder. The folder name is the name of the group.
Example:
lib/ ├── selenium-server-standalone.jar custom-commands/ ├── loginUser.js ├── attachPicture.js tests/ ├── logingroup | ├── login_test.js | └── otherlogin_test.js ├── addressbook | ├── addressbook_test.js | └── contact_test.js ├── chat | ├── chatwindow_test.js | ├── chatmessage_test.js | └── otherchat_test.js └── smoketests ├── smoke_test.js └── othersmoke_test.js
To run only the smoketests
group you would do the following:
$ nightwatch --group smoketests
Also, if you would want to skip running the smoketests
group you would do the following:
$ nightwatch --skipgroup smoketests
To skip multiple groups, just add them as comma-separated:
$ nightwatch --skipgroup addressbook,chat
Test Tags
You can also selectively target tests to run based on tags, such that a test may be belong to multiple tags. For example, you might have a login test that belongs to a login suite as well as a sanity suite.
The tagging can be accomplished by adding the @tags
property to a test module:
module.exports = {
'@tags': ['login', 'sanity'],
'demo login test': function (client) {
// test code
}
};
To select which tags to run, use the --tag
command line flag:
$ nightwatch --tag login
Specify multiple tags as:
$ nightwatch --tag login --tag something_else
To skip running tests with a specific tag, use the --skiptags
flag:
$ nightwatch --skiptags login
Or to skip multiple tags, add each tag you want to skip as comma-separated:
$ nightwatch --skiptags login,something_else
Disabling Tests
To prevent a test module from running, simply set the disabled
attribute in that module to true
, like so:
module.exports = {
'@disabled': true, // This will prevent the test module from running.
'sample test': function (client) {
// test code
}
};
This can be useful if you don't want to run certain tests that are known to be failing.
Disabling Individual Testcases
Disabling individual testcases isn't currently supported out of the box. However it can be achieved relatively straightforward with a simple work-around. By simply converting the test method to a string, Nightwatch will ignore it.
Here's an example:
module.exports = {
'sample test': function (client) {
// test code
},
// disabled
'other sample test': ''+function (client) {
// test code
}
};
Parallel Running
Starting with v0.5
Nightwatch supports the tests to be run in parallel. This works by specifying multiple environments in the command line, separated by comma. E.g.:
$ nightwatch -e default,chrome
The above will run two environments named default
and chrome
in parallel.
Terminal Output
Each environment will be run as a separate child_process
and the output will be sent to the main process.
To make the output easier to read, Nightwatch by default buffers the output from each child process and displays everything at the end, grouped by environment.
"live_output" : true
on the top level in your nightwatch.json
(e.g. after selenium
).
desiredCapabilities
) and then run them in parallel. In addition, using the filter
and exclude
options tests can be split per environment in order to be ran in parallel.
Via Workers
Version v0.7
introduces a new feature which allows the tests to be run in parallel. When this is enabled the test runner will launch a configurable number of child processes and then distribute the loaded tests over to be ran in parallel.
To enable test workers, set the test_workers
top-level property, like so:
"test_workers": {
"enabled": true,
"workers": "auto"
}
or, simply:
"test_workers": true
The
workers
option configures how many child processes can run concurrently.
"auto"
- determined by number of CPUs e.g. 4 CPUs means 4 workers{number}
- specifies an exact number of workers
Test concurrency is done at the file level. Each test file will fill a test worker slot. Individual tests/steps in a test file will not run concurrently.
detailed_output
to false
in your test settings for improved output readability.
Using Grunt
Grunt is a popular JavaScript task runner. Starting with version 0.6
Nightwatch is bundled with
an easy to use Grunt task which can be used in your existing Grunt-based build configuration for running the tests.
Usage
First, load the Nightwatch grunt task at the top in your Gruntfile.js
.
module.exports = function(grunt) {
var nightwatch = require('nightwatch');
nightwatch.initGrunt(grunt);
// ...
};
Task Configuration and Targets
The Nightwatch task will have one or more targets which can be used in various ways, one way being to map them to environments. Available settings are:
options
- the only available option so far iscwd
- current working directoryargv
- command-line arguments that would normally be passed to the Nightwatch runner (e.g.:env
);settings
- the test settings specified to a single Nightwatch environment.
Example
grunt.initConfig({
nightwatch: {
options: {
cwd: './'
},
'default' : {},
browserstack: {
argv: {
env: 'browserstack'
},
settings: {
silent: true
}
},
'all' : {
argv: {
env: 'default,browserstack'
}
},
}
});
Run the task as follows:
$ grunt nightwatch:default
or
$ grunt nightwatch:browserstack
There are also a few third-party Grunt plugins for Nightwatch which can be used instead, if you prefer. The most popular one is grunt-nightwatch.
Using Mocha
Starting with version 0.8
Nightwatch is bundled with a custom version of the popular Mocha test runner which allows running tests using Mocha, thus taking advantage of its interfaces and reporters.
Usage
There are two main ways in which you can use Mocha with Nightwatch.
From Nightwatch
Mocha is used as an alternative test runner to the built-in one. This is done by specifying the "test_runner"
option in the nightwatch.json
configuration file.
Custom options can also be specified for Mocha:
{
...
"test_runner" : {
"type" : "mocha",
"options" : {
"ui" : "bdd",
"reporter" : "list"
}
}
...
}
or simply:
{
...
"test_runner" : "mocha"
...
}
A complete list of Mocha options that are supported can be found here.
The test_runner
option can also be specified at test environment level:
{
"test_settings" : {
"mocha_tests" : {
"test_runner" : {
"type" : "mocha",
"options" : {
"ui" : "tdd",
"reporter" : "list"
}
}
}
}
...
}
Example
Writing a test in Mocha is the same as writing it in Nightwatch. Each testcase receives the client
object, hooks
also receiving a done
callback for async operations.
describe('Google demo test for Mocha', function() {
describe('with Nightwatch', function() {
before(function(client, done) {
done();
});
after(function(client, done) {
client.end(function() {
done();
});
});
afterEach(function(client, done) {
done();
});
beforeEach(function(client, done) {
done();
});
it('uses BDD to run the Google simple test', function(client) {
client
.url('http://google.com')
.expect.element('body').to.be.present.before(1000);
client.setValue('input[type=text]', ['nightwatch', client.Keys.ENTER])
.pause(1000)
.assert.containsText('#main', 'Night Watch');
});
});
});
When using the mocha test runner from Nightwatch some cli options are not available, like --retries
, --suiteRetries
, --reporter
.
Using the standard Mocha
Running Nightwatch tests with the standard Mocha it's also possible, though a bit more boilerplate code is involved and you need to manage the selenium server.
Example
var nightwatch = require('nightwatch');
describe('Github', function() {
var client = nightwatch.initClient({
silent : true
});
var browser = client.api();
this.timeout(99999999);
before(function() {
browser.perform(function() {
console.log('beforeAll')
});
});
beforeEach(function(done) {
browser.perform(function() {
console.log('beforeEach')
});
client.start(done);
});
it('Demo test GitHub', function (done) {
browser
.url('https://github.com/nightwatchjs/nightwatch')
.waitForElementVisible('body', 5000)
.assert.title('nightwatchjs/nightwatch · GitHub')
.waitForElementVisible('body', 1000)
.assert.visible('.container .breadcrumb a span')
.assert.containsText('.container .breadcrumb a span', 'nightwatch', 'Checking project title is set to nightwatch');
client.start(done);
});
afterEach(function() {
browser.perform(function() {
console.log('afterEach')
});
});
after(function(done) {
browser.end(function() {
console.log('afterAll')
});
client.start(done);
});
});
Using Page Objects
The Page Objects methodology is a popular pattern to write end-to-end tests by wrapping the pages or page fragments of a web app into objects. The purpose of a page object is to allow a software client to do anything and see anything that a human can by abstracting away the underlying html actions needed to access and manipulate the page.
A comprehensive introduction to Page Objects can be found in this article.
0.7
Nightwatch provides an enhanced and more powerful interface for creating page objects, significantly improved over the previous support. Page objects created prior to v0.7
will still continue to work however we recommend upgrading to the new version.
To use the new version, your page object must contain either the elements
or sections
property. Otherwise, Nightwatch will defer to the old.
Configuring Page Objects
To create a page object simply create an object with properties that describe the page. Each page object should be located in a separate file, located in a designated folder. Nightwatch reads the page objects from the folder (or folders) specified in the page_objects_path
configuration property.
The page_objects_path
property can also be an array of folders, allowing you thus to logically split the page objects into smaller groups.
The Url property
You can optionally add a url
property that designates the page's URL. To navigate to the page, you can call the navigate
method on the page object.
The URL will usually be defined as a string:
module.exports = {
url: 'http://google.com',
elements: {}
};
It can also be a function in case the URL is dynamic. One use case for this is to support different test environments. You can create a function that gets called in the context of the page, thus allowing you to do:
module.exports = {
url: function() {
return this.api.launchUrl + '/login';
},
elements: {}
};
Defining Elements
Most of the time, you will want to define elements on your page that your tests will interact with through commands and assertions. This is made simple using the elements
property so that all your elements are defined in a single place. Especially in larger integration tests, using elements
will go a long way to keep test code DRY.
Switching between css and xpath locate strategies is handled internally so you don't need to call useXpath
and useCss
in your tests. The default locateStrategy
is css but you can also specify xpath:
module.exports = {
elements: {
searchBar: {
selector: 'input[type=text]'
},
submit: {
selector: '//[@name="q"]',
locateStrategy: 'xpath'
}
}
};
Or if you're creating elements with the same locate strategy as is default, you can use the shorthand:
module.exports = {
elements: {
searchBar: 'input[type=text]'
}
};
Using the elements
property allows you to refer to the element by its name with an "@" prefix, rather than selector, when calling element commands and assertions (click
, etc).
Optionally, you can define an array of objects:
var sharedElements = {
mailLink: 'a[href*="mail.google.com"]'
};
module.exports = {
elements: [
sharedElements,
{ searchBar: 'input[type=text]' }
]
};
Putting elements
and url
together, say you have the following defined above saved as a google.js
file:
module.exports = {
url: 'http://google.com',
elements: {
searchBar: {
selector: 'input[type=text]'
},
submit: {
selector: '//[@name="q"]',
locateStrategy: 'xpath'
}
}
};
In your tests you will use it as follows:
module.exports = {
'Test': function (client) {
var google = client.page.google();
google.navigate()
.assert.title('Google')
.assert.visible('@searchBar')
.setValue('@searchBar', 'nightwatch')
.click('@submit');
client.end();
}
};
Defining Sections
Sometimes it is useful to define sections of a page. Sections do 2 things:
- Provide a level of namespacing under the page
- Provide element-level nesting so that any element defined within a section is a descendant of its parent section in the DOM
You can create sections using the sections
property:
module.exports = {
sections: {
menu: {
selector: '#gb',
elements: {
mail: {
selector: 'a[href="mail"]'
},
images: {
selector: 'a[href="imghp"]'
}
}
}
}
};
Your tests would use it as follows:
module.exports = {
'Test': function (client) {
var google = client.page.google();
google.expect.section('@menu').to.be.visible;
var menuSection = google.section.menu;
menuSection.expect.element('@mail').to.be.visible;
menuSection.expect.element('@images').to.be.visible;
menuSection.click('@mail');
client.end();
}
};
expect
assertions) returns that section for chaining. If desired, you can nest sections under other sections for complex DOM structures.
Example of nesting page object sections:
module.exports = {
sections: {
menu: {
selector: '#gb',
elements: {
mail: {
selector: 'a[href="mail"]'
},
images: {
selector: 'a[href="imghp"]'
}
},
sections: {
apps: {
selector: 'div.gb_pc',
elements: {
myAccount: {
selector: '#gb192'
},
googlePlus: {
selector: '#gb119'
}
}
}
}
}
}
};
Using a nested section in your test is straightforward:
module.exports = {
'Test': function (client) {
var google = client.page.google();
google.expect.section('@menu').to.be.visible;
var menuSection = google.section.menu;
var appSection = menuSection.section.apps;
menuSection.click('@appSection');
appSection.expect.element('@myAccount').to.be.visible;
appSection.expect.element('@googlePlus').to.be.visible;
client.end();
}
};
Writing Commands
You can add commands to your page object using the commands
property. This is a useful way to encapsulate logic about the page that would otherwise live in a test, or multiple tests.
Nightwatch will call the command on the context of the page or section. Client commands like pause
are available via this.api
. For chaining, each function should return the page object or section.
In this case, a command is used to encapsulate logic for clicking the submit button:
var googleCommands = {
submit: function() {
this.api.pause(1000);
return this.waitForElementVisible('@submitButton', 1000)
.click('@submitButton')
.waitForElementNotPresent('@submitButton');
}
};
module.exports = {
commands: [googleCommands],
elements: {
searchBar: {
selector: 'input[type=text]'
},
submitButton: {
selector: 'button[name=btnG]'
}
}
};
Then the test is simply:
module.exports = {
'Test': function (client) {
var google = client.page.google();
google.setValue('@searchBar', 'nightwatch')
.submit();
client.end();
}
};
Writing Custom Commands
Most of the time you will need to extend the Nightwatch commands to suit your own application needs. Doing that is only a matter of creating a separate folder and defining your own commands inside there, each one inside its own file.
Then specify the path to that folder inside the nightwatch.json
file, as the custom_commands_path
property. The command name is the name of the file itself.
There are two main ways in which you can define a custom command:
1) Function-style commands
This is the simplest form in which commands are defined, however they are also quite limited. Your command module needs to export a command
function, which needs to call at least one Nightwatch api method (such as .execute()
). This is due to a limitation of how the asynchronous queueing system of commands works. You can also wrap everything in a .perform()
call. Client commands like execute
and perform
are available via this
.
exports.command = function(file, callback) {
var self = this;
var imageData;
var fs = require('fs');
try {
var originalData = fs.readFileSync(file);
var base64Image = new Buffer(originalData, 'binary').toString('base64');
imageData = 'data:image/jpeg;base64,' + base64Image;
} catch (err) {
console.log(err);
throw "Unable to open file: " + file;
}
this.execute(
function(data) { // execute application specific code
App.resizePicture(data);
return true;
},
[imageData], // arguments array to be passed
function(result) {
if (typeof callback === "function") {
callback.call(self, result);
}
}
);
return this;
};
The example above defines a command (e.g. resizePicture.js) which loads an image file as data-URI
and calls a method named resizePicture
(via .execute()
), defined inside the application.
With this command, the test will look something like:
module.exports = {
"testing resize picture" : function (browser) {
browser
.url("http://app.host")
.waitForElementVisible("body")
.resizePicture("/var/www/pics/moon.jpg")
.assert.element(".container .picture-large")
.end();
}
};
2) Class-style commands
This is how most of the Nightwatch's own commands are written. Your command module needs to export a class constructor with a command
instance method representing the command function. Commands written like this should inherit from EventEmitter
and manually signal the complete
event, to indicate command completion.
Class-based command
methods are run in the context (the value of this
) of the class instance. The test api
object is available as this.api
or this.client.api
, where this.client
is the Nightwatch instance itself.
The example below is the .pause()
command:
var util = require('util');
var events = require('events');
function Pause() {
events.EventEmitter.call(this);
}
util.inherits(Pause, events.EventEmitter);
Pause.prototype.command = function(ms, cb) {
var self = this;
// If we don't pass the milliseconds, the client will
// be suspended indefinitely
if (!ms) {
return this;
}
setTimeout(function() {
// if we have a callback, call it right before the complete event
if (cb) {
cb.call(self.client.api);
}
self.emit('complete');
}, ms);
return this;
};
module.exports = Pause;
The "complete" event
Signaling the complete
event needs to be done inside an asynchronous action (e.g. a setTimeout
call). Command classes that do not extend EventEmitter
will be treated similar to command functions, requiring that the command
method calls at least one Nightwatch api method to be able to complete.
Using ES6 classes as custom commands is not supported at the moment. See nightwatchjs#1199 for more details.
Writing Custom Assertions
Nightwatch allows you to even define your own assertions, extending the available .assert
and .verify
namespaces.
Assertions implement a simple interface which is shared between built-in assertions and custom ones:
exports.assertion = function() {
/**
* The message which will be used in the test output and
* inside the XML reports
* @type {string}
*/
this.message;
/**
* A value to perform the assertion on. If a function is
* defined, its result will be used.
* @type {function|*}
*/
this.expected;
/**
* The method which performs the actual assertion. It is
* called with the result of the value method as the argument.
* @type {function}
*/
this.pass = function(value) {
};
/**
* The method which returns the value to be used on the
* assertion. It is called with the result of the command's
* callback as argument.
* @type {function}
*/
this.value = function(result) {
};
/**
* Performs a protocol command/action and its result is
* passed to the value method via the callback argument.
* @type {function}
*/
this.command = function(callback) {
return this;
};
};
Custom assertions also inherit from EventEmitter. To see some examples, check the assertions module on Github:
/nightwatch/tree/master/lib/selenium/assertions
Custom Reporter
If you'd like to define your own reporter in addition to the built-in ones (stdout and junit-xml) you can do so in two ways:
The --reporter
command-line argument
Interface:
module.exports = {
write : function(results, options, done) {
done();
}
};
The reporter
method in your external globals
file.
See the provided globalsModule.js for an example.
Example:
module.exports = {
reporter : function(results, done) {
console.log(results);
done();
}
};
Writing Unit Tests
Unit testing in Nightwatch has been refined in version 0.9
. Unit tests now written in Nightwatch are also fully compatible with Mocha's Exports interface, so you can use either test runners. In fact, all Nightwatch's unit tests have been rewritten so they can be ran with either Nightwatch or Mocha.
compatible_testcase_support
to true
in your test settings.
Unit tests written in versions prior to 0.9
will still continue to work however we recommend upgrading them.
Disabling automatic selenium session
Nightwatch automatically attempts to connect to the specified selenium server and create a session.
When running unit tests this needs to be disabled by setting the start_session
property to false
inside the selenium
settings group either on the root level or inside a specific environment.
Assertion framework
Starting with 0.9
, in the improved support for unit tests, the client
object is no longer passed as an argument to the test. The only argument passed now is the done
callback to be used for asynchronous tests.
You can use whatever assertion framework you like. Chai.js is quite a good one and very flexible. We use the internal Node.js assert
module in the Nightwatch unit tests.
You can still refer the client
object via this.client
in your tests.
Example
Here's a subset of the unit test for the utils.js
Nightwatch module:
var assert = require('assert');
var common = require('../../common.js');
var Utils = common.require('util/utils.js');
module.exports = {
'test Utils' : {
testFormatElapsedTime : function() {
var resultMs = Utils.formatElapsedTime(999);
assert.equal(resultMs, '999ms');
var resultSec = Utils.formatElapsedTime(1999);
assert.equal(resultSec, '1.999s');
var resultMin = Utils.formatElapsedTime(122299, true);
assert.equal(resultMin, '2m 2s / 122299ms');
},
testMakeFnAsync : function() {
function asyncFn(cb) {
cb();
}
function syncFn() {}
var convertedFn = Utils.makeFnAsync(1, syncFn);
var called = false;
convertedFn(function() {
called = true;
});
assert.equal(Utils.makeFnAsync(1, asyncFn), asyncFn);
assert.ok(called);
}
}
};
Asynchronous Unit Tests
The argument to the test function is the optional done
callback which signals the test is complete.
If present, the callback must be called when the async operation finishes.
Example
Here's unit test which checks if Nightwatch throws an error if you don't invoke the done
callback within a set time (10 ms).
module.exports = {
var path = require('path');
var assert = require('assert');
var common = require('../../common.js');
var CommandGlobals = require('../../lib/globals/commands.js');
var Runner = common.require('runner/run.js');
module.exports = {
'testRunner': {
before: function (done) {
CommandGlobals.beforeEach.call(this, done);
},
after: function (done) {
CommandGlobals.afterEach.call(this, done);
},
beforeEach: function () {
process.removeAllListeners('exit');
process.removeAllListeners('uncaughtException');
},
'test async unit test with timeout error': function (done) {
var testsPath = path.join(__dirname, '../../asynchookstests/unittest-async-timeout.js');
var globals = {
calls : 0,
asyncHookTimeout: 10
};
process.on('uncaughtException', function (err) {
assert.ok(err instanceof Error);
assert.equal(err.message, 'done() callback timeout of 10 ms was reached while executing "demoTest". ' +
'Make sure to call the done() callback when the operation finishes.');
done();
});
var runner = new Runner([testsPath], {
seleniumPort: 10195,
silent: true,
output: false,
persist_globals : true,
globals: globals,
compatible_testcase_support : true
}, {
output_folder : false,
start_session : false
});
runner.run().catch(function(err) {
done(err);
});
}
}
};
};
The complete test suite can be viewed on GitHub: https://github.com/nightwatchjs/nightwatch/tree/master/test/src/runner/testRunner.js
Running the Nightwatch unit tests
To get an idea of how running unit tests with Nightwatch works you can head over to our GitHub page, clone the project and follow the instructions on how to run the tests.
You can also check out Nightwatch's own complete test suite for examples: https://github.com/nightwatchjs/nightwatch/tree/master/test/src
Here's the configuration needed to run them:
{
"src_folders" : ["./test/src"],
"selenium" : {
"start_process" : false,
"start_session" : false
},
"test_settings" : {
"default" : {
"filter" : "*/.js",
"compatible_testcase_support" : true
}
}
}
Using a Combined Configuration
Below it's an example of how you can combine end-to-end tests and unit tests in the same nightwatch.json
configuration file.
Notice the usage of exclude
and filter
properties.
An empty exclude
means we want to reset its value and rely only on filter
.
{
"src_folders" : ["./examples/tests", "./examples/unittests"],
"output_folder" : "./examples/reports",
"selenium" : {
"start_process" : true,
"server_path" : "./bin/selenium-server-standalone.jar",
"log_path" : "",
"host" : "127.0.0.1",
"port" : 4444,
"cli_args" : {
"webdriver.chrome.driver" : "",
"webdriver.ie.driver" : ""
}
},
"test_settings" : {
"default" : {
"launch_url" : "http://localhost",
"selenium_port" : 4444,
"selenium_host" : "localhost",
"silent": true,
"screenshots" : {
"enabled" : false,
"path" : ""
},
"desiredCapabilities": {
"browserName": "firefox",
"javascriptEnabled": true,
"acceptSslCerts": true
},
"exclude" : "./examples/unittests/*"
},
"unittests" : {
"selenium" : {
"start_process" : false,
"start_session" : false
},
"filter" : "./examples/unittests/*",
"exclude" : ""
}
}
}
Code Coverage
At the moment, Nightwatch doesn't provide a coverage reporter but it is something that's being planned for a future release. In the meantime you can write a custom reporter which will output coverage data. See the custom reporter section for details and the Mocha HTMLCov reporter for how the reporter should look like.
3rd party coverage service
There are some hosted services which provide the reporting and metrics for you in a modern web interface. These services will typically require coverage data in LCOV format. Nightwatch uses coveralls.io.
For details on how an LCOV reporter should look like and how to integrate with your project, you can check out the mocha-lcov-reporter.