Testing

• Running tests
• Types of tests
  • End-to-end
  • Behaviour driven testing
  • Unit tests
• During development
  • Python
  • Javascript
  • BDD tests
  • End to end testing (e2e)

Running tests

Tests can be run in a variety of ways - for explicit details on how to run tests, see how tests are configured to run in .gitlab-ci.yml.

Types of tests

End-to-end

A complete app with frontend and backend is started. Several use-cases are executed through the browser simulating a user clicking and entering text. The tests are written in Javascript and executed using [webdriverIO] (http://webdriver.io/).

Behaviour driven testing

Tests (called features) are defined with Cucumber (opens new window) in bdd/tests. The tool behaving (opens new window) (built on top of behave (opens new window)) is used to communicate with webdriverIO to run the tests in a browser.

For both end-to-end tests (for the legacy frontend) and BDD tests, a docker compose service is defined to run Selenium Chrome, which is available at http://localhost:7900.

Unit tests

Testing of isolated Javascript/Typescript or Python functions.

During development

Python

If you want to run a specific API test while developing, enter your development container in VSCode, and select the Test-section in the left sidebar. From there you can run and debug specific tests.

Alternatively, you can use py.test <path> like you normally would, in the integrated terminal.

The database fixtures are setup in different conftest.py files.

Javascript

You can run yarn test-watch inside the container to watch for changes.

BDD tests

From the ELLA BDD tests dev container, simply run behave bdd/tests to run all tests, or run on single files. See Using behave (opens new window) for details on command line arguments.

End to end testing (e2e)

If you want to run tests on a specific database state, you can create a dump of that state to be reloaded later. See testdata/README (opens new window).

We use webdriver.io (opens new window) for testing.

CI tests include an end-to-end (e2e) test (make e2e-test). This will start Chrome inside the container and run the test suites. The e2e test can be run locally to check that the tests are passing but is unsuitable for authoring/editing tests.

To explore the e2e test data, start a local ELLA instance and import the e2e test data: make dbreset TESTSET=e2e

Local e2e

The following must be installed:

• Chrome
• Chromedriver

The ELLA app and the test execution (wdio) can be either run locally on your host machine or inside Docker:

1. Start chromedriver on your host machine:

   ./chromedriver  --port=4444 --whitelisted-ips= --url-base ''
   

2. Start the tests:

   make e2e-test-local [options]
   

This will connect to the locally running Chromedriver and run one or several test specs. You'll see a local chrome browser where a "ghost" will click and enter text.

You can put debug statements (browser.debug()) in your test spec to have the test execution stop and enter a REPL to interact with the browser. You can also open the dev tools in Chrome to dig around. Exit the REPL to have the test continue.

Relevant [options] for the make command:

Option	Explanation
DEBUG=true	Will make the browser visible (as opposed to headless), and increase test timeouts
CHROME_HOST	The IP address where the chromedriver is running. This will start a Chrome browser.
SPEC="\<path to test>"	Add this to run only a single/few tests. They must given as wdio-legacy/e2e/tests/.. (comma separated if multiple).
APP_URL	URL of the app to test, e.g http://localhost:8001. Make sure to use an ip/port that is accessible from within the container where the tests themselves are running. If not defined, the app running inside container of the test execution is used.

NOTE

Maximize the Chrome window to reduce the number of 'element-not-clickable' errors.

Misc

To get and test selectors in Chrome, use an extension, or search (Ctrl+F) in the Developer Tools.

You can connect a debugger to Node.js instance on port 5859 to play around.

Use browser.debug() in a test file to pause the execution of the tests. This will present a REPL (webdriverio >= 4.5.0) where can you interact with the webdriverio client to try out various commands, like browser.element(...). It's also useful to head over to the browser's console to experiment and inspect variables.

Hit Ctrl-C in the REPL to continue the test run. See more on https://webdriver.io/docs/repl/ (opens new window).

More info at https://webdriver.io/docs/debugging (opens new window)