Shared setup and teardown across suites

As a project grows, the same setup, checks and cleanup end up copy-pasted into every suite. A Rules subclass breaks that cycle: define the shared part once, attach it to any suite with one argument.

The problem: the same hooks in every suite

In a browser project, every suite checks the browser console for JavaScript errors after each test and closes the browser. Without shared rules, that code lives in every suite:

suites.pypython

@Suite()
class LoginSuite:

    @afterTest()
    def check_js_errors(self):
        logs = Browser.get_driver().get_log("browser")
        Browser.shutdown()
        errors = [entry["message"] for entry in logs if entry["level"] == "SEVERE"]
        assert not errors, errors

    @test()
    def valid_login(self):
        ...


@Suite()
class CheckoutSuite:

    @afterTest()
    def check_js_errors(self):
        ...  # the same lines again, in every suite

The solution: a Rules subclass

Move the shared logic into a Rules subclass and attach it with rules=. Raising in a rule hook fails the test, exactly like an @afterTest that raises:

browser_rules.pypython

from test_junkie.rules import Rules

from browser import Browser


class BrowserRules(Rules):

    def after_test(self, **kwargs):
        if not Browser.is_open():
            return  # the test failed before it opened a browser
        try:
            logs = Browser.get_driver().get_log("browser")
        finally:
            Browser.shutdown()  # closed even when the check below fails the test
        errors = [entry["message"] for entry in logs if entry["level"] == "SEVERE"]
        if errors:
            raise AssertionError("JavaScript errors: {}".format(errors))

suites.pypython

@Suite(rules=BrowserRules)
class LoginSuite:

    @test()
    def valid_login(self):
        ...


@Suite(rules=BrowserRules)
class CheckoutSuite:

    @test()
    def add_to_cart(self):
        ...

# both suites check for JavaScript errors and close the browser after every test - one definition

Browser here is a small helper that gives each thread its own WebDriver; part 1 of the Selenium series builds it.

Rules run around the suite's own hooks

A suite can have rules=BrowserRules and its own @beforeClass, @beforeTest, @afterTest and @afterClass. The rule's before_class and before_test run before the suite's hooks, its after_test and after_class run after them. Use the rule for the shared part and the suite's hooks for what's specific to it.

self in a rule is the rule, not the suite

Test Junkie creates one BrowserRules object per suite and calls the hooks on it. So self.driver = ... inside before_class sets an attribute on the rule object; the suite's tests never see it. Anything a test and a rule both need (a driver, an API session) should come from a shared helper like Browser.get_driver().

What the rule does get: self.kwargs["suite"], a copy of the suite's SuiteObject (its class name, owner, feature and so on), and in before_test / after_test, kwargs["test"], the TestObject that is about to run or just ran.

logging_rules.pypython

import logging

from test_junkie.rules import Rules


class LoggingRules(Rules):

    def before_class(self):
        logging.info("starting %s", self.kwargs["suite"].get_class_name())

    def before_test(self, **kwargs):
        logging.info("running %s", kwargs["test"].get_function_name())

Available hooks

Override any combination; the ones you don't override aren't called at all:

rules_reference.pypython

from test_junkie.rules import Rules


class MyRules(Rules):

    def before_class(self):
        # before the suite's @beforeClass - once per suite run, and once per suite parameter
        ...

    def before_test(self, **kwargs):
        # before each @beforeTest; kwargs["test"] is the TestObject about to run
        ...

    def after_test(self, **kwargs):
        # after each @afterTest; raising here fails the test
        ...

    def after_class(self):
        # after the suite's @afterClass
        ...

Real-world tip: a screenshot when a test fails

When a run fails, after_test can already see it: the failed run is recorded before the hook is called, a passing run only after. Name the file after kwargs["test"] so every test gets its own screenshot:

browser_rules.pypython

import time

from test_junkie.constants import TestCategory
from test_junkie.rules import Rules

from browser import Browser


class BrowserRules(Rules):

    def after_test(self, **kwargs):
        test = kwargs["test"]
        if Browser.is_open() and BrowserRules.failed(test):
            Browser.get_driver().save_screenshot("screenshots/{}-{}.png".format(
                test.get_function_name(), int(time.time())))
        ...  # then the JavaScript error check from above

    @staticmethod
    def failed(test):
        return any(data.get("status") in TestCategory.ALL_UN_SUCCESSFUL
                   for by_param in test.metrics.get_metrics().values() for data in by_param.values())

Parameters and retries

failed() looks at every recorded run of the test. For a parameterized or retried test an earlier failing variant or run also counts, so you may get an extra screenshot; the timestamp in the file name keeps them apart.