Multi-layer parametrization
Parametrize at the suite level and the test level. Write one test method, run it across every combination of environments, browsers or user roles, and get per-variant results, retries and reports.
Suite-level parameters
parameters= on @Suite runs the suite once per entry. Any hook or test that declares a suite_parameter argument receives the current entry:
@Suite(parameters=[ {"env": "https://staging.shop.example"}, {"env": "https://shop.example"}, ]) class CheckoutSuite: @beforeClass() def setup(self, suite_parameter): self.base_url = suite_parameter["env"] @test() def add_to_cart(self, suite_parameter): # runs twice: once against staging, once against production checkout(self.base_url)
Test-level parameters
parameters= on @test runs that test once per entry, within each suite run. The current entry is passed as parameter:
@Suite() class CheckoutSuite: @test(parameters=[ {"browser": "chrome"}, {"browser": "firefox"}, {"browser": "safari"}, ]) def checkout_works(self, parameter): driver = new_driver(parameter["browser"]) ...
Parameters can also come from a function that builds the list when the run starts, e.g. from a CSV file or a database: see data-driven testing.
Both layers together: every combination
With parameters= on both, Test Junkie runs every combination. Two environments × three browsers gives six variants from one method:
@Suite(parameters=[ {"env": "https://staging.shop.example"}, {"env": "https://shop.example"}, ]) class CheckoutSuite: @test(parameters=[ {"browser": "chrome"}, {"browser": "firefox"}, {"browser": "safari"}, ]) def checkout_works(self, parameter, suite_parameter): driver = new_driver(parameter["browser"]) checkout(driver, suite_parameter["env"]) # 6 variants: staging/chrome, staging/firefox, staging/safari, # production/chrome, production/firefox, production/safari
parameter and suite_parameter. The names are fixed: read parameter["browser"], there's no browser argument.Which functions take part
Whether a test runs once or once per suite parameter depends on its own arguments:
- Tests that declare
suite_parameterrun once per suite parameter. A test that doesn't declare it runs once in total, however many suite parameters there are. @beforeClassand@afterClassrun once per suite parameter whether or not they declare it; declare it to read the value.@beforeTestand@afterTestrun around every test run. If they declaresuite_parameter, it isNonearound a test that doesn't take suite parameters. They can't declareparameter.
@Suite(parameters=[{"env": "staging"}, {"env": "prod"}]) class CheckoutSuite: @beforeClass() def setup(self, suite_parameter): # runs twice: staging, then prod self.base_url = urls[suite_parameter["env"]] @afterClass() def teardown(self): # no suite_parameter, still runs twice: once per suite run cleanup() @beforeTest() def before_each(self, suite_parameter): # None before homepage_loads, which doesn't take suite parameters if suite_parameter is not None: log("starting a test on {}".format(suite_parameter["env"])) @test() def homepage_loads(self): # no suite_parameter: runs once in total ... @test() def checkout_works(self, suite_parameter): # runs twice: staging and prod checkout(self.base_url)
if suite_parameter is not None check above, suite_parameter["env"] raises a TypeError before homepage_loads, and that test is reported as an error without running.Per-variant results and retries
Each variant has its own result. If the production/safari variant fails, the others are unaffected, and retries apply per variant: only the variants that failed run again.
@test( retry=2, retry_on=[ConnectionError], parameters=[{"browser": "chrome"}, {"browser": "safari"}], ) def checkout_works(self, parameter): ... # if the safari variant raises a ConnectionError, only it runs again; # the chrome variant's result stands
In the console and the reports every variant is listed with its values, e.g. CheckoutSuite.checkout_works [suite: {'env': 'prod'}, {'browser': 'safari'}].