Running tests in parallel
Test Junkie runs suites and tests in parallel threads as soon as you raise the thread limits. Everything is allowed to run in parallel by default: you opt out with parallelized=False where code isn't thread-safe, and opt in to running a test's parameter variants in parallel.
Turn it on: thread limits
Both limits default to 1, which runs everything one at a time. Raise them on run():
from test_junkie.runner import Runner runner = Runner([LoginSuite, CheckoutSuite, SearchSuite]) runner.run( suite_multithreading_limit=3, # up to 3 suites at once test_multithreading_limit=8, # up to 8 tests in their own threads at once, across all suites )
From the command line it's tj run -S 3 -T 8. test_multithreading_limit is one pool shared by all suites, not a limit per suite.
Opting out: parallelized=False
Some suites and tests can't share the machine: a test that changes global settings, a suite that resets the database. Mark them parallelized=False:
@Suite(parallelized=False) class DatabaseMigrationSuite: # runs on its own: no other suite runs at the same time ... @Suite() class CheckoutSuite: @test() def add_to_cart(self): ... # runs in parallel with other tests @test(parallelized=False) def change_store_currency(self): ... # never overlaps another test
Parameter variants: parallelized_parameters=True
A parameterized test runs its variants one after another by default. parallelized_parameters=True runs them in parallel threads too (they count towards test_multithreading_limit):
@Suite() class CheckoutSuite: @test( parameters=["visa", "mastercard", "paypal"], parallelized_parameters=True, ) def pay_with(self, parameter): ... Runner([CheckoutSuite]).run(test_multithreading_limit=3) # all 3 payment methods run at the same time
parallelized (default True) decides whether a test may run alongside other tests. parallelized_parameters (default False) decides whether the variants of one test run alongside each other. Both need test_multithreading_limit above 1.Suite parameters run one after another
The runs of a suite with @Suite(parameters=...) happen one after another in that suite's thread: staging, then production. suite_multithreading_limit runs different suites at the same time; within each suite run, the tests use the test threads as usual.
@Suite(parameters=[{"env": "staging"}, {"env": "prod"}]) class CheckoutSuite: ... Runner([CheckoutSuite, SearchSuite]).run(suite_multithreading_limit=2, test_multithreading_limit=6) # CheckoutSuite (staging, then prod) runs alongside SearchSuite, # and the tests inside each run share the 6 test threads
self in @beforeClass (a driver, a session) is shared by all of them. Keep per-test state in local variables, or give each thread its own object, like the Browser helper in the Selenium series does.See what the threads did
tj run -m charts CPU and memory after the run, with how many tests ran at once and when each suite ran, so you can see whether more threads would help. The HTML report's threading analysis lists the slowest tests.