Open source · MIT licensed

The Python test runner built for precision.

▸ Light. Fast. No plugins.

Test Junkie runs your Python tests with parallel execution, retries, tags and HTML reports built in. One pip install and the whole toolkit is there.

Get Started View on GitHub
Behind this: the HTML report
The Essentials

Everything you'd expect

The building blocks of every framework — clean, decorator-based, zero boilerplate.

Suites & Tests

Plain Python classes with no base class to extend, so your IDE, linter and debugger just work.

@Suite@test
Read the docs →

Lifecycle Hooks

Setup and teardown declared on the class, each firing at exactly the right scope.

@beforeClass@afterClass@beforeTest@afterTest
Read the docs →

Automatic Retries

Re-run what fails, add retry policies for precision, and flag tests that pass only on retry as flaky.

retry=NRetryPolicy--fail-on-flaky
More on retries ↓

Built-in Reports

HTML, XML and JSON reports after every run, with no plugins and no config.

--html-report--xml-report--json-report
See a sample HTML report →
Beyond the basics

Built into the core

Parallel runs, retries, flaky-test detection and metadata ship in the package, with no plugins or glue code to add.

Parallel Execution You Can Tune

Performance

Suite threads and test threads are set separately, with no plugin or external orchestrator. When tests share something fragile, like a browser grid, a rate-limited API or a staging database, throttling, ramp-up and resource pools keep it upright without slowing everything else down.

  • -S 4 -T 20 runs up to 4 suites and 20 tests at once. Keep a suite or test out of the parallel pool with parallelized=False (such a test runs alone), and keep specific suites or tests apart with conflicts_with=[...].
  • Throttling spaces out starts across the whole run (--test-throttling 0.5), @Suite(throttling=0) exempts a fast suite, and --ramp-up 30 grows the thread count gradually.
  • Resource pools cap only the tests that need them: Limiter.pool("grid", max_concurrent=5) plus @test(uses="grid"). The rest of the run still uses all 20 threads.
Explore parallelism → Throttling & pools docs → 
# tj run -S 4 -T 20 --test-throttling 0.5 --ramp-up 30
Limiter.pool("grid", max_concurrent=5)       # 5 browsers
Limiter.pool("payments", max_concurrent=2,   # sandbox API:
             min_interval=0.5)               # 2 calls a second

@Suite(uses="grid", conflicts_with=[AdminSettingsSuite])
class CheckoutUiSuite:

    @test(parallelized=False)    # runs alone
    def resets_cart(self): ...

@Suite(uses="payments")            # sandbox API calls
class PaymentsApiSuite: ...

@Suite(throttling=0)                  # API checks: full speed
class PricingApiSuite: ...

Exception-aware Retries & Retry Policies

Reliability

Retry the failures worth retrying, as often and as patiently as each one deserves, and let real bugs fail on the first run.

  • retry=3 with retry_on=[ConnectionError, TimeoutError] / no_retry_on=[AssertionError]: network noise is retried, a wrong answer fails at once. Retries count per parameter combination.
  • A RetryPolicy matches by exception class or message (When(message="503")), and each condition has its own attempts, delay, backoff and jitter.
  • circuit=N stops retrying for the rest of the run once N tests with the policy still fail, max_time stops retrying a test after that many seconds, and reset="class" runs class setup again before the retry.
  • A test that passed only on a retry is flagged as flaky, and --fail-on-flaky fails the run: see below.
Explore retries → Retry policies docs → 
from test_junkie.retry import RetryPolicy, When

class Flaky(RetryPolicy):
    when = [When(ConnectionError, attempts=3, delay=1),
            When(message="503", attempts=2,
                 delay=20, backoff=2)]
    circuit = 5         # 5 tests still failing: stop

@Suite(feature="Payments", owner="payments-team")
class PaymentsApiSuite:

    @test(parameters=["visa", "amex"], retry=Flaky)
    def charges_card(self, parameter): ...

    @test(retry=3, retry_on=[ConnectionError],
          no_retry_on=[AssertionError])
    def refunds_card(self): ...

Flaky Tests, Flagged for CI

CI

A pass on the second try isn't the same as a pass. Test Junkie notices it whatever retried the test, and hands it to CI in the format CI already reads. Here is the Flaky policy above, run against the payments sandbox when the first visa charge hit a dropped connection.

  • test.is_flaky() / get_flaky() and "flaky" in the JSON report name every parameter combination that passed only on a retry.
  • tj run --flag-flaky lists them after the summary; --fail-on-flaky also fails the run (exit code 1).
  • The XML report records the failed runs the Maven Surefire way (<flakyFailure> / <flakyError>), so CI tools that read that layout, such as Jenkins with its Flaky Test Handler plugin, can mark the test flaky instead of green.
Flaky tests docs → How it works → 
$ tj run -s tests --fail-on-flaky --xml-report reports/
...
Flaky 1 ────────────────────────────────────────

  PaymentsApiSuite.charges_card [visa]  passed on run 2
      ConnectionError: Connection reset by peer: sandbox.payments.example

 FAILED   1 flaky test (--fail-on-flaky)  exit code 1

# reports/report.xml
<testcase name="charges_card[visa]" classname="PaymentsApiSuite"
          status="success" time="0.000">
  <flakyError type="ConnectionError"
      message="Connection reset by peer: sandbox.payments.example">
    <stackTrace>Traceback (most recent call last): ...</stackTrace>
  </flakyError>
</testcase>
# CI can mark it flaky, not green

Structured Metadata & CI Targeting

Observability

First-class owner, component, and tags on every test for CI filtering; priority= controls execution order — lower number runs first.

  • tj run --tags-any smoke in CI, full suite locally — no test selection scripts to maintain as the repo grows.
  • Meta.update() sets values from inside a running test, Meta.link() adds clickable links (tickets, admin pages), Meta.attach() adds files (responses, screenshots, logs) and Meta.append() builds lists. All kept per attempt and shown in listeners and the HTML, JSON and XML reports; small files are embedded in the HTML report.
Explore metadata → Meta docs → 
@test(
    owner="checkout-team",
    component="payment",
    priority=1,
    tags=["smoke", "regression"],
)
def test_checkout(self):
    order = api.checkout(cart_id="c-118")
    Meta.update(order_id=order.id)
    Meta.link("TJ-42",
              "https://jira.example.com/browse/TJ-42")
    Meta.attach("response.json", order.raw)
    assert order.status == 201

# tj run --tags-any smoke

Conflict Rules

Name what must never overlap (suites, tests or a mix) and everything else stays parallel.

conflicts_with=[...]tj audit conflicts
How it works →

Clear Outcomes

A broken login in setup is 1 error and 40 ignored tests, not 41 failures.

1 ERROR40 IGNORED
How it works →

Audit & CI Gate

See tests by owner, feature or tag without running them, and fail CI when one has no owner.

tj audit--fail-on-gaps
How it works →

Rerun From a Report

Rerun only what didn't pass, down to the parameter, from last night's report on any machine.

tj run --rerun report.json
How it works →

Σevery capability, one package

Next: all features

See the code behind every capability

Working Test Junkie code for every capability and the settings that control it.

See all features →

As simple or as complex as your use case

From a minimal test file to a full production setup, one API — add what you need, leave out what you don't.

Define tests
01

Getting started

Minimal, fully functional login.py 
from test_junkie.runner import Runner
from test_junkie.decorators import Suite, test

@Suite()
class LoginSuite:

    @test()
    def test_valid_login(self):
        assert login("admin", "pass")

    @test()
    def test_invalid_password(self):
        assert login("admin", "wrong") is False

Runner([LoginSuite]).run()

$ python login.py
LoginSuite  login.py  [||||||||||||||||||||||||]  2/2  0.00s
 PASSED   exit code 0
02

Full power

Parameters, parallel, retries, metadata, listeners checkout.py 
@Suite(
    parameters=[{"env": "staging"}, {"env": "prod"}],
    parallelized=True, listener=AlertListener,
)
class CheckoutSuite:
    @beforeClass()
    def setup(self, suite_parameter):
        self.driver = new_driver(suite_parameter["env"])

    @test(
        owner="checkout-team", component="payment",
        priority=1, tags=["smoke"], retry=3,
        retry_on=[ConnectionError],
        no_retry_on=[AssertionError],
        parameters=[{"browser": "chrome"},
                    {"browser": "safari"}],
        parallelized=True,
    )
    def test_checkout(self, parameter, suite_parameter): ...
Run it
CLI The same suites from the command line: tj run targets, filters and parallelizes 
# Run all tests in a directory
$ tj run -s tests/

# Filter by owner and component
$ tj run -s tests/ --owners checkout-team --components payment

# Parallel: 10 test threads, 2 suite threads
$ tj run -s tests/ -T 10 -S 2
Test Junkie 0.9a8 · Python 3.13.2 · Windows 11
  tests    tests/ · 3 suites, 18 tests · found in 0.01s
  mode     parallel · 2 suite threads · 10 test threads

LoginSuite     tests\login.py     [||||||||||||||||||||||||]    6/6  0.75s
CheckoutSuite  tests\checkout.py  [||||||||||||||||||||||||]    8/8  1.40s
SearchSuite    tests\search.py    [||||||||||||||||||||||||]    4/4  0.70s

Total                             [||||||||||||||||||||||||]  18/18  100%  1.5s

Summary ──────────────────────────────────────────────────────────

  Suite            Pass  Fail  Error  Ignore  Skip  Cancel     Time
  CheckoutSuite       8     ·      ·       ·     ·       ·    1.40s
  LoginSuite          6     ·      ·       ·     ·       ·    0.75s
  SearchSuite         4     ·      ·       ·     ·       ·    0.70s
  ─────────────────────────────────────────────────────────────────
  Total              18     ·      ·       ·     ·       ·    1.46s
  18 tests         100%

──────────────────────────────────────────────────────────────────

 PASSED   exit code 0