Exception-aware retries

A blanket retry hides real bugs behind flaky infrastructure. Test Junkie's retry_on and no_retry_on let you scope retries to exact exception types — so a connection timeout retries quietly, but a genuine assertion failure about response content fails on the first try, the way it should.

The problem with plain retry=

retry=2 on its own retries on any exception — assertion failures included. A real contract break might pass by luck on the second attempt, or just burn time failing three times instead of one:

test_api.pypython

@test(retry=2)  # retries on anything - including real bugs
def test_response_shape(self):
    data = api.get("/users/1")
    # if the API changed and "name" was renamed, this AssertionError
    # gets retried twice before failing - masking the real problem
    assert "name" in data

The allowlist: retry_on=

Pass retry_on=[...] to only retry when one of those exception types is raised. Everything else fails immediately:

test_api.pypython

import requests
from test_junkie.decorators import Suite, test

@Suite()
class UserApiSuite:

    @test(
        retry=2,
        retry_on=[
            requests.exceptions.ConnectionError,
            requests.exceptions.Timeout,
        ],
    )
    def test_response_shape(self):
        data = api.get("/users/1")
        # AssertionError is not in retry_on
        # a wrong shape fails once, honestly
        assert "name" in data

    @test(
        retry=2,
        retry_on=[requests.exceptions.ConnectionError],
    )
    def test_create_user(self):
        result = api.post("/users", {"name": "Alice"})
        assert result["id"] is not None

The blocklist: no_retry_on=

Sometimes it's easier to name the exceptions you want to exclude from retries. no_retry_on flips the logic to a blocklist — retry on anything except the named types:

test_api.pypython

@test(
    retry=2,
    no_retry_on=[AssertionError, ValueError],
)
def test_status_endpoint(self):
    # retried on connection errors, timeouts, DNS failures, etc -
    # just never on assertion or value errors
    response = requests.get("/status", timeout=5)
    assert response.status_code == 200

retry_on= and no_retry_on= are mutually exclusive

Use one or the other per test, not both. Use retry_on when the retryable exceptions are the minority (a short allowlist is simpler). Use no_retry_on when the non-retryable exceptions are the minority (a short blocklist is simpler).

Retries with parametrization

When a test has both parameters= and retry=, retries apply per variant. If the safari variant gets a timeout, only it retries — the chrome and firefox variants are unaffected:

checkout_suite.pypython

@test(
    retry=2,
    retry_on=[ConnectionError, TimeoutError],
    parameters=[
        {"browser": "chrome"},
        {"browser": "firefox"},
        {"browser": "safari"},
    ],
)
def test_checkout(self, parameter):
    browser = parameter["browser"]
    ...