Retry-safe API testing

A blanket retry hides real bugs behind flaky infrastructure. Test Junkie's retry_on and no_retry_on let a test retry only on the exception types you name - so a genuine assertion failure about the response still fails on the first try, the way it should.

test_httpbin_contract.pypython

import requests
from test_junkie.decorators import Suite, test

@Suite()
class HttpBinContractSuite:

    # only a transient network error is retried - up to twice
    @test(retry=2, retry_on=[requests.exceptions.ConnectionError, requests.exceptions.Timeout])
    def test_status_endpoint_is_reachable(self):
        response = requests.get("https://httpbin.org/status/200", timeout=5)
        assert response.status_code == 200

    @test(retry=2, retry_on=[requests.exceptions.ConnectionError, requests.exceptions.Timeout])
    def test_response_json_has_expected_shape(self):
        response = requests.get("https://httpbin.org/get", timeout=5)
        data = response.json()
        # AssertionError isn't in retry_on, so a wrong shape fails
        # once, honestly, instead of being masked by a retry
        assert "url" in data

Why not just retry=2 on its own?

Plain retry=2 retries on any exception, assertion failures included - so a real contract break gets silently re-run and might pass by luck, or just burns time failing the same way twice. Naming the exception types in retry_on keeps retries scoped to actual infrastructure flakiness.

The inverse: retry on everything except one thing

Sometimes the exception you want to exclude is easier to name than the ones you want to catch. no_retry_on flips the list to a blocklist instead of an allowlist:

test_httpbin_contract.pypython

@test(retry=2, no_retry_on=[AssertionError])
def test_status_endpoint_is_reachable(self):
    # retried on connection errors, timeouts, DNS failures, etc -
    # just never on the assertion itself
    response = requests.get("https://httpbin.org/status/200", timeout=5)
    assert response.status_code == 200

This is one concept, applied to one real API - see the full retry reference for suite-level retries, and parameters for retrying only the parameter sets that actually failed.