Metadata and targeting

Every @test can carry structured metadata — owner, component, tags, priority. That metadata is more than documentation: pass any of those attributes back to the runner and it becomes a filter, letting you run exactly the subset you need without touching the test code.

Annotating tests

Attach metadata directly on the decorator:

checkout_suite.pypython

@Suite(owner="checkout-team")
class CheckoutSuite:

    @test(
        owner="checkout-team",
        component="payment",
        tags=["smoke", "regression"],
        priority=1,
    )
    def test_card_payment(self):
        ...

    @test(
        owner="checkout-team",
        component="cart",
        tags=["regression"],
        priority=2,
    )
    def test_add_to_cart(self):
        ...

owner= inherits from @Suite if not set on @test

If a test has no owner=, it inherits the suite's owner=. Set it on @Suite as a default and override only on the tests that have a different owner.

Execution priority

priority= controls the order tests (and suites) run in — it is not a filter. Tests with a lower number run before tests with a higher number. Tests with no priority set run after all prioritized tests.

The full execution order within a run is:

checkout_suite.pypython

@Suite()
class CheckoutSuite:

    @test(priority=1)   # runs first
    def test_login(self): ...

    @test(priority=2)   # runs second
    def test_add_to_cart(self): ...

    @test(parallel=True)  # no priority — runs after prioritized tests
    def test_search(self): ...

    @test()              # no priority, not parallel — runs last
    def test_logout(self): ...

Three execution tiers

1. Tests with priority= set — run first, ascending order (1 before 2 before 3).
2. Tests with no priority but parallelized=True — run next.
3. Tests with no priority and parallelized=False — run last.

@Suite also accepts priority= with the same rules, ordering suites relative to each other.

Targeting at runtime

Pass the same attributes to runner.run() to filter which tests execute. Tests that don't match are skipped, not removed:

run_ci.pypython

from test_junkie.runner import Runner

runner = Runner([CheckoutSuite, LoginSuite, SearchSuite])

# CI smoke pass: priority 1 smoke-tagged tests only
runner.run(
    components=["payment", "cart"],
    tag_config={"run_on_match_any": ["smoke"]},
)

# triage: run only what the checkout team owns
runner.run(owners=["checkout-team"])

# nightly: full regression, no filter
runner.run()

Tag filtering options

tag_config supports four modes — any combination can be set at once:

run.pypython

runner.run(
    tag_config={
        # run if ANY of these tags match
        "run_on_match_any": ["smoke", "critical"],

        # run only if ALL of these tags match
        "run_on_match_all": ["regression", "payment"],

        # skip if ANY of these tags match
        "skip_on_match_any": ["wip"],

        # skip only if ALL of these tags match
        "skip_on_match_all": ["slow", "flaky"],
    }
)

Runtime metadata: Meta.update()

Static metadata on the decorator is useful for targeting. For values only known at run time — build IDs, dynamic results, environment state — use Meta.update() inside the test.

Non-parametrized test

When the test has no parameters, call Meta.update(self, ...) with just the keyword values you want to attach:

checkout_suite.pypython

from test_junkie.meta import Meta

@Suite()
class CheckoutSuite:

    @test(owner="checkout-team", component="payment")
    def test_card_payment(self):
        order = checkout(card="4111111111111111")
        Meta.update(self, order_id=order["id"], amount=order["total"])
        assert order["status"] == "confirmed"

Parametrized test

When the test has parameters, you must also pass parameter= and/or suite_parameter= to Meta.update(). These tell it which variant's metadata record to write to — without them, the update targets the None entry, which doesn't exist for a parametrized test and the update is silently lost:

checkout_suite.pypython

@Suite(parameters=[
    {"env": "staging"},
    {"env": "prod"},
])
class CheckoutSuite:

    @test(
        owner="checkout-team",
        parameters=[{"card": "visa"}, {"card": "amex"}],
    )
    def test_card_payment(self, parameter, suite_parameter):
        order = checkout(
            card=parameter["card"],
            env=suite_parameter["env"],
        )
        # pass both parameter and suite_parameter so Meta.update
        # writes to the correct variant record (e.g. prod/amex)
        Meta.update(
            self,
            parameter=parameter,
            suite_parameter=suite_parameter,
            order_id=order["id"],
            amount=order["total"],
        )
        assert order["status"] == "confirmed"

Pass only the parameters your test uses

If the test only has suite-level parameters (no test-level parameter=), pass only suite_parameter=suite_parameter. If it only has test-level parameters, pass only parameter=parameter. Pass both when the test declares both.

The attached values appear in the test's metadata record, accessible via test_obj.metrics.get_metrics() after the run.