command line interface

@suite

from test_junkie.decorators import Suite

@Suite()
class ExampleTestSuite:
    ...

@beforeclass

from test_junkie.decorators import beforeClass

...
@beforeClass()
def before_class():
    ...

This decorator does not support any special properties. However, the function which is decorated, can benefit from class level parametrisation.

@beforetest

from test_junkie.decorators import beforeTest

...
@beforeTest()
def before_test():
...

This decorator does not support any special properties. However, the function which is decorated, can benefit from class level parametrisation.

This decorator can be configured to be skipped by individual tests in the suite. See @test() for a usage examples.

@test

from test_junkie.decorators import test

...
@test()
def example_test():
...

This decorator supports many different properties which allow you to optimize execution of your tests.

@aftertest

from test_junkie.decorators import afterTest

...
@afterTest()
def after_test():
...

This decorator does not support any special properties. However, the function which is decorated, can benefit from class level parametrisation.

This decorator can be configured to be skipped by individual tests in the suite. See @test() for a usage examples.

@afterclass

from test_junkie.decorators import afterClass

...
@afterClass()
def after_class():
...

This decorator does not support any special properties. However, the function which is decorated, can benefit from class level parametrisation.

@grouprules

This decorator allows to define rules for a particular set of test suites and it must be used from within the subclass of the Rules.

Inside a function that is decorated with @GroupRules() decorator, you can import test suites and define groups which then you can use to create @beforeGroup() & @afterGroup() rules. You can define as many rules as you like. For a more complete example, see the Rules section.

@beforegroup

@GroupRules()
def group_rules(self):

    from ... import ExampleSuite1
    from ... import ExampleSuite2
    @beforeGroup([ExampleSuite1, ExampleSuite2])
    def before_group_a():
        # your code
        pass

    from ... import ExampleSuite3
    from ... import ExampleSuite4
    @beforeGroup([ExampleSuite3, ExampleSuite4])
    def before_group_b():
        # your code
        pass

@aftergroup

@GroupRules()
def group_rules(self):

    from ... import ExampleSuite1
    from ... import ExampleSuite2
    @afterGroup([ExampleSuite1, ExampleSuite2])
    def after_group_a():
        # your code
        pass

    from ... import ExampleSuite3
    from ... import ExampleSuite4
    @afterGroup([ExampleSuite3, ExampleSuite4])
    def after_group_b():
        # your code
        pass

skip

from test_junkie.decorators import Suite, test

def custom_skip_function(meta):
    # meta is optional
    # do your processing
    return True  # or False but must return a boolean

@Suite(skip=True)
or
@Suite(skip=custom_skip_function)
class ExampleSuite:

    @test(skip=True)
    or
    @test(skip=custom_skip_function)
    def example_test(self):
        pass

skip special functions

...
@test(skip_before_test=True,
      skip_after_test=True,
      skip_before_test_rule=True,
      skip_after_test_rule=True)
def example_test(self):
    pass
...

retry

from test_junkie.decorators import Suite, test

@Suite(retry=2)  # default is 1
class ExampleSuite:

    @test(retry=2)  # default is 1
    def example_test(self):
        # this test will run 4 times in total
        # suite_retry x test_retry = total test retries
        assert True is False

...
@test(retry=2, retry_on=[AssertionError])
def example_test(self):
    # will be retried
    assert True is False
...

...
@test(retry=2, no_retry_on=[AssertionError])
def example_test(self):
    # won't be retried
    assert True is False
...

parametrization

Test Junkie has the best parametrization engine of any test runner. Parametrization is allowed at suite and test level.

@Suite(parameters=[1, 2])
class ExampleSuite:

    @beforeClass()
    def before_class(self, suite_parameter):
        print(suite_parameter)

    @beforeTest()
    def before_test(self, suite_parameter):
        print(suite_parameter)

@Suite(parameters=[1, 2])
class ExampleSuite:

    @test(parameters=[10, 20])
    def example_test(self, parameter, suite_parameter):
        print(parameter)
        print(suite_parameter)

parallel execution

Test Junkie supports multi-threading out of the box and you can allocate N number of threads to be used as a threadpool for your suites and/or tests. Thread allocation is done through the run() method of the Runner instance.

enable parallelized parameters

...
@test(parallelized_parameters=True, parameters=[...])
def example_test(parameter):
    ...

disable parallelized mode

@Suite(parallelized=False)
class ExampleSuite:
    @test(parallelized=False)
    def example_test():
    ...

avoiding suite/test conflicts

@Suite(pr=[TestSuiteA])
class TestSuiteB:
    @test(pr=[TestSuiteC.example_test])
    def example_test():
    ...

rules

Rules allow for reusable logic that can be applied to @Suite() and/or group of suites, much like the Listeners.

In order to create rules, you need to create a new class and inherit from Rules. After that, overwrite the desirable functions and assign the newly created class to a suite of your choice using the @Suite() decorator.

from test_junkie.decorators import GroupRules, afterGroup
from test_junkie.rules import Rules


class TestRules(Rules):

    def __init__(self, **kwargs):

        Rules.__init__(self, **kwargs)

    def before_class(self):
        """
        @beforeClass() event handling applies
        """
        # your code
        pass

    def before_test(self, **kwargs):
        """
        @beforeTest() event handling applies
        through kwargs you can access a copy of the TestObject in the current context
        kwargs.get("test")
        """
        # your code
        pass

    def after_test(self, **kwargs):
        """
        @afterTest() event handling applies
        through kwargs you can access a copy of the TestObject in the current context
        kwargs.get("test")
        """
        # your code
        pass

    def after_class(self):
        """
        @afterClass() event handling applies
        """
        # your code
        pass

    @GroupRules()
    def group_rules(self):

        from my.suites.examples.ExampleSuite1 import ExampleSuite1
        from my.suites.examples.ExampleSuite2 import ExampleSuite2
        @beforeGroup([ExampleSuite1, ExampleSuite2])
        def before_group_a():
            # your code
            pass
        @afterGroup([ExampleSuite1, ExampleSuite2])
        def after_group_a():
            # your code
            pass

        from my.suites.examples.ExampleSuite3 import ExampleSuite3
        from my.suites.examples.ExampleSuite4 import ExampleSuite4
        @beforeGroup([ExampleSuite1, ExampleSuite2])
        def before_group_b():
            # your code
            pass
        @afterGroup([ExampleSuite3, ExampleSuite4])
        def after_group_b():
            # your code
            pass
                

priority

Test Junkie allows to influence execution priority for suites & tests. Note that I said influence. Priority cannot be guaranteed when running in threaded mode and having either parallelized properties set to False in any of your suites or parallel restriction configured. Test Junkie will always try to find a suite or a test to run in its current scope. So if it comes to a suite/test that is currently restricted and cannot be executed, it will move on without waiting for condition to clear. It was implemented this way on purpose to increase performance.

Priority is integer based and it starts at 1, which is the highest priority index, and goes up from there. To set priority use the priority property of the @Suite() or the @test() decorator.

1. Suites & Tests without priority and disabled parallelized execution get de-prioritized the most
2. Suites & Tests without priority and enabled parallelized execution get prioritized higher
3. Suites & Tests with priority get prioritised according to the priority that was set. However, they are always prioritised above those that do not have any priority

listeners

Test Junkie allows to define listeners which allow to execute code on a specific suite/test event. Defining listeners is optional. This feature is typically useful when building large frameworks as it allows for seamless integration for reporting, post processing of errors, calculation of test metrics, alerts, artifact collection etc.

from test_junkie.listener import Listener

class MyTestListener(Listener):

    def __init__(self, **kwargs):

        Listener.__init__(self, **kwargs)
    ...

on in progress

...
def on_in_progress(self, **kwargs):
    # your code
    print(kwargs)
    ...

on success

...
def on_success(self, **kwargs):
    # your code
    print(kwargs)
    ...

on failure

...
def on_failure(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on error

...
def on_error(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on ignore

...
def on_ignore(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on cancel

...
def on_cancel(self, **kwargs):
    # your code
    print(kwargs)
    ...

on skip

...
def on_skip(self, **kwargs):
    # your code
    print(kwargs)
    ...

on complete

...
def on_complete(self, **kwargs):
    # your code
    print(kwargs)
    ...

on class in progress

...
def on_class_in_progress(self, **kwargs):
    # your code
    print(kwargs)
    ...

on before class failure

...
def on_before_class_failure(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on before class error

...
def on_before_class_error(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on after class failure

...
def on_after_class_failure(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on after class error

...
def on_after_class_error(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on class skip

...
def on_class_skip(self, **kwargs):
    # your code
    print(kwargs)
    ...

on class cancel

...
def on_class_cancel(self, **kwargs):
    # your code
    print(kwargs)
    ...

on class ignore

...
def on_class_ignore(self, **kwargs):
    # exception & traceback available
    # your code
    print(kwargs)
    ...

on class complete

...
def on_class_complete(self, **kwargs):
    # your code
    print(kwargs)
    ...

on before group failure

...
def on_before_group_failure(self, **kwargs):
    # your code
    print(kwargs)
    ...

on before group error

...
def on_before_group_error(self, **kwargs):
    # your code
    print(kwargs)
    ...

on after group failure

...
def on_after_group_failure(self, **kwargs):
    # your code
    print(kwargs)
    ...

on after group error

...
def on_after_group_error(self, **kwargs):
    # your code
    print(kwargs)
    ...

meta

Meta has absolutely no effect on how Test Junkie runs the tests and can be of any data type. Meta is supported by the @Suite() & @test() decorators

Why is this useful? You can use meta to set properties such as:

1. Test name, suite name, description, expected results and anything else that can be useful in reports
2. Test case IDs - if you have a test management system, leverage it to link test scripts directly to the test cases and further integrations can be implemented from there
3. Bug ticket IDs - if you have a bug tracking system (like Jira), leverage it to link your test cases with issues that are already known and allow you to process failures in a different manner (as a known failure for instance).

from test_junkie.decorators import Suite, test
from test_junkie.meta import meta


@Suite(listener=MyTestListener, meta=meta(name="some value", id=123))
class ExampleSuite:

    @test(meta=meta(name="some value", expected="some value",
                    known_bugs=[111, 222, 333], id=321))
    def example_test(self):
        pass

accessing meta through listeners

Meta data that was set in the code above can be accessed through the Listeners like so:

from test_junkie.listener import Listener


class MyTestListener(Listener):

    def __init__(self, **kwargs):

        Listener.__init__(self, **kwargs)

    def on_success(self, **kwargs):

        class_meta = kwargs.get("properties").get("class_meta")
        test_meta = kwargs.get("properties").get("test_meta")

        print("Suite name: {name}".format(name=class_meta["name"]))
        print("Suite ID: {id}".format(id=class_meta["id"]))
        print("Test name: {name}".format(name=test_meta["name"]))
        print("Test ID: {id}".format(id=test_meta["id"]))
        print("Expected result: {expected}".format(expected=test_meta["expected"]))
        print("Known bugs: {bugs}".format(bugs=test_meta["known_bugs"]))

updating meta

Meta data can be updated and/or added from within @test() using Meta.update(). Keep in mind, only @test() level meta can be updated, @Suite() level meta should never change. Meta.update() takes 3 positional arguments, those arguments are required in order to locate correct TestObject:

1. self - class instance of the current test
2. parameter - if test is parameterized with tests level parameters. If test does not have test level parameters, do not pass anything.
3. suite_parameter - if test is parameterized with suite level parameters. If test does not have suite level parameters, do not pass anything.

from test_junkie.meta import Meta
...
@test(parameters=[...])
def example_test(self, parameter, suite_parameter):
    # this particular test is running with test and suite level parameters, thus to update the meta...
    Meta.update(self, parameter=parameter, suite_parameter=suite_parameter,
                name="new value", expected="new value")
...
@test(parameters=[...])
def example_test(self, parameter):
    # this particular test is running only with test level parameters, thus to update the meta...
    Meta.update(self, parameter=parameter, name="new value", expected="new value")
...
@test()
def example_test(self):
    # this particular test has no parameters, thus to update the meta...
    Meta.update(self, name="new value", expected="new value")
...

assign tests

Have a large QA team that uses the same framework? Test Junkie allows to assign suites and tests to specific members of the team. If using HTML report, it'll allow to see performance metrics broken down by assignees. In addition, assignees can be accessed in the Listeners which can be of use for custom reporting.

Both the @Suite() & @test() decorators supports assignees. Using the @Suite() decorator to set the owner will set assignees for all of the tests in scope of that suite. To overwrite assignee of any particular test, set the owner using the @test() decorator.

categorize suites by features

Categorizing the suites by the features that those suites are testing, not only allows to see KPI metrics broken down by features if using the HTML Report but also allows to run feature specific regression tests.

Features can be defined using the @Suite() decorator.

categorize tests by components

Categorizing the tests by the components that those tests are covering, not only allows to see KPI metrics broken down by components if using the HTML Report but also allows to run component specific regression tests.

Components can be defined using the @test() decorator.

tags

Similar to the Features & Components but tags have one-to-many mapping concept. Meaning a single test can be covering multiple areas of the platform, including more then one component and maybe even more then one feature. Ideally tests should be as small as possible and test tiny small functionality but in the real world its often not the case, especially if we are talking about end to end tests. So in those cases where test covers multiple areas of the platform, you can use tags to tell Test Junkie and your peers what the test is covering at a glance.

Tags are supported only by the @test() decorator and it, also, allows to see KPI metrics broken down by tags if using the HTML Report and allows to run tag specific regression.

Using tag_config which the run() method of the Runner instance can take, allows to pick which tags to run tests for and which to skip, giving a lot of flexibility.

tag config

1. run_on_match_all - Will run test cases that match all of the tags in the list
2. run_on_match_any - Will run test cases that match at least one tag in the list
3. skip_on_match_all - Will skip test cases that match all of the tags in the list
4. skip_on_match_any - Will skip test cases that match at least one tag in the list

All of the configs can be used at the same time. However, this is the order that will be honored: skip_on_match_all -> skip_on_match_any -> run_on_match_all -> run_on_match_any which ever matches first will be executed or skipped. If something is skipped, it will trigger the on_skip event.

from test_junkie.runner import Runner

runner = Runner([ExampleSuite1, ExampleSuite2, ExampleSuite3])
runner.run(tag_config={"run_on_match_all": ["...", "..."],
                       "run_on_match_any": ["...", "..."],
                       "skip_on_match_all ": ["...", "..."],
                       "skip_on_match_any ": ["...", "..."]})

html report

Test Junkie is tracking a number of metrics during test execution and it can generate HTML reports based on those metrics. Here you can see a live demo.

In order to generate a report like that:

from test_junkie.runner import Runner
runner = Runner(suites=[...], html_report="/path/to/file/report.html", monitor_resources=True)
runner.run()

monitor_resources=True enables monitoring of the MEM & CPU usage during test execution which will be rendered in the HTML report as a trending graph.

xml report

Test Junkie can also produce, Jenkins friendly, basic XML reports.

from test_junkie.runner import Runner
runner = Runner(suites=[...], xml_report="/path/to/file/report.xml")
runner.run()

json report

JSON reports are used under the hood for all of the other reports produced by Test Junkie. You can use JSON reports to slice the data in the way that is meaningful to you.

JSON reports can be extracted from a number of objects, all of which will be accessible after the test have finished executing:

from test_junkie.runner import Runner
runner = Runner(suites=[...])
aggregator = runner.run()

raw metrics

You can access raw metrics data from SuiteObject and/or from TestObject

suite_objects = runner.get_executed_suites()
for suite in suite_objects:
    test_objects = suite.get_test_objects()
    print(suite.metrics.get_metrics())
    for test in test_objects:
        print(test.metrics.get_metrics())

reports

You can, also, access reports that are created by Test Junkie in its JSON form via the Aggregator object.

print(aggregator.get_report_by_tags())
print(aggregator.get_report_by_features())
print(aggregator.get_basic_report())
print(aggregator.get_report_by_owner())

advanced test suite example

Following snippet show how to leverage decorator options in order to optimize the execution of your tests.

All of the suites are shown as if they are in one file, this is just for demonstration purposes, in reality you should have them all in separate files.

                        
from test_junkie.decorators import Suite, test, afterTest, beforeTest, beforeClass, afterClass
from test_junkie.meta import meta, Meta


@Suite(parameters=[{"login": "mike@example.com", "password": "example", "admin": True},
                   {"login": "sally@example.com", "password": "example", "admin": False}])
class LoginSuite:

    @beforeClass()
    def before_class(self, suite_parameter):  # yes, we just parameterized this function, seen that anywhere else?
        # Lets assume we have some code here to login with
        # username . . . suite_parameter["login"]
        # password . . . suite_parameter["password"]
        # This is our, hypothetical, pre-requirement before we run the tests
        # If this step were to fail, the tests would have been ignored
        pass

    @afterClass()
    def after_class(self):
        # Here, generally, we would have clean up logic.
        # For the sake of this example, lets assume we logout
        # from the account that we logged into during @beforeClass()
        # no `suite_parameter` in method signature,
        # logout process would likely be the same irrespective of the account
        pass

    @test(parameters=["page_url_1", "page_url_2", "page_url_3"])
    def validate_user_login_in_header(self, parameter, suite_parameter):
        # Lets assume that in this test case we are going to be validating
        # the header. We need to make sure that email that user logged in with
        # is displayed on every page so we will make this test parameterized.

        # By doing so we will know exactly which pages pass/fail without
        # writing any extra logic in the test itself to log all the failures
        # and complete testing all the pages which would be required if you
        # were to use a loop inside the test case for instance.

        # Now we would use something like Webdriver to open the parameter in order to land on the page
        # and assert that suite_parameter["username"] in the expected place
        pass

    @test(parameters=["admin_page_url_1", "admin_page_url_2"])
    def validate_access_rights(self, parameter, suite_parameter):
        # Similar to the above test case, but instead we are validating
        # access right privileges for different user groups.
        # Using same principal with the parameterized test approach.

        # Now we would also use Webdriver to open the parameter in order to land on the page
        # and assert that the page is accessible if suite_parameter["admin"] is True


@Suite(pr=[LoginSuite],
       parameters=[{"login": "mike@example.com", "password": "example", "user_id": 1},
                   {"login": "sally@example.com", "password": "example", "user_id": 2}])
class EditAccountCredentialsSuite:
    """
    It is risky to run this suite with the LoginSuite above because if
    the suites happen to run in parallel and credentials get updated
    it can cause the LoginSuite to fail during the login process.

    Therefore, we are going to restrict this suite using the `pr` property, this will insure that
    LoginSuite and EditAccountCredentialsSuite will never run in parallel thus removing any risk
    when you run Test Junkie in multi-threaded mode.
    """

    @test(priority=1, retry=2)  # this test, in case of failure, will be retried twice
    def reset_password(self, suite_parameter):  # this test is now parameterised with parameters from the suite
        # Lets assume in this test we will be resetting password of the
        # username . . . suite_parameter["login"]
        # and then validate that the hash value gets updated in the database

        # We will need to know login when submitting the passowrd reset request, thus we need to make sure that
        # we don't run this test in parallel with edit_login() test bellow.
        # We will use decorator properties to prioritize this test over anything else in this suite
        # which means it will get kicked off first and then we will disable parallelized mode for the
        # edit_login() test so it will have to wait for this test to finish.
        pass

    @test(parallelized=False, meta=meta(expected="Able to change account login"))
    def edit_login(self, suite_parameter):
        # Lets assume in this test we will be changing login for . . . suite_parameter["login"]
        # with the current number of tests and settings, this test will run last

        Meta.update(self, suite_parameter=suite_parameter, name="Edit Login: {}".format(suite_parameter["login"]))
        # Making this call, gives you option to update meta from within the test case
        # make sure, when making this call, you did not override suite_parameter with a different value
        # or update any of its content

    @afterClass()
    def after_class(self, suite_parameter):
        # Will reset everything back to default values for the
        # user . . . suite_parameter["user_id"]
        # and we know the original value based on suite_parameter["login"]
        # This will insure other suites that are using same credentials, wont be at risk
        pass


@Suite(listener=MyTestListener,  # This will assign a dedicated listener that you created
       retry=2,  # Suite will run up to 2 times but only for tests that did not pass
       owner="Chuck Norris",  # defined the owner of this suite, has effects on the reporting
       feature="Analytics",  # defines a feature that is being tested by the tests in this suite,
                             # has effects on the reporting and can be used by the Runner
                             # to run regression only for this feature
       meta=meta(name="Example",  # sets meta, most usable for custom reporting, accessible in MyTestListener
                 known_failures_ticket_ids=[1, 2, 3]))  # can use to reference bug tickets for instance in your reporting
class ExampleTestSuite:

    @beforeTest()
    def before_test(self):
        pass

    @afterTest()
    def after_test(self):
        pass

    @test(component="Datatable",  # defines the component that this test is validating,
                                  # has effects on the reporting and can be used by the Runner
                                  # to run regression only for this component
          tags=["table", "critical", "ui"],  # defines tags that this test is validating,
                                             # has effects on the reporting and can be used by the Runner
                                             # to run regression only for specific tags
          )
    def something_to_test1(self, parameter):
        pass

    @test(skip_before_test=True,  # if you don't want to run before_test for s specific test in the suite, no problem
          skip_after_test=True)  # also no problem, you are welcome!
    def something_to_test2(self):
        pass

running tests

This section shows examples on how to programmatically run tests with different configurations.

running in threaded mode

from test_junkie.runner import Runner
from ... import ExampleTestSuite1
from ... import ExampleTestSuite2
from ... import ExampleTestSuite3

runner = Runner([ExampleTestSuite1, ExampleTestSuite2, ExampleTestSuite3])
runner.run(suite_multithreading_limit=5, test_multithreading_limit=5)

running regression for a feature

Out of the suites used to initiate the runner instance, Test Junkie will run those that match the features.

runner = Runner(suites=[...])
runner.run(features=["Login"])

running regression for a component

Out of the suites used to initiate the runner instance, Test Junkie will run those that match the components.

runner = Runner(suites=[...])
runner.run(components=["OAuth"])

running regression for tags

Out of the suites used to initiate the runner instance, Test Junkie will run those that match the configuration of the tags.

runner = Runner(suites=[...])
runner.run(tag_config={"run_on_match_all": ["component_a", "critical"],
                       "skip_on_match_any": ["trivial", "known_failure"]})

running tests assigned to a specific person

Out of the suites used to initiate the runner instance, Test Junkie will run those that match the assignees.

runner = Runner(suites=[...])
runner.run(owners=["John Doe", "Jane Doe"])

running specific tests

Out of the suites used to initiate the runner instance, Test Junkie will run specific tests, all you have to do is specify the test object to run.

runner = Runner(suites=[...])
runner.run(tests=[ExampleTestSuite1.example_test_1, ExampleTestSuite1.example_test_2])

runner = Runner(suites=[...])
runner.run(tests=["example_test_1", "example_test_2"])

canceling tests

If you are integrating Test Junkie into a bigger framework, its possible that you would like to programmatically stop test execution. Good news that Test Junkie allows, gracefully, to do just that. If you call cancel() on the Runner Object, the Runner will start canceling suites and tests, which will trigger respective event listeners:

1. on_cancel
2. on_class_cancel

from test_junkie.runner import Runner
from ... import ExampleTestSuite1
from ... import ExampleTestSuite2
from ... import ExampleTestSuite3

runner = Runner([ExampleTestSuite1, ExampleTestSuite2, ExampleTestSuite3])
runner.run(suite_multithreading_limit=5, test_multithreading_limit=5)
runner.cancel()

runner

Runner object is used to run the tests. Runner can be initialized with a number of different properties:

1. suites - required parameter, takes a list of class objects decorated with @Suite(). This is how to tell Test Junkie which suites to care about.
2. monitor_resources - optional parameter, takes a bool. This turns on CPU & MEM monitoring. See HTML Report.
3. html_report - optional parameter, takes a str. This enables HTML Report generation.
4. xml_report - optional parameter, takes a str. This enables XML Report generation.
5. config - optional parameter, takes a path to file as a str. Config can be created using CLI and saved to a location of your choice. Explicitly passed in args to the Runner & Runner.run() will take precedence over all other settings, otherwise settings in the config will be used.

exposed methods

Runner exposes 3 methods:

1. run() - This method is special. Not only, as the name suggests, it initiates the actual test cycle but it, also, allows to define more configurations for running your tests. All of which are optional:
   1. features - takes a list of features. aka: runner.run(features=["Login"]).
   2. components - takes a list of components. aka: runner.run(components=["2FactorAuth"]).
   3. owners - takes a list of owners. aka: runner.run(owners=["John Cena"]).
   4. tag_config - takes a dict such as runner.run(tag_config={"run_on_match_all": ["pre_deploy", "critical"]}). See Tags for more.
   5. tests - takes a list of @test() objects or a list of strings. example:
      • - runner.run(tests=[LoginSuite.positive_login, LoginSuite.negative_login])
      • - runner.run(tests=["positive_login", "negative_login"])
   6. suite_multithreading_limit - takes an int which enables parallel execution for test suites.
   7. test_multithreading_limit - takes an int which enables parallel execution for tests.
   8. quiet - boolean, will silence all output of the tests.
2. cancel() - read more about canceling tests here.
3. get_executed_suites() - This will return a list of test_junkie.objects.SuiteObjects. SuiteObject can be used to analyze anything from test results to performance of tests and much much more.

limiter

Limiter allows to control throttling and truncation mechanism for Test Junkie. Note that Limiter has a "master switch". If Limiter.ACTIVE is set to False it will disable all of the limits including the default ones.

throttling

There are cases when you don't want to kick off suites and/or tests back to back when running tests in parallel. Test Junkie gives you the ability to throttle down how frequently a new @test() and/or @Suite() can get kicked off.

from test_junkie.objects import Limiter

# Suites will be kicked off on a 3 second interval
Limiter.SUITE_THROTTLING = 3

# Tests will be kicked off on a 1 second interval
Limiter.TEST_THROTTLING = 1

truncation

Test Junkie will truncate long exception messages to keep tracebacks to a sensible size which by default is 3000 characters. You can control the threshold limit. Note that this will also apply for the tracebacks/exceptions you see in the HTML report.

from test_junkie.objects import Limiter

# Will increase char limit to 10000 for all exception messages
Limiter.EXCEPTION_MESSAGE_LIMIT = 10000

# Will increase char limit to 10000 for all tracebacks
Limiter.TRACEBACK_LIMIT = 10000

suiteobject

coming soon

testobject

coming soon