Allow ordering on per-class or per-module basis, instead of just per-session

[Ferguzz]

If I mark a test first within a class, it should be possible to have that test run first in that class, not first overall (unless that class is the first to run).

Similarly, if I mark a test first within a module, it should be possible to have that test run first in that module, but not first overall (unless that module is the first to run).

Perhaps the best way to specify this would be a keyword argument like @pytest.mark.first(scope='class') or @pytest.mark.second(scope='session').

[krishn2014]

@ftobia can you provide me some pointers to start with for fixing this issue.

[jahan01]

has anyone found solution or workaround for this? I would be interested to know - tried implementing this feature on my own, but then test collection breaks when a function is outside of class.

[jahan01]

I ended up creating separate marker for class scoped ordering. For those who may need below is the snippet.

def sort_cls_items(items):
    grouped_items = {}

    for item in items:

        mark = item.get_closest_marker("order")

        if mark:
            priority = mark.args[0]
        else:
            priority = None

        grouped_items.setdefault(priority, []).append(item)

    sorted_items = []
    unordered_items = [grouped_items.pop(None, [])]

    start_list = sorted((i for i in grouped_items.items() if i[0] >= 0), key=operator.itemgetter(0))
    end_list = sorted((i for i in grouped_items.items() if i[0] < 0), key=operator.itemgetter(0))

    sorted_items.extend([i[1] for i in start_list])
    sorted_items.extend(unordered_items)
    sorted_items.extend([i[1] for i in end_list])

    result = [item for sublist in sorted_items for item in sublist]
    return result


# inspired from
# https://github.com/ftobia/pytest-ordering/blob/492697ee26633cc31d329c1ceaa468375ee8ee9c/pytest_ordering/__init__.py#L49
def pytest_collection_modifyitems(items: Item):
    """
    To change the order of pytest execution.
    Useful especially when inheriting from a base test class and want to change the execution order
    """

    module_items: Dict[str, Item] = {}
    sorted_items = []

    for item in items:
        module_items.setdefault(item.module.__name__, []).append(item)

    prev_cls = None
    cls_items: List[Item] = []
    for module, list_of_func in module_items.items():
        for func in list_of_func:
            if func.cls is None:
                sorted_items.append(func)
            else:
                if prev_cls != func.cls.__name__:
                    prev_cls = func.cls.__name__
                    sorted_items.extend(sort_cls_items(cls_items))
                    cls_items.clear()

                cls_items.append(func)

    sorted_items.extend(sort_cls_items(cls_items))

    result = [item for item in sorted_items]
    items[:] = result

e.g

# tests/test_play1.py
class BaseTest:
    def test_2(self):
        print("test_2")
        assert 1

    @pytest.mark.order(-10)
    def test_4(self):
        print("test_4")
        assert 1


class TestInheritance(BaseTest):
    @pytest.mark.order(1)
    def test_1(self):
        print("test_1")
        assert 1

    def test_3(self):
        print("test_3")
        assert 1

    @pytest.mark.order(-1)
    def test_5(self):
        print("test_5")
        assert 1

Test order:

tests/test_play1.py .test_1
.test_2
.test_3
.test_4
.test_5
..

PS: Use at your own discretion, not tested for all the edge cases. Also I think it may work only in python3.7+ where dict insertion order is preserved.

[mrbean-bremen]

FWIW, I just added a command line option to set the order scope in my fork.