autoused, session-scoped fake fs in django project

[konnerthg]

Describe the bug

Tests crash in a counterintuitive way when attempting to autouse a fixture which uses fs_session
Is this a limitation?

command_name = 'migrate', args = ()
options = {'database': 'default', 'interactive': False, 'run_syncdb': True, 'verbosity': 0}

def call_command(command_name, *args, **options):
    """
    Call the given command, with the given options and args/kwargs.

    This is the primary API you should use for calling specific commands.

    `command_name` may be a string or a command object. Using a string is
    preferred unless the command object is required for further processing or
    testing.

    Some examples:
        call_command('migrate')
        call_command('shell', plain=True)
        call_command('sqlmigrate', 'myapp')

        from django.core.management.commands import flush
        cmd = flush.Command()
        call_command(cmd, verbosity=0, interactive=False)
        # Do something with cmd ...
    """
    if isinstance(command_name, BaseCommand):
        # Command object passed in.
        command = command_name
        command_name = command.__class__.__module__.split(".")[-1]
    else:
        # Load the command object by name.
        try:
            app_name = get_commands()[command_name]
        except KeyError:

          raise CommandError("Unknown command: %r" % command_name)

E django.core.management.base.CommandError: Unknown command: 'migrate'

see full stack trace in the attachment.

trace.txt

How To Reproduce

The following minimal test case will probably produce the above result in any django project.

from django.contrib.auth.models import User
import pytest


@pytest.fixture(
    # remove this line and the test passes
    autouse=True
)
def auto_use_fake_fs(fs_session):
    yield


def test_fake_fs(db, auto_use_fake_fs):
    User.objects.create()

Your environment

Linux-6.14.0-37-generic-x86_64-with-glibc2.39
Python 3.12.3 (main, Nov 6 2025, 13:44:16) [GCC 13.3.0]
pyfakefs 6.0.0
pytest 9.0.2
django==6.0
pytest-django==4.11.1
pyfakefs==6.0.0

[mrbean-bremen]

Thanks - the error message is indeed counter-intuitive, most likely the error is in reality related to a mix-up of real and fake filesystems.
My guess is that this is an interaction with pytest-django, which has to use the real file-system, while it is patched.
Possible remedies could be (just out of my head):

• pause patching where appropriate (probably difficult, as this happens in the pytest-django fixture)
• map the django app with the database into the fake filesystem (depends on the implementation of the database commands - if they use a C interface, this won't work)
• use 'fs' in the relevant tests instead of fs_session (probably defeats the purpose)

I will have a closer look tonight to understand the actual problem.

[mrbean-bremen]

The problem is that django looks up the commands in the file system, but doesn't find them because it uses the fake fs.
The problem here is the order in which the fixtures are involved.
I you where to use the normal fs fixture instead of fs_session this should work. If there is no compelling reason why you need the session scoped version, I wouldn't use it, at it is always more prone to side effects.
A somewhat more obscure solution is something like this:

@pytest.fixture(autouse=True)
def auto_use_fake_fs(db, fs_session):  # note the added `db` fixture before `fs_session` to ensure the correct order
    yield

def test_fake_fs(db, fs):  # `fs` here is a reference to `fs_session`
    User.objects.create()

If none of these solutions work for you, we have to look further into this (specifically, how the pytest-django fixture works).
Mapping the django path into the fake file system could work, but with your setup the initialization order will get in the way:
First fs_session is invoked, then automatically the db fixture, and at that point (before we have a chance to map the file system into the fixture), the (fake) filesystem is already used.

You still may need to map part of your application and/or django into the fake filesystem, but I'm not sure what part. For reference:
to map the django package into the fake fs could look something like this:

def auto_use_fake_fs(fs_session):
    fs_session.add_real_directory(Path(django.__file__).parent)
    yield

I also remember having to map tzdata into the fake fs, though you may not need that.

[konnerthg]

Thank you for looking into this.

To give some background: in our use case, the fake fs has become quite large, affecting performance a lot. I thought, session-scoping it could help.

def auto_use_fake_fs(db, fs_session) works for this example indeed. as long as auto_use_fake_fs remains function scoped (because db is function scoped).

fs_session.add_real_directory(Path(django.__file__).parent) also works in the example I provided, but not in the real test suite. I don't yet understand why. Maybe it's related to your suggestion:

map the django app with the database into the fake filesystem (depends on the implementation of the database commands - if they use a C interface, this won't work)

I considered overwriting the pytest-django fixture, but it seems complicated. It feels like this would lead to other maintanance problems in the future.

Unless I am overlooking something here, I will consider this a limitation. And my approach unsuitable.

Is a clearer error message possible here? I presume this is a bit of an edge case.

Thank you

[mrbean-bremen]

Is a clearer error message possible here? I presume this is a bit of an edge case.

The problem is that the error comes from django, not from pyfakefs. pyfakefs just checks for some files in the (fake) filesystem, and (correctly) reports that the files are not there, and django then raises because it could not find the command (which it should have read from the filesystem). Mapping the django package into the fake fs would have helped here, but I don't see a place there this could be done at the moment. I may have another look at this later, but this is a problem/limitation related to another pytest plugin working on the filesystem.

Generally, django can be problematic because it does some dynamic stuff by reading code from the fs using open (instead of the usual open_code, which is used internally by the Python import system, and will usually not be patched by pyfakefs). For this reason, mapping your own django app may also be needed (as also described in the mentioned documentation. I had to do something similar at the time I worked with django and pyfakefs (a few years ago).

So, get django working with pyfakefs can be a bit of a challenge, unfortunately, though it should be possible.
As an aside: I myself actually started using pyfakefs (before I was a contributor/maintainer) specifically for django apps, where my home-grown simple fs abstraction fell short of the dynamic behavior of django.

tl;dr - I will think about it - maybe I will come up with a better idea how to handle this, but no promises here.

[mrbean-bremen]

A couple more remarks:

def auto_use_fake_fs(db, fs_session) works for this example indeed. as long as auto_use_fake_fs remains function scoped (because db is function scoped).

Actually, I wrote that wrong, it should have been:

def auto_use_fake_fs(django_db_setup, fs_session):
    yield

e.g. make sure the session-scoped db fixture is used before the session-scoped fs fixture.

fs_session.add_real_directory(Path(django.file).parent) also works in the example I provided, but not in the real test suite.

In the example, the memory-based SQLite-db is used, which django uses per default for the tests. If you use your real database (something like postgreSQL) in your test instead, this would indeed not work, because in this case the db access would most likely be via low-level C functions.

It is difficult to see what your actual problems are without seing the project (or an example project that has the same problems). Your current example is easy to reproduce and easy to fix, but I guess the real problems lie elsewhere.

[mrbean-bremen]

I will convert this into a discussion issue for reference. With the current information, I don't see a way to improve the current behavior besides the mentioned ideas, but I would be happy do add more functionality or documentation for this kind of issues, given some more information.

[konnerthg]

thank you

In the example, the memory-based SQLite-db is used, which django uses per default for the tests. If you use your real database (something like postgreSQL) in your test instead, this would indeed not work, because in this case the db access would most likely be via low-level C functions.

indeed with postgres this fix does not work. but i now understand why this limitation exists. thank you for clarification