Store pipx data in platform-specific user directories

[memchr]

• I have added an entry to docs/changelog.md

Summary of changes

Store pipx data in platform-specific user directories using platformdirs.

e.g. on Linux/Unix

• ~/.local/pipx/logs -> $XDG_STATE_HOME/pipx/logs
• ~/.local/pipx/shared -> $XDG_DATA_HOME/pipx/shared
• ~/.local/pipx/venvs -> $XDG_DATA_HOME/pipx/venvs
• ~/.local/pipx/.trash -> $XDG_DATA_HOME/pipx/trash
• ~/.local/pipx/.cache -> $XDG_CACHE_HOME/pipx

If ~/.local/pipx exists, revert to old behavior.

Fixes #722

Test plan

Tested by running

# platform linux - python 3.11.3, pytest 7.3.2, pluggy-1.0.0
python -m venv venv
. venv/bin/activate
nox -s tests-3.11

Results

nox > Running session tests-3.11
nox > Re-using existing virtual environment at .nox/tests-3-11.
nox > python -m pip install --upgrade pip
Requirement already satisfied: pip in ./.nox/tests-3-11/lib/python3.11/site-packages (23.1.2)
nox > python -m pip install wheel
nox > pip wheel --wheel-dir=/home/arthur/src/tmp/pipx/.nox/tests-3-11/prebuild_wheels jupyter==1.0.0
nox > python -m pip install -e . pytest pytest-cov
nox > python -m pip install pypiserver
Updating local tests package spec file cache...
nox > python scripts/update_package_cache.py testdata/tests_packages .pipx_tests/package_cache
Using the following file to specify needed package files:
    testdata/tests_packages/unix-python3.11.txt
Ensuring the following directory contains necessary package files:
    .pipx_tests/package_cache/3.11
MISSING FILES: 0
EXISTING (found) FILES: 167
LEFTOVER (unused) FILES: 0
nox > pytest --cov=pipx --cov-report= tests
================================================ test session starts =================================================
platform linux -- Python 3.11.3, pytest-7.3.2, pluggy-1.0.0
rootdir: /home/arthur/src/tmp/pipx
configfile: pyproject.toml
plugins: cov-4.1.0
collected 360 items / 124 deselected / 236 selected

tests/test_animate.py ...........                                                                              [  4%]
tests/test_completions.py .                                                                                    [  5%]
tests/test_emojis.py ........................                                                                  [ 15%]
tests/test_environment.py ..                                                                                   [ 16%]
tests/test_inject.py .......                                                                                   [ 19%]
tests/test_install.py .....................s...                                                                [ 29%]
tests/test_interpreter.py .......                                                                              [ 32%]
tests/test_list.py ........                                                                                    [ 36%]
tests/test_main.py ....                                                                                        [ 37%]
tests/test_package_specifier.py .........................................................                      [ 61%]
tests/test_pipx_metadata_file.py ......                                                                        [ 64%]
tests/test_reinstall.py .......                                                                                [ 67%]
tests/test_reinstall_all.py .....                                                                              [ 69%]
tests/test_run.py ....................................                                                         [ 84%]
tests/test_runpip.py .                                                                                         [ 85%]
tests/test_shared_libs.py ..                                                                                   [ 86%]
tests/test_uninject.py ...                                                                                     [ 87%]
tests/test_uninstall.py ................                                                                       [ 94%]
tests/test_uninstall_all.py ...                                                                                [ 95%]
tests/test_upgrade.py ........                                                                                 [ 98%]
tests/test_upgrade_all.py ...                                                                                  [100%]

============================= 235 passed, 1 skipped, 124 deselected in 185.51s (0:03:05) =============================
nox > Session tests-3.11 was successful.
nox > Running session cover
nox > Re-using existing virtual environment at .nox/cover.
nox > python -m pip install --upgrade pip
Requirement already satisfied: pip in ./.nox/cover/lib/python3.11/site-packages (23.1.2)
nox > python -m pip install coverage
nox > coverage report --show-missing --fail-under=70
Name                                 Stmts   Miss  Cover   Missing
------------------------------------------------------------------
src/pipx/__init__.py                     3      1    67%   4
src/pipx/__main__.py                     8      8     0%   1-14
src/pipx/animate.py                     80     16    80%   21-24, 104-108, 113-117, 122-126
src/pipx/colors.py                      30      2    93%   12, 30
src/pipx/commands/__init__.py           12      0   100%
src/pipx/commands/common.py            199     37    81%   56, 69-81, 87-97, 108, 116-117, 147, 180, 186, 268-269, 274-275, 305-307
src/pipx/commands/ensure_path.py        62     48    23%   31-51, 58-89, 94-147
src/pipx/commands/environment.py        13      0   100%
src/pipx/commands/inject.py             32      0   100%
src/pipx/commands/install.py            32      2    94%   39-40
src/pipx/commands/list_packages.py      80      7    91%   25, 35, 79-80, 89, 117, 134
src/pipx/commands/reinstall.py          53     12    77%   37-41, 51, 78-84, 113, 121-123, 126, 128
src/pipx/commands/run.py               154     26    83%   47, 52, 90, 111, 122-123, 129-132, 134, 229, 245-261, 313-314, 322-324, 353
src/pipx/commands/run_pip.py            11      1    91%   15
src/pipx/commands/uninject.py           65     10    85%   39-48, 51-52, 70, 85-86, 107, 112, 130
src/pipx/commands/uninstall.py          61      8    87%   98, 114-120, 129-130
src/pipx/commands/upgrade.py            71      4    94%   29, 60-61, 192
src/pipx/constants.py                   44      6    86%   20-25
src/pipx/emojis.py                      27      4    85%   34-37
src/pipx/interpreter.py                 42      7    83%   11-16, 61, 78
src/pipx/main.py                       323     29    91%   49-51, 134, 144, 152, 159, 173, 176-178, 199, 274, 277-281, 288, 714-715, 742-743, 822, 844, 859-860, 867, 877
src/pipx/package_specifier.py          113      1    99%   74
src/pipx/pipx_metadata_file.py          75      5    93%   20, 96, 136-137, 157
src/pipx/shared_libs.py                 69      7    90%   64, 80-81, 85-86, 118-119
src/pipx/util.py                       198     35    82%   45-48, 58, 61-62, 66-73, 91, 94-95, 109, 117-120, 203, 255, 287, 295-298, 302-305, 322-323, 338, 376, 391
src/pipx/venv.py                       213     31    85%   11-12, 61, 72, 104-106, 109, 139, 188, 204, 211-213, 269, 297, 320-322, 396, 409-417, 421, 427-434, 474, 479-480
src/pipx/venv_inspect.py               125     26    79%   12-13, 43, 79, 93-94, 99-105, 132, 153-161, 248, 258, 265
src/pipx/version.py                      2      0   100%
------------------------------------------------------------------
TOTAL                                 2197    333    85%
nox > coverage erase
nox > Session cover was successful.
nox > Ran multiple sessions:
nox > * tests-3.11: success
nox > * cover: success

[chrysle]

LGTM. @uranusjr could you take a look?

[uranusjr]

How does this affect existing installations? I feel we should add something in the documentation for this. Even if existing installations are not affected at all (i.e. will continue to use the old paths), we should add some notes to guide migration if users want to use the new paths.

[memchr]

Existing installations should not be affected.

Even if existing installations are not affected at all (i.e. will continue to use the old paths), we should add some notes to guide migration if users want to use the new paths.

Do you have any suggestions on where to put the note?

[uranusjr]

Since paths in docs have been changed to the new locations, maybe we can frame this as a potential question a user may have, e.g. files are not at the expected locations. An entry in Troubleshooting can explain why that is the case, it should not affect functionalities, provide a table to map old and new paths, and explain how to migrate if the user inclines (I assume by exporting and reinstalling all apps?)

[memchr]

I assume by exporting and reinstalling all apps?

Would it be better to add a new command/flag to handle the path change, or should it be done manually?

[chrysle]

Would it be better to add a new command/flag to handle the path change

I think that's not worth it... Most users will upgrade their pipx installations to latest anyway, I suppose. I suggest mentioning the version which introduces this in the note. For the minority who may not want to upgrade pipx because of various reasons, the maual way should be proposed (if even wanted).

[dukecat0]

Thanks for the PR!

[Baerbeisser]

Hmm, having moved ~/.local/* and ~/.config to ~/Data (i rebooted), pipx got dependency issues. So i deleted the now ~/Data/data/pipx (data being share) and installing now succeeds, but an empty ~/.local/share/man gets created and the binary symlinks get placed in ~/.local/bin.

My XDG variables:

XDG_BIN_HOME=/home/me/Data/bin
XDG_CACHE_HOME=/home/me/Data/cache
XDG_CONFIG_HOME=/home/me/Data/config
XDG_DATA_HOME=/home/me/Data/data
XDG_STATE_HOME=/home/me/Data/state

[memchr]

The spec doesn't include XDG_BIN_HOME.

[Baerbeisser]

Oh, right, it's a unofficial one. I also have XDG_LOG_HOME, because a lot of tools still use that (and i find it more convenient).

Still, XDG_DATA_HOME is set; it shouldn't create a .local/bin.

Is there somewhere a config i need to update/delete?