Allow user to choose package manager

[sliem]

I'm using pipenv a lot which is, and I quote, the officially recommended Python packaging tool from Python.org. Is there interest in supporting pipenv in this cookiecutter?

Something like this by @callumkift.

[pjbull]

I'd love to make pipenv an option. I think that it would need:

(1) An assessment of how the environment creation would interact with existing conda and vritualenv options.
(2) An assessment of how the existing requirements.txt based flow would need to change when using the pipenv option.
(3) It introduces enough complexity that I'd like it to be part of an effort that does a more robust job of running test cases.

Thoughts on any of these? Looks like the option you pointed to is more of a replacement than an option.

[callumkift]

Just thought I would quickly chime in. Yes, what I am doing is a replacement, rather than an option. It is much more of a quick replacement rather than a full fleshed out approach, e.g. pipenv is only used in the created repo, rather than the overall project and I have some pre-installed modules that I always use.

If this is considered something that the project will move forward on (option rather than replacement), I would love to help out.

[pjbull]

Yeah, I'm interested in determining what would be necessary to make pipenv work.

The integration may be a little sticky since we use the existing structure in a number of different ways. For example, we auto-detect conda/virtualenv for environment creation rather than making a user specify right now. We also feed requirements.txt into various commands in the Makefile.

If we have a good plan that doesn't result in a lot of end user complexity or code/maintenance complexity, then we'd be open to this.

[sliem]

Nice to see interests in pipenv support. I've given it some thoughts, and trying to answers your questions/concerns.

Regarding requirements.txt; pipenv itself do not use requirements.txt but can produce one when required (using pipenv lock -r). It's used in one place in the Makefile (in requirements).

The current Makefile assumes the user want to use conda environment if it is available which might not be true. I would stop auto-detections and promote it to an choice variable in cookiecutter.json:

"virtualenv": ["pipenv", "conda", "virtualenv"]

We can then use jinja2 conditionals to implement the choice in the Makefile. E.g.

lint:
{% if cookiecutter.virtualenv == 'pipenv' -%}
	pipenv run $(PYTHON_INTERPRETER) -m flake8 src
{% else -%}
	$(PYTHON_INTERPRETER) -m flake8 src

Note that with pipenv we don't have to manually manage the environment. To install a package is simply run pipenv install . pipenv run <script> to run <script> in the environment. In the lint example we would not need to activate (pipenv shell) before running make lint. Side-note: it would be nice if we could handle the conda env similarly. Do not know how.

Creating the environment would be something like:

create_environment:
{% if cookiecutter.virtualenv == 'pipenv' -%}
	@$(PIP_CMD) install -q pipenv
	@echo ">>> Creating virtual environment using pipenv."
	pipenv --python $(PYTHON_VERSION)
	@echo ">>> New pipenv environment is created."

{% elif cookiecutter.virtualenv == 'conda' -%}
	@echo ">>> Creating conda environment."
	conda create --name $(PROJECT_NAME) python=$(PYTHON_VERSION)
	@echo ">>> New conda env created. Activate with:\nsource activate $(PROJECT_NAME)"

{% else -%} # default to virtualenv+wrapper
	...

{% endif -%}

Honestly I would however drop support for virtualenv+wrapper for the preferred pipenv.

[hackalog]

I took some of this thread to heart when writing PR #124 . (e.g. allowing the user to specify conda/virtualenv explicitly). I'd happily extend that PR to include pipenv support.

The sticky point is that pipenv was specifically designed to obsolete a single requirements.txt file, preferring the separate lockfile mechanism that many other package managers now use.

(By the way, am I missing something with the jinja2 conditionals? They seem to eat whitespace when I use them (bad in a makefile), and I'm forced to use a macro-based workaround.)

So what's the path forward? In the long term, pip+virtualenv is on the way out (python has formally endorsed the pipenv approach as the future), and as mentioned over in PR #124, it's often preferable to use all conda (or conda plus minimal pip) when using certain packages. I think a reasonable start would be to let the user choose their package manager of choice (conda, virtualenv, pipenv), and then using whatever config file format is preferred for the chosen package manager. That punts the tricky business of mixing package managers (e.g. conda+pip, or pipenv+non-python-dependencies) upstream to the respective package manager projects.

[jraviotta]

I support the choice of package manager. I've had a hard time getting a project working because pip and conda are fighting over dependency management. It would be nice to offer the user the choice on project creation, generate the required resources and use that manager for all processes.

I was putting together a PR, but found this thread. Happy to help if anyone wants a hand.

[project-delphi]

Agree that pipenv support in package manager would allow broader usage. It's a pain to use pip, requirements.txt, venv and virtualenvwrapper tools for consistent builds otherwise.

[dovy-mus]

Hey, sorry to necro this old issue, but I have noticed lack of support for Poetry when choosing package manager. Honestly, I believe it is the best package/virtual environment manager for Python at this time. I could create a PR for this but maybe someone could add it as well?

[jayqi]

Closing—implemented for V2 with initial options for virtualenv, conda, pipenv.

Other options like Poetry, etc. can be considered in separate issues for future implementation.