Add ruff as default linting + formatting option

[chrisjkuch]

Closes #374
Closes #388

This PR adds ruff as the default linting and formatting option for new ccds projects.

Implementation notes

• I've slightly altered the Makefile command tests so that they now run within each venv harness. I check to ensure that linting and formatting run without error and that nothing is reformatted.
  • For whatever reason, I found that ruff piped to stdout and flake8/black/isort piped to stderr. Maybe there's a nuance here or I've done something wrong?
• I've cleaned up the generated pyproject.toml and removed setup.cfg if using ruff since it's no longer needed.
• I didn't stray very far from the default ruff config, changing only the line length. If we are opinionated about the default set of linting / formatting options, happy to add those, but I have no strong preferences.

Discussion

As mentioned, this PR makes ruff the default option. I think there's good arguments for this that mostly boil down to "it's simpler + faster":

• one tool for linting + formatting so fewer dependencies and simpler config
• no setup.cfg file so everything can live in pyproject.toml
• ruff is quite fast

However, ruff is still (relatively) new, whereas flake8/black/isort is more mature, and most projects won't notice the speed difference until they get quite large. I could be persuaded that we should introduce ruff without making it the default option.

[jayqi]

I'm in favor of having the default to be Ruff. It's clear that Ruff is the emerging standard, and they describe themselves as production-ready.

There are major projects that are fully using Ruff for linting and formatting.

• Pandas
• pytest

Lots of other major projects (e.g., pip, matplotlib, PyTorch) are using Ruff for linting over flake8, though are still using black for now.

[szapp]

It seems that slight adjustments to the linter rules are necessary to ensure similar behavior as the flake8/black/isort alternative. Most of the rules that flake8 ignores in the setup.cfg are ignored in the defaults of ruff as well, except for E731 if I am not mistaken.

If I am not wrong, these additional lines in pyproject.toml should do.

[tool.ruff.lint]
ignore = [
  "E731"  # lambda-assignment: Do not assign a lambda expression, use a def
]

---

Update: Additionally, the I (isort) rules should be added according to the readme of the charliermarsh.ruff vscode extension in order to enable organization of imports. It's not enabled by default.

[tool.ruff.lint]
ignore = ["E731"]  # Ignore lambda-assignment
extend-select = ["I"]  # Add sorting imports (isort ruleset) to defaults

[chrisjkuch]

@szapp thanks for catching the isort fix! I've opted to keep the default selections in ruff for now - I don't know that we're necessarily aiming for exact parity with black, as opposed to just a good starting point for a project.