Start a processing a BI pipeline by reading raw data into pandas DataFrames (a two dimensional representation much like an Excel sheet).
Proper setup is critical. Follow earlier instructions to set up your machine.
We used these one-time commands when first starting the project.
uv python pin 3.12
uv venv
uv sync --extra dev --extra docs --upgrade
uv run pre-commit install
uv run python --versionWindows (PowerShell):
.\.venv\Scripts\activatemacOS / Linux / WSL:
source .venv/bin/activateAs we progress, we'll use this daily workflow often.
Always start with git pull to check for any changes made to the GitHub repo.
git pullIf we need additional packages, we first add them to pyproject.toml. Add pre-commit to pyproject.toml if you haven't already.
- Update dependencies (for security and compatibility).
- Clean unused cached packages to free space.
- Use
git add .to stage all changes. - Run ruff and fix minor issues.
- Update pre-commit periodically.
- Run pre-commit quality checks on all code files (twice if needed, the first pass may fix things).
- Run tests.
In VS Code, open your repository, then open a terminal (Terminal / New Terminal) and run the following commands one at a time to check the code.
uv sync --extra dev --extra docs --upgrade
uv cache clean
git add .
uvx ruff check --fix
uvx pre-commit autoupdate
uv run pre-commit run --all-files
git add .
uv run pytestNOTE: The second git add . ensures any automatic fixes made by Ruff or pre-commit are included before testing or committing.
Make sure you have current doc dependencies, then build your docs, fix any errors, and serve them locally to test.
uv run mkdocs build --strict
uv run mkdocs serve- After running the serve command, the local URL of the docs will be provided. To open the site, press CTRL and click the provided link (at the same time) to view the documentation. On a Mac, use CMD and click.
- Press CTRL c (at the same time) to stop the hosting process.
This project includes demo code. Run the data_prep module to confirm everything is working.
In VS Code terminal, run:
uv run python -m analytics_project.data_prepAnytime we make working changes to code is a good time to git add-commit-push to GitHub.
- Stage your changes with git add.
- Commit your changes with a useful message in quotes.
- Push your work to GitHub.
git add .
git commit -m "describe your change in quotes"
git push -u origin mainThis will trigger the GitHub Actions workflow and publish your documentation via GitHub Pages.
With a working version safe in GitHub, start making changes to the code.
Before starting a new session, remember to do a git pull and keep your tools updated.
Each time forward progress is made, remember to git add-commit-push.