Skip to content

[StreamingDataLoader, 3/N] feat: implement StreamingDataSet and StreamingDataLoader #28

[StreamingDataLoader, 3/N] feat: implement StreamingDataSet and StreamingDataLoader

[StreamingDataLoader, 3/N] feat: implement StreamingDataSet and StreamingDataLoader #28

Workflow file for this run

# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Python package
on:
push:
branches: [ "main", "dev" ]
pull_request:
branches: [ "main", "dev" ]
jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
python-version: ["3.11"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest build pytest_asyncio
python -m build --wheel
pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install dist/*.whl
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest (TQ_ZERO_COPY_SERIALIZATION=False)
run: |
pytest
- name: Test with pytest (TQ_ZERO_COPY_SERIALIZATION=True)
run: |
ray stop --force
export TQ_ZERO_COPY_SERIALIZATION=True
pytest