Skip to content

Conversation

@codetheweb
Copy link
Contributor

@codetheweb codetheweb commented Jun 9, 2025

Description of changes

Speeds up test_embeddings.py against the cluster: ~25m -> ~16m. The test is now parallelized and checks for new collection versions at a higher frequency (the collection version change has the side effect of also speeding up test_filtering.py).

Test plan

How are these changes tested?

  • Tests pass locally with pytest for python, yarn test for js, cargo test for rust

@github-actions
Copy link

github-actions bot commented Jun 9, 2025

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

@codetheweb codetheweb marked this pull request as ready for review June 9, 2025 23:25
@codetheweb codetheweb requested a review from itaismith June 9, 2025 23:26
Copy link
Contributor

@itaismith itaismith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@codetheweb codetheweb enabled auto-merge (squash) June 10, 2025 22:20
@propel-code-bot
Copy link
Contributor

Parallelize and Accelerate test_embeddings.py for Cluster Execution

This PR optimizes and parallelizes the property-based test suite test_embeddings.py when run against a cluster, reducing runtime significantly (~25m to ~16m). Key improvements include more frequent polling for collection version changes, centralization of isolated database creation, and updates to related utils and workflow configuration to further accelerate other tests (e.g., test_filtering.py). The isolated database setup logic is now shared and reused across test modules, reducing overhead and inter-test dependencies.

Key Changes:
• Moved create_isolated_database utility from test_add.py to conftest.py for central test use.
• Updated test_embeddings.py and test_add.py to use create_isolated_database, improving test isolation.
• Modified wait_for_version_increase.py to decrease TIMEOUT_INTERVAL from 5 to 1 second for faster polling.
• Refactored workflow YAML to parallelize test_embeddings.py (in addition to test_add.py) in cluster jobs.
• Removed redundant/duplicated isolated database code and updated imports accordingly.

Affected Areas:
• chromadb/test/property/test_embeddings.py
• chromadb/test/property/test_add.py
• chromadb/test/conftest.py
• chromadb/test/utils/wait_for_version_increase.py
• .github/workflows/_python-tests.yml

This summary was automatically generated by @propel-code-bot

@codetheweb codetheweb merged commit 928db5b into main Jun 10, 2025
57 checks passed
@codetheweb codetheweb deleted the tst-speed-up-test-embeddings branch June 10, 2025 22:42
Inventrohyder pushed a commit to Inventrohyder/chroma that referenced this pull request Aug 5, 2025
## Description of changes

Speeds up `test_embeddings.py` against the cluster: `~25m -> ~16m`. The
test is now parallelized and checks for new collection versions at a
higher frequency (the collection version change has the side effect of
also speeding up `test_filtering.py`).

## Test plan

_How are these changes tested?_

- [x] Tests pass locally with `pytest` for python, `yarn test` for js,
`cargo test` for rust
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants