Skip to content

cache workflow resources#1173

Merged
jmartin-tech merged 3 commits intoNVIDIA:mainfrom
jmartin-tech:fix/cache-workflow-resources
Apr 20, 2025
Merged

cache workflow resources#1173
jmartin-tech merged 3 commits intoNVIDIA:mainfrom
jmartin-tech:fix/cache-workflow-resources

Conversation

@jmartin-tech
Copy link
Collaborator

@jmartin-tech jmartin-tech commented Apr 18, 2025

Implements a shared cache for most, though not all, huggingface models used in automation tests.

This reduces the traffic to hugginface for github actions by creating a hf-cache-shared action by leveraging the XDG_CACHE_HOME to place the cache in a consistent location relative to the runner working directory.

Due to the 10GB limit of all caches per repo this cache file is updated during the nightly test run and any PR will use the cache to reduce the models required to be retrieved from huggingface for each test execution.

This also enables manual execution of the Nightly task in forks of the repository.

Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
@jmartin-tech jmartin-tech added the tests Testing-related label Apr 18, 2025
Copy link
Collaborator

@leondz leondz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elegant and fast. Perhaps rename to garak-cache-shared because we do more than HF content.

Is there a way of monitoring cache utilisation/activity?

statuses: none

env:
XDG_CACHE_HOME: ${{ github.workspace }}/.cache
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice use of XDG

Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
@jmartin-tech jmartin-tech merged commit 8ca9024 into NVIDIA:main Apr 20, 2025
9 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Apr 20, 2025
@jmartin-tech jmartin-tech deleted the fix/cache-workflow-resources branch October 28, 2025 20:45
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

tests Testing-related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants