Skip to content

feat: serve cached digest if available#462

Merged
MickaelCa merged 5 commits intomainfrom
use-s3-as-cache
Jul 29, 2025
Merged

feat: serve cached digest if available#462
MickaelCa merged 5 commits intomainfrom
use-s3-as-cache

Conversation

@MickaelCa
Copy link
Copy Markdown
Member

  • Added methods to upload metadata alongside digest files to S3.
  • Implemented S3-based digest caching mechanism for improved efficiency.
  • Refactored digest storage logic to support both S3 and local storage.

@MickaelCa MickaelCa added push-container Allow the PR to build & push a docker container instead of just building it deploy-pr-temp-env labels Jul 27, 2025
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jul 27, 2025

⚙️ Preview environment was undeployed.

@MickaelCa MickaelCa marked this pull request as ready for review July 27, 2025 16:17
Comment thread src/server/query_processor.py Outdated
@MickaelCa MickaelCa force-pushed the use-s3-as-cache branch 2 times, most recently from acb00ea to dd6e541 Compare July 28, 2025 15:05
@MickaelCa MickaelCa requested a review from NicolasIRAGNE July 28, 2025 17:24
Copy link
Copy Markdown
Contributor

@NicolasIRAGNE NicolasIRAGNE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RGPD

ix-56h
ix-56h previously requested changes Jul 28, 2025
Copy link
Copy Markdown
Contributor

@ix-56h ix-56h left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we ingest a repo hello_world then try to ingest it again with include/exclude patterns the cache is not used ?

Anyway, LGTM !

Comment thread src/server/query_processor.py Outdated
Comment thread src/server/s3_utils.py
Comment thread src/server/s3_utils.py
Combine steps to resolve and assign commit in query_processor.
Copy link
Copy Markdown
Contributor

@NicolasIRAGNE NicolasIRAGNE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RPDC

@cyclotruc cyclotruc dismissed ix-56h’s stale review July 29, 2025 16:01

fine for now going with simple check 404, can easily switch back if needed

@MickaelCa MickaelCa merged commit efe5a26 into main Jul 29, 2025
25 of 44 checks passed
@MickaelCa MickaelCa deleted the use-s3-as-cache branch July 29, 2025 16:02
@coderamp-ci coderamp-ci Bot mentioned this pull request Jul 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deploy-pr-temp-env push-container Allow the PR to build & push a docker container instead of just building it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants