Skip to content

Serverless Indexing perf and stability improvements#150

Open
dmoore247 wants to merge 4 commits intomainfrom
fix/serverless-udf-memory
Open

Serverless Indexing perf and stability improvements#150
dmoore247 wants to merge 4 commits intomainfrom
fix/serverless-udf-memory

Conversation

@dmoore247
Copy link
Collaborator

@dmoore247 dmoore247 commented Oct 5, 2025

Fixes #149

  • put file hash under the if deep: test
  • put key deletion under if deep: test

From ~2 minutes before failing (on Serverless notebook compute)
To 20 seconds putting if deep: around file hash computation
To 10-11 seconds with dicom_utils change (putting if deep around key deletion)

- put file hash under the `if deep:` test
- put key deletion under `if deep:` test
@dmoore247 dmoore247 self-assigned this Oct 5, 2025
@dmoore247 dmoore247 requested a review from erinaldidb October 5, 2025 01:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[UDF_PYSPARK_USER_CODE_ERROR.MEMORY_LIMIT_SERVERLESS] Execution failed. ...

2 participants