Skip to content

Add resume functionality to to_hats#1276

Merged
smcguire-cmu merged 5 commits intomainfrom
sean/resume-to-hats
Mar 6, 2026
Merged

Add resume functionality to to_hats#1276
smcguire-cmu merged 5 commits intomainfrom
sean/resume-to-hats

Conversation

@smcguire-cmu
Copy link
Contributor

@smcguire-cmu smcguire-cmu commented Mar 5, 2026

Changes to_hats to write done and histogram files while writing, allowing the pipeline to be resumed by reading the progress in the directory.

Closes #760

@github-actions
Copy link

github-actions bot commented Mar 5, 2026

Before [2f9c470] After [09b4b1c] Ratio Benchmark (Parameter)
19.4±0.01s 20.0±0.02s 1.03 benchmarks.time_save_big_catalog
8.21±0.06s 8.38±0.07s 1.02 benchmarks.time_lazy_crossmatch_many_columns_all_suffixes
3.80±0.03s 3.83±0.04s 1.01 benchmarks.time_open_many_columns_all
46.8±1ms 47.4±1ms 1.01 benchmarks.time_polygon_search
8.30±0.04s 8.31±0.02s 1 benchmarks.time_lazy_crossmatch_many_columns_overlapping_suffixes
169±2ms 169±3ms 1 benchmarks.time_open_many_columns_list
27.9±0.1ms 27.5±0.1ms 0.99 benchmarks.time_box_filter_on_partition
7.41±0s 7.37±0.03s 0.99 benchmarks.time_create_large_catalog
1.10±0.01s 1.10±0.01s 0.99 benchmarks.time_create_midsize_catalog
378±4ms 373±7ms 0.99 benchmarks.time_open_many_columns_default

Click here to view all benchmarks.

@codecov
Copy link

codecov bot commented Mar 5, 2026

Codecov Report

❌ Patch coverage is 98.90110% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 96.78%. Comparing base (2f9c470) to head (3c452fd).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/lsdb/io/to_hats.py 98.90% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1276      +/-   ##
==========================================
+ Coverage   96.67%   96.78%   +0.11%     
==========================================
  Files          48       48              
  Lines        2976     3047      +71     
==========================================
+ Hits         2877     2949      +72     
+ Misses         99       98       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@smcguire-cmu smcguire-cmu changed the title Add resume functionality to hats Add resume functionality to to_hats Mar 6, 2026
Copy link
Contributor

@delucchi-cmu delucchi-cmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please tag the related issue. Looks like #760.

@smcguire-cmu smcguire-cmu merged commit cacdb78 into main Mar 6, 2026
12 checks passed
@smcguire-cmu smcguire-cmu deleted the sean/resume-to-hats branch March 6, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make it possible to resume LSDB pipeline

2 participants