Skip to content

compaso: add passthrough support#162

Merged
lgarrison merged 3 commits intomainfrom
passthrough
Dec 13, 2024
Merged

compaso: add passthrough support#162
lgarrison merged 3 commits intomainfrom
passthrough

Conversation

@lgarrison
Copy link
Member

For the halo light cone catalogs in the Backlight project, we're doing halo finding on the time slices, then interpolating them onto the light cone with merger trees. Many time slice halos won't appear in the light cone, so there's no point in saving them. When we create the "squashed" catalogs that remove unused halos, we don't want to unpack all the columns, we want them to stay in their original format (e.g. we don't want to unpack rvint subsamples, because the floats won't compress as well). But we still want to load things through the CompaSOHaloCatalog interface, because it does all sorts of non-trivial merging of the uncleaned and cleaned halo info and subsamples, as well as row-based filtering, which we're going to use to get rid of unused halos.

This PR adds a passthrough bool argument to the CHC constructor to load halo info and subsamples without unpacking them. This effectively provides a way to get the raw data through CHC. Most users won't need this unless they're writing a pipeline where they want to defer the unpacking of the data to a later stage (often at the end-user stage, as CHC usually does).

passthrough lets us keep data in its unpacked format, which is useful when you want to defer the unpacking as part of a pipeline, for example.
@lgarrison lgarrison merged commit db865d1 into main Dec 13, 2024
@lgarrison lgarrison deleted the passthrough branch December 13, 2024 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant