allows user to output code in chunks when simulation input (or output…

[boryanah]

…) is too large; no documentation yet (can add some)

@SandyYuan Could you take a look at this? I will add documentation to abacus_hod.py if it seems fine.

[SandyYuan]

looks fine by me.

[lgarrison]

Seems fine to me! Is there anything to consider with the indexing of particles that will change because the arrays are now broken up?

Also maybe params['numslabs'] = n_jump should be changed to reflect the actual number of slabs used, which may be fewer if this is the last chunk.

[SandyYuan]

I agree with the n_jump change. @lgarrison What kind of issues do you see with the particle array breaking up? I cant immediately think of any failure modes. The particles are only used to populate satellites I am fairly sure. So as long as we believe we are not dividing a single halo's particles between different chunks I think we are fine?

[lgarrison]

@SandyYuan If the halos are using a start and count to index into the particle arrays, then the start will change if the array is split up. This is how we index the halo_rv_A files from the halo_info files, but I forget if prepare_sim uses a different method. I see a phid array, so is each particle just keeping track of its halo ID instead?

@boryanah Did you want to make the n_jump change, or was there a reason to keep it as-is? Also, could you update the changelog?

[boryanah]

I was just about to say that I thought I made the n_jump change, but I guess I only changed end: if end > len(halo_info_fns): end = len(halo_info_fns). Let me very quickly reopen this and change it to params['numslabs'] = end-start.
I believe that the halo IDs are saved in the prepare_sim part of the processs and hence it probably doesn't matter what order the superslabs are read in and written out for abacus_hod?