fix: Adjust LGBM_DatasetCreateFromSampledColumn to handle distributed data

[svotaw]

This PR adds a new API related to LGBM_DatasetCreateFromSampledColumn that splits the current single parameter for "num_total_data" into 2, one for total local data and one for total distributed data. The new API is called LGBM_DatasetDistCreateFromSampledColumn, but that name can be changed. I kept the old one for back-compat.

See issue here: #5343

[StrikerRUS]

I believe we can take the opportunity to make breaking changes in the next major release and split the current num_total_row argument into two new ones in the current LGBM_DatasetCreateFromSampledColumn() function. For the local training these new arguments will be equal to each other. I think this is better for user experience than two similar separate functions.

[svotaw]

I believe we can take the opportunity to make breaking changes in the next major release and split the current num_total_row argument into two new ones in the current LGBM_DatasetCreateFromSampledColumn() function. For the local training these new arguments will be equal to each other. I think this is better for user experience than two similar separate functions.

Sounds fine to me. Should I go ahead and make those changes? Also, does that mean this will get checked in when approved, or will it need to wait for some other condition related to incrementing the major version?

[StrikerRUS]

Next release will be v4.0.0 - that's a fact, and I believe we should include this PR in it (#5153).

Should I go ahead and make those changes?

I think it's better to wait for some other opinion on this (@guolinke @shiyu1994 ).

[guolinke]

I agree with @StrikerRUS , please go ahead

[mhamilton723]

Hey @StrikerRUS, @jameslamb, @shiyu1994 just wanted to gently ping on this, anything else Scott should tackle before approval. Thanks so much for all of your help, time, and thoughtful review comments!

[shiyu1994]

@svotaw Thanks for your great contribution. The changes LGTM!

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.