Kate/splitter

[jihyeonyi]

Summary

Added task-specific annotation splitters.

• SplitforClassification
• SplitforMatchingReID
• SplitforDetection

How to test

python -m unittest -v tests/test_splitter.py

Checklist

• I submit my changes into the develop branch
• I have added description of my changes into CHANGELOG
• I have updated the documentation accordingly
• I have added tests to cover my changes
• I have linked related issues)

License

• I submit my code changes under the same MIT License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below)

# Copyright (C) 2020 Intel Corporation
#
# SPDX-License-Identifier: MIT

[zhiltsov-max]

Thank you for a great PR!

I'd suggest fixing code formatting (spaces in arguments) and string formatting to keep the project code in a single style.

[zhiltsov-max]

I think, this should be done closer to the actual use, because a lot can happen after constructor is called.

[jihyeonyi]

Random shuffling occurs several times.
Do you think I really need to set seed whenever I use the shuffling function?

[zhiltsov-max]

Just make sure the results are reproducible. I suggest moving seed setting as much close to the point of use, as possible.

[jihyeonyi]

Then, I'll add test codes to check the results are reproducible.

[jihyeonyi]

I need help regarding complexity.
I refactor the code and get complexities like below.

$ python -m mccabe datumaro/plugins/splitter.py --min 10
188:4: 'SplitforMatchingReID.__init__' 16
307:4: 'SplitforMatchingReID._rebalancing' 12
370:4: 'SplitforDetection.__init__' 19

But Codacy gives 35 for SplitforMatchingReID.init and 20 for SplitforDetection.init
Why they're different? And how can resolve this issue?

[zhiltsov-max]

Complexity from these tools is not something we look at. Don't bother with this.

[jihyeonyi]

Here, I use 'group' property to split 'test' subset into 'query' and 'gallery' set.
It seems that the default subset doesn't support a hierarchy.
In the original implementation, we add "Auxiliary" attribute to the annotation.
Do you think I have to change the current implementation w.r.t the original one?
Or do I have to implement the subset hierarchy?
Please share your opinion.

[zhiltsov-max]

Yes, there in no exact support for split hierarchy yet. To do what you want (as a workaround?), I'd create 2 extra subsets - test_gallery and test_query, this would look quite natural (at least, in Datumaro). Technically, subsets are extractors themselves, so it is possible to extend the implementation with some nesting. But it would require lots of changes in the importing/exporting code.

[jihyeonyi]

Actually, the gallery and query sets are not subsets. They're a kind of tag.
I need more time to resolve this, so let's leave this as it is at the moment.
I'll resolve this issue while applying the command line feature.

[jihyeonyi]

I've rebased and squashed some commits.
I'd like to resolve the gallery/query tagging issue and support command line execution in the future pull request.