2290 Add support to update data in Datasets

[Nic-Ma]

Fixes #2290 .

Description

This PR added support to update data after several epochs in Datasets.

Status

Ready

Types of changes

• Non-breaking change (fix or new feature that would not break existing functionality).
• Breaking change (fix or new feature that would cause existing functionality to change).
• New tests added to cover the changes.
• Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
• Quick tests passed locally by running ./runtests.sh --quick --unittests.
• In-line docstrings updated.
• Documentation updated, tested make html command in the docs/ folder.

[Nic-Ma]

/black

[Nic-Ma]

/black

[wyli]

I think the new method could be set_data instead of update_data? not sure about the overall stability in the multiprocess loading because there will be concurrent read and write for the cache. maybe the PersistentDataset is ok but how about the other datasets?

[Nic-Ma]

/black

[Nic-Ma]

Hi @wyli ,

Thanks for your review and suggestions.
I updated the PR according to your comments.
Could you please help review it again?
I already added multi-processing test based on workers of DataLoader:
https://github.com/Project-MONAI/MONAI/pull/2436/files#diff-eea046ab7338e5acd9b172b24ec5a704a755dd10bfae2056b046d598d064ecf9R97

Thanks in advance.

[Nic-Ma]

Hi @wyli ,

BTW, I already added the doc-string in CacheDataset to highlight that it requires persistent_workers=False in DataLoader:
https://github.com/Project-MONAI/MONAI/pull/2436/files#diff-e682e654b07b9d753cac73f5a782e70644c82a84d30ebc757ec294a261845108R541

Thanks.

[wyli]

thanks, just need to change the docstring update_data() to set_data()