Split transform #4153

bhashemian · 2022-04-20T17:51:03Z

Description

Implement Split transform that splits an image based on a grid. This transform is particularly useful in pathology pipelines.

Status

Ready/Work in progress/Hold

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

* Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <[email protected]> * Add unittests for WSIReader Signed-off-by: Behrooz <[email protected]> * Add image mode for output validation Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Update references to new WSIReader Signed-off-by: Behrooz <[email protected]> * Remove legacy WSIReader Signed-off-by: Behrooz <[email protected]> * Update unittests Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * sort imports Signed-off-by: Behrooz <[email protected]> * Clean up imports Signed-off-by: Behrooz <[email protected]> * Update docstrings Signed-off-by: Behrooz <[email protected]> * Update docs and docstrings Signed-off-by: Behrooz <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <[email protected]> * Remove redundant checking Signed-off-by: Behrooz <[email protected]> * Update read and other methods Signed-off-by: Behrooz <[email protected]> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <[email protected]> * Make workaround for CuImage objects Signed-off-by: Behrooz <[email protected]> * Add unittests for multi image reading Signed-off-by: Behrooz <[email protected]> * Update a note about cucim Signed-off-by: Behrooz <[email protected]> * Update type hints and docstrings Signed-off-by: Behrooz <[email protected]>

Signed-off-by: Behrooz <[email protected]>

…form

Signed-off-by: Behrooz <[email protected]>

docs/source/transforms.rst

monai/transforms/spatial/array.py

monai/transforms/spatial/dictionary.py

monai/transforms/spatial/array.py

tests/test_split_transform.py

tests/test_split_transform_dict.py

Signed-off-by: Behrooz <[email protected]>

bhashemian · 2022-04-22T16:10:41Z

Any other comments, @Nic-Ma? Thanks

Signed-off-by: Behrooz <[email protected]>

Nic-Ma

Thanks for the quick update!
Looks good to me.
Let's address the sample image issue in another ticket #4161 later.

Thanks.

Signed-off-by: Behrooz <[email protected]>

bhashemian · 2022-04-22T22:11:08Z

Thank you @Nic-Ma! It just need a /build!

Nic-Ma · 2022-04-23T00:17:55Z

/build

Nic-Ma · 2022-04-23T01:57:41Z

/build

Nic-Ma · 2022-04-23T02:58:01Z

Hi @drbeh ,

The Blossom CI failed due to these cuCIM / tifffile errors:
https://pipelines.actions.githubusercontent.com/serviceHosts/b9c0ace7-8939-446b-80c4-ee7a711ae9fe/_apis/pipelines/1/runs/45363/signedlogcontent/2?urlExpires=2022-04-23T02%3A55%3A51.0143983Z&urlSigningMethod=HMACV1&urlSignature=xQZGiCa2Q7diUmOHn8359OQby9lS7KkNJZL1HYq9oSI%3D

Could you please help take a look?

Thanks.

2022-04-23T02:27:55.1788893Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1789153Z [2022-04-23T02:27:07.931Z] ERROR: test_read_malformats_0 (tests.test_wsireader_new.TestCuCIM)
2022-04-23T02:27:55.1789423Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1789624Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1789973Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
2022-04-23T02:27:55.1790175Z [2022-04-23T02:27:07.931Z]     return func(*(a + p.args), **p.kwargs)
2022-04-23T02:27:55.1790532Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader_new.py", line 187, in test_read_malformats
2022-04-23T02:27:55.1790721Z [2022-04-23T02:27:07.931Z]     file_path = save_gray_tiff(
2022-04-23T02:27:55.1791068Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader_new.py", line 106, in save_gray_tiff
2022-04-23T02:27:55.1791406Z [2022-04-23T02:27:07.931Z]     imwrite(filename, img_gray, shape=img_gray.shape, photometric="rgb")
2022-04-23T02:27:55.1791713Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1791966Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1792273Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 2133, in write
2022-04-23T02:27:55.1792451Z [2022-04-23T02:27:07.931Z]     raise RuntimeError(
2022-04-23T02:27:55.1792831Z [2022-04-23T02:27:07.931Z] RuntimeError: None != 2 <StoredShape(frames=1, separate_samples=1, depth=1, length=16, width=16, contig_samples=1, extrasamples=0)>
2022-04-23T02:27:55.1792963Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1793186Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1793486Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_0 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1793745Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1793943Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1794273Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1794503Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1794847Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1795077Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1795417Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1795663Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1795963Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1796211Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1796517Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1796653Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1796874Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1797161Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_1 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1797428Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1797627Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1797955Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1798129Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1798472Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1798702Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1799037Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1799282Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1799581Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1799828Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1800086Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1800217Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1800431Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1800725Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_2 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1800992Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1801188Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1801512Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1801687Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1802025Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1802252Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1802583Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1802829Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1803177Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1803423Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1803678Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1803799Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1804019Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1804310Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_3 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1804572Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1804766Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1805130Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1805309Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1805650Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1805876Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1806204Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1806447Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1806744Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1806992Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1807250Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1807372Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1807592Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1807883Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_4 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1808146Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1808343Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1808664Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1808836Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1809181Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1809409Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1809736Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1809980Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1810276Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1810522Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1810765Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1810893Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1811116Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1811455Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_5 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1811717Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1811913Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1812238Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1812415Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1812754Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1812981Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1813354Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1813606Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1813903Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1814140Z [2022-04-23T02:27:07.931Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1814390Z [2022-04-23T02:27:07.931Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1814520Z [2022-04-23T02:27:07.931Z] 
2022-04-23T02:27:55.1814743Z [2022-04-23T02:27:07.931Z] ======================================================================
2022-04-23T02:27:55.1815033Z [2022-04-23T02:27:07.931Z] ERROR: test_read_patches_cucim_6 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1815296Z [2022-04-23T02:27:07.931Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1815495Z [2022-04-23T02:27:07.931Z] Traceback (most recent call last):
2022-04-23T02:27:55.1815872Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1816079Z [2022-04-23T02:27:07.931Z]     prepare_test_data()
2022-04-23T02:27:55.1816419Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1816645Z [2022-04-23T02:27:07.931Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1816974Z [2022-04-23T02:27:07.931Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1817217Z [2022-04-23T02:27:07.931Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1817504Z [2022-04-23T02:27:07.931Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1817753Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1818007Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1818138Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1818357Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1818645Z [2022-04-23T02:27:07.932Z] ERROR: test_read_patches_cucim_7 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1818906Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1819101Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1819422Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1819594Z [2022-04-23T02:27:07.932Z]     prepare_test_data()
2022-04-23T02:27:55.1819933Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1820220Z [2022-04-23T02:27:07.932Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1820548Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1820787Z [2022-04-23T02:27:07.932Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1821081Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1821326Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1821576Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1821705Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1821969Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1822263Z [2022-04-23T02:27:07.932Z] ERROR: test_read_patches_cucim_8 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1822526Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1822721Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1823042Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1823215Z [2022-04-23T02:27:07.932Z]     prepare_test_data()
2022-04-23T02:27:55.1823553Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1823778Z [2022-04-23T02:27:07.932Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1824101Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1824345Z [2022-04-23T02:27:07.932Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1824640Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1824884Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1825135Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1825269Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1825490Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1825779Z [2022-04-23T02:27:07.932Z] ERROR: test_read_patches_cucim_9 (tests.test_lesion_froc.TestEvaluateTumorFROC)
2022-04-23T02:27:55.1826048Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1826246Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1826568Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 307, in setUp
2022-04-23T02:27:55.1826742Z [2022-04-23T02:27:07.932Z]     prepare_test_data()
2022-04-23T02:27:55.1827079Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 52, in prepare_test_data
2022-04-23T02:27:55.1827296Z [2022-04-23T02:27:07.932Z]     save_as_tif("temp_ground_truth_0", ground_truth)
2022-04-23T02:27:55.1827623Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_lesion_froc.py", line 34, in save_as_tif
2022-04-23T02:27:55.1827866Z [2022-04-23T02:27:07.932Z]     imwrite(file_path, array, compress="jpeg", tile=(16, 16))
2022-04-23T02:27:55.1828164Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1828457Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1828707Z [2022-04-23T02:27:07.932Z] TypeError: write() got an unexpected keyword argument 'compress'
2022-04-23T02:27:55.1828838Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1829057Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1829315Z [2022-04-23T02:27:07.932Z] ERROR: test_read_malformats_0 (tests.test_wsireader.TestCuCIM)
2022-04-23T02:27:55.1829574Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1829774Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1830122Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
2022-04-23T02:27:55.1830376Z [2022-04-23T02:27:07.932Z]     return func(*(a + p.args), **p.kwargs)
2022-04-23T02:27:55.1830720Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 194, in test_read_malformats
2022-04-23T02:27:55.1830908Z [2022-04-23T02:27:07.932Z]     file_path = save_gray_tiff(
2022-04-23T02:27:55.1831316Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 118, in save_gray_tiff
2022-04-23T02:27:55.1831588Z [2022-04-23T02:27:07.932Z]     imwrite(filename, img_gray, shape=img_gray.shape, photometric="rgb")
2022-04-23T02:27:55.1831891Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1832140Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1832447Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 2133, in write
2022-04-23T02:27:55.1832623Z [2022-04-23T02:27:07.932Z]     raise RuntimeError(
2022-04-23T02:27:55.1832997Z [2022-04-23T02:27:07.932Z] RuntimeError: None != 2 <StoredShape(frames=1, separate_samples=1, depth=1, length=16, width=16, contig_samples=1, extrasamples=0)>
2022-04-23T02:27:55.1833129Z [2022-04-23T02:27:07.932Z] 
2022-04-23T02:27:55.1833351Z [2022-04-23T02:27:07.932Z] ======================================================================
2022-04-23T02:27:55.1833614Z [2022-04-23T02:27:07.932Z] ERROR: test_read_malformats_0 (tests.test_wsireader.TestTiffFile)
2022-04-23T02:27:55.1833877Z [2022-04-23T02:27:07.932Z] ----------------------------------------------------------------------
2022-04-23T02:27:55.1834066Z [2022-04-23T02:27:07.932Z] Traceback (most recent call last):
2022-04-23T02:27:55.1834407Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/parameterized/parameterized.py", line 533, in standalone_func
2022-04-23T02:27:55.1834621Z [2022-04-23T02:27:07.932Z]     return func(*(a + p.args), **p.kwargs)
2022-04-23T02:27:55.1834975Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 194, in test_read_malformats
2022-04-23T02:27:55.1835165Z [2022-04-23T02:27:07.932Z]     file_path = save_gray_tiff(
2022-04-23T02:27:55.1835505Z [2022-04-23T02:27:07.932Z]   File "/home/jenkins/agent/workspace/MONAI-premerge/monai/tests/test_wsireader.py", line 118, in save_gray_tiff
2022-04-23T02:27:55.1835777Z [2022-04-23T02:27:07.932Z]     imwrite(filename, img_gray, shape=img_gray.shape, photometric="rgb")
2022-04-23T02:27:55.1836074Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 989, in imwrite
2022-04-23T02:27:55.1836320Z [2022-04-23T02:27:07.932Z]     result = tif.write(data, shape=shape, dtype=dtype, **kwargs)
2022-04-23T02:27:55.1836621Z [2022-04-23T02:27:07.932Z]   File "/opt/conda/lib/python3.8/site-packages/tifffile/tifffile.py", line 2133, in write
2022-04-23T02:27:55.1836852Z [2022-04-23T02:27:07.932Z]     raise RuntimeError(
2022-04-23T02:27:55.1837223Z [2022-04-23T02:27:07.932Z] RuntimeError: None != 2 <StoredShape(frames=1, separate_samples=1, depth=1, length=16, width=16, contig_samples=1, extrasamples=0)>

Nic-Ma · 2022-04-23T09:55:58Z

BTW, I saw you added new test files as "XXX_new.py", would you rename the file name later to remove the "_new" postfix?

Thanks.

Nic-Ma · 2022-04-23T14:24:14Z

Hi @drbeh ,

Above CI tests error is because Tifffile package released a new version yesterday and has some breaking changes.
I slightly changed the CI test data and arg name, fixed it in PR: #4162.

Thanks.

Nic-Ma · 2022-04-23T16:42:19Z

/build

* 4095 Add bundle download (#4114) * draft download Signed-off-by: Yiheng Wang <[email protected]> * update bundle download Signed-off-by: Yiheng Wang <[email protected]> * add url and load Signed-off-by: Yiheng Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename args and remove a few places Signed-off-by: Yiheng Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix flake8 issue Signed-off-by: Yiheng Wang <[email protected]> * enhance with reviews Signed-off-by: Yiheng Wang <[email protected]> * add instantiate for load Signed-off-by: Yiheng Wang <[email protected]> * fix black error Signed-off-by: Yiheng Wang <[email protected]> * add unittest Signed-off-by: Yiheng Wang <[email protected]> * add load to docs Signed-off-by: Yiheng Wang <[email protected]> * add skip Signed-off-by: Yiheng Wang <[email protected]> * add schemaerror Signed-off-by: Yiheng Wang <[email protected]> * fix partial places Signed-off-by: Yiheng Wang <[email protected]> * download zip bundle Signed-off-by: Yiheng Wang <[email protected]> * [DLMED] restore Exception for test Signed-off-by: Nic Ma <[email protected]> * update ts features Signed-off-by: Yiheng Wang <[email protected]> * add config_files test case Signed-off-by: Yiheng Wang <[email protected]> * enhance docstring example for args_file Signed-off-by: Yiheng Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nic Ma <[email protected]> * Disable pylint error and fix CI tests of new tifffile (#4162) * workaround Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix tifffile issue Signed-off-by: Nic Ma <[email protected]> * Fixed an error in DiNTS model implementation and enabled act and norm layer options (#4157) * fixed a bug Signed-off-by: dongy <[email protected]> * autofix Signed-off-by: dongy <[email protected]> * update test case Signed-off-by: dongy <[email protected]> Co-authored-by: dongy <[email protected]> * Split transform (#4153) * Redesign whole slide image reading (#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <[email protected]> * Add unittests for WSIReader Signed-off-by: Behrooz <[email protected]> * Add image mode for output validation Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Update references to new WSIReader Signed-off-by: Behrooz <[email protected]> * Remove legacy WSIReader Signed-off-by: Behrooz <[email protected]> * Update unittests Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * sort imports Signed-off-by: Behrooz <[email protected]> * Clean up imports Signed-off-by: Behrooz <[email protected]> * Update docstrings Signed-off-by: Behrooz <[email protected]> * Update docs and docstrings Signed-off-by: Behrooz <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <[email protected]> * Remove redundant checking Signed-off-by: Behrooz <[email protected]> * Update read and other methods Signed-off-by: Behrooz <[email protected]> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <[email protected]> * Make workaround for CuImage objects Signed-off-by: Behrooz <[email protected]> * Add unittests for multi image reading Signed-off-by: Behrooz <[email protected]> * Update a note about cucim Signed-off-by: Behrooz <[email protected]> * Update type hints and docstrings Signed-off-by: Behrooz <[email protected]> * Implement Split transform Signed-off-by: Behrooz <[email protected]> * Add unittests Signed-off-by: Behrooz <[email protected]> * Update formatting Signed-off-by: Behrooz <[email protected]> * Implement SplitDict Signed-off-by: Behrooz <[email protected]> * Add unittests for SplitDict Signed-off-by: Behrooz <[email protected]> * Add docs Signed-off-by: Behrooz <[email protected]> * Remove images from docs Signed-off-by: Behrooz <[email protected]> * Address all comments Signed-off-by: Behrooz <[email protected]> * Add example and size check Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Revert references to new wsireader Signed-off-by: Behrooz <[email protected]> * Add missing comma Signed-off-by: Behrooz <[email protected]> * fix bundle download test issue (#4169) Signed-off-by: Yiheng Wang <[email protected]> * 4094 Enhance `ckpt_export` to save config files (#4159) * [DLMED] enhance checkpoint export Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to comments Signed-off-by: Nic Ma <[email protected]> * Move RGB/RGBA checks to base class (#4171) Signed-off-by: Behrooz <[email protected]> Co-authored-by: Nic Ma <[email protected]> * [CICD] To support temp dgx runner (#4175) * Support new temp dgx runner Signed-off-by: Peixin Li <[email protected]> * atol 1e-5 Signed-off-by: Wenqi Li <[email protected]> Co-authored-by: Wenqi Li <[email protected]> * Test fix for AMP kwargs (#4178) Signed-off-by: Eric Kerfoot <[email protected]> Co-authored-by: Yiheng Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nic Ma <[email protected]> Co-authored-by: dongyang0122 <[email protected]> Co-authored-by: dongy <[email protected]> Co-authored-by: Behrooz <[email protected]> Co-authored-by: Peixin <[email protected]> Co-authored-by: Eric Kerfoot <[email protected]>

* Redesign whole slide image reading (Project-MONAI#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <[email protected]> * Add unittests for WSIReader Signed-off-by: Behrooz <[email protected]> * Add image mode for output validation Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Update references to new WSIReader Signed-off-by: Behrooz <[email protected]> * Remove legacy WSIReader Signed-off-by: Behrooz <[email protected]> * Update unittests Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * sort imports Signed-off-by: Behrooz <[email protected]> * Clean up imports Signed-off-by: Behrooz <[email protected]> * Update docstrings Signed-off-by: Behrooz <[email protected]> * Update docs and docstrings Signed-off-by: Behrooz <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <[email protected]> * Remove redundant checking Signed-off-by: Behrooz <[email protected]> * Update read and other methods Signed-off-by: Behrooz <[email protected]> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <[email protected]> * Make workaround for CuImage objects Signed-off-by: Behrooz <[email protected]> * Add unittests for multi image reading Signed-off-by: Behrooz <[email protected]> * Update a note about cucim Signed-off-by: Behrooz <[email protected]> * Update type hints and docstrings Signed-off-by: Behrooz <[email protected]> * Implement Split transform Signed-off-by: Behrooz <[email protected]> * Add unittests Signed-off-by: Behrooz <[email protected]> * Update formatting Signed-off-by: Behrooz <[email protected]> * Implement SplitDict Signed-off-by: Behrooz <[email protected]> * Add unittests for SplitDict Signed-off-by: Behrooz <[email protected]> * Add docs Signed-off-by: Behrooz <[email protected]> * Remove images from docs Signed-off-by: Behrooz <[email protected]> * Address all comments Signed-off-by: Behrooz <[email protected]> * Add example and size check Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Revert references to new wsireader Signed-off-by: Behrooz <[email protected]> * Add missing comma Signed-off-by: Behrooz <[email protected]>

bhashemian added 8 commits April 19, 2022 10:28

Implement Split transform

14b189a

Signed-off-by: Behrooz <[email protected]>

Add unittests

0ffd9dc

Signed-off-by: Behrooz <[email protected]>

Update formatting

a6c0209

Signed-off-by: Behrooz <[email protected]>

Implement SplitDict

651c6bf

Signed-off-by: Behrooz <[email protected]>

Add unittests for SplitDict

7bb2335

Signed-off-by: Behrooz <[email protected]>

Merge branch 'dev' of github.com:Project-MONAI/MONAI into split-trans…

c2b602c

…form

Add docs

dfb2ecd

Signed-off-by: Behrooz <[email protected]>

bhashemian marked this pull request as ready for review April 20, 2022 19:05

bhashemian requested review from Nic-Ma and wyli April 20, 2022 19:06

Nic-Ma reviewed Apr 21, 2022

View reviewed changes

bhashemian added 6 commits April 22, 2022 12:39

Merge dev branch

befc178

Signed-off-by: Behrooz <[email protected]>

Remove images from docs

072203d

Signed-off-by: Behrooz <[email protected]>

Address all comments

f712f8a

Signed-off-by: Behrooz <[email protected]>

Merge branch 'dev' into split-transform

5b99ee4

Add example and size check

bf55c05

Signed-off-by: Behrooz <[email protected]>

Update docs

630eb2a

Signed-off-by: Behrooz <[email protected]>

bhashemian added 2 commits April 22, 2022 16:13

Revert references to new wsireader

cc97cc1

Signed-off-by: Behrooz <[email protected]>

Add missing comma

ab23363

Signed-off-by: Behrooz <[email protected]>

bhashemian enabled auto-merge (squash) April 22, 2022 16:16

bhashemian requested a review from Nic-Ma April 22, 2022 16:22

Nic-Ma approved these changes Apr 22, 2022

View reviewed changes

bhashemian added 2 commits April 22, 2022 16:45

Fix size

1b68e48

Signed-off-by: Behrooz <[email protected]>

Merge branch 'dev' into split-transform

388bc9e

Merge branch 'dev' into split-transform

7b81217

bhashemian merged commit 0cdec83 into Project-MONAI:dev Apr 23, 2022

bhashemian deleted the split-transform branch February 22, 2023 15:31

Split transform #4153

Split transform #4153

Uh oh!

Conversation

bhashemian commented Apr 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Status

Types of changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 22, 2022

Uh oh!

Nic-Ma left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bhashemian commented Apr 22, 2022

Uh oh!

Nic-Ma commented Apr 23, 2022

Uh oh!

Nic-Ma commented Apr 23, 2022

Uh oh!

Nic-Ma commented Apr 23, 2022

Uh oh!

Nic-Ma commented Apr 23, 2022

Uh oh!

Nic-Ma commented Apr 23, 2022

Uh oh!

Nic-Ma commented Apr 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bhashemian commented Apr 20, 2022 •

edited

Loading

Nic-Ma left a comment •

edited

Loading