Redesign whole slide image reading #4107

bhashemian · 2022-04-12T01:20:13Z

Description

This PR is a re-design and re-implementation of WSIReader while keeping the API the same as before and provide full backward compatibility (support for all the backends will come in the next PRs). This PR implements BaseWSIReader, WSIReader, and CuCIMWSIReader.

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: Behrooz <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: Behrooz <[email protected]>

…ader

monai/data/wsi_reader.py

Signed-off-by: Behrooz <[email protected]>

…sireader

monai/data/wsi_reader.py

Signed-off-by: Behrooz <[email protected]>

bhashemian · 2022-04-18T15:02:07Z

Hi @Nic-Ma, do you have any other comments here? Thanks

Signed-off-by: Behrooz <[email protected]>

Nic-Ma

Hi @drbeh ,

Thanks for your quick update!
I put some minor comments inline, others look good to me.

Thanks.

tests/test_wsireader.py

monai/data/wsi_reader.py

Signed-off-by: Behrooz <[email protected]>

…sireader

bhashemian · 2022-04-19T14:27:31Z

Hi @Nic-Ma, thanks for your comments. I have addressed them all and will merge this PR but please let me know if you have any comment and I will include it in the following PRs.

This reverts commit 78340a9.

Nic-Ma · 2022-04-19T14:46:27Z

Cool, thanks for your quick update.
Looks good to me.

* Redesign whole slide image reading (#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <[email protected]> * Add unittests for WSIReader Signed-off-by: Behrooz <[email protected]> * Add image mode for output validation Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Update references to new WSIReader Signed-off-by: Behrooz <[email protected]> * Remove legacy WSIReader Signed-off-by: Behrooz <[email protected]> * Update unittests Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * sort imports Signed-off-by: Behrooz <[email protected]> * Clean up imports Signed-off-by: Behrooz <[email protected]> * Update docstrings Signed-off-by: Behrooz <[email protected]> * Update docs and docstrings Signed-off-by: Behrooz <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <[email protected]> * Remove redundant checking Signed-off-by: Behrooz <[email protected]> * Update read and other methods Signed-off-by: Behrooz <[email protected]> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <[email protected]> * Make workaround for CuImage objects Signed-off-by: Behrooz <[email protected]> * Add unittests for multi image reading Signed-off-by: Behrooz <[email protected]> * Update a note about cucim Signed-off-by: Behrooz <[email protected]> * Update type hints and docstrings Signed-off-by: Behrooz <[email protected]> * Implement Split transform Signed-off-by: Behrooz <[email protected]> * Add unittests Signed-off-by: Behrooz <[email protected]> * Update formatting Signed-off-by: Behrooz <[email protected]> * Implement SplitDict Signed-off-by: Behrooz <[email protected]> * Add unittests for SplitDict Signed-off-by: Behrooz <[email protected]> * Add docs Signed-off-by: Behrooz <[email protected]> * Remove images from docs Signed-off-by: Behrooz <[email protected]> * Address all comments Signed-off-by: Behrooz <[email protected]> * Add example and size check Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Revert references to new wsireader Signed-off-by: Behrooz <[email protected]> * Add missing comma Signed-off-by: Behrooz <[email protected]>

* 4095 Add bundle download (#4114) * draft download Signed-off-by: Yiheng Wang <[email protected]> * update bundle download Signed-off-by: Yiheng Wang <[email protected]> * add url and load Signed-off-by: Yiheng Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename args and remove a few places Signed-off-by: Yiheng Wang <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix flake8 issue Signed-off-by: Yiheng Wang <[email protected]> * enhance with reviews Signed-off-by: Yiheng Wang <[email protected]> * add instantiate for load Signed-off-by: Yiheng Wang <[email protected]> * fix black error Signed-off-by: Yiheng Wang <[email protected]> * add unittest Signed-off-by: Yiheng Wang <[email protected]> * add load to docs Signed-off-by: Yiheng Wang <[email protected]> * add skip Signed-off-by: Yiheng Wang <[email protected]> * add schemaerror Signed-off-by: Yiheng Wang <[email protected]> * fix partial places Signed-off-by: Yiheng Wang <[email protected]> * download zip bundle Signed-off-by: Yiheng Wang <[email protected]> * [DLMED] restore Exception for test Signed-off-by: Nic Ma <[email protected]> * update ts features Signed-off-by: Yiheng Wang <[email protected]> * add config_files test case Signed-off-by: Yiheng Wang <[email protected]> * enhance docstring example for args_file Signed-off-by: Yiheng Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nic Ma <[email protected]> * Disable pylint error and fix CI tests of new tifffile (#4162) * workaround Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix tifffile issue Signed-off-by: Nic Ma <[email protected]> * Fixed an error in DiNTS model implementation and enabled act and norm layer options (#4157) * fixed a bug Signed-off-by: dongy <[email protected]> * autofix Signed-off-by: dongy <[email protected]> * update test case Signed-off-by: dongy <[email protected]> Co-authored-by: dongy <[email protected]> * Split transform (#4153) * Redesign whole slide image reading (#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <[email protected]> * Add unittests for WSIReader Signed-off-by: Behrooz <[email protected]> * Add image mode for output validation Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Update references to new WSIReader Signed-off-by: Behrooz <[email protected]> * Remove legacy WSIReader Signed-off-by: Behrooz <[email protected]> * Update unittests Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * sort imports Signed-off-by: Behrooz <[email protected]> * Clean up imports Signed-off-by: Behrooz <[email protected]> * Update docstrings Signed-off-by: Behrooz <[email protected]> * Update docs and docstrings Signed-off-by: Behrooz <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <[email protected]> * Remove redundant checking Signed-off-by: Behrooz <[email protected]> * Update read and other methods Signed-off-by: Behrooz <[email protected]> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <[email protected]> * Make workaround for CuImage objects Signed-off-by: Behrooz <[email protected]> * Add unittests for multi image reading Signed-off-by: Behrooz <[email protected]> * Update a note about cucim Signed-off-by: Behrooz <[email protected]> * Update type hints and docstrings Signed-off-by: Behrooz <[email protected]> * Implement Split transform Signed-off-by: Behrooz <[email protected]> * Add unittests Signed-off-by: Behrooz <[email protected]> * Update formatting Signed-off-by: Behrooz <[email protected]> * Implement SplitDict Signed-off-by: Behrooz <[email protected]> * Add unittests for SplitDict Signed-off-by: Behrooz <[email protected]> * Add docs Signed-off-by: Behrooz <[email protected]> * Remove images from docs Signed-off-by: Behrooz <[email protected]> * Address all comments Signed-off-by: Behrooz <[email protected]> * Add example and size check Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Revert references to new wsireader Signed-off-by: Behrooz <[email protected]> * Add missing comma Signed-off-by: Behrooz <[email protected]> * fix bundle download test issue (#4169) Signed-off-by: Yiheng Wang <[email protected]> * 4094 Enhance `ckpt_export` to save config files (#4159) * [DLMED] enhance checkpoint export Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to comments Signed-off-by: Nic Ma <[email protected]> * Move RGB/RGBA checks to base class (#4171) Signed-off-by: Behrooz <[email protected]> Co-authored-by: Nic Ma <[email protected]> * [CICD] To support temp dgx runner (#4175) * Support new temp dgx runner Signed-off-by: Peixin Li <[email protected]> * atol 1e-5 Signed-off-by: Wenqi Li <[email protected]> Co-authored-by: Wenqi Li <[email protected]> * Test fix for AMP kwargs (#4178) Signed-off-by: Eric Kerfoot <[email protected]> Co-authored-by: Yiheng Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nic Ma <[email protected]> Co-authored-by: dongyang0122 <[email protected]> Co-authored-by: dongy <[email protected]> Co-authored-by: Behrooz <[email protected]> Co-authored-by: Peixin <[email protected]> Co-authored-by: Eric Kerfoot <[email protected]>

* Redesign whole slide image reading (Project-MONAI#4107) * Redesign BaseWSIReader, WSIReader, CuCIMWSIReader Signed-off-by: Behrooz <[email protected]> * Add unittests for WSIReader Signed-off-by: Behrooz <[email protected]> * Add image mode for output validation Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Update references to new WSIReader Signed-off-by: Behrooz <[email protected]> * Remove legacy WSIReader Signed-off-by: Behrooz <[email protected]> * Update unittests Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * sort imports Signed-off-by: Behrooz <[email protected]> * Clean up imports Signed-off-by: Behrooz <[email protected]> * Update docstrings Signed-off-by: Behrooz <[email protected]> * Update docs and docstrings Signed-off-by: Behrooz <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix a typo Signed-off-by: Behrooz <[email protected]> * Remove redundant checking Signed-off-by: Behrooz <[email protected]> * Update read and other methods Signed-off-by: Behrooz <[email protected]> * Update wsireader to support multi image and update docstrings Signed-off-by: Behrooz <[email protected]> * Make workaround for CuImage objects Signed-off-by: Behrooz <[email protected]> * Add unittests for multi image reading Signed-off-by: Behrooz <[email protected]> * Update a note about cucim Signed-off-by: Behrooz <[email protected]> * Update type hints and docstrings Signed-off-by: Behrooz <[email protected]> * Implement Split transform Signed-off-by: Behrooz <[email protected]> * Add unittests Signed-off-by: Behrooz <[email protected]> * Update formatting Signed-off-by: Behrooz <[email protected]> * Implement SplitDict Signed-off-by: Behrooz <[email protected]> * Add unittests for SplitDict Signed-off-by: Behrooz <[email protected]> * Add docs Signed-off-by: Behrooz <[email protected]> * Remove images from docs Signed-off-by: Behrooz <[email protected]> * Address all comments Signed-off-by: Behrooz <[email protected]> * Add example and size check Signed-off-by: Behrooz <[email protected]> * Update docs Signed-off-by: Behrooz <[email protected]> * Revert references to new wsireader Signed-off-by: Behrooz <[email protected]> * Add missing comma Signed-off-by: Behrooz <[email protected]>

bhashemian added 4 commits April 12, 2022 01:18

Redesign BaseWSIReader, WSIReader, CuCIMWSIReader

2baa344

Signed-off-by: Behrooz <[email protected]>

Add unittests for WSIReader

e46fea0

Signed-off-by: Behrooz <[email protected]>

Add image mode for output validation

f946a63

Signed-off-by: Behrooz <[email protected]>

Merge branch 'dev' into new-wsireader

9b41019

bhashemian changed the base branch from dev to pathology April 12, 2022 20:44

bhashemian added 8 commits April 13, 2022 13:08

Update docs

9c15ea3

Signed-off-by: Behrooz <[email protected]>

Update references to new WSIReader

e004538

Signed-off-by: Behrooz <[email protected]>

Remove legacy WSIReader

df0a61e

Signed-off-by: Behrooz <[email protected]>

Update unittests

b64c087

Signed-off-by: Behrooz <[email protected]>

Update docs

7f09e73

Signed-off-by: Behrooz <[email protected]>

sort imports

d95e1c2

Signed-off-by: Behrooz <[email protected]>

Clean up imports

025376b

Signed-off-by: Behrooz <[email protected]>

Update docstrings

3e6fb27

Signed-off-by: Behrooz <[email protected]>

bhashemian marked this pull request as ready for review April 13, 2022 20:32

bhashemian requested review from Nic-Ma and wyli April 13, 2022 20:33

bhashemian and others added 5 commits April 13, 2022 20:36

Update docs and docstrings

b8a2444

Signed-off-by: Behrooz <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

0268a03

for more information, see https://pre-commit.ci

Fix a typo

f2b14aa

Signed-off-by: Behrooz <[email protected]>

Remove redundant checking

aed0ac5

Signed-off-by: Behrooz <[email protected]>

Merge branch 'new-wsireader' of github.com:drbeh/MONAI into new-wsire…

272d11f

…ader

Nic-Ma reviewed Apr 14, 2022

View reviewed changes

monai/data/wsi_reader.py Outdated Show resolved Hide resolved

monai/data/wsi_reader.py Show resolved Hide resolved

bhashemian added 2 commits April 14, 2022 17:34

Update read and other methods

ae4ff17

Signed-off-by: Behrooz <[email protected]>

Merge branch 'pathology' of github.com:Project-MONAI/MONAI into new-w…

e0bad98

…sireader

Nic-Ma reviewed Apr 15, 2022

View reviewed changes

monai/data/wsi_reader.py Outdated Show resolved Hide resolved

monai/data/wsi_reader.py Show resolved Hide resolved

monai/data/wsi_reader.py Show resolved Hide resolved

bhashemian added 3 commits April 18, 2022 14:43

Update wsireader to support multi image and update docstrings

e5a7a18

Signed-off-by: Behrooz <[email protected]>

Make workaround for CuImage objects

da6b675

Signed-off-by: Behrooz <[email protected]>

Add unittests for multi image reading

1ed8154

Signed-off-by: Behrooz <[email protected]>

Update a note about cucim

77f8747

Signed-off-by: Behrooz <[email protected]>

Nic-Ma reviewed Apr 19, 2022

View reviewed changes

bhashemian added 2 commits April 19, 2022 13:52

Update type hints and docstrings

36e5b17

Signed-off-by: Behrooz <[email protected]>

Merge branch 'pathology' of github.com:Project-MONAI/MONAI into new-w…

0328fe6

…sireader

bhashemian merged commit 78340a9 into Project-MONAI:pathology Apr 19, 2022

bhashemian added a commit that referenced this pull request Apr 19, 2022

Revert "Redesign whole slide image reading (#4107)"

36c77bb

This reverts commit 78340a9.

bhashemian mentioned this pull request Apr 19, 2022

New wsireader #4147

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Redesign whole slide image reading #4107

Redesign whole slide image reading #4107

Uh oh!

bhashemian commented Apr 12, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 18, 2022

Uh oh!

Nic-Ma left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 19, 2022

Uh oh!

Nic-Ma commented Apr 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Redesign whole slide image reading #4107

Redesign whole slide image reading #4107

Uh oh!

Conversation

bhashemian commented Apr 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 18, 2022

Uh oh!

Nic-Ma left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bhashemian commented Apr 19, 2022

Uh oh!

Nic-Ma commented Apr 19, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bhashemian commented Apr 12, 2022 •

edited

Loading