Skip to content

Enable ViTReciproCAM for VisionTransformer backbone (cls task)#2403

Merged
sovrasov merged 10 commits intoopen-edge-platform:developfrom
negvet:et/vit_reciprocam
Aug 3, 2023
Merged

Enable ViTReciproCAM for VisionTransformer backbone (cls task)#2403
sovrasov merged 10 commits intoopen-edge-platform:developfrom
negvet:et/vit_reciprocam

Conversation

@negvet
Copy link
Copy Markdown
Contributor

@negvet negvet commented Jul 26, 2023

Summary

Enable ViTReciproCAM to support cls models with VisionTransformer backbone, e.g. deit.

Can be merged, but cannot be actually used in Geti until #2098 is resoled (featute vector and saliency maps are coupled). Can be used right away only for torch runtime.

Validated in OTX:

  • Torch task (infer, explain APIs).
  • Export to OV IR.
  • OV task (infer (async, sync), explain APIs).

Torch task examples:
image

OV task examples:
image

Multilabel:
image

How to test

Checklist

  • I have added unit tests to cover my changes.​
  • I have added integration tests to cover my changes.​
  • I have added e2e tests for validation.
  • I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
  • I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
  • I have linked related issues.

License

  • I submit my code changes under the same Apache License that covers the project.
    Feel free to contact the maintainers if that's a concern.
  • I have updated the license header for each file (see an example below).
# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

@github-actions github-actions bot added ALGO Any changes in OTX Algo Tasks implementation TEST Any changes in tests labels Jul 26, 2023
@negvet negvet added FEATURE New feature & functionality TEST Any changes in tests and removed ALGO Any changes in OTX Algo Tasks implementation TEST Any changes in tests labels Jul 26, 2023
@negvet negvet added this to the 1.5.0 milestone Jul 26, 2023
@github-actions github-actions bot added the ALGO Any changes in OTX Algo Tasks implementation label Jul 31, 2023
@negvet negvet changed the title Support ViTReciproCAM for deit Enable ViTReciproCAM for deit Jul 31, 2023
@negvet negvet changed the title Enable ViTReciproCAM for deit Enable ViTReciproCAM for VisionTransformer backbone (cls task) Jul 31, 2023
@negvet negvet marked this pull request as ready for review July 31, 2023 15:05
@negvet negvet requested a review from a team as a code owner July 31, 2023 15:05
@jaegukhyun jaegukhyun requested a review from wonjuleee August 1, 2023 01:30
jaegukhyun
jaegukhyun previously approved these changes Aug 1, 2023
Copy link
Copy Markdown
Contributor

@jaegukhyun jaegukhyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@eunwoosh eunwoosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your effort :) I left some comments. please take a look.

@negvet negvet requested a review from sovrasov August 1, 2023 15:29
jaegukhyun
jaegukhyun previously approved these changes Aug 2, 2023
@sovrasov sovrasov merged commit 171d614 into open-edge-platform:develop Aug 3, 2023
@negvet negvet deleted the et/vit_reciprocam branch August 3, 2023 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ALGO Any changes in OTX Algo Tasks implementation FEATURE New feature & functionality TEST Any changes in tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants