Skip to content

bug/gmsd: Add alpha parameter from the paper#162

Merged
snk4tr merged 7 commits intomasterfrom
bug/gmsd
Sep 8, 2020
Merged

bug/gmsd: Add alpha parameter from the paper#162
snk4tr merged 7 commits intomasterfrom
bug/gmsd

Conversation

@jzakirov
Copy link
Collaborator

@jzakirov jzakirov commented Aug 4, 2020

Partially closes #161

Proposed Changes

  • After comparing current implementation with original paper I found missing alpha term in gradient similarity maps computation. Adding it slightly increases overall performance for both MS-GMSD and MS-GMSDc, but it's still far from values in the paper
  • Paper makes no reference about range of image values used in computations. After changing input range from 1.0 to 255.0 performance dropped by ~30%, so I assume that values should be in [0, 1]
  • There is also no reference about constant value in similarity maps computation. Currently we are using 0.0026 as in original GMSD paper. It's unlikely that it's so important for the final result.

@jzakirov jzakirov marked this pull request as ready for review August 11, 2020 09:47
@jzakirov
Copy link
Collaborator Author

I looked through paper again and couldn't find anything else, that's missing.
@denproc do you want to take a look at that?

We can merge this small fix, but issue with performance degradation will be left opened.

@codecov
Copy link

codecov bot commented Aug 11, 2020

Codecov Report

Merging #162 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #162   +/-   ##
=======================================
  Coverage   94.81%   94.81%           
=======================================
  Files          25       25           
  Lines        1697     1698    +1     
=======================================
+ Hits         1609     1610    +1     
  Misses         88       88           
Flag Coverage Δ
#unittests 94.81% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
piq/functional/base.py 90.62% <100.00%> (ø)
piq/gmsd.py 100.00% <100.00%> (ø)

@denproc
Copy link
Collaborator

denproc commented Aug 14, 2020

@zakajd

  1. Do you mean GMSD instead of MS_GMSD? In Potential bug in MS-GMSDc #161 you mentioned GMSD
  2. Making the pixel values in [0,255] requires the constant for similarity map T to be equal 170

@jzakirov
Copy link
Collaborator Author

@denproc Ready to be merged after test pass

@denproc
Copy link
Collaborator

denproc commented Aug 20, 2020

@zakajd, I tested MS-GMSD on our test images. I got the same result for both [0,1] and [0,255] ranges to be used inside the multi_scale_gmsd. Also, added alpha parameter to MultiScaleGMSDLoss.
Do you get the same results (let's discuss it after your vacation 🙃)?

@jzakirov
Copy link
Collaborator Author

jzakirov commented Sep 8, 2020

Hey, @denproc
It took me a bit longer to take a look at this PR. Your changes make results bad again.
MS-GMSD: SRCC 0.812, KRCC 0.645
MS-GMSDc: SRCC 0.814, KRCC 0.647

This is expected, because there are other range-specific values, such as beta1, beta2 etc.
I changed range to be in [0, 255] for multiscale and it works fine again.
MS-GMSD: SRCC 0.812, KRCC 0.645
MS-GMSDc: SRCC 0.887, KRCC 0.710
Let's keep it this way. Alpha parameter should be the same, because it's range invariant.

@snk4tr ready for review and merge.

@sonarqubecloud
Copy link

sonarqubecloud bot commented Sep 8, 2020

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities (and Security Hotspot 0 Security Hotspots to review)
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@snk4tr snk4tr merged commit 477f0f7 into master Sep 8, 2020
@snk4tr snk4tr deleted the bug/gmsd branch September 8, 2020 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Potential bug in MS-GMSDc

3 participants