Extend benchmarks with computation of scores for NR and DB metrics

[snk4tr]

Closes #265

Proposed Changes

• Add computation of correlation scores for no-reference metrics on all datasets
• Add computation of correlation scores for distribution-based (DB) metrics on all datasets
• Add description of how DB metrics are computed + reference to the paper on IQA for MRI

[codecov]

Codecov Report

Merging #270 (797502a) into master (de38340) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #270   +/-   ##
=======================================
  Coverage   94.04%   94.04%           
=======================================
  Files          34       34           
  Lines        2485     2485           
=======================================
  Hits         2337     2337           
  Misses        148      148           

Flag	Coverage Δ
unittests	94.04% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

[snk4tr]

Ready for review.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[jzakirov]

I like the implementation, thanks @snk4tr
Let's add benchmarks results into README table and also some explanation on how values for distribution-based metrics are computed.

[snk4tr]

@zakajd about including results on PIPAL to the table: the paper on the PIPAL dataset has the following table:

The problem with this table is that there is no single column for evaluation of metrics on all types of artefacts. Hence, we need to either

• Select only one type of artefacts and use it for comparison between reference numbers and our results or
• Compute the reference score for all types of artefacts, average it and provide in the table without any reference or
• Not add PIPAL in the comparison table

At this point in time I prefer the first option but the the question of which distortion type to choose arises. Personally, I think that so called "Traditional distortions" (Gaussian noise, blur etc.) is the most logical choice but I would like to know what you think.

[snk4tr]

I also think that we should not add values for distribution-based metrics in the table even though the benchmarking tool lets us to compute them. The reason is that the main purpose of the table in the README file is to provide a reference on how close our implementations of metrics are to the original ones. At this point in time, there are no reference values for distribution-based metrics, which means that there is nothing to compare with.

[jzakirov]

@snk4tr PIPAL paper is a bit confusing. They actually do evaluate metrics on all distortions (see Fig. 4), but don’t publish numbers. In their next work with SWDN metric (https://arxiv.org/pdf/2011.15002.pdf) they post some results in Table 5.

So we can

1. Ask authors to provide the data
2. Use available data from SWDN paper

[snk4tr]

@zakajd I sent a request to provide the data. I see that values from the SWDN metric paper are already there. I will add our values for all metrics in this PR. Hopefully the authors will kindly provide us the data soon.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[denproc]

Great work. Just some minor changes.

Line 200: I would change it to "Feature Based"

[snk4tr]

@denproc @zakajd ready for review.

[denproc]

LGTM, just a minor change to make documentation consistent

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication