Compress RectAreaLight BRDF Data and include in three.js build files

[abelnation]

This issue can serve as discussion for improving the preliminary RectAreaLight implementation being reviewed in #9234.

Currently, the BRDF for the RectAreaLight implementation is approximated using float textures totaling ~80kb when the data is included as discrete, raw, float values. Methods to compress that data need to be explored in order for it to be viable to include the data directly in the three.js source.

Possible methods include:

• fit a 3d surface function to each of the five 2d float data sets
• compress the data (e.g. via a kmeans vectorization, or precision reduction)

@bhouston @WestLangley @tschw

[abelnation]

Here is an attempt using kmeans vectorization to reduce the data sets to 256 discrete values and store 1byte indices into those "codebooks". Original data and the vectorization can be seen in the screenshot below. The results are not great, resulting in some pretty serious banding artifacts.

Demo of results can be seen here: http://groovemechanic.net/three.js.compress/examples/#webgl_lights_rectarealight

[WestLangley]

I support what you are trying to do, but as I tried to explain elsewhere, your metric is in the wrong space, IMHO.

The goal is to approximate a BRDF with a particular LTC function. The LTC function has 5 parameters, which have been specified by the authors of the paper.

You are proposing an approach that results in a different set of 5 parameters. If you change the parameters, then the LTC function will change, and it may no longer approximated the BRDF correctly.

You don't want to measure how well you are fitting the 5 parameters. You need to measure how well you are fitting the BRDF with your modified method. Changes in the 5 parameters can potentially result in large changes in the BRDF -- or in an approximated BRDF that is not properly normalized.

In any event, thank you for all your efforts; they are much-appreciated. We will get there... : - )

[abelnation]

@WestLangley Your reasoning makes sense. TBH, this is an area where I'm rather inexperienced, though it's certainly been enjoyable to experiment.

Your reasoning essentially boils down to doing exactly what the authors of the paper did: for each angle of view, fit a linear transformation matrix to the basic clamped cosine distribution, such that the resulting function matches the GGX BRDF as closely as possible. I also noticed just now that the paper authors have published their fitting code, which is great.

The real question is, is there a better method to compress the resulting fitted data that is smaller than what the authors propose? They represent the BRDF fitting for each angle as the values in the linear transform matrix, 5 vals, where 5 vals are needed for each angle of view. What other approaches could be used to represent the same fitting without requiring as much data (while not completely deviating from the model proposed by the paper)?

[WestLangley]

@abelnation OK, we are on the same page. Data compression is not my area of expertise. If you choose to go down that path, we have to make sure your metric is a reasonable one. In the near term, my personal goal is to verify that you have correctly implemented the algorithm as described in the paper. Sorry it is taking a long time for me to get to it -- it is not for a lack of interest. : - )

[sam-g-steel]

@WestLangley @abelnation I've been looking into compressing the two tables and I think I may have found something! So far I've been playing around with an algorithm that produces no noticeable artifacts and reduces ltc_mag from 40k bytes to roughly 4k bytes of us this includes decoding logic. What is our target size? 20kb???

[sam-g-steel]

Note: the compression algrothem is neural net based. See... http://codepen.io/sam-g-steel/pen/GWMXjB?editors=1011. It takes about 2.5 million iterations to get good results

I've a three.js demo that will be on code pin soon too. I'll comment when available.

[mrdoob]

Note: the compression algrothem is neural net based. See... http://codepen.io/sam-g-steel/pen/GWMXjB?editors=1011. It takes about 2.5 million iterations to get good results

404 (Page Not Found)

[mrdoob]

This sounds super good though!

[sam-g-steel]

@mrdoob
Cool! about the 404... It looks like my pen was private
Still working on a few things.
Todo list...

• Compress single channel data ie... LTC_MAG
• Display compression error
• [wip] Compress multi-channel data, LTC_MAT
• finish uncompression code... It currently works but could be more efficient, smaller and faster
• Have a demo showing that this works and how much smaller the code is

All of the code in the pen is to encode the image
I also have code that will decode it

http://codepen.io/sam-g-steel/pen/GWMXjB

To get decent results it takes 2.5million iterations ~8hrs on dell xps 1550
Fortunatly decoding happens in the single diget milliseconds :)

[bhouston]

This is all brilliant and only possible because Three.JS is such an attractive open source project that it brings in so many with varied talents. Best regards, Ben Houston (Work: 613-422-4009, Cell: 613-762-4113, Skype: ben.exocortex, Twitter: @exocortexcom) http://exocortex.com | https://clara.io | http://threekit.com

…

On Sun, Mar 19, 2017 at 2:38 PM, Samuel Sylvester ***@***.***> wrote: @mrdoob <https://github.com/mrdoob> Cool! about the 404... It looks like my pen was private Still working on a few things. Todo list... - Compress single channel data ie... LTC_MAG - [wip] Compress multi-channel data, LTC_MAT - finish uncompression code... It currently works but could be more efficient, smaller and faster - Have a demo showing that this works and how much smaller the code is All of the code in the pen is to encode the image I also have code that will decode it http://codepen.io/sam-g-steel/pen/GWMXjB — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#9373 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAj6_cBVnO_w8Ijah7Vt1ydMr1wAOjm-ks5rnaBPgaJpZM4JSMWI> .

[WestLangley]

@sam-g-steel These are data textures and the data has meaning in the context of the BRDF we are using. What is the metric you are using as a measure of convergence? That fact that these images "look similar" is not sufficient.

I am assuming the uncompressed data is not the same as the original data. Is that a correct assumption?

[sam-g-steel]

@WestLangley
Yes, that is correct... the uncompressed data is not the same as the original table. But this could be a good thing, the table is limited to a 64x64 table of 16bit floats the uncompressed data is resolution independent and can easily be rendered to a 128, 256, or even 4096 square texture at 64bits per channel... Ok, we probably can't use that kind of precision, but you get my point... this might eventually yield more desirable results.

Also, I understand that just looking at the images and saying that they look similar is not good enough.

However, I have a THREE.js demo that I'm working on that will demonstrate the results that this kind of compression can yield. LTC_MAG compression is working well I just want to get LTC_MAT(much harder) working too before I publish the demo.

Also note, the longer I train the neural network the more accurate the results...
I'll have accuracy measures available soon, next few days...

It looks like this experiment is going to take me another week to finish...

[abelnation]

It's also important to note that the original data is *also* fitted to an ideal function, so the source data should not be viewed as a "pure" source of truth.

…

Sent from my phone

On Mar 21, 2017, at 11:16 AM, Samuel Sylvester ***@***.***> wrote: @WestLangley Yes, that is correct... the uncompressed data is not the same as the original table. Also, I am aware of how the data is used so I understand that just looking at the images and saying that they look similar is not good enough. However, I have a THREE.js demo that I'm working on a demo that will demonstrate the results that this kind of compression can yield. LTC_MAG compression is working well I just want to get LTC_MAT(much harder) working too before I publish the demo. Also note, the longer I train the neural network the more accurate the results... I'll have accuracy measures available soon, next few days... It looks like this experiment is going to take me another week to finish... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[sam-g-steel]

OK, the codepen now shows compression errors on an exponential scale... (1-(1-delta^10))2
I've also improved compression time and rate from 2.5 million iterations at 8hrs to 100k iterations at under 30min!