Cleanup FixedPrecisionTensor

[iamtrask]

Describe the bug
FixedPrecision tensor has become a catch-all for a myraid of functionality which have nothing to do with fixing precision. At present it seems to include:

0. The ability to model Fixed Precision Numbers using underlying Integer representations
1. BigInt functionality allowing for large precision
2. Convolutional functionality
3. many references to specific subclass types (such as AdditiveSharingTensor).
4. an implementation of pow() which seems unrelated to FixedPrecisionRepresentation
5. Newton-shulz inversion
6. approximation of exponentiation
7. approximation of sigmoid
8. approximation of log
9. etc...

All functionality other than (0) should be removed from this class and refactored into appropriate other classes.

[iamtrask]

I will begin working on a properly decoupled BigInt tensor

[LaRiffle]

That's right!

1. So actually once you are in Fixed-precision, you can't do anymore pow, exp, simoid, log, inversion the way you did previously, so you have to redefine them, and I'd say this is the good place.
2. Less important, some functions like Convolutional should indeed be put in our dedicated nn module where rnn already lies.
3. Having references to AdditiveSharingTensor is super annoying, but from our design seems hard to avoid, since we want to operate tensors with different types (FPT & AST)

Actually, we could follow the design principle of another lib and define fixed-precision not as a tensor type but as an encoder which is attached to a tensor. This sounds weird but:

• It reconciliate FPT & AST in a single type and solves 3.
• It diminishes the number of tensors types and reduces the tensor chain size
• It is more readable for non-expert users

To avoid the limitation of having to redefine the encoder for the HomomorphicTensor, etc, encoder could be supplied by a inheritance mechanism, but then @karlhigley is not going to be happy lol

In a nutshell

• I don't think 1. is that of a problem
• Issue 2. can be solved easily
• Issue 3. needs more intelligence, to make for example the mul function of FPT readable

[karlhigley]

My only ask would be to supply the encoder as a constructor param with a default value (instead of as an additional parent class via inheritance.) 🙂

[iamtrask]

I don't see the semantic difference between an encoder and a tensor type (something used in our tensor chain).

…

On Sun, Jan 12, 2020 at 3:33 PM Karl Higley ***@***.***> wrote: My only ask would be to supply the encoder as a constructor param with a default value (instead of as an additional parent class via inheritance.) 🙂 — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2897?email_source=notifications&email_token=ABBAZESX7SJQJSF2TBSA6W3Q5MZ3VA5CNFSM4KFVASI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIW4VYI#issuecomment-573426401>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABBAZEXYAJMMHOXGJSAMKZ3Q5MZ3VANCNFSM4KFVASIQ> .

[karlhigley]

I'm not sure I fully understand it either, just chiming in on the point about inheritance. 😄

[LaRiffle]

The difference here is that using the encoder, you don't add the FPT node in the tensor chain, so you're simpler & faster. But this reflexion is far from being top priority for the moment^^

[iamtrask]

Relatively to the performance hit of doing secure computation I think you wouldn't wouldn't even notice the extra node in the tensor chain, but in exchange we get re-usable, more modular code. I'm not opposed to an encoder existing long term, but I have very strong convictions about making FPT it's own module because literally every encrypted computation tensor type will need it.

[ADMoreau]

I'll move the conv2d. Should I put it in the nn module or should I put it in some other class. If another class, are there any suggestions? I jsut want to make sure I'm sticking to a design strategy I may not be aware of.

[LaRiffle]

In nn is good!
But maybe conv2d has to transformed into a class, or maybe not, I let you choose the best option here :)

[iamtrask]

Important question conv2d get _exactly_ the same values in the same positions as torch's Conv2d? Aka, can I train with PyTorch's conv2d then swap it out for this one and have it automatically work? We've been working on an implementation which is this precise in nn, but it doesn't have all features yet.

…

On Mon, Jan 27, 2020 at 10:15 AM Théo Ryffel ***@***.***> wrote: In nn is good! But maybe conv2d has to transformed into a class, or maybe not, I let you choose the best option here :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2897?email_source=notifications&email_token=ABBAZES3L37ASBK5DPZ7B5LQ72X3JA5CNFSM4KFVASI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJ67EHY#issuecomment-578679327>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABBAZESEHSFC33QBUVHLZ2LQ72X3JANCNFSM4KFVASIQ> .

[ADMoreau]

Sure I'll give it a shot

[stephenjfox]

@ADMoreau Are you still working on this?
@iamtrask is this still relevant?