Is reporting the model diff supported?

[erksch]

Question

Hey there! Thanks for the awesome work on the Android front!

The demo works for training on the dataset but I can not see any code that is related to actually reporting the model diff to PyGrid. Is this already in the demo, or even supported by the library?

Regards

[vkkhare]

Hi @erksch reporting the diffs to pygrid is not yet functional. It should be fairly easy to add though. We have plans to add it just haven't implemented it.

[vkkhare]

If you need it on priority, you can do it without altering the kotlin code. Just create a plan by pysyft that evaluates the diff between two sets of tensors. You can then serialise this diff and send it to pygrid by the report function already present in SyftJob.

Ofcourse if you are interested, we welcome all the contributions

[erksch]

Thank you for your quick response!
What do you mean by not altering the kotlin code? I would say I'd need to calculate the diff somehow and wrap it into a State object. Like in this example in python:

diff = [ model_params[i] - updated_model_params[i] for i in range(len(model_params)) ]
diff_state = State(
    owner=hook.local_worker,
    state_placeholders=[PlaceHolder().instantiate(param) for param in diff]
)

As you mentioned SyftJob already has a report method and takes a diff State object and sends it to the websocket. But how do I create the diff in Kotlin? I think that would be easier than creating a plan for that.

[erksch]

Okay this is what I came up with

fun comptueDiff(originalState: State, newState: State): State {
  val diffTensors = mutableListOf<SyftTensor>()

  for (i in 0 until originalState.syftTensors.size) {
    val originalTensor = originalState.syftTensors[i].getTorchTensor()
    val originalFloats = originalTensor.dataAsFloatArray

    val newTensor = newState.syftTensors[i].getTorchTensor()
    val newFloats = newTensor.dataAsFloatArray

    val diffFloats = FloatArray(originalFloats.size)

    for (j in originalFloats.indices) {
      diffFloats[j] = originalFloats[j] - newFloats[j]
    }

    val diffTensor = Tensor.fromBlob(diffFloats, originalTensor.shape())
    val diffSyftTensor = SyftTensor.fromTorchTensor(diffTensor)
    diffTensors.add(diffSyftTensor)
 }

 return State(newState.placeholders, diffTensors)
}

In full action it looks like this:

fun training_process(...) {
  // Copy state in a way so it does not change during training
  val originalState = State(model.modelState!!.placeholders.toMutableList(), model.modelState!!.syftTensors.toMutableList())
  .... training ...
  val diff = comptueDiff(originalState, model.modelState!!)
  mnistJob.report(diff)
}

I can confirm that it works!

[vkkhare]

I actually wrote a method to evaluate a diff in Kotlin and it is quite similar to what you came up with. It's left commented in SyftModel because of slow speed.

https://github.com/OpenMined/KotlinSyft/blob/dev/syftlib/src/main/java/org/openmined/syft/proto/SyftModel.kt#L40

The problem of writing this in Kotlin is we have to sequentially iterate over all the indices and subtract them manually. This is way too slow if the model parameters' dimensions explode. There are few vectorization alternatives in Java like Bigdecimal but they only handle integer in substraction.
The only way I could find to have a highly parallelized implementation while being able to support all the syft tensor types was doing that via a diff plan. Yet another issue with such type casting is the possibility of memory overflow when upcasting float16 to float64 or byte to int
Thus, I am inclined towards writing a diff torchscript and storing it in library assets so that the user does not need to explicitly define it. This would have been a lot more easier if Kotlin had a numpy like library. Afaik Kotlin math is in active development but is farther in future

[erksch]

Okay, I see the problem, I didn't really think of performance yet. This stuff is much easier in python :D
If you already have some code in mind for creating a diffing plan it would be awesome to share a small notebook or add it somewhere in the repo.

[vkkhare]

Crudely, I would go about the lines of how a usual plan is implemented just removing all the custom arguments from it. https://github.com/OpenMined/PySyft/blob/v0.2.5/examples/experimental/FL%20Training%20Plan/Create%20Plan.ipynb

@sy.func2plan(args_shape=args_shape)
def diff_plan(*model_params):   # we can also feed in two lists if torchscript supports
    num_params = len(model_params)/2
    old_params = model_params[:num_params]
    new_params = model_params[num_params:]
    diff = [new - old for old,new in zip(old_params,new_params)]
    return (*diff)

[erksch]

Ah right, I thought the plan function would also include reporting the diff to the grid but this could just be done in Kotlin.

[vkkhare]

But looking at this, I feel it cannot be a static file after all and we would need to generate this plan based on the model (maybe while hosting the plan?). At Kotlin end, we should load the diff plan and syft model should execute that plan.