Dropout inference performance

[eric-haibin-lin]

Dropout is a commonly used component for NN training. Currently it is however, not well optimized for inference. Currently for inference, the dropout op produces two outputs (out, mask), and copies the input to one of them (out). Ideally for inference we should

1. check kWriteInplace and avoid the copy
2. produces a single output instead of two

For (1), it should be straightforward - just check req in the op implementation (like the nd.identity op https://github.com/apache/incubator-mxnet/blob/30655f9ef58daa778d6ea049940068f0ff9a21bf/src/operator/tensor/elemwise_unary_op.h#L305). For (2), however, we need to know runtime (is_train=False/True) information, which is not available when inferring the number of outputs in the op. @junrushao1994 we should also consider this information for backend op registration.

cc @TaoLv @pengzhao-intel

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Feature

[eric-haibin-lin]

We can probably add a graph pass for bind(is_train=False).
cc @szha