Use null pointer when allocating 0 memory for CUDA

[hobofan]

A CUDA cuMemAlloc returns an error when trying to allocate 0 bytes.

We should wrap the CUDA driver call so that trying to allocate 0 bytes returns a null pointer. Before implementing that it should be checked if providing cuMemFree with a null pointer is valid.

The recommended workaround for now is allocating 1 byte instead 0 bytes.

[bklooste]

Why is the caller making a mem_alloc request for nothing .. ?

Why not fault or return an option pointer ? An option pointer can also handle more errors like overflow but obviously you get the issue of fatal error vs non fatal.

[hobofan]

In collenchyma-nn the required memory size for some operations is calculated and if allocating 0 bytes leads to an error, we have to introduce some special handling. I am sure that there are a lot of similar use cases. The standard malloc also has a similar behaviour, of returning a null pointer.

I think on a high level we should mimic malloc for the usability aspect. Under the controlled circumstances we call it there, the only error that should be possible is a OOM, which should panic.
On a lower level we of course want to mimic the CUDA API and give people the possibility to shoot themselves in the foot when allocating 0 bytes.

[bklooste]

A 0 check is a triviality ,and that pattern does repeats a lot because most clients can not be trusted but the same argument is their for returning null . I'm not sure about mimicking malloc , Malloc does not need to deal with different architectures on one system , for native and SIMD your will definitely get into aligned_alloc(16 vs 64 vs other arch ) , you may also have 32 bit vs 64 bit support , numa alloc_local and a null 32 vs null 64 . So this will get more complicated later.

Null return can also be EINVAL for allign . It is also probably very useful to know where the out of memory came from , especially when using multiple devices ( GPUs) , or mixed GPU / Native.

Not saying they have to be here they can be in the next layer but it should be considered.

[hobofan]

I'm not sure about mimicking malloc , Malloc does not need to deal with different architectures on one system

With the CUDA driver API, given a specific context we also don't have to deal with different architectures, as far as I can see it.

It is also probably very useful to know where the out of memory came from , especially when using multiple devices ( GPUs) , or mixed GPU / Native.

Yeah, that makes sense.