Constant cache issues

[neworderofjamie]

As discussed in https://github.com/orgs/genn-team/discussions/586, constant cache still seems to be overflowing with relatively complex models. Possibilities:

• cudaDeviceProp.totalConstMem is returning something different
• CUDA is using more constant cache itself somewhere (see Problem with constant cache allocation #352 for previous analysis)
  • Oddly increasing the constantCacheOverhead doesn't seem to solve this problem for all model sizes which is worrying
• There's a bug in GeNN's constant cache usage calculation

[neworderofjamie]

Examining object files provided by user using cuobjdump --dump-resource-usage XXXX.o, it appears that each object uses the following amount of constant cache:

neuronUpdate.o = 1928 bytes
synapseUpdate.o = 63576 bytes
init.o = 72 bytes
customUpdate.o = 72 bytes
runner.o = 72 bytes

= 65720 bytes which is more than 64K

This suggests that CUDA itself is still only using 72 bytes per module so constantCacheOverhead is still correct

[neworderofjamie]

I thought this was a CUDA 12 issues but it happens on my machine with CUDA 11.8 which is both good and bad

neuronUpdate
CONSTANT[3]:344

synapseUpdate
CONSTANT[3]:65032

init
CONSTANT[3]:72

customUpdate
CONSTANT[3]:72

runner
CONSTANT[3]:72

= 65592 = 0x10038 bytes which is not only more than 64K but also corresponds to the number in the error nvlink error : File uses too much global constant data (0x10038 bytes, 0x10000 max) .

Extracting the sizes of the symbols in the constant cache (which ones are here can be determined by using cuobjdump --dump-elf-symbols XXX.o) obtained from cuobjdump --dump-elf XXX.o gives:

size	name	size dec
0x3900	d_mergedPresynapticUpdateGroup0	14592
0xab00	d_mergedPresynapticUpdateGroup1	43776
0x500	d_mergedPresynapticUpdateGroup3	1280
0x400	d_mergedPresynapticUpdateGroup5	1024
0x180	d_mergedPresynapticUpdateGroupStartID0	384
0x480	d_mergedPresynapticUpdateGroupStartID1	1152
0x180	d_mergedPresynapticUpdateGroupStartID2	384
0x40	d_mergedPresynapticUpdateGroupStartID3	64
0x80	d_mergedPresynapticUpdateGroupStartID4	128
0x40	d_mergedPresynapticUpdateGroupStartID5	64
0x40	d_mergedPresynapticUpdateGroupStartID6	64
0x80	d_mergedPostsynapticUpdateGroupStartID0	128
0x180	d_mergedPostsynapticUpdateGroupStartID1	384
0x480	d_mergedPostsynapticUpdateGroupStartID2	1152
0x180	d_mergedPostsynapticUpdateGroupStartID3	384
0x30	d_mergedNeuronUpdateGroupStartID0	48
0x40	d_mergedNeuronUpdateGroupStartID1	64
0x10	d_mergedNeuronUpdateGroupStartID2	16
0x10	d_mergedNeuronUpdateGroupStartID3	16
0x40	d_mergedNeuronPrevSpikeTimeUpdateGroupStartID0	64
0x40	d_mergedNeuronPrevSpikeTimeUpdateGroupStartID1	64

Which gives 65232 bytes and, after adding 72*5 bytes for the CUDA LUTs in each module, gives 65592=0x10038 which matches the error message so definitely another bug in GeNN's constant cache usage calculation 😞

[neworderofjamie]

Doling out of the constant cache to the merged groups is pretty fool-proof but, based on the above, there are 4560 bytes of indices (as well as the 72*5 bytes of CUDA LUTs) so only up to 60616 bytes should be available for merged groups. However, GeNN was calculating 60744 bytes due to not including d_mergedNeuronPrevSpikeTimeUpdateGroupStartIDXXX