Attempting to create a variable using a compound with size larger or equal than 2**16 bytes fails

[smndvgnc]

Hi,

I recently ran into this issue with the python API (see this).
It appears it is coming from the C library, so I wrote this code snippet to verify.

#include <stdio.h>
#include <netcdf.h>

#define FILE_NAME "output.nc"
#define FILT_SIZE 8192

typedef struct Filter{
    double values[FILT_SIZE];
} Filter;

int main() {
    int ncid; // NetCDF file ID
    int dimid; // Dimension ID
    int varid; // Variable ID
    Filter filters;

    // Create NetCDF file
    if (nc_create(FILE_NAME, NC_NETCDF4, &ncid) != NC_NOERR) {
        fprintf(stderr, "Error creating NetCDF file.\n");
        return 1;
    }

    // Define a dimension
    if (nc_def_dim(ncid, "dim", 1, &dimid) != NC_NOERR) {
        fprintf(stderr, "Error defining dimension.\n");
        return 1;
    }

    // Define a compound type
    nc_type compound_type;
    if (nc_def_compound(ncid, sizeof(Filter), "Filter", &compound_type) != NC_NOERR) {
        fprintf(stderr, "Error defining compound type.\n");
        return 1;
    }
    int nelts = FILT_SIZE;
    if (nc_insert_array_compound(ncid, compound_type, "values", NC_COMPOUND_OFFSET(Filter, values), NC_DOUBLE, 1, &nelts) != NC_NOERR) {
        fprintf(stderr, "Error inserting member into compound type.\n");
        return 1;
    }

    // Define the variable using the compound type
    if (nc_def_var(ncid, "variable", compound_type, 1, &dimid, &varid) != NC_NOERR) {
        fprintf(stderr, "Error defining variable.\n");
        return 1;
    }

    // Write the data
    if (nc_put_var(ncid, varid, &filters) != NC_NOERR) {
        fprintf(stderr, "Error writing data.\n");
        return 1;
    }

    // Close the NetCDF file
    if (nc_close(ncid) != NC_NOERR) {
        fprintf(stderr, "Error closing NetCDF file.\n");
        return 1;
    }

    printf("NetCDF file created successfully.\n");
    return 0;
}

This code fails at runtime with libnetcdf>=4.9.0. If you modify FILT_SIZE to have a value lower or equal to 8191 (2**16 - 1 bytes), it works and produces a valid output file.
With libnetcdf=4.8.1, the code works no matter what the FILT_SIZE is.

[smndvgnc]

Forgot to mention I tested this on both:

• macOS Ventura (up-to-date)
• Ubuntu 18.04

The installation itself of libnetcdf was done with mamba.

[DennisHeimbigner]

I have just now gotten to this. I can confirm the error.
I created a pure HDF5 equivalent and it worked,
so the error is somewhere in the netcdf-c library.

[DennisHeimbigner]

I have been working on this with no luck.