Trim does not trim trailing whitespace

[nbauernfeind]

Description

Trailing white space remains even when Trim is set to true.

Steps to reproduce

Load the attached csv into a dh-core repl:

from deephaven import read_csv
t = read_csv("test.csv", trim=True, delimiter=",")

Expected results

I'd expect the Copy Cell Unformatted (right-click dropdown menu on a table cell in the deephaven-core web-ui) to demonstrate that all cells are properly trimmed.

Actual results

The 2nd and 3rd rows for all three columns have trailing whitespace.

test.csv

[nbauernfeind]

@kosak it looks like we need to add trim support to processUnquotedMode.

    public boolean grabNext(final ByteSlice dest, final MutableBoolean lastInRow)
            throws CsvReaderException {
        spillBuffer.clear();
        startOffset = offset;

        if (ignoreSurroundingSpaces) {
            skipWhitespace();
        }
        if (!tryEnsureMore()) {
            return false;
        }

        // Is first char the quote char?
        if (buffer[offset] == quoteChar) {
            ++offset;
            processQuotedMode(dest, lastInRow);
            if (trim) {
                trimWhitespace(dest);
            }
        } else {
            processUnquotedMode(dest, lastInRow);
        }
        return true;
    }

It seem that this ignores the leading whitespace:

        if (ignoreSurroundingSpaces) {
            skipWhitespace();
        }

However, in processUnquotedMode(dest, lastInRow); we have no code path to drop the trailing whitespace.

Perhaps the intended solution was for skipWhitespace(); to also remove trailing spaces?

[kosak]

Hi, thanks for the bug report. The issue has been fixed in Deephaven Core 0.27.1 and Deephaven CSV 0.11.0