NUL-delimited output without raw conversion

[lgarron]

Suppose I'd like to parse a JSON array and output its members:

echo '[4,"foo", "[]",[],{"hi": "world"}]' | jq ".[]"

There are a few conventions for newline-delimited JSON values, but this requires having a JSON parser. Fortunately, the NUL delimiter convention is pretty wide-spread and makes it easy for another process to split the values.

Unfortunately, jq only has the --raw-output0, which removes quotes from strings.

echo '[4,"foo", "[]",[],{"hi": "world"}]' | jq ".[]" --raw-output0

As a result:

• Some entries of the output are no longer parseable as JSON (e.g. "foo" → foo)
• Certain entries are serialized in the same way, making it impossible to recover the original data (e.g. "[]" and [] both serialize to []).

This could be solved with a flag like --output0 that applies the NUL delimiter without raw formatting.

[wader]

Hi, it got removed in #2683 as a JSON string can contain NUL bytes so raw string output would be ambiguous.

$ jq -n '"\u0000"' --raw-output0
jq: error (at <unknown>): Cannot dump a string containing NUL with --raw-output0 option

But there are some workaround if you're ok with the ambiguity:

$ echo '[4,"foo", "[]",[],{"hi": "world"}]' | jq -j '.[] | tojson, "\u0000"' | hexdump -C
00000000  34 00 22 66 6f 6f 22 00  22 5b 5d 22 00 5b 5d 00  |4."foo"."[]".[].|
00000010  7b 22 68 69 22 3a 22 77  6f 72 6c 64 22 7d 00     |{"hi":"world"}.|
0000001f

$ echo '[4,"foo", "[]",[],{"hi": "world"}]' | jq -j 'map(tojson) | join("\u0000")' | hexdump -C
00000000  34 00 22 66 6f 6f 22 00  22 5b 5d 22 00 5b 5d 00  |4."foo"."[]".[].|
00000010  7b 22 68 69 22 3a 22 77  6f 72 6c 64 22 7d        |{"hi":"world"}|
0000001e

[lgarron]

Hi, it got removed in #2683 as a JSON string can contain NUL bytes so raw string output would be ambiguous.

Hmm, I must be missing something. If the output entries are non-raw JSON, then NUL can always be encoded inside strings using \0000, right?

From what I can tell, the main concerns in that thread are about the interaction of -0 with raw output, which is specifically not what I'm asking to do.

jq -j '.[] | tojson, "\u0000"'

Thanks! I don't suppose you know of a way to also serialize the strings in the JSON using \0000 rather than the NUL char?