dd: get rid of line buffered stdout

[svlv]

Line-buffered stdout causes partial write and read operations in dd,
which is an issue when writing binary data to stdout. Partial writes can
lead to data loss and require passing iflag=fullblock to ensure that the
exact number of bytes is read.

It fixes partial writes mentioned in the PR #8840 and the issue #9119

Partial writes can be reproduced by the following calls

dd bs=1024 if=/dev/random count=1 | dd bs=1024 of=/dev/null count=1
1+0 records in
1+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000333775 s, 1.0 MB/s
0+1 records in
0+1 records out
678 bytes copied, 0.00151917 s, 678 kB/s

dd bs=1024 if=/dev/random count=1 | dd bs=1024 of=/dev/null count=1
1+0 records in
1+0 records out
0+1 records in
0+1 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000438309 s, 1.0 MB/s
677 bytes copied, 0.00130174 s, 677 kB/s

dd bs=1024 if=/dev/random count=1 | dd bs=1024 of=/dev/null count=1
0+1 records in
0+1 records out
904 bytes copied, 0.00121703 s, 904 kB/s
1+0 records in
1+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000428462 s, 1.0 MB/s Error flushing stdout: Broken pipe (os error 32)

As you can see, each call copies a different number of bytes, and in some cases a broken pipe can occur. The broken pipe happens when the second dd closes the pipe too early.

The issue with line-buffered stdout is well-known:
https://ericswpark.com/blog/2025/2025-01-23-buffering-by-block-in-rust/

For the fix I was using the workaround shared here - rust-lang/rust#58326 (comment)

This is almost my first snippet in Rust, so please don’t judge too strictly.

[ChrisDryden]

I believe we have a shared library that handles this behavior: OwnedFileDescriptorOrHandle. My understanding is that treating the Fd as a file even if its Stdout will also solve the issue of bypassing using the LineWriter.

There is the performance tradeoff of cloning the Fd, but it also means we can avoid the unsafe and it wont close stdout on drop.

I did a check to see if OwnedFileDescriptorOrHandle works and using the examples you provided it seems to solve the issue you are describing

[svlv]

Thanks for the good points. I’ve applied the comments.

[codspeed-hq]

Merging this PR will degrade performance by 4.77%

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 280 untouched benchmarks
⏩ 38 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	Memory	cp_large_file[16]	114.1 KB	119.8 KB	-4.77%
⚡	Memory	numfmt_large_numbers_si[10000]	4.8 MB	4.6 MB	+4.08%

---

_{Comparing svlv:dd-get-rid-of-line-buffered-stdout (ebcfca3) with main (038a08b)}

Footnotes

1. 38 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[ChrisDryden]

Those memory tests were just added today, can ignore them

[github-actions]

GNU testsuite comparison:

GNU test failed: tests/tail/follow-name. tests/tail/follow-name is passing on 'main'. Maybe you have to rebase?