Optimize output rendering with direct byte buffer

[gnodet]

Summary

The terminal output hot path — AttributedCharSequence.toAnsi() — allocates intermediate Strings for every rendered line segment. Since ANSI escape sequences are 100% ASCII and represent the bulk of the output during redisplay, writing them directly as bytes to a reusable buffer would eliminate unnecessary String allocations and charset encoding overhead.

Current hot path

On every keystroke, LineReaderImpl.redisplay() triggers:

AttributedString.print(terminal)
  → toAnsi(terminal)                    // allocates StringBuilder
    → "38;2;" + r + ";" + g + ";" + b   // String concat for each color
    → sb.toString()                     // allocates final String
  → terminal.writer().print(String)     // PrintWriter
    → OutputStreamWriter encodes UTF-8  // charset encoding
      → FastBufferedOutputStream        // already buffered

For a prompt line with 5 style regions, this means ~8 toAnsi() calls per keystroke, each creating:

• 1 StringBuilder allocation
• 5-10+ intermediate String allocations (color codes, attribute codes)
• 1 final toString() String
• UTF-8 charset encoding of content that is overwhelmingly ASCII

Proposed solution

Add a ByteArrayBuilder utility to org.jline.utils that provides:

public class ByteArrayBuilder {
    // ANSI escape sequence helpers
    ByteArrayBuilder csi();                    // append ESC [ (2 bytes, no String)
    ByteArrayBuilder appendAscii(String s);    // direct byte copy, no charset encoding
    ByteArrayBuilder appendInt(int value);     // digits to ASCII bytes directly
    ByteArrayBuilder appendUtf8(String s);     // full encoding only when needed

    // Buffer management
    byte[] buffer();
    int length();
    void reset();                              // reuse across frames
}

Key optimizations

1. appendInt(int) — avoid Integer.toString()
Color codes like 38;2;128;64;255 are currently built via String concatenation. appendInt() writes ASCII digit bytes directly to the buffer without creating a String:

// Before: allocates String
sb.append("38;2;").append(r).append(";").append(g).append(";").append(b);

// After: zero allocation
buf.appendAscii("38;2;").appendInt(r).appendAscii(";").appendInt(g).appendAscii(";").appendInt(b);

2. appendAscii(String) — bypass charset encoding
ANSI escape sequences (\033[0m, \033[38;2;...m, cursor positioning) are pure ASCII. Writing them byte-by-byte avoids the OutputStreamWriter charset encoding path entirely.

3. appendUtf8(String) — fast path for ASCII content
Most terminal text is ASCII. An optimized UTF-8 encoder can fast-path single-byte characters and only invoke full encoding for multi-byte content (CJK, emoji, box-drawing).

4. Reusable across frames
reset() clears the write position without deallocating the buffer, so the same buffer is reused for every redisplay cycle — no per-frame allocation.

Implementation plan

All changes are internal — the public API is unchanged.

Step 1: Add ByteArrayBuilder to org.jline.utils

• Reusable byte buffer with exponential growth
• csi(), appendAscii(), appendInt(), appendUtf8() methods
• ~200 lines

Step 2: Add internal toAnsiBytes() to AttributedCharSequence

• Same logic as toAnsi(), but writes to ByteArrayBuilder instead of StringBuilder
• Returns the byte buffer directly
• The existing toAnsi() can delegate to this and wrap in a String for backward compatibility

Step 3: Add internal byte-level write to terminal output

• Package-private writeBytes(byte[], int, int) on AbstractTerminal that writes directly to FastBufferedOutputStream, bypassing PrintWriter/OutputStreamWriter
• The public writer() method remains unchanged

Step 4: Wire Display.rawPrint() to use byte path

• rawPrint(AttributedString) calls toAnsiBytes() and writes via writeBytes() when available
• Falls back to print(terminal) for non-optimized terminals

Step 5 (optional): Optimize Curses.tputs()

• Terminal capability expansion (cursor movement, clear) currently returns Strings
• Could output directly to ByteArrayBuilder for cursor positioning sequences

Expected impact

Metric	Before	After
String allocations per keystroke (5 style regions)	~40-50	~0 (for ANSI output)
Charset encoding calls per keystroke	~8-10	~1 (only for actual multi-byte text)
GC pressure during fast typing	Moderate	Minimal
Output latency for full-screen redraw	Baseline	~30-40% reduction (est.)

Non-breaking guarantees

• terminal.writer() still returns PrintWriter — unchanged
• AttributedString.toAnsi() still works — unchanged (may delegate internally)
• Display public API — unchanged
• External consumers of toAnsi() — unchanged

---

Claude Code on behalf of Guillaume Nodet