Post v2: Make `ansi` driver even less dependent on Win32 APIs

[tig]

Approach

Use a background async read pattern inside WindowsVTInputHelper: a background task calls Stream.ReadAsync in a loop, buffering results into a concurrent collection. The existing Peek()/TryRead() API continues to work but reads from the buffer instead of P/Invoke. This avoids needing to change InputImpl or IInput interfaces.

Current Architecture (for reference)

Input pipeline flow

1. Input thread (InputImpl.Run()): loops while (Peek()) { Read() } with 20ms delay
2. AnsiInput.Peek() → delegates to WindowsVTInputHelper.Peek() for WindowsVT platform
3. AnsiInput.Read() → calls WindowsVTInputHelper.TryRead(buffer, out bytesRead), converts bytes to UTF-8 chars, yields them
4. InputImpl.Run() enqueues yielded chars into InputQueue (a ConcurrentQueue<char>)
5. Main thread (ProcessQueue()): dequeues from InputQueue, calls AnsiInputProcessor.Process(char) → Parser.ProcessInput() → matches ANSI responses, raises keyboard/mouse events

ANSI request/response flow

1. AnsiSizeMonitor.SendSizeQuery() → AnsiRequestScheduler.SendOrSchedule() → writes ESC[18t to stdout via driver.WriteRaw()
2. Terminal responds with ESC[8;height;widtht on stdin
3. Response bytes flow through the input pipeline (steps 1-5 above)
4. AnsiResponseParser matches the response terminator/value and invokes ResponseReceived callback
5. Scheduler has 100ms throttle, 1s stale timeout for unanswered requests

Initialization order (MainLoopCoordinator)

1. StartInputTaskAsync(app) is called
2. RunInput(app) starts on background thread (acquires lock):
   • Creates AnsiInput → WindowsVTInputHelper.TryEnable() (opens handle, sets console mode)
   • Calls BuildDriverIfPossible() (driver built only if output also exists)
   • Releases lock, then calls _input.Run() (starts Peek/Read loop)
3. BootMainLoop(app) runs on main thread (acquires lock):
   • Creates _output
   • Calls BuildDriverIfPossible() → if both input+output exist, builds driver
   • SizeMonitor.Initialize(_driver) → sends initial size query
   • _startupSemaphore.Release()
4. StartInputTaskAsync awaits semaphore, then returns
5. Main event loop starts → RunIteration() → ProcessQueue() begins processing

Key files

• Terminal.Gui/Drivers/WindowsDriver/WindowsVTInputHelper.cs — the file to modify
• Terminal.Gui/Drivers/AnsiDriver/AnsiInput.cs — calls Peek() (line 170) and TryRead() (line 195)
• Terminal.Gui/Drivers/Input/InputImpl.cs — the Peek/Read/20ms-delay loop
• Terminal.Gui/Drivers/AnsiDriver/AnsiInputProcessor.cs — processes chars through parser
• Terminal.Gui/Drivers/AnsiHandling/AnsiRequestScheduler.cs — manages ANSI request lifecycle
• Terminal.Gui/Drivers/AnsiDriver/AnsiSizeMonitor.cs — sends CSI_ReportWindowSizeInChars
• Terminal.Gui/App/MainLoop/MainLoopCoordinator.cs — initialization and threading

VTTest Reference Implementation

VTTest/AsyncStreamInputReader.cs is the pattern to follow:

• Uses Console.OpenStandardInput() on Windows
• Background Task.Run(async () => { ReadAsync loop }) with CancellationToken
• BlockingCollection<(byte[] Data, int Count)> (not ConcurrentQueue) — consumer blocks via _queue.Take()
• Consumer Read() method BLOCKS until data available (unlike our TryRead which is non-blocking)
• ReadAsync(buf.AsMemory(0, buf.Length), _cts.Token) — uses Memory<byte> overload
• Always copies buffer before enqueuing: var copy = new byte[n]; Array.Copy(buf, 0, copy, 0, n);
• Enqueues 0-byte reads as-is (Ctrl+Z workaround handled by consumer)
• Calls _queue.CompleteAdding() in finally block when loop exits
• Dispose: cancel → wait 1s → dispose stream → dispose queue → dispose CTS

Changes to WindowsVTInputHelper.cs

Remove:

• ReadFile P/Invoke declaration
• GetNumberOfConsoleInputEvents P/Invoke declaration

Keep:

• GetStdHandle, GetConsoleMode, SetConsoleMode (still needed — no .NET equivalent for VT input mode flags)
• All console mode constants and setup logic
• InputHandle property (used internally for Get/SetConsoleMode)

Add:

• using System.Collections.Concurrent;
• Stream? _stdinStream — opened via Console.OpenStandardInput()
• CancellationTokenSource? _cts for clean shutdown
• Task? _readTask for the background read loop
• ConcurrentQueue<(byte[] Buffer, int Count)> _readQueue for buffering
• ReadLoopAsync(CancellationToken ct) — background method:
  • Calls _stdinStream!.ReadAsync(buffer.AsMemory(0, buffer.Length), ct) in a loop
  • Copies data before enqueuing
  • For 0-byte reads (Ctrl+Z bug): enqueues ([0x1A], 1)
  • Catches OperationCanceledException for clean shutdown

Modify:

• TryEnable() — after setting console mode, open stdin stream and start background task
• Peek() — !_readQueue.IsEmpty instead of GetNumberOfConsoleInputEvents
• TryRead() — _readQueue.TryDequeue() instead of ReadFile
• Restore() — cancel background task, wait, dispose stream/CTS, then restore console mode

No changes to AnsiInput.cs, InputImpl.cs, or IInput.

Known Issues from First Attempt

The implementation compiled and passed all unit tests (14520 parallelizable + 1000 non-parallel) but broke ANSI request/response at runtime. Root cause unknown. Hypotheses:

1. Data truncation: Background reader used 1024-byte buffer but AnsiInput.Read() passes 256-byte buffer to TryRead(). If ReadAsync returns >256 bytes in one call, excess is silently dropped.

2. Timing/ordering: Background reader starts in TryEnable() before the ANSI request scheduler is set up. Unsolicited data arriving early could desync the parser.

3. Stream vs ReadFile behavioral difference: Console.OpenStandardInput() wraps the same handle but .NET's ConsoleStream.ReadAsync may have different buffering or blocking behavior. On .NET, ReadAsync on a non-seekable stream schedules synchronous Read on a thread pool thread.

4. ConcurrentQueue vs BlockingCollection: VTTest uses BlockingCollection with blocking Take(). Our ConcurrentQueue with non-blocking TryDequeue may miss data in edge cases, or the different consumer pattern may matter.

5. CancellationToken ineffective on console ReadFile: The Memory<byte> overload of ReadAsync on a Windows console stream may not support cancellation — the underlying ReadFile is a blocking kernel call.

No Unit Test Coverage for This Path

There is no unit test that exercises the ANSI request/response flow through WindowsVTInputHelper. All existing tests use either:

• Mocked IAnsiResponseParser (AnsiRequestSchedulerTests, AnsiRequestSchedulerCollisionTests)
• Direct parser.ProcessInput() calls (AnsiRequestSchedulerRaceTests)
• InjectInput() test injection (AnsiInputTestableTests)

Prerequisites Before Re-attempting

1. Write a unit test for ANSI request/response through the input pipeline
2. Investigate the regression — add diagnostic logging to identify where response bytes are lost
3. Consider BlockingCollection instead of ConcurrentQueue (matching VTTest)
4. Match buffer sizes — use 256-byte buffer in background reader to match caller