Skip to content

Introduce vtable-based async stream implementation#627

Closed
arnetheduck wants to merge 3 commits intomasterfrom
vtbl-stream
Closed

Introduce vtable-based async stream implementation#627
arnetheduck wants to merge 3 commits intomasterfrom
vtbl-stream

Conversation

@arnetheduck
Copy link
Copy Markdown
Member

Currently, async readers and writers are implemented using a mechanism where each layer implements a read/write loop that reads/writes from an intermediate buffer - a write to a stream is placed in a queue and a write loop reads from that queue, processes the data and forward it to the next "layer".

The sink layer then writes the data to a transport and translates the errors and exceptions coming back.

While this approach works, it has a few downsides:

  • Each layer introduces additional latency - for example, a http body writer that has a bounded writer and maybe one more layer has to go through 3-4 async loop iterations before the data reaches the socket - this means a lot of additional latency specially in a busy applications where the loop has lots of work to do
  • For string/seq, each layer also introduces at least two full data copies - in and out of the queue (plus the usual async overhead)
  • The "top" layer must keep track of the full stack of streams underneath to close them, ie HttpBodyWriter has a seq[...] of streams it needs to manage - logically though, it should only have to keep track of one.
  • The buffering is mandatory - there's no way to write a lightweight "observer" stream that doesn't use the buffer - this is due to how the extension mechanism has flow control for selecting implementation using an if/else construct that cannot be extended by the stream.
  • Every cal lhas to go through the if/else construct every time even though the conditions are fixed per stream type

This commit introduces a (classic) alternative where streams inherit from each other - this allows each stream to choose its own buffering strategy and thus avoid the queue/loop/copy problems.

Closing streams can now be implemented using an ownership cascade where the top layer closes the layer under it when requested but no longer has to manage multiple layers.

There are a few downsides too - the API wide meaning that each layer has to implement a large number of functions - readOnce, readExactly and so on - in the current implementation, only the loop needs replacing.

The aim of this first step is to introduce the VTable which removes the if/else construct in favor of function pointers that are assigned on construction - more work is needed to take advantage of the vtables and solve the other issues.

@arnetheduck arnetheduck marked this pull request as draft March 27, 2026 19:37
@arnetheduck arnetheduck force-pushed the vtbl-stream branch 10 times, most recently from b4a3be7 to ac27115 Compare April 9, 2026 06:42
Currently, async readers and writers are implemented using a mechanism
where each layer implements a read/write loop that reads/writes from an
intermediate buffer - a write to a stream is placed in a queue and a
write loop reads from that queue, processes the data and forward it to
the next "layer".

The sink layer then writes the data to a transport and translates the
errors and exceptions coming back.

While this approach works, it has a few downsides:

* Each layer introduces additional latency - for example, a http body
writer that has a bounded writer and maybe one more layer has to go
through 3-4 async loop iterations before the data reaches the socket -
this means a lot of additional latency specially in a busy applications
where the loop has lots of work to do
* For string/seq, each layer also introduces at least two full data
copies - in and out of the queue (plus the usual async overhead)
* The "top" layer must keep track of the full stack of streams
underneath to close them, ie HttpBodyWriter has a `seq[...]` of streams
it needs to manage - logically though, it should only have to keep track
of one.
* The buffering is mandatory - there's no way to write a lightweight
"observer" stream that doesn't use the buffer - this is due to how the
extension mechanism has flow control for selecting implementation using
an `if/else` construct that cannot be extended by the stream.
* Every cal lhas to go through the `if/else` construct every time even
though the conditions are fixed per stream type

This commit introduces a (classic) alternative where streams inherit
from each other - this allows each stream to choose its own buffering
strategy and thus avoid the queue/loop/copy problems.

Closing streams can now be implemented using an ownership cascade where
the top layer closes the layer under it when requested but no longer has
to manage multiple layers.

There are a few downsides too - the API wide meaning that each layer has
to implement a large number of functions - readOnce, readExactly and so
on - in the current implementation, only the loop needs replacing.

The aim of this first step is to introduce the VTable which removes the
`if/else` construct in favor of function pointers that are assigned on
construction - more work is needed to take advantage of the vtables and
solve the other issues.
@arnetheduck
Copy link
Copy Markdown
Member Author

#644 is the production version of this PR

@arnetheduck arnetheduck deleted the vtbl-stream branch April 10, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant