Introduce vtable-based async stream implementation#627
Closed
arnetheduck wants to merge 3 commits intomasterfrom
Closed
Introduce vtable-based async stream implementation#627arnetheduck wants to merge 3 commits intomasterfrom
arnetheduck wants to merge 3 commits intomasterfrom
Conversation
b4a3be7 to
ac27115
Compare
Currently, async readers and writers are implemented using a mechanism where each layer implements a read/write loop that reads/writes from an intermediate buffer - a write to a stream is placed in a queue and a write loop reads from that queue, processes the data and forward it to the next "layer". The sink layer then writes the data to a transport and translates the errors and exceptions coming back. While this approach works, it has a few downsides: * Each layer introduces additional latency - for example, a http body writer that has a bounded writer and maybe one more layer has to go through 3-4 async loop iterations before the data reaches the socket - this means a lot of additional latency specially in a busy applications where the loop has lots of work to do * For string/seq, each layer also introduces at least two full data copies - in and out of the queue (plus the usual async overhead) * The "top" layer must keep track of the full stack of streams underneath to close them, ie HttpBodyWriter has a `seq[...]` of streams it needs to manage - logically though, it should only have to keep track of one. * The buffering is mandatory - there's no way to write a lightweight "observer" stream that doesn't use the buffer - this is due to how the extension mechanism has flow control for selecting implementation using an `if/else` construct that cannot be extended by the stream. * Every cal lhas to go through the `if/else` construct every time even though the conditions are fixed per stream type This commit introduces a (classic) alternative where streams inherit from each other - this allows each stream to choose its own buffering strategy and thus avoid the queue/loop/copy problems. Closing streams can now be implemented using an ownership cascade where the top layer closes the layer under it when requested but no longer has to manage multiple layers. There are a few downsides too - the API wide meaning that each layer has to implement a large number of functions - readOnce, readExactly and so on - in the current implementation, only the loop needs replacing. The aim of this first step is to introduce the VTable which removes the `if/else` construct in favor of function pointers that are assigned on construction - more work is needed to take advantage of the vtables and solve the other issues.
Member
Author
|
#644 is the production version of this PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, async readers and writers are implemented using a mechanism where each layer implements a read/write loop that reads/writes from an intermediate buffer - a write to a stream is placed in a queue and a write loop reads from that queue, processes the data and forward it to the next "layer".
The sink layer then writes the data to a transport and translates the errors and exceptions coming back.
While this approach works, it has a few downsides:
seq[...]of streams it needs to manage - logically though, it should only have to keep track of one.if/elseconstruct that cannot be extended by the stream.if/elseconstruct every time even though the conditions are fixed per stream typeThis commit introduces a (classic) alternative where streams inherit from each other - this allows each stream to choose its own buffering strategy and thus avoid the queue/loop/copy problems.
Closing streams can now be implemented using an ownership cascade where the top layer closes the layer under it when requested but no longer has to manage multiple layers.
There are a few downsides too - the API wide meaning that each layer has to implement a large number of functions - readOnce, readExactly and so on - in the current implementation, only the loop needs replacing.
The aim of this first step is to introduce the VTable which removes the
if/elseconstruct in favor of function pointers that are assigned on construction - more work is needed to take advantage of the vtables and solve the other issues.