[Feature Request] Bufferpools and memory management

[abiesps]

Is your feature request related to a problem? Please describe

In continuation to discussion here, (#18833). We have found that having mmap works well in case of SSDs based deployments, however for slower storage devices mmap really transfers the underlying storage device latencies to query latencies. We found that a) mmap does synchronous reads on page faults and mmap also captures the carrier thread while using the virtual thread. Having better control on IO and caching has proved to be helpful in hiding higher disk seek latencies of slower storage devices.

There is another use-case of buffer pool for index level encryption, which is parallely being discussed here (opensearch-project/opensearch-storage-encryption#25 (comment))

Describe the solution you'd like

We are proposing to provide an alternative to mmap directory in opensearch. We are proposing to build a buffer pool that can be used for managing the pages of the file without going to page-cache. This buffer pool should go hand in hand with direct io like directories.
At a high level, we can either build our own buffer pool using MemorySegment apis of java, or we can also use netty based buffer pools. In my experiments, I used netty based buffer-pools but I can see for index-level-encryption there is a memory segment based buffer pool implementation is in progress.

Related component

No response

Describe alternatives you've considered

No response

Additional context

No response

[abiesps]

I am going to work on this.

[kumargu]

At a high level, we can either build our own buffer pool using MemorySegment apis of java, or we can also use netty based buffer pools.

Netty based ByteBuffer pools can be off-heap? Off-heap vs On heap buffers are going to be a deal a breaker.

[abiesps]

Yes netty do provide off heap byte buffers. I'll cover this in a design proposal with possible ways to implement this with pros and cons of approaches.