[Feature Request] 64-bit roaring bitmap support

[pdeaudney]

Is your feature request related to a problem? Please describe

N/A, this is a capability uplift request to support a 64-bit roaring bitmap term filter on long fields.

Describe the solution you'd like

Add roaring64 support to the bitmap term filter, to support long fields.
Allow a mapping like:

{
  "mappings": {
    "properties": {
      "user_id": { "type": "long" }
    }
  }
}

With a query like:

{
  "query": {
    "terms": {
      "user_id": ["<base64-encoded Roaring64NavigableMap portable bytes>"],
      "value_type": "bitmap"
    }
  }
}

Related component

Search:Query Capabilities

Describe alternatives you've considered

Just use 32-bit roaring bitmap term filters? 🙅

Additional context

Historically this wasn't possible due to the lack of a portable 64-bit roaring bitmap format.

Ref: #15638

This is now addressed in the roaring specification & Roaring64NavigableMap allows for a compatible base64 encoding of a roaring64.
https://github.com/RoaringBitmap/RoaringFormatSpec

So we should be able to support a bitmap term filter over long fields now (I think)?

[sandeshkr419]

[Search Triage] @bowenlan-amzn Can you please take a look?

Related discussion on a previous issue: #18422

[divyaruhil]

Hi @sandeshkr419 @pdeaudney,

I’m excited to contribute and have raised a PR for this. Would appreciate it if you could take a look and share your feedback.

Thanks!

[pdeaudney]

@divyaruhil, I've started a PR on this too, #20598 is where I got to with it. I need to put some more time into validating this more (and setting up my local dev environment).

Not going to lie, that was all prompted so I want to take some more time to go through it. It has been a largish number of years since I have written java.
I am wondering about some performance implications of some of the choices I made with the LLM in there.

I am on the opensearch slack, happy for anyone to DM me with critique/feedback.