Skip to content

Implement memory reservation on ingestion#884

Open
allenss-amazon wants to merge 1 commit intovalkey-io:mainfrom
allenss-amazon:dime
Open

Implement memory reservation on ingestion#884
allenss-amazon wants to merge 1 commit intovalkey-io:mainfrom
allenss-amazon:dime

Conversation

@allenss-amazon
Copy link
Copy Markdown
Member

No description provided.

Signed-off-by: Allen Samuels <allenss@amazon.com>
*max_nonvector_search_results_fetched);
}

/// Register the "--mutation-weight-vector" flag. Controls the weight multiplier
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want these to be App or Dev configs?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, we can mark all of these as Dev configs for now

itr->second.from_backfill = from_backfill;
itr->second.from_multi = from_multi;
itr->second.sequence_number = sequence_number;
itr->second.weighted_buffer.resize(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add documentation on weighted_buffer explaining what this helps us solve?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if this resolves the problem. If it does we can clean this up.

Comment on lines +1769 to +1770
if (attr_itr != attributes_.end()) {
switch (attr_itr->second.GetIndex()->GetIndexerType()) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can each Attribute type instead implement a Weight function where we just return the weight of the config? It will help avoid this switch statement / case handling here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. A virtual function would have been a better choice.

KarthikSubbarao added a commit that referenced this pull request Mar 27, 2026
Add memory reservation based on mutation queue

This is picking up the change from here:
#884 by @allenss-amazon

---------

Signed-off-by: Allen Samuels <allenss@amazon.com>
Signed-off-by: KarthikSubbarao <karthikrs2021@gmail.com>
Co-authored-by: Allen Samuels <allenss@amazon.com>
rileydes-improving pushed a commit to Bit-Quill/valkey-search that referenced this pull request Apr 1, 2026
Add memory reservation based on mutation queue

This is picking up the change from here:
valkey-io#884 by @allenss-amazon

---------

Signed-off-by: Allen Samuels <allenss@amazon.com>
Signed-off-by: KarthikSubbarao <karthikrs2021@gmail.com>
Co-authored-by: Allen Samuels <allenss@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants