Skip to content

Implement Index Default Timeout#936

Open
bandalgomsu wants to merge 6 commits intovalkey-io:mainfrom
bandalgomsu:gh-383
Open

Implement Index Default Timeout#936
bandalgomsu wants to merge 6 commits intovalkey-io:mainfrom
bandalgomsu:gh-383

Conversation

@bandalgomsu
Copy link
Copy Markdown
Contributor

Implement default search timeout per index and setting default search timeout in FT.CREATE

in FT.SEARCH or FT.AGGREGATE, timeout is applied in the following order of priority

  1. FT.SEARCH TIMEOUT
  2. FT.CREATE SEARCH_TIMEOUT
  3. search.default-timeout-ms

issue : #383

Signed-off-by: Su Ko <rhtn1128@gmail.com>
keys are loaded. The key itself can be loaded by specifying `@__key`. For vector queries, the distance can also be loaded by using the name of that field.
- `SLOP <slop>` (Optional): Specifies a slop value for proximity matching of terms.
- `TIMEOUT <timeout>` (optional): Lets you set a timeout value for the search command. This must be an integer in milliseconds.
- `TIMEOUT <timeout>` (optional): Lets you set a timeout value for the search command. This must be an integer in milliseconds. If an index was created with `SEARCH_TIMEOUT`, this query-level `TIMEOUT` value takes precedence.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe say that this "If present, overrides the default TIMEOUT value. The default timeout can be specified by the SEARCH_TIMEOUT option for this index or the global search.default-timeout-ms configuration value.


- `SKIPINITIALSCAN` (optional): If specified, this option skips the normal backfill operation for an index. If this option is specified, pre-existing keys which match the `PREFIX` clause will not be loaded into the index during a backfill operation. This clause has no effect on processing of key mutations _after_ an index is created, i.e., keys which are mutated after an index is created and satisfy the data type and `PREFIX` clause will be inserted into that index.

- `SEARCH_TIMEOUT <timeout>` (optional): Sets the default timeout in milliseconds for `FT.SEARCH` and `FT.AGGREGATE` requests on this index. A query-level `TIMEOUT` argument overrides this value.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think search_timeout is misleading because it applies to both ft.search and ft.aggregate. Perhaps QUERY_TIMEOUT ?

- `SOMESHARDS` (Optional): If specified, the command will generate a best-effort reply if all shards have not responded within the timeout interval.
- `SORTBY <field> [ASC | DESC]` (Optional): If present, results are sorted according the value of the specified field and the optional sort-direction instruction. By default, vector results are sorted in distance order and non-vector results are not sorted in any particular order. Sorting is applied before the `LIMIT` clause is applied.
- `TIMEOUT <timeout>` (optional): Lets you set a timeout value for the search command. This must be an integer in milliseconds.
- `TIMEOUT <timeout>` (optional): Lets you set a timeout value for the search command. This must be an integer in milliseconds. If an index was created with `SEARCH_TIMEOUT`, this query-level `TIMEOUT` value takes precedence.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be same as ft.aggregate language.

"Pausepoint search_entries_fetcher was not hit"

# Hold the query at pausepoint long enough to exceed index timeout.
time.sleep(0.2)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there a sleep here? Can't there be something specific to wait for? Timed sleeps almost always create flaky tests.

parameters->index_schema,
schema_manager.GetIndexSchema(db_num, parameters->index_schema_name));
parameters->timeout_ms =
parameters->index_schema->GetSearchTimeoutMs() > 0
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code precludes setting a timeout value to 0, which is a potential future compatibility issue. Why not set the default value before the command is parsed?

repeated string stop_words = 11;
uint32 min_stem_size = 12;
bool skip_initial_scan = 13;
uint32 search_timeout_ms = 14;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be optional, just like std::option so that if it's not specified, then the current global default is used.

absl::string_view score_field;
absl::string_view payload_field;
bool skip_initial_scan{false};
uint32_t search_timeout_ms{0};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be std::optional

uint64_t fingerprint_{0};
uint32_t version_{0};
bool skip_initial_scan_{false};
uint32_t search_timeout_ms_{0};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be std::optional

Signed-off-by: Su Ko <rhtn1128@gmail.com>
Signed-off-by: Su Ko <rhtn1128@gmail.com>
Signed-off-by: Su Ko <rhtn1128@gmail.com>
Signed-off-by: Su Ko <rhtn1128@gmail.com>
Signed-off-by: Su Ko <rhtn1128@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants