Skip to content

[WIP] New ETag Commands Syntax + ETag Object Support#1478

Draft
TalZaccai wants to merge 320 commits intodevfrom
talzacc/storage-v2-etag
Draft

[WIP] New ETag Commands Syntax + ETag Object Support#1478
TalZaccai wants to merge 320 commits intodevfrom
talzacc/storage-v2-etag

Conversation

@TalZaccai
Copy link
Contributor

@TalZaccai TalZaccai commented Dec 19, 2025

<<This PR is still a work in progress>>

This PR introduces a notion of "meta commands" - commands that "envelop" existing data commands.
The meta commands introduced in this PR are ETag related - EXECWITHETAG (execute a command and add the dest key's etag to the output), as well as EXECIFMATCH, EXECIFNOTMATCH, EXECIFGREATER (conditionally-execute a command based on the dest key's ETag in relation to the ETag meta-argument).

These command are able to replace the existing etag-related commands and command arguments, and in-turn support many more (including object commands).
For example:
DELIFGREATER key etag -> EXECIFGREATER etag DEL key
GETIFNOTMATCH key etag -> EXECIFNOTMATCH etag GET key
GETWITHETAG key -> EXECWITHETAG GET key
SETIFMATCH key value etag -> EXECIFMATCH etag SET key value

To achieve this logic, we parse the meta command and its meta-arguments into the SessionParseState. Currently this logic supports a constant number of meta-arguments which is defined in the Arity property in the meta-command's RespCommandInfo (arg count = arity - 1).
When the parse state gets handed to the appropriate ISessionFunctions method, the method decides whether the operation should get executed (in the case of conditional-execution) and to what value should the record's ETag advance (if at all).
The "special output" (in case the ETag needs to be written along with the command output) is handled by the ISessionFunctions method in the main and unified case, and in the IGarnetObject.Operate method in the object case.

To-do:

  • Add new supported meta-commands, remove existing etag commands and command arguments & update usages
  • Implement meta-command and meta-argument parsing in the RESP command parser.
  • Update SessionParseState to parse and hold meta-arguments
  • Update InputHeader to include meta-command and update commands
  • Implement ETag-related meta-command handling in ISessionFunctions for all 3 contexts
  • Implement meta-command-based ETag output for all object commands
  • Handle ETags in operations that do not call into ISessionFunctions (internal transactions)
  • Handle ETags in custom operations
  • Further clean-up of ETag logic in ISessionFunctions
  • Introduce leaner AOF serialization for commands with single long meta-argument
  • Support programmatic commands supplying long etag without serializing to parse state

Testing & Benchmarking:

  • Expand existing main-store ETag tests to include all meta-commands
  • Verify all write commands result in ETag advancement
  • Verify ETag output added to all commands when meta-command present
  • Verify running meta-commands within transactions
  • Verify ETag advancement + ETag output with custom commands
  • Verify benchmarking numbers on commands without ETags
  • Consider adding benchmarking logic for meta-commands with basic commands / basic commands with ETags

Add YCSB --di (Delete and reInsert) option
…ies (eliminating need for ISessionFunctions calls for those operations)
- Rename SingleWriter() => InitialWriter(), ConcurrentWriter() => InPlaceWriter()
- Rename SinglecDeleter() => InitialDeleter(), ConcurrentDeleter() => InPlaceDeleter()
- Consolidate SingleReader() and ConcurrentReader() to just Reader()
…ail, and CopyToReadCache now being fully internal operations.
… no-longer-needed allocator functions that are now done by DLR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants