C#: Implement transparent compression feature by prateek-kumar-improving · Pull Request #213 · valkey-io/valkey-glide-csharp

prateek-kumar-improving · 2026-02-23T05:23:43Z

Summary

Implementing C# bindings for the transparent compression feature added in valkey-io/valkey-glide#4759

Issue Link

This pull request is linked to issue: valkey-io/valkey-glide#5120
Related Issue: valkey-io/valkey-glide#5299

Features and Behaviour Changes

Adds the necessary language bindings in C# to configure a client with compression enabled.
Added unit and integration tests for compression scenarios.
Compression Backends Supported: Zstd and Lz4.
Statistics available using BaseClient.GetStatistics().

Implementation

Rust FFI

rust/src/lib.rs - compress_cmd helper, get_statistics FFI export, command() integration
rust/src/ffi.rs - FFI bindings for compression config and statistics

C# Public API

sources/Valkey.Glide/CompressionConfig.cs - Compression configuration class
sources/Valkey.Glide/Statistics.cs - Statistics record (compression, connections, pub/sub)
sources/Valkey.Glide/abstract_Enums/CompressionBackend.cs - Zstd/Lz4 enum
sources/Valkey.Glide/ConnectionConfiguration.cs - WithCompression() builder method
sources/Valkey.Glide/BaseClient.cs - GetStatistics() method

C# FFI Internals

sources/Valkey.Glide/Internals/FFI.structs.cs - FFI structs for CompressionConfig and Statistics
sources/Valkey.Glide/Internals/FFI.methods.cs - FFI method bindings

Tests

tests/Valkey.Glide.IntegrationTests/CompressionTests.cs - Integration tests
tests/Valkey.Glide.UnitTests/CompressionConfigTests.cs - Unit tests

Documentation:

README.md - Added compression documentation and examples

Limitations

This feature is only limited to GET and SET for now.

Testing

All tests passing locally and in CI. Both unit tests and integration tests added.

Checklist

This Pull Request is related to one issue.
Commit message has a detailed description of what changed and why.
Tests are added or updated and all checks pass.
CHANGELOG.md, README.md, DEVELOPER.md, and other documentation files are updated.
Destination branch is correct - main or release
Create merge commit if merging release branch into main, squash otherwise.

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

…rc/ffi.rs Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

Updated submodule from 2ea4018f6 to 0429c1201 to include: - Transparent Compression Feature (#4759) - Go bindings for compression (#5359) This update enables compression functionality in the Rust core. Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

…ture

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

currantw · 2026-03-24T20:10:32Z

C# example for compression added to README

C# documentation is being shifted to valkey-glide-docs. Can you please raise a pull request to add that documentation there (and remove it here)? Please update the description accordingly.

currantw

Reviewed implementation. Will review tests once addressed.

currantw · 2026-03-25T14:30:10Z

rust/src/ffi.rs

+        compression_config: if config.has_compression_config {
+            Some(glide_core::compression::CompressionConfig {
+                enabled: config.compression_config.enabled,
+                min_compression_size: config.compression_config.min_compression_size,
+                compression_level: if config.compression_config.compression_level == 0 {
+                    None
+                } else {
+                    Some(config.compression_config.compression_level)
+                },
+                backend: match config.compression_config.backend {
+                    CompressionBackend::Zstd => {
+                        glide_core::compression::CompressionBackendType::Zstd
+                    }
+                    CompressionBackend::Lz4 => {
+                        glide_core::compression::CompressionBackendType::Lz4
+                    }
+                },
+            })
+        } else {
+            None
+        },


I think this can be simplified quite a bit using then_some (this also removes the unnecessary brackets from the match block):

Suggested change

compression_config: if config.has_compression_config {

Some(glide_core::compression::CompressionConfig {

enabled: config.compression_config.enabled,

min_compression_size: config.compression_config.min_compression_size,

compression_level: if config.compression_config.compression_level == 0 {

None

} else {

Some(config.compression_config.compression_level)

},

backend: match config.compression_config.backend {

CompressionBackend::Zstd => {

glide_core::compression::CompressionBackendType::Zstd

}

CompressionBackend::Lz4 => {

glide_core::compression::CompressionBackendType::Lz4

}

},

})

} else {

None

},

compression_config: config.has_compression_config.then_some(

glide_core::compression::CompressionConfig {

enabled: config.compression_config.enabled,

min_compression_size: config.compression_config.min_compression_size,

compression_level: (config.compression_config.compression_level != 0)

.then_some(config.compression_config.compression_level),

backend: match config.compression_config.backend {

CompressionBackend::Zstd => glide_core::compression::CompressionBackendType::Zstd,

CompressionBackend::Lz4 => glide_core::compression::CompressionBackendType::Lz4,

},

},

),

currantw · 2026-03-25T14:43:14Z

sources/Valkey.Glide/CompressionConfig.cs

+        if (minCompressionSize < MinCompressionSizeLimit)
+        {
+            throw new ArgumentException($"minCompressionSize must be at least {MinCompressionSizeLimit} bytes", nameof(minCompressionSize));
+        }


I think ArgumentOutOfRangeException might be better here?

currantw · 2026-03-25T15:02:03Z

sources/Valkey.Glide/CompressionConfig.cs

Are these StackExchange.Redis-compatible types? Otherwise, I think they should just be in sources rather than sources/abstract_APITypes. The distinction between these is certainly not clear, but that is my understanding of the separation!

Moreover, I think we should generally endeavour to have one class (excepting private classes) per file? That seems to be the pattern (loosely followed) in the code base.

See the refactoring PR I just raised along these lines: #246

currantw · 2026-03-25T15:03:06Z

sources/Valkey.Glide/abstract_APITypes/CompressionConfig.cs

+/// Enables automatic compression of values before sending to the server
+/// and decompression when receiving from the server.
+/// </summary>
+[StructLayout(LayoutKind.Sequential)]


Marshalling related code should be isolated in FFI.structs.cs, not exposed in public APIs. We probably need to split this into two separate classes? See #246

currantw · 2026-03-25T15:04:04Z

sources/Valkey.Glide/abstract_Enums/CompressionBackend.cs

+    /// <summary>
+    /// Zstandard (zstd) compression backend.
+    /// Provides high compression ratios with good performance.
+    /// Default compression level: 3


Don't think we need to specify the default compression level here?

currantw · 2026-03-25T15:50:25Z

sources/Valkey.Glide/Internals/FFI.structs.cs

+    /// <summary>
+    /// Statistics structure containing telemetry data from Rust core.
+    /// </summary>
+    [StructLayout(LayoutKind.Sequential)]


As mentioned elsewhere, we need to separate the FFI classes from the API ones.

currantw · 2026-03-25T15:50:40Z

sources/Valkey.Glide/Internals/FFI.structs.cs

+        public ulong TotalConnections;
+        public ulong TotalClients;
+        public ulong TotalValuesCompressed;
+        public ulong TotalValuesDecompressed;
+        public ulong TotalOriginalBytes;
+        public ulong TotalBytesCompressed;
+        public ulong TotalBytesDecompressed;
+        public ulong CompressionSkippedCount;


Can these be documented please?

currantw · 2026-03-25T15:51:38Z

sources/Valkey.Glide/Internals/FFI.structs.cs

+    /// Statistics structure containing telemetry data from Rust core.
+    /// </summary>
+    [StructLayout(LayoutKind.Sequential)]
+    internal struct Statistics


Should this be readonly like others in this class? Similarly, should fields also be readonly?

currantw · 2026-03-25T16:02:22Z

README.md

 - **[Cluster Scan](https://github.com/valkey-io/valkey-glide/wiki/General-Concepts#cluster-scan)** – Unified key iteration across shards using a consistent, high-level API
 - **[Batching (Pipeline and Transaction)](https://github.com/valkey-io/valkey-glide/wiki/General-Concepts#batching-pipeline-and-transaction)** – Execute multiple commands efficiently in a single network roundtrip
 - **[OpenTelemetry](https://github.com/valkey-io/valkey-glide/wiki/General-Concepts#opentelemetry)** – Integrated tracing support for enhanced observability
+- **[Transparent Compression](#compression-operations)** – Automatic value compression with Zstd and LZ4 backends to reduce bandwidth and storage


Probably also need to update CHANGELOG.md?

currantw · 2026-03-25T16:03:09Z

README.md

+### Compression Operations
+
+```csharp
+// Enable transparent compression with Zstd
+var config = new StandaloneClientConfigurationBuilder()
+    .WithAddress("localhost", 6379)
+    .WithCompression(CompressionConfig.Zstd())
+    .Build();
+
+await using var client = await GlideClient.CreateClient(config);
+
+// Values are automatically compressed/decompressed
+await client.StringSetAsync("large_data", new string('a', 10000));
+var retrieved = await client.StringGetAsync("large_data");
+
+// Monitor compression statistics
+var stats = BaseClient.GetCompressionStatistics();
+Console.WriteLine($"Compression ratio: {stats.CompressionRatio:F2}x");
+Console.WriteLine($"Space saved: {stats.SpaceSavedPercent:F2}%");
+
+// Use LZ4 for faster compression
+var lz4Config = new StandaloneClientConfigurationBuilder()
+    .WithAddress("localhost", 6379)
+    .WithCompression(CompressionConfig.Lz4(compressionLevel: 5, minCompressionSize: 128))
+    .Build();
+```


This should be part of valkey-glide-docs instead. Please raise PR for that and link here (as described in other comment).

Removed readme.md update from this PR. Documentation is not updated for compression feature for now. Will be updated later after all work related to the feature is done.

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

…ics.cs Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

…ture Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

prateek-kumar-improving added 15 commits February 22, 2026 21:23

C#: Implement transparent compression feature

f1b0167

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update readme.md

5532d11

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Fix compression tests

4fc88eb

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Fix compression tests

8cf5b84

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update lint issues

63acbfd

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C# Update submodules

d56c08a

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

Rust: Update lib.rs

2b09d7b

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

PHP: Update rust/src/lib.rs

fd3eaa3

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update CompressionTests.cs

120e83f

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update rust/src/ffi.rs

14430a8

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update CompressionConfig.cs, CompressionConfigTests.cs and rust/s…

dd8b37e

…rc/ffi.rs Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update CompressionConfig.cs, CompressionConfigTests.cs and rust/s…

ef4f55f

…rc/ffi.rs Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Fix compression tests

825f9ef

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Fix lint for compression tests

34ff162

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

prateek-kumar-improving force-pushed the csharp/implement-transparent-compression-feature branch from 4bb939e to 34ff162 Compare February 26, 2026 17:12

prateek-kumar-improving added 2 commits February 26, 2026 10:13

Merge branch 'main' into csharp/implement-transparent-compression-fea…

4b83974

…ture

refactor: add constant for minimum compression size limit

95f5740

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

prateek-kumar-improving marked this pull request as ready for review February 26, 2026 21:19

prateek-kumar-improving requested a review from alexr-bq February 26, 2026 21:21

prateek-kumar-improving closed this Feb 26, 2026

prateek-kumar-improving reopened this Feb 26, 2026

prateek-kumar-improving added 4 commits February 26, 2026 14:34

refactor: add constants for magic numbers in compression unit tests

76931f8

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

refactor: remove debug log and add constants for magic numbers in tests

c4b90ac

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

refactor: remove debug log from CreateClient

a3cbabf

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

refactor: remove debug log from WithCompression

2d12936

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

prateek-kumar-improving requested a review from currantw February 26, 2026 22:46

prateek-kumar-improving self-assigned this Feb 27, 2026

prateek-kumar-improving added the C# 🥅 label Feb 27, 2026

currantw requested changes Mar 25, 2026

View reviewed changes

prateek-kumar-improving added 15 commits April 3, 2026 22:32

C#: Update ffi.rs

7e68d1b

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update CompressionConfig.cs and CompressionStatistics.cs

baed61e

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update CompressionConfig.cs

7c18b69

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update CompressionConfig.cs and ffi.rs

15ad5d1

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update BaseClient.cs, CompressionConfig.cs and CompressionStatist…

34bce98

…ics.cs Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update changelog

1289715

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

Merge branch 'main' into csharp/implement-transparent-compression-fea…

fefde0b

…ture Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update BaseClient.cs and FFI.structs.cs

4da890b

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update readme

d4ce24f

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update BaseClient.cs

1d4318b

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Add Statistics.cs

a9280c6

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Update CompressionConfigTests.cs

b4a80aa

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Fix lint issues

c127e82

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C# Fix lint issues

95cf978

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

C#: Add tests for new commands for compression feature

4a73b95

Signed-off-by: Prateek Kumar <prateek.kumar@improving.com>

Conversation

prateek-kumar-improving commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Issue Link

Features and Behaviour Changes

Implementation

Rust FFI

C# Public API

C# FFI Internals

Tests

Documentation:

Limitations

Testing

Checklist

Uh oh!

currantw commented Mar 24, 2026

Uh oh!

currantw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

prateek-kumar-improving commented Feb 23, 2026 •

edited

Loading