[FEATURE] Key-dependent cache entry options

[gleb-osokin]

Hi, I was not sure about the exact flow for the feature request/PR, so I decided to just create the PR directly. Let me know, if I need to create a corresponding issue and link it to this PR 😄

Problem

I want to be able to configure individual cache entry options for a given subset of keys (e.g., all keys, starting with "foo-") in a generic manner, that is, via standard IOptions mechanism, provided by Microsoft out of the box. This is beneficial for:

• storing all the configuration in one place
• overriding the configuration via common means (i.e., environment variables, appsettings.*.json files, AWS parameter store, etc.)
• cache API calls simplification: cache consumers don't need to know, how things are configured.

Currently, only the default entry options are exposed, which means I can only setup them for all entries. However, if I want to setup entry options in a more granular manner, I need to resort to adjusting the code at the call site, providing modified options argument.

Solution

Provide new FusionCacheOptions.KeyDependentEntryOptions property, that applies options per cache key template:

{
    "DefaultEntryOptions": { ... },
    "KeyDependentEntryOptions": [
        { "KeyTemplate": "foo-", "Options": { ... },
        { "KeyTemplate": "bar-", "Options": { ... },
    ]
}

Also, provide a public IKeyedFusionCacheEntryOptionsProvider interface, so that custom key lookup strategy can be used.

By default, the KeyPrefixBasedEntryOptionsProvider is used to select the corresponding options. It returns the entry options for the longest matching prefix for the given cache key. If no options are found, DefaultEntryOptions are returned.

On every cache operation that involves the cache key, get the entry options from the following sources, until one is found:
expliticly provided entry options -> key-dependent entry options -> default entry options

Public API updates

1. FusionCacheOptions has a new KeyDependentEntryOptions (get/set) accessor.
2. FusionCache constructor now accepts an additional optional IKeyedFusionCacheEntryOptionsProvider parameter. This is the only place, that might cause backward compatibility issues in some very specific circumstances, If this is not acceptable, then we can think of a different approach of setting the provider on cache (i.e. via a property).
   Two built-in implementations for the interface are provided: KeyPrefixBasedEntryOptionsProvider and NullKeyedEntryOptionsProvider.
3. new overload FusionCacheEntryOptions FusionCache.CreateEntryOptions(string key, Action<FusionCacheEntryOptions>? setupAction = null, TimeSpan? duration = null) has been added as an addition to the existing one, that doesn't accept the key. I'm not sure if it makes sense to mark the existing one as Obsolete, since it is no longer used. I kept it as is for now.
4. IFusionCacheBuilder now has a new IFusionCacheBuilder WithKeyDependentEntryOptionsProvider(this IFusionCacheBuilder builder, Action<CustomServiceRegistration<IKeyedFusionCacheEntryOptionsProvider>> config) extension method to setup custom IKeyedFusionCacheEntryOptionsProvider provider.

Implementation details and limitations

The current implementation of KeyPrefixBasedEntryOptionsProvider is instantiated upon FusionCache creation, which means later updates to the KeyPrefixBasedEntryOptions members will have only limited effect. That is, changes to the entry options themselves will be applied, however it is not possible to add or remove mappings, or adjust key templates for mappings after cache creation.

Default KeyPrefixBasedEntryOptionsProvider does not support multiple configurations for the same prefix: the internal .ToDictionary() call will throw on multiple entries with the same KeyTemplate. This is done on purpose, to avoid unexpected behavior and hard-to-track bugs later at runtime.

Current prefix search algorithm is a simple binary search among the pre-sorted array of prefixes. Assuming the amount of mappings will stay reasonably small, it shouldn't give too much of an overhead. Otherwise a better data structure, e.g., prefix tree can be assumed. Also, when key-dependent options are not set (that is the case for all existing setups), the only overhead is an additional length + null check, which should be negligible.

IMPORTANT: keys are considered exactly as they are passed in to the public APIs, that is before applying the CacheKeyPrefix adjustment. The reason for this is that CacheKeyPrefix is set up only once for the entire cache, so it is expected to stay the same for all keys.

---

PS The PR is pretty big, sorry for that. Luckily, the majority of changes are to the XML comments 😸
Also, the project itself is already very nicely structured, so incorporating the options selection fits in perfectly in the existing APIs, without affecting any lower-level caching logic.

And thank you once again for the awesome library! 🚀

[jodydonetti]

Hi @gleb-osokin , I'm reading your description - and will finish asap - but I was wondering if you already knew about multiple named caches: it already supports multiple caches each configured in a totally different way, with their own DefaultEntryOptions, the standard named options pattern, keyed services and all the rest.

Will update when I have finished reading, but wanted to share that immediately.

Let me know, thanks!

[gleb-osokin]

Hi @jodydonetti , thank you so much for taking a look at this!

I see that named caches is a very powerful feature indeed. What I wanted to avoid is having to inject multiple different caches into the consumer - they shouldn't really care, what kind of cache it is, they know only the cache key and the data, while all the rest can be governed through the configuration/setup. Ideally I'd want to specify in appsettings.json only those properties of key-dependent entry options, that are different from the default, whereas all the rest would come from DefaultEntryOptions. This is achievable on the configuration stage, even before the FusionCache setup.

In reality, I think, the most common setting that is going to be set up in those key-dependent options is the cache duration. At least that's the case for my setup.

Let me know if all that makes sense 😅

[jodydonetti]

Hi @gleb-osokin , sorry for the delay but with v2 and all the things happened after that I've been carried away.

This gave me some food for thought, but after some consideration I feel like this should not be a part of FusionCache itself since it's a very niche feature, and one that can introduce potential unintended consequences (like a match on some internal stuff, maybe used for tagging or similar).
I see this a candidate for an ext method or similar (like an extra GetOrSet overload) in a private project/package, and not a full fledged official feature.
Anyway I'd like to thank you for the contribution, it gave me food for thoughts and allowed me to see a new scenario which I haven't thought of before, which is always a net positive!

[gleb-osokin-gen]

Hi @jodydonetti , many thanks, this totally makes sense. To be frank, I also thought that such a change might be a bit of an overkill and too use case-specific for a general caching library. I nevertheless wanted to share my thoughts about the configuration and I really appreciate that you've taken a look at those.

Do you think, though, that a custom options factory (like here, but with a more generic name) might become beneficial as a possible extension point? Then any options-building strategy could be applied independently from the library itself.

[jodydonetti]

Hi @gleb-osokin , I'm getting back to this.

Do you think, though, that a custom options factory (like here, but with a more generic name) might become beneficial as a possible extension point? Then any options-building strategy could be applied independently from the library itself.

So to recap, after thinking a bit about it, what about this?

public class FusionCacheOptions {
  // ...
  public Func<string, FusionCacheEntryOptions, FusionCacheEntryOptions> DefaultEntryOptionsBuilder { get; set; }
}

The signature is such that your method will receive both the cache key and the normal DefaultEntryOptions, so you can do whatever you want with it:

// 1. BASIC:
// - ALWAYS CREATE A NEW ONE
DefaultEntryOptionsBuilder = (key, deo) => {
  return new FusionCacheEntryOptions().SetFailSafe(...);
};

// 2. MORE COMPLEX:
// - IF KEY STARTS WITH "foo/" -> CREATE A NEW ONE
// - OTHERWISE JUST RETURN THE DEFAULT ONE
DefaultEntryOptionsBuilder = (key, deo) => {
  if (key.StartsWith("foo/")) {
    // CREATE A NEW ONE
    return new FusionCacheEntryOptions().SetFailSafe(...);
  }
  
  // RETURN THE DEFAULT ONE
  return deo;
};

// 3. MOST COMPLEX:
// - IF KEY STARTS WITH "foo/" -> CREATE A NEW ONE
// - IF KEY STARTS WITH "bar/" -> DUPLICATE THE DEFAULT ONE AND CHANGE SOMETHING
// - OTHERWISE JUST RETURN THE DEFAULT ONE
DefaultEntryOptionsBuilder = (key, deo) => {
  if (key.StartsWith("foo/")) {
    return new FusionCacheEntryOptions().SetFailSafe(...);
  }
  
  if (key.StartsWith("bar/")) {
    return deo.Duplicate().SetDuration(...);
  }
  
  return deo;
};

Important

NOTE: mind you nothing is confirmed yet. As of now I'm playing with the idea, but I still need to implement it and see what happens in all edge cases, etc.

Let me know what you think, thanks.

[gleb-osokin-gen]

Thanks @jodydonetti , I very much like the idea! This will enable any custom per-key configuration, exactly what I need.
The default entry options (deo, in this case), should probably already be duplicated from source by the time the builder is called, right? Overall, it looks terrific already, thank you for investing time in this!

[jodydonetti]

The default entry options (deo, in this case), should probably already be duplicated from source by the time the builder is called, right?

I was thinking the opposite, mostly because of saving resources: if the lambda just need to return the defaults or if it needs to return a completely new one, duplicating the defaults would be a waste.
I was thinking about putting something evident in the xml comments to make it clear.
Thoughts?

Overall, it looks terrific already, thank you for investing time in this!

Thanks to you for giving the idea!

[gleb-osokin-gen]

As far as I understood from the current code, the default entry options are stored as a single instance within the cache object. So aren't we risking breaking things by exposing it to the builder? Especially considering the multi-threaded nature of the cache operations... It is way too easy to modify the provided instance instead of creating a new one.

As an alternative, we could have an argument, that allows us to create the duplicate on demand:

DefaultEntryOptionsBuilder = (EntryOptionsBuilderArgs args) => {
    if (args.Key.StartsWith("foo/") {
        return args.CreateDefaultEntryOptions().SetDuration(...);
    }

    return null; // should use unchanged default
};

[jodydonetti]

As far as I understood from the current code, the default entry options are stored as a single instance within the cache object.

Correct.

So aren't we risking breaking things by exposing it to the builder? Especially considering the multi-threaded nature of the cache operations... It is way too easy to modify the provided instance instead of creating a new one.

Also correct. One note here: today is already possible to access cache.DefaultEntryOptions, BUT it's also less error prone.
So, I agree with you here.

As an alternative, we could have an argument, that allows us to create the duplicate on demand:

DefaultEntryOptionsBuilder = (EntryOptionsBuilderArgs args) => {
    if (args.Key.StartsWith("foo/") {
        return args.CreateDefaultEntryOptions().SetDuration(...);
    }

    return null; // should use unchanged default
};

And this, my friends, is why it's good to publicly share a design you think it's good, and get back an even better design in return!

Good point, I think I'll follow your suggestion.

Will update, thanks again.

[jodydonetti]

UPDATE: moved here for better tracking.

Please answer there, thanks!

---

Hi @gleb-osokin-gen , I'm exploring the implementation of the feature, looking at edge cases and whatnot.

Something that I'd like to know your view on is how to define this new thing.
I see 2 possibilities to define the "entry options builder", as either one of these:

"is used to get a per-key DefaultEntryOptions"

or:

"is used to get a per-key entry options when nothing is passed to the main method call (eg: GetOrSet)"

The difference is subtle, and it is mainly about what should happen when something like this is called:

cache.GetOrSet(
  "foo",
  factory,
  options => options.SetDuration(...)
);

Right now FusionCache takes the DefaultEntryOptions, duplicates it, and apply the logic specified with options => options....

So the question is: with this new "entry options builder" thing should it:

"start from the normal DefaultEntryOptions"

or:

"start from the per-key one, if the logic has been specified"

?

I have a clear answer in my mind for... reasons, but I'd like to see your point of view.

Thanks!

[jodydonetti]

Hi @gleb-osokin-gen , meanwhile I created a separate issue to keep track of this, see here.

I'm copy pasting there my last question, so can you answer there for better readability by the community?

Thanks!

[jodydonetti]

Hi all, v2.3.0 is finally out 🥳