Introduce a settable class-level cache for Version

[sirosen]

Opened as a draft to enable discussion.

I took a crack at caching Version instances using __new__, as suggested by @notatallshaw in some prior conversation.
The initial results are promising but also raise questions.

First, there's the question of the interface proposed here.
Allowing the cache to be passed in from the outside has some nice benefits, in that any cache policy is fully in the control of the caller.
But it does force the Version cache to operate in terms of some external object -- the caching behavior is only as efficient as the choice of mapping type for the cache.

Without caching enabled, there is a measurable slowdown of ~10% for several benchmarks.
With a Version.set_constructor_cache({}) call added, there's an improvement of as much as 80%.
The added overhead seems quite high, arguably too high.
EDIT: the CI runs show an even more severe penalty than my local runs. It may be that using a contextvar is just too costly.

Overall, I'm curious where the maintainers think this might go as an idea, and about suggestions for ways to alter the approach.

---

Version._constructor_cache is a contextvar (as a classvar) containing
a mutable mapping of strings-to-Version-instances, or None.

Version.__new__ now checks the cache and will return the same instance
of Version for the same string input if there is a cache hit.

Version.__init__ is updated ot check for an existing _release
attribute in order to determine whether or not it's being called on an
already initialized instance and short-circuit appropriately.

In order to enable and control the cache, Version has a new public API,
Version.set_constructor_cache, which accepts a mutable mapping or None
(with None being the indicator for "disable the cache").
set_constructor_cache documents that the setting is a contextvar, which
provides callers with sufficient information to make reasonable decisions
about concurrency.

No attempt is made to protect against data races if the same cache is
used in multiple threads. However, it is valid to use a single
cache in such use cases and to protectively lock around calls to
packaging functionality. Failing to lock in such usages could result
in data races and handling Version objects which are in the midst of
being initialized in another thread, due to the short-circuiting check
in Version.__init__.

Policies around cache lifetimes and evictions, e.g., an LRU policy, are
entirely external to the cache interface, which only requires that a
cache is provided.

[sirosen]

I've rewritten to a global cache because the penalty paid for a contextvar was so big. The design remains the same, and the docstring for the cache setting method now notes that there is no locking around the cache because it's significant to proper usage.

I also added a new benchmark, which runs a copy of the Version constructor benchmark with the cache enabled. Because there are already duplicates in version_sample.txt (intentionally?), but they don't dominate the sample, this already should show some results. We could "juice the numbers" here by running the loop of that benchmark twice, but I'm not sure what's appropriate on that front.

---

Update:
I've looked at the benchmark results and they're still poor. I think this approach might just not be viable. I'm trying a few more tuning tricks but I'm not sure anything will work in this space.

[sirosen]

I pushed a final commit with my last ditch effort on this, but I can already tell from local benchmarks that it's not going to work. I'm self-closing and will try to re-approach the issue without relying on __new__.

[notatallshaw]

Thanks a lot for trying, the dispatch to a Python __new__ is probably not well optimized in CPython 🙁