Investigate how Aries-VCR is performing paging when querying Solr

[WadeBarnes]

As discussed some of the articles listed here certain workarounds for paging can be problematic for Solr. Particular search queries seem to have an effect on the search-engine container (Solr), causing it to use a significant amount of memory.

Example queries (note the high page numbers):

/api/v4/search/topic?format=json&ordering=-revoked_date&page=149713
/api/v4/search/topic?format=api&ordering=revoked_date&page=2721
/api/v4/search/credential/facets?format=api&page=318768
/api/v4/search/credential?ordering=effective_date&page=318772

This issue is related to the investigation happening here; bcgov/orgbook-configurations#137

[WadeBarnes]

Try this first; #758

[WadeBarnes]

Disabling the Solr caches has been ineffective; #758 (comment), #758

I think it's time we get together to discuss next steps on how to proceed. A good start would be to assign someone to review, and then we can go over the tools we have for monitoring and what we've discovered about the issue so far.

[WadeBarnes]

The issues we were seeing with Solr were caused by unbound queries (#762) and likely made worse by the paging implementation.

[WadeBarnes]

Since we added support in the proxy to limit paging queries we have not had any issues with SOLR, indicating the way the paging queries are being constructed and executed is an issue.