logging: add SetFilteredKlogLogger to show client-go throttling logs

[sttts]

Description of your changes

It's hard to understand which component is causing slowness. This PR enables client throttling logging, both for server-side and client-side, i.e. these pass:

switch {
  case len(retryInfo) > 0:
    message = fmt.Sprintf("Waited for %v, %s - request: %s:%s", latency, retryInfo, r.verb, r.URL().String())
  default:
    message = fmt.Sprintf("Waited for %v due to client-side throttling, not priority and fairness, request: %s:%s", latency, r.verb, r.URL().String())
}

For example:

Waited for 1.033772408s due to client-side throttling, not priority and fairness, request: GET:https://api.example.org/apis/pkg.crossplane.io/v1?timeout=32s
Waited for 580ms, request: GET:https://api.example.org/apis/pkg.crossplane.io/v1?timeout=32s - retries: 1, retry-after: 1s - retry-reason: due to server-side throttling, FlowSchema UID: ""

This is just in Crossplane. It probably needs a log setup helper in controller-runtime and then vendored into providers.

Of course, this information could also be understood from metrics, but metrics are complicated, hard to discover and need tons of knowledge to understand right.

Mr Prosser: But the plans were on display…
Arthur: On display? I eventually had to go down to the cellar to find them.
Mr Prosser: That’s the display department.
Arthur: With a torch.
Mr Prosser: The lights had probably gone out.
Arthur: So had the stairs.
Mr Prosser: But look, you found the notice, didn’t you?
Arthur: Yes yes I did. It was on display at the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying beware of the leopard.

I have:

• Read and followed Crossplane's contribution process.
• Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

Tested in crossplane/crossplane#5419.

[negz]

One nit but otherwise LGTM

[negz]

Perhaps this should have something WRT "request throttling" in the name, to make it obvious why it's filtering. Unless you think we might include more in future?

[sttts]

Was thinking that maybe we will add more some day, i.e. this is more like the standard klog setup for crossplane components.