outbound: add backend and route metadata to errors

[hawkw]

PRs #2418 and #2419 add per-route and per-backend request timeouts
configured by the OutboundPolicies API to the MatchedRoute and
MatchedBackend layers in the outbound ClientPolicy stack,
respectively. This means that — unlike in the ServiceProfile stack —
two separate request timeouts can be configured in ClientPolicy
stacks. However, because both the MatchedRoute and MatchedBackend
layers are in the HTTP logical stack, the errors emitted by both
timeouts will have a LogicalError as their most specific error
metadata, meaning that the log messages and l5d-proxy-error headers
recorded for these timeouts do not indicate whether the timeout that
failed the request was the route request timeout or the backend request
timeout.

In order to ensure this information is recorded and exposed to the user,
this branch adds two new error wrapper types, one of which enriches an
error with a RouteRef's metadata, and one of which enriches an error
with a BackendRef's metadata. The MatchedRoute stack now wraps all
errors with RouteRef metadata, and the MatchedBackend stack wraps
errors with BackendRef metadata. This way, when the route timeout
fails a request, the error will include the route metadata, while when
the backend request timeout fails a request, the error will include both
the route and backend metadata.

Adding these new error wrappers also has the additional side benefit of
adding this metadata to errors returned by filters, allowing users to
distinguish between errors emitted by a filter on a route rule and
errors emitted by a per-backend filter. Also, any other errors emitted
lower in the stack for requests that are handled by a client policy
stack will now also include this metadata, which seems generally useful.

Example errors, taken from a proxy unit test:

backend request:

logical service logical.test.svc.cluster.local:666: route httproute.test.timeout-route: backend service.test.test-svc:666: HTTP response timeout after 1s

route request:

logical service logical.test.svc.cluster.local:666: route httproute.test.timeout-route: HTTP response timeout after 2s

[adleong]

This looks good. Can you give examples of what these errors look like in practice for route timeouts and backend timeouts? i.e. in l5d-proxy-error

[hawkw]

@adleong

This looks good. Can you give examples of what these errors look like in practice for route timeouts and backend timeouts? i.e. in l5d-proxy-error

backend request:

logical service logical.test.svc.cluster.local:666: route httproute.test.timeout-route: backend service.test.test-svc: HTTP response timeout after 1s

route request:

logical service logical.test.svc.cluster.local:666: route httproute.test.timeout-route: HTTP response timeout after 2s

(these are from a proxy unit test, so i may have formatted the metadata slightly differently from how the policy controller would)