diff --git a/.github/ISSUE_TEMPLATE/conformance.md b/.github/ISSUE_TEMPLATE/conformance.md new file mode 100644 index 0000000000..1643a02f76 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/conformance.md @@ -0,0 +1,11 @@ +--- +name: Conformance Test +about: Suggest a new Conformance Test +labels: area/conformance + +--- + + +**What would you like to be added**: + +**Why this is needed**: diff --git a/.github/ISSUE_TEMPLATE/enhancement.md b/.github/ISSUE_TEMPLATE/enhancement.md index b58a4fcaef..99e48046a6 100644 --- a/.github/ISSUE_TEMPLATE/enhancement.md +++ b/.github/ISSUE_TEMPLATE/enhancement.md @@ -6,12 +6,15 @@ labels: kind/feature --- + + **What would you like to be added**: **Why this is needed**: - diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 4c6170d9fb..41dc9f8355 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,8 +1,8 @@ **What type of PR is this?** @@ -23,15 +27,15 @@ Add one of the following kinds: /kind cleanup /kind documentation /kind feature -/kind design /kind gep +/kind test Optionally add one or more of the following kinds if applicable: -/kind api-change /kind deprecation /kind failing-test /kind flake /kind regression +/area conformance --> **What this PR does / why we need it**: diff --git a/.gitignore b/.gitignore index a8a804e1fa..85908ee122 100644 --- a/.gitignore +++ b/.gitignore @@ -55,3 +55,4 @@ admission /cache .venv/ release/ +site-src/geps diff --git a/.golangci.yml b/.golangci.yml index 14b5a1a786..5f0027814c 100644 --- a/.golangci.yml +++ b/.golangci.yml @@ -1,6 +1,8 @@ run: timeout: 10m issues-exit-code: 1 + max-issues-per-linter: 0 + max-same-issues: 0 tests: true skip-dirs-use-default: true modules-download-mode: readonly @@ -9,12 +11,20 @@ run: linters: fast: false enable: - - gofmt + - errcheck + - exportloopref + - gocritic + - gofumpt - goimports - - revive + - gomodguard + - gosec - govet - misspell - - exportloopref + - revive + - unconvert + - unparam + - unused + - whitespace disable: - scopelint disable-all: false @@ -36,7 +46,16 @@ linters-settings: check-shadowing: true misspell: locale: US - ignore-words: + ignore-words: [] + gomodguard: + blocked: + # List of blocked modules. + modules: + - io/ioutil: + recommendations: + - io + - os + reason: "Deprecation of package ioutil in Go 1.16." issues: exclude-rules: diff --git a/.yamllint.yaml b/.yamllint.yaml new file mode 100644 index 0000000000..fe59aaa5eb --- /dev/null +++ b/.yamllint.yaml @@ -0,0 +1,57 @@ +--- + +ignore: | + .github/ + .golangci.yml + mkdocs.yml + cloudbuild.yaml + config/crd/kustomization.yaml + +rules: + braces: + min-spaces-inside: 0 + max-spaces-inside: 0 + min-spaces-inside-empty: -1 + max-spaces-inside-empty: -1 + brackets: + min-spaces-inside: 0 + max-spaces-inside: 1 + min-spaces-inside-empty: -1 + max-spaces-inside-empty: -1 + colons: + max-spaces-before: 0 + max-spaces-after: 1 + commas: + max-spaces-before: 1 + min-spaces-after: 1 + max-spaces-after: 1 + comments: + level: warning + require-starting-space: false + min-spaces-from-content: 1 + comments-indentation: + level: warning + document-end: disable + document-start: disable + empty-lines: + max: 2 + max-start: 0 + max-end: 1 + empty-values: + forbid-in-block-mappings: false + forbid-in-flow-mappings: true + hyphens: + max-spaces-after: 1 + indentation: + spaces: 2 + indent-sequences: consistent # be consistent: don't mix indentation styles in one file. + check-multi-line-strings: false + key-duplicates: enable + key-ordering: disable + new-line-at-end-of-file: enable + new-lines: + type: unix + trailing-spaces: disable + truthy: + check-keys: false # GitHub Actions uses "on:" as a key + level: warning diff --git a/CHANGELOG.md b/CHANGELOG.md index 8fab3221bb..8baebf1a7e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ ## Table of Contents +- [v0.7.1](#v071) +- [v0.7.0](#v070) +- [v0.7.0-rc2](#v070-rc2) +- [v0.7.0-rc1](#v070-rc1) +- [v0.6.2](#v062) +- [v0.6.1](#v061) - [v0.6.0](#v060) - [v0.6.0-rc2](#v060-rc2) - [v0.6.0-rc1](#v060-rc1) @@ -21,6 +27,370 @@ - [v0.1.0-rc2](#v010-rc2) - [v0.1.0-rc1](#v010-rc1) +# v0.7.1 + +This is a patch release that includes small fixes, clarifications, and +conformance tests as a follow up to the v0.7.0 release. + +## Changes by Kind + +### Conformance Tests + +- Fixed an issues causing conformance tests to fail when using IPv6 addresses. + (#2024, @howardjohn) +- HTTPRoute connectivity is in now enforced in conformance tests if a relevant + ReferenceGrant gets deleted. (#1853, @pmalek) +- New: Conformance tests for HTTP request mirroring. (#1912, @liorlieberman) +- Fixes to port and scheme redirect tests: Tests now send HTTPS requests with + consistent SNI and Host, Gateway now has the correct SANs. (#2039, @sunjaybhatia) +- TLSRoute test now waits for namespaces to be ready. (#2067, @skriss) + +### Validating Webhook + +- Webhook config works with "restricted" Pod Security level. (#2016, @jcpunk) + +### Clarifications + +- HTTPRoute Method matching precedence has been clarified. (#2054, + @gauravkghildiyal) +- Implementations MUST ignore any port value specified in the HTTP Host header + while performing a match against HTTPRoute.Hostnames. (#1980, + @gauravkghildiyal) +- HTTPRoute: Clarified that exact path matches are truly exact, both trailing + slashes and capitalization are meaningful. (#2055, @robscott) +- Gateway: Clarified that AttachedRoutes should only consider Routes that have + been accepted. (#2050, @mlavacca) + +# v0.7.0 + +The v0.7.0 release focuses on refining and stabilizing existing APIs. This +included a focus on both conformance tests and clarifying ambiguous parts of the +API spec. + +## Features Graduating to Standard +In addition to those broad focuses, 2 features are graduating to the +standard channel: + +* GEP-1323: Response Header Modifiers (#1905, @robscott) +* GEP-726: Path Redirects and Rewrites (#1905, @robscott) + +## GEPs +There are a lot of interesting GEPs in the pipeline right now, but only some of +these GEPs have made it to experimental status in time for v0.7.0. The GEPs +highlighted below are both in an experimental state and are either entirely new +(GEP-1748) or had significant new concepts introduced (GEP-713): + +### GEP-713: Policy Attachment +This GEP received a major update, splitting policy attachment into two +categories "Direct" and "Inherited". The new "Direct" mode enables a simplified +form of policy attachment for targeting a single resource (#1565, @youngnick). + +### GEP-1748: Gateway API Interaction with Multi-Cluster Services +A new GEP was introduced to define how Gateway API interacts with Multi-Cluster +Services. At a high level, this states that ServiceImports have "Extended" +support and can be used anywhere Services can throughout the API. There's a lot +more nuance here, so for the full details, refer to the GEP. (#1843, @robscott) + +## Other Changes by Kind + +### Status Changes + +- The "Ready" Gateway and Listener condition has been reserved for future use. + (#1888, @howardjohn) +- The UnsupportedAddress Listener condition reason has been moved to a Gateway + condition reason. (#1888, @howardjohn) +- The AddressNotAssigned Gateway condition reasons has moved from Accepted to + Programmed. (#1888, @howardjohn) +- The NoResources Gateway condition reasons has moved from Ready to Programmed. + (#1888, @howardjohn) + +### Spec Cleanup + +- Clarification that port redirects should not add port number to Location + header for HTTP and HTTPS requests on 80 and 443. (#1908, @robscott) +- Port redirect when empty will depend on the configured Redirect scheme (#1880, + @gauravkghildiyal) +- Updated spec to clarify that Exact matches have precedence over Prefix matches + and RegularExpression matches have implementation specific precedence. (#1855, + @Xunzhuo) +- The `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer is no + longer required and is now just recommended. (#1917, @howardjohn) + +### Validation Fixes + +- Removes GRPCRoute method match defaulting to allow for matching all requests, + or matching only by header. (#1753, @skriss) +- Update route validation to comply with RFC-3986 "p-char" characters. (#1644, + @jackstine) +- Illegal names like " " will be not allowed for query param name in + HTTPQueryParamMatch. (#1796, @gyohuangxin) +* Webhook: Port is now considered when validating that ParentRefs are unique + (#1995, @howardjohn) + +### Conformance + +- No conformance tests run by default anymore, including tests for GatewayClass + and Gateway. A new SupportGateway feature must be opted into in order to run + those tests (similar to what we've done previously for ReferenceGrant and + HTTPRoute). Also with this release, `EnableAllSupportedFeatures` enables all + Gateway AND Mesh features (where previously that was just Gateway). (#1894, + @shaneutt) +- Gateways must publish the "Programmed" condition. (#1732, @robscott) +- Add `all-features` flag to enable all supported feature conformance tests. + (#1642, @gyohuangxin) +- A new SkipTests field has been added to the conformance test options to + opt-out of specific tests. (#1578, @mlavacca) +- Added: conformance tests for http rewrite host and path filters. (#1622, + @LiorLieberman) +- In Conformance tests, when a Route references a gateway having no listener + whose allowedRoutes criteria permit the route, the reason + NotAllowedByListeners should be used for the accepted condition. (#1669, + @mlavacca) +- Support configurable timeout for GatewayObservedGenerationBump (#1887, + @Xunzhuo) +- The conformance test HTTPRouteInvalidCrossNamespaceParentRef now requires the + HTTPRoute accepted condition to be failing with the ParentRefNotPermitted + reason. (#1694, @mlavacca) +- The conformance tests always check that the HTTPRoute ResolvedRefs condition + is enforced, even when the status is true. (#1668, @mlavacca) +- Checks for the NotAllowedByListeners reason on the HTTPRoute's Accepted: false + condition in the HTTPRouteInvalidCrossNamespaceParentRef conformance test. + (#1714, @skriss) +- Added conformance test to verify that path matching precedence is + implemented correctly. (#1855, @Xunzhuo) +- Remove a test that only covered redirect status without any other changes. + (#2007, @robscott) +- Port redirect when empty will depend on the configured Redirect scheme (#1880, + @gauravkghildiyal) +- Fixes for mesh conformance tests (#2017, @keithmattix) + +### Documentation + +- Updated outdated content on list of resources in installation guide page. + (#1857, @randmonkey) +- Fix description of ReferenceGrant example in documentation by making it use + the correct resources. (#1864, @matteoolivi) +- Fix grammar mistake in ReferenceGrant implementation guidelines. (#1865, + @matteoolivi) + +# v0.7.0-rc2 + +We expect this to be our final release candidate before launching v0.7.0. This +release candidate includes a variety of clarifications and conformance updates. +The changelog below represents the changes since v0.7.0-rc1. + +## Changes by Kind + +### Spec Clarification + +- Port redirect when empty will depend on the configured Redirect scheme (#1880, + @gauravkghildiyal) + +### Conformance + +- Remove a test that only covered redirect status without any other changes. + (#2007, @robscott) +- Port redirect when empty will depend on the configured Redirect scheme (#1880, + @gauravkghildiyal) + +### Validation Fixes + +* Webhook: Port is now considered when validating that ParentRefs are unique + (#1995, @howardjohn) + +# v0.7.0-rc1 + +## Changes by Kind + +### Graduating to Standard + +- GEP-1323: Response Header Modifier has graduated to standard (#1905, + @robscott) +- GEP-726: Path Redirects and Rewrites has graduated to the standard channel. + (#1874, @robscott) + +### Experimental GEPs + +- The Policy Attachment GEP received a major update, splitting policy attachment + into two categories "Direct" and "Inherited". The new "Direct" mode enables a + simplified form of policy attachment for targeting a single resource (#1565, + @youngnick) +- A new GEP was introduced to define how Gateway API interacts with + Multi-Cluster Services (#1843, @robscott) + +### Status Changes + +- The "Ready" Gateway and Listener condition has been reserved for future use. + (#1888, @howardjohn) +- The UnsupportedAddress Listener condition reason has been moved to a Gateway + condition reason. (#1888, @howardjohn) +- The AddressNotAssigned Gateway condition reasons has moved from Accepted to + Programmed. (#1888, @howardjohn) +- The NoResources Gateway condition reasons has moved from Ready to Programmed. + (#1888, @howardjohn) + +### Spec Cleanup + +- Clarification that port redirects should not add port number to Location + header for HTTP and HTTPS requests on 80 and 443. (#1908, @robscott) +- Updated spec to clarify that Exact matches have precedence over Prefix matches + and RegularExpression matches have implementation specific precedence. (#1855, + @Xunzhuo) +- The `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer is no + longer required and is now just recommended. (#1917, @howardjohn) + +### Validation Fixes + +- Removes GRPCRoute method match defaulting to allow for matching all requests, + or matching only by header. (#1753, @skriss) +- Update route validation to comply with RFC-3986 "p-char" characters. (#1644, + @jackstine) +- Illegal names like " " will be not allowed for query param name in + HTTPQueryParamMatch. (#1796, @gyohuangxin) + +### Conformance + +- No conformance tests run by default anymore, including tests for GatewayClass + and Gateway. A new SupportGateway feature must be opted into in order to run + those tests (similar to what we've done previously for ReferenceGrant and + HTTPRoute). Also with this release, `EnableAllSupportedFeatures` enables all + Gateway AND Mesh features (where previously that was just Gateway). (#1894, + @shaneutt) +- Gateways must publish the "Programmed" condition. (#1732, @robscott) +- Add `all-features` flag to enable all supported feature conformance tests. + (#1642, @gyohuangxin) +- A new SkipTests field has been added to the conformance test options to + opt-out of specific tests. (#1578, @mlavacca) +- Added: conformance tests for http rewrite host and path filters. (#1622, + @LiorLieberman) +- In Conformance tests, when a Route references a gateway having no listener + whose allowedRoutes criteria permit the route, the reason + NotAllowedByListeners should be used for the accepted condition. (#1669, + @mlavacca) +- Support configurable timeout for GatewayObservedGenerationBump (#1887, + @Xunzhuo) +- The conformance test HTTPRouteInvalidCrossNamespaceParentRef now requires the + HTTPRoute accepted condition to be failing with the ParentRefNotPermitted + reason. (#1694, @mlavacca) +- The conformance tests always check that the HTTPRoute ResolvedRefs condition + is enforced, even when the status is true. (#1668, @mlavacca) +- Checks for the NotAllowedByListeners reason on the HTTPRoute's Accepted: false + condition in the HTTPRouteInvalidCrossNamespaceParentRef conformance test. + (#1714, @skriss) +- Added conformance test to verify that path matching precedence is + implemented correctly. (#1855, @Xunzhuo) + +### Documentation + +- Updated outdated content on list of resources in installation guide page. + (#1857, @randmonkey) +- Fix description of ReferenceGrant example in documentation by making it use + the correct resources. (#1864, @matteoolivi) +- Fix grammar mistake in ReferenceGrant implementation guidelines. (#1865, + @matteoolivi) + +# v0.6.2 + +This is a patch release that predominantly includes updated conformance tests +for implementations to implement. + +For all major changes since the `v0.5.x` release series, please see the +[v0.6.0](/#v060) release notes. + +## Maintenance + +- As per [changes in upstream to container image registries] we replaced all + usage of the k8s.gcr.io registry with registry.k8s.io. + (#1736, @shaneutt) + +[changes in upstream to container image registries]:https://github.com/kubernetes/k8s.io/issues/4738 + +## Bug Fixes + +- Fix invalid HTTP redirect/rewrite examples. + (#1787, @Xunzhuo) + +## Conformance Test Updates + +- The `HTTPRouteInvalidCrossNamespaceParentRef` conformance test now checks for + the `NotAllowedByListeners` reason on the `HTTPRoute`'s `Accepted: false` + condition to better indicate why the route was note accepted. + (#1714, @skriss) +- A conformance test was added for `HTTPRoute` to cover the behavior of a + non-matching `SectionName` similar to what was already present for + `ListenerPort`. + (#1719, @zaunist) +- Fixed an issue where tests may fail erroneously on the removal of resources + that are already removed. + (#1745, @mlavacca) +- Logging in conformance utilities related to resource's `ObservedGeneration` + has been improved to emit the `ObservedGenerations that are found for the + purpose of making it easier to debug test failures and be more verbose about + the objects in question. + (#1761, @briantkennedy) + (#1763, @briantkennedy) +- Patch instead of update in some places in conformance tests to reduce noise + in logs. + (#1760, @michaelbeaumont) +- Added `AttachedRoutes` testing to conformance tests. + (#1624, @ChaningHwang) +- The conformance tests always check that the HTTPRoute ResolvedRefs condition + is enforced, even when the status is true. + (#1668, @mlavacca) + +# v0.6.1 + +This is a patch release that predominantly includes updated conformance tests +for implementations to implement. + +For all major changes since the `v0.5.x` release series, please see the +[v0.6.0](/#v060) release notes. + +## Bug Fixes + +- Our regex for validating path characters was updated to accurately identify + "p-chars" as per RFC-3986. + (#1644, @jackstine) +- An erroneous "namespace" field was present in our webhook ClusterRoleBindings + and has been removed. + (#1684, @tao12345666333) + +## New Features + +- Conditions for Policies have been added to the Golang library, enabling + Go-based implementations to re-use those for their downstream Policies. + (#1682, @mmamczur) + +## Conformance Test Updates + +- Added conformance tests for checking Port, Scheme and Path to the extended and + experimental features. + (#1611, @LiorLieberman) +- Added conformance tests for HTTP rewrite + (#1622, #1628, @LiorLieberman) +- Added more conformance tests for path matching to catch known edge cases. + (#1627, @sunjayBhatia) +- Added some initial conformance tests for TLSRoute passthrough. + (#1579, @candita) +- Added conformance tests that exercise NotAllowedByListeners reason. + (#1669, @mlavacca) +- Loosen the Accepted check in GatewayClass observed generation tests to + provide a more realistic test for implementations. + (#1655, @arkodg) +- A "SkipTests" field has been added to accomodate implementations in + running subsets of the tests as needed, this can be particularly helpful + for new implementations that want to add conformance iteratively. + (#1578, @mlavacca) +- Fixed a broken test for GRPCRoute that caused an erronous failure. + (#1692, @arkodg) +- Added "all-features" flag to conformance test to enable all supported + features on test runs. + (#1642, @gyohuangxin) +- Fixed usage of `net/http` default client in conformance test suite + (#1617, @howardjohn) +- Fixed missing reference to NoMatchingParent in godoc + (#1671, @mlavacca) + # v0.6.0 ## Major Changes @@ -723,7 +1093,7 @@ The following changes have been made since v0.3.0: Implemented in [#754](https://github.com/kubernetes-sigs/gateway-api/pull/754). Further documentation was added in [#762](https://github.com/kubernetes-sigs/gateway-api/pull/762). -* Safer cross-namespace references ([GEP-709](https://gateway-api.sigs.k8s.io/geps/gep-709/)): +* Safer cross-namespace references ([GEP-709](https://gateway-api.sigs.k8s.io/geps/gep-709/)): This concerns (currently), references from Routes to Backends, and Gateways to Secrets. The new behavior is: * By default, references across namespaces are not permitted; creating a @@ -785,7 +1155,7 @@ The following changes have been made since v0.3.0: GEP added in [#749](https://github.com/kubernetes-sigs/gateway-api/pull/749). Implemented in [#768](https://github.com/kubernetes-sigs/gateway-api/pull/768). - [GEP-851](https://github.com/kubernetes-sigs/gateway-api/blob/main/site-src/geps/gep-851.md) + [GEP-851](https://github.com/kubernetes-sigs/gateway-api/blob/main/geps/gep-851.md) was a follow up on this change that allowed multiple Certificate Refs per Gateway Listener. This was implemented in [#852](https://github.com/kubernetes-sigs/gateway-api/pull/852). @@ -800,7 +1170,7 @@ The following changes have been made since v0.3.0: ### Small Changes * Extension points within match blocks from all Routes have been removed [#829](https://github.com/kubernetes-sigs/gateway-api/pull/829). Implements - [GEP-820](https://github.com/kubernetes-sigs/gateway-api/blob/main/site-src/geps/gep-820.md). + [GEP-820](https://github.com/kubernetes-sigs/gateway-api/blob/main/geps/gep-820.md). These extension points have been removed because they are currently not used, are poorly understood, and we don't have good use cases for them. We may consider re-adding them in the future. @@ -947,11 +1317,11 @@ The following changes have been made since v0.4.0-rc1: ### GEP implementations * Replace `CertificateRef` field with `CertificateRefs` in `GatewayTLSConfig`. [#852](https://github.com/kubernetes-sigs/gateway-api/pull/852). This implements -[GEP-851](https://github.com/kubernetes-sigs/gateway-api/blob/main/site-src/geps/gep-851.md), +[GEP-851](https://github.com/kubernetes-sigs/gateway-api/blob/main/geps/gep-851.md), Allow Multiple Certificate Refs per Gateway Listener. * Extension points within match blocks from all Routes have been removed [#829](https://github.com/kubernetes-sigs/gateway-api/pull/829). Implements -[GEP-820](https://github.com/kubernetes-sigs/gateway-api/blob/main/site-src/geps/gep-820.md). +[GEP-820](https://github.com/kubernetes-sigs/gateway-api/blob/main/geps/gep-820.md). These extension points have been removed because they are currently not used, are poorly understood, and we don't have good use cases for them. We may consider re-adding them in the future. diff --git a/Makefile b/Makefile index 9cf42b3118..bf686793f2 100644 --- a/Makefile +++ b/Makefile @@ -52,6 +52,7 @@ ROOT := $(abspath $(TOP)) # Command-line flags passed to "go test" for the conformance # test. These are passed after the "-args" flag. CONFORMANCE_FLAGS ?= +GO_TEST_FLAGS ?= all: generate vet fmt verify test @@ -81,12 +82,17 @@ vet: # Run go test against code test: - go test -race -cover ./pkg/... + go test -race -cover ./pkg/... ./apis/... ./conformance/utils/... # Run conformance tests against controller implementation .PHONY: conformance conformance: - go test -v ./conformance/... -args ${CONFORMANCE_FLAGS} + go test ${GO_TEST_FLAGS} -v ./conformance -args ${CONFORMANCE_FLAGS} + +# Run experimental conformance tests against controller implementation +.PHONY: conformance.experimental +conformance.experimental: + go test ${GO_TEST_FLAGS} --tags experimental -v ./conformance -run TestExperimentalConformance -args ${CONFORMANCE_FLAGS} # Install CRD's and example resources to a pre-existing cluster. .PHONY: install diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES index c4bdfc152c..99bcbbfc06 100644 --- a/OWNERS_ALIASES +++ b/OWNERS_ALIASES @@ -4,8 +4,8 @@ aliases: # Reference: https://github.com/kubernetes/org/blob/main/OWNERS_ALIASES sig-network-leads: - - caseydavenport - - dcbw + - mikezappa87 + - shaneutt - thockin # Reference: https://github.com/kubernetes/org/blob/main/config/kubernetes-sigs/sig-network/teams.yaml @@ -15,12 +15,28 @@ aliases: - shaneutt - youngnick + emeritus-gateway-api-maintainers: + - danehans + - hbagdi + - jpeach + gateway-api-mesh-leads: - howardjohn - keithmattix + - kflynn + + emeritus-gateway-api-mesh-leads: - mikemorris - emeritus-gateway-api-maintainers: - - danehans - - hbagdi - - jpeach + gateway-api-conformance-reviewers: + - arkodg + - LiorLieberman + - michaelbeaumont + - mlavacca + - sunjayBhatia + - xunzhuo + + gateway-api-conformance-approvers: + - arkodg + - mlavacca + - sunjayBhatia diff --git a/README.md b/README.md index c5fbc020e3..da716c9b07 100644 --- a/README.md +++ b/README.md @@ -5,8 +5,8 @@ the specification and Custom Resource Definitions (CRDs). ## Status -The latest supported version is `v1beta1` as released by the [v0.6.0 -release](https://github.com/kubernetes-sigs/gateway-api/releases/tag/v0.6.0) of +The latest supported version is `v1beta1` as released by the [v0.7.0 +release](https://github.com/kubernetes-sigs/gateway-api/releases/tag/v0.7.0) of this project. This version of the API is has beta level support for the following resources: @@ -33,9 +33,9 @@ to understand the API and the use-cases it targets. ### Getting started -Once you have a good understanding of the API at a higher-level, check out -[getting started][getting-started] to install your first Gateway controller and try out -one of the guides. +Once you have a good understanding of the API at a higher-level, check out +[getting started][getting-started] to install your first Gateway controller and try out +one of the guides. ### References diff --git a/RELEASE.md b/RELEASE.md index bac28eed70..e12df697d2 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -37,34 +37,73 @@ release. The following steps must be done by one of the [Gateway API maintainers][gateway-api-team]: -For a major or minor release: +For a **PATCH** release: +- Create a new branch in your fork named something like `/release-x.x.x`. Use the new branch + in the upcoming steps. +- Use `git` to cherry-pick all relevant PRs into your branch. +- Update `pkg/generator/main.go` with the new semver tag and any updates to the API review URL. +- Run the following command `BASE_REF=vmajor.minor.patch make generate` which + will update generated docs and webhook with the correct version info. (Note + that you can't test with these YAMLs yet as they contain references to + elements which wont exist until the tag is cut and image is promoted to + production registry.) +- Create a pull request of the `/release-x.x.x` branch into the `release-x.x` branch upstream + (which should already exist since this is a patch release). Add a hold on this PR waiting for at least + one maintainer/codeowner to provide a `lgtm`. +- Verify the CI tests pass and merge the PR into `release-x.x`. +- Create a tag using the `HEAD` of the `release-x.x` branch. This can be done using the `git` CLI or + Github's [release][release] page. +- Run the `make build-install-yaml` command which will generate install files in the `release/` directory. + Attach these files to the Github release. +- Update the `README.md` and `site-src/guides/index.md` files to point links and examples to the new release. + +For a **MAJOR** or **MINOR** release: - Cut a `release-major.minor` branch that we can tag things in as needed. - Check out the `release-major.minor` release branch locally. - Update `pkg/generator/main.go` with the new semver tag and any updates to the API review URL. -- Run the following command `BASE_REF=vmajor.minor.patch make generate` which will update generated docs - and webhook with the correct version info. Note that the YAMLs will not work until the tag is actually - published in the next step. -- Publish a new Git tag. This can be done using the `git` CLI or Github's [release][release] - page. -- Run the `make build-install-yaml` command which will generate - install files in the `release/` directory -- Attach these files to the Github release. -- Update the `README.md` as needed for any latest release references. - -For an RC release: +- Run the following command `BASE_REF=vmajor.minor.patch make generate` which + will update generated docs and webhook with the correct version info. (Note + that you can't test with these YAMLs yet as they contain references to + elements which wont exist until the tag is cut and image is promoted to + production registry.) +- Verify the CI tests pass before continuing. +- Create a tag using the `HEAD` of the `release-x.x` branch. This can be done using the `git` CLI or + Github's [release][release] page. +- Run the `make build-install-yaml` command which will generate install files in the `release/` directory. + Attach these files to the Github release. +- Update the `README.md` and `site-src/guides/index.md` files to point links and examples to the new release. + +For an **RC** release: - Update `pkg/generator/main.go` with the new semver tag and any updates to the API review URL. -- Run the following command `BASE_REF=vmajor.minor.patch make generate` which will update generated docs - and webhook with the correct version info. Note that the YAMLs will not work until the tag is actually - published in the next step. +- Run the following command `BASE_REF=vmajor.minor.patch make generate` which + will update generated docs and webhook with the correct version info. (Note + that you can't test with these YAMLs yet as they contain references to + elements which wont exist until the tag is cut and image is promoted to + production registry.) - Include the changelog update in this PR. - Merge the update PR. - Tag the release using the commit on `main` where the changelog update merged. This can be done using the `git` CLI or Github's [release][release] page. - Run the `make build-install-yaml` command which will generate - install files in the `release/` directory + install files in the `release/` directory. - Attach these files to the Github release. +### Promoting images to production registry +Gateway API follows the standard kubernetes image promotion process described [here][kubernetes-image-promotion]. + +1. Once the tag has been cut and the image is available in the staging registry, + identify the SHA-256 image digest of the image that you want to promote. +2. Modify the + [k8s-staging-gateway-api/images.yaml](https://github.com/kubernetes/k8s.io/blob/main/registry.k8s.io/images/k8s-staging-gateway-api/images.yaml) + file under [kubernetes/k8s.io](https://github.com/kubernetes/k8s.io) + repository and add the image digest along with the new tag under the correct + component. + 1. Currently, the following images are included: `admission-server`, `echo-server` +3. Create a PR with the above changes. +4. Image will get promoted by [automated prow jobs][kubernetes-image-promotion] + once the PR merges + [release]: https://github.com/kubernetes-sigs/gateway-api/releases [gateway-api-team]: https://github.com/kubernetes/org/blob/main/config/kubernetes-sigs/sig-network/teams.yaml - +[kubernetes-image-promotion]: https://github.com/kubernetes/k8s.io/tree/main/registry.k8s.io#image-promoter diff --git a/apis/v1alpha2/doc.go b/apis/v1alpha2/doc.go index 0fcba7318c..68e1659551 100644 --- a/apis/v1alpha2/doc.go +++ b/apis/v1alpha2/doc.go @@ -16,6 +16,7 @@ limitations under the License. // Package v1alpha2 contains API Schema definitions for the // gateway.networking.k8s.io API group. +// // +kubebuilder:object:generate=true // +groupName=gateway.networking.k8s.io package v1alpha2 diff --git a/apis/v1alpha2/gateway_types.go b/apis/v1alpha2/gateway_types.go index a2e540ae6a..9eed72d50d 100644 --- a/apis/v1alpha2/gateway_types.go +++ b/apis/v1alpha2/gateway_types.go @@ -26,6 +26,7 @@ import ( // +kubebuilder:object:root=true // +kubebuilder:resource:categories=gateway-api,shortName=gtw // +kubebuilder:subresource:status +// +kubebuilder:unservedversion // +kubebuilder:deprecatedversion:warning="The v1alpha2 version of Gateway has been deprecated and will be removed in a future release of the API. Please upgrade to v1beta1." // +kubebuilder:printcolumn:name="Class",type=string,JSONPath=`.spec.gatewayClassName` // +kubebuilder:printcolumn:name="Address",type=string,JSONPath=`.status.addresses[*].value` diff --git a/apis/v1alpha2/gatewayclass_types.go b/apis/v1alpha2/gatewayclass_types.go index 69d012d123..87d08ebffc 100644 --- a/apis/v1alpha2/gatewayclass_types.go +++ b/apis/v1alpha2/gatewayclass_types.go @@ -27,6 +27,7 @@ import ( // +kubebuilder:object:root=true // +kubebuilder:resource:categories=gateway-api,scope=Cluster,shortName=gc // +kubebuilder:subresource:status +// +kubebuilder:unservedversion // +kubebuilder:deprecatedversion:warning="The v1alpha2 version of GatewayClass has been deprecated and will be removed in a future release of the API. Please upgrade to v1beta1." // +kubebuilder:printcolumn:name="Controller",type=string,JSONPath=`.spec.controllerName` // +kubebuilder:printcolumn:name="Accepted",type=string,JSONPath=`.status.conditions[?(@.type=="Accepted")].status` @@ -44,7 +45,7 @@ import ( // If implementations choose to propagate GatewayClass changes to existing // Gateways, that MUST be clearly documented by the implementation. // -// Whenever one or more Gateways are using a GatewayClass, implementations MUST +// Whenever one or more Gateways are using a GatewayClass, implementations SHOULD // add the `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer on the // associated GatewayClass. This ensures that a GatewayClass associated with a // Gateway is not deleted while in use. diff --git a/apis/v1alpha2/grpcroute_types.go b/apis/v1alpha2/grpcroute_types.go index dc5f558d99..1e623f20c6 100644 --- a/apis/v1alpha2/grpcroute_types.go +++ b/apis/v1alpha2/grpcroute_types.go @@ -18,6 +18,8 @@ package v1alpha2 import ( metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + + "sigs.k8s.io/gateway-api/apis/v1beta1" ) // +genclient @@ -149,7 +151,7 @@ type GRPCRouteSpec struct { Rules []GRPCRouteRule `json:"rules,omitempty"` } -// GRPCRouteRule defines the semantics for matching an gRPC request based on +// GRPCRouteRule defines the semantics for matching a gRPC request based on // conditions (matches), processing it (filters), and forwarding the request to // an API object (backendRefs). type GRPCRouteRule struct { @@ -205,7 +207,6 @@ type GRPCRouteRule struct { // // +optional // +kubebuilder:validation:MaxItems=8 - // +kubebuilder:default={{method: {type: "Exact"}}} Matches []GRPCRouteMatch `json:"matches,omitempty"` // Filters define the filters that are applied to requests that match @@ -222,8 +223,15 @@ type GRPCRouteRule struct { // - Implementation-specific custom filters have no API guarantees across // implementations. // - // Specifying a core filter multiple times has unspecified or - // implementation-specific conformance. + // Specifying the same filter multiple times is not supported unless explicitly + // indicated in the filter. + // + // If an implementation can not support a combination of filters, it must clearly + // document that limitation. In cases where incompatible or unsupported + // filters are specified and cause the `Accepted` condition to be set to status + // `False`, implementations may use the `IncompatibleFilters` reason to specify + // this configuration error. + // // Support: Core // // +optional @@ -286,7 +294,6 @@ type GRPCRouteMatch struct { // not specified, all services and methods will match. // // +optional - // +kubebuilder:default={type: "Exact"} Method *GRPCMethodMatch `json:"method,omitempty"` // Headers specifies gRPC request header matchers. Multiple match values are @@ -321,12 +328,8 @@ type GRPCMethodMatch struct { // // At least one of Service and Method MUST be a non-empty string. // - // A GRPC Service must be a valid Protobuf Type Name - // (https://protobuf.com/docs/language-spec#type-references). - // // +optional // +kubebuilder:validation:MaxLength=1024 - // +kubebuilder:validation:Pattern=`^(?i)\.?[a-z_][a-z_0-9]*(\.[a-z_][a-z_0-9]*)*$` Service *string `json:"service,omitempty"` // Value of the method to match against. If left empty or omitted, will @@ -334,12 +337,8 @@ type GRPCMethodMatch struct { // // At least one of Service and Method MUST be a non-empty string. // - // A GRPC Method must be a valid Protobuf Method - // (https://protobuf.com/docs/language-spec#methods). - // // +optional // +kubebuilder:validation:MaxLength=1024 - // +kubebuilder:validation:Pattern=`^[A-Za-z_][A-Za-z_0-9]*$` Method *string `json:"method,omitempty"` } @@ -419,10 +418,7 @@ const ( GRPCHeaderMatchRegularExpression GRPCHeaderMatchType = "RegularExpression" ) -// +kubebuilder:validation:MinLength=1 -// +kubebuilder:validation:MaxLength=256 -// +kubebuilder:validation:Pattern=`^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$` -type GRPCHeaderName string +type GRPCHeaderName v1beta1.HeaderName // GRPCRouteFilterType identifies a type of GRPCRoute filter. type GRPCRouteFilterType string @@ -513,13 +509,16 @@ type GRPCRouteFilter struct { // Support: Extended // // +optional - // ResponseHeaderModifier *HTTPHeaderFilter `json:"responseHeaderModifier,omitempty"` // RequestMirror defines a schema for a filter that mirrors requests. // Requests are sent to the specified destination, but responses from // that destination are ignored. // + // This filter can be used multiple times within the same rule. Note that + // not all implementations will be able to support mirroring to multiple + // backends. + // // Support: Extended // // +optional @@ -532,6 +531,7 @@ type GRPCRouteFilter struct { // // Support: Implementation-specific // + // This filter can be used multiple times within the same rule. // +optional ExtensionRef *LocalObjectReference `json:"extensionRef,omitempty"` } @@ -562,6 +562,8 @@ type GRPCBackendRef struct { // // Support: Core for Kubernetes Service // + // Support: Extended for Kubernetes ServiceImport + // // Support: Implementation-specific for any other resource // // Support for weight: Core diff --git a/apis/v1alpha2/httproute_types.go b/apis/v1alpha2/httproute_types.go index ac1e009cbb..20e1e32586 100644 --- a/apis/v1alpha2/httproute_types.go +++ b/apis/v1alpha2/httproute_types.go @@ -26,6 +26,7 @@ import ( // +kubebuilder:object:root=true // +kubebuilder:resource:categories=gateway-api // +kubebuilder:subresource:status +// +kubebuilder:unservedversion // +kubebuilder:deprecatedversion:warning="The v1alpha2 version of HTTPRoute has been deprecated and will be removed in a future release of the API. Please upgrade to v1beta1." // +kubebuilder:printcolumn:name="Hostnames",type=string,JSONPath=`.spec.hostnames` // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` @@ -111,12 +112,8 @@ type HeaderMatchType = v1beta1.HeaderMatchType // headers are not currently supported by this type. // // * "/invalid" - "/" is an invalid character -// -// +kubebuilder:validation:MinLength=1 -// +kubebuilder:validation:MaxLength=256 -// +kubebuilder:validation:Pattern=`^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$` // +k8s:deepcopy-gen=false -type HTTPHeaderName = v1beta1.HTTPHeaderName +type HTTPHeaderName = v1beta1.HeaderName // HTTPHeaderMatch describes how to select a HTTP route by matching HTTP request // headers. diff --git a/apis/v1alpha2/object_reference_types.go b/apis/v1alpha2/object_reference_types.go index 80fdd522ce..f2798e597b 100644 --- a/apis/v1alpha2/object_reference_types.go +++ b/apis/v1alpha2/object_reference_types.go @@ -45,10 +45,10 @@ type SecretObjectReference = v1beta1.SecretObjectReference // specific to BackendRef. It includes a few additional fields and features // than a regular ObjectReference. // -// Note that when a namespace is specified, a ReferenceGrant object -// is required in the referent namespace to allow that namespace's -// owner to accept the reference. See the ReferenceGrant documentation -// for details. +// Note that when a namespace different than the local namespace is specified, a +// ReferenceGrant object is required in the referent namespace to allow that +// namespace's owner to accept the reference. See the ReferenceGrant +// documentation for details. // // The API object must be valid in the cluster; the Group and Kind must // be registered in the cluster for this reference to be valid. diff --git a/apis/v1alpha2/policy_types.go b/apis/v1alpha2/policy_types.go index cbf96ed31a..cf151ea23d 100644 --- a/apis/v1alpha2/policy_types.go +++ b/apis/v1alpha2/policy_types.go @@ -39,3 +39,44 @@ type PolicyTargetReference struct { // +optional Namespace *Namespace `json:"namespace,omitempty"` } + +// PolicyConditionType is a type of condition for a policy. This type should be +// used with a Policy resource Status.Conditions field. +type PolicyConditionType string + +// PolicyConditionReason is a reason for a policy condition. +type PolicyConditionReason string + +const ( + // PolicyConditionAccepted indicates whether the policy has been accepted or + // rejected by a targeted resource, and why. + // + // Possible reasons for this condition to be True are: + // + // * "Accepted" + // + // Possible reasons for this condition to be False are: + // + // * "Conflicted" + // * "Invalid" + // * "TargetNotFound" + // + PolicyConditionAccepted PolicyConditionType = "Accepted" + + // PolicyReasonAccepted is used with the "Accepted" condition when the policy + // has been accepted by the targeted resource. + PolicyReasonAccepted PolicyConditionReason = "Accepted" + + // PolicyReasonConflicted is used with the "Accepted" condition when the + // policy has not been accepted by a targeted resource because there is + // another policy that targets the same resource and a merge is not possible. + PolicyReasonConflicted PolicyConditionReason = "Conflicted" + + // PolicyReasonInvalid is used with the "Accepted" condition when the policy + // is syntactically or semantically invalid. + PolicyReasonInvalid PolicyConditionReason = "Invalid" + + // PolicyReasonTargetNotFound is used with the "Accepted" condition when the + // policy is attached to an invalid target resource. + PolicyReasonTargetNotFound PolicyConditionReason = "TargetNotFound" +) diff --git a/apis/v1alpha2/referencegrant_types.go b/apis/v1alpha2/referencegrant_types.go index 8d2955100c..1cecc75fab 100644 --- a/apis/v1alpha2/referencegrant_types.go +++ b/apis/v1alpha2/referencegrant_types.go @@ -25,8 +25,8 @@ import ( // +genclient // +kubebuilder:object:root=true // +kubebuilder:resource:categories=gateway-api,shortName=refgrant -// +kubebuilder:storageversion // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` +// +kubebuilder:deprecatedversion:warning="The v1alpha2 version of ReferenceGrant has been deprecated and will be removed in a future release of the API. Please upgrade to v1beta1." // ReferenceGrant identifies kinds of resources in other namespaces that are // trusted to reference the specified kinds of resources in the same namespace @@ -36,8 +36,12 @@ import ( // Additional Reference Grants can be used to add to the set of trusted // sources of inbound references for the namespace they are defined within. // -// All cross-namespace references in Gateway API (with the exception of cross-namespace -// Gateway-route attachment) require a ReferenceGrant. +// A ReferenceGrant is required for all cross-namespace references in Gateway API +// (with the exception of cross-namespace Route-Gateway attachment, which is +// governed by the AllowedRoutes configuration on the Gateway, and cross-namespace +// Service ParentRefs on a "consumer" mesh Route, which defines routing rules +// applicable only to workloads in the Route namespace). ReferenceGrants allowing +// a reference from a Route to a Service are only applicable to BackendRefs. // // ReferenceGrant is a form of runtime verification allowing users to assert // which cross-namespace object references are permitted. Implementations that diff --git a/apis/v1alpha2/shared_types.go b/apis/v1alpha2/shared_types.go index e759b5f454..bba8e95160 100644 --- a/apis/v1alpha2/shared_types.go +++ b/apis/v1alpha2/shared_types.go @@ -50,10 +50,10 @@ type PortNumber = v1beta1.PortNumber // BackendRef defines how a Route should forward a request to a Kubernetes // resource. // -// Note that when a namespace is specified, a ReferenceGrant object -// is required in the referent namespace to allow that namespace's -// owner to accept the reference. See the ReferenceGrant documentation -// for details. +// Note that when a namespace different than the local namespace is specified, a +// ReferenceGrant object is required in the referent namespace to allow that +// namespace's owner to accept the reference. See the ReferenceGrant +// documentation for details. // +k8s:deepcopy-gen=false type BackendRef = v1beta1.BackendRef diff --git a/apis/v1alpha2/tcproute_types.go b/apis/v1alpha2/tcproute_types.go index f60677c1a7..fe927ab8d4 100644 --- a/apis/v1alpha2/tcproute_types.go +++ b/apis/v1alpha2/tcproute_types.go @@ -68,6 +68,8 @@ type TCPRouteRule struct { // // Support: Core for Kubernetes Service // + // Support: Extended for Kubernetes ServiceImport + // // Support: Implementation-specific for any other resource // // Support for weight: Extended diff --git a/apis/v1alpha2/tlsroute_types.go b/apis/v1alpha2/tlsroute_types.go index e5a4955354..afe34d82d6 100644 --- a/apis/v1alpha2/tlsroute_types.go +++ b/apis/v1alpha2/tlsroute_types.go @@ -112,6 +112,8 @@ type TLSRouteRule struct { // // Support: Core for Kubernetes Service // + // Support: Extended for Kubernetes ServiceImport + // // Support: Implementation-specific for any other resource // // Support for weight: Extended diff --git a/apis/v1alpha2/udproute_types.go b/apis/v1alpha2/udproute_types.go index eaa4f5c21a..9e3770c293 100644 --- a/apis/v1alpha2/udproute_types.go +++ b/apis/v1alpha2/udproute_types.go @@ -67,6 +67,9 @@ type UDPRouteRule struct { // the packets, then 80% of packets must be dropped instead. // // Support: Core for Kubernetes Service + // + // Support: Extended for Kubernetes ServiceImport + // // Support: Implementation-specific for any other resource // // Support for weight: Extended diff --git a/apis/v1alpha2/validation/common.go b/apis/v1alpha2/validation/common.go index 431d5185ab..00b5a4ca25 100644 --- a/apis/v1alpha2/validation/common.go +++ b/apis/v1alpha2/validation/common.go @@ -23,11 +23,9 @@ import ( gatewayvalidationv1b1 "sigs.k8s.io/gateway-api/apis/v1beta1/validation" ) -var ( - // validateParentRefs validates ParentRefs SectionName must be set and uique - // when ParentRefs includes 2 or more references to the same parent - validateParentRefs = gatewayvalidationv1b1.ValidateParentRefs -) +// validateParentRefs validates ParentRefs SectionName must be set and uique +// when ParentRefs includes 2 or more references to the same parent +var validateParentRefs = gatewayvalidationv1b1.ValidateParentRefs // validateBackendRefServicePort validates whether or not a port was specified // for a backendRef which refers to a corev1.Service, asserting that the port diff --git a/apis/v1alpha2/validation/grpcroute.go b/apis/v1alpha2/validation/grpcroute.go index 74709b8de9..d7e452d41c 100644 --- a/apis/v1alpha2/validation/grpcroute.go +++ b/apis/v1alpha2/validation/grpcroute.go @@ -17,7 +17,9 @@ limitations under the License. package validation import ( + "fmt" "net/http" + "regexp" "strings" "k8s.io/apimachinery/pkg/util/validation/field" @@ -30,7 +32,12 @@ var ( // repeated multiple times in a rule. repeatableGRPCRouteFilters = []gatewayv1a2.GRPCRouteFilterType{ gatewayv1a2.GRPCRouteFilterExtensionRef, + gatewayv1a2.GRPCRouteFilterRequestMirror, } + validServiceName = `^(?i)\.?[a-z_][a-z_0-9]*(\.[a-z_][a-z_0-9]*)*$` + validServiceNameRegex = regexp.MustCompile(validServiceName) + validMethodName = `^[A-Za-z_][A-Za-z_0-9]*$` + validMethodNameRegex = regexp.MustCompile(validMethodName) ) // ValidateGRPCRoute validates GRPCRoute according to the Gateway API specification. @@ -63,13 +70,26 @@ func validateGRPCRouteRules(rules []gatewayv1a2.GRPCRouteRule, path *field.Path) return errs } -// validateRuleMatches validates that at least one of the fields Service or Method of -// GRPCMethodMatch to be specified +// validateRuleMatches validates GRPCMethodMatch func validateRuleMatches(matches []gatewayv1a2.GRPCRouteMatch, path *field.Path) field.ErrorList { var errs field.ErrorList for i, m := range matches { - if m.Method != nil && m.Method.Service == nil && m.Method.Method == nil { - errs = append(errs, field.Required(path.Index(i).Child("method"), "one or both of `service` or `method` must be specified")) + if m.Method != nil { + if m.Method.Service == nil && m.Method.Method == nil { + errs = append(errs, field.Required(path.Index(i).Child("method"), "one or both of `service` or `method` must be specified")) + } + // GRPCRoute method matcher admits two types: Exact and RegularExpression. + // If not specified, the match will be treated as type Exact (also the default value for this field). + if m.Method.Type == nil || *m.Method.Type == gatewayv1a2.GRPCMethodMatchExact { + if m.Method.Service != nil && !validServiceNameRegex.MatchString(*m.Method.Service) { + errs = append(errs, field.Invalid(path.Index(i).Child("method"), *m.Method.Service, + fmt.Sprintf("must only contain valid characters (matching %s)", validServiceName))) + } + if m.Method.Method != nil && !validMethodNameRegex.MatchString(*m.Method.Method) { + errs = append(errs, field.Invalid(path.Index(i).Child("method"), *m.Method.Method, + fmt.Sprintf("must only contain valid characters (matching %s)", validMethodName))) + } + } } if m.Headers != nil { errs = append(errs, validateGRPCHeaderMatches(m.Headers, path.Index(i).Child("headers"))...) @@ -146,7 +166,7 @@ func validateGRPCRouteFilters(filters []gatewayv1a2.GRPCRouteFilter, path *field } errs = append(errs, validateGRPCRouteFilterType(filter, path.Index(i))...) } - // custom filters don't have any validation + // repeatableGRPCRouteFilters filters can be used more than once for _, key := range repeatableGRPCRouteFilters { delete(counts, key) } diff --git a/apis/v1alpha2/validation/grpcroute_test.go b/apis/v1alpha2/validation/grpcroute_test.go index 67e8f5743a..a7c1c8fac4 100644 --- a/apis/v1alpha2/validation/grpcroute_test.go +++ b/apis/v1alpha2/validation/grpcroute_test.go @@ -29,8 +29,9 @@ import ( func TestValidateGRPCRoute(t *testing.T) { t.Parallel() - service := "foo" - method := "login" + service := "foo.Test.Example" + method := "Login" + regex := ".*" tests := []struct { name string @@ -87,6 +88,115 @@ func TestValidateGRPCRoute(t *testing.T) { }, }, }, + { + name: "GRPCRoute use regex in service and method with undefined match type", + rules: []gatewayv1a2.GRPCRouteRule{ + { + Matches: []gatewayv1a2.GRPCRouteMatch{ + { + Method: &gatewayv1a2.GRPCMethodMatch{ + Service: ®ex, + Method: ®ex, + }, + }, + }, + }, + }, + errs: field.ErrorList{ + { + Type: field.ErrorTypeInvalid, + BadValue: regex, + Field: "spec.rules[0].matches[0].method", + Detail: `must only contain valid characters (matching ^(?i)\.?[a-z_][a-z_0-9]*(\.[a-z_][a-z_0-9]*)*$)`, + }, + { + Type: field.ErrorTypeInvalid, + BadValue: regex, + Field: "spec.rules[0].matches[0].method", + Detail: `must only contain valid characters (matching ^[A-Za-z_][A-Za-z_0-9]*$)`, + }, + }, + }, + { + name: "GRPCRoute use regex in service and method with match type Exact", + rules: []gatewayv1a2.GRPCRouteRule{ + { + Matches: []gatewayv1a2.GRPCRouteMatch{ + { + Method: &gatewayv1a2.GRPCMethodMatch{ + Service: ®ex, + Method: ®ex, + Type: ptrTo(gatewayv1a2.GRPCMethodMatchExact), + }, + }, + }, + }, + }, + errs: field.ErrorList{ + { + Type: field.ErrorTypeInvalid, + BadValue: regex, + Field: "spec.rules[0].matches[0].method", + Detail: `must only contain valid characters (matching ^(?i)\.?[a-z_][a-z_0-9]*(\.[a-z_][a-z_0-9]*)*$)`, + }, + { + Type: field.ErrorTypeInvalid, + BadValue: regex, + Field: "spec.rules[0].matches[0].method", + Detail: `must only contain valid characters (matching ^[A-Za-z_][A-Za-z_0-9]*$)`, + }, + }, + }, + { + name: "GRPCRoute use regex in service and method with match type RegularExpression", + rules: []gatewayv1a2.GRPCRouteRule{ + { + Matches: []gatewayv1a2.GRPCRouteMatch{ + { + Method: &gatewayv1a2.GRPCMethodMatch{ + Service: ®ex, + Method: ®ex, + Type: ptrTo(gatewayv1a2.GRPCMethodMatchRegularExpression), + }, + }, + }, + }, + }, + errs: field.ErrorList{}, + }, + { + name: "GRPCRoute use valid service and method with undefined match type", + rules: []gatewayv1a2.GRPCRouteRule{ + { + Matches: []gatewayv1a2.GRPCRouteMatch{ + { + Method: &gatewayv1a2.GRPCMethodMatch{ + Service: &service, + Method: &method, + }, + }, + }, + }, + }, + errs: field.ErrorList{}, + }, + { + name: "GRPCRoute use valid service and method with match type Exact", + rules: []gatewayv1a2.GRPCRouteRule{ + { + Matches: []gatewayv1a2.GRPCRouteMatch{ + { + Method: &gatewayv1a2.GRPCMethodMatch{ + Service: &service, + Method: &method, + Type: ptrTo(gatewayv1a2.GRPCMethodMatchExact), + }, + }, + }, + }, + }, + errs: field.ErrorList{}, + }, { name: "GRPCRoute with duplicate ExtensionRef filters", rules: []gatewayv1a2.GRPCRouteRule{ @@ -126,10 +236,38 @@ func TestValidateGRPCRoute(t *testing.T) { }}, }, }, + }, + { + name: "invalid GRPCRoute with duplicate RequestHeaderModifier filters", + rules: []gatewayv1a2.GRPCRouteRule{ + { + Filters: []gatewayv1a2.GRPCRouteFilter{{ + Type: "RequestHeaderModifier", + RequestHeaderModifier: &gatewayv1a2.HTTPHeaderFilter{ + Set: []gatewayv1a2.HTTPHeader{ + { + Name: "special-header", + Value: "foo", + }, + }, + }, + }, { + Type: "RequestHeaderModifier", + RequestHeaderModifier: &gatewayv1a2.HTTPHeaderFilter{ + Add: []gatewayv1a2.HTTPHeader{ + { + Name: "my-header", + Value: "bar", + }, + }, + }, + }}, + }, + }, errs: field.ErrorList{ { Type: field.ErrorTypeInvalid, - BadValue: "RequestMirror", + BadValue: "RequestHeaderModifier", Field: "spec.rules[0].filters", Detail: "cannot be used multiple times in the same rule", }, @@ -195,8 +333,8 @@ func TestValidateGRPCBackendUniqueFilters(t *testing.T) { }, }}, }, { - name: "invalid grpcRoute Rules duplicate mirror filter", - errCount: 1, + name: "valid grpcRoute Rules duplicate mirror filter", + errCount: 0, rules: []gatewayv1a2.GRPCRouteRule{{ BackendRefs: []gatewayv1a2.GRPCBackendRef{ { diff --git a/apis/v1alpha2/validation/httproute_test.go b/apis/v1alpha2/validation/httproute_test.go index 91242832e9..159ab4f587 100644 --- a/apis/v1alpha2/validation/httproute_test.go +++ b/apis/v1alpha2/validation/httproute_test.go @@ -29,7 +29,6 @@ import ( func TestValidateHTTPRoute(t *testing.T) { testService := gatewayv1a2.ObjectName("test-service") - specialService := gatewayv1a2.ObjectName("special-service") pathPrefixMatchType := gatewayv1b1.PathMatchPathPrefix tests := []struct { @@ -103,20 +102,20 @@ func TestValidateHTTPRoute(t *testing.T) { }, Filters: []gatewayv1a2.HTTPRouteFilter{ { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1a2.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1a2.BackendObjectReference{ - Name: testService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), }, }, }, { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1a2.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1a2.BackendObjectReference{ - Name: specialService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("bar"), }, }, }, @@ -186,11 +185,13 @@ func TestValidateHTTPRoute(t *testing.T) { }, Filters: []gatewayv1a2.HTTPRouteFilter{ { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1a2.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1a2.BackendObjectReference{ - Name: testService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), + Type: gatewayv1b1.HTTPRouteFilterResponseHeaderModifier, + ResponseHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Add: []gatewayv1b1.HTTPHeader{ + { + Name: "extra-header", + Value: "foo", + }, }, }, }, @@ -206,11 +207,13 @@ func TestValidateHTTPRoute(t *testing.T) { }, }, { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1a2.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1a2.BackendObjectReference{ - Name: testService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), + Type: gatewayv1b1.HTTPRouteFilterResponseHeaderModifier, + ResponseHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Set: []gatewayv1b1.HTTPHeader{ + { + Name: "other-header", + Value: "bat", + }, }, }, }, @@ -225,15 +228,6 @@ func TestValidateHTTPRoute(t *testing.T) { }, }, }, - { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1a2.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1a2.BackendObjectReference{ - Name: specialService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), - }, - }, - }, }, }, }, @@ -471,8 +465,8 @@ func TestValidateHTTPBackendUniqueFilters(t *testing.T) { }, }}, }, { - name: "invalid httpRoute Rules duplicate mirror filter", - errCount: 1, + name: "valid httpRoute Rules duplicate mirror filter", + errCount: 0, rules: []gatewayv1a2.HTTPRouteRule{{ BackendRefs: []gatewayv1a2.HTTPBackendRef{ { @@ -868,7 +862,7 @@ func TestValidateHTTPRouteTypeMatchesField(t *testing.T) { StatusCode: new(int), }, }, - errCount: 0, + errCount: 1, }, { name: "invalid HTTPRouteFilterRequestRedirect type filter with non-matching field", routeFilter: gatewayv1a2.HTTPRouteFilter{ diff --git a/apis/v1beta1/doc.go b/apis/v1beta1/doc.go index d29a3c14a2..328100aee8 100644 --- a/apis/v1beta1/doc.go +++ b/apis/v1beta1/doc.go @@ -16,6 +16,7 @@ limitations under the License. // Package v1beta1 contains API Schema definitions for the // gateway.networking.k8s.io API group. +// // +kubebuilder:object:generate=true // +groupName=gateway.networking.k8s.io package v1beta1 diff --git a/apis/v1beta1/gateway_types.go b/apis/v1beta1/gateway_types.go index 8f4d3ff70d..559d6942e5 100644 --- a/apis/v1beta1/gateway_types.go +++ b/apis/v1beta1/gateway_types.go @@ -41,7 +41,7 @@ type Gateway struct { // Status defines the current state of Gateway. // - // +kubebuilder:default={conditions: {{type: "Accepted", status: "Unknown", reason:"NotReconciled", message:"Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"}}} + // +kubebuilder:default={conditions: {{type: "Accepted", status: "Unknown", reason:"Pending", message:"Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"},{type: "Programmed", status: "Unknown", reason:"Pending", message:"Waiting for controller", lastTransitionTime: "1970-01-01T00:00:00Z"}}} Status GatewayStatus `json:"status,omitempty"` } @@ -72,6 +72,19 @@ type GatewaySpec struct { // Each listener in a Gateway must have a unique combination of Hostname, // Port, and Protocol. // + // Within the HTTP Conformance Profile, the below combinations of port and + // protocol are considered Core and MUST be supported: + // + // 1. Port: 80, Protocol: HTTP + // 2. Port: 443, Protocol: HTTPS + // + // Within the TLS Conformance Profile, the below combinations of port and + // protocol are considered Core and MUST be supported: + // + // 1. Port: 443, Protocol: TLS + // + // Port and protocol combinations not listed above are considered Extended. + // // An implementation MAY group Listeners by Port and then collapse each // group of Listeners into a single Listener if the implementation // determines that the Listeners in the group are "compatible". An @@ -111,6 +124,11 @@ type GatewaySpec struct { // +listMapKey=name // +kubebuilder:validation:MinItems=1 // +kubebuilder:validation:MaxItems=64 + // +kubebuilder:validation:XValidation:message="tls must be specified for protocols ['HTTPS', 'TLS']",rule="self.all(l, l.protocol in ['HTTPS', 'TLS'] ? has(l.tls) : true)" + // +kubebuilder:validation:XValidation:message="tls must not be specified for protocols ['HTTP', 'TCP', 'UDP']",rule="self.all(l, l.protocol in ['HTTP', 'TCP', 'UDP'] ? !has(l.tls) : true)" + // +kubebuilder:validation:XValidation:message="hostname must not be specified for protocols ['TCP', 'UDP']",rule="self.all(l, l.protocol in ['TCP', 'UDP'] ? (!has(l.hostname) || l.hostname == '') : true)" + // +kubebuilder:validation:XValidation:message="Listener name must be unique within the Gateway",rule="self.all(l1, self.exists_one(l2, l1.name == l2.name))" + // +kubebuilder:validation:XValidation:message="Combination of port, protocol and hostname must be unique for each listener",rule="self.all(l1, self.exists_one(l2, l1.port == l2.port && l1.protocol == l2.protocol && (has(l1.hostname) && has(l2.hostname) ? l1.hostname == l2.hostname : true)))" Listeners []Listener `json:"listeners"` // Addresses requested for this Gateway. This is optional and behavior can @@ -138,7 +156,10 @@ type GatewaySpec struct { // Support: Extended // // +optional + // // +kubebuilder:validation:MaxItems=16 + // +kubebuilder:validation:XValidation:message="IPAddress values must be unique",rule="self.all(a1, a1.type == 'IPAddress' ? self.exists_one(a2, a2.type == a1.type && a2.value == a1.value) : true )" + // +kubebuilder:validation:XValidation:message="Hostname values must be unique",rule="self.all(a1, a1.type == 'Hostname' ? self.exists_one(a2, a2.type == a1.type && a2.value == a1.value) : true )" Addresses []GatewayAddress `json:"addresses,omitempty"` } @@ -286,6 +307,8 @@ const ( ) // GatewayTLSConfig describes a TLS configuration. +// +// +kubebuilder:validation:XValidation:message="certificateRefs must be specified when TLSModeType is Terminate",rule="self.mode == 'Terminate' ? size(self.certificateRefs) > 0 : true" type GatewayTLSConfig struct { // Mode defines the TLS behavior for the TLS session initiated by the client. // There are two possible modes: @@ -416,6 +439,7 @@ const ( type RouteNamespaces struct { // From indicates where Routes will be selected for this Gateway. Possible // values are: + // // * All: Routes in all namespaces may be used by this Gateway. // * Selector: Routes in namespaces selected by the selector may be used by // this Gateway. @@ -450,6 +474,8 @@ type RouteGroupKind struct { } // GatewayAddress describes an address that can be bound to a Gateway. +// +// +kubebuilder:validation:XValidation:message="Hostname value must only contain valid characters (matching ^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$)",rule="self.type == 'Hostname' ? self.value.matches('^(\\\\*\\\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$'): true" type GatewayAddress struct { // Type of the address. // @@ -467,6 +493,26 @@ type GatewayAddress struct { Value string `json:"value"` } +// GatewayStatusAddress describes an address that is bound to a Gateway. +// +// +kubebuilder:validation:XValidation:message="Hostname value must only contain valid characters (matching ^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$)",rule="self.type == 'Hostname' ? self.value.matches('^(\\\\*\\\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$'): true" +type GatewayStatusAddress struct { + // Type of the address. + // + // +optional + // +kubebuilder:default=IPAddress + Type *AddressType `json:"type,omitempty"` + + // Value of the address. The validity of the values will depend + // on the type and support by the controller. + // + // Examples: `1.2.3.4`, `128::1`, `my-ip-address`. + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Value string `json:"value"` +} + // GatewayStatus defines the observed state of Gateway. type GatewayStatus struct { // Addresses lists the IP addresses that have actually been @@ -475,8 +521,9 @@ type GatewayStatus struct { // assigns an address from a reserved pool. // // +optional + // // +kubebuilder:validation:MaxItems=16 - Addresses []GatewayAddress `json:"addresses,omitempty"` + Addresses []GatewayStatusAddress `json:"addresses,omitempty"` // Conditions describe the current conditions of the Gateway. // @@ -488,6 +535,7 @@ type GatewayStatus struct { // Known condition types are: // // * "Accepted" + // * "Programmed" // * "Ready" // // +optional @@ -535,6 +583,8 @@ const ( // // * "Invalid" // * "Pending" + // * "NoResources" + // * "AddressNotAssigned" // // Possible reasons for this condition to be Unknown are: // @@ -549,9 +599,20 @@ const ( // true. GatewayReasonProgrammed GatewayConditionReason = "Programmed" - // This reason is used with the "Programmed" condition when the Listener is + // This reason is used with the "Programmed" and "Accepted" conditions when the Gateway is // syntactically or semantically invalid. GatewayReasonInvalid GatewayConditionReason = "Invalid" + + // This reason is used with the "Programmed" condition when the + // Gateway is not scheduled because insufficient infrastructure + // resources are available. + GatewayReasonNoResources GatewayConditionReason = "NoResources" + + // This reason is used with the "Programmed" condition when none of the requested + // addresses have been assigned to the Gateway. This reason can be used to + // express a range of circumstances, including (but not limited to) IPAM + // address exhaustion, address not yet allocated, or a named address not being found. + GatewayReasonAddressNotAssigned GatewayConditionReason = "AddressNotAssigned" ) const ( @@ -563,11 +624,14 @@ const ( // Possible reasons for this condition to be True are: // // * "Accepted" + // * "ListenersNotValid" // // Possible reasons for this condition to be False are: // + // * "Invalid" // * "NotReconciled" - // * "NoResources" + // * "UnsupportedAddress" + // * "ListenersNotValid" // // Possible reasons for this condition to be Unknown are: // @@ -578,76 +642,65 @@ const ( // interoperability. GatewayConditionAccepted GatewayConditionType = "Accepted" - // Deprecated: use "Accepted" instead. - GatewayConditionScheduled GatewayConditionType = "Scheduled" - // This reason is used with the "Accepted" condition when the condition is // True. GatewayReasonAccepted GatewayConditionReason = "Accepted" + // This reason is used with the "Accepted" condition when one or + // more Listeners have an invalid or unsupported configuration + // and cannot be configured on the Gateway. + // This can be the reason when "Accepted" is "True" or "False", depending on whether + // the listener being invalid causes the entire Gateway to not be accepted. + GatewayReasonListenersNotValid GatewayConditionReason = "ListenersNotValid" + + // This reason is used with the "Accepted" and "Programmed" + // conditions when the status is "Unknown" and no controller has reconciled + // the Gateway. + GatewayReasonPending GatewayConditionReason = "Pending" + + // This reason is used with the "Accepted" condition when the Gateway could not be configured + // because the requested address is not supported. This reason could be used in a number of + // instances, including: + // + // * The address is already in use. + // * The type of address is not supported by the implementation. + GatewayReasonUnsupportedAddress GatewayConditionReason = "UnsupportedAddress" +) + +const ( + // Deprecated: use "Accepted" instead. + GatewayConditionScheduled GatewayConditionType = "Scheduled" + // This reason is used with the "Scheduled" condition when the condition is // True. // // Deprecated: use the "Accepted" condition with reason "Accepted" instead. GatewayReasonScheduled GatewayConditionReason = "Scheduled" - // This reason is used with the "Accepted", "Programmed" and "Ready" - // conditions when the status is "Unknown" and no controller has reconciled - // the Gateway. - GatewayReasonPending GatewayConditionReason = "Pending" - // Deprecated: Use "Pending" instead. GatewayReasonNotReconciled GatewayConditionReason = "NotReconciled" - - // This reason is used with the "Accepted" condition when the - // Gateway is not scheduled because insufficient infrastructure - // resources are available. - GatewayReasonNoResources GatewayConditionReason = "NoResources" ) const ( - // Ready is an optional Condition that has Extended support. When it's set, - // the condition indicates whether the Gateway has been completely configured - // and traffic is ready to flow through the data plane immediately. - // - // If both the "ListenersNotValid" and "ListenersNotReady" - // reasons are true, the Gateway controller should prefer the - // "ListenersNotValid" reason. + // "Ready" is a condition type reserved for future use. It should not be used by implementations. // - // Possible reasons for this condition to be true are: + // If used in the future, "Ready" will represent the final state where all configuration is confirmed good + // _and has completely propagated to the data plane_. That is, it is a _guarantee_ that, as soon as something + // sees the Condition as `true`, then connections will be correctly routed _immediately_. // - // * "Ready" + // This is a very strong guarantee, and to date no implementation has satisfied it enough to implement it. + // This reservation can be discussed in the future if necessary. // - // Possible reasons for this condition to be False are: - // - // * "ListenersNotValid" - // * "ListenersNotReady" - // * "AddressNotAssigned" - // - // Controllers may raise this condition with other reasons, - // but should prefer to use the reasons listed above to improve - // interoperability. + // Note: This condition is not really "deprecated", but rather "reserved"; however, deprecated triggers Go linters + // to alert about usage. + // Deprecated: Ready is reserved for future use GatewayConditionReady GatewayConditionType = "Ready" - // This reason is used with the "Ready" condition when the condition is - // true. + // Deprecated: Ready is reserved for future use GatewayReasonReady GatewayConditionReason = "Ready" - // This reason is used with the "Ready" condition when one or - // more Listeners have an invalid or unsupported configuration - // and cannot be configured on the Gateway. - GatewayReasonListenersNotValid GatewayConditionReason = "ListenersNotValid" - - // This reason is used with the "Ready" condition when one or - // more Listeners are not ready to serve traffic. + // Deprecated: Ready is reserved for future use GatewayReasonListenersNotReady GatewayConditionReason = "ListenersNotReady" - - // This reason is used with the "Ready" condition when none of the requested - // addresses have been assigned to the Gateway. This reason can be used to - // express a range of circumstances, including (but not limited to) IPAM - // address exhaustion, invalid or unsupported address requests, or a named - // address not being found. - GatewayReasonAddressNotAssigned GatewayConditionReason = "AddressNotAssigned" ) // ListenerStatus is the status associated with a Listener. @@ -668,7 +721,7 @@ type ListenerStatus struct { // +kubebuilder:validation:MaxItems=8 SupportedKinds []RouteGroupKind `json:"supportedKinds"` - // AttachedRoutes represents the total number of Routes that have been + // AttachedRoutes represents the total number of accepted Routes that have been // successfully attached to this Listener. AttachedRoutes int32 `json:"attachedRoutes"` @@ -726,15 +779,17 @@ const ( ) const ( - // This condition indicates that, even though the listener is - // syntactically and semantically valid, the controller is not able - // to configure it on the underlying Gateway infrastructure. + // This condition indicates that the listener is syntactically and + // semantically valid, and that all features used in the listener's spec are + // supported. + // + // In general, a Listener will be marked as Accepted when the supplied + // configuration will generate at least some data plane configuration. // - // A Listener is specified as a logical requirement, but needs to be - // configured on a network endpoint (i.e. address and port) by a - // controller. The controller may be unable to attach the Listener - // if it specifies an unsupported requirement, or prerequisite - // resources are not available. + // For example, a Listener with an unsupported protocol will never generate + // any data plane config, and so will have Accepted set to `false.` + // Conversely, a Listener that does not have any Routes will be able to + // generate data plane config, and so will have Accepted set to `true`. // // Possible reasons for this condition to be True are: // @@ -744,7 +799,6 @@ const ( // // * "PortUnavailable" // * "UnsupportedProtocol" - // * "UnsupportedAddress" // // Possible reasons for this condition to be Unknown are: // @@ -780,14 +834,6 @@ const ( // Listener could not be attached to be Gateway because its // protocol type is not supported. ListenerReasonUnsupportedProtocol ListenerConditionReason = "UnsupportedProtocol" - - // This reason is used with the "Accepted" condition when the Listener could - // not be attached to the Gateway because the requested address is not - // supported. This reason could be used in a number of instances, including: - // - // * The address is already in use. - // * The type of address is not supported by the implementation. - ListenerReasonUnsupportedAddress ListenerConditionReason = "UnsupportedAddress" ) const ( @@ -869,34 +915,6 @@ const ( // This reason is used with the "Programmed" condition when the condition is // true. ListenerReasonProgrammed ListenerConditionReason = "Programmed" -) - -const ( - // Ready is an optional Condition that has Extended support. When it's set, - // the condition indicates whether the Listener has been configured on the - // Gateway and traffic is ready to flow through the data plane immediately. - // - // Possible reasons for this condition to be True are: - // - // * "Ready" - // - // Possible reasons for this condition to be False are: - // - // * "Invalid" - // * "Pending" - // - // Possible reasons for this condition to be Unknown are: - // - // * "Pending" - // - // Controllers may raise this condition with other reasons, - // but should prefer to use the reasons listed above to improve - // interoperability. - ListenerConditionReady ListenerConditionType = "Ready" - - // This reason is used with the "Ready" condition when the condition is - // true. - ListenerReasonReady ListenerConditionReason = "Ready" // This reason is used with the "Ready" and "Programmed" conditions when the // Listener is syntactically or semantically invalid. @@ -907,3 +925,22 @@ const ( // online and ready to accept client traffic. ListenerReasonPending ListenerConditionReason = "Pending" ) + +const ( + // "Ready" is a condition type reserved for future use. It should not be used by implementations. + // Note: This condition is not really "deprecated", but rather "reserved"; however, deprecated triggers Go linters + // to alert about usage. + // + // If used in the future, "Ready" will represent the final state where all configuration is confirmed good + // _and has completely propagated to the data plane_. That is, it is a _guarantee_ that, as soon as something + // sees the Condition as `true`, then connections will be correctly routed _immediately_. + // + // This is a very strong guarantee, and to date no implementation has satisfied it enough to implement it. + // This reservation can be discussed in the future if necessary. + // + // Deprecated: Ready is reserved for future use + ListenerConditionReady ListenerConditionType = "Ready" + + // Deprecated: Ready is reserved for future use + ListenerReasonReady ListenerConditionReason = "Ready" +) diff --git a/apis/v1beta1/gatewayclass_types.go b/apis/v1beta1/gatewayclass_types.go index 9507e5a529..9d166700d0 100644 --- a/apis/v1beta1/gatewayclass_types.go +++ b/apis/v1beta1/gatewayclass_types.go @@ -42,7 +42,7 @@ import ( // If implementations choose to propagate GatewayClass changes to existing // Gateways, that MUST be clearly documented by the implementation. // -// Whenever one or more Gateways are using a GatewayClass, implementations MUST +// Whenever one or more Gateways are using a GatewayClass, implementations SHOULD // add the `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer on the // associated GatewayClass. This ensures that a GatewayClass associated with a // Gateway is not deleted while in use. @@ -57,6 +57,9 @@ type GatewayClass struct { // Status defines the current state of GatewayClass. // + // Implementations MUST populate status on all GatewayClass resources which + // specify their controller name. + // // +kubebuilder:default={conditions: {{type: "Accepted", status: "Unknown", message: "Waiting for controller", reason: "Waiting", lastTransitionTime: "1970-01-01T00:00:00Z"}}} Status GatewayClassStatus `json:"status,omitempty"` } @@ -78,6 +81,8 @@ type GatewayClassSpec struct { // This field is not mutable and cannot be empty. // // Support: Core + // + // +kubebuilder:validation:XValidation:message="Value is immutable",rule="self == oldSelf" ControllerName GatewayController `json:"controllerName"` // ParametersRef is a reference to a resource that contains the configuration diff --git a/apis/v1beta1/httproute_types.go b/apis/v1beta1/httproute_types.go index 1dc7e2ede0..7cfda8f7ec 100644 --- a/apis/v1beta1/httproute_types.go +++ b/apis/v1beta1/httproute_types.go @@ -56,9 +56,14 @@ type HTTPRouteList struct { type HTTPRouteSpec struct { CommonRouteSpec `json:",inline"` - // Hostnames defines a set of hostname that should match against the HTTP - // Host header to select a HTTPRoute to process the request. This matches - // the RFC 1123 definition of a hostname with 2 notable exceptions: + // Hostnames defines a set of hostnames that should match against the HTTP Host + // header to select a HTTPRoute used to process the request. Implementations + // MUST ignore any port value specified in the HTTP Host header while + // performing a match and (absent of any applicable header modification + // configuration) MUST forward this header unmodified to the backend. + // + // Valid values for Hostnames are determined by RFC 1123 definition of a + // hostname with 2 notable exceptions: // // 1. IPs are not allowed. // 2. A hostname may be prefixed with a wildcard label (`*.`). The wildcard @@ -120,6 +125,12 @@ type HTTPRouteSpec struct { // HTTPRouteRule defines semantics for matching an HTTP request based on // conditions (matches), processing it (filters), and forwarding the request to // an API object (backendRefs). +// +// +kubebuilder:validation:XValidation:message="RequestRedirect filter must not be used together with backendRefs",rule="(has(self.backendRefs) && size(self.backendRefs) > 0) ? (!has(self.filters) || self.filters.all(f, !has(f.requestRedirect))): true" +// +kubebuilder:validation:XValidation:message="When using RequestRedirect filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified",rule="(has(self.filters) && self.filters.exists_one(f, has(f.requestRedirect) && has(f.requestRedirect.path) && f.requestRedirect.path.type == 'ReplacePrefixMatch' && has(f.requestRedirect.path.replacePrefixMatch))) ? ((size(self.matches) != 1 || !has(self.matches[0].path) || self.matches[0].path.type != 'PathPrefix') ? false : true) : true" +// +kubebuilder:validation:XValidation:message="When using URLRewrite filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified",rule="(has(self.filters) && self.filters.exists_one(f, has(f.urlRewrite) && has(f.urlRewrite.path) && f.urlRewrite.path.type == 'ReplacePrefixMatch' && has(f.urlRewrite.path.replacePrefixMatch))) ? ((size(self.matches) != 1 || !has(self.matches[0].path) || self.matches[0].path.type != 'PathPrefix') ? false : true) : true" +// +kubebuilder:validation:XValidation:message="Within backendRefs, when using RequestRedirect filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified",rule="(has(self.backendRefs) && self.backendRefs.exists_one(b, (has(b.filters) && b.filters.exists_one(f, has(f.requestRedirect) && has(f.requestRedirect.path) && f.requestRedirect.path.type == 'ReplacePrefixMatch' && has(f.requestRedirect.path.replacePrefixMatch))) )) ? ((size(self.matches) != 1 || !has(self.matches[0].path) || self.matches[0].path.type != 'PathPrefix') ? false : true) : true" +// +kubebuilder:validation:XValidation:message="Within backendRefs, When using URLRewrite filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified",rule="(has(self.backendRefs) && self.backendRefs.exists_one(b, (has(b.filters) && b.filters.exists_one(f, has(f.urlRewrite) && has(f.urlRewrite.path) && f.urlRewrite.path.type == 'ReplacePrefixMatch' && has(f.urlRewrite.path.replacePrefixMatch))) )) ? ((size(self.matches) != 1 || !has(self.matches[0].path) || self.matches[0].path.type != 'PathPrefix') ? false : true) : true" type HTTPRouteRule struct { // Matches define conditions used for matching the rule against incoming // HTTP requests. Each match is independent, i.e. this rule will be matched @@ -154,11 +165,15 @@ type HTTPRouteRule struct { // Proxy or Load Balancer routing configuration generated from HTTPRoutes // MUST prioritize matches based on the following criteria, continuing on // ties. Across all rules specified on applicable Routes, precedence must be - // given to the match with the largest number of: + // given to the match having: + // + // * "Exact" path match. + // * "Prefix" path match with largest number of characters. + // * Method match. + // * Largest number of header matches. + // * Largest number of query param matches. // - // * Characters in a matching path. - // * Header matches. - // * Query param matches. + // Note: The precedence of RegularExpression path matches are implementation-specific. // // If ties still exist across multiple Routes, matching precedence MUST be // determined in order of the following criteria, continuing on ties: @@ -192,19 +207,26 @@ type HTTPRouteRule struct { // - Implementation-specific custom filters have no API guarantees across // implementations. // - // Specifying a core filter multiple times has unspecified or - // implementation-specific conformance. + // Specifying the same filter multiple times is not supported unless explicitly + // indicated in the filter. // // All filters are expected to be compatible with each other except for the // URLRewrite and RequestRedirect filters, which may not be combined. If an // implementation can not support other combinations of filters, they must clearly - // document that limitation. In all cases where incompatible or unsupported - // filters are specified, implementations MUST add a warning condition to status. + // document that limitation. In cases where incompatible or unsupported + // filters are specified and cause the `Accepted` condition to be set to status + // `False`, implementations may use the `IncompatibleFilters` reason to specify + // this configuration error. // // Support: Core // // +optional // +kubebuilder:validation:MaxItems=16 + // +kubebuilder:validation:XValidation:message="May specify either httpRouteFilterRequestRedirect or httpRouteFilterRequestRewrite, but not both",rule="!(self.exists(f, f.type == 'RequestRedirect') && self.exists(f, f.type == 'URLRewrite'))" + // +kubebuilder:validation:XValidation:message="RequestHeaderModifier filter cannot be repeated",rule="self.exists(f, f.type == 'RequestHeaderModifier') ? self.exists_one(f, f.type == 'RequestHeaderModifier') : true" + // +kubebuilder:validation:XValidation:message="ResponseHeaderModifier filter cannot be repeated",rule="self.exists(f, f.type == 'ResponseHeaderModifier') ? self.exists_one(f, f.type == 'ResponseHeaderModifier') : true" + // +kubebuilder:validation:XValidation:message="RequestRedirect filter cannot be repeated",rule="self.exists(f, f.type == 'RequestRedirect') ? self.exists_one(f, f.type == 'RequestRedirect') : true" + // +kubebuilder:validation:XValidation:message="URLRewrite filter cannot be repeated",rule="self.exists(f, f.type == 'URLRewrite') ? self.exists_one(f, f.type == 'URLRewrite') : true" Filters []HTTPRouteFilter `json:"filters,omitempty"` // BackendRefs defines the backend(s) where matching requests should be @@ -232,6 +254,8 @@ type HTTPRouteRule struct { // // Support: Core for Kubernetes Service // + // Support: Extended for Kubernetes ServiceImport + // // Support: Implementation-specific for any other resource // // Support for weight: Core @@ -264,7 +288,9 @@ type HTTPRouteRule struct { type PathMatchType string const ( - // Matches the URL path exactly and with case sensitivity. + // Matches the URL path exactly and with case sensitivity. This means that + // an exact path match on `/abc` will only match requests to `/abc`, NOT + // `/abc/`, `/Abc`, or `/abcd`. PathMatchExact PathMatchType = "Exact" // Matches based on a URL path prefix split by `/`. Matching is @@ -291,6 +317,18 @@ const ( ) // HTTPPathMatch describes how to select a HTTP route by matching the HTTP request path. +// +// +kubebuilder:validation:XValidation:message="value must be an absolute path and start with '/' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? self.value.startsWith('/') : true" +// +kubebuilder:validation:XValidation:message="must not contain '//' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.contains('//') : true" +// +kubebuilder:validation:XValidation:message="must not contain '/./' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.contains('/./') : true" +// +kubebuilder:validation:XValidation:message="must not contain '/../' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.contains('/../') : true" +// +kubebuilder:validation:XValidation:message="must not contain '%2f' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.contains('%2f') : true" +// +kubebuilder:validation:XValidation:message="must not contain '%2F' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.contains('%2F') : true" +// +kubebuilder:validation:XValidation:message="must not contain '#' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.contains('#') : true" +// +kubebuilder:validation:XValidation:message="must not end with '/..' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.endsWith('/..') : true" +// +kubebuilder:validation:XValidation:message="must not end with '/.' when type one of ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? !self.value.endsWith('/.') : true" +// +kubebuilder:validation:XValidation:message="type must be one of ['Exact', 'PathPrefix', 'RegularExpression']",rule="self.type == 'Exact' || self.type == 'PathPrefix' || self.type == 'RegularExpression'" +// +kubebuilder:validation:XValidation:message="must only contain valid characters (matching ^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$) for types ['Exact', 'PathPrefix']",rule="(self.type == 'Exact' || self.type == 'PathPrefix') ? self.value.matches(r\"\"\"^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$\"\"\") : true" type HTTPPathMatch struct { // Type specifies how to match against the path Value. // @@ -343,12 +381,8 @@ const ( // // - ":method" - ":" is an invalid character. This means that HTTP/2 pseudo // headers are not currently supported by this type. -// - "/invalid" - "/" is an invalid character -// -// +kubebuilder:validation:MinLength=1 -// +kubebuilder:validation:MaxLength=256 -// +kubebuilder:validation:Pattern=`^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$` -type HTTPHeaderName string +// - "/invalid" - "/ " is an invalid character +type HTTPHeaderName HeaderName // HTTPHeaderMatch describes how to select a HTTP route by matching HTTP request // headers. @@ -449,10 +483,7 @@ type HTTPQueryParamMatch struct { // // Users SHOULD NOT route traffic based on repeated query params to guard // themselves against potential differences in the implementations. - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=256 - Name string `json:"name"` + Name HTTPHeaderName `json:"name"` // Value is the value of HTTP query param to be matched. // @@ -552,6 +583,19 @@ type HTTPRouteMatch struct { // examples include request or response modification, implementing // authentication strategies, rate-limiting, and traffic shaping. API // guarantee/conformance is defined based on the type of the filter. +// +// +kubebuilder:validation:XValidation:message="filter.requestHeaderModifier must be nil if the filter.type is not RequestHeaderModifier",rule="!(has(self.requestHeaderModifier) && self.type != 'RequestHeaderModifier')" +// +kubebuilder:validation:XValidation:message="filter.requestHeaderModifier must be specified for RequestHeaderModifier filter.type",rule="!(!has(self.requestHeaderModifier) && self.type == 'RequestHeaderModifier')" +// +kubebuilder:validation:XValidation:message="filter.responseHeaderModifier must be nil if the filter.type is not ResponseHeaderModifier",rule="!(has(self.responseHeaderModifier) && self.type != 'ResponseHeaderModifier')" +// +kubebuilder:validation:XValidation:message="filter.responseHeaderModifier must be specified for ResponseHeaderModifier filter.type",rule="!(!has(self.responseHeaderModifier) && self.type == 'ResponseHeaderModifier')" +// +kubebuilder:validation:XValidation:message="filter.requestMirror must be nil if the filter.type is not RequestMirror",rule="!(has(self.requestMirror) && self.type != 'RequestMirror')" +// +kubebuilder:validation:XValidation:message="filter.requestMirror must be specified for RequestMirror filter.type",rule="!(!has(self.requestMirror) && self.type == 'RequestMirror')" +// +kubebuilder:validation:XValidation:message="filter.requestRedirect must be nil if the filter.type is not RequestRedirect",rule="!(has(self.requestRedirect) && self.type != 'RequestRedirect')" +// +kubebuilder:validation:XValidation:message="filter.requestRedirect must be specified for RequestRedirect filter.type",rule="!(!has(self.requestRedirect) && self.type == 'RequestRedirect')" +// +kubebuilder:validation:XValidation:message="filter.urlRewrite must be nil if the filter.type is not URLRewrite",rule="!(has(self.urlRewrite) && self.type != 'URLRewrite')" +// +kubebuilder:validation:XValidation:message="filter.urlRewrite must be specified for URLRewrite filter.type",rule="!(!has(self.urlRewrite) && self.type == 'URLRewrite')" +// +kubebuilder:validation:XValidation:message="filter.extensionRef must be nil if the filter.type is not ExtensionRef",rule="!(has(self.extensionRef) && self.type != 'ExtensionRef')" +// +kubebuilder:validation:XValidation:message="filter.extensionRef must be specified for ExtensionRef filter.type",rule="!(!has(self.extensionRef) && self.type == 'ExtensionRef')" type HTTPRouteFilter struct { // Type identifies the type of filter to apply. As with other API fields, // types are classified into three conformance levels: @@ -587,8 +631,7 @@ type HTTPRouteFilter struct { // Reason of `UnsupportedValue`. // // +unionDiscriminator - // +kubebuilder:validation:Enum=RequestHeaderModifier;RequestMirror;RequestRedirect;ExtensionRef - // + // +kubebuilder:validation:Enum=RequestHeaderModifier;ResponseHeaderModifier;RequestMirror;RequestRedirect;URLRewrite;ExtensionRef Type HTTPRouteFilterType `json:"type"` // RequestHeaderModifier defines a schema for a filter that modifies request @@ -605,13 +648,16 @@ type HTTPRouteFilter struct { // Support: Extended // // +optional - // ResponseHeaderModifier *HTTPHeaderFilter `json:"responseHeaderModifier,omitempty"` // RequestMirror defines a schema for a filter that mirrors requests. // Requests are sent to the specified destination, but responses from // that destination are ignored. // + // This filter can be used multiple times within the same rule. Note that + // not all implementations will be able to support mirroring to multiple + // backends. + // // Support: Extended // // +optional @@ -629,7 +675,6 @@ type HTTPRouteFilter struct { // // Support: Extended // - // // +optional URLRewrite *HTTPURLRewriteFilter `json:"urlRewrite,omitempty"` @@ -638,6 +683,8 @@ type HTTPRouteFilter struct { // "networking.example.net"). ExtensionRef MUST NOT be used for core and // extended filters. // + // This filter can be used multiple times within the same rule. + // // Support: Implementation-specific // // +optional @@ -662,7 +709,6 @@ const ( // Support in HTTPRouteRule: Extended // // Support in HTTPBackendRef: Extended - // HTTPRouteFilterResponseHeaderModifier HTTPRouteFilterType = "ResponseHeaderModifier" // HTTPRouteFilterRequestRedirect can be used to redirect a request to @@ -683,8 +729,6 @@ const ( // Support in HTTPRouteRule: Extended // // Support in HTTPBackendRef: Extended - // - // HTTPRouteFilterURLRewrite HTTPRouteFilterType = "URLRewrite" // HTTPRouteFilterRequestMirror can be used to mirror HTTP requests to a @@ -821,7 +865,11 @@ const ( ) // HTTPPathModifier defines configuration for path modifiers. -// +// +// +kubebuilder:validation:XValidation:message="replaceFullPath must be specified when type is set to 'ReplaceFullPath'",rule="self.type == 'ReplaceFullPath' ? has(self.replaceFullPath) : true" +// +kubebuilder:validation:XValidation:message="type must be 'ReplaceFullPath' when replaceFullPath is set",rule="has(self.replaceFullPath) ? self.type == 'ReplaceFullPath' : true" +// +kubebuilder:validation:XValidation:message="replacePrefixMatch must be specified when type is set to 'ReplacePrefixMatch'",rule="self.type == 'ReplacePrefixMatch' ? has(self.replacePrefixMatch) : true" +// +kubebuilder:validation:XValidation:message="type must be 'ReplacePrefixMatch' when replacePrefixMatch is set",rule="has(self.replacePrefixMatch) ? self.type == 'ReplacePrefixMatch' : true" type HTTPPathModifier struct { // Type defines the type of path modifier. Additional types may be // added in a future release of the API. @@ -833,21 +881,20 @@ type HTTPPathModifier struct { // Accepted Condition for the Route to `status: False`, with a // Reason of `UnsupportedValue`. // - // // +kubebuilder:validation:Enum=ReplaceFullPath;ReplacePrefixMatch Type HTTPPathModifierType `json:"type"` // ReplaceFullPath specifies the value with which to replace the full path // of a request during a rewrite or redirect. // - // // +kubebuilder:validation:MaxLength=1024 // +optional ReplaceFullPath *string `json:"replaceFullPath,omitempty"` // ReplacePrefixMatch specifies the value with which to replace the prefix // match of a request during a rewrite or redirect. For example, a request - // to "/foo/bar" with a prefix match of "/foo" would be modified to "/bar". + // to "/foo/bar" with a prefix match of "/foo" and a ReplacePrefixMatch + // of "/xyz" would be modified to "/xyz/bar". // // Note that this matches the behavior of the PathPrefix match type. This // matches full path elements. A path element refers to the list of labels @@ -855,7 +902,24 @@ type HTTPPathModifier struct { // ignored. For example, the paths `/abc`, `/abc/`, and `/abc/def` would all // match the prefix `/abc`, but the path `/abcd` would not. // - // + // ReplacePrefixMatch is only compatible with a `PathPrefix` HTTPRouteMatch. + // Using any other HTTPRouteMatch type on the same HTTPRouteRule will result in + // the implementation setting the Accepted Condition for the Route to `status: False`. + // + // Request Path | Prefix Match | Replace Prefix | Modified Path + // -------------|--------------|----------------|---------- + // /foo/bar | /foo | /xyz | /xyz/bar + // /foo/bar | /foo | /xyz/ | /xyz/bar + // /foo/bar | /foo/ | /xyz | /xyz/bar + // /foo/bar | /foo/ | /xyz/ | /xyz/bar + // /foo | /foo | /xyz | /xyz + // /foo/ | /foo | /xyz | /xyz/ + // /foo/bar | /foo | | /bar + // /foo/ | /foo | | / + // /foo | /foo | | / + // /foo/ | /foo | / | / + // /foo | /foo | / | / + // // +kubebuilder:validation:MaxLength=1024 // +optional ReplacePrefixMatch *string `json:"replacePrefixMatch,omitempty"` @@ -867,6 +931,9 @@ type HTTPRequestRedirectFilter struct { // Scheme is the scheme to be used in the value of the `Location` header in // the response. When empty, the scheme of the request is used. // + // Scheme redirects can affect the port of the redirect, for more information, + // refer to the documentation for the port field of this filter. + // // Note that values may be added to this enum, implementations // must ensure that unknown values will not cause a crash. // @@ -882,7 +949,7 @@ type HTTPRequestRedirectFilter struct { // Hostname is the hostname to be used in the value of the `Location` // header in the response. - // When empty, the hostname of the request is used. + // When empty, the hostname in the `Host` header of the request is used. // // Support: Core // @@ -895,13 +962,29 @@ type HTTPRequestRedirectFilter struct { // // Support: Extended // - // // +optional Path *HTTPPathModifier `json:"path,omitempty"` // Port is the port to be used in the value of the `Location` // header in the response. - // When empty, port (if specified) of the request is used. + // + // If no port is specified, the redirect port MUST be derived using the + // following rules: + // + // * If redirect scheme is not-empty, the redirect port MUST be the well-known + // port associated with the redirect scheme. Specifically "http" to port 80 + // and "https" to port 443. If the redirect scheme does not have a + // well-known port, the listener port of the Gateway SHOULD be used. + // * If redirect scheme is empty, the redirect port MUST be the Gateway + // Listener port. + // + // Implementations SHOULD NOT add the port number in the 'Location' + // header in the following cases: + // + // * A Location header that will use HTTP (whether that is determined via + // the Listener protocol or the Scheme field) _and_ use port 80. + // * A Location header that will use HTTPS (whether that is determined via + // the Listener protocol or the Scheme field) _and_ use port 443. // // Support: Extended // @@ -930,15 +1013,12 @@ type HTTPRequestRedirectFilter struct { // MUST NOT be used on the same Route rule as a HTTPRequestRedirect filter. // // Support: Extended -// -// type HTTPURLRewriteFilter struct { // Hostname is the value to be used to replace the Host header value during // forwarding. // // Support: Extended // - // // +optional Hostname *PreciseHostname `json:"hostname,omitempty"` @@ -946,7 +1026,6 @@ type HTTPURLRewriteFilter struct { // // Support: Extended // - // // +optional Path *HTTPPathModifier `json:"path,omitempty"` } @@ -1016,6 +1095,12 @@ type HTTPBackendRef struct { // // +optional // +kubebuilder:validation:MaxItems=16 + // +kubebuilder:validation:XValidation:message="May specify either httpRouteFilterRequestRedirect or httpRouteFilterRequestRewrite, but not both",rule="!(self.exists(f, f.type == 'RequestRedirect') && self.exists(f, f.type == 'URLRewrite'))" + // +kubebuilder:validation:XValidation:message="May specify either httpRouteFilterRequestRedirect or httpRouteFilterRequestRewrite, but not both",rule="!(self.exists(f, f.type == 'RequestRedirect') && self.exists(f, f.type == 'URLRewrite'))" + // +kubebuilder:validation:XValidation:message="RequestHeaderModifier filter cannot be repeated",rule="self.exists(f, f.type == 'RequestHeaderModifier') ? self.exists_one(f, f.type == 'RequestHeaderModifier') : true" + // +kubebuilder:validation:XValidation:message="ResponseHeaderModifier filter cannot be repeated",rule="self.exists(f, f.type == 'ResponseHeaderModifier') ? self.exists_one(f, f.type == 'ResponseHeaderModifier') : true" + // +kubebuilder:validation:XValidation:message="RequestRedirect filter cannot be repeated",rule="self.exists(f, f.type == 'RequestRedirect') ? self.exists_one(f, f.type == 'RequestRedirect') : true" + // +kubebuilder:validation:XValidation:message="URLRewrite filter cannot be repeated",rule="self.exists(f, f.type == 'URLRewrite') ? self.exists_one(f, f.type == 'URLRewrite') : true" Filters []HTTPRouteFilter `json:"filters,omitempty"` } diff --git a/apis/v1beta1/object_reference_types.go b/apis/v1beta1/object_reference_types.go index dd4929a9e0..4ef1c37891 100644 --- a/apis/v1beta1/object_reference_types.go +++ b/apis/v1beta1/object_reference_types.go @@ -65,10 +65,10 @@ type SecretObjectReference struct { // Namespace is the namespace of the backend. When unspecified, the local // namespace is inferred. // - // Note that when a namespace is specified, a ReferenceGrant object - // is required in the referent namespace to allow that namespace's - // owner to accept the reference. See the ReferenceGrant documentation - // for details. + // Note that when a namespace different than the local namespace is specified, + // a ReferenceGrant object is required in the referent namespace to allow that + // namespace's owner to accept the reference. See the ReferenceGrant + // documentation for details. // // Support: Core // @@ -80,10 +80,10 @@ type SecretObjectReference struct { // specific to BackendRef. It includes a few additional fields and features // than a regular ObjectReference. // -// Note that when a namespace is specified, a ReferenceGrant object -// is required in the referent namespace to allow that namespace's -// owner to accept the reference. See the ReferenceGrant documentation -// for details. +// Note that when a namespace different than the local namespace is specified, a +// ReferenceGrant object is required in the referent namespace to allow that +// namespace's owner to accept the reference. See the ReferenceGrant +// documentation for details. // // The API object must be valid in the cluster; the Group and Kind must // be registered in the cluster for this reference to be valid. @@ -91,6 +91,8 @@ type SecretObjectReference struct { // References to objects with invalid Group and Kind are not valid, and must // be rejected by the implementation, with appropriate Conditions set // on the containing object. +// +// +kubebuilder:validation:XValidation:message="Must have port for Service reference",rule="(size(self.group) == 0 && self.kind == 'Service') ? has(self.port) : true" type BackendObjectReference struct { // Group is the group of the referent. For example, "gateway.networking.k8s.io". // When unspecified or empty string, core API group is inferred. @@ -99,9 +101,21 @@ type BackendObjectReference struct { // +kubebuilder:default="" Group *Group `json:"group,omitempty"` - // Kind is kind of the referent. For example "HTTPRoute" or "Service". + // Kind is the Kubernetes resource kind of the referent. For example + // "Service". + // // Defaults to "Service" when not specified. // + // ExternalName services can refer to CNAME DNS records that may live + // outside of the cluster and as such are difficult to reason about in + // terms of conformance. They also may not be safe to forward to (see + // CVE-2021-25740 for more information). Implementations SHOULD NOT + // support ExternalName Services. + // + // Support: Core (Services with a type other than ExternalName) + // + // Support: Implementation-specific (Services with type ExternalName) + // // +optional // +kubebuilder:default=Service Kind *Kind `json:"kind,omitempty"` @@ -112,10 +126,10 @@ type BackendObjectReference struct { // Namespace is the namespace of the backend. When unspecified, the local // namespace is inferred. // - // Note that when a namespace is specified, a ReferenceGrant object - // is required in the referent namespace to allow that namespace's - // owner to accept the reference. See the ReferenceGrant documentation - // for details. + // Note that when a namespace different than the local namespace is specified, + // a ReferenceGrant object is required in the referent namespace to allow that + // namespace's owner to accept the reference. See the ReferenceGrant + // documentation for details. // // Support: Core // diff --git a/apis/v1beta1/referencegrant_types.go b/apis/v1beta1/referencegrant_types.go index 668a73ea9c..fe3517d062 100644 --- a/apis/v1beta1/referencegrant_types.go +++ b/apis/v1beta1/referencegrant_types.go @@ -22,6 +22,7 @@ import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" // +kubebuilder:object:root=true // +kubebuilder:resource:categories=gateway-api,shortName=refgrant // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp` +// +kubebuilder:storageversion // ReferenceGrant identifies kinds of resources in other namespaces that are // trusted to reference the specified kinds of resources in the same namespace diff --git a/apis/v1beta1/shared_types.go b/apis/v1beta1/shared_types.go index f973e4b329..c54103cce9 100644 --- a/apis/v1beta1/shared_types.go +++ b/apis/v1beta1/shared_types.go @@ -21,9 +21,14 @@ import ( ) // ParentReference identifies an API object (usually a Gateway) that can be considered -// a parent of this resource (usually a route). The only kind of parent resource -// with "Core" support is Gateway. This API may be extended in the future to -// support additional kinds of parent resources, such as HTTPRoute. +// a parent of this resource (usually a route). There are two kinds of parent resources +// with "Core" support: +// +// * Gateway (Gateway conformance profile) +// * Service (Mesh conformance profile, experimental, ClusterIP Services only) +// +// This API may be extended in the future to support additional kinds of parent +// resources. // // The API object must be valid in the cluster; the Group and Kind must // be registered in the cluster for this reference to be valid. @@ -41,9 +46,12 @@ type ParentReference struct { // Kind is kind of the referent. // - // Support: Core (Gateway) + // There are two kinds of parent resources with "Core" support: + // + // * Gateway (Gateway conformance profile) + // * Service (Mesh conformance profile, experimental, ClusterIP Services only) // - // Support: Implementation-specific (Other Resources) + // Support for other resources is Implementation-Specific. // // +kubebuilder:default=Gateway // +optional @@ -58,6 +66,16 @@ type ParentReference struct { // Gateway has the AllowedRoutes field, and ReferenceGrant provides a // generic way to enable any other kind of cross-namespace reference. // + // ParentRefs from a Route to a Service in the same namespace are "producer" + // routes, which apply default routing rules to inbound connections from + // any namespace to the Service. + // + // ParentRefs from a Route to a Service in a different namespace are + // "consumer" routes, and these routing rules are only applied to outbound + // connections originating from the same namespace as the Route, for which + // the intended destination of the connections are a Service targeted as a + // ParentRef of the Route. + // // Support: Core // // +optional @@ -74,6 +92,11 @@ type ParentReference struct { // * Gateway: Listener Name. When both Port (experimental) and SectionName // are specified, the name and port of the selected listener must match // both specified values. + // * Service: Port Name. When both Port (experimental) and SectionName + // are specified, the name and port of the selected listener must match + // both specified values. Note that attaching Routes to Services as Parents + // is part of experimental Mesh support and is not supported for any other + // purpose. // // Implementations MAY choose to support attaching Routes to other resources. // If that is the case, they MUST clearly document how SectionName is @@ -104,6 +127,10 @@ type ParentReference struct { // and SectionName are specified, the name and port of the selected listener // must match both specified values. // + // When the parent resource is a Service, this targets a specific port in the + // Service spec. When both Port (experimental) and SectionName are specified, + // the name and port of the selected port must match both specified values. + // // Implementations MAY choose to support other parent resources. // Implementations supporting other types of parent resources MUST clearly // document how/if Port is interpreted. @@ -130,15 +157,25 @@ type CommonRouteSpec struct { // to be attached to. Note that the referenced parent resource needs to // allow this for the attachment to be complete. For Gateways, that means // the Gateway needs to allow attachment from Routes of this kind and - // namespace. + // namespace. For Services, that means the Service must either be in the same + // namespace for a "producer" route, or the mesh implementation must support + // and allow "consumer" routes for the referenced Service. ReferenceGrant is + // not applicable for governing ParentRefs to Services - it is not possible to + // create a "producer" route for a Service in a different namespace from the + // Route. + // + // There are two kinds of parent resources with "Core" support: + // + // * Gateway (Gateway conformance profile) + // * Service (Mesh conformance profile, experimental, ClusterIP Services only) // - // The only kind of parent resource with "Core" support is Gateway. This API - // may be extended in the future to support additional kinds of parent - // resources such as one of the route kinds. + // This API may be extended in the future to support additional kinds of parent + // resources. // // It is invalid to reference an identical parent more than once. It is // valid to reference multiple distinct sections within the same parent - // resource, such as 2 Listeners within a Gateway. + // resource, such as two separate Listeners on the same Gateway or two separate + // ports on the same Service. // // It is possible to separately reference multiple distinct objects that may // be collapsed by an implementation. For example, some implementations may @@ -150,7 +187,17 @@ type CommonRouteSpec struct { // rules. Cross-namespace references are only valid if they are explicitly // allowed by something in the namespace they are referring to. For example, // Gateway has the AllowedRoutes field, and ReferenceGrant provides a - // generic way to enable any other kind of cross-namespace reference. + // generic way to enable other kinds of cross-namespace reference. + // + // ParentRefs from a Route to a Service in the same namespace are "producer" + // routes, which apply default routing rules to inbound connections from + // any namespace to the Service. + // + // ParentRefs from a Route to a Service in a different namespace are + // "consumer" routes, and these routing rules are only applied to outbound + // connections originating from the same namespace as the Route, for which + // the intended destination of the connections are a Service targeted as a + // ParentRef of the Route. // // +optional // +kubebuilder:validation:MaxItems=32 @@ -166,10 +213,10 @@ type PortNumber int32 // BackendRef defines how a Route should forward a request to a Kubernetes // resource. // -// Note that when a namespace is specified, a ReferenceGrant object -// is required in the referent namespace to allow that namespace's -// owner to accept the reference. See the ReferenceGrant documentation -// for details. +// Note that when a namespace different than the local namespace is specified, a +// ReferenceGrant object is required in the referent namespace to allow that +// namespace's owner to accept the reference. See the ReferenceGrant +// documentation for details. type BackendRef struct { // BackendObjectReference references a Kubernetes object. BackendObjectReference `json:",inline"` @@ -215,7 +262,6 @@ const ( // * "NoMatchingListenerHostname" // * "NoMatchingParent" // * "UnsupportedValue" - // * "ParentRefNotPermitted" // // Possible reasons for this condition to be Unknown are: // @@ -249,15 +295,15 @@ const ( // is not recognized. RouteReasonUnsupportedValue RouteConditionReason = "UnsupportedValue" - // This reason is used with the "Accepted" condition when the route has not - // been accepted by a Gateway because it has a cross-namespace parentRef, - // but no ReferenceGrant in the other namespace allows such a reference. - RouteReasonParentRefNotPermitted RouteConditionReason = "ParentRefNotPermitted" - // This reason is used with the "Accepted" when a controller has not yet // reconciled the route. RouteReasonPending RouteConditionReason = "Pending" + // This reason is used with the "Accepted" condition when there + // are incompatible filters present on a route rule (for example if + // the URLRewrite and RequestRedirect are both present on an HTTPRoute). + RouteReasonIncompatibleFilters RouteConditionReason = "IncompatibleFilters" + // This condition indicates whether the controller was able to resolve all // the object references for the Route. // @@ -552,6 +598,14 @@ type AnnotationValue string // +kubebuilder:validation:Pattern=`^Hostname|IPAddress|NamedAddress|[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$` type AddressType string +// HeaderName is the name of a header or query parameter. +// +// +kubebuilder:validation:MinLength=1 +// +kubebuilder:validation:MaxLength=256 +// +kubebuilder:validation:Pattern=`^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$` +// +k8s:deepcopy-gen=false +type HeaderName string + const ( // A textual representation of a numeric IP address. IPv4 // addresses must be in dotted-decimal form. IPv6 addresses diff --git a/apis/v1beta1/validation/common.go b/apis/v1beta1/validation/common.go index 626911ed8f..8edda6a107 100644 --- a/apis/v1beta1/validation/common.go +++ b/apis/v1beta1/validation/common.go @@ -35,10 +35,14 @@ func ValidateParentRefs(parentRefs []gatewayv1b1.ParentReference, path *field.Pa namespace gatewayv1b1.Namespace kind gatewayv1b1.Kind } - parentRefsSectionMap := make(map[sameKindParentRefs][]gatewayv1b1.SectionName) + type parentQualifier struct { + section gatewayv1b1.SectionName + port gatewayv1b1.PortNumber + } + parentRefsSectionMap := make(map[sameKindParentRefs]sets.Set[parentQualifier]) for i, p := range parentRefs { - targetParentRefs := sameKindParentRefs{name: p.Name, namespace: *new(gatewayv1b1.Namespace), kind: *new(gatewayv1b1.Kind)} - targetSection := new(gatewayv1b1.SectionName) + targetParentRefs := sameKindParentRefs{name: p.Name, namespace: gatewayv1b1.Namespace(""), kind: gatewayv1b1.Kind("")} + pq := parentQualifier{} if p.Namespace != nil { targetParentRefs.namespace = *p.Namespace } @@ -46,19 +50,33 @@ func ValidateParentRefs(parentRefs []gatewayv1b1.ParentReference, path *field.Pa targetParentRefs.kind = *p.Kind } if p.SectionName != nil { - targetSection = p.SectionName + pq.section = *p.SectionName + } + if p.Port != nil { + pq.port = *p.Port } if s, ok := parentRefsSectionMap[targetParentRefs]; ok { - if len(s[0]) == 0 || len(*targetSection) == 0 { - errs = append(errs, field.Required(path.Child("parentRefs"), "sectionNames must be specified when more than one parentRef refers to the same parent")) + if s.UnsortedList()[0] == (parentQualifier{}) || pq == (parentQualifier{}) { + errs = append(errs, field.Required(path.Child("parentRefs"), "sectionNames or ports must be specified when more than one parentRef refers to the same parent")) return errs } - if sets.New(s...).Has(*targetSection) { - errs = append(errs, field.Invalid(path.Index(i).Child("parentRefs").Child("sectionName"), targetSection, "must be unique when ParentRefs includes 2 or more references to the same parent")) + if s.Has(pq) { + fieldPath := path.Index(i).Child("parentRefs") + var val any + if len(pq.section) > 0 { + fieldPath = fieldPath.Child("sectionName") + val = pq.section + } else { + fieldPath = fieldPath.Child("port") + val = pq.port + } + errs = append(errs, field.Invalid(fieldPath, val, "must be unique when ParentRefs includes 2 or more references to the same parent")) return errs } + parentRefsSectionMap[targetParentRefs].Insert(pq) + } else { + parentRefsSectionMap[targetParentRefs] = sets.New(pq) } - parentRefsSectionMap[targetParentRefs] = append(parentRefsSectionMap[targetParentRefs], *targetSection) } return errs } diff --git a/apis/v1beta1/validation/common_test.go b/apis/v1beta1/validation/common_test.go index 8ec6e74cd7..9f841f3498 100644 --- a/apis/v1beta1/validation/common_test.go +++ b/apis/v1beta1/validation/common_test.go @@ -17,6 +17,7 @@ limitations under the License. package validation import ( + "strings" "testing" "k8s.io/apimachinery/pkg/util/validation/field" @@ -24,7 +25,7 @@ import ( gatewayv1b1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) -var path = *new(field.Path) +var path = field.Path{} func TestValidateParentRefs(t *testing.T) { namespace := gatewayv1b1.Namespace("example-namespace") @@ -36,7 +37,7 @@ func TestValidateParentRefs(t *testing.T) { tests := []struct { name string parentRefs []gatewayv1b1.ParentReference - errCount int + err string }{{ name: "valid ParentRefs includes 1 reference", parentRefs: []gatewayv1b1.ParentReference{ @@ -47,7 +48,7 @@ func TestValidateParentRefs(t *testing.T) { SectionName: §ionA, }, }, - errCount: 0, + err: "", }, { name: "valid ParentRefs includes 2 references", parentRefs: []gatewayv1b1.ParentReference{ @@ -64,7 +65,7 @@ func TestValidateParentRefs(t *testing.T) { SectionName: §ionB, }, }, - errCount: 0, + err: "", }, { name: "valid ParentRefs when different references have the same section name", parentRefs: []gatewayv1b1.ParentReference{ @@ -81,7 +82,7 @@ func TestValidateParentRefs(t *testing.T) { SectionName: §ionA, }, }, - errCount: 0, + err: "", }, { name: "valid ParentRefs includes more references to the same parent", parentRefs: []gatewayv1b1.ParentReference{ @@ -104,7 +105,7 @@ func TestValidateParentRefs(t *testing.T) { SectionName: §ionC, }, }, - errCount: 0, + err: "", }, { name: "invalid ParentRefs due to the same section names to the same parentRefs", parentRefs: []gatewayv1b1.ParentReference{ @@ -121,7 +122,7 @@ func TestValidateParentRefs(t *testing.T) { SectionName: §ionA, }, }, - errCount: 1, + err: "must be unique when ParentRefs", }, { name: "invalid ParentRefs due to section names not set to the same ParentRefs", parentRefs: []gatewayv1b1.ParentReference{ @@ -132,7 +133,7 @@ func TestValidateParentRefs(t *testing.T) { Name: "example", }, }, - errCount: 1, + err: "sectionNames or ports must be specified", }, { name: "invalid ParentRefs due to more same section names to the same ParentRefs", parentRefs: []gatewayv1b1.ParentReference{ @@ -157,7 +158,7 @@ func TestValidateParentRefs(t *testing.T) { SectionName: §ionA, }, }, - errCount: 1, + err: "sectionNames or ports must be specified", }, { name: "invalid ParentRefs when one ParentRef section name not set to the same ParentRefs", parentRefs: []gatewayv1b1.ParentReference{ @@ -172,7 +173,7 @@ func TestValidateParentRefs(t *testing.T) { SectionName: §ionA, }, }, - errCount: 1, + err: "sectionNames or ports must be specified", }, { name: "invalid ParentRefs when next ParentRef section name not set to the same ParentRefs", parentRefs: []gatewayv1b1.ParentReference{ @@ -187,7 +188,84 @@ func TestValidateParentRefs(t *testing.T) { SectionName: nil, }, }, - errCount: 1, + err: "sectionNames or ports must be specified", + }, { + name: "valid ParentRefs with multiple port references to the same parent", + parentRefs: []gatewayv1b1.ParentReference{ + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(81)), + }, + }, + err: "", + }, { + name: "valid ParentRefs with multiple mixed references to the same parent", + parentRefs: []gatewayv1b1.ParentReference{ + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + { + Name: "example", + Namespace: &namespace, + SectionName: §ionA, + }, + }, + err: "", + }, { + name: "invalid ParentRefs due to same port references to the same parent", + parentRefs: []gatewayv1b1.ParentReference{ + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + err: "port: Invalid value: 80: must be unique when ParentRefs", + }, { + name: "invalid ParentRefs due to mixed port references to the same parent", + parentRefs: []gatewayv1b1.ParentReference{ + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + { + Name: "example", + Namespace: &namespace, + Port: nil, + }, + }, + err: "Required value: sectionNames or ports must be specified", + }, { + name: "valid ParentRefs with multiple same port references to different section of a parent", + parentRefs: []gatewayv1b1.ParentReference{ + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + SectionName: §ionA, + }, + { + Name: "example", + Namespace: &namespace, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + SectionName: §ionB, + }, + }, + err: "", }} for _, tc := range tests { @@ -196,8 +274,16 @@ func TestValidateParentRefs(t *testing.T) { ParentRefs: tc.parentRefs, } errs := ValidateParentRefs(spec.ParentRefs, path.Child("spec")) - if len(errs) != tc.errCount { - t.Errorf("got %d errors, want %d errors: %s", len(errs), tc.errCount, errs) + if tc.err == "" { + if len(errs) != 0 { + t.Errorf("got %d errors, want none: %s", len(errs), errs) + } + } else { + if errs == nil { + t.Errorf("got no errors, want %q", tc.err) + } else if !strings.Contains(errs.ToAggregate().Error(), tc.err) { + t.Errorf("got %d errors, want %q: %s", len(errs), tc.err, errs) + } } }) } diff --git a/apis/v1beta1/validation/gateway.go b/apis/v1beta1/validation/gateway.go index 60d7f55305..3119efb2c0 100644 --- a/apis/v1beta1/validation/gateway.go +++ b/apis/v1beta1/validation/gateway.go @@ -18,7 +18,10 @@ package validation import ( "fmt" + "net/netip" + "regexp" + "k8s.io/apimachinery/pkg/util/sets" "k8s.io/apimachinery/pkg/util/validation/field" gatewayv1b1 "sigs.k8s.io/gateway-api/apis/v1beta1" @@ -41,6 +44,9 @@ var ( gatewayv1b1.HTTPSProtocolType: {}, gatewayv1b1.TLSProtocolType: {}, } + + validHostnameAddress = `^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$` + validHostnameRegexp = regexp.MustCompile(validHostnameAddress) ) // ValidateGateway validates gw according to the Gateway API specification. @@ -59,6 +65,7 @@ func ValidateGateway(gw *gatewayv1b1.Gateway) field.ErrorList { func ValidateGatewaySpec(spec *gatewayv1b1.GatewaySpec, path *field.Path) field.ErrorList { var errs field.ErrorList errs = append(errs, validateGatewayListeners(spec.Listeners, path.Child("listeners"))...) + errs = append(errs, validateGatewayAddresses(spec.Addresses, path.Child("addresses"))...) return errs } @@ -69,10 +76,12 @@ func validateGatewayListeners(listeners []gatewayv1b1.Listener, path *field.Path errs = append(errs, ValidateListenerTLSConfig(listeners, path)...) errs = append(errs, validateListenerHostname(listeners, path)...) errs = append(errs, ValidateTLSCertificateRefs(listeners, path)...) + errs = append(errs, ValidateListenerNames(listeners, path)...) + errs = append(errs, validateHostnameProtocolPort(listeners, path)...) return errs } -// validateListenerTLSConfig validates TLS config must be set when protocol is HTTPS or TLS, +// ValidateListenerTLSConfig validates TLS config must be set when protocol is HTTPS or TLS, // and TLS config shall not be present when protocol is HTTP, TCP or UDP func ValidateListenerTLSConfig(listeners []gatewayv1b1.Listener, path *field.Path) field.ErrorList { var errs field.ErrorList @@ -112,7 +121,74 @@ func ValidateTLSCertificateRefs(listeners []gatewayv1b1.Listener, path *field.Pa for i, c := range listeners { if isProtocolInSubset(c.Protocol, protocolsTLSRequired) && c.TLS != nil { if *c.TLS.Mode == gatewayv1b1.TLSModeTerminate && len(c.TLS.CertificateRefs) == 0 { - errs = append(errs, field.Forbidden(path.Index(i).Child("tls").Child("certificateRefs"), fmt.Sprintln("should be set and not empty when TLSModeType is Terminate"))) + errs = append(errs, field.Forbidden(path.Index(i).Child("tls").Child("certificateRefs"), "should be set and not empty when TLSModeType is Terminate")) + } + } + } + return errs +} + +// ValidateListenerNames validates the names of the listeners +// must be unique within the Gateway +func ValidateListenerNames(listeners []gatewayv1b1.Listener, path *field.Path) field.ErrorList { + var errs field.ErrorList + nameMap := make(map[gatewayv1b1.SectionName]struct{}, len(listeners)) + for i, c := range listeners { + if _, found := nameMap[c.Name]; found { + errs = append(errs, field.Duplicate(path.Index(i).Child("name"), "must be unique within the Gateway")) + } + nameMap[c.Name] = struct{}{} + } + return errs +} + +// validateHostnameProtocolPort validates that the combination of port, protocol, and hostname are +// unique for each listener. +func validateHostnameProtocolPort(listeners []gatewayv1b1.Listener, path *field.Path) field.ErrorList { + var errs field.ErrorList + hostnameProtocolPortSets := sets.Set[string]{} + for i, listener := range listeners { + hostname := new(gatewayv1b1.Hostname) + if listener.Hostname != nil { + hostname = listener.Hostname + } + protocol := listener.Protocol + port := listener.Port + hostnameProtocolPort := fmt.Sprintf("%s:%s:%d", *hostname, protocol, port) + if hostnameProtocolPortSets.Has(hostnameProtocolPort) { + errs = append(errs, field.Duplicate(path.Index(i), "combination of port, protocol, and hostname must be unique for each listener")) + } else { + hostnameProtocolPortSets.Insert(hostnameProtocolPort) + } + } + return errs +} + +// validateGatewayAddresses validates whether fields of addresses are set according +// to the Gateway API specification. +func validateGatewayAddresses(addresses []gatewayv1b1.GatewayAddress, path *field.Path) field.ErrorList { + var errs field.ErrorList + ipAddrSet, hostnameAddrSet := sets.Set[string]{}, sets.Set[string]{} + for i, address := range addresses { + if address.Type != nil { + if *address.Type == gatewayv1b1.IPAddressType { + if _, err := netip.ParseAddr(address.Value); err != nil { + errs = append(errs, field.Invalid(path.Index(i), address.Value, "invalid ip address")) + } + if ipAddrSet.Has(address.Value) { + errs = append(errs, field.Duplicate(path.Index(i), address.Value)) + } else { + ipAddrSet.Insert(address.Value) + } + } else if *address.Type == gatewayv1b1.HostnameAddressType { + if !validHostnameRegexp.MatchString(address.Value) { + errs = append(errs, field.Invalid(path.Index(i), address.Value, fmt.Sprintf("must only contain valid characters (matching %s)", validHostnameAddress))) + } + if hostnameAddrSet.Has(address.Value) { + errs = append(errs, field.Duplicate(path.Index(i), address.Value)) + } else { + hostnameAddrSet.Insert(address.Value) + } } } } diff --git a/apis/v1beta1/validation/gateway_test.go b/apis/v1beta1/validation/gateway_test.go index dfb3a1877c..533c4e97f4 100644 --- a/apis/v1beta1/validation/gateway_test.go +++ b/apis/v1beta1/validation/gateway_test.go @@ -17,9 +17,11 @@ limitations under the License. package validation import ( + "fmt" "testing" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/util/validation/field" gatewayv1b1 "sigs.k8s.io/gateway-api/apis/v1beta1" ) @@ -49,52 +51,80 @@ func TestValidateGateway(t *testing.T) { tlsConfig := gatewayv1b1.GatewayTLSConfig{} testCases := map[string]struct { - mutate func(gw *gatewayv1b1.Gateway) - expectErrsOnFields []string + mutate func(gw *gatewayv1b1.Gateway) + expectErrs []field.Error }{ "tls config present with http protocol": { mutate: func(gw *gatewayv1b1.Gateway) { gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPProtocolType gw.Spec.Listeners[0].TLS = &tlsConfig }, - expectErrsOnFields: []string{"spec.listeners[0].tls"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].tls", + Detail: "should be empty for protocol HTTP", + BadValue: "", + }, + }, }, "tls config present with tcp protocol": { mutate: func(gw *gatewayv1b1.Gateway) { gw.Spec.Listeners[0].Protocol = gatewayv1b1.TCPProtocolType gw.Spec.Listeners[0].TLS = &tlsConfig }, - expectErrsOnFields: []string{"spec.listeners[0].tls"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].tls", + Detail: "should be empty for protocol TCP", + BadValue: "", + }, + }, }, "tls config not set with https protocol": { mutate: func(gw *gatewayv1b1.Gateway) { gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPSProtocolType }, - expectErrsOnFields: []string{"spec.listeners[0].tls"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].tls", + Detail: "must be set for protocol HTTPS", + BadValue: "", + }, + }, }, "tls config not set with tls protocol": { mutate: func(gw *gatewayv1b1.Gateway) { gw.Spec.Listeners[0].Protocol = gatewayv1b1.TLSProtocolType }, - expectErrsOnFields: []string{"spec.listeners[0].tls"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].tls", + Detail: "must be set for protocol TLS", + BadValue: "", + }, + }, }, "tls config not set with http protocol": { mutate: func(gw *gatewayv1b1.Gateway) { gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPProtocolType }, - expectErrsOnFields: nil, + expectErrs: nil, }, "tls config not set with tcp protocol": { mutate: func(gw *gatewayv1b1.Gateway) { gw.Spec.Listeners[0].Protocol = gatewayv1b1.TCPProtocolType }, - expectErrsOnFields: nil, + expectErrs: nil, }, "tls config not set with udp protocol": { mutate: func(gw *gatewayv1b1.Gateway) { gw.Spec.Listeners[0].Protocol = gatewayv1b1.UDPProtocolType }, - expectErrsOnFields: nil, + expectErrs: nil, }, "hostname present with tcp protocol": { mutate: func(gw *gatewayv1b1.Gateway) { @@ -102,7 +132,14 @@ func TestValidateGateway(t *testing.T) { gw.Spec.Listeners[0].Hostname = &hostname gw.Spec.Listeners[0].Protocol = gatewayv1b1.TCPProtocolType }, - expectErrsOnFields: []string{"spec.listeners[0].hostname"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].hostname", + Detail: "should be empty for protocol TCP", + BadValue: "", + }, + }, }, "hostname present with udp protocol": { mutate: func(gw *gatewayv1b1.Gateway) { @@ -110,7 +147,14 @@ func TestValidateGateway(t *testing.T) { gw.Spec.Listeners[0].Hostname = &hostname gw.Spec.Listeners[0].Protocol = gatewayv1b1.UDPProtocolType }, - expectErrsOnFields: []string{"spec.listeners[0].hostname"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].hostname", + Detail: "should be empty for protocol UDP", + BadValue: "", + }, + }, }, "certificatedRefs not set with https protocol and TLS terminate mode": { mutate: func(gw *gatewayv1b1.Gateway) { @@ -121,7 +165,14 @@ func TestValidateGateway(t *testing.T) { gw.Spec.Listeners[0].TLS = &tlsConfig gw.Spec.Listeners[0].TLS.Mode = &tlsMode }, - expectErrsOnFields: []string{"spec.listeners[0].tls.certificateRefs"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].tls.certificateRefs", + Detail: "should be set and not empty when TLSModeType is Terminate", + BadValue: "", + }, + }, }, "certificatedRefs not set with tls protocol and TLS terminate mode": { mutate: func(gw *gatewayv1b1.Gateway) { @@ -132,7 +183,236 @@ func TestValidateGateway(t *testing.T) { gw.Spec.Listeners[0].TLS = &tlsConfig gw.Spec.Listeners[0].TLS.Mode = &tlsMode }, - expectErrsOnFields: []string{"spec.listeners[0].tls.certificateRefs"}, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeForbidden, + Field: "spec.listeners[0].tls.certificateRefs", + Detail: "should be set and not empty when TLSModeType is Terminate", + BadValue: "", + }, + }, + }, + "names are not unique within the Gateway": { + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + hostnameBar := gatewayv1b1.Hostname("bar.com") + gw.Spec.Listeners[0].Name = "foo" + gw.Spec.Listeners[0].Hostname = &hostnameFoo + gw.Spec.Listeners = append(gw.Spec.Listeners, + gatewayv1b1.Listener{ + Name: "foo", + Hostname: &hostnameBar, + }, + ) + }, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeDuplicate, + Field: "spec.listeners[1].name", + BadValue: "must be unique within the Gateway", + }, + }, + }, + "combination of port, protocol, and hostname are not unique for each listener": { + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + gw.Spec.Listeners[0].Name = "foo" + gw.Spec.Listeners[0].Hostname = &hostnameFoo + gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPProtocolType + gw.Spec.Listeners[0].Port = 80 + gw.Spec.Listeners = append(gw.Spec.Listeners, + gatewayv1b1.Listener{ + Name: "bar", + Hostname: &hostnameFoo, + Protocol: gatewayv1b1.HTTPProtocolType, + Port: 80, + }, + ) + }, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeDuplicate, + Field: "spec.listeners[1]", + BadValue: "combination of port, protocol, and hostname must be unique for each listener", + }, + }, + }, + "combination of port and protocol are not unique for each listener when hostnames not set": { + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners[0].Name = "foo" + gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPProtocolType + gw.Spec.Listeners[0].Port = 80 + gw.Spec.Listeners = append(gw.Spec.Listeners, + gatewayv1b1.Listener{ + Name: "bar", + Protocol: gatewayv1b1.HTTPProtocolType, + Port: 80, + }, + ) + }, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeDuplicate, + Field: "spec.listeners[1]", + BadValue: "combination of port, protocol, and hostname must be unique for each listener", + }, + }, + }, + "port is unique when protocol and hostname are the same": { + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + gw.Spec.Listeners[0].Name = "foo" + gw.Spec.Listeners[0].Hostname = &hostnameFoo + gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPProtocolType + gw.Spec.Listeners[0].Port = 80 + gw.Spec.Listeners = append(gw.Spec.Listeners, + gatewayv1b1.Listener{ + Name: "bar", + Hostname: &hostnameFoo, + Protocol: gatewayv1b1.HTTPProtocolType, + Port: 8080, + }, + ) + }, + expectErrs: nil, + }, + "hostname is unique when protocol and port are the same": { + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + hostnameBar := gatewayv1b1.Hostname("bar.com") + gw.Spec.Listeners[0].Name = "foo" + gw.Spec.Listeners[0].Hostname = &hostnameFoo + gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPProtocolType + gw.Spec.Listeners[0].Port = 80 + gw.Spec.Listeners = append(gw.Spec.Listeners, + gatewayv1b1.Listener{ + Name: "bar", + Hostname: &hostnameBar, + Protocol: gatewayv1b1.HTTPProtocolType, + Port: 80, + }, + ) + }, + expectErrs: nil, + }, + "protocol is unique when port and hostname are the same": { + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + tlsConfigFoo := tlsConfig + tlsModeFoo := gatewayv1b1.TLSModeType("Terminate") + tlsConfigFoo.Mode = &tlsModeFoo + tlsConfigFoo.CertificateRefs = []gatewayv1b1.SecretObjectReference{ + { + Name: "FooCertificateRefs", + }, + } + gw.Spec.Listeners[0].Name = "foo" + gw.Spec.Listeners[0].Hostname = &hostnameFoo + gw.Spec.Listeners[0].Protocol = gatewayv1b1.HTTPSProtocolType + gw.Spec.Listeners[0].Port = 8000 + gw.Spec.Listeners[0].TLS = &tlsConfigFoo + gw.Spec.Listeners = append(gw.Spec.Listeners, + gatewayv1b1.Listener{ + Name: "bar", + Hostname: &hostnameFoo, + Protocol: gatewayv1b1.TLSProtocolType, + Port: 8000, + TLS: &tlsConfigFoo, + }, + ) + }, + expectErrs: nil, + }, + "ip address and hostname in addresses are valid": { + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Addresses = []gatewayv1b1.GatewayAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4", + }, + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1111:2222:3333:4444::", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "foo.bar", + }, + } + }, + expectErrs: nil, + }, + "ip address and hostname in addresses are invalid": { + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Addresses = []gatewayv1b1.GatewayAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4:8080", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "*foo/bar", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "12:34:56::", + }, + } + }, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeInvalid, + Field: "spec.addresses[0]", + Detail: "invalid ip address", + BadValue: "1.2.3.4:8080", + }, + { + Type: field.ErrorTypeInvalid, + Field: "spec.addresses[1]", + Detail: fmt.Sprintf("must only contain valid characters (matching %s)", validHostnameAddress), + BadValue: "*foo/bar", + }, + { + Type: field.ErrorTypeInvalid, + Field: "spec.addresses[2]", + Detail: fmt.Sprintf("must only contain valid characters (matching %s)", validHostnameAddress), + BadValue: "12:34:56::", + }, + }, + }, + "duplicate ip address or hostname": { + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Addresses = []gatewayv1b1.GatewayAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4", + }, + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "foo.bar", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "foo.bar", + }, + } + }, + expectErrs: []field.Error{ + { + Type: field.ErrorTypeDuplicate, + Field: "spec.addresses[1]", + BadValue: "1.2.3.4", + }, + { + Type: field.ErrorTypeDuplicate, + Field: "spec.addresses[3]", + BadValue: "foo.bar", + }, + }, }, } @@ -142,12 +422,21 @@ func TestValidateGateway(t *testing.T) { gw := baseGateway.DeepCopy() tc.mutate(gw) errs := ValidateGateway(gw) - if len(tc.expectErrsOnFields) != len(errs) { - t.Fatalf("Expected %d errors, got %d errors: %v", len(tc.expectErrsOnFields), len(errs), errs) + if len(tc.expectErrs) != len(errs) { + t.Fatalf("Expected %d errors, got %d errors: %v", len(tc.expectErrs), len(errs), errs) } for i, err := range errs { - if err.Field != tc.expectErrsOnFields[i] { - t.Errorf("Expected error on field: %s, got: %s", tc.expectErrsOnFields[i], err.Error()) + if err.Type != tc.expectErrs[i].Type { + t.Errorf("Expected error on type: %s, got: %s", tc.expectErrs[i].Type, err.Type) + } + if err.Field != tc.expectErrs[i].Field { + t.Errorf("Expected error on field: %s, got: %s", tc.expectErrs[i].Field, err.Field) + } + if err.Detail != tc.expectErrs[i].Detail { + t.Errorf("Expected error on detail: %s, got: %s", tc.expectErrs[i].Detail, err.Detail) + } + if err.BadValue != tc.expectErrs[i].BadValue { + t.Errorf("Expected error on bad value: %s, got: %s", tc.expectErrs[i].BadValue, err.BadValue) } } }) diff --git a/apis/v1beta1/validation/httproute.go b/apis/v1beta1/validation/httproute.go index 506c953558..8dbe4e8f3d 100644 --- a/apis/v1beta1/validation/httproute.go +++ b/apis/v1beta1/validation/httproute.go @@ -32,6 +32,7 @@ var ( // repeated multiple times in a rule. repeatableHTTPRouteFilters = []gatewayv1b1.HTTPRouteFilterType{ gatewayv1b1.HTTPRouteFilterExtensionRef, + gatewayv1b1.HTTPRouteFilterRequestMirror, } // Invalid path sequences and suffixes, primarily related to directory traversal @@ -55,6 +56,7 @@ func ValidateHTTPRouteSpec(spec *gatewayv1b1.HTTPRouteSpec, path *field.Path) fi var errs field.ErrorList for i, rule := range spec.Rules { errs = append(errs, validateHTTPRouteFilters(rule.Filters, rule.Matches, path.Child("rules").Index(i))...) + errs = append(errs, validateRequestRedirectFiltersWithBackendRefs(rule, path.Child("rules").Index(i))...) for j, backendRef := range rule.BackendRefs { errs = append(errs, validateHTTPRouteFilters(backendRef.Filters, rule.Matches, path.Child("rules").Index(i).Child("backendRefs").Index(j))...) } @@ -77,6 +79,17 @@ func ValidateHTTPRouteSpec(spec *gatewayv1b1.HTTPRouteSpec, path *field.Path) fi return errs } +// validateRequestRedirectFiltersWithBackendRefs validates that RequestRedirect filters are not used with backendRefs +func validateRequestRedirectFiltersWithBackendRefs(rule gatewayv1b1.HTTPRouteRule, path *field.Path) field.ErrorList { + var errs field.ErrorList + for _, filter := range rule.Filters { + if filter.RequestRedirect != nil && len(rule.BackendRefs) > 0 { + errs = append(errs, field.Invalid(path.Child("filters"), gatewayv1b1.HTTPRouteFilterRequestRedirect, "RequestRedirect filter is not allowed with backendRefs")) + } + } + return errs +} + // validateHTTPRouteBackendServicePorts validates that v1.Service backends always have a port. func validateHTTPRouteBackendServicePorts(rules []gatewayv1b1.HTTPRouteRule, path *field.Path) field.ErrorList { var errs field.ErrorList @@ -125,15 +138,16 @@ func validateHTTPRouteFilters(filters []gatewayv1b1.HTTPRouteFilter, matches []g } errs = append(errs, validateHTTPRouteFilterTypeMatchesValue(filter, path.Index(i))...) } - // custom filters don't have any validation - for _, key := range repeatableHTTPRouteFilters { - delete(counts, key) - } if counts[gatewayv1b1.HTTPRouteFilterRequestRedirect] > 0 && counts[gatewayv1b1.HTTPRouteFilterURLRewrite] > 0 { errs = append(errs, field.Invalid(path.Child("filters"), gatewayv1b1.HTTPRouteFilterRequestRedirect, "may specify either httpRouteFilterRequestRedirect or httpRouteFilterRequestRewrite, but not both")) } + // repeatableHTTPRouteFilters filters can be used more than once + for _, key := range repeatableHTTPRouteFilters { + delete(counts, key) + } + for filterType, count := range counts { if count > 1 { errs = append(errs, field.Invalid(path.Child("filters"), filterType, "cannot be used multiple times in the same rule")) diff --git a/apis/v1beta1/validation/httproute_test.go b/apis/v1beta1/validation/httproute_test.go index 4e886ce0f3..c157dc3704 100644 --- a/apis/v1beta1/validation/httproute_test.go +++ b/apis/v1beta1/validation/httproute_test.go @@ -28,7 +28,6 @@ import ( func TestValidateHTTPRoute(t *testing.T) { testService := gatewayv1b1.ObjectName("test-service") - specialService := gatewayv1b1.ObjectName("special-service") pathPrefixMatchType := gatewayv1b1.PathMatchPathPrefix tests := []struct { @@ -102,20 +101,20 @@ func TestValidateHTTPRoute(t *testing.T) { }, Filters: []gatewayv1b1.HTTPRouteFilter{ { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1b1.BackendObjectReference{ - Name: testService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), }, }, }, { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1b1.BackendObjectReference{ - Name: specialService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("bar"), }, }, }, @@ -172,7 +171,7 @@ func TestValidateHTTPRoute(t *testing.T) { }, }, { name: "invalid httpRoute with multiple duplicate filters", - errCount: 3, + errCount: 2, rules: []gatewayv1b1.HTTPRouteRule{ { Matches: []gatewayv1b1.HTTPRouteMatch{ @@ -184,15 +183,6 @@ func TestValidateHTTPRoute(t *testing.T) { }, }, Filters: []gatewayv1b1.HTTPRouteFilter{ - { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1b1.BackendObjectReference{ - Name: testService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), - }, - }, - }, { Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ @@ -204,15 +194,6 @@ func TestValidateHTTPRoute(t *testing.T) { }, }, }, - { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1b1.BackendObjectReference{ - Name: testService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), - }, - }, - }, { Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ @@ -235,15 +216,6 @@ func TestValidateHTTPRoute(t *testing.T) { }, }, }, - { - Type: gatewayv1b1.HTTPRouteFilterRequestMirror, - RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ - BackendRef: gatewayv1b1.BackendObjectReference{ - Name: specialService, - Port: ptrTo(gatewayv1b1.PortNumber(8080)), - }, - }, - }, { Type: gatewayv1b1.HTTPRouteFilterResponseHeaderModifier, ResponseHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ @@ -638,8 +610,8 @@ func TestValidateHTTPBackendUniqueFilters(t *testing.T) { }, }}, }, { - name: "invalid httpRoute Rules duplicate mirror filter", - errCount: 1, + name: "valid httpRoute Rules duplicate mirror filter", + errCount: 0, rules: []gatewayv1b1.HTTPRouteRule{{ BackendRefs: []gatewayv1b1.HTTPBackendRef{ { @@ -1049,7 +1021,7 @@ func TestValidateHTTPRouteTypeMatchesField(t *testing.T) { StatusCode: new(int), }, }, - errCount: 0, + errCount: 1, }, { name: "invalid HTTPRouteFilterRequestRedirect type filter with non-matching field", routeFilter: gatewayv1b1.HTTPRouteFilter{ @@ -1140,3 +1112,67 @@ func TestValidateHTTPRouteTypeMatchesField(t *testing.T) { }) } } + +func TestValidateRequestRedirectFiltersWithNoBackendRef(t *testing.T) { + testService := gatewayv1b1.ObjectName("test-service") + tests := []struct { + name string + rules []gatewayv1b1.HTTPRouteRule + errCount int + }{ + { + name: "backendref with request redirect httpRoute filter", + errCount: 1, + rules: []gatewayv1b1.HTTPRouteRule{ + { + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Scheme: ptrTo("https"), + StatusCode: ptrTo(301), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + }, + }, + }, + }, + }, { + name: "request redirect without backendref in httpRoute filter", + errCount: 0, + rules: []gatewayv1b1.HTTPRouteRule{ + { + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Scheme: ptrTo("https"), + StatusCode: ptrTo(301), + }, + }, + }, + }, + }, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + var errs field.ErrorList + route := gatewayv1b1.HTTPRoute{Spec: gatewayv1b1.HTTPRouteSpec{Rules: tc.rules}} + errs = ValidateHTTPRoute(&route) + if len(errs) != tc.errCount { + t.Errorf("got %d errors, want %d errors: %s", len(errs), tc.errCount, errs) + } + }) + } +} diff --git a/apis/v1beta1/zz_generated.deepcopy.go b/apis/v1beta1/zz_generated.deepcopy.go index afda0d4dd2..699576a302 100644 --- a/apis/v1beta1/zz_generated.deepcopy.go +++ b/apis/v1beta1/zz_generated.deepcopy.go @@ -350,7 +350,7 @@ func (in *GatewayStatus) DeepCopyInto(out *GatewayStatus) { *out = *in if in.Addresses != nil { in, out := &in.Addresses, &out.Addresses - *out = make([]GatewayAddress, len(*in)) + *out = make([]GatewayStatusAddress, len(*in)) for i := range *in { (*in)[i].DeepCopyInto(&(*out)[i]) } @@ -381,6 +381,26 @@ func (in *GatewayStatus) DeepCopy() *GatewayStatus { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *GatewayStatusAddress) DeepCopyInto(out *GatewayStatusAddress) { + *out = *in + if in.Type != nil { + in, out := &in.Type, &out.Type + *out = new(AddressType) + **out = **in + } +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new GatewayStatusAddress. +func (in *GatewayStatusAddress) DeepCopy() *GatewayStatusAddress { + if in == nil { + return nil + } + out := new(GatewayStatusAddress) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *GatewayTLSConfig) DeepCopyInto(out *GatewayTLSConfig) { *out = *in diff --git a/cmd/admission/main.go b/cmd/admission/main.go index 7e983e1b0f..af8bfc1ded 100644 --- a/cmd/admission/main.go +++ b/cmd/admission/main.go @@ -27,6 +27,7 @@ import ( "os/signal" "sync" "syscall" + "time" "k8s.io/klog/v2" @@ -70,7 +71,8 @@ func main() { } server := &http.Server{ - Addr: ":8443", + Addr: ":8443", + ReadHeaderTimeout: 10 * time.Second, // for Potential Slowloris Attack (G112) // Require at least TLS12 to satisfy golint G402. TLSConfig: &tls.Config{MinVersion: tls.VersionTLS12, Certificates: []tls.Certificate{certs}}, } diff --git a/config/crd/experimental/gateway.networking.k8s.io_gatewayclasses.yaml b/config/crd/experimental/gateway.networking.k8s.io_gatewayclasses.yaml index 95a7f65c70..810e530dfc 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_gatewayclasses.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_gatewayclasses.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: gatewayclasses.gateway.networking.k8s.io @@ -49,7 +49,7 @@ spec: to GatewayClass or associated parameters. If implementations choose to propagate GatewayClass changes to existing Gateways, that MUST be clearly documented by the implementation. \n Whenever one or more Gateways are using a GatewayClass, - implementations MUST add the `gateway-exists-finalizer.gateway.networking.k8s.io` + implementations SHOULD add the `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer on the associated GatewayClass. This ensures that a GatewayClass associated with a Gateway is not deleted while in use. \n GatewayClass is a Cluster level resource." @@ -78,6 +78,9 @@ spec: minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$ type: string + x-kubernetes-validations: + - message: Value is immutable + rule: self == oldSelf description: description: Description helps describe a GatewayClass with more details. maxLength: 64 @@ -132,7 +135,9 @@ spec: reason: Waiting status: Unknown type: Accepted - description: Status defines the current state of GatewayClass. + description: "Status defines the current state of GatewayClass. \n Implementations + MUST populate status on all GatewayClass resources which specify their + controller name." properties: conditions: default: @@ -219,7 +224,7 @@ spec: required: - spec type: object - served: true + served: false storage: false subresources: status: {} @@ -249,7 +254,7 @@ spec: to GatewayClass or associated parameters. If implementations choose to propagate GatewayClass changes to existing Gateways, that MUST be clearly documented by the implementation. \n Whenever one or more Gateways are using a GatewayClass, - implementations MUST add the `gateway-exists-finalizer.gateway.networking.k8s.io` + implementations SHOULD add the `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer on the associated GatewayClass. This ensures that a GatewayClass associated with a Gateway is not deleted while in use. \n GatewayClass is a Cluster level resource." @@ -278,6 +283,9 @@ spec: minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$ type: string + x-kubernetes-validations: + - message: Value is immutable + rule: self == oldSelf description: description: Description helps describe a GatewayClass with more details. maxLength: 64 @@ -332,7 +340,9 @@ spec: reason: Waiting status: Unknown type: Accepted - description: Status defines the current state of GatewayClass. + description: "Status defines the current state of GatewayClass. \n Implementations + MUST populate status on all GatewayClass resources which specify their + controller name." properties: conditions: default: diff --git a/config/crd/experimental/gateway.networking.k8s.io_gateways.yaml b/config/crd/experimental/gateway.networking.k8s.io_gateways.yaml index 95b6d245ef..cd4c774225 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_gateways.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_gateways.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: gateways.gateway.networking.k8s.io @@ -73,10 +73,24 @@ spec: manner, assigning an appropriate set of Addresses. \n The implementation MUST bind all Listeners to every GatewayAddress that it assigns to the Gateway and add a corresponding entry in GatewayStatus.Addresses. - \n Support: Extended" + \n Support: Extended \n " items: description: GatewayAddress describes an address that can be bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -95,8 +109,20 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array + x-kubernetes-validations: + - message: IPAddress values must be unique + rule: 'self.all(a1, a1.type == ''IPAddress'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' + - message: Hostname values must be unique + rule: 'self.all(a1, a1.type == ''Hostname'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' gatewayClassName: description: GatewayClassName used for this Gateway. This is the name of a GatewayClass resource. @@ -108,27 +134,34 @@ spec: logical endpoints that are bound on this Gateway's addresses. At least one Listener MUST be specified. \n Each listener in a Gateway must have a unique combination of Hostname, Port, and Protocol. - \n An implementation MAY group Listeners by Port and then collapse - each group of Listeners into a single Listener if the implementation - determines that the Listeners in the group are \"compatible\". An - implementation MAY also group together and collapse compatible Listeners - belonging to different Gateways. \n For example, an implementation - might consider Listeners to be compatible with each other if all - of the following conditions are met: \n 1. Either each Listener - within the group specifies the \"HTTP\" Protocol or each Listener - within the group specifies either the \"HTTPS\" or \"TLS\" Protocol. - \n 2. Each Listener within the group specifies a Hostname that is - unique within the group. \n 3. As a special case, one Listener within - a group may omit Hostname, in which case this Listener matches when - no other Listener matches. \n If the implementation does collapse - compatible Listeners, the hostname provided in the incoming client - request MUST be matched to a Listener to find the correct set of - Routes. The incoming hostname MUST be matched using the Hostname - field for each Listener in order of most to least specific. That - is, exact matches must be processed before wildcard matches. \n - If this field specifies multiple Listeners that have the same Port - value but are not compatible, the implementation must raise a \"Conflicted\" - condition in the Listener status. \n Support: Core" + \n Within the HTTP Conformance Profile, the below combinations of + port and protocol are considered Core and MUST be supported: \n + 1. Port: 80, Protocol: HTTP 2. Port: 443, Protocol: HTTPS \n Within + the TLS Conformance Profile, the below combinations of port and + protocol are considered Core and MUST be supported: \n 1. Port: + 443, Protocol: TLS \n Port and protocol combinations not listed + above are considered Extended. \n An implementation MAY group Listeners + by Port and then collapse each group of Listeners into a single + Listener if the implementation determines that the Listeners in + the group are \"compatible\". An implementation MAY also group together + and collapse compatible Listeners belonging to different Gateways. + \n For example, an implementation might consider Listeners to be + compatible with each other if all of the following conditions are + met: \n 1. Either each Listener within the group specifies the \"HTTP\" + Protocol or each Listener within the group specifies either the + \"HTTPS\" or \"TLS\" Protocol. \n 2. Each Listener within the group + specifies a Hostname that is unique within the group. \n 3. As a + special case, one Listener within a group may omit Hostname, in + which case this Listener matches when no other Listener matches. + \n If the implementation does collapse compatible Listeners, the + hostname provided in the incoming client request MUST be matched + to a Listener to find the correct set of Routes. The incoming hostname + MUST be matched using the Hostname field for each Listener in order + of most to least specific. That is, exact matches must be processed + before wildcard matches. \n If this field specifies multiple Listeners + that have the same Port value but are not compatible, the implementation + must raise a \"Conflicted\" condition in the Listener status. \n + Support: Core" items: description: Listener embodies the concept of a logical endpoint where a Gateway accepts network connections. @@ -201,7 +234,7 @@ spec: from: default: Same description: "From indicates where Routes will be selected - for this Gateway. Possible values are: * All: Routes + for this Gateway. Possible values are: \n * All: Routes in all namespaces may be used by this Gateway. * Selector: Routes in namespaces selected by the selector may be used by this Gateway. * Same: Only Routes in the @@ -379,11 +412,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. - \n Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to - allow that namespace's owner to accept the reference. - See the ReferenceGrant documentation for details. - \n Support: Core" + \n Note that when a namespace different than the + local namespace is specified, a ReferenceGrant object + is required in the referent namespace to allow that + namespace's owner to accept the reference. See the + ReferenceGrant documentation for details. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -431,6 +465,11 @@ spec: maxProperties: 16 type: object type: object + x-kubernetes-validations: + - message: certificateRefs must be specified when TLSModeType + is Terminate + rule: 'self.mode == ''Terminate'' ? size(self.certificateRefs) + > 0 : true' required: - name - port @@ -442,6 +481,24 @@ spec: x-kubernetes-list-map-keys: - name x-kubernetes-list-type: map + x-kubernetes-validations: + - message: tls must be specified for protocols ['HTTPS', 'TLS'] + rule: 'self.all(l, l.protocol in [''HTTPS'', ''TLS''] ? has(l.tls) + : true)' + - message: tls must not be specified for protocols ['HTTP', 'TCP', + 'UDP'] + rule: 'self.all(l, l.protocol in [''HTTP'', ''TCP'', ''UDP''] ? + !has(l.tls) : true)' + - message: hostname must not be specified for protocols ['TCP', 'UDP'] + rule: 'self.all(l, l.protocol in [''TCP'', ''UDP''] ? (!has(l.hostname) + || l.hostname == '''') : true)' + - message: Listener name must be unique within the Gateway + rule: self.all(l1, self.exists_one(l2, l1.name == l2.name)) + - message: Combination of port, protocol and hostname must be unique + for each listener + rule: 'self.all(l1, self.exists_one(l2, l1.port == l2.port && l1.protocol + == l2.protocol && (has(l1.hostname) && has(l2.hostname) ? l1.hostname + == l2.hostname : true)))' required: - gatewayClassName - listeners @@ -451,19 +508,38 @@ spec: conditions: - lastTransitionTime: "1970-01-01T00:00:00Z" message: Waiting for controller - reason: NotReconciled + reason: Pending status: Unknown type: Accepted + - lastTransitionTime: "1970-01-01T00:00:00Z" + message: Waiting for controller + reason: Pending + status: Unknown + type: Programmed description: Status defines the current state of Gateway. properties: addresses: - description: Addresses lists the IP addresses that have actually been - bound to the Gateway. These addresses may differ from the addresses + description: "Addresses lists the IP addresses that have actually + been bound to the Gateway. These addresses may differ from the addresses in the Spec, e.g. if the Gateway automatically assigns an address - from a reserved pool. + from a reserved pool. \n " items: - description: GatewayAddress describes an address that can be bound + description: GatewayStatusAddress describes an address that is bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -482,6 +558,11 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array conditions: @@ -501,7 +582,7 @@ spec: the `GatewayConditionType` and `GatewayConditionReason` constants so that operators and tools can converge on a common vocabulary to describe Gateway state. \n Known condition types are: \n * \"Accepted\" - * \"Ready\"" + * \"Programmed\" * \"Ready\"" items: description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct @@ -580,8 +661,8 @@ spec: description: ListenerStatus is the status associated with a Listener. properties: attachedRoutes: - description: AttachedRoutes represents the total number of Routes - that have been successfully attached to this Listener. + description: AttachedRoutes represents the total number of accepted + Routes that have been successfully attached to this Listener. format: int32 type: integer conditions: @@ -716,7 +797,7 @@ spec: required: - spec type: object - served: true + served: false storage: false subresources: status: {} @@ -770,10 +851,24 @@ spec: manner, assigning an appropriate set of Addresses. \n The implementation MUST bind all Listeners to every GatewayAddress that it assigns to the Gateway and add a corresponding entry in GatewayStatus.Addresses. - \n Support: Extended" + \n Support: Extended \n " items: description: GatewayAddress describes an address that can be bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -792,8 +887,20 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array + x-kubernetes-validations: + - message: IPAddress values must be unique + rule: 'self.all(a1, a1.type == ''IPAddress'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' + - message: Hostname values must be unique + rule: 'self.all(a1, a1.type == ''Hostname'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' gatewayClassName: description: GatewayClassName used for this Gateway. This is the name of a GatewayClass resource. @@ -805,27 +912,34 @@ spec: logical endpoints that are bound on this Gateway's addresses. At least one Listener MUST be specified. \n Each listener in a Gateway must have a unique combination of Hostname, Port, and Protocol. - \n An implementation MAY group Listeners by Port and then collapse - each group of Listeners into a single Listener if the implementation - determines that the Listeners in the group are \"compatible\". An - implementation MAY also group together and collapse compatible Listeners - belonging to different Gateways. \n For example, an implementation - might consider Listeners to be compatible with each other if all - of the following conditions are met: \n 1. Either each Listener - within the group specifies the \"HTTP\" Protocol or each Listener - within the group specifies either the \"HTTPS\" or \"TLS\" Protocol. - \n 2. Each Listener within the group specifies a Hostname that is - unique within the group. \n 3. As a special case, one Listener within - a group may omit Hostname, in which case this Listener matches when - no other Listener matches. \n If the implementation does collapse - compatible Listeners, the hostname provided in the incoming client - request MUST be matched to a Listener to find the correct set of - Routes. The incoming hostname MUST be matched using the Hostname - field for each Listener in order of most to least specific. That - is, exact matches must be processed before wildcard matches. \n - If this field specifies multiple Listeners that have the same Port - value but are not compatible, the implementation must raise a \"Conflicted\" - condition in the Listener status. \n Support: Core" + \n Within the HTTP Conformance Profile, the below combinations of + port and protocol are considered Core and MUST be supported: \n + 1. Port: 80, Protocol: HTTP 2. Port: 443, Protocol: HTTPS \n Within + the TLS Conformance Profile, the below combinations of port and + protocol are considered Core and MUST be supported: \n 1. Port: + 443, Protocol: TLS \n Port and protocol combinations not listed + above are considered Extended. \n An implementation MAY group Listeners + by Port and then collapse each group of Listeners into a single + Listener if the implementation determines that the Listeners in + the group are \"compatible\". An implementation MAY also group together + and collapse compatible Listeners belonging to different Gateways. + \n For example, an implementation might consider Listeners to be + compatible with each other if all of the following conditions are + met: \n 1. Either each Listener within the group specifies the \"HTTP\" + Protocol or each Listener within the group specifies either the + \"HTTPS\" or \"TLS\" Protocol. \n 2. Each Listener within the group + specifies a Hostname that is unique within the group. \n 3. As a + special case, one Listener within a group may omit Hostname, in + which case this Listener matches when no other Listener matches. + \n If the implementation does collapse compatible Listeners, the + hostname provided in the incoming client request MUST be matched + to a Listener to find the correct set of Routes. The incoming hostname + MUST be matched using the Hostname field for each Listener in order + of most to least specific. That is, exact matches must be processed + before wildcard matches. \n If this field specifies multiple Listeners + that have the same Port value but are not compatible, the implementation + must raise a \"Conflicted\" condition in the Listener status. \n + Support: Core" items: description: Listener embodies the concept of a logical endpoint where a Gateway accepts network connections. @@ -898,7 +1012,7 @@ spec: from: default: Same description: "From indicates where Routes will be selected - for this Gateway. Possible values are: * All: Routes + for this Gateway. Possible values are: \n * All: Routes in all namespaces may be used by this Gateway. * Selector: Routes in namespaces selected by the selector may be used by this Gateway. * Same: Only Routes in the @@ -1076,11 +1190,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. - \n Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to - allow that namespace's owner to accept the reference. - See the ReferenceGrant documentation for details. - \n Support: Core" + \n Note that when a namespace different than the + local namespace is specified, a ReferenceGrant object + is required in the referent namespace to allow that + namespace's owner to accept the reference. See the + ReferenceGrant documentation for details. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -1128,6 +1243,11 @@ spec: maxProperties: 16 type: object type: object + x-kubernetes-validations: + - message: certificateRefs must be specified when TLSModeType + is Terminate + rule: 'self.mode == ''Terminate'' ? size(self.certificateRefs) + > 0 : true' required: - name - port @@ -1139,6 +1259,24 @@ spec: x-kubernetes-list-map-keys: - name x-kubernetes-list-type: map + x-kubernetes-validations: + - message: tls must be specified for protocols ['HTTPS', 'TLS'] + rule: 'self.all(l, l.protocol in [''HTTPS'', ''TLS''] ? has(l.tls) + : true)' + - message: tls must not be specified for protocols ['HTTP', 'TCP', + 'UDP'] + rule: 'self.all(l, l.protocol in [''HTTP'', ''TCP'', ''UDP''] ? + !has(l.tls) : true)' + - message: hostname must not be specified for protocols ['TCP', 'UDP'] + rule: 'self.all(l, l.protocol in [''TCP'', ''UDP''] ? (!has(l.hostname) + || l.hostname == '''') : true)' + - message: Listener name must be unique within the Gateway + rule: self.all(l1, self.exists_one(l2, l1.name == l2.name)) + - message: Combination of port, protocol and hostname must be unique + for each listener + rule: 'self.all(l1, self.exists_one(l2, l1.port == l2.port && l1.protocol + == l2.protocol && (has(l1.hostname) && has(l2.hostname) ? l1.hostname + == l2.hostname : true)))' required: - gatewayClassName - listeners @@ -1148,19 +1286,38 @@ spec: conditions: - lastTransitionTime: "1970-01-01T00:00:00Z" message: Waiting for controller - reason: NotReconciled + reason: Pending status: Unknown type: Accepted + - lastTransitionTime: "1970-01-01T00:00:00Z" + message: Waiting for controller + reason: Pending + status: Unknown + type: Programmed description: Status defines the current state of Gateway. properties: addresses: - description: Addresses lists the IP addresses that have actually been - bound to the Gateway. These addresses may differ from the addresses + description: "Addresses lists the IP addresses that have actually + been bound to the Gateway. These addresses may differ from the addresses in the Spec, e.g. if the Gateway automatically assigns an address - from a reserved pool. + from a reserved pool. \n " items: - description: GatewayAddress describes an address that can be bound + description: GatewayStatusAddress describes an address that is bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -1179,6 +1336,11 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array conditions: @@ -1198,7 +1360,7 @@ spec: the `GatewayConditionType` and `GatewayConditionReason` constants so that operators and tools can converge on a common vocabulary to describe Gateway state. \n Known condition types are: \n * \"Accepted\" - * \"Ready\"" + * \"Programmed\" * \"Ready\"" items: description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct @@ -1277,8 +1439,8 @@ spec: description: ListenerStatus is the status associated with a Listener. properties: attachedRoutes: - description: AttachedRoutes represents the total number of Routes - that have been successfully attached to this Listener. + description: AttachedRoutes represents the total number of accepted + Routes that have been successfully attached to this Listener. format: int32 type: integer conditions: diff --git a/config/crd/experimental/gateway.networking.k8s.io_grpcroutes.yaml b/config/crd/experimental/gateway.networking.k8s.io_grpcroutes.yaml index 1a21f263f4..49c8480d3c 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_grpcroutes.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_grpcroutes.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: grpcroutes.gateway.networking.k8s.io @@ -127,13 +127,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -141,16 +149,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -164,8 +181,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -185,7 +205,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -200,18 +228,22 @@ spec: a Route must apply to a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY choose - to support other parent resources. Implementations supporting - other types of parent resources MUST clearly document how/if - Port is interpreted. \n For the purpose of status, an attachment - is considered successful as long as the parent resource accepts - it partially. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + must match both specified values. \n When the parent resource + is a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are specified, + the name and port of the selected port must match both specified + values. \n Implementations MAY choose to support other parent + resources. Implementations supporting other types of parent + resources MUST clearly document how/if Port is interpreted. + \n For the purpose of status, an attachment is considered + successful as long as the parent resource accepts it partially. + For example, Gateway listeners can restrict which Routes can + attach to them by Route kind, namespace, or hostname. If 1 + of 2 Gateway listeners accept attachment from the referencing + Route, the Route MUST be considered successfully attached. + If no Gateway listeners accept attachment from this Route, + the Route MUST be considered detached from the Gateway. \n + Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -222,19 +254,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -251,7 +287,7 @@ spec: type: Exact description: Rules are a list of GRPC matchers, filters and actions. items: - description: GRPCRouteRule defines the semantics for matching an + description: GRPCRouteRule defines the semantics for matching a gRPC request based on conditions (matches), processing it (filters), and forwarding the request to an API object (backendRefs). properties: @@ -301,7 +337,9 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n Support: Implementation-specific \n + This filter can be used multiple times within + the same rule." properties: group: description: Group is the group of the referent. @@ -439,7 +477,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from - that destination are ignored. \n Support: Extended" + that destination are ignored. \n This filter can + be used multiple times within the same rule. Note + that not all implementations will be able to support + mirroring to multiple backends. \n Support: Extended" properties: backendRef: description: "BackendRef references a resource @@ -474,9 +515,20 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. - For example "HTTPRoute" or "Service". - Defaults to "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to + CNAME DNS records that may live outside + of the cluster and as such are difficult + to reason about in terms of conformance. + They also may not be safe to forward to + (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with + a type other than ExternalName) \n Support: + Implementation-specific (Services with + type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -490,7 +542,8 @@ spec: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n Note that - when a namespace is specified, a ReferenceGrant + when a namespace different than the local + namespace is specified, a ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the ReferenceGrant @@ -516,13 +569,17 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind + == ''Service'') ? has(self.port) : true' required: - backendRef type: object responseHeaderModifier: description: "ResponseHeaderModifier defines a schema for a filter that modifies response headers. \n - Support: Extended \n " + Support: Extended" properties: add: description: "Add adds the given header(s) (name, @@ -674,9 +731,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -689,11 +754,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -731,6 +796,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 type: array filters: @@ -743,8 +812,13 @@ spec: all implementations that support GRPCRoute. - Implementers are encouraged to support extended filters. - Implementation-specific custom filters have no API guarantees across implementations. - \n Specifying a core filter multiple times has unspecified - or implementation-specific conformance. Support: Core" + \n Specifying the same filter multiple times is not supported + unless explicitly indicated in the filter. \n If an implementation + can not support a combination of filters, it must clearly + document that limitation. In cases where incompatible or unsupported + filters are specified and cause the `Accepted` condition to + be set to status `False`, implementations may use the `IncompatibleFilters` + reason to specify this configuration error. \n Support: Core" items: description: GRPCRouteFilter defines processing steps that must be completed during the request or response lifecycle. @@ -760,7 +834,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n Support: Implementation-specific \n This + filter can be used multiple times within the same rule." properties: group: description: Group is the group of the referent. For @@ -890,7 +965,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from that destination are - ignored. \n Support: Extended" + ignored. \n This filter can be used multiple times within + the same rule. Note that not all implementations will + be able to support mirroring to multiple backends. \n + Support: Extended" properties: backendRef: description: "BackendRef references a resource where @@ -923,9 +1001,18 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For - example "HTTPRoute" or "Service". Defaults to - "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to CNAME + DNS records that may live outside of the cluster + and as such are difficult to reason about in + terms of conformance. They also may not be safe + to forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with a + type other than ExternalName) \n Support: Implementation-specific + (Services with type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -938,11 +1025,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace - is inferred. \n Note that when a namespace is - specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's - owner to accept the reference. See the ReferenceGrant - documentation for details. \n Support: Core" + is inferred. \n Note that when a namespace different + than the local namespace is specified, a ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept the + reference. See the ReferenceGrant documentation + for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -962,13 +1050,17 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' required: - backendRef type: object responseHeaderModifier: description: "ResponseHeaderModifier defines a schema for a filter that modifies response headers. \n Support: - Extended \n " + Extended" properties: add: description: "Add adds the given header(s) (name, @@ -1101,9 +1193,6 @@ spec: maxItems: 16 type: array matches: - default: - - method: - type: Exact description: "Matches define conditions used for matching the rule against incoming gRPC requests. Each match is independent, i.e. this rule will be matched if **any** one of the matches @@ -1186,8 +1275,6 @@ spec: - name x-kubernetes-list-type: map method: - default: - type: Exact description: Method specifies a gRPC request service/method matcher. If this field is not specified, all services and methods will match. @@ -1196,19 +1283,15 @@ spec: description: "Value of the method to match against. If left empty or omitted, will match all services. \n At least one of Service and Method MUST be a - non-empty string. \n A GRPC Method must be a valid - Protobuf Method (https://protobuf.com/docs/language-spec#methods)." + non-empty string." maxLength: 1024 - pattern: ^[A-Za-z_][A-Za-z_0-9]*$ type: string service: description: "Value of the service to match against. If left empty or omitted, will match any service. \n At least one of Service and Method MUST be a - non-empty string. \n A GRPC Service must be a valid - Protobuf Type Name (https://protobuf.com/docs/language-spec#type-references)." + non-empty string." maxLength: 1024 - pattern: ^(?i)\.?[a-z_][a-z_0-9]*(\.[a-z_][a-z_0-9]*)*$ type: string type: default: Exact @@ -1374,9 +1457,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -1396,7 +1481,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -1412,8 +1506,12 @@ spec: a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener must match - both specified values. \n Implementations MAY choose to - support other parent resources. Implementations supporting + both specified values. \n When the parent resource is + a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are + specified, the name and port of the selected port must + match both specified values. \n Implementations MAY choose + to support other parent resources. Implementations supporting other types of parent resources MUST clearly document how/if Port is interpreted. \n For the purpose of status, an attachment is considered successful as long as the @@ -1424,7 +1522,7 @@ spec: the Route MUST be considered successfully attached. If no Gateway listeners accept attachment from this Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + \n Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -1435,8 +1533,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of diff --git a/config/crd/experimental/gateway.networking.k8s.io_httproutes.yaml b/config/crd/experimental/gateway.networking.k8s.io_httproutes.yaml index b0b0c1e036..1d0f027249 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_httproutes.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_httproutes.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: httproutes.gateway.networking.k8s.io @@ -52,13 +52,17 @@ spec: description: Spec defines the desired state of HTTPRoute. properties: hostnames: - description: "Hostnames defines a set of hostname that should match - against the HTTP Host header to select a HTTPRoute to process the - request. This matches the RFC 1123 definition of a hostname with - 2 notable exceptions: \n 1. IPs are not allowed. 2. A hostname may - be prefixed with a wildcard label (`*.`). The wildcard label must - appear by itself as the first label. \n If a hostname is specified - by both the Listener and HTTPRoute, there must be at least one intersecting + description: "Hostnames defines a set of hostnames that should match + against the HTTP Host header to select a HTTPRoute used to process + the request. Implementations MUST ignore any port value specified + in the HTTP Host header while performing a match and (absent of + any applicable header modification configuration) MUST forward this + header unmodified to the backend. \n Valid values for Hostnames + are determined by RFC 1123 definition of a hostname with 2 notable + exceptions: \n 1. IPs are not allowed. 2. A hostname may be prefixed + with a wildcard label (`*.`). The wildcard label must appear by + itself as the first label. \n If a hostname is specified by both + the Listener and HTTPRoute, there must be at least one intersecting hostname for the HTTPRoute to be attached to the Listener. For example: \n * A Listener with `test.example.com` as the hostname matches HTTPRoutes that have either not specified any hostnames, or have @@ -110,13 +114,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -124,16 +136,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -147,8 +168,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -168,7 +192,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -183,18 +215,22 @@ spec: a Route must apply to a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY choose - to support other parent resources. Implementations supporting - other types of parent resources MUST clearly document how/if - Port is interpreted. \n For the purpose of status, an attachment - is considered successful as long as the parent resource accepts - it partially. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + must match both specified values. \n When the parent resource + is a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are specified, + the name and port of the selected port must match both specified + values. \n Implementations MAY choose to support other parent + resources. Implementations supporting other types of parent + resources MUST clearly document how/if Port is interpreted. + \n For the purpose of status, an attachment is considered + successful as long as the parent resource accepts it partially. + For example, Gateway listeners can restrict which Routes can + attach to them by Route kind, namespace, or hostname. If 1 + of 2 Gateway listeners accept attachment from the referencing + Route, the Route MUST be considered successfully attached. + If no Gateway listeners accept attachment from this Route, + the Route MUST be considered detached from the Gateway. \n + Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -205,19 +241,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -256,7 +296,8 @@ spec: \n For example, if two backends are specified with equal weights, and one is invalid, 50 percent of traffic must receive a 500. Implementations may choose how that 50 percent is determined. - \n Support: Core for Kubernetes Service \n Support: Implementation-specific + \n Support: Core for Kubernetes Service \n Support: Extended + for Kubernetes ServiceImport \n Support: Implementation-specific for any other resource \n Support for weight: Core" items: description: HTTPBackendRef defines how a HTTPRoute should @@ -284,7 +325,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times + within the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. @@ -422,7 +464,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from - that destination are ignored. \n Support: Extended" + that destination are ignored. \n This filter can + be used multiple times within the same rule. Note + that not all implementations will be able to support + mirroring to multiple backends. \n Support: Extended" properties: backendRef: description: "BackendRef references a resource @@ -457,9 +502,20 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. - For example "HTTPRoute" or "Service". - Defaults to "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to + CNAME DNS records that may live outside + of the cluster and as such are difficult + to reason about in terms of conformance. + They also may not be safe to forward to + (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with + a type other than ExternalName) \n Support: + Implementation-specific (Services with + type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -473,7 +529,8 @@ spec: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n Note that - when a namespace is specified, a ReferenceGrant + when a namespace different than the local + namespace is specified, a ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the ReferenceGrant @@ -499,6 +556,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind + == ''Service'') ? has(self.port) : true' required: - backendRef type: object @@ -511,7 +572,8 @@ spec: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. When empty, the hostname - of the request is used. \n Support: Core" + in the `Host` header of the request is used. + \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -521,14 +583,12 @@ spec: modify the path of the incoming request. The modified path is then used to construct the `Location` header. When empty, the request - path is used as-is. \n Support: Extended \n - " + path is used as-is. \n Support: Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies - the value with which to replace the full - path of a request during a rewrite or - redirect. \n " + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -536,8 +596,9 @@ spec: the value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" - with a prefix match of \"/foo\" would - be modified to \"/bar\". \n Note that + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list of labels @@ -546,7 +607,28 @@ spec: For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path `/abcd` would not. - \n " + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" maxLength: 1024 type: string type: @@ -559,7 +641,7 @@ spec: values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -567,11 +649,45 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value of the `Location` header in the - response. When empty, port (if specified) - of the request is used. \n Support: Extended" + response. \n If no port is specified, the + redirect port MUST be derived using the following + rules: \n * If redirect scheme is not-empty, + the redirect port MUST be the well-known port + associated with the redirect scheme. Specifically + \"http\" to port 80 and \"https\" to port + 443. If the redirect scheme does not have + a well-known port, the listener port of the + Gateway SHOULD be used. * If redirect scheme + is empty, the redirect port MUST be the Gateway + Listener port. \n Implementations SHOULD NOT + add the port number in the 'Location' header + in the following cases: \n * A Location header + that will use HTTP (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 80. * A Location header that + will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -580,8 +696,11 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request - is used. \n Note that values may be added - to this enum, implementations must ensure + is used. \n Scheme redirects can affect the + port of the redirect, for more information, + refer to the documentation for the port field + of this filter. \n Note that values may be + added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition @@ -610,7 +729,7 @@ spec: responseHeaderModifier: description: "ResponseHeaderModifier defines a schema for a filter that modifies response headers. \n - Support: Extended \n " + Support: Extended" properties: add: description: "Add adds the given header(s) (name, @@ -745,7 +864,7 @@ spec: a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - RequestHeaderModifier - ResponseHeaderModifier @@ -757,25 +876,24 @@ spec: urlRewrite: description: "URLRewrite defines a schema for a filter that modifies a request during forwarding. - \n Support: Extended \n " + \n Support: Extended" properties: hostname: description: "Hostname is the value to be used to replace the Host header value during forwarding. - \n Support: Extended \n " + \n Support: Extended" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string path: description: "Path defines a path rewrite. \n - Support: Extended \n " + Support: Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies - the value with which to replace the full - path of a request during a rewrite or - redirect. \n " + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -783,8 +901,9 @@ spec: the value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" - with a prefix match of \"/foo\" would - be modified to \"/bar\". \n Note that + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list of labels @@ -793,7 +912,28 @@ spec: For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path `/abcd` would not. - \n " + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" maxLength: 1024 type: string type: @@ -806,7 +946,7 @@ spec: values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -814,12 +954,97 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil + if the filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type + != ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type + == ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil + if the filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type + != ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for + RequestMirror filter.type + rule: '!(!has(self.requestMirror) && self.type == + ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the + filter.type is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != + ''RequestRedirect'')' + - message: filter.requestRedirect must be specified + for RequestRedirect filter.type + rule: '!(!has(self.requestRedirect) && self.type == + ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for + ExtensionRef filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') + ? self.exists_one(f, f.type == ''RequestRedirect'') + : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' group: default: "" description: Group is the group of the referent. For example, @@ -830,9 +1055,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -845,11 +1078,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -887,6 +1120,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 type: array filters: @@ -899,14 +1136,15 @@ spec: all implementations. - Implementers are encouraged to support extended filters. - Implementation-specific custom filters have no API guarantees across implementations. \n Specifying - a core filter multiple times has unspecified or implementation-specific - conformance. \n All filters are expected to be compatible - with each other except for the URLRewrite and RequestRedirect + the same filter multiple times is not supported unless explicitly + indicated in the filter. \n All filters are expected to be + compatible with each other except for the URLRewrite and RequestRedirect filters, which may not be combined. If an implementation can not support other combinations of filters, they must clearly - document that limitation. In all cases where incompatible - or unsupported filters are specified, implementations MUST - add a warning condition to status. \n Support: Core" + document that limitation. In cases where incompatible or unsupported + filters are specified and cause the `Accepted` condition to + be set to status `False`, implementations may use the `IncompatibleFilters` + reason to specify this configuration error. \n Support: Core" items: description: HTTPRouteFilter defines processing steps that must be completed during the request or response lifecycle. @@ -922,7 +1160,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times within + the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. For @@ -1052,7 +1291,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from that destination are - ignored. \n Support: Extended" + ignored. \n This filter can be used multiple times within + the same rule. Note that not all implementations will + be able to support mirroring to multiple backends. \n + Support: Extended" properties: backendRef: description: "BackendRef references a resource where @@ -1085,9 +1327,18 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For - example "HTTPRoute" or "Service". Defaults to - "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to CNAME + DNS records that may live outside of the cluster + and as such are difficult to reason about in + terms of conformance. They also may not be safe + to forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with a + type other than ExternalName) \n Support: Implementation-specific + (Services with type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -1100,11 +1351,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace - is inferred. \n Note that when a namespace is - specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's - owner to accept the reference. See the ReferenceGrant - documentation for details. \n Support: Core" + is inferred. \n Note that when a namespace different + than the local namespace is specified, a ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept the + reference. See the ReferenceGrant documentation + for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -1124,6 +1376,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' required: - backendRef type: object @@ -1135,8 +1391,8 @@ spec: hostname: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. - When empty, the hostname of the request is used. - \n Support: Core" + When empty, the hostname in the `Host` header of + the request is used. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -1146,12 +1402,12 @@ spec: the path of the incoming request. The modified path is then used to construct the `Location` header. When empty, the request path is used as-is. \n Support: - Extended \n " + Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies the value + description: ReplaceFullPath specifies the value with which to replace the full path of a request - during a rewrite or redirect. \n " + during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -1159,7 +1415,8 @@ spec: value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" with a prefix - match of \"/foo\" would be modified to \"/bar\". + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list @@ -1167,7 +1424,26 @@ spec: When specified, a trailing `/` is ignored. For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path - `/abcd` would not. \n " + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" maxLength: 1024 type: string type: @@ -1178,8 +1454,7 @@ spec: unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route - to `status: False`, with a Reason of `UnsupportedValue`. - \n " + to `status: False`, with a Reason of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -1187,11 +1462,43 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value - of the `Location` header in the response. When empty, - port (if specified) of the request is used. \n Support: - Extended" + of the `Location` header in the response. \n If + no port is specified, the redirect port MUST be + derived using the following rules: \n * If redirect + scheme is not-empty, the redirect port MUST be the + well-known port associated with the redirect scheme. + Specifically \"http\" to port 80 and \"https\" to + port 443. If the redirect scheme does not have a + well-known port, the listener port of the Gateway + SHOULD be used. * If redirect scheme is empty, the + redirect port MUST be the Gateway Listener port. + \n Implementations SHOULD NOT add the port number + in the 'Location' header in the following cases: + \n * A Location header that will use HTTP (whether + that is determined via the Listener protocol or + the Scheme field) _and_ use port 80. * A Location + header that will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) _and_ + use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -1200,7 +1507,10 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request is used. \n - Note that values may be added to this enum, implementations + Scheme redirects can affect the port of the redirect, + for more information, refer to the documentation + for the port field of this filter. \n Note that + values may be added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for @@ -1228,7 +1538,7 @@ spec: responseHeaderModifier: description: "ResponseHeaderModifier defines a schema for a filter that modifies response headers. \n Support: - Extended \n " + Extended" properties: add: description: "Add adds the given header(s) (name, @@ -1352,8 +1662,7 @@ spec: implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition - for the Route to `status: False`, with a Reason of `UnsupportedValue`. - \n " + for the Route to `status: False`, with a Reason of `UnsupportedValue`." enum: - RequestHeaderModifier - ResponseHeaderModifier @@ -1365,24 +1674,24 @@ spec: urlRewrite: description: "URLRewrite defines a schema for a filter that modifies a request during forwarding. \n Support: - Extended \n " + Extended" properties: hostname: description: "Hostname is the value to be used to replace the Host header value during forwarding. - \n Support: Extended \n " + \n Support: Extended" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string path: description: "Path defines a path rewrite. \n Support: - Extended \n " + Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies the value + description: ReplaceFullPath specifies the value with which to replace the full path of a request - during a rewrite or redirect. \n " + during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -1390,7 +1699,8 @@ spec: value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" with a prefix - match of \"/foo\" would be modified to \"/bar\". + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list @@ -1398,7 +1708,26 @@ spec: When specified, a trailing `/` is ignored. For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path - `/abcd` would not. \n " + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" maxLength: 1024 type: string type: @@ -1409,8 +1738,7 @@ spec: unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route - to `status: False`, with a Reason of `UnsupportedValue`. - \n " + to `status: False`, with a Reason of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -1418,12 +1746,89 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil if the + filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type != + ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type == + ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil if the + filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type != + ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for RequestMirror + filter.type + rule: '!(!has(self.requestMirror) && self.type == ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the filter.type + is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != ''RequestRedirect'')' + - message: filter.requestRedirect must be specified for RequestRedirect + filter.type + rule: '!(!has(self.requestRedirect) && self.type == ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for ExtensionRef + filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') && + self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') ? self.exists_one(f, + f.type == ''RequestRedirect'') : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' matches: default: - path: @@ -1445,19 +1850,21 @@ spec: request. \n Proxy or Load Balancer routing configuration generated from HTTPRoutes MUST prioritize matches based on the following criteria, continuing on ties. Across all rules specified on - applicable Routes, precedence must be given to the match with - the largest number of: \n * Characters in a matching path. - * Header matches. * Query param matches. \n If ties still - exist across multiple Routes, matching precedence MUST be - determined in order of the following criteria, continuing - on ties: \n * The oldest Route based on creation timestamp. - * The Route appearing first in alphabetical order by \"{namespace}/{name}\". - \n If ties still exist within an HTTPRoute, matching precedence - MUST be granted to the FIRST matching rule (in list order) - with a match meeting the above criteria. \n When no rules - matching a request have been successfully attached to the - parent a request is coming from, a HTTP 404 status code MUST - be returned." + applicable Routes, precedence must be given to the match having: + \n * \"Exact\" path match. * \"Prefix\" path match with largest + number of characters. * Method match. * Largest number of + header matches. * Largest number of query param matches. \n + Note: The precedence of RegularExpression path matches are + implementation-specific. \n If ties still exist across multiple + Routes, matching precedence MUST be determined in order of + the following criteria, continuing on ties: \n * The oldest + Route based on creation timestamp. * The Route appearing first + in alphabetical order by \"{namespace}/{name}\". \n If ties + still exist within an HTTPRoute, matching precedence MUST + be granted to the FIRST matching rule (in list order) with + a match meeting the above criteria. \n When no rules matching + a request have been successfully attached to the parent a + request is coming from, a HTTP 404 status code MUST be returned." items: description: "HTTPRouteMatch defines the predicate used to match requests to a given action. Multiple match types are @@ -1565,6 +1972,53 @@ spec: maxLength: 1024 type: string type: object + x-kubernetes-validations: + - message: value must be an absolute path and start with + '/' when type one of ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.startsWith(''/'') : true' + - message: must not contain '//' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''//'') : true' + - message: must not contain '/./' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/./'') : true' + - message: must not contain '/../' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/../'') : true' + - message: must not contain '%2f' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2f'') : true' + - message: must not contain '%2F' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2F'') : true' + - message: must not contain '#' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''#'') : true' + - message: must not end with '/..' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/..'') : true' + - message: must not end with '/.' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/.'') : true' + - message: type must be one of ['Exact', 'PathPrefix', + 'RegularExpression'] + rule: self.type == 'Exact' || self.type == 'PathPrefix' + || self.type == 'RegularExpression' + - message: must only contain valid characters (matching + ^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$) + for types ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.matches(r"""^(?:[-A-Za-z0-9/._~!$&''()*+,;=:@]|[%][0-9a-fA-F]{2})+$""") + : true' queryParams: description: "QueryParams specifies HTTP query parameter matchers. Multiple match values are ANDed together, @@ -1595,6 +2049,7 @@ spec: differences in the implementations." maxLength: 256 minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ type: string type: default: Exact @@ -1630,6 +2085,46 @@ spec: maxItems: 8 type: array type: object + x-kubernetes-validations: + - message: RequestRedirect filter must not be used together with + backendRefs + rule: '(has(self.backendRefs) && size(self.backendRefs) > 0) ? + (!has(self.filters) || self.filters.all(f, !has(f.requestRedirect))): + true' + - message: When using RequestRedirect filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + ? ((size(self.matches) != 1 || !has(self.matches[0].path) || + self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: When using URLRewrite filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' + - message: Within backendRefs, when using RequestRedirect filter + with path.replacePrefixMatch, exactly one PathPrefix match must + be specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + )) ? ((size(self.matches) != 1 || !has(self.matches[0].path) + || self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: Within backendRefs, When using URLRewrite filter with + path.replacePrefixMatch, exactly one PathPrefix match must be + specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) )) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' maxItems: 16 type: array type: object @@ -1778,9 +2273,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -1800,7 +2297,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -1816,8 +2322,12 @@ spec: a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener must match - both specified values. \n Implementations MAY choose to - support other parent resources. Implementations supporting + both specified values. \n When the parent resource is + a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are + specified, the name and port of the selected port must + match both specified values. \n Implementations MAY choose + to support other parent resources. Implementations supporting other types of parent resources MUST clearly document how/if Port is interpreted. \n For the purpose of status, an attachment is considered successful as long as the @@ -1828,7 +2338,7 @@ spec: the Route MUST be considered successfully attached. If no Gateway listeners accept attachment from this Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + \n Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -1839,8 +2349,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of @@ -1872,7 +2387,7 @@ spec: required: - spec type: object - served: true + served: false storage: false subresources: status: {} @@ -1907,13 +2422,17 @@ spec: description: Spec defines the desired state of HTTPRoute. properties: hostnames: - description: "Hostnames defines a set of hostname that should match - against the HTTP Host header to select a HTTPRoute to process the - request. This matches the RFC 1123 definition of a hostname with - 2 notable exceptions: \n 1. IPs are not allowed. 2. A hostname may - be prefixed with a wildcard label (`*.`). The wildcard label must - appear by itself as the first label. \n If a hostname is specified - by both the Listener and HTTPRoute, there must be at least one intersecting + description: "Hostnames defines a set of hostnames that should match + against the HTTP Host header to select a HTTPRoute used to process + the request. Implementations MUST ignore any port value specified + in the HTTP Host header while performing a match and (absent of + any applicable header modification configuration) MUST forward this + header unmodified to the backend. \n Valid values for Hostnames + are determined by RFC 1123 definition of a hostname with 2 notable + exceptions: \n 1. IPs are not allowed. 2. A hostname may be prefixed + with a wildcard label (`*.`). The wildcard label must appear by + itself as the first label. \n If a hostname is specified by both + the Listener and HTTPRoute, there must be at least one intersecting hostname for the HTTPRoute to be attached to the Listener. For example: \n * A Listener with `test.example.com` as the hostname matches HTTPRoutes that have either not specified any hostnames, or have @@ -1965,13 +2484,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -1979,16 +2506,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -2002,8 +2538,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -2023,7 +2562,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -2038,18 +2585,22 @@ spec: a Route must apply to a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY choose - to support other parent resources. Implementations supporting - other types of parent resources MUST clearly document how/if - Port is interpreted. \n For the purpose of status, an attachment - is considered successful as long as the parent resource accepts - it partially. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + must match both specified values. \n When the parent resource + is a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are specified, + the name and port of the selected port must match both specified + values. \n Implementations MAY choose to support other parent + resources. Implementations supporting other types of parent + resources MUST clearly document how/if Port is interpreted. + \n For the purpose of status, an attachment is considered + successful as long as the parent resource accepts it partially. + For example, Gateway listeners can restrict which Routes can + attach to them by Route kind, namespace, or hostname. If 1 + of 2 Gateway listeners accept attachment from the referencing + Route, the Route MUST be considered successfully attached. + If no Gateway listeners accept attachment from this Route, + the Route MUST be considered detached from the Gateway. \n + Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -2060,19 +2611,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -2111,7 +2666,8 @@ spec: \n For example, if two backends are specified with equal weights, and one is invalid, 50 percent of traffic must receive a 500. Implementations may choose how that 50 percent is determined. - \n Support: Core for Kubernetes Service \n Support: Implementation-specific + \n Support: Core for Kubernetes Service \n Support: Extended + for Kubernetes ServiceImport \n Support: Implementation-specific for any other resource \n Support for weight: Core" items: description: HTTPBackendRef defines how a HTTPRoute should @@ -2139,7 +2695,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times + within the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. @@ -2277,7 +2834,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from - that destination are ignored. \n Support: Extended" + that destination are ignored. \n This filter can + be used multiple times within the same rule. Note + that not all implementations will be able to support + mirroring to multiple backends. \n Support: Extended" properties: backendRef: description: "BackendRef references a resource @@ -2312,9 +2872,20 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. - For example "HTTPRoute" or "Service". - Defaults to "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to + CNAME DNS records that may live outside + of the cluster and as such are difficult + to reason about in terms of conformance. + They also may not be safe to forward to + (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with + a type other than ExternalName) \n Support: + Implementation-specific (Services with + type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -2328,7 +2899,8 @@ spec: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n Note that - when a namespace is specified, a ReferenceGrant + when a namespace different than the local + namespace is specified, a ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the ReferenceGrant @@ -2354,6 +2926,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind + == ''Service'') ? has(self.port) : true' required: - backendRef type: object @@ -2366,7 +2942,8 @@ spec: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. When empty, the hostname - of the request is used. \n Support: Core" + in the `Host` header of the request is used. + \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -2376,14 +2953,12 @@ spec: modify the path of the incoming request. The modified path is then used to construct the `Location` header. When empty, the request - path is used as-is. \n Support: Extended \n - " + path is used as-is. \n Support: Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies - the value with which to replace the full - path of a request during a rewrite or - redirect. \n " + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -2391,8 +2966,9 @@ spec: the value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" - with a prefix match of \"/foo\" would - be modified to \"/bar\". \n Note that + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list of labels @@ -2401,7 +2977,28 @@ spec: For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path `/abcd` would not. - \n " + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" maxLength: 1024 type: string type: @@ -2414,7 +3011,7 @@ spec: values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -2422,11 +3019,45 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value of the `Location` header in the - response. When empty, port (if specified) - of the request is used. \n Support: Extended" + response. \n If no port is specified, the + redirect port MUST be derived using the following + rules: \n * If redirect scheme is not-empty, + the redirect port MUST be the well-known port + associated with the redirect scheme. Specifically + \"http\" to port 80 and \"https\" to port + 443. If the redirect scheme does not have + a well-known port, the listener port of the + Gateway SHOULD be used. * If redirect scheme + is empty, the redirect port MUST be the Gateway + Listener port. \n Implementations SHOULD NOT + add the port number in the 'Location' header + in the following cases: \n * A Location header + that will use HTTP (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 80. * A Location header that + will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -2435,8 +3066,11 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request - is used. \n Note that values may be added - to this enum, implementations must ensure + is used. \n Scheme redirects can affect the + port of the redirect, for more information, + refer to the documentation for the port field + of this filter. \n Note that values may be + added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition @@ -2465,7 +3099,7 @@ spec: responseHeaderModifier: description: "ResponseHeaderModifier defines a schema for a filter that modifies response headers. \n - Support: Extended \n " + Support: Extended" properties: add: description: "Add adds the given header(s) (name, @@ -2600,7 +3234,7 @@ spec: a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - RequestHeaderModifier - ResponseHeaderModifier @@ -2612,25 +3246,24 @@ spec: urlRewrite: description: "URLRewrite defines a schema for a filter that modifies a request during forwarding. - \n Support: Extended \n " + \n Support: Extended" properties: hostname: description: "Hostname is the value to be used to replace the Host header value during forwarding. - \n Support: Extended \n " + \n Support: Extended" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string path: description: "Path defines a path rewrite. \n - Support: Extended \n " + Support: Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies - the value with which to replace the full - path of a request during a rewrite or - redirect. \n " + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -2638,8 +3271,9 @@ spec: the value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" - with a prefix match of \"/foo\" would - be modified to \"/bar\". \n Note that + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list of labels @@ -2648,7 +3282,28 @@ spec: For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path `/abcd` would not. - \n " + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" maxLength: 1024 type: string type: @@ -2661,7 +3316,7 @@ spec: values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -2669,12 +3324,97 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil + if the filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type + != ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type + == ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil + if the filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type + != ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for + RequestMirror filter.type + rule: '!(!has(self.requestMirror) && self.type == + ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the + filter.type is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != + ''RequestRedirect'')' + - message: filter.requestRedirect must be specified + for RequestRedirect filter.type + rule: '!(!has(self.requestRedirect) && self.type == + ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for + ExtensionRef filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') + ? self.exists_one(f, f.type == ''RequestRedirect'') + : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' group: default: "" description: Group is the group of the referent. For example, @@ -2685,9 +3425,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -2700,11 +3448,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -2742,6 +3490,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 type: array filters: @@ -2754,14 +3506,15 @@ spec: all implementations. - Implementers are encouraged to support extended filters. - Implementation-specific custom filters have no API guarantees across implementations. \n Specifying - a core filter multiple times has unspecified or implementation-specific - conformance. \n All filters are expected to be compatible - with each other except for the URLRewrite and RequestRedirect + the same filter multiple times is not supported unless explicitly + indicated in the filter. \n All filters are expected to be + compatible with each other except for the URLRewrite and RequestRedirect filters, which may not be combined. If an implementation can not support other combinations of filters, they must clearly - document that limitation. In all cases where incompatible - or unsupported filters are specified, implementations MUST - add a warning condition to status. \n Support: Core" + document that limitation. In cases where incompatible or unsupported + filters are specified and cause the `Accepted` condition to + be set to status `False`, implementations may use the `IncompatibleFilters` + reason to specify this configuration error. \n Support: Core" items: description: HTTPRouteFilter defines processing steps that must be completed during the request or response lifecycle. @@ -2777,7 +3530,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times within + the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. For @@ -2907,7 +3661,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from that destination are - ignored. \n Support: Extended" + ignored. \n This filter can be used multiple times within + the same rule. Note that not all implementations will + be able to support mirroring to multiple backends. \n + Support: Extended" properties: backendRef: description: "BackendRef references a resource where @@ -2940,9 +3697,18 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For - example "HTTPRoute" or "Service". Defaults to - "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to CNAME + DNS records that may live outside of the cluster + and as such are difficult to reason about in + terms of conformance. They also may not be safe + to forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with a + type other than ExternalName) \n Support: Implementation-specific + (Services with type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -2955,11 +3721,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace - is inferred. \n Note that when a namespace is - specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's - owner to accept the reference. See the ReferenceGrant - documentation for details. \n Support: Core" + is inferred. \n Note that when a namespace different + than the local namespace is specified, a ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept the + reference. See the ReferenceGrant documentation + for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -2979,6 +3746,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' required: - backendRef type: object @@ -2990,8 +3761,8 @@ spec: hostname: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. - When empty, the hostname of the request is used. - \n Support: Core" + When empty, the hostname in the `Host` header of + the request is used. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -3001,12 +3772,12 @@ spec: the path of the incoming request. The modified path is then used to construct the `Location` header. When empty, the request path is used as-is. \n Support: - Extended \n " + Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies the value + description: ReplaceFullPath specifies the value with which to replace the full path of a request - during a rewrite or redirect. \n " + during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -3014,7 +3785,8 @@ spec: value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" with a prefix - match of \"/foo\" would be modified to \"/bar\". + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list @@ -3022,7 +3794,26 @@ spec: When specified, a trailing `/` is ignored. For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path - `/abcd` would not. \n " + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" maxLength: 1024 type: string type: @@ -3033,8 +3824,7 @@ spec: unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route - to `status: False`, with a Reason of `UnsupportedValue`. - \n " + to `status: False`, with a Reason of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -3042,11 +3832,43 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value - of the `Location` header in the response. When empty, - port (if specified) of the request is used. \n Support: - Extended" + of the `Location` header in the response. \n If + no port is specified, the redirect port MUST be + derived using the following rules: \n * If redirect + scheme is not-empty, the redirect port MUST be the + well-known port associated with the redirect scheme. + Specifically \"http\" to port 80 and \"https\" to + port 443. If the redirect scheme does not have a + well-known port, the listener port of the Gateway + SHOULD be used. * If redirect scheme is empty, the + redirect port MUST be the Gateway Listener port. + \n Implementations SHOULD NOT add the port number + in the 'Location' header in the following cases: + \n * A Location header that will use HTTP (whether + that is determined via the Listener protocol or + the Scheme field) _and_ use port 80. * A Location + header that will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) _and_ + use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -3055,7 +3877,10 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request is used. \n - Note that values may be added to this enum, implementations + Scheme redirects can affect the port of the redirect, + for more information, refer to the documentation + for the port field of this filter. \n Note that + values may be added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for @@ -3083,7 +3908,7 @@ spec: responseHeaderModifier: description: "ResponseHeaderModifier defines a schema for a filter that modifies response headers. \n Support: - Extended \n " + Extended" properties: add: description: "Add adds the given header(s) (name, @@ -3207,8 +4032,7 @@ spec: implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition - for the Route to `status: False`, with a Reason of `UnsupportedValue`. - \n " + for the Route to `status: False`, with a Reason of `UnsupportedValue`." enum: - RequestHeaderModifier - ResponseHeaderModifier @@ -3220,24 +4044,24 @@ spec: urlRewrite: description: "URLRewrite defines a schema for a filter that modifies a request during forwarding. \n Support: - Extended \n " + Extended" properties: hostname: description: "Hostname is the value to be used to replace the Host header value during forwarding. - \n Support: Extended \n " + \n Support: Extended" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string path: description: "Path defines a path rewrite. \n Support: - Extended \n " + Extended" properties: replaceFullPath: - description: "ReplaceFullPath specifies the value + description: ReplaceFullPath specifies the value with which to replace the full path of a request - during a rewrite or redirect. \n " + during a rewrite or redirect. maxLength: 1024 type: string replacePrefixMatch: @@ -3245,7 +4069,8 @@ spec: value with which to replace the prefix match of a request during a rewrite or redirect. For example, a request to \"/foo/bar\" with a prefix - match of \"/foo\" would be modified to \"/bar\". + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". \n Note that this matches the behavior of the PathPrefix match type. This matches full path elements. A path element refers to the list @@ -3253,7 +4078,26 @@ spec: When specified, a trailing `/` is ignored. For example, the paths `/abc`, `/abc/`, and `/abc/def` would all match the prefix `/abc`, but the path - `/abcd` would not. \n " + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" maxLength: 1024 type: string type: @@ -3264,8 +4108,7 @@ spec: unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route - to `status: False`, with a Reason of `UnsupportedValue`. - \n " + to `status: False`, with a Reason of `UnsupportedValue`." enum: - ReplaceFullPath - ReplacePrefixMatch @@ -3273,12 +4116,89 @@ spec: required: - type type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil if the + filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type != + ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type == + ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil if the + filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type != + ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for RequestMirror + filter.type + rule: '!(!has(self.requestMirror) && self.type == ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the filter.type + is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != ''RequestRedirect'')' + - message: filter.requestRedirect must be specified for RequestRedirect + filter.type + rule: '!(!has(self.requestRedirect) && self.type == ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for ExtensionRef + filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') && + self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') ? self.exists_one(f, + f.type == ''RequestRedirect'') : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' matches: default: - path: @@ -3300,19 +4220,21 @@ spec: request. \n Proxy or Load Balancer routing configuration generated from HTTPRoutes MUST prioritize matches based on the following criteria, continuing on ties. Across all rules specified on - applicable Routes, precedence must be given to the match with - the largest number of: \n * Characters in a matching path. - * Header matches. * Query param matches. \n If ties still - exist across multiple Routes, matching precedence MUST be - determined in order of the following criteria, continuing - on ties: \n * The oldest Route based on creation timestamp. - * The Route appearing first in alphabetical order by \"{namespace}/{name}\". - \n If ties still exist within an HTTPRoute, matching precedence - MUST be granted to the FIRST matching rule (in list order) - with a match meeting the above criteria. \n When no rules - matching a request have been successfully attached to the - parent a request is coming from, a HTTP 404 status code MUST - be returned." + applicable Routes, precedence must be given to the match having: + \n * \"Exact\" path match. * \"Prefix\" path match with largest + number of characters. * Method match. * Largest number of + header matches. * Largest number of query param matches. \n + Note: The precedence of RegularExpression path matches are + implementation-specific. \n If ties still exist across multiple + Routes, matching precedence MUST be determined in order of + the following criteria, continuing on ties: \n * The oldest + Route based on creation timestamp. * The Route appearing first + in alphabetical order by \"{namespace}/{name}\". \n If ties + still exist within an HTTPRoute, matching precedence MUST + be granted to the FIRST matching rule (in list order) with + a match meeting the above criteria. \n When no rules matching + a request have been successfully attached to the parent a + request is coming from, a HTTP 404 status code MUST be returned." items: description: "HTTPRouteMatch defines the predicate used to match requests to a given action. Multiple match types are @@ -3420,6 +4342,53 @@ spec: maxLength: 1024 type: string type: object + x-kubernetes-validations: + - message: value must be an absolute path and start with + '/' when type one of ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.startsWith(''/'') : true' + - message: must not contain '//' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''//'') : true' + - message: must not contain '/./' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/./'') : true' + - message: must not contain '/../' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/../'') : true' + - message: must not contain '%2f' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2f'') : true' + - message: must not contain '%2F' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2F'') : true' + - message: must not contain '#' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''#'') : true' + - message: must not end with '/..' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/..'') : true' + - message: must not end with '/.' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/.'') : true' + - message: type must be one of ['Exact', 'PathPrefix', + 'RegularExpression'] + rule: self.type == 'Exact' || self.type == 'PathPrefix' + || self.type == 'RegularExpression' + - message: must only contain valid characters (matching + ^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$) + for types ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.matches(r"""^(?:[-A-Za-z0-9/._~!$&''()*+,;=:@]|[%][0-9a-fA-F]{2})+$""") + : true' queryParams: description: "QueryParams specifies HTTP query parameter matchers. Multiple match values are ANDed together, @@ -3450,6 +4419,7 @@ spec: differences in the implementations." maxLength: 256 minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ type: string type: default: Exact @@ -3485,6 +4455,46 @@ spec: maxItems: 8 type: array type: object + x-kubernetes-validations: + - message: RequestRedirect filter must not be used together with + backendRefs + rule: '(has(self.backendRefs) && size(self.backendRefs) > 0) ? + (!has(self.filters) || self.filters.all(f, !has(f.requestRedirect))): + true' + - message: When using RequestRedirect filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + ? ((size(self.matches) != 1 || !has(self.matches[0].path) || + self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: When using URLRewrite filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' + - message: Within backendRefs, when using RequestRedirect filter + with path.replacePrefixMatch, exactly one PathPrefix match must + be specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + )) ? ((size(self.matches) != 1 || !has(self.matches[0].path) + || self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: Within backendRefs, When using URLRewrite filter with + path.replacePrefixMatch, exactly one PathPrefix match must be + specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) )) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' maxItems: 16 type: array type: object @@ -3633,9 +4643,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -3655,7 +4667,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -3671,8 +4692,12 @@ spec: a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener must match - both specified values. \n Implementations MAY choose to - support other parent resources. Implementations supporting + both specified values. \n When the parent resource is + a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are + specified, the name and port of the selected port must + match both specified values. \n Implementations MAY choose + to support other parent resources. Implementations supporting other types of parent resources MUST clearly document how/if Port is interpreted. \n For the purpose of status, an attachment is considered successful as long as the @@ -3683,7 +4708,7 @@ spec: the Route MUST be considered successfully attached. If no Gateway listeners accept attachment from this Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + \n Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -3694,8 +4719,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of diff --git a/config/crd/experimental/gateway.networking.k8s.io_referencegrants.yaml b/config/crd/experimental/gateway.networking.k8s.io_referencegrants.yaml index 9bf4e7e34e..b52b79d69b 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_referencegrants.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_referencegrants.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: referencegrants.gateway.networking.k8s.io @@ -24,6 +24,9 @@ spec: - jsonPath: .metadata.creationTimestamp name: Age type: date + deprecated: true + deprecationWarning: The v1alpha2 version of ReferenceGrant has been deprecated + and will be removed in a future release of the API. Please upgrade to v1beta1. name: v1alpha2 schema: openAPIV3Schema: @@ -32,13 +35,17 @@ spec: namespace as the policy. \n Each ReferenceGrant can be used to represent a unique trust relationship. Additional Reference Grants can be used to add to the set of trusted sources of inbound references for the namespace - they are defined within. \n All cross-namespace references in Gateway API - (with the exception of cross-namespace Gateway-route attachment) require - a ReferenceGrant. \n ReferenceGrant is a form of runtime verification allowing - users to assert which cross-namespace object references are permitted. Implementations - that support ReferenceGrant MUST NOT permit cross-namespace references which - have no grant, and MUST respond to the removal of a grant by revoking the - access that the grant allowed. \n Support: Core" + they are defined within. \n A ReferenceGrant is required for all cross-namespace + references in Gateway API (with the exception of cross-namespace Route-Gateway + attachment, which is governed by the AllowedRoutes configuration on the + Gateway, and cross-namespace Service ParentRefs on a \"consumer\" mesh Route, + which defines routing rules applicable only to workloads in the Route namespace). + ReferenceGrants allowing a reference from a Route to a Service are only + applicable to BackendRefs. \n ReferenceGrant is a form of runtime verification + allowing users to assert which cross-namespace object references are permitted. + Implementations that support ReferenceGrant MUST NOT permit cross-namespace + references which have no grant, and MUST respond to the removal of a grant + by revoking the access that the grant allowed. \n Support: Core" properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation @@ -143,7 +150,7 @@ spec: type: object type: object served: true - storage: true + storage: false subresources: {} - additionalPrinterColumns: - jsonPath: .metadata.creationTimestamp @@ -268,7 +275,7 @@ spec: type: object type: object served: true - storage: false + storage: true subresources: {} status: acceptedNames: diff --git a/config/crd/experimental/gateway.networking.k8s.io_tcproutes.yaml b/config/crd/experimental/gateway.networking.k8s.io_tcproutes.yaml index 448be8fef1..0306e917d4 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_tcproutes.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_tcproutes.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: tcproutes.gateway.networking.k8s.io @@ -49,13 +49,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -63,16 +71,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -86,8 +103,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -107,7 +127,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -122,18 +150,22 @@ spec: a Route must apply to a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY choose - to support other parent resources. Implementations supporting - other types of parent resources MUST clearly document how/if - Port is interpreted. \n For the purpose of status, an attachment - is considered successful as long as the parent resource accepts - it partially. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + must match both specified values. \n When the parent resource + is a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are specified, + the name and port of the selected port must match both specified + values. \n Implementations MAY choose to support other parent + resources. Implementations supporting other types of parent + resources MUST clearly document how/if Port is interpreted. + \n For the purpose of status, an attachment is considered + successful as long as the parent resource accepts it partially. + For example, Gateway listeners can restrict which Routes can + attach to them by Route kind, namespace, or hostname. If 1 + of 2 Gateway listeners accept attachment from the referencing + Route, the Route MUST be considered successfully attached. + If no Gateway listeners accept attachment from this Route, + the Route MUST be considered detached from the Gateway. \n + Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -144,19 +176,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -179,15 +215,16 @@ spec: attempts to this backend. Connection rejections must respect weight; if an invalid backend is requested to have 80% of connections, then 80% of connections must be rejected instead. - \n Support: Core for Kubernetes Service \n Support: Implementation-specific + \n Support: Core for Kubernetes Service \n Support: Extended + for Kubernetes ServiceImport \n Support: Implementation-specific for any other resource \n Support for weight: Extended" items: description: "BackendRef defines how a Route should forward a request to a Kubernetes resource. \n Note that when a - namespace is specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's owner - to accept the reference. See the ReferenceGrant documentation - for details." + namespace different than the local namespace is specified, + a ReferenceGrant object is required in the referent namespace + to allow that namespace's owner to accept the reference. + See the ReferenceGrant documentation for details." properties: group: default: "" @@ -199,9 +236,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -214,11 +259,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -256,6 +301,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 minItems: 1 type: array @@ -411,9 +460,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -433,7 +484,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -449,8 +509,12 @@ spec: a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener must match - both specified values. \n Implementations MAY choose to - support other parent resources. Implementations supporting + both specified values. \n When the parent resource is + a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are + specified, the name and port of the selected port must + match both specified values. \n Implementations MAY choose + to support other parent resources. Implementations supporting other types of parent resources MUST clearly document how/if Port is interpreted. \n For the purpose of status, an attachment is considered successful as long as the @@ -461,7 +525,7 @@ spec: the Route MUST be considered successfully attached. If no Gateway listeners accept attachment from this Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + \n Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -472,8 +536,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of diff --git a/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml b/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml index 350ae9af2a..d4e8cda49d 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: tlsroutes.gateway.networking.k8s.io @@ -95,13 +95,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -109,16 +117,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -132,8 +149,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -153,7 +173,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -168,18 +196,22 @@ spec: a Route must apply to a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY choose - to support other parent resources. Implementations supporting - other types of parent resources MUST clearly document how/if - Port is interpreted. \n For the purpose of status, an attachment - is considered successful as long as the parent resource accepts - it partially. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + must match both specified values. \n When the parent resource + is a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are specified, + the name and port of the selected port must match both specified + values. \n Implementations MAY choose to support other parent + resources. Implementations supporting other types of parent + resources MUST clearly document how/if Port is interpreted. + \n For the purpose of status, an attachment is considered + successful as long as the parent resource accepts it partially. + For example, Gateway listeners can restrict which Routes can + attach to them by Route kind, namespace, or hostname. If 1 + of 2 Gateway listeners accept attachment from the referencing + Route, the Route MUST be considered successfully attached. + If no Gateway listeners accept attachment from this Route, + the Route MUST be considered detached from the Gateway. \n + Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -190,19 +222,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -228,15 +264,16 @@ spec: code. Request rejections must respect weight; if an invalid backend is requested to have 80% of requests, then 80% of requests must be rejected instead. \n Support: Core for Kubernetes - Service \n Support: Implementation-specific for any other - resource \n Support for weight: Extended" + Service \n Support: Extended for Kubernetes ServiceImport + \n Support: Implementation-specific for any other resource + \n Support for weight: Extended" items: description: "BackendRef defines how a Route should forward a request to a Kubernetes resource. \n Note that when a - namespace is specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's owner - to accept the reference. See the ReferenceGrant documentation - for details." + namespace different than the local namespace is specified, + a ReferenceGrant object is required in the referent namespace + to allow that namespace's owner to accept the reference. + See the ReferenceGrant documentation for details." properties: group: default: "" @@ -248,9 +285,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -263,11 +308,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -305,6 +350,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 minItems: 1 type: array @@ -460,9 +509,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -482,7 +533,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -498,8 +558,12 @@ spec: a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener must match - both specified values. \n Implementations MAY choose to - support other parent resources. Implementations supporting + both specified values. \n When the parent resource is + a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are + specified, the name and port of the selected port must + match both specified values. \n Implementations MAY choose + to support other parent resources. Implementations supporting other types of parent resources MUST clearly document how/if Port is interpreted. \n For the purpose of status, an attachment is considered successful as long as the @@ -510,7 +574,7 @@ spec: the Route MUST be considered successfully attached. If no Gateway listeners accept attachment from this Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + \n Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -521,8 +585,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of diff --git a/config/crd/experimental/gateway.networking.k8s.io_udproutes.yaml b/config/crd/experimental/gateway.networking.k8s.io_udproutes.yaml index c4dcba99b7..8a2e96d071 100644 --- a/config/crd/experimental/gateway.networking.k8s.io_udproutes.yaml +++ b/config/crd/experimental/gateway.networking.k8s.io_udproutes.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: experimental creationTimestamp: null name: udproutes.gateway.networking.k8s.io @@ -49,13 +49,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -63,16 +71,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -86,8 +103,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -107,7 +127,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -122,18 +150,22 @@ spec: a Route must apply to a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY choose - to support other parent resources. Implementations supporting - other types of parent resources MUST clearly document how/if - Port is interpreted. \n For the purpose of status, an attachment - is considered successful as long as the parent resource accepts - it partially. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + must match both specified values. \n When the parent resource + is a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are specified, + the name and port of the selected port must match both specified + values. \n Implementations MAY choose to support other parent + resources. Implementations supporting other types of parent + resources MUST clearly document how/if Port is interpreted. + \n For the purpose of status, an attachment is considered + successful as long as the parent resource accepts it partially. + For example, Gateway listeners can restrict which Routes can + attach to them by Route kind, namespace, or hostname. If 1 + of 2 Gateway listeners accept attachment from the referencing + Route, the Route MUST be considered successfully attached. + If no Gateway listeners accept attachment from this Route, + the Route MUST be considered detached from the Gateway. \n + Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -144,19 +176,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -179,15 +215,16 @@ spec: attempts to this backend. Packet drops must respect weight; if an invalid backend is requested to have 80% of the packets, then 80% of packets must be dropped instead. \n Support: Core - for Kubernetes Service Support: Implementation-specific for - any other resource \n Support for weight: Extended" + for Kubernetes Service \n Support: Extended for Kubernetes + ServiceImport \n Support: Implementation-specific for any + other resource \n Support for weight: Extended" items: description: "BackendRef defines how a Route should forward a request to a Kubernetes resource. \n Note that when a - namespace is specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's owner - to accept the reference. See the ReferenceGrant documentation - for details." + namespace different than the local namespace is specified, + a ReferenceGrant object is required in the referent namespace + to allow that namespace's owner to accept the reference. + See the ReferenceGrant documentation for details." properties: group: default: "" @@ -199,9 +236,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -214,11 +259,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -256,6 +301,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 minItems: 1 type: array @@ -411,9 +460,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -433,7 +484,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -449,8 +509,12 @@ spec: a specific port as opposed to a listener(s) whose port(s) may be changed. When both Port and SectionName are specified, the name and port of the selected listener must match - both specified values. \n Implementations MAY choose to - support other parent resources. Implementations supporting + both specified values. \n When the parent resource is + a Service, this targets a specific port in the Service + spec. When both Port (experimental) and SectionName are + specified, the name and port of the selected port must + match both specified values. \n Implementations MAY choose + to support other parent resources. Implementations supporting other types of parent resources MUST clearly document how/if Port is interpreted. \n For the purpose of status, an attachment is considered successful as long as the @@ -461,7 +525,7 @@ spec: the Route MUST be considered successfully attached. If no Gateway listeners accept attachment from this Route, the Route MUST be considered detached from the Gateway. - \n Support: Extended \n " + \n Support: Extended \n " format: int32 maximum: 65535 minimum: 1 @@ -472,8 +536,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of diff --git a/config/crd/kustomization.yaml b/config/crd/kustomization.yaml index be0beb328c..ac7ec7971b 100644 --- a/config/crd/kustomization.yaml +++ b/config/crd/kustomization.yaml @@ -2,3 +2,4 @@ resources: - standard/gateway.networking.k8s.io_gatewayclasses.yaml - standard/gateway.networking.k8s.io_gateways.yaml - standard/gateway.networking.k8s.io_httproutes.yaml +- standard/gateway.networking.k8s.io_referencegrants.yaml diff --git a/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml b/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml index a11f8b05ac..dc54d98882 100644 --- a/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml +++ b/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: standard creationTimestamp: null name: gatewayclasses.gateway.networking.k8s.io @@ -49,7 +49,7 @@ spec: to GatewayClass or associated parameters. If implementations choose to propagate GatewayClass changes to existing Gateways, that MUST be clearly documented by the implementation. \n Whenever one or more Gateways are using a GatewayClass, - implementations MUST add the `gateway-exists-finalizer.gateway.networking.k8s.io` + implementations SHOULD add the `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer on the associated GatewayClass. This ensures that a GatewayClass associated with a Gateway is not deleted while in use. \n GatewayClass is a Cluster level resource." @@ -78,6 +78,9 @@ spec: minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$ type: string + x-kubernetes-validations: + - message: Value is immutable + rule: self == oldSelf description: description: Description helps describe a GatewayClass with more details. maxLength: 64 @@ -132,7 +135,9 @@ spec: reason: Waiting status: Unknown type: Accepted - description: Status defines the current state of GatewayClass. + description: "Status defines the current state of GatewayClass. \n Implementations + MUST populate status on all GatewayClass resources which specify their + controller name." properties: conditions: default: @@ -219,7 +224,7 @@ spec: required: - spec type: object - served: true + served: false storage: false subresources: status: {} @@ -249,7 +254,7 @@ spec: to GatewayClass or associated parameters. If implementations choose to propagate GatewayClass changes to existing Gateways, that MUST be clearly documented by the implementation. \n Whenever one or more Gateways are using a GatewayClass, - implementations MUST add the `gateway-exists-finalizer.gateway.networking.k8s.io` + implementations SHOULD add the `gateway-exists-finalizer.gateway.networking.k8s.io` finalizer on the associated GatewayClass. This ensures that a GatewayClass associated with a Gateway is not deleted while in use. \n GatewayClass is a Cluster level resource." @@ -278,6 +283,9 @@ spec: minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-._~%!$&'()*+,;=:]+$ type: string + x-kubernetes-validations: + - message: Value is immutable + rule: self == oldSelf description: description: Description helps describe a GatewayClass with more details. maxLength: 64 @@ -332,7 +340,9 @@ spec: reason: Waiting status: Unknown type: Accepted - description: Status defines the current state of GatewayClass. + description: "Status defines the current state of GatewayClass. \n Implementations + MUST populate status on all GatewayClass resources which specify their + controller name." properties: conditions: default: diff --git a/config/crd/standard/gateway.networking.k8s.io_gateways.yaml b/config/crd/standard/gateway.networking.k8s.io_gateways.yaml index 488d230167..70d81f576a 100644 --- a/config/crd/standard/gateway.networking.k8s.io_gateways.yaml +++ b/config/crd/standard/gateway.networking.k8s.io_gateways.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: standard creationTimestamp: null name: gateways.gateway.networking.k8s.io @@ -73,10 +73,24 @@ spec: manner, assigning an appropriate set of Addresses. \n The implementation MUST bind all Listeners to every GatewayAddress that it assigns to the Gateway and add a corresponding entry in GatewayStatus.Addresses. - \n Support: Extended" + \n Support: Extended \n " items: description: GatewayAddress describes an address that can be bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -95,8 +109,20 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array + x-kubernetes-validations: + - message: IPAddress values must be unique + rule: 'self.all(a1, a1.type == ''IPAddress'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' + - message: Hostname values must be unique + rule: 'self.all(a1, a1.type == ''Hostname'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' gatewayClassName: description: GatewayClassName used for this Gateway. This is the name of a GatewayClass resource. @@ -108,27 +134,34 @@ spec: logical endpoints that are bound on this Gateway's addresses. At least one Listener MUST be specified. \n Each listener in a Gateway must have a unique combination of Hostname, Port, and Protocol. - \n An implementation MAY group Listeners by Port and then collapse - each group of Listeners into a single Listener if the implementation - determines that the Listeners in the group are \"compatible\". An - implementation MAY also group together and collapse compatible Listeners - belonging to different Gateways. \n For example, an implementation - might consider Listeners to be compatible with each other if all - of the following conditions are met: \n 1. Either each Listener - within the group specifies the \"HTTP\" Protocol or each Listener - within the group specifies either the \"HTTPS\" or \"TLS\" Protocol. - \n 2. Each Listener within the group specifies a Hostname that is - unique within the group. \n 3. As a special case, one Listener within - a group may omit Hostname, in which case this Listener matches when - no other Listener matches. \n If the implementation does collapse - compatible Listeners, the hostname provided in the incoming client - request MUST be matched to a Listener to find the correct set of - Routes. The incoming hostname MUST be matched using the Hostname - field for each Listener in order of most to least specific. That - is, exact matches must be processed before wildcard matches. \n - If this field specifies multiple Listeners that have the same Port - value but are not compatible, the implementation must raise a \"Conflicted\" - condition in the Listener status. \n Support: Core" + \n Within the HTTP Conformance Profile, the below combinations of + port and protocol are considered Core and MUST be supported: \n + 1. Port: 80, Protocol: HTTP 2. Port: 443, Protocol: HTTPS \n Within + the TLS Conformance Profile, the below combinations of port and + protocol are considered Core and MUST be supported: \n 1. Port: + 443, Protocol: TLS \n Port and protocol combinations not listed + above are considered Extended. \n An implementation MAY group Listeners + by Port and then collapse each group of Listeners into a single + Listener if the implementation determines that the Listeners in + the group are \"compatible\". An implementation MAY also group together + and collapse compatible Listeners belonging to different Gateways. + \n For example, an implementation might consider Listeners to be + compatible with each other if all of the following conditions are + met: \n 1. Either each Listener within the group specifies the \"HTTP\" + Protocol or each Listener within the group specifies either the + \"HTTPS\" or \"TLS\" Protocol. \n 2. Each Listener within the group + specifies a Hostname that is unique within the group. \n 3. As a + special case, one Listener within a group may omit Hostname, in + which case this Listener matches when no other Listener matches. + \n If the implementation does collapse compatible Listeners, the + hostname provided in the incoming client request MUST be matched + to a Listener to find the correct set of Routes. The incoming hostname + MUST be matched using the Hostname field for each Listener in order + of most to least specific. That is, exact matches must be processed + before wildcard matches. \n If this field specifies multiple Listeners + that have the same Port value but are not compatible, the implementation + must raise a \"Conflicted\" condition in the Listener status. \n + Support: Core" items: description: Listener embodies the concept of a logical endpoint where a Gateway accepts network connections. @@ -201,7 +234,7 @@ spec: from: default: Same description: "From indicates where Routes will be selected - for this Gateway. Possible values are: * All: Routes + for this Gateway. Possible values are: \n * All: Routes in all namespaces may be used by this Gateway. * Selector: Routes in namespaces selected by the selector may be used by this Gateway. * Same: Only Routes in the @@ -379,11 +412,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. - \n Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to - allow that namespace's owner to accept the reference. - See the ReferenceGrant documentation for details. - \n Support: Core" + \n Note that when a namespace different than the + local namespace is specified, a ReferenceGrant object + is required in the referent namespace to allow that + namespace's owner to accept the reference. See the + ReferenceGrant documentation for details. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -431,6 +465,11 @@ spec: maxProperties: 16 type: object type: object + x-kubernetes-validations: + - message: certificateRefs must be specified when TLSModeType + is Terminate + rule: 'self.mode == ''Terminate'' ? size(self.certificateRefs) + > 0 : true' required: - name - port @@ -442,6 +481,24 @@ spec: x-kubernetes-list-map-keys: - name x-kubernetes-list-type: map + x-kubernetes-validations: + - message: tls must be specified for protocols ['HTTPS', 'TLS'] + rule: 'self.all(l, l.protocol in [''HTTPS'', ''TLS''] ? has(l.tls) + : true)' + - message: tls must not be specified for protocols ['HTTP', 'TCP', + 'UDP'] + rule: 'self.all(l, l.protocol in [''HTTP'', ''TCP'', ''UDP''] ? + !has(l.tls) : true)' + - message: hostname must not be specified for protocols ['TCP', 'UDP'] + rule: 'self.all(l, l.protocol in [''TCP'', ''UDP''] ? (!has(l.hostname) + || l.hostname == '''') : true)' + - message: Listener name must be unique within the Gateway + rule: self.all(l1, self.exists_one(l2, l1.name == l2.name)) + - message: Combination of port, protocol and hostname must be unique + for each listener + rule: 'self.all(l1, self.exists_one(l2, l1.port == l2.port && l1.protocol + == l2.protocol && (has(l1.hostname) && has(l2.hostname) ? l1.hostname + == l2.hostname : true)))' required: - gatewayClassName - listeners @@ -451,19 +508,38 @@ spec: conditions: - lastTransitionTime: "1970-01-01T00:00:00Z" message: Waiting for controller - reason: NotReconciled + reason: Pending status: Unknown type: Accepted + - lastTransitionTime: "1970-01-01T00:00:00Z" + message: Waiting for controller + reason: Pending + status: Unknown + type: Programmed description: Status defines the current state of Gateway. properties: addresses: - description: Addresses lists the IP addresses that have actually been - bound to the Gateway. These addresses may differ from the addresses + description: "Addresses lists the IP addresses that have actually + been bound to the Gateway. These addresses may differ from the addresses in the Spec, e.g. if the Gateway automatically assigns an address - from a reserved pool. + from a reserved pool. \n " items: - description: GatewayAddress describes an address that can be bound + description: GatewayStatusAddress describes an address that is bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -482,6 +558,11 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array conditions: @@ -501,7 +582,7 @@ spec: the `GatewayConditionType` and `GatewayConditionReason` constants so that operators and tools can converge on a common vocabulary to describe Gateway state. \n Known condition types are: \n * \"Accepted\" - * \"Ready\"" + * \"Programmed\" * \"Ready\"" items: description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct @@ -580,8 +661,8 @@ spec: description: ListenerStatus is the status associated with a Listener. properties: attachedRoutes: - description: AttachedRoutes represents the total number of Routes - that have been successfully attached to this Listener. + description: AttachedRoutes represents the total number of accepted + Routes that have been successfully attached to this Listener. format: int32 type: integer conditions: @@ -716,7 +797,7 @@ spec: required: - spec type: object - served: true + served: false storage: false subresources: status: {} @@ -770,10 +851,24 @@ spec: manner, assigning an appropriate set of Addresses. \n The implementation MUST bind all Listeners to every GatewayAddress that it assigns to the Gateway and add a corresponding entry in GatewayStatus.Addresses. - \n Support: Extended" + \n Support: Extended \n " items: description: GatewayAddress describes an address that can be bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -792,8 +887,20 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array + x-kubernetes-validations: + - message: IPAddress values must be unique + rule: 'self.all(a1, a1.type == ''IPAddress'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' + - message: Hostname values must be unique + rule: 'self.all(a1, a1.type == ''Hostname'' ? self.exists_one(a2, + a2.type == a1.type && a2.value == a1.value) : true )' gatewayClassName: description: GatewayClassName used for this Gateway. This is the name of a GatewayClass resource. @@ -805,27 +912,34 @@ spec: logical endpoints that are bound on this Gateway's addresses. At least one Listener MUST be specified. \n Each listener in a Gateway must have a unique combination of Hostname, Port, and Protocol. - \n An implementation MAY group Listeners by Port and then collapse - each group of Listeners into a single Listener if the implementation - determines that the Listeners in the group are \"compatible\". An - implementation MAY also group together and collapse compatible Listeners - belonging to different Gateways. \n For example, an implementation - might consider Listeners to be compatible with each other if all - of the following conditions are met: \n 1. Either each Listener - within the group specifies the \"HTTP\" Protocol or each Listener - within the group specifies either the \"HTTPS\" or \"TLS\" Protocol. - \n 2. Each Listener within the group specifies a Hostname that is - unique within the group. \n 3. As a special case, one Listener within - a group may omit Hostname, in which case this Listener matches when - no other Listener matches. \n If the implementation does collapse - compatible Listeners, the hostname provided in the incoming client - request MUST be matched to a Listener to find the correct set of - Routes. The incoming hostname MUST be matched using the Hostname - field for each Listener in order of most to least specific. That - is, exact matches must be processed before wildcard matches. \n - If this field specifies multiple Listeners that have the same Port - value but are not compatible, the implementation must raise a \"Conflicted\" - condition in the Listener status. \n Support: Core" + \n Within the HTTP Conformance Profile, the below combinations of + port and protocol are considered Core and MUST be supported: \n + 1. Port: 80, Protocol: HTTP 2. Port: 443, Protocol: HTTPS \n Within + the TLS Conformance Profile, the below combinations of port and + protocol are considered Core and MUST be supported: \n 1. Port: + 443, Protocol: TLS \n Port and protocol combinations not listed + above are considered Extended. \n An implementation MAY group Listeners + by Port and then collapse each group of Listeners into a single + Listener if the implementation determines that the Listeners in + the group are \"compatible\". An implementation MAY also group together + and collapse compatible Listeners belonging to different Gateways. + \n For example, an implementation might consider Listeners to be + compatible with each other if all of the following conditions are + met: \n 1. Either each Listener within the group specifies the \"HTTP\" + Protocol or each Listener within the group specifies either the + \"HTTPS\" or \"TLS\" Protocol. \n 2. Each Listener within the group + specifies a Hostname that is unique within the group. \n 3. As a + special case, one Listener within a group may omit Hostname, in + which case this Listener matches when no other Listener matches. + \n If the implementation does collapse compatible Listeners, the + hostname provided in the incoming client request MUST be matched + to a Listener to find the correct set of Routes. The incoming hostname + MUST be matched using the Hostname field for each Listener in order + of most to least specific. That is, exact matches must be processed + before wildcard matches. \n If this field specifies multiple Listeners + that have the same Port value but are not compatible, the implementation + must raise a \"Conflicted\" condition in the Listener status. \n + Support: Core" items: description: Listener embodies the concept of a logical endpoint where a Gateway accepts network connections. @@ -898,7 +1012,7 @@ spec: from: default: Same description: "From indicates where Routes will be selected - for this Gateway. Possible values are: * All: Routes + for this Gateway. Possible values are: \n * All: Routes in all namespaces may be used by this Gateway. * Selector: Routes in namespaces selected by the selector may be used by this Gateway. * Same: Only Routes in the @@ -1076,11 +1190,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. - \n Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to - allow that namespace's owner to accept the reference. - See the ReferenceGrant documentation for details. - \n Support: Core" + \n Note that when a namespace different than the + local namespace is specified, a ReferenceGrant object + is required in the referent namespace to allow that + namespace's owner to accept the reference. See the + ReferenceGrant documentation for details. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -1128,6 +1243,11 @@ spec: maxProperties: 16 type: object type: object + x-kubernetes-validations: + - message: certificateRefs must be specified when TLSModeType + is Terminate + rule: 'self.mode == ''Terminate'' ? size(self.certificateRefs) + > 0 : true' required: - name - port @@ -1139,6 +1259,24 @@ spec: x-kubernetes-list-map-keys: - name x-kubernetes-list-type: map + x-kubernetes-validations: + - message: tls must be specified for protocols ['HTTPS', 'TLS'] + rule: 'self.all(l, l.protocol in [''HTTPS'', ''TLS''] ? has(l.tls) + : true)' + - message: tls must not be specified for protocols ['HTTP', 'TCP', + 'UDP'] + rule: 'self.all(l, l.protocol in [''HTTP'', ''TCP'', ''UDP''] ? + !has(l.tls) : true)' + - message: hostname must not be specified for protocols ['TCP', 'UDP'] + rule: 'self.all(l, l.protocol in [''TCP'', ''UDP''] ? (!has(l.hostname) + || l.hostname == '''') : true)' + - message: Listener name must be unique within the Gateway + rule: self.all(l1, self.exists_one(l2, l1.name == l2.name)) + - message: Combination of port, protocol and hostname must be unique + for each listener + rule: 'self.all(l1, self.exists_one(l2, l1.port == l2.port && l1.protocol + == l2.protocol && (has(l1.hostname) && has(l2.hostname) ? l1.hostname + == l2.hostname : true)))' required: - gatewayClassName - listeners @@ -1148,19 +1286,38 @@ spec: conditions: - lastTransitionTime: "1970-01-01T00:00:00Z" message: Waiting for controller - reason: NotReconciled + reason: Pending status: Unknown type: Accepted + - lastTransitionTime: "1970-01-01T00:00:00Z" + message: Waiting for controller + reason: Pending + status: Unknown + type: Programmed description: Status defines the current state of Gateway. properties: addresses: - description: Addresses lists the IP addresses that have actually been - bound to the Gateway. These addresses may differ from the addresses + description: "Addresses lists the IP addresses that have actually + been bound to the Gateway. These addresses may differ from the addresses in the Spec, e.g. if the Gateway automatically assigns an address - from a reserved pool. + from a reserved pool. \n " items: - description: GatewayAddress describes an address that can be bound + description: GatewayStatusAddress describes an address that is bound to a Gateway. + oneOf: + - properties: + type: + enum: + - IPAddress + value: + anyOf: + - format: ipv4 + - format: ipv6 + - properties: + type: + not: + enum: + - IPAddress properties: type: default: IPAddress @@ -1179,6 +1336,11 @@ spec: required: - value type: object + x-kubernetes-validations: + - message: Hostname value must only contain valid characters (matching + ^(\*\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$) + rule: 'self.type == ''Hostname'' ? self.value.matches(''^(\\*\\.)?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$''): + true' maxItems: 16 type: array conditions: @@ -1198,7 +1360,7 @@ spec: the `GatewayConditionType` and `GatewayConditionReason` constants so that operators and tools can converge on a common vocabulary to describe Gateway state. \n Known condition types are: \n * \"Accepted\" - * \"Ready\"" + * \"Programmed\" * \"Ready\"" items: description: "Condition contains details for one aspect of the current state of this API Resource. --- This struct is intended for direct @@ -1277,8 +1439,8 @@ spec: description: ListenerStatus is the status associated with a Listener. properties: attachedRoutes: - description: AttachedRoutes represents the total number of Routes - that have been successfully attached to this Listener. + description: AttachedRoutes represents the total number of accepted + Routes that have been successfully attached to this Listener. format: int32 type: integer conditions: diff --git a/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml b/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml index aad28fce44..c5aed280e5 100644 --- a/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml +++ b/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: standard creationTimestamp: null name: httproutes.gateway.networking.k8s.io @@ -52,13 +52,17 @@ spec: description: Spec defines the desired state of HTTPRoute. properties: hostnames: - description: "Hostnames defines a set of hostname that should match - against the HTTP Host header to select a HTTPRoute to process the - request. This matches the RFC 1123 definition of a hostname with - 2 notable exceptions: \n 1. IPs are not allowed. 2. A hostname may - be prefixed with a wildcard label (`*.`). The wildcard label must - appear by itself as the first label. \n If a hostname is specified - by both the Listener and HTTPRoute, there must be at least one intersecting + description: "Hostnames defines a set of hostnames that should match + against the HTTP Host header to select a HTTPRoute used to process + the request. Implementations MUST ignore any port value specified + in the HTTP Host header while performing a match and (absent of + any applicable header modification configuration) MUST forward this + header unmodified to the backend. \n Valid values for Hostnames + are determined by RFC 1123 definition of a hostname with 2 notable + exceptions: \n 1. IPs are not allowed. 2. A hostname may be prefixed + with a wildcard label (`*.`). The wildcard label must appear by + itself as the first label. \n If a hostname is specified by both + the Listener and HTTPRoute, there must be at least one intersecting hostname for the HTTPRoute to be attached to the Listener. For example: \n * A Listener with `test.example.com` as the hostname matches HTTPRoutes that have either not specified any hostnames, or have @@ -110,13 +114,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -124,16 +136,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -147,8 +168,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -168,7 +192,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -179,19 +211,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -230,7 +266,8 @@ spec: \n For example, if two backends are specified with equal weights, and one is invalid, 50 percent of traffic must receive a 500. Implementations may choose how that 50 percent is determined. - \n Support: Core for Kubernetes Service \n Support: Implementation-specific + \n Support: Core for Kubernetes Service \n Support: Extended + for Kubernetes ServiceImport \n Support: Implementation-specific for any other resource \n Support for weight: Core" items: description: HTTPBackendRef defines how a HTTPRoute should @@ -258,7 +295,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times + within the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. @@ -396,7 +434,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from - that destination are ignored. \n Support: Extended" + that destination are ignored. \n This filter can + be used multiple times within the same rule. Note + that not all implementations will be able to support + mirroring to multiple backends. \n Support: Extended" properties: backendRef: description: "BackendRef references a resource @@ -431,9 +472,20 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. - For example "HTTPRoute" or "Service". - Defaults to "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to + CNAME DNS records that may live outside + of the cluster and as such are difficult + to reason about in terms of conformance. + They also may not be safe to forward to + (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with + a type other than ExternalName) \n Support: + Implementation-specific (Services with + type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -447,7 +499,8 @@ spec: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n Note that - when a namespace is specified, a ReferenceGrant + when a namespace different than the local + namespace is specified, a ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the ReferenceGrant @@ -473,6 +526,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind + == ''Service'') ? has(self.port) : true' required: - backendRef type: object @@ -485,16 +542,122 @@ spec: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. When empty, the hostname - of the request is used. \n Support: Core" + in the `Host` header of the request is used. + \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string + path: + description: "Path defines parameters used to + modify the path of the incoming request. The + modified path is then used to construct the + `Location` header. When empty, the request + path is used as-is. \n Support: Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies + the value with which to replace the prefix + match of a request during a rewrite or + redirect. For example, a request to \"/foo/bar\" + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that + this matches the behavior of the PathPrefix + match type. This matches full path elements. + A path element refers to the list of labels + in the path split by the `/` separator. + When specified, a trailing `/` is ignored. + For example, the paths `/abc`, `/abc/`, + and `/abc/def` would all match the prefix + `/abc`, but the path `/abcd` would not. + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path + modifier. Additional types may be added + in a future release of the API. \n Note + that values may be added to this enum, + implementations must ensure that unknown + values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the + Route to `status: False`, with a Reason + of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value of the `Location` header in the - response. When empty, port (if specified) - of the request is used. \n Support: Extended" + response. \n If no port is specified, the + redirect port MUST be derived using the following + rules: \n * If redirect scheme is not-empty, + the redirect port MUST be the well-known port + associated with the redirect scheme. Specifically + \"http\" to port 80 and \"https\" to port + 443. If the redirect scheme does not have + a well-known port, the listener port of the + Gateway SHOULD be used. * If redirect scheme + is empty, the redirect port MUST be the Gateway + Listener port. \n Implementations SHOULD NOT + add the port number in the 'Location' header + in the following cases: \n * A Location header + that will use HTTP (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 80. * A Location header that + will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -503,8 +666,11 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request - is used. \n Note that values may be added - to this enum, implementations must ensure + is used. \n Scheme redirects can affect the + port of the redirect, for more information, + refer to the documentation for the port field + of this filter. \n Note that values may be + added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition @@ -530,6 +696,113 @@ spec: - 302 type: integer type: object + responseHeaderModifier: + description: "ResponseHeaderModifier defines a schema + for a filter that modifies response headers. \n + Support: Extended" + properties: + add: + description: "Add adds the given header(s) (name, + value) to the request before the action. It + appends to any existing values associated + with the header name. \n Input: GET /foo HTTP/1.1 + my-header: foo \n Config: add: - name: \"my-header\" + value: \"bar,baz\" \n Output: GET /foo HTTP/1.1 + my-header: foo,bar,baz" + items: + description: HTTPHeader represents an HTTP + Header name and value as defined by RFC + 7230. + properties: + name: + description: "Name is the name of the + HTTP Header to be matched. Name matching + MUST be case insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an + equivalent name MUST be considered for + a match. Subsequent entries with an + equivalent header name MUST be ignored. + Due to the case-insensitivity of header + names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP + Header to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + remove: + description: "Remove the given header(s) from + the HTTP request before the action. The value + of Remove is a list of HTTP header names. + Note that the header names are case-insensitive + (see https://datatracker.ietf.org/doc/html/rfc2616#section-4.2). + \n Input: GET /foo HTTP/1.1 my-header1: foo + my-header2: bar my-header3: baz \n Config: + remove: [\"my-header1\", \"my-header3\"] \n + Output: GET /foo HTTP/1.1 my-header2: bar" + items: + type: string + maxItems: 16 + type: array + set: + description: "Set overwrites the request with + the given header (name, value) before the + action. \n Input: GET /foo HTTP/1.1 my-header: + foo \n Config: set: - name: \"my-header\" + value: \"bar\" \n Output: GET /foo HTTP/1.1 + my-header: bar" + items: + description: HTTPHeader represents an HTTP + Header name and value as defined by RFC + 7230. + properties: + name: + description: "Name is the name of the + HTTP Header to be matched. Name matching + MUST be case insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an + equivalent name MUST be considered for + a match. Subsequent entries with an + equivalent header name MUST be ignored. + Due to the case-insensitivity of header + names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP + Header to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object type: description: "Type identifies the type of filter to apply. As with other API fields, types are @@ -561,18 +834,187 @@ spec: a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - RequestHeaderModifier + - ResponseHeaderModifier - RequestMirror - RequestRedirect + - URLRewrite - ExtensionRef type: string + urlRewrite: + description: "URLRewrite defines a schema for a + filter that modifies a request during forwarding. + \n Support: Extended" + properties: + hostname: + description: "Hostname is the value to be used + to replace the Host header value during forwarding. + \n Support: Extended" + maxLength: 253 + minLength: 1 + pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ + type: string + path: + description: "Path defines a path rewrite. \n + Support: Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies + the value with which to replace the prefix + match of a request during a rewrite or + redirect. For example, a request to \"/foo/bar\" + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that + this matches the behavior of the PathPrefix + match type. This matches full path elements. + A path element refers to the list of labels + in the path split by the `/` separator. + When specified, a trailing `/` is ignored. + For example, the paths `/abc`, `/abc/`, + and `/abc/def` would all match the prefix + `/abc`, but the path `/abcd` would not. + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path + modifier. Additional types may be added + in a future release of the API. \n Note + that values may be added to this enum, + implementations must ensure that unknown + values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the + Route to `status: False`, with a Reason + of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' + type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil + if the filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type + != ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type + == ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil + if the filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type + != ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for + RequestMirror filter.type + rule: '!(!has(self.requestMirror) && self.type == + ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the + filter.type is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != + ''RequestRedirect'')' + - message: filter.requestRedirect must be specified + for RequestRedirect filter.type + rule: '!(!has(self.requestRedirect) && self.type == + ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for + ExtensionRef filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') + ? self.exists_one(f, f.type == ''RequestRedirect'') + : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' group: default: "" description: Group is the group of the referent. For example, @@ -583,9 +1025,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -598,11 +1048,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -640,6 +1090,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 type: array filters: @@ -652,14 +1106,15 @@ spec: all implementations. - Implementers are encouraged to support extended filters. - Implementation-specific custom filters have no API guarantees across implementations. \n Specifying - a core filter multiple times has unspecified or implementation-specific - conformance. \n All filters are expected to be compatible - with each other except for the URLRewrite and RequestRedirect + the same filter multiple times is not supported unless explicitly + indicated in the filter. \n All filters are expected to be + compatible with each other except for the URLRewrite and RequestRedirect filters, which may not be combined. If an implementation can not support other combinations of filters, they must clearly - document that limitation. In all cases where incompatible - or unsupported filters are specified, implementations MUST - add a warning condition to status. \n Support: Core" + document that limitation. In cases where incompatible or unsupported + filters are specified and cause the `Accepted` condition to + be set to status `False`, implementations may use the `IncompatibleFilters` + reason to specify this configuration error. \n Support: Core" items: description: HTTPRouteFilter defines processing steps that must be completed during the request or response lifecycle. @@ -675,7 +1130,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times within + the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. For @@ -805,7 +1261,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from that destination are - ignored. \n Support: Extended" + ignored. \n This filter can be used multiple times within + the same rule. Note that not all implementations will + be able to support mirroring to multiple backends. \n + Support: Extended" properties: backendRef: description: "BackendRef references a resource where @@ -838,9 +1297,18 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For - example "HTTPRoute" or "Service". Defaults to - "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to CNAME + DNS records that may live outside of the cluster + and as such are difficult to reason about in + terms of conformance. They also may not be safe + to forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with a + type other than ExternalName) \n Support: Implementation-specific + (Services with type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -853,11 +1321,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace - is inferred. \n Note that when a namespace is - specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's - owner to accept the reference. See the ReferenceGrant - documentation for details. \n Support: Core" + is inferred. \n Note that when a namespace different + than the local namespace is specified, a ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept the + reference. See the ReferenceGrant documentation + for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -877,6 +1346,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' required: - backendRef type: object @@ -888,17 +1361,114 @@ spec: hostname: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. - When empty, the hostname of the request is used. - \n Support: Core" + When empty, the hostname in the `Host` header of + the request is used. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string + path: + description: "Path defines parameters used to modify + the path of the incoming request. The modified path + is then used to construct the `Location` header. + When empty, the request path is used as-is. \n Support: + Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the value + with which to replace the full path of a request + during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies the + value with which to replace the prefix match + of a request during a rewrite or redirect. For + example, a request to \"/foo/bar\" with a prefix + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". + \n Note that this matches the behavior of the + PathPrefix match type. This matches full path + elements. A path element refers to the list + of labels in the path split by the `/` separator. + When specified, a trailing `/` is ignored. For + example, the paths `/abc`, `/abc/`, and `/abc/def` + would all match the prefix `/abc`, but the path + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path modifier. + Additional types may be added in a future release + of the API. \n Note that values may be added + to this enum, implementations must ensure that + unknown values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the Route + to `status: False`, with a Reason of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value - of the `Location` header in the response. When empty, - port (if specified) of the request is used. \n Support: - Extended" + of the `Location` header in the response. \n If + no port is specified, the redirect port MUST be + derived using the following rules: \n * If redirect + scheme is not-empty, the redirect port MUST be the + well-known port associated with the redirect scheme. + Specifically \"http\" to port 80 and \"https\" to + port 443. If the redirect scheme does not have a + well-known port, the listener port of the Gateway + SHOULD be used. * If redirect scheme is empty, the + redirect port MUST be the Gateway Listener port. + \n Implementations SHOULD NOT add the port number + in the 'Location' header in the following cases: + \n * A Location header that will use HTTP (whether + that is determined via the Listener protocol or + the Scheme field) _and_ use port 80. * A Location + header that will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) _and_ + use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -907,7 +1477,10 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request is used. \n - Note that values may be added to this enum, implementations + Scheme redirects can affect the port of the redirect, + for more information, refer to the documentation + for the port field of this filter. \n Note that + values may be added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for @@ -932,6 +1505,106 @@ spec: - 302 type: integer type: object + responseHeaderModifier: + description: "ResponseHeaderModifier defines a schema + for a filter that modifies response headers. \n Support: + Extended" + properties: + add: + description: "Add adds the given header(s) (name, + value) to the request before the action. It appends + to any existing values associated with the header + name. \n Input: GET /foo HTTP/1.1 my-header: foo + \n Config: add: - name: \"my-header\" value: \"bar,baz\" + \n Output: GET /foo HTTP/1.1 my-header: foo,bar,baz" + items: + description: HTTPHeader represents an HTTP Header + name and value as defined by RFC 7230. + properties: + name: + description: "Name is the name of the HTTP Header + to be matched. Name matching MUST be case + insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an equivalent + name MUST be considered for a match. Subsequent + entries with an equivalent header name MUST + be ignored. Due to the case-insensitivity + of header names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP Header + to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + remove: + description: "Remove the given header(s) from the + HTTP request before the action. The value of Remove + is a list of HTTP header names. Note that the header + names are case-insensitive (see https://datatracker.ietf.org/doc/html/rfc2616#section-4.2). + \n Input: GET /foo HTTP/1.1 my-header1: foo my-header2: + bar my-header3: baz \n Config: remove: [\"my-header1\", + \"my-header3\"] \n Output: GET /foo HTTP/1.1 my-header2: + bar" + items: + type: string + maxItems: 16 + type: array + set: + description: "Set overwrites the request with the + given header (name, value) before the action. \n + Input: GET /foo HTTP/1.1 my-header: foo \n Config: + set: - name: \"my-header\" value: \"bar\" \n Output: + GET /foo HTTP/1.1 my-header: bar" + items: + description: HTTPHeader represents an HTTP Header + name and value as defined by RFC 7230. + properties: + name: + description: "Name is the name of the HTTP Header + to be matched. Name matching MUST be case + insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an equivalent + name MUST be considered for a match. Subsequent + entries with an equivalent header name MUST + be ignored. Due to the case-insensitivity + of header names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP Header + to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object type: description: "Type identifies the type of filter to apply. As with other API fields, types are classified into @@ -959,19 +1632,173 @@ spec: implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition - for the Route to `status: False`, with a Reason of `UnsupportedValue`. - \n " + for the Route to `status: False`, with a Reason of `UnsupportedValue`." enum: - RequestHeaderModifier + - ResponseHeaderModifier - RequestMirror - RequestRedirect + - URLRewrite - ExtensionRef type: string + urlRewrite: + description: "URLRewrite defines a schema for a filter + that modifies a request during forwarding. \n Support: + Extended" + properties: + hostname: + description: "Hostname is the value to be used to + replace the Host header value during forwarding. + \n Support: Extended" + maxLength: 253 + minLength: 1 + pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ + type: string + path: + description: "Path defines a path rewrite. \n Support: + Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the value + with which to replace the full path of a request + during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies the + value with which to replace the prefix match + of a request during a rewrite or redirect. For + example, a request to \"/foo/bar\" with a prefix + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". + \n Note that this matches the behavior of the + PathPrefix match type. This matches full path + elements. A path element refers to the list + of labels in the path split by the `/` separator. + When specified, a trailing `/` is ignored. For + example, the paths `/abc`, `/abc/`, and `/abc/def` + would all match the prefix `/abc`, but the path + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path modifier. + Additional types may be added in a future release + of the API. \n Note that values may be added + to this enum, implementations must ensure that + unknown values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the Route + to `status: False`, with a Reason of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' + type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil if the + filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type != + ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type == + ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil if the + filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type != + ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for RequestMirror + filter.type + rule: '!(!has(self.requestMirror) && self.type == ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the filter.type + is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != ''RequestRedirect'')' + - message: filter.requestRedirect must be specified for RequestRedirect + filter.type + rule: '!(!has(self.requestRedirect) && self.type == ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for ExtensionRef + filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') && + self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') ? self.exists_one(f, + f.type == ''RequestRedirect'') : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' matches: default: - path: @@ -993,19 +1820,21 @@ spec: request. \n Proxy or Load Balancer routing configuration generated from HTTPRoutes MUST prioritize matches based on the following criteria, continuing on ties. Across all rules specified on - applicable Routes, precedence must be given to the match with - the largest number of: \n * Characters in a matching path. - * Header matches. * Query param matches. \n If ties still - exist across multiple Routes, matching precedence MUST be - determined in order of the following criteria, continuing - on ties: \n * The oldest Route based on creation timestamp. - * The Route appearing first in alphabetical order by \"{namespace}/{name}\". - \n If ties still exist within an HTTPRoute, matching precedence - MUST be granted to the FIRST matching rule (in list order) - with a match meeting the above criteria. \n When no rules - matching a request have been successfully attached to the - parent a request is coming from, a HTTP 404 status code MUST - be returned." + applicable Routes, precedence must be given to the match having: + \n * \"Exact\" path match. * \"Prefix\" path match with largest + number of characters. * Method match. * Largest number of + header matches. * Largest number of query param matches. \n + Note: The precedence of RegularExpression path matches are + implementation-specific. \n If ties still exist across multiple + Routes, matching precedence MUST be determined in order of + the following criteria, continuing on ties: \n * The oldest + Route based on creation timestamp. * The Route appearing first + in alphabetical order by \"{namespace}/{name}\". \n If ties + still exist within an HTTPRoute, matching precedence MUST + be granted to the FIRST matching rule (in list order) with + a match meeting the above criteria. \n When no rules matching + a request have been successfully attached to the parent a + request is coming from, a HTTP 404 status code MUST be returned." items: description: "HTTPRouteMatch defines the predicate used to match requests to a given action. Multiple match types are @@ -1113,6 +1942,53 @@ spec: maxLength: 1024 type: string type: object + x-kubernetes-validations: + - message: value must be an absolute path and start with + '/' when type one of ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.startsWith(''/'') : true' + - message: must not contain '//' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''//'') : true' + - message: must not contain '/./' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/./'') : true' + - message: must not contain '/../' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/../'') : true' + - message: must not contain '%2f' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2f'') : true' + - message: must not contain '%2F' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2F'') : true' + - message: must not contain '#' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''#'') : true' + - message: must not end with '/..' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/..'') : true' + - message: must not end with '/.' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/.'') : true' + - message: type must be one of ['Exact', 'PathPrefix', + 'RegularExpression'] + rule: self.type == 'Exact' || self.type == 'PathPrefix' + || self.type == 'RegularExpression' + - message: must only contain valid characters (matching + ^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$) + for types ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.matches(r"""^(?:[-A-Za-z0-9/._~!$&''()*+,;=:@]|[%][0-9a-fA-F]{2})+$""") + : true' queryParams: description: "QueryParams specifies HTTP query parameter matchers. Multiple match values are ANDed together, @@ -1143,6 +2019,7 @@ spec: differences in the implementations." maxLength: 256 minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ type: string type: default: Exact @@ -1178,6 +2055,46 @@ spec: maxItems: 8 type: array type: object + x-kubernetes-validations: + - message: RequestRedirect filter must not be used together with + backendRefs + rule: '(has(self.backendRefs) && size(self.backendRefs) > 0) ? + (!has(self.filters) || self.filters.all(f, !has(f.requestRedirect))): + true' + - message: When using RequestRedirect filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + ? ((size(self.matches) != 1 || !has(self.matches[0].path) || + self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: When using URLRewrite filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' + - message: Within backendRefs, when using RequestRedirect filter + with path.replacePrefixMatch, exactly one PathPrefix match must + be specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + )) ? ((size(self.matches) != 1 || !has(self.matches[0].path) + || self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: Within backendRefs, When using URLRewrite filter with + path.replacePrefixMatch, exactly one PathPrefix match must be + specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) )) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' maxItems: 16 type: array type: object @@ -1326,9 +2243,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -1348,7 +2267,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -1359,8 +2287,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of @@ -1392,7 +2325,7 @@ spec: required: - spec type: object - served: true + served: false storage: false subresources: status: {} @@ -1427,13 +2360,17 @@ spec: description: Spec defines the desired state of HTTPRoute. properties: hostnames: - description: "Hostnames defines a set of hostname that should match - against the HTTP Host header to select a HTTPRoute to process the - request. This matches the RFC 1123 definition of a hostname with - 2 notable exceptions: \n 1. IPs are not allowed. 2. A hostname may - be prefixed with a wildcard label (`*.`). The wildcard label must - appear by itself as the first label. \n If a hostname is specified - by both the Listener and HTTPRoute, there must be at least one intersecting + description: "Hostnames defines a set of hostnames that should match + against the HTTP Host header to select a HTTPRoute used to process + the request. Implementations MUST ignore any port value specified + in the HTTP Host header while performing a match and (absent of + any applicable header modification configuration) MUST forward this + header unmodified to the backend. \n Valid values for Hostnames + are determined by RFC 1123 definition of a hostname with 2 notable + exceptions: \n 1. IPs are not allowed. 2. A hostname may be prefixed + with a wildcard label (`*.`). The wildcard label must appear by + itself as the first label. \n If a hostname is specified by both + the Listener and HTTPRoute, there must be at least one intersecting hostname for the HTTPRoute to be attached to the Listener. For example: \n * A Listener with `test.example.com` as the hostname matches HTTPRoutes that have either not specified any hostnames, or have @@ -1485,13 +2422,21 @@ spec: that a Route wants to be attached to. Note that the referenced parent resource needs to allow this for the attachment to be complete. For Gateways, that means the Gateway needs to allow attachment from - Routes of this kind and namespace. \n The only kind of parent resource - with \"Core\" support is Gateway. This API may be extended in the - future to support additional kinds of parent resources such as one - of the route kinds. \n It is invalid to reference an identical parent + Routes of this kind and namespace. For Services, that means the + Service must either be in the same namespace for a \"producer\" + route, or the mesh implementation must support and allow \"consumer\" + routes for the referenced Service. ReferenceGrant is not applicable + for governing ParentRefs to Services - it is not possible to create + a \"producer\" route for a Service in a different namespace from + the Route. \n There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services only) \n This + API may be extended in the future to support additional kinds of + parent resources. \n It is invalid to reference an identical parent more than once. It is valid to reference multiple distinct sections - within the same parent resource, such as 2 Listeners within a Gateway. - \n It is possible to separately reference multiple distinct objects + within the same parent resource, such as two separate Listeners + on the same Gateway or two separate ports on the same Service. \n + It is possible to separately reference multiple distinct objects that may be collapsed by an implementation. For example, some implementations may choose to merge compatible Gateway Listeners together. If that is the case, the list of routes attached to those resources should @@ -1499,16 +2444,25 @@ spec: boundaries, there are specific rules. Cross-namespace references are only valid if they are explicitly allowed by something in the namespace they are referring to. For example, Gateway has the AllowedRoutes - field, and ReferenceGrant provides a generic way to enable any other - kind of cross-namespace reference." + field, and ReferenceGrant provides a generic way to enable other + kinds of cross-namespace reference. \n ParentRefs from a Route to + a Service in the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any namespace + to the Service. \n ParentRefs from a Route to a Service in a different + namespace are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same namespace + as the Route, for which the intended destination of the connections + are a Service targeted as a ParentRef of the Route." items: description: "ParentReference identifies an API object (usually a Gateway) that can be considered a parent of this resource (usually - a route). The only kind of parent resource with \"Core\" support - is Gateway. This API may be extended in the future to support - additional kinds of parent resources, such as HTTPRoute. \n The - API object must be valid in the cluster; the Group and Kind must - be registered in the cluster for this reference to be valid." + a route). There are two kinds of parent resources with \"Core\" + support: \n * Gateway (Gateway conformance profile) * Service + (Mesh conformance profile, experimental, ClusterIP Services only) + \n This API may be extended in the future to support additional + kinds of parent resources. \n The API object must be valid in + the cluster; the Group and Kind must be registered in the cluster + for this reference to be valid." properties: group: default: gateway.networking.k8s.io @@ -1522,8 +2476,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: Core - (Gateway) \n Support: Implementation-specific (Other Resources)" + description: "Kind is kind of the referent. \n There are two + kinds of parent resources with \"Core\" support: \n * Gateway + (Gateway conformance profile) * Service (Mesh conformance + profile, experimental, ClusterIP Services only) \n Support + for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -1543,7 +2500,15 @@ spec: the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace reference. - \n Support: Core" + \n ParentRefs from a Route to a Service in the same namespace + are \"producer\" routes, which apply default routing rules + to inbound connections from any namespace to the Service. + \n ParentRefs from a Route to a Service in a different namespace + are \"consumer\" routes, and these routing rules are only + applied to outbound connections originating from the same + namespace as the Route, for which the intended destination + of the connections are a Service targeted as a ParentRef of + the Route. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -1554,19 +2519,23 @@ spec: interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener must match both - specified values. \n Implementations MAY choose to support - attaching Routes to other resources. If that is the case, - they MUST clearly document how SectionName is interpreted. - \n When unspecified (empty string), this will reference the - entire resource. For the purpose of status, an attachment - is considered successful if at least one section in the parent - resource accepts it. For example, Gateway listeners can restrict - which Routes can attach to them by Route kind, namespace, - or hostname. If 1 of 2 Gateway listeners accept attachment - from the referencing Route, the Route MUST be considered successfully - attached. If no Gateway listeners accept attachment from this - Route, the Route MUST be considered detached from the Gateway. - \n Support: Core" + specified values. * Service: Port Name. When both Port (experimental) + and SectionName are specified, the name and port of the selected + listener must match both specified values. Note that attaching + Routes to Services as Parents is part of experimental Mesh + support and is not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. + If that is the case, they MUST clearly document how SectionName + is interpreted. \n When unspecified (empty string), this will + reference the entire resource. For the purpose of status, + an attachment is considered successful if at least one section + in the parent resource accepts it. For example, Gateway listeners + can restrict which Routes can attach to them by Route kind, + namespace, or hostname. If 1 of 2 Gateway listeners accept + attachment from the referencing Route, the Route MUST be considered + successfully attached. If no Gateway listeners accept attachment + from this Route, the Route MUST be considered detached from + the Gateway. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ @@ -1605,7 +2574,8 @@ spec: \n For example, if two backends are specified with equal weights, and one is invalid, 50 percent of traffic must receive a 500. Implementations may choose how that 50 percent is determined. - \n Support: Core for Kubernetes Service \n Support: Implementation-specific + \n Support: Core for Kubernetes Service \n Support: Extended + for Kubernetes ServiceImport \n Support: Implementation-specific for any other resource \n Support for weight: Core" items: description: HTTPBackendRef defines how a HTTPRoute should @@ -1633,7 +2603,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times + within the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. @@ -1771,7 +2742,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from - that destination are ignored. \n Support: Extended" + that destination are ignored. \n This filter can + be used multiple times within the same rule. Note + that not all implementations will be able to support + mirroring to multiple backends. \n Support: Extended" properties: backendRef: description: "BackendRef references a resource @@ -1806,9 +2780,20 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. - For example "HTTPRoute" or "Service". - Defaults to "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to + CNAME DNS records that may live outside + of the cluster and as such are difficult + to reason about in terms of conformance. + They also may not be safe to forward to + (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with + a type other than ExternalName) \n Support: + Implementation-specific (Services with + type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -1822,7 +2807,8 @@ spec: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n Note that - when a namespace is specified, a ReferenceGrant + when a namespace different than the local + namespace is specified, a ReferenceGrant object is required in the referent namespace to allow that namespace's owner to accept the reference. See the ReferenceGrant @@ -1848,6 +2834,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind + == ''Service'') ? has(self.port) : true' required: - backendRef type: object @@ -1860,16 +2850,122 @@ spec: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. When empty, the hostname - of the request is used. \n Support: Core" + in the `Host` header of the request is used. + \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string + path: + description: "Path defines parameters used to + modify the path of the incoming request. The + modified path is then used to construct the + `Location` header. When empty, the request + path is used as-is. \n Support: Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies + the value with which to replace the prefix + match of a request during a rewrite or + redirect. For example, a request to \"/foo/bar\" + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that + this matches the behavior of the PathPrefix + match type. This matches full path elements. + A path element refers to the list of labels + in the path split by the `/` separator. + When specified, a trailing `/` is ignored. + For example, the paths `/abc`, `/abc/`, + and `/abc/def` would all match the prefix + `/abc`, but the path `/abcd` would not. + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path + modifier. Additional types may be added + in a future release of the API. \n Note + that values may be added to this enum, + implementations must ensure that unknown + values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the + Route to `status: False`, with a Reason + of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value of the `Location` header in the - response. When empty, port (if specified) - of the request is used. \n Support: Extended" + response. \n If no port is specified, the + redirect port MUST be derived using the following + rules: \n * If redirect scheme is not-empty, + the redirect port MUST be the well-known port + associated with the redirect scheme. Specifically + \"http\" to port 80 and \"https\" to port + 443. If the redirect scheme does not have + a well-known port, the listener port of the + Gateway SHOULD be used. * If redirect scheme + is empty, the redirect port MUST be the Gateway + Listener port. \n Implementations SHOULD NOT + add the port number in the 'Location' header + in the following cases: \n * A Location header + that will use HTTP (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 80. * A Location header that + will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) + _and_ use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -1878,8 +2974,11 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request - is used. \n Note that values may be added - to this enum, implementations must ensure + is used. \n Scheme redirects can affect the + port of the redirect, for more information, + refer to the documentation for the port field + of this filter. \n Note that values may be + added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition @@ -1905,6 +3004,113 @@ spec: - 302 type: integer type: object + responseHeaderModifier: + description: "ResponseHeaderModifier defines a schema + for a filter that modifies response headers. \n + Support: Extended" + properties: + add: + description: "Add adds the given header(s) (name, + value) to the request before the action. It + appends to any existing values associated + with the header name. \n Input: GET /foo HTTP/1.1 + my-header: foo \n Config: add: - name: \"my-header\" + value: \"bar,baz\" \n Output: GET /foo HTTP/1.1 + my-header: foo,bar,baz" + items: + description: HTTPHeader represents an HTTP + Header name and value as defined by RFC + 7230. + properties: + name: + description: "Name is the name of the + HTTP Header to be matched. Name matching + MUST be case insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an + equivalent name MUST be considered for + a match. Subsequent entries with an + equivalent header name MUST be ignored. + Due to the case-insensitivity of header + names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP + Header to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + remove: + description: "Remove the given header(s) from + the HTTP request before the action. The value + of Remove is a list of HTTP header names. + Note that the header names are case-insensitive + (see https://datatracker.ietf.org/doc/html/rfc2616#section-4.2). + \n Input: GET /foo HTTP/1.1 my-header1: foo + my-header2: bar my-header3: baz \n Config: + remove: [\"my-header1\", \"my-header3\"] \n + Output: GET /foo HTTP/1.1 my-header2: bar" + items: + type: string + maxItems: 16 + type: array + set: + description: "Set overwrites the request with + the given header (name, value) before the + action. \n Input: GET /foo HTTP/1.1 my-header: + foo \n Config: set: - name: \"my-header\" + value: \"bar\" \n Output: GET /foo HTTP/1.1 + my-header: bar" + items: + description: HTTPHeader represents an HTTP + Header name and value as defined by RFC + 7230. + properties: + name: + description: "Name is the name of the + HTTP Header to be matched. Name matching + MUST be case insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an + equivalent name MUST be considered for + a match. Subsequent entries with an + equivalent header name MUST be ignored. + Due to the case-insensitivity of header + names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP + Header to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object type: description: "Type identifies the type of filter to apply. As with other API fields, types are @@ -1936,18 +3142,187 @@ spec: a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for the Route to `status: False`, with a Reason - of `UnsupportedValue`. \n " + of `UnsupportedValue`." enum: - RequestHeaderModifier + - ResponseHeaderModifier - RequestMirror - RequestRedirect + - URLRewrite - ExtensionRef type: string + urlRewrite: + description: "URLRewrite defines a schema for a + filter that modifies a request during forwarding. + \n Support: Extended" + properties: + hostname: + description: "Hostname is the value to be used + to replace the Host header value during forwarding. + \n Support: Extended" + maxLength: 253 + minLength: 1 + pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ + type: string + path: + description: "Path defines a path rewrite. \n + Support: Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the + value with which to replace the full path + of a request during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies + the value with which to replace the prefix + match of a request during a rewrite or + redirect. For example, a request to \"/foo/bar\" + with a prefix match of \"/foo\" and a + ReplacePrefixMatch of \"/xyz\" would be + modified to \"/xyz/bar\". \n Note that + this matches the behavior of the PathPrefix + match type. This matches full path elements. + A path element refers to the list of labels + in the path split by the `/` separator. + When specified, a trailing `/` is ignored. + For example, the paths `/abc`, `/abc/`, + and `/abc/def` would all match the prefix + `/abc`, but the path `/abcd` would not. + \n ReplacePrefixMatch is only compatible + with a `PathPrefix` HTTPRouteMatch. Using + any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the + Route to `status: False`. \n Request Path + | Prefix Match | Replace Prefix | Modified + Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | + /xyz/ | /xyz/bar /foo/bar | + /foo/ | /xyz | /xyz/bar + /foo/bar | /foo/ | /xyz/ | + /xyz/bar /foo | /foo | + /xyz | /xyz /foo/ | /foo + \ | /xyz | /xyz/ /foo/bar + \ | /foo | | + /bar /foo/ | /foo | | / /foo | /foo | + | / /foo/ | /foo + \ | / | / /foo | + /foo | / | /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path + modifier. Additional types may be added + in a future release of the API. \n Note + that values may be added to this enum, + implementations must ensure that unknown + values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the + Route to `status: False`, with a Reason + of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified + when type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? + has(self.replaceFullPath) : true' + - message: type must be 'ReplaceFullPath' when + replaceFullPath is set + rule: 'has(self.replaceFullPath) ? self.type + == ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified + when type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' + ? has(self.replacePrefixMatch) : true' + - message: type must be 'ReplacePrefixMatch' + when replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' + type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil + if the filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type + != ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type + == ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil + if the filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type + != ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for + RequestMirror filter.type + rule: '!(!has(self.requestMirror) && self.type == + ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the + filter.type is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != + ''RequestRedirect'')' + - message: filter.requestRedirect must be specified + for RequestRedirect filter.type + rule: '!(!has(self.requestRedirect) && self.type == + ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for + ExtensionRef filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') + && self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') + ? self.exists_one(f, f.type == ''RequestRedirect'') + : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' group: default: "" description: Group is the group of the referent. For example, @@ -1958,9 +3333,17 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For example - "HTTPRoute" or "Service". Defaults to "Service" when - not specified. + description: "Kind is the Kubernetes resource kind of + the referent. For example \"Service\". \n Defaults to + \"Service\" when not specified. \n ExternalName services + can refer to CNAME DNS records that may live outside + of the cluster and as such are difficult to reason about + in terms of conformance. They also may not be safe to + forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName Services. + \n Support: Core (Services with a type other than ExternalName) + \n Support: Implementation-specific (Services with type + ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -1973,11 +3356,11 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace is inferred. \n - Note that when a namespace is specified, a ReferenceGrant - object is required in the referent namespace to allow - that namespace's owner to accept the reference. See - the ReferenceGrant documentation for details. \n Support: - Core" + Note that when a namespace different than the local + namespace is specified, a ReferenceGrant object is required + in the referent namespace to allow that namespace's + owner to accept the reference. See the ReferenceGrant + documentation for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -2015,6 +3398,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' maxItems: 16 type: array filters: @@ -2027,14 +3414,15 @@ spec: all implementations. - Implementers are encouraged to support extended filters. - Implementation-specific custom filters have no API guarantees across implementations. \n Specifying - a core filter multiple times has unspecified or implementation-specific - conformance. \n All filters are expected to be compatible - with each other except for the URLRewrite and RequestRedirect + the same filter multiple times is not supported unless explicitly + indicated in the filter. \n All filters are expected to be + compatible with each other except for the URLRewrite and RequestRedirect filters, which may not be combined. If an implementation can not support other combinations of filters, they must clearly - document that limitation. In all cases where incompatible - or unsupported filters are specified, implementations MUST - add a warning condition to status. \n Support: Core" + document that limitation. In cases where incompatible or unsupported + filters are specified and cause the `Accepted` condition to + be set to status `False`, implementations may use the `IncompatibleFilters` + reason to specify this configuration error. \n Support: Core" items: description: HTTPRouteFilter defines processing steps that must be completed during the request or response lifecycle. @@ -2050,7 +3438,8 @@ spec: extension to the \"filter\" behavior. For example, resource \"myroutefilter\" in group \"networking.example.net\"). ExtensionRef MUST NOT be used for core and extended - filters. \n Support: Implementation-specific" + filters. \n This filter can be used multiple times within + the same rule. \n Support: Implementation-specific" properties: group: description: Group is the group of the referent. For @@ -2180,7 +3569,10 @@ spec: description: "RequestMirror defines a schema for a filter that mirrors requests. Requests are sent to the specified destination, but responses from that destination are - ignored. \n Support: Extended" + ignored. \n This filter can be used multiple times within + the same rule. Note that not all implementations will + be able to support mirroring to multiple backends. \n + Support: Extended" properties: backendRef: description: "BackendRef references a resource where @@ -2213,9 +3605,18 @@ spec: type: string kind: default: Service - description: Kind is kind of the referent. For - example "HTTPRoute" or "Service". Defaults to - "Service" when not specified. + description: "Kind is the Kubernetes resource + kind of the referent. For example \"Service\". + \n Defaults to \"Service\" when not specified. + \n ExternalName services can refer to CNAME + DNS records that may live outside of the cluster + and as such are difficult to reason about in + terms of conformance. They also may not be safe + to forward to (see CVE-2021-25740 for more information). + Implementations SHOULD NOT support ExternalName + Services. \n Support: Core (Services with a + type other than ExternalName) \n Support: Implementation-specific + (Services with type ExternalName)" maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -2228,11 +3629,12 @@ spec: namespace: description: "Namespace is the namespace of the backend. When unspecified, the local namespace - is inferred. \n Note that when a namespace is - specified, a ReferenceGrant object is required - in the referent namespace to allow that namespace's - owner to accept the reference. See the ReferenceGrant - documentation for details. \n Support: Core" + is inferred. \n Note that when a namespace different + than the local namespace is specified, a ReferenceGrant + object is required in the referent namespace + to allow that namespace's owner to accept the + reference. See the ReferenceGrant documentation + for details. \n Support: Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -2252,6 +3654,10 @@ spec: required: - name type: object + x-kubernetes-validations: + - message: Must have port for Service reference + rule: '(size(self.group) == 0 && self.kind == ''Service'') + ? has(self.port) : true' required: - backendRef type: object @@ -2263,17 +3669,114 @@ spec: hostname: description: "Hostname is the hostname to be used in the value of the `Location` header in the response. - When empty, the hostname of the request is used. - \n Support: Core" + When empty, the hostname in the `Host` header of + the request is used. \n Support: Core" maxLength: 253 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ type: string + path: + description: "Path defines parameters used to modify + the path of the incoming request. The modified path + is then used to construct the `Location` header. + When empty, the request path is used as-is. \n Support: + Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the value + with which to replace the full path of a request + during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies the + value with which to replace the prefix match + of a request during a rewrite or redirect. For + example, a request to \"/foo/bar\" with a prefix + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". + \n Note that this matches the behavior of the + PathPrefix match type. This matches full path + elements. A path element refers to the list + of labels in the path split by the `/` separator. + When specified, a trailing `/` is ignored. For + example, the paths `/abc`, `/abc/`, and `/abc/def` + would all match the prefix `/abc`, but the path + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path modifier. + Additional types may be added in a future release + of the API. \n Note that values may be added + to this enum, implementations must ensure that + unknown values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the Route + to `status: False`, with a Reason of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' port: description: "Port is the port to be used in the value - of the `Location` header in the response. When empty, - port (if specified) of the request is used. \n Support: - Extended" + of the `Location` header in the response. \n If + no port is specified, the redirect port MUST be + derived using the following rules: \n * If redirect + scheme is not-empty, the redirect port MUST be the + well-known port associated with the redirect scheme. + Specifically \"http\" to port 80 and \"https\" to + port 443. If the redirect scheme does not have a + well-known port, the listener port of the Gateway + SHOULD be used. * If redirect scheme is empty, the + redirect port MUST be the Gateway Listener port. + \n Implementations SHOULD NOT add the port number + in the 'Location' header in the following cases: + \n * A Location header that will use HTTP (whether + that is determined via the Listener protocol or + the Scheme field) _and_ use port 80. * A Location + header that will use HTTPS (whether that is determined + via the Listener protocol or the Scheme field) _and_ + use port 443. \n Support: Extended" format: int32 maximum: 65535 minimum: 1 @@ -2282,7 +3785,10 @@ spec: description: "Scheme is the scheme to be used in the value of the `Location` header in the response. When empty, the scheme of the request is used. \n - Note that values may be added to this enum, implementations + Scheme redirects can affect the port of the redirect, + for more information, refer to the documentation + for the port field of this filter. \n Note that + values may be added to this enum, implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition for @@ -2307,6 +3813,106 @@ spec: - 302 type: integer type: object + responseHeaderModifier: + description: "ResponseHeaderModifier defines a schema + for a filter that modifies response headers. \n Support: + Extended" + properties: + add: + description: "Add adds the given header(s) (name, + value) to the request before the action. It appends + to any existing values associated with the header + name. \n Input: GET /foo HTTP/1.1 my-header: foo + \n Config: add: - name: \"my-header\" value: \"bar,baz\" + \n Output: GET /foo HTTP/1.1 my-header: foo,bar,baz" + items: + description: HTTPHeader represents an HTTP Header + name and value as defined by RFC 7230. + properties: + name: + description: "Name is the name of the HTTP Header + to be matched. Name matching MUST be case + insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an equivalent + name MUST be considered for a match. Subsequent + entries with an equivalent header name MUST + be ignored. Due to the case-insensitivity + of header names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP Header + to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + remove: + description: "Remove the given header(s) from the + HTTP request before the action. The value of Remove + is a list of HTTP header names. Note that the header + names are case-insensitive (see https://datatracker.ietf.org/doc/html/rfc2616#section-4.2). + \n Input: GET /foo HTTP/1.1 my-header1: foo my-header2: + bar my-header3: baz \n Config: remove: [\"my-header1\", + \"my-header3\"] \n Output: GET /foo HTTP/1.1 my-header2: + bar" + items: + type: string + maxItems: 16 + type: array + set: + description: "Set overwrites the request with the + given header (name, value) before the action. \n + Input: GET /foo HTTP/1.1 my-header: foo \n Config: + set: - name: \"my-header\" value: \"bar\" \n Output: + GET /foo HTTP/1.1 my-header: bar" + items: + description: HTTPHeader represents an HTTP Header + name and value as defined by RFC 7230. + properties: + name: + description: "Name is the name of the HTTP Header + to be matched. Name matching MUST be case + insensitive. (See https://tools.ietf.org/html/rfc7230#section-3.2). + \n If multiple entries specify equivalent + header names, the first entry with an equivalent + name MUST be considered for a match. Subsequent + entries with an equivalent header name MUST + be ignored. Due to the case-insensitivity + of header names, \"foo\" and \"Foo\" are considered + equivalent." + maxLength: 256 + minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ + type: string + value: + description: Value is the value of HTTP Header + to be matched. + maxLength: 4096 + minLength: 1 + type: string + required: + - name + - value + type: object + maxItems: 16 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object type: description: "Type identifies the type of filter to apply. As with other API fields, types are classified into @@ -2334,19 +3940,173 @@ spec: implementations must ensure that unknown values will not cause a crash. \n Unknown values here must result in the implementation setting the Accepted Condition - for the Route to `status: False`, with a Reason of `UnsupportedValue`. - \n " + for the Route to `status: False`, with a Reason of `UnsupportedValue`." enum: - RequestHeaderModifier + - ResponseHeaderModifier - RequestMirror - RequestRedirect + - URLRewrite - ExtensionRef type: string + urlRewrite: + description: "URLRewrite defines a schema for a filter + that modifies a request during forwarding. \n Support: + Extended" + properties: + hostname: + description: "Hostname is the value to be used to + replace the Host header value during forwarding. + \n Support: Extended" + maxLength: 253 + minLength: 1 + pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$ + type: string + path: + description: "Path defines a path rewrite. \n Support: + Extended" + properties: + replaceFullPath: + description: ReplaceFullPath specifies the value + with which to replace the full path of a request + during a rewrite or redirect. + maxLength: 1024 + type: string + replacePrefixMatch: + description: "ReplacePrefixMatch specifies the + value with which to replace the prefix match + of a request during a rewrite or redirect. For + example, a request to \"/foo/bar\" with a prefix + match of \"/foo\" and a ReplacePrefixMatch of + \"/xyz\" would be modified to \"/xyz/bar\". + \n Note that this matches the behavior of the + PathPrefix match type. This matches full path + elements. A path element refers to the list + of labels in the path split by the `/` separator. + When specified, a trailing `/` is ignored. For + example, the paths `/abc`, `/abc/`, and `/abc/def` + would all match the prefix `/abc`, but the path + `/abcd` would not. \n ReplacePrefixMatch is + only compatible with a `PathPrefix` HTTPRouteMatch. + Using any other HTTPRouteMatch type on the same + HTTPRouteRule will result in the implementation + setting the Accepted Condition for the Route + to `status: False`. \n Request Path | Prefix + Match | Replace Prefix | Modified Path -------------|--------------|----------------|---------- + /foo/bar | /foo | /xyz | + /xyz/bar /foo/bar | /foo | /xyz/ + \ | /xyz/bar /foo/bar | /foo/ | + /xyz | /xyz/bar /foo/bar | /foo/ + \ | /xyz/ | /xyz/bar /foo | + /foo | /xyz | /xyz /foo/ | + /foo | /xyz | /xyz/ /foo/bar + \ | /foo | | /bar + /foo/ | /foo | + | / /foo | /foo | + | / /foo/ | /foo | / | + / /foo | /foo | / | + /" + maxLength: 1024 + type: string + type: + description: "Type defines the type of path modifier. + Additional types may be added in a future release + of the API. \n Note that values may be added + to this enum, implementations must ensure that + unknown values will not cause a crash. \n Unknown + values here must result in the implementation + setting the Accepted Condition for the Route + to `status: False`, with a Reason of `UnsupportedValue`." + enum: + - ReplaceFullPath + - ReplacePrefixMatch + type: string + required: + - type + type: object + x-kubernetes-validations: + - message: replaceFullPath must be specified when + type is set to 'ReplaceFullPath' + rule: 'self.type == ''ReplaceFullPath'' ? has(self.replaceFullPath) + : true' + - message: type must be 'ReplaceFullPath' when replaceFullPath + is set + rule: 'has(self.replaceFullPath) ? self.type == + ''ReplaceFullPath'' : true' + - message: replacePrefixMatch must be specified when + type is set to 'ReplacePrefixMatch' + rule: 'self.type == ''ReplacePrefixMatch'' ? has(self.replacePrefixMatch) + : true' + - message: type must be 'ReplacePrefixMatch' when + replacePrefixMatch is set + rule: 'has(self.replacePrefixMatch) ? self.type + == ''ReplacePrefixMatch'' : true' + type: object required: - type type: object + x-kubernetes-validations: + - message: filter.requestHeaderModifier must be nil if the + filter.type is not RequestHeaderModifier + rule: '!(has(self.requestHeaderModifier) && self.type != + ''RequestHeaderModifier'')' + - message: filter.requestHeaderModifier must be specified + for RequestHeaderModifier filter.type + rule: '!(!has(self.requestHeaderModifier) && self.type == + ''RequestHeaderModifier'')' + - message: filter.responseHeaderModifier must be nil if the + filter.type is not ResponseHeaderModifier + rule: '!(has(self.responseHeaderModifier) && self.type != + ''ResponseHeaderModifier'')' + - message: filter.responseHeaderModifier must be specified + for ResponseHeaderModifier filter.type + rule: '!(!has(self.responseHeaderModifier) && self.type + == ''ResponseHeaderModifier'')' + - message: filter.requestMirror must be nil if the filter.type + is not RequestMirror + rule: '!(has(self.requestMirror) && self.type != ''RequestMirror'')' + - message: filter.requestMirror must be specified for RequestMirror + filter.type + rule: '!(!has(self.requestMirror) && self.type == ''RequestMirror'')' + - message: filter.requestRedirect must be nil if the filter.type + is not RequestRedirect + rule: '!(has(self.requestRedirect) && self.type != ''RequestRedirect'')' + - message: filter.requestRedirect must be specified for RequestRedirect + filter.type + rule: '!(!has(self.requestRedirect) && self.type == ''RequestRedirect'')' + - message: filter.urlRewrite must be nil if the filter.type + is not URLRewrite + rule: '!(has(self.urlRewrite) && self.type != ''URLRewrite'')' + - message: filter.urlRewrite must be specified for URLRewrite + filter.type + rule: '!(!has(self.urlRewrite) && self.type == ''URLRewrite'')' + - message: filter.extensionRef must be nil if the filter.type + is not ExtensionRef + rule: '!(has(self.extensionRef) && self.type != ''ExtensionRef'')' + - message: filter.extensionRef must be specified for ExtensionRef + filter.type + rule: '!(!has(self.extensionRef) && self.type == ''ExtensionRef'')' maxItems: 16 type: array + x-kubernetes-validations: + - message: May specify either httpRouteFilterRequestRedirect + or httpRouteFilterRequestRewrite, but not both + rule: '!(self.exists(f, f.type == ''RequestRedirect'') && + self.exists(f, f.type == ''URLRewrite''))' + - message: RequestHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestHeaderModifier'') + ? self.exists_one(f, f.type == ''RequestHeaderModifier'') + : true' + - message: ResponseHeaderModifier filter cannot be repeated + rule: 'self.exists(f, f.type == ''ResponseHeaderModifier'') + ? self.exists_one(f, f.type == ''ResponseHeaderModifier'') + : true' + - message: RequestRedirect filter cannot be repeated + rule: 'self.exists(f, f.type == ''RequestRedirect'') ? self.exists_one(f, + f.type == ''RequestRedirect'') : true' + - message: URLRewrite filter cannot be repeated + rule: 'self.exists(f, f.type == ''URLRewrite'') ? self.exists_one(f, + f.type == ''URLRewrite'') : true' matches: default: - path: @@ -2368,19 +4128,21 @@ spec: request. \n Proxy or Load Balancer routing configuration generated from HTTPRoutes MUST prioritize matches based on the following criteria, continuing on ties. Across all rules specified on - applicable Routes, precedence must be given to the match with - the largest number of: \n * Characters in a matching path. - * Header matches. * Query param matches. \n If ties still - exist across multiple Routes, matching precedence MUST be - determined in order of the following criteria, continuing - on ties: \n * The oldest Route based on creation timestamp. - * The Route appearing first in alphabetical order by \"{namespace}/{name}\". - \n If ties still exist within an HTTPRoute, matching precedence - MUST be granted to the FIRST matching rule (in list order) - with a match meeting the above criteria. \n When no rules - matching a request have been successfully attached to the - parent a request is coming from, a HTTP 404 status code MUST - be returned." + applicable Routes, precedence must be given to the match having: + \n * \"Exact\" path match. * \"Prefix\" path match with largest + number of characters. * Method match. * Largest number of + header matches. * Largest number of query param matches. \n + Note: The precedence of RegularExpression path matches are + implementation-specific. \n If ties still exist across multiple + Routes, matching precedence MUST be determined in order of + the following criteria, continuing on ties: \n * The oldest + Route based on creation timestamp. * The Route appearing first + in alphabetical order by \"{namespace}/{name}\". \n If ties + still exist within an HTTPRoute, matching precedence MUST + be granted to the FIRST matching rule (in list order) with + a match meeting the above criteria. \n When no rules matching + a request have been successfully attached to the parent a + request is coming from, a HTTP 404 status code MUST be returned." items: description: "HTTPRouteMatch defines the predicate used to match requests to a given action. Multiple match types are @@ -2488,6 +4250,53 @@ spec: maxLength: 1024 type: string type: object + x-kubernetes-validations: + - message: value must be an absolute path and start with + '/' when type one of ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.startsWith(''/'') : true' + - message: must not contain '//' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''//'') : true' + - message: must not contain '/./' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/./'') : true' + - message: must not contain '/../' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''/../'') : true' + - message: must not contain '%2f' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2f'') : true' + - message: must not contain '%2F' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''%2F'') : true' + - message: must not contain '#' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.contains(''#'') : true' + - message: must not end with '/..' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/..'') : true' + - message: must not end with '/.' when type one of ['Exact', + 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? !self.value.endsWith(''/.'') : true' + - message: type must be one of ['Exact', 'PathPrefix', + 'RegularExpression'] + rule: self.type == 'Exact' || self.type == 'PathPrefix' + || self.type == 'RegularExpression' + - message: must only contain valid characters (matching + ^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$) + for types ['Exact', 'PathPrefix'] + rule: '(self.type == ''Exact'' || self.type == ''PathPrefix'') + ? self.value.matches(r"""^(?:[-A-Za-z0-9/._~!$&''()*+,;=:@]|[%][0-9a-fA-F]{2})+$""") + : true' queryParams: description: "QueryParams specifies HTTP query parameter matchers. Multiple match values are ANDed together, @@ -2518,6 +4327,7 @@ spec: differences in the implementations." maxLength: 256 minLength: 1 + pattern: ^[A-Za-z0-9!#$%&'*+\-.^_\x60|~]+$ type: string type: default: Exact @@ -2553,6 +4363,46 @@ spec: maxItems: 8 type: array type: object + x-kubernetes-validations: + - message: RequestRedirect filter must not be used together with + backendRefs + rule: '(has(self.backendRefs) && size(self.backendRefs) > 0) ? + (!has(self.filters) || self.filters.all(f, !has(f.requestRedirect))): + true' + - message: When using RequestRedirect filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + ? ((size(self.matches) != 1 || !has(self.matches[0].path) || + self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: When using URLRewrite filter with path.replacePrefixMatch, + exactly one PathPrefix match must be specified + rule: '(has(self.filters) && self.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' + - message: Within backendRefs, when using RequestRedirect filter + with path.replacePrefixMatch, exactly one PathPrefix match must + be specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.requestRedirect) + && has(f.requestRedirect.path) && f.requestRedirect.path.type + == ''ReplacePrefixMatch'' && has(f.requestRedirect.path.replacePrefixMatch))) + )) ? ((size(self.matches) != 1 || !has(self.matches[0].path) + || self.matches[0].path.type != ''PathPrefix'') ? false : true) + : true' + - message: Within backendRefs, When using URLRewrite filter with + path.replacePrefixMatch, exactly one PathPrefix match must be + specified + rule: '(has(self.backendRefs) && self.backendRefs.exists_one(b, + (has(b.filters) && b.filters.exists_one(f, has(f.urlRewrite) + && has(f.urlRewrite.path) && f.urlRewrite.path.type == ''ReplacePrefixMatch'' + && has(f.urlRewrite.path.replacePrefixMatch))) )) ? ((size(self.matches) + != 1 || !has(self.matches[0].path) || self.matches[0].path.type + != ''PathPrefix'') ? false : true) : true' maxItems: 16 type: array type: object @@ -2701,9 +4551,11 @@ spec: type: string kind: default: Gateway - description: "Kind is kind of the referent. \n Support: - Core (Gateway) \n Support: Implementation-specific (Other - Resources)" + description: "Kind is kind of the referent. \n There are + two kinds of parent resources with \"Core\" support: \n + * Gateway (Gateway conformance profile) * Service (Mesh + conformance profile, experimental, ClusterIP Services + only) \n Support for other resources is Implementation-Specific." maxLength: 63 minLength: 1 pattern: ^[a-zA-Z]([-a-zA-Z0-9]*[a-zA-Z0-9])?$ @@ -2723,7 +4575,16 @@ spec: in the namespace they are referring to. For example: Gateway has the AllowedRoutes field, and ReferenceGrant provides a generic way to enable any other kind of cross-namespace - reference. \n Support: Core" + reference. \n ParentRefs from a Route to a Service in + the same namespace are \"producer\" routes, which apply + default routing rules to inbound connections from any + namespace to the Service. \n ParentRefs from a Route to + a Service in a different namespace are \"consumer\" routes, + and these routing rules are only applied to outbound connections + originating from the same namespace as the Route, for + which the intended destination of the connections are + a Service targeted as a ParentRef of the Route. \n Support: + Core" maxLength: 63 minLength: 1 pattern: ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ @@ -2734,8 +4595,13 @@ spec: is interpreted as the following: \n * Gateway: Listener Name. When both Port (experimental) and SectionName are specified, the name and port of the selected listener - must match both specified values. \n Implementations MAY - choose to support attaching Routes to other resources. + must match both specified values. * Service: Port Name. + When both Port (experimental) and SectionName are specified, + the name and port of the selected listener must match + both specified values. Note that attaching Routes to Services + as Parents is part of experimental Mesh support and is + not supported for any other purpose. \n Implementations + MAY choose to support attaching Routes to other resources. If that is the case, they MUST clearly document how SectionName is interpreted. \n When unspecified (empty string), this will reference the entire resource. For the purpose of diff --git a/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml b/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml index b57e096591..91b75ecad4 100644 --- a/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml +++ b/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml @@ -2,8 +2,8 @@ apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: - api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1538 - gateway.networking.k8s.io/bundle-version: v0.6.0 + api-approved.kubernetes.io: https://github.com/kubernetes-sigs/gateway-api/pull/1923 + gateway.networking.k8s.io/bundle-version: v0.7.1-dev gateway.networking.k8s.io/channel: standard creationTimestamp: null name: referencegrants.gateway.networking.k8s.io @@ -24,6 +24,9 @@ spec: - jsonPath: .metadata.creationTimestamp name: Age type: date + deprecated: true + deprecationWarning: The v1alpha2 version of ReferenceGrant has been deprecated + and will be removed in a future release of the API. Please upgrade to v1beta1. name: v1alpha2 schema: openAPIV3Schema: @@ -32,13 +35,17 @@ spec: namespace as the policy. \n Each ReferenceGrant can be used to represent a unique trust relationship. Additional Reference Grants can be used to add to the set of trusted sources of inbound references for the namespace - they are defined within. \n All cross-namespace references in Gateway API - (with the exception of cross-namespace Gateway-route attachment) require - a ReferenceGrant. \n ReferenceGrant is a form of runtime verification allowing - users to assert which cross-namespace object references are permitted. Implementations - that support ReferenceGrant MUST NOT permit cross-namespace references which - have no grant, and MUST respond to the removal of a grant by revoking the - access that the grant allowed. \n Support: Core" + they are defined within. \n A ReferenceGrant is required for all cross-namespace + references in Gateway API (with the exception of cross-namespace Route-Gateway + attachment, which is governed by the AllowedRoutes configuration on the + Gateway, and cross-namespace Service ParentRefs on a \"consumer\" mesh Route, + which defines routing rules applicable only to workloads in the Route namespace). + ReferenceGrants allowing a reference from a Route to a Service are only + applicable to BackendRefs. \n ReferenceGrant is a form of runtime verification + allowing users to assert which cross-namespace object references are permitted. + Implementations that support ReferenceGrant MUST NOT permit cross-namespace + references which have no grant, and MUST respond to the removal of a grant + by revoking the access that the grant allowed. \n Support: Core" properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation @@ -143,7 +150,7 @@ spec: type: object type: object served: true - storage: true + storage: false subresources: {} - additionalPrinterColumns: - jsonPath: .metadata.creationTimestamp @@ -268,7 +275,7 @@ spec: type: object type: object served: true - storage: false + storage: true subresources: {} status: acceptedNames: diff --git a/config/webhook/0-namespace.yaml b/config/webhook/0-namespace.yaml index 3dbea03a71..c837108d96 100644 --- a/config/webhook/0-namespace.yaml +++ b/config/webhook/0-namespace.yaml @@ -2,3 +2,7 @@ apiVersion: v1 kind: Namespace metadata: name: gateway-system + labels: + pod-security.kubernetes.io/enforce: restricted + pod-security.kubernetes.io/audit: restricted + pod-security.kubernetes.io/warn: restricted diff --git a/config/webhook/admission_webhook.yaml b/config/webhook/admission_webhook.yaml index 5929278663..6ea55d1d92 100644 --- a/config/webhook/admission_webhook.yaml +++ b/config/webhook/admission_webhook.yaml @@ -3,22 +3,22 @@ kind: ValidatingWebhookConfiguration metadata: name: gateway-api-admission webhooks: - - name: validate.gateway.networking.k8s.io - matchPolicy: Equivalent - rules: - - operations: [ "CREATE" , "UPDATE" ] - apiGroups: [ "gateway.networking.k8s.io" ] - apiVersions: [ "v1alpha2", "v1beta1" ] - resources: [ "gateways", "gatewayclasses", "httproutes" ] - failurePolicy: Fail - sideEffects: None - admissionReviewVersions: - - v1 - clientConfig: - service: - name: gateway-api-admission-server - namespace: gateway-system - path: "/validate" +- name: validate.gateway.networking.k8s.io + matchPolicy: Equivalent + rules: + - operations: [ "CREATE" , "UPDATE" ] + apiGroups: [ "gateway.networking.k8s.io" ] + apiVersions: [ "v1alpha2", "v1beta1" ] + resources: [ "gateways", "gatewayclasses", "httproutes" ] + failurePolicy: Fail + sideEffects: None + admissionReviewVersions: + - v1 + clientConfig: + service: + name: gateway-api-admission-server + namespace: gateway-system + path: "/validate" --- apiVersion: v1 kind: Service @@ -30,9 +30,9 @@ metadata: spec: type: ClusterIP ports: - - name: https-webhook - port: 443 - targetPort: 8443 + - name: https-webhook + port: 443 + targetPort: 8443 selector: name: gateway-api-admission-server --- @@ -55,32 +55,41 @@ spec: name: gateway-api-admission-server spec: containers: - - name: webhook - image: gcr.io/k8s-staging-gateway-api/admission-server:v0.6.0 - imagePullPolicy: Always - args: - - -logtostderr - - --tlsCertFile=/etc/certs/cert - - --tlsKeyFile=/etc/certs/key - - -v=10 - - 2>&1 - ports: - - containerPort: 8443 - name: webhook - resources: - limits: - memory: 50Mi - cpu: 100m - requests: - memory: 50Mi - cpu: 100m - volumeMounts: - - name: webhook-certs - mountPath: /etc/certs - readOnly: true - securityContext: - readOnlyRootFilesystem: true - volumes: + - name: webhook + image: registry.k8s.io/gateway-api/admission-server:v0.7.1 + imagePullPolicy: IfNotPresent + args: + - -logtostderr + - --tlsCertFile=/etc/certs/cert + - --tlsKeyFile=/etc/certs/key + - -v=10 + - 2>&1 + ports: + - containerPort: 8443 + name: webhook + resources: + limits: + memory: 50Mi + cpu: 100m + requests: + memory: 50Mi + cpu: 100m + volumeMounts: - name: webhook-certs - secret: - secretName: gateway-api-admission + mountPath: /etc/certs + readOnly: true + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 65532 + runAsGroup: 65532 + capabilities: + drop: + - "ALL" + seccompProfile: + type: RuntimeDefault + volumes: + - name: webhook-certs + secret: + secretName: gateway-api-admission diff --git a/config/webhook/certificate_config.yaml b/config/webhook/certificate_config.yaml index 9ee7450a93..c8f72659b7 100644 --- a/config/webhook/certificate_config.yaml +++ b/config/webhook/certificate_config.yaml @@ -13,13 +13,13 @@ metadata: labels: name: gateway-api rules: - - apiGroups: - - admissionregistration.k8s.io - resources: - - validatingwebhookconfigurations - verbs: - - get - - update +- apiGroups: + - admissionregistration.k8s.io + resources: + - validatingwebhookconfigurations + verbs: + - get + - update --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -33,9 +33,9 @@ roleRef: kind: ClusterRole name: gateway-api-admission subjects: - - kind: ServiceAccount - name: gateway-api-admission - namespace: gateway-system +- kind: ServiceAccount + name: gateway-api-admission + namespace: gateway-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role @@ -46,13 +46,13 @@ metadata: name: gateway-api-webhook namespace: gateway-system rules: - - apiGroups: - - '' - resources: - - secrets - verbs: - - get - - create +- apiGroups: + - '' + resources: + - secrets + verbs: + - get + - create --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding @@ -67,9 +67,9 @@ roleRef: kind: Role name: gateway-api-admission subjects: - - kind: ServiceAccount - name: gateway-api-admission - namespace: gateway-system +- kind: ServiceAccount + name: gateway-api-admission + namespace: gateway-system --- apiVersion: batch/v1 kind: Job @@ -87,24 +87,36 @@ spec: name: gateway-api-webhook spec: containers: - - name: create - image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1 - imagePullPolicy: IfNotPresent - args: - - create - - --host=gateway-api-admission-server,gateway-api-admission-server.gateway-system.svc - - --namespace=gateway-system - - --secret-name=gateway-api-admission - env: - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace + - name: create + image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1 + imagePullPolicy: IfNotPresent + args: + - create + - --host=gateway-api-admission-server,gateway-api-admission-server.gateway-system.svc + - --namespace=gateway-system + - --secret-name=gateway-api-admission + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 2000 + runAsGroup: 2000 + capabilities: + drop: + - "ALL" + seccompProfile: + type: RuntimeDefault restartPolicy: OnFailure serviceAccountName: gateway-api-admission securityContext: runAsNonRoot: true runAsUser: 2000 + runAsGroup: 2000 --- apiVersion: batch/v1 kind: Job @@ -121,24 +133,36 @@ spec: name: gateway-api-webhook spec: containers: - - name: patch - image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1 - imagePullPolicy: IfNotPresent - args: - - patch - - --webhook-name=gateway-api-admission - - --namespace=gateway-system - - --patch-mutating=false - - --patch-validating=true - - --secret-name=gateway-api-admission - - --patch-failure-policy=Fail - env: - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace + - name: patch + image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1 + imagePullPolicy: IfNotPresent + args: + - patch + - --webhook-name=gateway-api-admission + - --namespace=gateway-system + - --patch-mutating=false + - --patch-validating=true + - --secret-name=gateway-api-admission + - --patch-failure-policy=Fail + env: + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 2000 + runAsGroup: 2000 + capabilities: + drop: + - "ALL" + seccompProfile: + type: RuntimeDefault restartPolicy: OnFailure serviceAccountName: gateway-api-admission securityContext: runAsNonRoot: true runAsUser: 2000 + runAsGroup: 2000 diff --git a/conformance/OWNERS b/conformance/OWNERS index f866d12bd1..d418082b53 100644 --- a/conformance/OWNERS +++ b/conformance/OWNERS @@ -2,9 +2,9 @@ # See the OWNERS_ALIASES file at https://github.com/kubernetes-sigs/gateway-api/blob/main/OWNERS_ALIASES for a list of members for each alias. approvers: - - sig-network-leads - - gateway-api-maintainers + - gateway-api-conformance-approvers + - gateway-api-mesh-leads reviewers: - - gateway-api-maintainers + - gateway-api-conformance-reviewers - gateway-api-mesh-leads diff --git a/conformance/apis/v1alpha1/conformancereport.go b/conformance/apis/v1alpha1/conformancereport.go new file mode 100644 index 0000000000..d3cb97b425 --- /dev/null +++ b/conformance/apis/v1alpha1/conformancereport.go @@ -0,0 +1,77 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// ConformanceReport is a report of conformance testing results including the +// specific conformance profiles that were tested and the results of the tests +// with summaries and statistics. +type ConformanceReport struct { + metav1.TypeMeta `json:",inline"` + Implementation `json:"implementation"` + + // Date indicates the date that this report was generated. + Date string `json:"date"` + + // GatewayAPIVersion indicates which release version of Gateway API this + // test report was made for. + GatewayAPIVersion string `json:"gatewayAPIVersion"` + + // ProfileReports is a list of the individual reports for each conformance + // profile that was enabled for a test run. + ProfileReports []ProfileReport `json:"profiles"` +} + +// Implementation provides metadata information on the downstream +// implementation of Gateway API which ran conformance tests. +type Implementation struct { + // Organization refers to the company, group or individual which maintains + // the named implementation. Organizations can provide reports for any + // number of distinct Gateway API implementations they maintain, but need + // to identify themselves using this organization field for grouping. + Organization string `json:"organization"` + + // Project indicates the name of the project or repository for a Gateway API + // implementation. + Project string `json:"project"` + + // URL indicates a human-usable URL where more information about the + // implementation can be found. For open source projects this should + // generally link to the code repository. + URL string `json:"url"` + + // Version indicates the version of the implementation that was used for + // testing. This should generally be a semver version when applicable. + Version string `json:"version"` + + // Contact is contact information for the maintainers so that Gateway API + // maintainers can get ahold of them as needed. Ideally this should be + // Github usernames (in the form of `@`) or team names (in the + // form of `@/`), but when that's not possible it can be email + // addresses. + // Rather than Github usernames or email addresses you can provide a URL to the relevant + // support pages for the project. Ideally this would be something like the issue creation page + // on a repository, but for projects without a publicly exposed repository a general support + // page URL can be provided. + Contact []string `json:"contact"` +} diff --git a/conformance/apis/v1alpha1/doc.go b/conformance/apis/v1alpha1/doc.go new file mode 100644 index 0000000000..d9581574ae --- /dev/null +++ b/conformance/apis/v1alpha1/doc.go @@ -0,0 +1,39 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// V1alpha1 includes alpha maturity API types and utilities for creating and +// handling the results of conformance test runs. These types are _only_ +// intended for use by the conformance test suite OR external test suites that +// are written in Golang and execute the conformance test suite as a Golang +// library. +// +// Note that currently all sub-packages are considered "experimental" in that +// they aren't intended for general use or to be distributed as part of a +// release so there is no way to use them by default when using the Golang +// library at this time. If you don't know for sure that you want to use these +// features, then you should not use them. If you would like to opt into these +// unreleased features use Go build tags to enable them, e.g.: +// +// $ GOFLAGS='-tags=experimental' go test ./conformance/... -args ${CONFORMANCE_ARGS} +// +// Please note that everything here is considered experimental and subject to +// change. Expect breaking changes and/or complete removals if you start using +// them. + +package v1alpha1 diff --git a/conformance/apis/v1alpha1/profilereport.go b/conformance/apis/v1alpha1/profilereport.go new file mode 100644 index 0000000000..f321435eac --- /dev/null +++ b/conformance/apis/v1alpha1/profilereport.go @@ -0,0 +1,73 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +// ProfileReport is the generated report for the test results of a specific +// named conformance profile. +type ProfileReport struct { + // Name indicates the name of the conformance profile (e.g. "HTTP", + // "TLS", "Mesh", e.t.c.). + Name string `json:"name"` + + // Core indicates the core support level which includes the set of tests + // which are the minimum the implementation must pass to be considered at + // all conformant. + Core Status `json:"core"` + + // Extended indicates the extended support level which includes additional + // optional features which the implementation may choose to implement + // support for, but are not required. + Extended *ExtendedStatus `json:"extended,omitempty"` +} + +// ExtendedStatus shows the testing results for the extended support level. +type ExtendedStatus struct { + Status `json:",inline"` + + // SupportedFeatures indicates which extended features were flagged as + // supported by the implementation and tests will be attempted for. + SupportedFeatures []string `json:"supportedFeatures,omitempty"` + + // UnsupportedFeatures indicates which extended features the implementation + // does not have support for and therefore will not attempt to test. + UnsupportedFeatures []string `json:"unsupportedFeatures,omitempty"` +} + +// Status includes details on the results of a test. +type Status struct { + Result `json:"result"` + + // Summary is a human-readable message intended for end-users to understand + // the overall status at a glance. + Summary string `json:"summary"` + + // Statistics includes numerical statistics on the result of the test run. + Statistics `json:"statistics"` + + // SkippedTests indicates which tests were explicitly disabled in the test + // suite. Skipping tests for Core level support implicitly identifies the + // results as being partial and the implementation will not be considered + // conformant at any level. + SkippedTests []string `json:"skippedTests,omitempty"` + + // FailedTests indicates which tests were failing during the execution of + // test suite. + FailedTests []string `json:"failedTests,omitempty"` +} diff --git a/conformance/apis/v1alpha1/result.go b/conformance/apis/v1alpha1/result.go new file mode 100644 index 0000000000..8f47272af9 --- /dev/null +++ b/conformance/apis/v1alpha1/result.go @@ -0,0 +1,38 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +// Result is a simple high-level summary describing the conclusion of a test +// run. +type Result string + +var ( + // Success indicates that the test run concluded in all required tests + // passing. + Success Result = "success" + + // Partial indicates that the test run concluded in some of the required + // tests passing without any failures, but some were skipped. + Partial Result = "partial" + + // Failure indicates that the test run concluded in one ore more tests + // failing to complete successfully. + Failure Result = "failure" +) diff --git a/conformance/apis/v1alpha1/statistics.go b/conformance/apis/v1alpha1/statistics.go new file mode 100644 index 0000000000..e64451a674 --- /dev/null +++ b/conformance/apis/v1alpha1/statistics.go @@ -0,0 +1,35 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1alpha1 + +// Statistics includes numerical summaries of the number of conformance tests +// that passed, failed or were intentionally skipped. +type Statistics struct { + // Passed indicates how many tests completed successfully. + Passed uint32 + + // Skipped indicates how many tests were intentionally not run, whether due + // to lack of feature support or whether they were explicitly disabled in + // the test suite. + Skipped uint32 + + // Failed indicates how many tests were unsuccessful. + Failed uint32 +} diff --git a/conformance/base/manifests.yaml b/conformance/base/manifests.yaml index 7ab20d575f..674710bc03 100644 --- a/conformance/base/manifests.yaml +++ b/conformance/base/manifests.yaml @@ -30,6 +30,27 @@ spec: --- apiVersion: gateway.networking.k8s.io/v1beta1 kind: Gateway +metadata: + name: same-namespace-with-https-listener + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: https + port: 443 + protocol: HTTPS + allowedRoutes: + namespaces: + from: Same + tls: + certificateRefs: + - group: "" + kind: Secret + name: tls-validity-checks-certificate + namespace: gateway-conformance-infra +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway metadata: name: all-namespaces namespace: gateway-conformance-infra @@ -70,9 +91,9 @@ spec: selector: app: infra-backend-v1 ports: - - protocol: TCP - port: 8080 - targetPort: 3000 + - protocol: TCP + port: 8080 + targetPort: 3000 --- apiVersion: apps/v1 kind: Deployment @@ -117,9 +138,9 @@ spec: selector: app: infra-backend-v2 ports: - - protocol: TCP - port: 8080 - targetPort: 3000 + - protocol: TCP + port: 8080 + targetPort: 3000 --- apiVersion: apps/v1 kind: Deployment @@ -163,9 +184,9 @@ spec: selector: app: infra-backend-v3 ports: - - protocol: TCP - port: 8080 - targetPort: 3000 + - protocol: TCP + port: 8080 + targetPort: 3000 --- apiVersion: apps/v1 kind: Deployment @@ -209,9 +230,9 @@ spec: selector: app: tls-backend ports: - - protocol: TCP - port: 443 - targetPort: 8443 + - protocol: TCP + port: 443 + targetPort: 8443 --- apiVersion: apps/v1 kind: Deployment @@ -237,18 +258,18 @@ spec: - name: secret-volume mountPath: /etc/secret-volume env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - - name: TLS_SERVER_CERT - value: /etc/secret-volume/crt - - name: TLS_SERVER_PRIVKEY - value: /etc/secret-volume/key + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: TLS_SERVER_CERT + value: /etc/secret-volume/crt + - name: TLS_SERVER_PRIVKEY + value: /etc/secret-volume/key resources: requests: cpu: 10m @@ -271,6 +292,68 @@ metadata: --- apiVersion: v1 kind: Service +metadata: + name: tls-backend + namespace: gateway-conformance-app-backend +spec: + selector: + app: tls-backend + ports: + - protocol: TCP + port: 443 + targetPort: 8443 +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: tls-backend + namespace: gateway-conformance-app-backend + labels: + app: tls-backend +spec: + replicas: 1 + selector: + matchLabels: + app: tls-backend + template: + metadata: + labels: + app: tls-backend + spec: + containers: + - name: tls-backend + image: gcr.io/k8s-staging-ingressconformance/echoserver:v20221109-7ee2f3e + volumeMounts: + - name: secret-volume + mountPath: /etc/secret-volume + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: TLS_SERVER_CERT + value: /etc/secret-volume/crt + - name: TLS_SERVER_PRIVKEY + value: /etc/secret-volume/key + resources: + requests: + cpu: 10m + volumes: + - name: secret-volume + secret: + secretName: tls-passthrough-checks-certificate + items: + - key: tls.crt + path: crt + - key: tls.key + path: key +--- +apiVersion: v1 +kind: Service metadata: name: app-backend-v1 namespace: gateway-conformance-app-backend @@ -278,9 +361,9 @@ spec: selector: app: app-backend-v1 ports: - - protocol: TCP - port: 8080 - targetPort: 3000 + - protocol: TCP + port: 8080 + targetPort: 3000 --- apiVersion: apps/v1 kind: Deployment @@ -324,9 +407,9 @@ spec: selector: app: app-backend-v2 ports: - - protocol: TCP - port: 8080 - targetPort: 3000 + - protocol: TCP + port: 8080 + targetPort: 3000 --- apiVersion: apps/v1 kind: Deployment @@ -377,9 +460,9 @@ spec: selector: app: web-backend ports: - - protocol: TCP - port: 8080 - targetPort: 3000 + - protocol: TCP + port: 8080 + targetPort: 3000 --- apiVersion: apps/v1 kind: Deployment diff --git a/conformance/conformance_test.go b/conformance/conformance_test.go index 91a788c667..32072b89c6 100644 --- a/conformance/conformance_test.go +++ b/conformance/conformance_test.go @@ -18,7 +18,6 @@ limitations under the License. package conformance_test import ( - "strings" "testing" "sigs.k8s.io/gateway-api/apis/v1alpha2" @@ -27,6 +26,7 @@ import ( "sigs.k8s.io/gateway-api/conformance/utils/flags" "sigs.k8s.io/gateway-api/conformance/utils/suite" + "k8s.io/client-go/kubernetes" _ "k8s.io/client-go/plugin/pkg/client/auth" "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/client/config" @@ -41,35 +41,38 @@ func TestConformance(t *testing.T) { if err != nil { t.Fatalf("Error initializing Kubernetes client: %v", err) } + clientset, err := kubernetes.NewForConfig(cfg) + if err != nil { + t.Fatalf("Error initializing Kubernetes REST client: %v", err) + } + v1alpha2.AddToScheme(client.Scheme()) v1beta1.AddToScheme(client.Scheme()) - supportedFeatures := parseSupportedFeatures(*flags.SupportedFeatures) - exemptFeatures := parseSupportedFeatures(*flags.ExemptFeatures) - for feature := range exemptFeatures { - supportedFeatures[feature] = false - } + supportedFeatures := suite.ParseSupportedFeatures(*flags.SupportedFeatures) + exemptFeatures := suite.ParseSupportedFeatures(*flags.ExemptFeatures) + skipTests := suite.ParseSkipTests(*flags.SkipTests) + namespaceLabels := suite.ParseNamespaceLabels(*flags.NamespaceLabels) - t.Logf("Running conformance tests with %s GatewayClass\n cleanup: %t\n debug: %t\n supported features: [%v]\n exempt features: [%v]", - *flags.GatewayClassName, *flags.CleanupBaseResources, *flags.ShowDebug, *flags.SupportedFeatures, *flags.ExemptFeatures) + t.Logf("Running conformance tests with %s GatewayClass\n cleanup: %t\n debug: %t\n enable all features: %t \n supported features: [%v]\n exempt features: [%v]", + *flags.GatewayClassName, *flags.CleanupBaseResources, *flags.ShowDebug, *flags.EnableAllSupportedFeatures, *flags.SupportedFeatures, *flags.ExemptFeatures) cSuite := suite.New(suite.Options{ - Client: client, - GatewayClassName: *flags.GatewayClassName, - Debug: *flags.ShowDebug, - CleanupBaseResources: *flags.CleanupBaseResources, - SupportedFeatures: supportedFeatures, + Client: client, + RestConfig: cfg, + // This clientset is needed in addition to the client only because + // controller-runtime client doesn't support non CRUD sub-resources yet (https://github.com/kubernetes-sigs/controller-runtime/issues/452). + Clientset: clientset, + GatewayClassName: *flags.GatewayClassName, + Debug: *flags.ShowDebug, + CleanupBaseResources: *flags.CleanupBaseResources, + SupportedFeatures: supportedFeatures, + ExemptFeatures: exemptFeatures, + EnableAllSupportedFeatures: *flags.EnableAllSupportedFeatures, + NamespaceLabels: namespaceLabels, + SkipTests: skipTests, }) cSuite.Setup(t) - cSuite.Run(t, tests.ConformanceTests) -} -// parseSupportedFeatures parses flag arguments and converts the string to -// map[suite.SupportedFeature]bool -func parseSupportedFeatures(f string) map[suite.SupportedFeature]bool { - res := map[suite.SupportedFeature]bool{} - for _, value := range strings.Split(f, ",") { - res[suite.SupportedFeature(value)] = true - } - return res + cSuite.Run(t, tests.ConformanceTests) } diff --git a/conformance/embed.go b/conformance/embed.go index 11b8a12a99..067654755b 100644 --- a/conformance/embed.go +++ b/conformance/embed.go @@ -18,5 +18,5 @@ package conformance import "embed" -//go:embed tests/* base/* +//go:embed tests/* base/* mesh/* var Manifests embed.FS diff --git a/conformance/experimental_conformance_test.go b/conformance/experimental_conformance_test.go new file mode 100644 index 0000000000..01bcacc547 --- /dev/null +++ b/conformance/experimental_conformance_test.go @@ -0,0 +1,150 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package conformance_test + +import ( + "os" + "testing" + + "k8s.io/apimachinery/pkg/util/sets" + "k8s.io/client-go/kubernetes" + "k8s.io/client-go/rest" + "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/controller-runtime/pkg/client/config" + "sigs.k8s.io/yaml" + + "sigs.k8s.io/gateway-api/apis/v1alpha2" + "sigs.k8s.io/gateway-api/apis/v1beta1" + confv1a1 "sigs.k8s.io/gateway-api/conformance/apis/v1alpha1" + "sigs.k8s.io/gateway-api/conformance/tests" + "sigs.k8s.io/gateway-api/conformance/utils/flags" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +var ( + cfg *rest.Config + k8sClientset *kubernetes.Clientset + mgrClient client.Client + supportedFeatures sets.Set[suite.SupportedFeature] + exemptFeatures sets.Set[suite.SupportedFeature] + namespaceLabels map[string]string + implementation *confv1a1.Implementation + conformanceProfiles sets.Set[suite.ConformanceProfileName] + skipTests []string +) + +func TestExperimentalConformance(t *testing.T) { + var err error + cfg, err = config.GetConfig() + if err != nil { + t.Fatalf("Error loading Kubernetes config: %v", err) + } + mgrClient, err = client.New(cfg, client.Options{}) + if err != nil { + t.Fatalf("Error initializing Kubernetes client: %v", err) + } + k8sClientset, err = kubernetes.NewForConfig(cfg) + if err != nil { + t.Fatalf("Error initializing Kubernetes REST client: %v", err) + } + + v1alpha2.AddToScheme(mgrClient.Scheme()) + v1beta1.AddToScheme(mgrClient.Scheme()) + + // standard conformance flags + supportedFeatures = suite.ParseSupportedFeatures(*flags.SupportedFeatures) + exemptFeatures = suite.ParseSupportedFeatures(*flags.ExemptFeatures) + skipTests = suite.ParseSkipTests(*flags.SkipTests) + namespaceLabels = suite.ParseNamespaceLabels(*flags.NamespaceLabels) + + // experimental conformance flags + conformanceProfiles = suite.ParseConformanceProfiles(*flags.ConformanceProfiles) + + if conformanceProfiles.Len() > 0 { + // if some conformance profiles have been set, run the experimental conformance suite... + implementation, err = suite.ParseImplementation( + *flags.ImplementationOrganization, + *flags.ImplementationProject, + *flags.ImplementationUrl, + *flags.ImplementationVersion, + *flags.ImplementationContact, + ) + if err != nil { + t.Fatalf("Error parsing implementation's details: %v", err) + } + testExperimentalConformance(t) + } else { + // ...otherwise run the standard conformance suite. + t.Run("standard conformance tests", TestConformance) + } +} + +func testExperimentalConformance(t *testing.T) { + t.Logf("Running experimental conformance tests with %s GatewayClass\n cleanup: %t\n debug: %t\n enable all features: %t \n supported features: [%v]\n exempt features: [%v]", + *flags.GatewayClassName, *flags.CleanupBaseResources, *flags.ShowDebug, *flags.EnableAllSupportedFeatures, *flags.SupportedFeatures, *flags.ExemptFeatures) + + cSuite, err := suite.NewExperimentalConformanceTestSuite( + suite.ExperimentalConformanceOptions{ + Options: suite.Options{ + Client: mgrClient, + RestConfig: cfg, + // This clientset is needed in addition to the client only because + // controller-runtime client doesn't support non CRUD sub-resources yet (https://github.com/kubernetes-sigs/controller-runtime/issues/452). + Clientset: k8sClientset, + GatewayClassName: *flags.GatewayClassName, + Debug: *flags.ShowDebug, + CleanupBaseResources: *flags.CleanupBaseResources, + SupportedFeatures: supportedFeatures, + ExemptFeatures: exemptFeatures, + EnableAllSupportedFeatures: *flags.EnableAllSupportedFeatures, + NamespaceLabels: namespaceLabels, + SkipTests: skipTests, + }, + Implementation: *implementation, + ConformanceProfiles: conformanceProfiles, + }) + if err != nil { + t.Fatalf("error creating experimental conformance test suite: %v", err) + } + + cSuite.Setup(t) + cSuite.Run(t, tests.ConformanceTests) + report, err := cSuite.Report() + if err != nil { + t.Fatalf("error generating conformance profile report: %v", err) + } + writeReport(t.Logf, *report, *flags.ReportOutput) +} + +func writeReport(logf func(string, ...any), report confv1a1.ConformanceReport, output string) error { + rawReport, err := yaml.Marshal(report) + if err != nil { + return err + } + + if output != "" { + if err = os.WriteFile(output, rawReport, 0644); err != nil { + return err + } + } + logf("Conformance report:\n%s", string(rawReport)) + + return nil +} diff --git a/conformance/mesh/manifests.yaml b/conformance/mesh/manifests.yaml new file mode 100644 index 0000000000..0b26dcc474 --- /dev/null +++ b/conformance/mesh/manifests.yaml @@ -0,0 +1,172 @@ +--- +apiVersion: v1 +kind: Namespace +metadata: + name: gateway-conformance-mesh + labels: + gateway-conformance: mesh +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: echo-v1 + namespace: gateway-conformance-mesh + labels: + app: echo +spec: + selector: + matchLabels: + app: echo + version: v1 + template: + metadata: + labels: + app: echo + version: v1 + spec: + containers: + - name: echo + image: gcr.io/k8s-staging-gateway-api/echo-server:v20230505-v0.7.0-rc1-10-g497e67da + imagePullPolicy: IfNotPresent + args: + - --tcp=9090 + - --port=80 + - --port=8080 + - --grpc=7070 + - --port=443 + - --tls=443 + - --crt=/cert.crt + - --key=/cert.key +--- +apiVersion: v1 +kind: Service +metadata: + name: echo-v1 + namespace: gateway-conformance-mesh +spec: + selector: + app: echo + version: v1 + ports: + - name: http + port: 80 + appProtocol: http + - name: http-alt + port: 8080 + appProtocol: http + - name: https + port: 443 + - name: tcp + port: 9090 + - name: grpc + port: 7070 + appProtocol: grpc +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: echo-v2 + namespace: gateway-conformance-mesh + labels: + app: echo +spec: + selector: + matchLabels: + app: echo + version: v2 + template: + metadata: + labels: + app: echo + version: v2 + spec: + containers: + - name: echo + image: gcr.io/k8s-staging-gateway-api/echo-server:v20230505-v0.7.0-rc1-10-g497e67da + imagePullPolicy: IfNotPresent + args: + - --tcp=9090 + - --port=80 + - --port=8080 + - --grpc=7070 + - --port=443 + - --tls=443 + - --crt=/cert.crt + - --key=/cert.key +--- +apiVersion: v1 +kind: Service +metadata: + name: echo-v2 + namespace: gateway-conformance-mesh +spec: + selector: + app: echo + version: v2 + ports: + - name: http + port: 80 + appProtocol: http + - name: http-alt + port: 8080 + appProtocol: http + - name: https + port: 443 + - name: tcp + port: 9090 + - name: grpc + port: 7070 + appProtocol: grpc +--- +apiVersion: v1 +kind: Service +metadata: + name: echo + namespace: gateway-conformance-mesh +spec: + selector: + app: echo + ports: + - name: http + port: 80 + appProtocol: http + - name: http-alt + port: 8080 + appProtocol: http + - name: https + port: 443 + - name: tcp + port: 9090 + - name: grpc + port: 7070 + appProtocol: grpc +--- +apiVersion: v1 +kind: Namespace +metadata: + name: gateway-conformance-mesh-consumer + labels: + gateway-conformance: mesh-consumer +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: echo-v1 + namespace: gateway-conformance-mesh-consumer + labels: + app: echo +spec: + selector: + matchLabels: + app: echo + version: v1 + template: + metadata: + labels: + app: echo + version: v1 + spec: + containers: + - name: echo + image: gcr.io/k8s-staging-gateway-api/echo-server:v20230505-v0.7.0-rc1-10-g497e67da + imagePullPolicy: IfNotPresent diff --git a/conformance/results/cilium.yaml b/conformance/results/cilium.yaml new file mode 100644 index 0000000000..caa4159189 --- /dev/null +++ b/conformance/results/cilium.yaml @@ -0,0 +1,46 @@ +apiVersion: gateway.networking.k8s.io/v1alpha1 +date: "2023-08-06T12:21:32+10:00" +gatewayAPIVersion: v0.7.1 +implementation: + contact: + - https://github.com/cilium/community/blob/main/roles/Maintainers.md + organization: cilium + project: cilium + url: https://github.com/cilium/cilium + version: v1.14.0 +kind: ConformanceReport +profiles: + - core: + result: success + statistics: + Failed: 0 + Passed: 28 + Skipped: 0 + summary: "First conformance result for Cilium" + extended: + result: success + statistics: + Failed: 0 + Passed: 3 + Skipped: 0 + summary: "" + supportedFeatures: + - HTTPResponseHeaderModification + - HTTPRouteQueryParamMatching + - HTTPRouteMethodMatching + unsupportedFeatures: + - HTTPRouteSchemeRedirect + - HTTPRouteHostRewrite + - HTTPRouteRequestMirror + - HTTPRoutePathRedirect + - HTTPRoutePathRewrite + - HTTPRoutePortRedirect + name: HTTP + - core: + result: success + statistics: + Failed: 0 + Passed: 11 + Skipped: 0 + summary: "First conformance result for Cilium" + name: TLS diff --git a/conformance/results/contour.yaml b/conformance/results/contour.yaml new file mode 100644 index 0000000000..85b86c9937 --- /dev/null +++ b/conformance/results/contour.yaml @@ -0,0 +1,45 @@ +apiVersion: gateway.networking.k8s.io/v1alpha1 +date: "2023-07-25T22:11:56Z" +gatewayAPIVersion: "e9859d057b43d177176a9d3d6b15dbdd27fe7e46" +implementation: + contact: + - '@projectcontour/maintainers' + organization: projectcontour + project: contour + url: https://github.com/projectcontour/contour + version: "f6c112e60a40ceb9d448354473a8fe7d1a726790" +kind: ConformanceReport +profiles: +- core: + result: success + statistics: + Failed: 0 + Passed: 28 + Skipped: 0 + summary: "" + extended: + result: success + statistics: + Failed: 0 + Passed: 9 + Skipped: 0 + summary: "" + supportedFeatures: + - HTTPRouteHostRewrite + - HTTPRouteQueryParamMatching + - HTTPResponseHeaderModification + - HTTPRouteSchemeRedirect + - HTTPRoutePathRedirect + - HTTPRouteMethodMatching + - HTTPRoutePortRedirect + - HTTPRoutePathRewrite + - HTTPRouteRequestMirror + name: HTTP +- core: + result: success + statistics: + Failed: 0 + Passed: 10 + Skipped: 0 + summary: "" + name: TLS diff --git a/conformance/results/istio.yaml b/conformance/results/istio.yaml new file mode 100644 index 0000000000..7346aedf4d --- /dev/null +++ b/conformance/results/istio.yaml @@ -0,0 +1,56 @@ +apiVersion: gateway.networking.k8s.io/v1alpha1 +date: "2023-07-24T14:26:14-07:00" +gatewayAPIVersion: 4c436290f2bfeaeaf0b1a367449e16de3e0addc4 +implementation: + contact: + - '@istio/maintainers' + organization: istio.io + project: istio + url: istio.io + # Istio does not yet official support conformance profiles while it is experimental + version: https://github.com/istio/istio/pull/46148 +kind: ConformanceReport +profiles: +- core: + result: success + statistics: + Failed: 0 + Passed: 28 + Skipped: 0 + summary: "" + extended: + result: success + statistics: + Failed: 0 + Passed: 9 + Skipped: 0 + summary: "" + supportedFeatures: + - HTTPRouteQueryParamMatching + - HTTPRouteMethodMatching + - HTTPRoutePortRedirect + - HTTPRouteRequestMirror + - HTTPResponseHeaderModification + - HTTPRoutePathRedirect + - HTTPRouteHostRewrite + - HTTPRouteSchemeRedirect + - HTTPRoutePathRewrite + name: HTTP +- core: + result: success + statistics: + Failed: 0 + Passed: 10 + Skipped: 0 + summary: "" + name: TLS +- core: + result: partial + skippedTests: + - MeshFrontendHostname + statistics: + Failed: 0 + Passed: 2 + Skipped: 1 + summary: "" + name: MESH diff --git a/conformance/results/kong-kubernetes-ingress-controller.yaml b/conformance/results/kong-kubernetes-ingress-controller.yaml new file mode 100644 index 0000000000..4476fb5241 --- /dev/null +++ b/conformance/results/kong-kubernetes-ingress-controller.yaml @@ -0,0 +1,41 @@ +apiVersion: gateway.networking.k8s.io/v1alpha1 +date: "2023-07-17T14:54:07+02:00" +gatewayAPIVersion: v0.7.1 +implementation: + organization: kong + project: kubernetes-ingress-controller + url: github.com/kong/kubernetes-ingress-controller + version: "2.10.3" + contact: + - https://github.com/Kong/kubernetes-ingress-controller/issues/new/choose +kind: ConformanceReport +profileReports: + - core: + result: partial + skippedTests: + - HTTPRouteHeaderMatching + statistics: + Failed: 0 + Passed: 27 + Skipped: 1 + summary: | + 'Very first conformance profile.' + extended: + result: success + statistics: + Failed: 0 + Passed: 1 + Skipped: 0 + summary: "" + supportedFeatures: + - HTTPRouteMethodMatching + unsupportedFeatures: + - HTTPRoutePathRewrite + - HTTPRouteRequestMirror + - HTTPRoutePortRedirect + - HTTPRoutePathRedirect + - HTTPRouteQueryParamMatching + - HTTPRouteMethodMatching + - HTTPRouteSchemeRedirect + - HTTPRouteHostRewrite + name: HTTP diff --git a/conformance/results/kuma.yaml b/conformance/results/kuma.yaml new file mode 100644 index 0000000000..d7604b85f8 --- /dev/null +++ b/conformance/results/kuma.yaml @@ -0,0 +1,45 @@ +apiVersion: gateway.networking.k8s.io/v1alpha1 +date: "2023-07-27T18:08:52Z" +gatewayAPIVersion: v0.7.1 +implementation: + contact: + - "@kumahq/kuma-maintainers" + organization: kumahq + project: kuma + url: https://github.com/kumahq/kuma + version: 0.0.0-preview.vcf063c043 +kind: ConformanceReport +profiles: + - core: + result: success + statistics: + Failed: 0 + Passed: 29 + Skipped: 0 + summary: "" + extended: + result: success + statistics: + Failed: 0 + Passed: 9 + Skipped: 0 + summary: "" + supportedFeatures: + - HTTPRoutePathRewrite + - HTTPResponseHeaderModification + - HTTPRouteSchemeRedirect + - HTTPRouteHostRewrite + - HTTPRouteQueryParamMatching + - HTTPRoutePortRedirect + - HTTPRoutePathRedirect + - HTTPRouteMethodMatching + - HTTPRouteRequestMirror + name: HTTP + - core: + result: success + statistics: + Failed: 0 + Passed: 3 + Skipped: 0 + summary: "" + name: MESH diff --git a/conformance/results/nginx-kubernetes-gateway.yaml b/conformance/results/nginx-kubernetes-gateway.yaml new file mode 100644 index 0000000000..7693ef8c63 --- /dev/null +++ b/conformance/results/nginx-kubernetes-gateway.yaml @@ -0,0 +1,38 @@ +apiVersion: gateway.networking.k8s.io/v1alpha1 +date: "2023-07-25T20:48:53Z" +gatewayAPIVersion: v0.7.1 +implementation: + contact: + - https://github.com/nginxinc/nginx-kubernetes-gateway/discussions/new/choose + organization: nginxinc + project: nginx-kubernetes-gateway + url: github.com/nginxinc/nginx-kubernetes-gateway + version: "v0.5.0" +kind: ConformanceReport +profiles: +- core: + result: success + statistics: + Failed: 0 + Passed: 28 + Skipped: 0 + summary: All core HTTP features are supported + extended: + result: success + statistics: + Failed: 0 + Passed: 4 + Skipped: 0 + summary: Some extended HTTP features are supported + supportedFeatures: + - HTTPRouteMethodMatching + - HTTPRouteSchemeRedirect + - HTTPRouteQueryParamMatching + - HTTPRoutePortRedirect + unsupportedFeatures: + - HTTPResponseHeaderModification + - HTTPRouteHostRewrite + - HTTPRoutePathRedirect + - HTTPRoutePathRewrite + - HTTPRouteRequestMirror + name: HTTP diff --git a/conformance/tests/gateway-invalid-route-kind.go b/conformance/tests/gateway-invalid-route-kind.go index 33777ea453..8d2fe26926 100644 --- a/conformance/tests/gateway-invalid-route-kind.go +++ b/conformance/tests/gateway-invalid-route-kind.go @@ -34,7 +34,10 @@ func init() { var GatewayInvalidRouteKind = suite.ConformanceTest{ ShortName: "GatewayInvalidRouteKind", Description: "A Gateway in the gateway-conformance-infra namespace should fail to become ready an invalid Route kind is specified.", - Manifests: []string{"tests/gateway-invalid-route-kind.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + }, + Manifests: []string{"tests/gateway-invalid-route-kind.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { t.Run("Gateway listener should have a false ResolvedRefs condition with reason InvalidRouteKinds and no supportedKinds", func(t *testing.T) { gwNN := types.NamespacedName{Name: "gateway-only-invalid-route-kind", Namespace: "gateway-conformance-infra"} @@ -46,6 +49,7 @@ var GatewayInvalidRouteKind = suite.ConformanceTest{ Status: metav1.ConditionFalse, Reason: string(v1beta1.ListenerReasonInvalidRouteKinds), }}, + AttachedRoutes: 0, }} kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) @@ -64,6 +68,7 @@ var GatewayInvalidRouteKind = suite.ConformanceTest{ Status: metav1.ConditionFalse, Reason: string(v1beta1.ListenerReasonInvalidRouteKinds), }}, + AttachedRoutes: 0, }} kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) diff --git a/conformance/tests/gateway-invalid-route-kind.yaml b/conformance/tests/gateway-invalid-route-kind.yaml index 0af669fc27..7318d962d6 100644 --- a/conformance/tests/gateway-invalid-route-kind.yaml +++ b/conformance/tests/gateway-invalid-route-kind.yaml @@ -13,7 +13,7 @@ spec: namespaces: from: All kinds: - - kind: InvalidRoute + - kind: InvalidRoute --- apiVersion: gateway.networking.k8s.io/v1beta1 kind: Gateway @@ -30,5 +30,5 @@ spec: namespaces: from: All kinds: - - kind: InvalidRoute - - kind: HTTPRoute + - kind: InvalidRoute + - kind: HTTPRoute diff --git a/conformance/tests/gateway-invalid-tls-certificateref.go b/conformance/tests/gateway-invalid-tls-certificateref.go index dad35605a0..8119e3a673 100644 --- a/conformance/tests/gateway-invalid-tls-certificateref.go +++ b/conformance/tests/gateway-invalid-tls-certificateref.go @@ -34,7 +34,10 @@ func init() { var GatewayInvalidTLSConfiguration = suite.ConformanceTest{ ShortName: "GatewayInvalidTLSConfiguration", Description: "A Gateway should fail to become ready if the Gateway has an invalid TLS configuration", - Manifests: []string{"tests/gateway-invalid-tls-certificateref.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + }, + Manifests: []string{"tests/gateway-invalid-tls-certificateref.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { listeners := []v1beta1.ListenerStatus{{ Name: v1beta1.SectionName("https"), @@ -47,6 +50,7 @@ var GatewayInvalidTLSConfiguration = suite.ConformanceTest{ Status: metav1.ConditionFalse, Reason: string(v1beta1.ListenerReasonInvalidCertificateRef), }}, + AttachedRoutes: 0, }} testCases := []struct { diff --git a/conformance/tests/gateway-modify-listeners.go b/conformance/tests/gateway-modify-listeners.go new file mode 100644 index 0000000000..fffed977fd --- /dev/null +++ b/conformance/tests/gateway-modify-listeners.go @@ -0,0 +1,185 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "context" + "testing" + "time" + + "github.com/stretchr/testify/require" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" + + "sigs.k8s.io/gateway-api/apis/v1beta1" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, GatewayModifyListeners) +} + +var GatewayModifyListeners = suite.ConformanceTest{ + ShortName: "GatewayModifyListeners", + Description: "A Gateway in the gateway-conformance-infra namespace should handle adding and removing listeners.", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + }, + Manifests: []string{"tests/gateway-modify-listeners.yaml"}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + t.Run("should be able to add a listener that then becomes available for routing traffic", func(t *testing.T) { + gwNN := types.NamespacedName{Name: "gateway-add-listener", Namespace: "gateway-conformance-infra"} + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + + namespaces := []string{"gateway-conformance-infra"} + kubernetes.NamespacesMustBeReady(t, s.Client, s.TimeoutConfig, namespaces) + + // verify that the implementation is tracking the most recent resource changes + kubernetes.GatewayMustHaveLatestConditions(t, s.Client, s.TimeoutConfig, gwNN) + + original := &v1beta1.Gateway{} + err := s.Client.Get(ctx, gwNN, original) + require.NoErrorf(t, err, "error getting Gateway: %v", err) + + all := v1beta1.NamespacesFromAll + + mutate := original.DeepCopy() + + // add a new listener to the Gateway spec + hostname := v1beta1.Hostname("data.test.com") + mutate.Spec.Listeners = append(mutate.Spec.Listeners, v1beta1.Listener{ + Name: "http", + Port: 80, + Protocol: v1beta1.HTTPProtocolType, + Hostname: &hostname, + AllowedRoutes: &v1beta1.AllowedRoutes{ + Namespaces: &v1beta1.RouteNamespaces{From: &all}, + }, + }) + + err = s.Client.Patch(ctx, mutate, client.MergeFrom(original)) + require.NoErrorf(t, err, "error patching the Gateway: %v", err) + + // Ensure the generation and observedGeneration sync up + kubernetes.NamespacesMustBeReady(t, s.Client, s.TimeoutConfig, namespaces) + + listeners := []v1beta1.ListenerStatus{ + { + Name: v1beta1.SectionName("https"), + SupportedKinds: []v1beta1.RouteGroupKind{{ + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }}, + Conditions: []metav1.Condition{{ + Type: string(v1beta1.ListenerConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }}, + AttachedRoutes: 1, + }, + { + Name: v1beta1.SectionName("http"), + SupportedKinds: []v1beta1.RouteGroupKind{{ + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }}, + Conditions: []metav1.Condition{{ + Type: string(v1beta1.ListenerConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }}, + AttachedRoutes: 1, + }, + } + + kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) + + // verify that the implementation continues to keep up to date with the resource changes we've been making + kubernetes.GatewayMustHaveLatestConditions(t, s.Client, s.TimeoutConfig, gwNN) + + updated := &v1beta1.Gateway{} + err = s.Client.Get(ctx, gwNN, updated) + require.NoErrorf(t, err, "error getting Gateway: %v", err) + + require.NotEqual(t, original.Generation, updated.Generation, "generation should change after an update") + }) + + t.Run("should be able to remove listeners, which would then stop routing the relevant traffic", func(t *testing.T) { + gwNN := types.NamespacedName{Name: "gateway-remove-listener", Namespace: "gateway-conformance-infra"} + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + + namespaces := []string{"gateway-conformance-infra"} + kubernetes.NamespacesMustBeReady(t, s.Client, s.TimeoutConfig, namespaces) + + // verify that the implementation is tracking the most recent resource changes + kubernetes.GatewayMustHaveLatestConditions(t, s.Client, s.TimeoutConfig, gwNN) + + original := &v1beta1.Gateway{} + err := s.Client.Get(ctx, gwNN, original) + require.NoErrorf(t, err, "error getting Gateway: %v", err) + + mutate := original.DeepCopy() + require.Equalf(t, 2, len(mutate.Spec.Listeners), "the gateway must have 2 listeners") + + // remove the "https" Gateway listener, leaving only the "http" listener + var newListeners []v1beta1.Listener + for _, listener := range mutate.Spec.Listeners { + if listener.Name == "http" { + newListeners = append(newListeners, listener) + } + } + mutate.Spec.Listeners = newListeners + + err = s.Client.Patch(ctx, mutate, client.MergeFrom(original)) + require.NoErrorf(t, err, "error patching the Gateway: %v", err) + + // Ensure the generation and observedGeneration sync up + kubernetes.NamespacesMustBeReady(t, s.Client, s.TimeoutConfig, namespaces) + + listeners := []v1beta1.ListenerStatus{ + { + Name: v1beta1.SectionName("http"), + SupportedKinds: []v1beta1.RouteGroupKind{{ + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }}, + Conditions: []metav1.Condition{{ + Type: string(v1beta1.ListenerConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }}, + AttachedRoutes: 1, + }, + } + + kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) + + // verify that the implementation continues to keep up to date with the resource changes we've been making + kubernetes.GatewayMustHaveLatestConditions(t, s.Client, s.TimeoutConfig, gwNN) + + updated := &v1beta1.Gateway{} + err = s.Client.Get(ctx, gwNN, updated) + require.NoErrorf(t, err, "error getting Gateway: %v", err) + + require.NotEqual(t, original.Generation, updated.Generation, "generation should change after an update") + }) + }, +} diff --git a/conformance/tests/gateway-modify-listeners.yaml b/conformance/tests/gateway-modify-listeners.yaml new file mode 100644 index 0000000000..aaa53c6d08 --- /dev/null +++ b/conformance/tests/gateway-modify-listeners.yaml @@ -0,0 +1,79 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway-add-listener + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: https + port: 443 + protocol: HTTPS + hostname: "secure.test.com" + allowedRoutes: + namespaces: + from: All + tls: + certificateRefs: + - group: "" + kind: Secret + name: tls-validity-checks-certificate + namespace: gateway-conformance-infra +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-1 + namespace: gateway-conformance-infra +spec: + parentRefs: + - kind: Gateway + name: gateway-add-listener + namespace: gateway-conformance-infra + rules: + - backendRefs: + - name: infra-backend-v1 + port: 8080 +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway-remove-listener + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: https + port: 443 + protocol: HTTPS + hostname: "secure.test.com" + allowedRoutes: + namespaces: + from: All + tls: + certificateRefs: + - group: "" + kind: Secret + name: tls-validity-checks-certificate + namespace: gateway-conformance-infra + - name: http + port: 80 + protocol: HTTP + allowedRoutes: + namespaces: + from: All +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-2 + namespace: gateway-conformance-infra +spec: + parentRefs: + - kind: Gateway + name: gateway-remove-listener + namespace: gateway-conformance-infra + rules: + - backendRefs: + - name: infra-backend-v1 + port: 8080 diff --git a/conformance/tests/gateway-observed-generation-bump.go b/conformance/tests/gateway-observed-generation-bump.go index d7b5d87bda..9f63772ba0 100644 --- a/conformance/tests/gateway-observed-generation-bump.go +++ b/conformance/tests/gateway-observed-generation-bump.go @@ -23,6 +23,7 @@ import ( "github.com/stretchr/testify/require" "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/gateway-api/apis/v1beta1" "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" @@ -35,10 +36,13 @@ func init() { var GatewayObservedGenerationBump = suite.ConformanceTest{ ShortName: "GatewayObservedGenerationBump", - Description: "A Gateway in the gateway-conformance-infra namespace should update the observedGeneration in all of it's Status.Conditions after an update to the spec", - Manifests: []string{"tests/gateway-observed-generation-bump.yaml"}, + Description: "A Gateway in the gateway-conformance-infra namespace should update the observedGeneration in all of its Status.Conditions after an update to the spec", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportGatewayPort8080, + }, + Manifests: []string{"tests/gateway-observed-generation-bump.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { - gwNN := types.NamespacedName{Name: "gateway-observed-generation-bump", Namespace: "gateway-conformance-infra"} t.Run("observedGeneration should increment", func(t *testing.T) { @@ -46,15 +50,15 @@ var GatewayObservedGenerationBump = suite.ConformanceTest{ defer cancel() namespaces := []string{"gateway-conformance-infra"} - kubernetes.NamespacesMustBeAccepted(t, s.Client, s.TimeoutConfig, namespaces) + kubernetes.NamespacesMustBeReady(t, s.Client, s.TimeoutConfig, namespaces) + + // Sanity check + kubernetes.GatewayMustHaveLatestConditions(t, s.Client, s.TimeoutConfig, gwNN) original := &v1beta1.Gateway{} err := s.Client.Get(ctx, gwNN, original) require.NoErrorf(t, err, "error getting Gateway: %v", err) - // Sanity check - kubernetes.GatewayMustHaveLatestConditions(t, original) - all := v1beta1.NamespacesFromAll mutate := original.DeepCopy() @@ -69,19 +73,19 @@ var GatewayObservedGenerationBump = suite.ConformanceTest{ }, }) - err = s.Client.Update(ctx, mutate) - require.NoErrorf(t, err, "error updating the Gateway: %v", err) + err = s.Client.Patch(ctx, mutate, client.MergeFrom(original)) + require.NoErrorf(t, err, "error patching the Gateway: %v", err) // Ensure the generation and observedGeneration sync up - kubernetes.NamespacesMustBeAccepted(t, s.Client, s.TimeoutConfig, namespaces) + kubernetes.NamespacesMustBeReady(t, s.Client, s.TimeoutConfig, namespaces) + + // Sanity check + kubernetes.GatewayMustHaveLatestConditions(t, s.Client, s.TimeoutConfig, gwNN) updated := &v1beta1.Gateway{} err = s.Client.Get(ctx, gwNN, updated) require.NoErrorf(t, err, "error getting Gateway: %v", err) - // Sanity check - kubernetes.GatewayMustHaveLatestConditions(t, updated) - require.NotEqual(t, original.Generation, updated.Generation, "generation should change after an update") }) }, diff --git a/conformance/tests/gateway-secret-invalid-reference-grant.go b/conformance/tests/gateway-secret-invalid-reference-grant.go index 4ae38fd55f..87c10f3af3 100644 --- a/conformance/tests/gateway-secret-invalid-reference-grant.go +++ b/conformance/tests/gateway-secret-invalid-reference-grant.go @@ -34,8 +34,11 @@ func init() { var GatewaySecretInvalidReferenceGrant = suite.ConformanceTest{ ShortName: "GatewaySecretInvalidReferenceGrant", Description: "A Gateway in the gateway-conformance-infra namespace should fail to become ready if the Gateway has a certificateRef for a Secret in the gateway-conformance-web-backend namespace and a ReferenceGrant exists but does not grant permission to that specific Secret", - Features: []suite.SupportedFeature{suite.SupportReferenceGrant}, - Manifests: []string{"tests/gateway-secret-invalid-reference-grant.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/gateway-secret-invalid-reference-grant.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { gwNN := types.NamespacedName{Name: "gateway-secret-invalid-reference-grant", Namespace: "gateway-conformance-infra"} @@ -51,6 +54,7 @@ var GatewaySecretInvalidReferenceGrant = suite.ConformanceTest{ Status: metav1.ConditionFalse, Reason: string(v1beta1.ListenerReasonRefNotPermitted), }}, + AttachedRoutes: 0, }} kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) diff --git a/conformance/tests/gateway-secret-missing-reference-grant.go b/conformance/tests/gateway-secret-missing-reference-grant.go index d9c3254fa9..2b4e747bb2 100644 --- a/conformance/tests/gateway-secret-missing-reference-grant.go +++ b/conformance/tests/gateway-secret-missing-reference-grant.go @@ -34,8 +34,11 @@ func init() { var GatewaySecretMissingReferenceGrant = suite.ConformanceTest{ ShortName: "GatewaySecretMissingReferenceGrant", Description: "A Gateway in the gateway-conformance-infra namespace should fail to become programmed if the Gateway has a certificateRef for a Secret in the gateway-conformance-web-backend namespace and a ReferenceGrant granting permission to the Secret does not exist", - Features: []suite.SupportedFeature{suite.SupportReferenceGrant}, - Manifests: []string{"tests/gateway-secret-missing-reference-grant.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/gateway-secret-missing-reference-grant.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { gwNN := types.NamespacedName{Name: "gateway-secret-missing-reference-grant", Namespace: "gateway-conformance-infra"} @@ -51,6 +54,7 @@ var GatewaySecretMissingReferenceGrant = suite.ConformanceTest{ Status: metav1.ConditionFalse, Reason: string(v1beta1.ListenerReasonRefNotPermitted), }}, + AttachedRoutes: 0, }} kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) diff --git a/conformance/tests/gateway-secret-reference-grant-all-in-namespace.go b/conformance/tests/gateway-secret-reference-grant-all-in-namespace.go index c91344a7f8..1fb5fed88c 100644 --- a/conformance/tests/gateway-secret-reference-grant-all-in-namespace.go +++ b/conformance/tests/gateway-secret-reference-grant-all-in-namespace.go @@ -34,10 +34,13 @@ func init() { var GatewaySecretReferenceGrantAllInNamespace = suite.ConformanceTest{ ShortName: "GatewaySecretReferenceGrantAllInNamespace", Description: "A Gateway in the gateway-conformance-infra namespace should become programmed if the Gateway has a certificateRef for a Secret in the gateway-conformance-web-backend namespace and a ReferenceGrant granting permission to all Secrets in the namespace exists", - Features: []suite.SupportedFeature{suite.SupportReferenceGrant}, - Manifests: []string{"tests/gateway-secret-reference-grant-all-in-namespace.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/gateway-secret-reference-grant-all-in-namespace.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { - gwNN := types.NamespacedName{Name: "gateway-secret-reference-grant", Namespace: "gateway-conformance-infra"} + gwNN := types.NamespacedName{Name: "gateway-secret-reference-grant-all-in-namespace", Namespace: "gateway-conformance-infra"} t.Run("Gateway listener should have a true ResolvedRefs condition and a true Programmed condition", func(t *testing.T) { listeners := []v1beta1.ListenerStatus{{ @@ -50,9 +53,10 @@ var GatewaySecretReferenceGrantAllInNamespace = suite.ConformanceTest{ { Type: string(v1beta1.ListenerConditionProgrammed), Status: metav1.ConditionTrue, - Reason: string(v1beta1.ListenerConditionProgrammed), + Reason: string(v1beta1.ListenerReasonProgrammed), }, }, + AttachedRoutes: 0, }} kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) diff --git a/conformance/tests/gateway-secret-reference-grant-all-in-namespace.yaml b/conformance/tests/gateway-secret-reference-grant-all-in-namespace.yaml index aaed62f8b8..034c9f7f5b 100644 --- a/conformance/tests/gateway-secret-reference-grant-all-in-namespace.yaml +++ b/conformance/tests/gateway-secret-reference-grant-all-in-namespace.yaml @@ -1,7 +1,7 @@ apiVersion: gateway.networking.k8s.io/v1beta1 kind: Gateway metadata: - name: gateway-secret-reference-grant + name: gateway-secret-reference-grant-all-in-namespace namespace: gateway-conformance-infra spec: gatewayClassName: "{GATEWAY_CLASS_NAME}" diff --git a/conformance/tests/gateway-secret-reference-grant-specific.go b/conformance/tests/gateway-secret-reference-grant-specific.go index 649d17e4db..6d9c092fc5 100644 --- a/conformance/tests/gateway-secret-reference-grant-specific.go +++ b/conformance/tests/gateway-secret-reference-grant-specific.go @@ -34,10 +34,13 @@ func init() { var GatewaySecretReferenceGrantSpecific = suite.ConformanceTest{ ShortName: "GatewaySecretReferenceGrantSpecific", Description: "A Gateway in the gateway-conformance-infra namespace should become programmed if the Gateway has a certificateRef for a Secret in the gateway-conformance-web-backend namespace and a ReferenceGrant granting permission to the specific Secret exists", - Features: []suite.SupportedFeature{suite.SupportReferenceGrant}, - Manifests: []string{"tests/gateway-secret-reference-grant-specific.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/gateway-secret-reference-grant-specific.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { - gwNN := types.NamespacedName{Name: "gateway-secret-reference-grant", Namespace: "gateway-conformance-infra"} + gwNN := types.NamespacedName{Name: "gateway-secret-reference-grant-specific", Namespace: "gateway-conformance-infra"} t.Run("Gateway listener should have a true ResolvedRefs condition and a true Programmed condition", func(t *testing.T) { listeners := []v1beta1.ListenerStatus{{ @@ -53,6 +56,7 @@ var GatewaySecretReferenceGrantSpecific = suite.ConformanceTest{ Reason: string(v1beta1.ListenerReasonProgrammed), }, }, + AttachedRoutes: 0, }} kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) diff --git a/conformance/tests/gateway-secret-reference-grant-specific.yaml b/conformance/tests/gateway-secret-reference-grant-specific.yaml index 0eb1e5fbec..0f15710900 100644 --- a/conformance/tests/gateway-secret-reference-grant-specific.yaml +++ b/conformance/tests/gateway-secret-reference-grant-specific.yaml @@ -1,7 +1,7 @@ apiVersion: gateway.networking.k8s.io/v1beta1 kind: Gateway metadata: - name: gateway-secret-reference-grant + name: gateway-secret-reference-grant-specific namespace: gateway-conformance-infra spec: gatewayClassName: "{GATEWAY_CLASS_NAME}" diff --git a/conformance/tests/gateway-with-attached-routes.go b/conformance/tests/gateway-with-attached-routes.go new file mode 100644 index 0000000000..81de412d7b --- /dev/null +++ b/conformance/tests/gateway-with-attached-routes.go @@ -0,0 +1,114 @@ +/* +Copyright 2022 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + + "sigs.k8s.io/gateway-api/apis/v1beta1" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, GatewayWithAttachedRoutes) +} + +var GatewayWithAttachedRoutes = suite.ConformanceTest{ + ShortName: "GatewayWithAttachedRoutes", + Description: "A Gateway in the gateway-conformance-infra namespace should be attached to routes.", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + }, + Manifests: []string{"tests/gateway-with-attached-routes.yaml"}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + t.Run("Gateway listener should have one valid http routes attached", func(t *testing.T) { + gwNN := types.NamespacedName{Name: "gateway-with-one-attached-route", Namespace: "gateway-conformance-infra"} + listeners := []v1beta1.ListenerStatus{{ + Name: v1beta1.SectionName("http"), + SupportedKinds: []v1beta1.RouteGroupKind{{ + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }}, + Conditions: []metav1.Condition{{ + Type: string(v1beta1.ListenerConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }}, + AttachedRoutes: 1, + }} + + kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) + }) + + t.Run("Gateway listener should have two valid http routes attached", func(t *testing.T) { + gwNN := types.NamespacedName{Name: "gateway-with-two-attached-routes", Namespace: "gateway-conformance-infra"} + listeners := []v1beta1.ListenerStatus{{ + Name: v1beta1.SectionName("http"), + SupportedKinds: []v1beta1.RouteGroupKind{{ + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }}, + Conditions: []metav1.Condition{{ + Type: string(v1beta1.ListenerConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }}, + AttachedRoutes: 2, + }} + + kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) + }) + + t.Run("Gateway listener should have attached route by specifying the sectionName", func(t *testing.T) { + gwNN := types.NamespacedName{Name: "gateway-with-two-listeners-and-one-attached-route", Namespace: "gateway-conformance-infra"} + listeners := []v1beta1.ListenerStatus{ + { + Name: v1beta1.SectionName("http-unattached"), + SupportedKinds: []v1beta1.RouteGroupKind{{ + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }}, + Conditions: []metav1.Condition{{ + Type: string(v1beta1.ListenerConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }}, + AttachedRoutes: 0, + }, + { + Name: v1beta1.SectionName("http"), + SupportedKinds: []v1beta1.RouteGroupKind{{ + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }}, + Conditions: []metav1.Condition{{ + Type: string(v1beta1.ListenerConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }}, + AttachedRoutes: 1, + }, + } + + kubernetes.GatewayStatusMustHaveListeners(t, s.Client, s.TimeoutConfig, gwNN, listeners) + }) + }, +} diff --git a/conformance/tests/gateway-with-attached-routes.yaml b/conformance/tests/gateway-with-attached-routes.yaml new file mode 100644 index 0000000000..c5a572c362 --- /dev/null +++ b/conformance/tests/gateway-with-attached-routes.yaml @@ -0,0 +1,129 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway-with-one-attached-route + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: http + port: 80 + protocol: HTTP + allowedRoutes: + kinds: + - kind: HTTPRoute + namespaces: + from: Selector + selector: + matchLabels: + # This label is added automatically as of K8s 1.22 + # to all namespaces + kubernetes.io/metadata.name: gateway-conformance-infra +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-1 + namespace: gateway-conformance-infra +spec: + parentRefs: + - kind: Gateway + name: gateway-with-one-attached-route + namespace: gateway-conformance-infra + rules: + - backendRefs: + - name: infra-backend-v1 + port: 8080 +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway-with-two-attached-routes + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: http + port: 80 + protocol: HTTP + allowedRoutes: + kinds: + - kind: HTTPRoute + namespaces: + from: Selector + selector: + matchLabels: + # This label is added automatically as of K8s 1.22 + # to all namespaces + kubernetes.io/metadata.name: gateway-conformance-infra +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-2 + namespace: gateway-conformance-infra +spec: + parentRefs: + - kind: Gateway + name: gateway-with-two-attached-routes + namespace: gateway-conformance-infra + rules: + - backendRefs: + - name: infra-backend-v1 + port: 8080 +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-3 + namespace: gateway-conformance-infra +spec: + parentRefs: + - kind: Gateway + name: gateway-with-two-attached-routes + namespace: gateway-conformance-infra + rules: + - backendRefs: + - name: infra-backend-v1 + port: 8080 +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway-with-two-listeners-and-one-attached-route + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: http-unattached + port: 8080 + protocol: HTTP + allowedRoutes: + kinds: + - kind: HTTPRoute + namespaces: + from: All + - name: http + port: 80 + protocol: HTTP + allowedRoutes: + kinds: + - kind: HTTPRoute + namespaces: + from: All +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-4 + namespace: gateway-conformance-infra +spec: + parentRefs: + - kind: Gateway + name: gateway-with-two-listeners-and-one-attached-route + namespace: gateway-conformance-infra + sectionName: http + rules: + - backendRefs: + - name: infra-backend-v1 + port: 8080 diff --git a/conformance/tests/gatewayclass-observed-generation-bump.go b/conformance/tests/gatewayclass-observed-generation-bump.go index 0a699ff514..b3e3c693fc 100644 --- a/conformance/tests/gatewayclass-observed-generation-bump.go +++ b/conformance/tests/gatewayclass-observed-generation-bump.go @@ -23,6 +23,7 @@ import ( "github.com/stretchr/testify/require" "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/gateway-api/apis/v1beta1" "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" @@ -34,8 +35,10 @@ func init() { } var GatewayClassObservedGenerationBump = suite.ConformanceTest{ - ShortName: "GatewayClassObservedGenerationBump", - Features: []suite.SupportedFeature{suite.SupportGatewayClassObservedGenerationBump}, + ShortName: "GatewayClassObservedGenerationBump", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + }, Description: "A GatewayClass should update the observedGeneration in all of it's Status.Conditions after an update to the spec", Manifests: []string{"tests/gatewayclass-observed-generation-bump.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { @@ -58,8 +61,8 @@ var GatewayClassObservedGenerationBump = suite.ConformanceTest{ desc := "new" mutate.Spec.Description = &desc - err = s.Client.Update(ctx, mutate) - require.NoErrorf(t, err, "error updating the GatewayClass: %v", err) + err = s.Client.Patch(ctx, mutate, client.MergeFrom(original)) + require.NoErrorf(t, err, "error patching the GatewayClass: %v", err) // Ensure the generation and observedGeneration sync up kubernetes.GWCMustHaveAcceptedConditionAny(t, s.Client, s.TimeoutConfig, gwc.Name) diff --git a/conformance/tests/httproute-cross-namespace.go b/conformance/tests/httproute-cross-namespace.go index db6ff61109..eaa3fc3562 100644 --- a/conformance/tests/httproute-cross-namespace.go +++ b/conformance/tests/httproute-cross-namespace.go @@ -33,11 +33,16 @@ func init() { var HTTPRouteCrossNamespace = suite.ConformanceTest{ ShortName: "HTTPRouteCrossNamespace", Description: "A single HTTPRoute in the gateway-conformance-web-backend namespace should attach to Gateway in another namespace", - Manifests: []string{"tests/httproute-cross-namespace.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-cross-namespace.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "cross-namespace", Namespace: "gateway-conformance-web-backend"} gwNN := types.NamespacedName{Name: "backend-namespaces", Namespace: "gateway-conformance-infra"} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) t.Run("Simple HTTP request should reach web-backend", func(t *testing.T) { http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, http.ExpectedResponse{ diff --git a/conformance/tests/httproute-disallowed-kind.go b/conformance/tests/httproute-disallowed-kind.go index d072c8b9e9..b334f36f8f 100644 --- a/conformance/tests/httproute-disallowed-kind.go +++ b/conformance/tests/httproute-disallowed-kind.go @@ -34,15 +34,20 @@ func init() { var HTTPRouteDisallowedKind = suite.ConformanceTest{ ShortName: "HTTPRouteDisallowedKind", Description: "A single HTTPRoute in the gateway-conformance-infra namespace should fail to attach to a Gateway with no listeners that allow the HTTPRoute kind", - Features: []suite.SupportedFeature{suite.SupportTLSRoute}, - Manifests: []string{"tests/httproute-disallowed-kind.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportTLSRoute, + }, + Manifests: []string{"tests/httproute-disallowed-kind.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { // This test creates an additional Gateway in the gateway-conformance-infra // namespace so we have to wait for it to be ready. - kubernetes.NamespacesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, []string{"gateway-conformance-infra"}) + kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, []string{"gateway-conformance-infra"}) routeNN := types.NamespacedName{Name: "disallowed-kind", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "tlsroutes-only", Namespace: "gateway-conformance-infra"} + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) t.Run("Route should not have been accepted with reason NotAllowedByListeners", func(t *testing.T) { kubernetes.HTTPRouteMustHaveCondition(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN, metav1.Condition{ diff --git a/conformance/tests/httproute-exact-path-matching.go b/conformance/tests/httproute-exact-path-matching.go index a543834aa5..6e399eb846 100644 --- a/conformance/tests/httproute-exact-path-matching.go +++ b/conformance/tests/httproute-exact-path-matching.go @@ -33,12 +33,17 @@ func init() { var HTTPExactPathMatching = suite.ConformanceTest{ ShortName: "HTTPExactPathMatching", Description: "A single HTTPRoute with exact path matching for different backends", - Manifests: []string{"tests/httproute-exact-path-matching.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-exact-path-matching.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "exact-matching", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{ { @@ -58,6 +63,9 @@ var HTTPExactPathMatching = suite.ConformanceTest{ }, { Request: http.Request{Path: "/two/"}, Response: http.Response{StatusCode: 404}, + }, { + Request: http.Request{Path: "/Two"}, + Response: http.Response{StatusCode: 404}, }, } diff --git a/conformance/tests/httproute-header-matching.go b/conformance/tests/httproute-header-matching.go index 51349f5018..66c9f5795b 100644 --- a/conformance/tests/httproute-header-matching.go +++ b/conformance/tests/httproute-header-matching.go @@ -33,12 +33,17 @@ func init() { var HTTPRouteHeaderMatching = suite.ConformanceTest{ ShortName: "HTTPRouteHeaderMatching", Description: "A single HTTPRoute with header matching for different backends", - Manifests: []string{"tests/httproute-header-matching.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-header-matching.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "header-matching", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{{ Request: http.Request{Path: "/", Headers: map[string]string{"Version": "one"}}, diff --git a/conformance/tests/httproute-hostname-intersection.go b/conformance/tests/httproute-hostname-intersection.go index cd52d4fede..f4adc7bc49 100644 --- a/conformance/tests/httproute-hostname-intersection.go +++ b/conformance/tests/httproute-hostname-intersection.go @@ -35,14 +35,18 @@ func init() { var HTTPRouteHostnameIntersection = suite.ConformanceTest{ ShortName: "HTTPRouteHostnameIntersection", Description: "HTTPRoutes should attach to listeners only if they have intersecting hostnames, and should accept requests only for the intersecting hostnames", - Manifests: []string{"tests/httproute-hostname-intersection.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-hostname-intersection.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" gwNN := types.NamespacedName{Name: "httproute-hostname-intersection", Namespace: ns} // This test creates an additional Gateway in the gateway-conformance-infra // namespace so we have to wait for it to be ready. - kubernetes.NamespacesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, []string{ns}) + kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, []string{ns}) t.Run("HTTPRoutes that do intersect with listener hostnames", func(t *testing.T) { routes := []types.NamespacedName{ @@ -52,6 +56,9 @@ var HTTPRouteHostnameIntersection = suite.ConformanceTest{ {Namespace: ns, Name: "wildcard-host-matches-listener-wildcard-host"}, } gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routes...) + for _, routeNN := range routes { + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) + } var testCases []http.ExpectedResponse @@ -62,6 +69,13 @@ var HTTPRouteHostnameIntersection = suite.ConformanceTest{ Backend: "infra-backend-v1", Namespace: ns, }, + // Port value within the Host header MUST not be considered while + // performing match against hostname. + http.ExpectedResponse{ + Request: http.Request{Host: "very.specific.com:1234", Path: "/s1"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, http.ExpectedResponse{ Request: http.Request{Host: "non.matching.com", Path: "/s1"}, Response: http.Response{StatusCode: 404}, diff --git a/conformance/tests/httproute-hostname-intersection.yaml b/conformance/tests/httproute-hostname-intersection.yaml index 3c6773202b..aa5864be0c 100644 --- a/conformance/tests/httproute-hostname-intersection.yaml +++ b/conformance/tests/httproute-hostname-intersection.yaml @@ -40,7 +40,7 @@ spec: hostnames: - non.matching.com - "*.nonmatchingwildcard.io" - - very.specific.com # matches listener-1's specific host + - very.specific.com # matches listener-1's specific host rules: - matches: - path: @@ -62,9 +62,9 @@ spec: hostnames: - non.matching.com - wildcard.io - - foo.wildcard.io # matches listener-2's wildcard host - - bar.wildcard.io # matches listener-2's wildcard host - - foo.bar.wildcard.io # matches listener-2's wildcard host + - foo.wildcard.io # matches listener-2's wildcard host + - bar.wildcard.io # matches listener-2's wildcard host + - foo.bar.wildcard.io # matches listener-2's wildcard host rules: - matches: - path: diff --git a/conformance/tests/httproute-invalid-backendref-nonexistent.go b/conformance/tests/httproute-invalid-backendref-nonexistent.go index 97f5907b13..318432dea3 100644 --- a/conformance/tests/httproute-invalid-backendref-nonexistent.go +++ b/conformance/tests/httproute-invalid-backendref-nonexistent.go @@ -35,7 +35,11 @@ func init() { var HTTPRouteInvalidNonExistentBackendRef = suite.ConformanceTest{ ShortName: "HTTPRouteInvalidNonExistentBackendRef", Description: "A single HTTPRoute in the gateway-conformance-infra namespace should set a ResolvedRefs status False with reason BackendNotFound and return 500 when binding to a Gateway in the same namespace if the route has a BackendRef Service that does not exist", - Manifests: []string{"tests/httproute-invalid-backendref-nonexistent.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-invalid-backendref-nonexistent.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "invalid-nonexistent-backend-ref", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} @@ -62,6 +66,5 @@ var HTTPRouteInvalidNonExistentBackendRef = suite.ConformanceTest{ Response: http.Response{StatusCode: 500}, }) }) - }, } diff --git a/conformance/tests/httproute-invalid-backendref-unknown-kind.go b/conformance/tests/httproute-invalid-backendref-unknown-kind.go index b100ae85e4..83bad184e3 100644 --- a/conformance/tests/httproute-invalid-backendref-unknown-kind.go +++ b/conformance/tests/httproute-invalid-backendref-unknown-kind.go @@ -35,7 +35,11 @@ func init() { var HTTPRouteInvalidBackendRefUnknownKind = suite.ConformanceTest{ ShortName: "HTTPRouteInvalidBackendRefUnknownKind", Description: "A single HTTPRoute in the gateway-conformance-infra namespace should set a ResolvedRefs status False with reason InvalidKind when attempting to bind to a Gateway in the same namespace if the route has a BackendRef that points to an unknown Kind.", - Manifests: []string{"tests/httproute-invalid-backendref-unknown-kind.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-invalid-backendref-unknown-kind.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "invalid-backend-ref-unknown-kind", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} @@ -63,6 +67,5 @@ var HTTPRouteInvalidBackendRefUnknownKind = suite.ConformanceTest{ Response: http.Response{StatusCode: 500}, }) }) - }, } diff --git a/conformance/tests/httproute-invalid-cross-namespace-backend-ref.go b/conformance/tests/httproute-invalid-cross-namespace-backend-ref.go index 11b2f1d933..8d5dcbf765 100644 --- a/conformance/tests/httproute-invalid-cross-namespace-backend-ref.go +++ b/conformance/tests/httproute-invalid-cross-namespace-backend-ref.go @@ -35,8 +35,12 @@ func init() { var HTTPRouteInvalidCrossNamespaceBackendRef = suite.ConformanceTest{ ShortName: "HTTPRouteInvalidCrossNamespaceBackendRef", Description: "A single HTTPRoute in the gateway-conformance-infra namespace should set a ResolvedRefs status False with reason RefNotPermitted when attempting to bind to a Gateway in the same namespace if the route has a BackendRef Service in the gateway-conformance-web-backend namespace and a ReferenceGrant granting permission to route to that Service does not exist", - Features: []suite.SupportedFeature{suite.SupportReferenceGrant}, - Manifests: []string{"tests/httproute-invalid-cross-namespace-backend-ref.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/httproute-invalid-cross-namespace-backend-ref.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "invalid-cross-namespace-backend-ref", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} @@ -45,7 +49,6 @@ var HTTPRouteInvalidCrossNamespaceBackendRef = suite.ConformanceTest{ gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) t.Run("HTTPRoute with a cross-namespace BackendRef and no ReferenceGrant has a ResolvedRefs Condition with status False and Reason RefNotPermitted", func(t *testing.T) { - resolvedRefsCond := metav1.Condition{ Type: string(v1beta1.RouteConditionResolvedRefs), Status: metav1.ConditionFalse, @@ -64,6 +67,5 @@ var HTTPRouteInvalidCrossNamespaceBackendRef = suite.ConformanceTest{ Response: http.Response{StatusCode: 500}, }) }) - }, } diff --git a/conformance/tests/httproute-invalid-cross-namespace-parent-ref.go b/conformance/tests/httproute-invalid-cross-namespace-parent-ref.go index 77f90302ad..82bf00e5c7 100644 --- a/conformance/tests/httproute-invalid-cross-namespace-parent-ref.go +++ b/conformance/tests/httproute-invalid-cross-namespace-parent-ref.go @@ -34,17 +34,25 @@ func init() { var HTTPRouteInvalidCrossNamespaceParentRef = suite.ConformanceTest{ ShortName: "HTTPRouteInvalidCrossNamespaceParentRef", Description: "A single HTTPRoute in the gateway-conformance-web-backend namespace should fail to attach to a Gateway in another namespace that it is not allowed to", - Manifests: []string{"tests/httproute-invalid-cross-namespace-parent-ref.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-invalid-cross-namespace-parent-ref.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} routeNN := types.NamespacedName{Name: "invalid-cross-namespace-parent-ref", Namespace: "gateway-conformance-web-backend"} + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) - // The Route must have an Accepted Condition with a ParentRefNotPermitted Reason. - t.Run("HTTPRoute with a cross-namespace ParentRef where no ReferenceGrants allows such a reference, has an Accepted Condition with status False and Reason ParentRefNotPermitted", func(t *testing.T) { + // When running conformance tests, implementations are expected to have visibility across all namespaces, and + // must be setting this condition on routes that are not allowed. However, outside of conformance testing, + // it's also valid for implementations to run in modes where they only have access to a limited subset of + // namespaces, in which case they are not obligated to populate this condition on routes they cannot access. + t.Run("HTTPRoute should have an Accepted: false condition with reason NotAllowedByListeners", func(t *testing.T) { acceptedCond := metav1.Condition{ Type: string(v1beta1.RouteConditionAccepted), Status: metav1.ConditionFalse, - Reason: string(v1beta1.RouteReasonParentRefNotPermitted), + Reason: string(v1beta1.RouteReasonNotAllowedByListeners), } kubernetes.HTTPRouteMustHaveCondition(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN, acceptedCond) diff --git a/conformance/tests/httproute-invalid-cross-namespace-parent-ref.yaml b/conformance/tests/httproute-invalid-cross-namespace-parent-ref.yaml index 7e5810b3ff..aed587f2e1 100644 --- a/conformance/tests/httproute-invalid-cross-namespace-parent-ref.yaml +++ b/conformance/tests/httproute-invalid-cross-namespace-parent-ref.yaml @@ -6,6 +6,7 @@ metadata: spec: parentRefs: - name: same-namespace + namespace: gateway-conformance-infra rules: - backendRefs: - name: web-backend diff --git a/conformance/tests/httproute-invalid-parentref-not-matching-listener-port.go b/conformance/tests/httproute-invalid-parentref-not-matching-listener-port.go index dda394d110..a49e44478f 100644 --- a/conformance/tests/httproute-invalid-parentref-not-matching-listener-port.go +++ b/conformance/tests/httproute-invalid-parentref-not-matching-listener-port.go @@ -34,11 +34,16 @@ func init() { var HTTPRouteInvalidParentRefNotMatchingListenerPort = suite.ConformanceTest{ ShortName: "HTTPRouteInvalidParentRefNotMatchingListenerPort", Description: "A single HTTPRoute in the gateway-conformance-infra namespace should set the Accepted status to False with reason NoMatchingParent when attempting to bind to a Gateway that does not have a matching ListenerPort.", - Features: []suite.SupportedFeature{suite.SupportRouteDestinationPortMatching}, - Manifests: []string{"tests/httproute-invalid-parentref-not-matching-listener-port.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportRouteDestinationPortMatching, + }, + Manifests: []string{"tests/httproute-invalid-parentref-not-matching-listener-port.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "httproute-listener-not-matching-route-port", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) // The Route must have an Accepted Condition with a NoMatchingParent Reason. t.Run("HTTPRoute with no matching port in ParentRef has an Accepted Condition with status False and Reason NoMatchingParent", func(t *testing.T) { diff --git a/conformance/tests/httproute-invalid-parentref-not-matching-section-name.go b/conformance/tests/httproute-invalid-parentref-not-matching-section-name.go new file mode 100644 index 0000000000..0fe909f514 --- /dev/null +++ b/conformance/tests/httproute-invalid-parentref-not-matching-section-name.go @@ -0,0 +1,65 @@ +/* +Copyright 2022 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + + "sigs.k8s.io/gateway-api/apis/v1beta1" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, HTTPRouteInvalidParentRefNotMatchingSectionName) +} + +var HTTPRouteInvalidParentRefNotMatchingSectionName = suite.ConformanceTest{ + ShortName: "HTTPRouteInvalidParentRefNotMatchingSectionName", + Description: "A single HTTPRoute in the gateway-conformance-infra namespace should set the Accepted status to False with reason NoMatchingParent when attempting to bind to a Gateway that does not have a matching SectionName.", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-invalid-parentref-not-matching-section-name.yaml"}, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { + routeNN := types.NamespacedName{Name: "httproute-listener-not-matching-section-name", Namespace: "gateway-conformance-infra"} + gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} + + // The Route must have an Accepted Condition with a NoMatchingParent Reason. + t.Run("HTTPRoute with no matching sectionName in ParentRef has an Accepted Condition with status False and Reason NoMatchingParent", func(t *testing.T) { + resolvedRefsCond := metav1.Condition{ + Type: string(v1beta1.RouteConditionAccepted), + Status: metav1.ConditionFalse, + Reason: string(v1beta1.RouteReasonNoMatchingParent), + } + + kubernetes.HTTPRouteMustHaveCondition(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN, resolvedRefsCond) + }) + + t.Run("Route should not have Parents accepted in status", func(t *testing.T) { + kubernetes.HTTPRouteMustHaveNoAcceptedParents(t, suite.Client, suite.TimeoutConfig, routeNN) + }) + + t.Run("Gateway should have 0 Routes attached", func(t *testing.T) { + kubernetes.GatewayMustHaveZeroRoutes(t, suite.Client, suite.TimeoutConfig, gwNN) + }) + }, +} diff --git a/conformance/tests/httproute-invalid-parentref-not-matching-section-name.yaml b/conformance/tests/httproute-invalid-parentref-not-matching-section-name.yaml new file mode 100644 index 0000000000..b449965d98 --- /dev/null +++ b/conformance/tests/httproute-invalid-parentref-not-matching-section-name.yaml @@ -0,0 +1,17 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: httproute-listener-not-matching-section-name + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: same-namespace + namespace: gateway-conformance-infra + port: 80 + # mismatched sectionName here (http1 is not an available gateway listener) triggers NoMatchingParent reason + sectionName: http1 + rules: + - backendRefs: + - name: infra-backend-v1 + kind: Service + port: 8080 diff --git a/conformance/tests/httproute-invalid-reference-grant.go b/conformance/tests/httproute-invalid-reference-grant.go new file mode 100644 index 0000000000..e209f0f24a --- /dev/null +++ b/conformance/tests/httproute-invalid-reference-grant.go @@ -0,0 +1,71 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + + "sigs.k8s.io/gateway-api/apis/v1beta1" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, HTTPRouteInvalidReferenceGrant) +} + +var HTTPRouteInvalidReferenceGrant = suite.ConformanceTest{ + ShortName: "HTTPRouteInvalidReferenceGrant", + Description: "A single HTTPRoute in the gateway-conformance-infra namespace, with a backendRef in another namespace without valid ReferenceGrant, should have the ResolvedRefs condition set to False and not forward HTTP requests to any backend", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/httproute-invalid-reference-grant.yaml"}, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { + routeNN := types.NamespacedName{Name: "reference-grant", Namespace: "gateway-conformance-infra"} + gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} + gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + + t.Run("HTTPRoute with BackendRef in another namespace and no ReferenceGrant covering the Service has a ResolvedRefs Condition with status False and Reason RefNotPermitted", func(t *testing.T) { + resolvedRefsCond := metav1.Condition{ + Type: string(v1beta1.RouteConditionResolvedRefs), + Status: metav1.ConditionFalse, + Reason: string(v1beta1.RouteReasonRefNotPermitted), + } + + kubernetes.HTTPRouteMustHaveCondition(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN, resolvedRefsCond) + }) + + t.Run("Simple HTTP request not should reach web-backend", func(t *testing.T) { + http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, http.ExpectedResponse{ + Request: http.Request{ + Method: "GET", + Path: "/", + }, + Response: http.Response{StatusCode: 500}, + Backend: "web-backend", + Namespace: "gateway-conformance-web-backend", + }) + }) + }, +} diff --git a/conformance/tests/httproute-invalid-reference-grant.yaml b/conformance/tests/httproute-invalid-reference-grant.yaml new file mode 100644 index 0000000000..5aab35f2cd --- /dev/null +++ b/conformance/tests/httproute-invalid-reference-grant.yaml @@ -0,0 +1,119 @@ +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-namespace + namespace: gateway-conformance-infra +spec: + from: + - group: gateway.networking.k8s.io + kind: HTTPRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Service + name: web-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-from-group + namespace: gateway-conformance-web-backend +spec: + from: + - group: not-the-group-youre-looking-for + kind: HTTPRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Service + name: web-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-from-kind + namespace: gateway-conformance-web-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: Gateway + namespace: gateway-conformance-infra + to: + - group: "" + kind: Service + name: web-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-from-namespace + namespace: gateway-conformance-web-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: HTTPRoute + namespace: not-the-namespace-youre-looking-for + to: + - group: "" + kind: Service + name: web-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-to-group + namespace: gateway-conformance-web-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: HTTPRoute + namespace: gateway-conformance-infra + to: + - group: not-the-group-youre-looking-for + kind: Service + name: web-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-to-kind + namespace: gateway-conformance-web-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: HTTPRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Secret + name: web-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-to-name + namespace: gateway-conformance-web-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: HTTPRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Secret + name: not-the-service-youre-looking-for +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: reference-grant + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: same-namespace + rules: + - backendRefs: + - name: web-backend + namespace: gateway-conformance-web-backend + port: 8080 diff --git a/conformance/tests/httproute-listener-hostname-matching.go b/conformance/tests/httproute-listener-hostname-matching.go index 9ab780b64e..6b326ce1f9 100644 --- a/conformance/tests/httproute-listener-hostname-matching.go +++ b/conformance/tests/httproute-listener-hostname-matching.go @@ -33,25 +33,31 @@ func init() { var HTTPRouteListenerHostnameMatching = suite.ConformanceTest{ ShortName: "HTTPRouteListenerHostnameMatching", Description: "Multiple HTTP listeners with the same port and different hostnames, each with a different HTTPRoute", - Manifests: []string{"tests/httproute-listener-hostname-matching.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-listener-hostname-matching.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" // This test creates an additional Gateway in the gateway-conformance-infra // namespace so we have to wait for it to be ready. - kubernetes.NamespacesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, []string{ns}) + kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, []string{ns}) + routeNN1 := types.NamespacedName{Name: "backend-v1", Namespace: ns} + routeNN2 := types.NamespacedName{Name: "backend-v2", Namespace: ns} + routeNN3 := types.NamespacedName{Name: "backend-v3", Namespace: ns} gwNN := types.NamespacedName{Name: "httproute-listener-hostname-matching", Namespace: ns} - _ = kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN, "listener-1"), - types.NamespacedName{Namespace: ns, Name: "backend-v1"}, - ) - _ = kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN, "listener-2"), - types.NamespacedName{Namespace: ns, Name: "backend-v2"}, - ) - gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN, "listener-3", "listener-4"), - types.NamespacedName{Namespace: ns, Name: "backend-v3"}, - ) + kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN, "listener-1"), routeNN1) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN1, gwNN) + + kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN, "listener-2"), routeNN2) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN2, gwNN) + + gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN, "listener-3", "listener-4"), routeNN3) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN3, gwNN) testCases := []http.ExpectedResponse{{ Request: http.Request{Host: "bar.com", Path: "/"}, diff --git a/conformance/tests/httproute-matching-across-routes.go b/conformance/tests/httproute-matching-across-routes.go index 4ab46a3d92..1ca6938e7d 100644 --- a/conformance/tests/httproute-matching-across-routes.go +++ b/conformance/tests/httproute-matching-across-routes.go @@ -33,13 +33,19 @@ func init() { var HTTPRouteMatchingAcrossRoutes = suite.ConformanceTest{ ShortName: "HTTPRouteMatchingAcrossRoutes", Description: "Two HTTPRoutes with path matching for different backends", - Manifests: []string{"tests/httproute-matching-across-routes.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-matching-across-routes.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN1 := types.NamespacedName{Name: "matching-part1", Namespace: ns} routeNN2 := types.NamespacedName{Name: "matching-part2", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN1, routeNN2) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN1, gwNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN2, gwNN) testCases := []http.ExpectedResponse{{ Request: http.Request{ diff --git a/conformance/tests/httproute-matching.go b/conformance/tests/httproute-matching.go index 596f04cbf5..4cdc64af61 100644 --- a/conformance/tests/httproute-matching.go +++ b/conformance/tests/httproute-matching.go @@ -33,12 +33,17 @@ func init() { var HTTPRouteMatching = suite.ConformanceTest{ ShortName: "HTTPRouteMatching", Description: "A single HTTPRoute with path and header matching for different backends", - Manifests: []string{"tests/httproute-matching.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-matching.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "matching", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{{ Request: http.Request{Path: "/"}, diff --git a/conformance/tests/httproute-method-matching.go b/conformance/tests/httproute-method-matching.go index dc136cc4a0..fd1393a2f6 100644 --- a/conformance/tests/httproute-method-matching.go +++ b/conformance/tests/httproute-method-matching.go @@ -34,12 +34,17 @@ var HTTPRouteMethodMatching = suite.ConformanceTest{ ShortName: "HTTPRouteMethodMatching", Description: "A single HTTPRoute with method matching for different backends", Manifests: []string{"tests/httproute-method-matching.yaml"}, - Features: []suite.SupportedFeature{suite.SupportHTTPRouteMethodMatching}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRouteMethodMatching, + }, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "method-matching", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{ { @@ -56,6 +61,68 @@ var HTTPRouteMethodMatching = suite.ConformanceTest{ }, } + // Combinations of method matching with other core matches. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/path1", Method: "GET"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "one"}, Path: "/", Method: "PUT"}, + Backend: "infra-backend-v2", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "two"}, Path: "/path2", Method: "POST"}, + Backend: "infra-backend-v3", + Namespace: ns, + }, + }...) + + // Ensure that combinations of matches which are OR'd together match + // even if only one of them is used in the request. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/path3", Method: "PATCH"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "three"}, Path: "/path4", Method: "DELETE"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + }...) + + // Ensure that combinations of match types which are ANDed together do not match + // when only a subset of match types is used in the request. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/", Method: "PUT"}, + Response: http.Response{StatusCode: 404}, + }, + { + Request: http.Request{Path: "/path4", Method: "DELETE"}, + Response: http.Response{StatusCode: 404}, + }, + }...) + + // For requests that satisfy multiple matches, ensure precedence order + // defined by the Gateway API spec is maintained. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/path5", Method: "PATCH"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "four"}, Path: "/", Method: "PATCH"}, + Backend: "infra-backend-v2", + Namespace: ns, + }, + }...) + for i := range testCases { // Declare tc here to avoid loop variable // reuse issues across parallel tests. diff --git a/conformance/tests/httproute-method-matching.yaml b/conformance/tests/httproute-method-matching.yaml index b75d60439e..5396f084ee 100644 --- a/conformance/tests/httproute-method-matching.yaml +++ b/conformance/tests/httproute-method-matching.yaml @@ -17,3 +17,70 @@ spec: backendRefs: - name: infra-backend-v2 port: 8080 + + # Combinations with core match types. + - matches: + - path: + type: PathPrefix + value: /path1 + method: GET + backendRefs: + - name: infra-backend-v1 + port: 8080 + - matches: + - headers: + - name: version + value: one + method: PUT + backendRefs: + - name: infra-backend-v2 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /path2 + headers: + - name: version + value: two + method: POST + backendRefs: + - name: infra-backend-v3 + port: 8080 + + # Match of the form (cond1 AND cond2) OR (cond3 AND cond4 AND cond5) + - matches: + - path: + type: PathPrefix + value: /path3 + method: PATCH + - path: + type: PathPrefix + value: /path4 + headers: + - name: version + value: three + method: DELETE + backendRefs: + - name: infra-backend-v1 + port: 8080 + + # Matches for checking precedence. + - matches: + - path: + type: PathPrefix + value: /path5 + backendRefs: + - name: infra-backend-v1 + port: 8080 + - matches: + - method: PATCH + backendRefs: + - name: infra-backend-v2 + port: 8080 + - matches: + - headers: + - name: version + value: four + backendRefs: + - name: infra-backend-v3 + port: 8080 diff --git a/conformance/tests/httproute-observed-generation-bump.go b/conformance/tests/httproute-observed-generation-bump.go index 590dcc6fed..ade4a5927f 100644 --- a/conformance/tests/httproute-observed-generation-bump.go +++ b/conformance/tests/httproute-observed-generation-bump.go @@ -24,6 +24,7 @@ import ( "github.com/stretchr/testify/require" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/gateway-api/apis/v1beta1" "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" @@ -37,27 +38,24 @@ func init() { var HTTPRouteObservedGenerationBump = suite.ConformanceTest{ ShortName: "HTTPRouteObservedGenerationBump", Description: "A HTTPRoute in the gateway-conformance-infra namespace should update the observedGeneration in all of it's Status.Conditions after an update to the spec", - Manifests: []string{"tests/httproute-observed-generation-bump.yaml"}, - Test: func(t *testing.T, s *suite.ConformanceTestSuite) { - + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-observed-generation-bump.yaml"}, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "observed-generation-bump", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} - acceptedCondition := metav1.Condition{ - Type: string(v1beta1.RouteConditionAccepted), - Status: metav1.ConditionTrue, - Reason: "", // any reason - } - t.Run("observedGeneration should increment", func(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), time.Minute) defer cancel() namespaces := []string{"gateway-conformance-infra"} - kubernetes.NamespacesMustBeAccepted(t, s.Client, s.TimeoutConfig, namespaces) + kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, namespaces) original := &v1beta1.HTTPRoute{} - err := s.Client.Get(ctx, routeNN, original) + err := suite.Client.Get(ctx, routeNN, original) require.NoErrorf(t, err, "error getting HTTPRoute: %v", err) // Sanity check @@ -65,13 +63,18 @@ var HTTPRouteObservedGenerationBump = suite.ConformanceTest{ mutate := original.DeepCopy() mutate.Spec.Rules[0].BackendRefs[0].Name = "infra-backend-v2" - err = s.Client.Update(ctx, mutate) - require.NoErrorf(t, err, "error updating the HTTPRoute: %v", err) + err = suite.Client.Patch(ctx, mutate, client.MergeFrom(original)) + require.NoErrorf(t, err, "error patching the HTTPRoute: %v", err) - kubernetes.HTTPRouteMustHaveCondition(t, s.Client, s.TimeoutConfig, routeNN, gwNN, acceptedCondition) + kubernetes.HTTPRouteMustHaveCondition(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN, metav1.Condition{ + Type: string(v1beta1.RouteConditionAccepted), + Status: metav1.ConditionTrue, + Reason: "", // any reason + }) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) updated := &v1beta1.HTTPRoute{} - err = s.Client.Get(ctx, routeNN, updated) + err = suite.Client.Get(ctx, routeNN, updated) require.NoErrorf(t, err, "error getting Gateway: %v", err) // Sanity check diff --git a/conformance/tests/httproute-partially-invalid-via-reference-grant.go b/conformance/tests/httproute-partially-invalid-via-reference-grant.go index dff80bb749..1869309632 100644 --- a/conformance/tests/httproute-partially-invalid-via-reference-grant.go +++ b/conformance/tests/httproute-partially-invalid-via-reference-grant.go @@ -35,8 +35,12 @@ func init() { var HTTPRoutePartiallyInvalidViaInvalidReferenceGrant = suite.ConformanceTest{ ShortName: "HTTPRoutePartiallyInvalidViaInvalidReferenceGrant", Description: "A single HTTPRoute in the gateway-conformance-infra namespace should attach to a Gateway in the same namespace if the route has a backendRef Service in the gateway-conformance-app-backend namespace and a ReferenceGrant exists but does not grant permission to route to that specific Service", - Features: []suite.SupportedFeature{suite.SupportReferenceGrant}, - Manifests: []string{"tests/httproute-partially-invalid-via-reference-grant.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/httproute-partially-invalid-via-reference-grant.yaml"}, Test: func(t *testing.T, s *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "invalid-reference-grant", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} @@ -45,7 +49,6 @@ var HTTPRoutePartiallyInvalidViaInvalidReferenceGrant = suite.ConformanceTest{ gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, s.Client, s.TimeoutConfig, s.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) t.Run("HTTPRoute with BackendRef in another namespace and no ReferenceGrant covering the Service has a ResolvedRefs Condition with status False and Reason RefNotPermitted", func(t *testing.T) { - resolvedRefsCond := metav1.Condition{ Type: string(v1beta1.RouteConditionResolvedRefs), Status: metav1.ConditionFalse, @@ -76,6 +79,5 @@ var HTTPRoutePartiallyInvalidViaInvalidReferenceGrant = suite.ConformanceTest{ Namespace: "gateway-conformance-app-backend", }) }) - }, } diff --git a/conformance/tests/httproute-partially-invalid-via-reference-grant.yaml b/conformance/tests/httproute-partially-invalid-via-reference-grant.yaml index 41181b7665..0d584b67e4 100644 --- a/conformance/tests/httproute-partially-invalid-via-reference-grant.yaml +++ b/conformance/tests/httproute-partially-invalid-via-reference-grant.yaml @@ -20,17 +20,17 @@ metadata: namespace: gateway-conformance-infra spec: parentRefs: - - name: same-namespace + - name: same-namespace rules: - - matches: - - path: - type: PathPrefix - value: "/v2" - backendRefs: - - name: app-backend-v2 - namespace: gateway-conformance-app-backend - port: 8080 - - backendRefs: - - name: app-backend-v1 - namespace: gateway-conformance-app-backend - port: 8080 + - matches: + - path: + type: PathPrefix + value: "/v2" + backendRefs: + - name: app-backend-v2 + namespace: gateway-conformance-app-backend + port: 8080 + - backendRefs: + - name: app-backend-v1 + namespace: gateway-conformance-app-backend + port: 8080 diff --git a/conformance/tests/httproute-path-match-order.go b/conformance/tests/httproute-path-match-order.go new file mode 100644 index 0000000000..a620e11901 --- /dev/null +++ b/conformance/tests/httproute-path-match-order.go @@ -0,0 +1,84 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "k8s.io/apimachinery/pkg/types" + + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, HTTPRoutePathMatchOrder) +} + +var HTTPRoutePathMatchOrder = suite.ConformanceTest{ + ShortName: "HTTPRoutePathMatchOrder", + Description: "An HTTPRoute where there are multiple matches routing to any given backend follows match order precedence", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-path-match-order.yaml"}, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { + ns := "gateway-conformance-infra" + routeNN := types.NamespacedName{Namespace: ns, Name: "path-matching-order"} + gwNN := types.NamespacedName{Namespace: ns, Name: "same-namespace"} + gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) + + testCases := []http.ExpectedResponse{ + { + Request: http.Request{Path: "/match/exact/one"}, + Backend: "infra-backend-v3", + Namespace: ns, + }, { + Request: http.Request{Path: "/match/exact"}, + Backend: "infra-backend-v2", + Namespace: ns, + }, { + Request: http.Request{Path: "/match"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, { + Request: http.Request{Path: "/match/prefix/one/any"}, + Backend: "infra-backend-v2", + Namespace: ns, + }, { + Request: http.Request{Path: "/match/prefix/any"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, { + Request: http.Request{Path: "/match/any"}, + Backend: "infra-backend-v3", + Namespace: ns, + }, + } + + for i := range testCases { + tc := testCases[i] + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + t.Parallel() + http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, tc) + }) + } + }, +} diff --git a/conformance/tests/httproute-path-match-order.yaml b/conformance/tests/httproute-path-match-order.yaml new file mode 100644 index 0000000000..134ae21b49 --- /dev/null +++ b/conformance/tests/httproute-path-match-order.yaml @@ -0,0 +1,51 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: path-matching-order + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: same-namespace + rules: + - matches: + - path: + type: Exact + value: /match + backendRefs: + - name: infra-backend-v1 + port: 8080 + - matches: + - path: + type: Exact + value: /match/exact + backendRefs: + - name: infra-backend-v2 + port: 8080 + - matches: + - path: + type: Exact + value: /match/exact/one + backendRefs: + - name: infra-backend-v3 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /match/ + backendRefs: + - name: infra-backend-v3 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /match/prefix/ + backendRefs: + - name: infra-backend-v1 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /match/prefix/one + backendRefs: + - name: infra-backend-v2 + port: 8080 diff --git a/conformance/tests/httproute-query-param-matching.go b/conformance/tests/httproute-query-param-matching.go index e5a513150e..79c76f7aad 100644 --- a/conformance/tests/httproute-query-param-matching.go +++ b/conformance/tests/httproute-query-param-matching.go @@ -34,14 +34,17 @@ var HTTPRouteQueryParamMatching = suite.ConformanceTest{ ShortName: "HTTPRouteQueryParamMatching", Description: "A single HTTPRoute with query param matching for different backends", Manifests: []string{"tests/httproute-query-param-matching.yaml"}, - Features: []suite.SupportedFeature{suite.SupportHTTPRouteQueryParamMatching}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRouteQueryParamMatching, + }, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { - var ( - ns = "gateway-conformance-infra" - routeNN = types.NamespacedName{Namespace: ns, Name: "query-param-matching"} - gwNN = types.NamespacedName{Namespace: ns, Name: "same-namespace"} - gwAddr = kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) - ) + ns := "gateway-conformance-infra" + routeNN := types.NamespacedName{Namespace: ns, Name: "query-param-matching"} + gwNN := types.NamespacedName{Namespace: ns, Name: "same-namespace"} + gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{{ Request: http.Request{Path: "/?animal=whale"}, @@ -81,6 +84,68 @@ var HTTPRouteQueryParamMatching = suite.ConformanceTest{ Response: http.Response{StatusCode: 404}, }} + // Combinations of query param matching with other core matches. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/path1?animal=whale"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "one"}, Path: "/?animal=whale"}, + Backend: "infra-backend-v2", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "two"}, Path: "/path2?animal=whale"}, + Backend: "infra-backend-v3", + Namespace: ns, + }, + }...) + + // Ensure that combinations of matches which are OR'd together match + // even if only one of them is used in the request. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/path3?animal=shark"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "three"}, Path: "/path4?animal=kraken"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + }...) + + // Ensure that combinations of match types which are ANDed together do not match + // when only a subset of match types is used in the request. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/?animal=shark"}, + Response: http.Response{StatusCode: 404}, + }, + { + Request: http.Request{Path: "/path4?animal=kraken"}, + Response: http.Response{StatusCode: 404}, + }, + }...) + + // For requests that satisfy multiple matches, ensure precedence order + // defined by the Gateway API spec is maintained. + testCases = append(testCases, []http.ExpectedResponse{ + { + Request: http.Request{Path: "/path5?animal=hydra"}, + Backend: "infra-backend-v1", + Namespace: ns, + }, + { + Request: http.Request{Headers: map[string]string{"version": "four"}, Path: "/?animal=hydra"}, + Backend: "infra-backend-v3", + Namespace: ns, + }, + }...) + for i := range testCases { tc := testCases[i] t.Run(tc.GetTestCaseName(i), func(t *testing.T) { diff --git a/conformance/tests/httproute-query-param-matching.yaml b/conformance/tests/httproute-query-param-matching.yaml index c4ddfe206c..2b9db8141c 100644 --- a/conformance/tests/httproute-query-param-matching.yaml +++ b/conformance/tests/httproute-query-param-matching.yaml @@ -33,3 +33,82 @@ spec: backendRefs: - name: infra-backend-v3 port: 8080 + + # Combinations with core match types. + - matches: + - path: + type: PathPrefix + value: /path1 + queryParams: + - name: animal + value: whale + backendRefs: + - name: infra-backend-v1 + port: 8080 + - matches: + - headers: + - name: version + value: one + queryParams: + - name: animal + value: whale + backendRefs: + - name: infra-backend-v2 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /path2 + headers: + - name: version + value: two + queryParams: + - name: animal + value: whale + backendRefs: + - name: infra-backend-v3 + port: 8080 + + # Match of the form (cond1 AND cond2) OR (cond3 AND cond4 AND cond5) + - matches: + - path: + type: PathPrefix + value: /path3 + queryParams: + - name: animal + value: shark + - path: + type: PathPrefix + value: /path4 + headers: + - name: version + value: three + queryParams: + - name: animal + value: kraken + backendRefs: + - name: infra-backend-v1 + port: 8080 + + # Matches for checking precedence. + - matches: + - path: + type: PathPrefix + value: /path5 + backendRefs: + - name: infra-backend-v1 + port: 8080 + - matches: + - queryParams: + - name: animal + value: hydra + backendRefs: + - name: infra-backend-v2 + port: 8080 + - matches: + - headers: + - name: version + value: four + backendRefs: + - name: infra-backend-v3 + port: 8080 diff --git a/conformance/tests/httproute-redirect-host-and-status.go b/conformance/tests/httproute-redirect-host-and-status.go index d360abb28f..4deec42c40 100644 --- a/conformance/tests/httproute-redirect-host-and-status.go +++ b/conformance/tests/httproute-redirect-host-and-status.go @@ -34,47 +34,44 @@ func init() { var HTTPRouteRedirectHostAndStatus = suite.ConformanceTest{ ShortName: "HTTPRouteRedirectHostAndStatus", Description: "An HTTPRoute with hostname and statusCode redirect filters", - Manifests: []string{"tests/httproute-redirect-host-and-status.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-redirect-host-and-status.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "redirect-host-and-status", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) - testCases := []http.ExpectedResponse{{ - Request: http.Request{ - Path: "/hostname-redirect", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Host: "example.org", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/status-code-301", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 301, - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/host-and-status", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 301, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Host: "example.org", + testCases := []http.ExpectedResponse{ + { + Request: http.Request{ + Path: "/hostname-redirect", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Host: "example.org", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/host-and-status", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 301, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Host: "example.org", + }, + Namespace: ns, }, - Namespace: ns, - }, } for i := range testCases { // Declare tc here to avoid loop variable diff --git a/conformance/tests/httproute-redirect-host-and-status.yaml b/conformance/tests/httproute-redirect-host-and-status.yaml index 21505a0723..c6de2465bb 100644 --- a/conformance/tests/httproute-redirect-host-and-status.yaml +++ b/conformance/tests/httproute-redirect-host-and-status.yaml @@ -15,14 +15,6 @@ spec: - type: RequestRedirect requestRedirect: hostname: example.org - - matches: - - path: - type: PathPrefix - value: /status-code-301 - filters: - - type: RequestRedirect - requestRedirect: - statusCode: 301 - matches: - path: type: PathPrefix diff --git a/conformance/tests/httproute-redirect-path.go b/conformance/tests/httproute-redirect-path.go index 880fb053ea..e54e35abb7 100644 --- a/conformance/tests/httproute-redirect-path.go +++ b/conformance/tests/httproute-redirect-path.go @@ -35,88 +35,94 @@ var HTTPRouteRedirectPath = suite.ConformanceTest{ ShortName: "HTTPRouteRedirectPath", Description: "An HTTPRoute with scheme redirect filter", Manifests: []string{"tests/httproute-redirect-path.yaml"}, - Features: []suite.SupportedFeature{suite.SupportHTTPRoutePathRedirect}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRoutePathRedirect, + }, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "redirect-path", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) - testCases := []http.ExpectedResponse{{ - Request: http.Request{ - Path: "/original-prefix/lemon", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Path: "/replacement-prefix/lemon", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/full/path/original", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Path: "/full-path-replacement", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/path-and-host", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Host: "example.org", - Path: "/replacement-prefix", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/path-and-status", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 301, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Path: "/replacement-prefix", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/full-path-and-host", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Host: "example.org", - Path: "/replacement-full", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/full-path-and-status", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 301, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Path: "/replacement-full", + testCases := []http.ExpectedResponse{ + { + Request: http.Request{ + Path: "/original-prefix/lemon", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Path: "/replacement-prefix/lemon", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/full/path/original", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Path: "/full-path-replacement", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/path-and-host", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Host: "example.org", + Path: "/replacement-prefix", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/path-and-status", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 301, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Path: "/replacement-prefix", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/full-path-and-host", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Host: "example.org", + Path: "/replacement-full", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/full-path-and-status", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 301, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Path: "/replacement-full", + }, + Namespace: ns, }, - Namespace: ns, - }, } for i := range testCases { // Declare tc here to avoid loop variable diff --git a/conformance/tests/httproute-redirect-port-and-scheme.go b/conformance/tests/httproute-redirect-port-and-scheme.go new file mode 100644 index 0000000000..17ee173245 --- /dev/null +++ b/conformance/tests/httproute-redirect-port-and-scheme.go @@ -0,0 +1,310 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "k8s.io/apimachinery/pkg/types" + + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/roundtripper" + "sigs.k8s.io/gateway-api/conformance/utils/suite" + "sigs.k8s.io/gateway-api/conformance/utils/tls" +) + +func init() { + ConformanceTests = append(ConformanceTests, HTTPRouteRedirectPortAndScheme) +} + +var HTTPRouteRedirectPortAndScheme = suite.ConformanceTest{ + ShortName: "HTTPRouteRedirectPortAndScheme", + Description: "An HTTPRoute with port and scheme redirect filter", + Manifests: []string{"tests/httproute-redirect-port-and-scheme.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRoutePortRedirect, + suite.SupportGatewayPort8080, + }, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { + ns := "gateway-conformance-infra" + + gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} + routeNN := types.NamespacedName{Name: "http-route-for-listener-on-port-80", Namespace: ns} + gwAddr80 := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) + + gwNN = types.NamespacedName{Name: "same-namespace-with-http-listener-on-8080", Namespace: ns} + routeNN = types.NamespacedName{Name: "http-route-for-listener-on-port-8080", Namespace: ns} + gwAddr8080 := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) + + gwNN = types.NamespacedName{Name: "same-namespace-with-https-listener", Namespace: ns} + routeNN = types.NamespacedName{Name: "http-route-for-listener-on-port-443", Namespace: ns} + gwAddr443 := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) + + certNN := types.NamespacedName{Name: "tls-validity-checks-certificate", Namespace: ns} + cPem, keyPem, err := GetTLSSecret(suite.Client, certNN) + if err != nil { + t.Fatalf("unexpected error finding TLS secret: %v", err) + } + + // NOTE: In all the test cases, a missing value of expected Port within + // RedirectRequest implies that it is still valid and acceptable for a + // port to be specified in the redirect if it corresponds to the scheme + // (80 and 443). + + //////////////////////////////////////////////////////////////////////////// + // Test cases that use http-route-for-listener-on-port-80 + //////////////////////////////////////////////////////////////////////////// + + testCases := []http.ExpectedResponse{ + { + Request: http.Request{ + Path: "/scheme-nil-and-port-nil", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/scheme-nil-and-port-80", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/scheme-nil-and-port-8080", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Port: "8080", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/scheme-https-and-port-nil", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/scheme-https-and-port-443", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/scheme-https-and-port-8443", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Port: "8443", + Host: "example.org", + }, + Namespace: ns, + }, + } + + for i := range testCases { + tc := testCases[i] + t.Run("http-listener-on-80/"+tc.GetTestCaseName(i), func(t *testing.T) { + t.Parallel() + http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr80, tc) + }) + } + + //////////////////////////////////////////////////////////////////////////// + // Test cases that use same-namespace-with-http-listener-on-8080 + //////////////////////////////////////////////////////////////////////////// + + testCases = []http.ExpectedResponse{ + { + Request: http.Request{ + Path: "/scheme-nil-and-port-nil", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Port: "8080", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/scheme-nil-and-port-80", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/scheme-https-and-port-nil", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Host: "example.org", + }, + Namespace: ns, + }, + } + + for i := range testCases { + tc := testCases[i] + t.Run("http-listener-on-8080/"+tc.GetTestCaseName(i), func(t *testing.T) { + t.Parallel() + http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr8080, tc) + }) + } + + //////////////////////////////////////////////////////////////////////////// + // Test cases that use http-route-for-listener-on-port-443 + //////////////////////////////////////////////////////////////////////////// + + testCases = []http.ExpectedResponse{ + { + Request: http.Request{ + Host: "example.org", + Path: "/scheme-nil-and-port-nil", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Host: "example.org", + Path: "/scheme-nil-and-port-443", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Host: "example.org", + Path: "/scheme-nil-and-port-8443", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Port: "8443", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Host: "example.org", + Path: "/scheme-http-and-port-nil", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Host: "example.org", + Path: "/scheme-http-and-port-80", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Host: "example.org", + }, + Namespace: ns, + }, + { + Request: http.Request{ + Host: "example.org", + Path: "/scheme-http-and-port-8080", + UnfollowRedirect: true, + }, + Response: http.Response{StatusCode: 302}, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "http", + Port: "8080", + Host: "example.org", + }, + Namespace: ns, + }, + } + + for i := range testCases { + tc := testCases[i] + t.Run("https-listener-on-443/"+tc.GetTestCaseName(i), func(t *testing.T) { + t.Parallel() + tls.MakeTLSRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr443, cPem, keyPem, "example.org", tc) + }) + } + }, +} diff --git a/conformance/tests/httproute-redirect-port-and-scheme.yaml b/conformance/tests/httproute-redirect-port-and-scheme.yaml new file mode 100644 index 0000000000..1b6d72cd01 --- /dev/null +++ b/conformance/tests/httproute-redirect-port-and-scheme.yaml @@ -0,0 +1,181 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: same-namespace-with-http-listener-on-8080 + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: http + port: 8080 + protocol: HTTP + allowedRoutes: + namespaces: + from: Same +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-for-listener-on-port-80 + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: same-namespace + rules: + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-nil + filters: + - type: RequestRedirect + requestRedirect: + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-80 + filters: + - type: RequestRedirect + requestRedirect: + port: 80 + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-8080 + filters: + - type: RequestRedirect + requestRedirect: + port: 8080 + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-https-and-port-nil + filters: + - type: RequestRedirect + requestRedirect: + scheme: "https" + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-https-and-port-443 + filters: + - type: RequestRedirect + requestRedirect: + scheme: "https" + port: 443 + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-https-and-port-8443 + filters: + - type: RequestRedirect + requestRedirect: + scheme: "https" + port: 8443 + hostname: example.org +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-for-listener-on-port-8080 + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: same-namespace-with-http-listener-on-8080 + rules: + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-nil + filters: + - type: RequestRedirect + requestRedirect: + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-80 + filters: + - type: RequestRedirect + requestRedirect: + port: 80 + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-https-and-port-nil + filters: + - type: RequestRedirect + requestRedirect: + scheme: "https" + hostname: example.org +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-route-for-listener-on-port-443 + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: same-namespace-with-https-listener + rules: + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-nil + filters: + - type: RequestRedirect + requestRedirect: + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-443 + filters: + - type: RequestRedirect + requestRedirect: + port: 443 + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-nil-and-port-8443 + filters: + - type: RequestRedirect + requestRedirect: + port: 8443 + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-http-and-port-nil + filters: + - type: RequestRedirect + requestRedirect: + scheme: "http" + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-http-and-port-80 + filters: + - type: RequestRedirect + requestRedirect: + scheme: "http" + port: 80 + hostname: example.org + - matches: + - path: + type: PathPrefix + value: /scheme-http-and-port-8080 + filters: + - type: RequestRedirect + requestRedirect: + scheme: "http" + port: 8080 + hostname: example.org +--- diff --git a/conformance/tests/httproute-redirect-port.go b/conformance/tests/httproute-redirect-port.go index 58d4fd6b12..ec2149bb91 100644 --- a/conformance/tests/httproute-redirect-port.go +++ b/conformance/tests/httproute-redirect-port.go @@ -35,64 +35,70 @@ var HTTPRouteRedirectPort = suite.ConformanceTest{ ShortName: "HTTPRouteRedirectPort", Description: "An HTTPRoute with a port redirect filter", Manifests: []string{"tests/httproute-redirect-port.yaml"}, - Features: []suite.SupportedFeature{suite.SupportHTTPRoutePortRedirect}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRoutePortRedirect, + }, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "redirect-port", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) - testCases := []http.ExpectedResponse{{ - Request: http.Request{ - Path: "/port", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Port: "8083", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/port-and-host", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Host: "example.org", - Port: "8083", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/port-and-status", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 301, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Port: "8083", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/port-and-host-and-status", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Port: "8083", - Host: "example.org", + testCases := []http.ExpectedResponse{ + { + Request: http.Request{ + Path: "/port", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Port: "8083", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/port-and-host", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Host: "example.org", + Port: "8083", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/port-and-status", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 301, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Port: "8083", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/port-and-host-and-status", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Port: "8083", + Host: "example.org", + }, + Namespace: ns, }, - Namespace: ns, - }, } for i := range testCases { // Declare tc here to avoid loop variable diff --git a/conformance/tests/httproute-redirect-scheme.go b/conformance/tests/httproute-redirect-scheme.go index d2cc34f251..9ce7bd836d 100644 --- a/conformance/tests/httproute-redirect-scheme.go +++ b/conformance/tests/httproute-redirect-scheme.go @@ -35,64 +35,70 @@ var HTTPRouteRedirectScheme = suite.ConformanceTest{ ShortName: "HTTPRouteRedirectScheme", Description: "An HTTPRoute with a scheme redirect filter", Manifests: []string{"tests/httproute-redirect-scheme.yaml"}, - Features: []suite.SupportedFeature{suite.SupportHTTPRouteSchemeRedirect}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRouteSchemeRedirect, + }, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "redirect-scheme", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) - testCases := []http.ExpectedResponse{{ - Request: http.Request{ - Path: "/scheme", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Scheme: "https", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/scheme-and-host", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Host: "example.org", - Scheme: "https", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/scheme-and-status", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 301, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Scheme: "https", - }, - Namespace: ns, - }, { - Request: http.Request{ - Path: "/scheme-and-host-and-status", - UnfollowRedirect: true, - }, - Response: http.Response{ - StatusCode: 302, - }, - RedirectRequest: &roundtripper.RedirectRequest{ - Scheme: "https", - Host: "example.org", + testCases := []http.ExpectedResponse{ + { + Request: http.Request{ + Path: "/scheme", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/scheme-and-host", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Host: "example.org", + Scheme: "https", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/scheme-and-status", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 301, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + }, + Namespace: ns, + }, { + Request: http.Request{ + Path: "/scheme-and-host-and-status", + UnfollowRedirect: true, + }, + Response: http.Response{ + StatusCode: 302, + }, + RedirectRequest: &roundtripper.RedirectRequest{ + Scheme: "https", + Host: "example.org", + }, + Namespace: ns, }, - Namespace: ns, - }, } for i := range testCases { // Declare tc here to avoid loop variable diff --git a/conformance/tests/httproute-reference-grant.go b/conformance/tests/httproute-reference-grant.go index ae096ecadf..0b53fb92c5 100644 --- a/conformance/tests/httproute-reference-grant.go +++ b/conformance/tests/httproute-reference-grant.go @@ -17,10 +17,14 @@ limitations under the License. package tests import ( + "context" "testing" + "github.com/stretchr/testify/require" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/gateway-api/apis/v1beta1" "sigs.k8s.io/gateway-api/conformance/utils/http" "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" "sigs.k8s.io/gateway-api/conformance/utils/suite" @@ -33,15 +37,20 @@ func init() { var HTTPRouteReferenceGrant = suite.ConformanceTest{ ShortName: "HTTPRouteReferenceGrant", Description: "A single HTTPRoute in the gateway-conformance-infra namespace, with a backendRef in the gateway-conformance-web-backend namespace, should attach to Gateway in the gateway-conformance-infra namespace", - Features: []suite.SupportedFeature{suite.SupportReferenceGrant}, - Manifests: []string{"tests/httproute-reference-grant.yaml"}, - Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/httproute-reference-grant.yaml"}, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { routeNN := types.NamespacedName{Name: "reference-grant", Namespace: "gateway-conformance-infra"} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: "gateway-conformance-infra"} - gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, s.Client, s.TimeoutConfig, s.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) t.Run("Simple HTTP request should reach web-backend", func(t *testing.T) { - http.MakeRequestAndExpectEventuallyConsistentResponse(t, s.RoundTripper, s.TimeoutConfig, gwAddr, http.ExpectedResponse{ + http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, http.ExpectedResponse{ Request: http.Request{ Method: "GET", Path: "/", @@ -51,5 +60,25 @@ var HTTPRouteReferenceGrant = suite.ConformanceTest{ Namespace: "gateway-conformance-web-backend", }) }) + + ctx, cancel := context.WithTimeout(context.Background(), suite.TimeoutConfig.DeleteTimeout) + defer cancel() + rg := v1beta1.ReferenceGrant{ + ObjectMeta: metav1.ObjectMeta{ + Name: "reference-grant", + Namespace: "gateway-conformance-web-backend", + }, + } + require.NoError(t, suite.Client.Delete(ctx, &rg)) + + t.Run("Simple HTTP request should return 500 after deleting the relevant reference grant", func(t *testing.T) { + http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, http.ExpectedResponse{ + Request: http.Request{ + Method: "GET", + Path: "/", + }, + Response: http.Response{StatusCode: 500}, + }) + }) }, } diff --git a/conformance/tests/httproute-reference-grant.yaml b/conformance/tests/httproute-reference-grant.yaml index f93d1b21d2..a7f73bb73e 100644 --- a/conformance/tests/httproute-reference-grant.yaml +++ b/conformance/tests/httproute-reference-grant.yaml @@ -20,9 +20,9 @@ metadata: namespace: gateway-conformance-infra spec: parentRefs: - - name: same-namespace + - name: same-namespace rules: - - backendRefs: - - name: web-backend - namespace: gateway-conformance-web-backend - port: 8080 + - backendRefs: + - name: web-backend + namespace: gateway-conformance-web-backend + port: 8080 diff --git a/conformance/tests/httproute-request-header-modifier.go b/conformance/tests/httproute-request-header-modifier.go index 14220b3442..fef660e95a 100644 --- a/conformance/tests/httproute-request-header-modifier.go +++ b/conformance/tests/httproute-request-header-modifier.go @@ -33,12 +33,17 @@ func init() { var HTTPRouteRequestHeaderModifier = suite.ConformanceTest{ ShortName: "HTTPRouteRequestHeaderModifier", Description: "An HTTPRoute has request header modifier filters applied correctly", - Manifests: []string{"tests/httproute-request-header-modifier.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-request-header-modifier.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "request-header-modifier", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{{ Request: http.Request{ diff --git a/conformance/tests/httproute-request-mirror.go b/conformance/tests/httproute-request-mirror.go new file mode 100644 index 0000000000..0f06ad3a62 --- /dev/null +++ b/conformance/tests/httproute-request-mirror.go @@ -0,0 +1,96 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "k8s.io/apimachinery/pkg/types" + + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, HTTPRouteRequestMirror) +} + +var HTTPRouteRequestMirror = suite.ConformanceTest{ + ShortName: "HTTPRouteRequestMirror", + Description: "An HTTPRoute with request mirror filter", + Manifests: []string{"tests/httproute-request-mirror.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRouteRequestMirror, + }, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { + ns := "gateway-conformance-infra" + routeNN := types.NamespacedName{Name: "request-mirror", Namespace: ns} + gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} + gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + + testCases := []http.ExpectedResponse{ + { + Request: http.Request{ + Path: "/mirror", + }, + ExpectedRequest: &http.ExpectedRequest{ + Request: http.Request{ + Path: "/mirror", + }, + }, + Backend: "infra-backend-v1", + MirroredTo: "infra-backend-v2", + Namespace: ns, + }, { + Request: http.Request{ + Path: "/mirror-and-modify-headers", + Headers: map[string]string{ + "X-Header-Remove": "remove-val", + "X-Header-Add-Append": "append-val-1", + }, + }, + ExpectedRequest: &http.ExpectedRequest{ + Request: http.Request{ + Path: "/mirror-and-modify-headers", + Headers: map[string]string{ + "X-Header-Add": "header-val-1", + "X-Header-Add-Append": "append-val-1,header-val-2", + "X-Header-Set": "set-overwrites-values", + }, + }, + AbsentHeaders: []string{"X-Header-Remove"}, + }, + Namespace: ns, + Backend: "infra-backend-v1", + MirroredTo: "infra-backend-v2", + }, + } + for i := range testCases { + // Declare tc here to avoid loop variable + // reuse issues across parallel tests. + tc := testCases[i] + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + t.Parallel() + http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, tc) + http.ExpectMirroredRequest(t, suite.Client, suite.Clientset, ns, tc.MirroredTo, tc.Request.Path) + }) + } + }, +} diff --git a/conformance/tests/httproute-request-mirror.yaml b/conformance/tests/httproute-request-mirror.yaml new file mode 100644 index 0000000000..b742b795d2 --- /dev/null +++ b/conformance/tests/httproute-request-mirror.yaml @@ -0,0 +1,51 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: request-mirror + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: same-namespace + rules: + - matches: + - path: + type: PathPrefix + value: /mirror + filters: + - type: RequestMirror + requestMirror: + backendRef: + name: infra-backend-v2 + namespace: gateway-conformance-infra + port: 8080 + backendRefs: + - name: infra-backend-v1 + port: 8080 + namespace: gateway-conformance-infra + - matches: + - path: + type: PathPrefix + value: /mirror-and-modify-headers + filters: + - type: RequestHeaderModifier + requestHeaderModifier: + set: + - name: X-Header-Set + value: set-overwrites-values + add: + - name: X-Header-Add + value: header-val-1 + - name: X-Header-Add-Append + value: header-val-2 + remove: + - X-Header-Remove + - type: RequestMirror + requestMirror: + backendRef: + name: infra-backend-v2 + namespace: gateway-conformance-infra + port: 8080 + backendRefs: + - name: infra-backend-v1 + port: 8080 + namespace: gateway-conformance-infra diff --git a/conformance/tests/httproute-response-header-modifier.go b/conformance/tests/httproute-response-header-modifier.go index dd6323545c..01fa311abb 100644 --- a/conformance/tests/httproute-response-header-modifier.go +++ b/conformance/tests/httproute-response-header-modifier.go @@ -33,13 +33,18 @@ func init() { var HTTPRouteResponseHeaderModifier = suite.ConformanceTest{ ShortName: "HTTPRouteResponseHeaderModifier", Description: "An HTTPRoute has response header modifier filters applied correctly", - Features: []suite.SupportedFeature{suite.SupportHTTPResponseHeaderModification}, - Manifests: []string{"tests/httproute-response-header-modifier.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPResponseHeaderModification, + }, + Manifests: []string{"tests/httproute-response-header-modifier.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "response-header-modifier", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{{ Request: http.Request{ @@ -163,6 +168,48 @@ var HTTPRouteResponseHeaderModifier = suite.ConformanceTest{ }, Backend: "infra-backend-v1", Namespace: ns, + }, { + Request: http.Request{ + Path: "/response-and-request-header-modifiers", + Headers: map[string]string{ + "X-Header-Remove": "remove-val", + "X-Header-Add-Append": "append-val-1", + "X-Header-Echo": "echo", + }, + }, + BackendSetResponseHeaders: map[string]string{ + "X-Header-Set-2": "set-val-2", + "X-Header-Add-2": "add-val-2", + "X-Header-Remove-2": "remove-val-2", + "Another-Header": "another-header-val", + "X-Header-Remove-1": "remove-val-1", + "X-Header-Echo": "echo", + }, + ExpectedRequest: &http.ExpectedRequest{ + Request: http.Request{ + Path: "/response-and-request-header-modifiers", + Headers: map[string]string{ + "X-Header-Add": "header-val-1", + "X-Header-Set": "set-overwrites-values", + "X-Header-Add-Append": "append-val-1,header-val-2", + "X-Header-Echo": "echo", + }, + }, + AbsentHeaders: []string{"X-Header-Remove"}, + }, + Response: http.Response{ + Headers: map[string]string{ + "X-Header-Set-1": "header-set-1", + "X-Header-Set-2": "header-set-2", + "X-Header-Add-1": "header-add-1", + "X-Header-Add-2": "add-val-2,header-add-2", + "Another-Header": "another-header-val", + "X-Header-Echo": "echo", + }, + AbsentHeaders: []string{"X-Header-Remove-1", "X-Header-Remove-2"}, + }, + Backend: "infra-backend-v1", + Namespace: ns, }} for i := range testCases { diff --git a/conformance/tests/httproute-response-header-modifier.yaml b/conformance/tests/httproute-response-header-modifier.yaml index 071ab38878..68f7fc5d95 100644 --- a/conformance/tests/httproute-response-header-modifier.yaml +++ b/conformance/tests/httproute-response-header-modifier.yaml @@ -96,3 +96,38 @@ spec: backendRefs: - name: infra-backend-v1 port: 8080 + - matches: + - path: + type: PathPrefix + value: /response-and-request-header-modifiers + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: X-Header-Set-1 + value: header-set-1 + - name: X-Header-Set-2 + value: header-set-2 + add: + - name: X-Header-Add-1 + value: header-add-1 + - name: X-Header-Add-2 + value: header-add-2 + remove: + - X-Header-Remove-1 + - X-Header-Remove-2 + - type: RequestHeaderModifier + requestHeaderModifier: + set: + - name: X-Header-Set + value: set-overwrites-values + add: + - name: X-Header-Add + value: header-val-1 + - name: X-Header-Add-Append + value: header-val-2 + remove: + - X-Header-Remove + backendRefs: + - name: infra-backend-v1 + port: 8080 diff --git a/conformance/tests/httproute-rewrite-host.go b/conformance/tests/httproute-rewrite-host.go index 59a392874e..8cf4bd7c39 100644 --- a/conformance/tests/httproute-rewrite-host.go +++ b/conformance/tests/httproute-rewrite-host.go @@ -34,12 +34,17 @@ var HTTPRouteRewriteHost = suite.ConformanceTest{ ShortName: "HTTPRouteRewriteHost", Description: "An HTTPRoute with hostname rewrite filter", Manifests: []string{"tests/httproute-rewrite-host.yaml"}, - Features: []suite.SupportedFeature{suite.SupportHTTPRouteHostRewrite}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRouteHostRewrite, + }, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "rewrite-host", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{ { @@ -68,6 +73,29 @@ var HTTPRouteRewriteHost = suite.ConformanceTest{ }, Backend: "infra-backend-v2", Namespace: ns, + }, { + Request: http.Request{ + Path: "/rewrite-host-and-modify-headers", + Host: "rewrite.example", + Headers: map[string]string{ + "X-Header-Remove": "remove-val", + "X-Header-Add-Append": "append-val-1", + }, + }, + ExpectedRequest: &http.ExpectedRequest{ + Request: http.Request{ + Path: "/rewrite-host-and-modify-headers", + Host: "test.example.org", + Headers: map[string]string{ + "X-Header-Add": "header-val-1", + "X-Header-Add-Append": "append-val-1,header-val-2", + "X-Header-Set": "set-overwrites-values", + }, + }, + AbsentHeaders: []string{"X-Header-Remove"}, + }, + Backend: "infra-backend-v2", + Namespace: ns, }, } for i := range testCases { diff --git a/conformance/tests/httproute-rewrite-host.yaml b/conformance/tests/httproute-rewrite-host.yaml index 2e94734c32..c8a260ab68 100644 --- a/conformance/tests/httproute-rewrite-host.yaml +++ b/conformance/tests/httproute-rewrite-host.yaml @@ -7,23 +7,46 @@ spec: hostnames: - "rewrite.example" parentRefs: - - name: same-namespace + - name: same-namespace rules: - - matches: - - path: - type: PathPrefix - value: /one - filters: - - type: URLRewrite - urlRewrite: - hostname: one.example.org - backendRefs: - - name: infra-backend-v1 - port: 8080 - - filters: - - type: URLRewrite - urlRewrite: - hostname: example.org - backendRefs: - - name: infra-backend-v2 - port: 8080 + - matches: + - path: + type: PathPrefix + value: /one + filters: + - type: URLRewrite + urlRewrite: + hostname: one.example.org + backendRefs: + - name: infra-backend-v1 + port: 8080 + - filters: + - type: URLRewrite + urlRewrite: + hostname: example.org + backendRefs: + - name: infra-backend-v2 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /rewrite-host-and-modify-headers + filters: + - type: URLRewrite + urlRewrite: + hostname: test.example.org + - type: RequestHeaderModifier + requestHeaderModifier: + set: + - name: X-Header-Set + value: set-overwrites-values + add: + - name: X-Header-Add + value: header-val-1 + - name: X-Header-Add-Append + value: header-val-2 + remove: + - X-Header-Remove + backendRefs: + - name: infra-backend-v2 + port: 8080 diff --git a/conformance/tests/httproute-rewrite-path.go b/conformance/tests/httproute-rewrite-path.go index 2e8539e9e7..bc9b849c87 100644 --- a/conformance/tests/httproute-rewrite-path.go +++ b/conformance/tests/httproute-rewrite-path.go @@ -34,12 +34,17 @@ var HTTPRouteRewritePath = suite.ConformanceTest{ ShortName: "HTTPRouteRewritePath", Description: "An HTTPRoute with path rewrite filter", Manifests: []string{"tests/httproute-rewrite-path.yaml"}, - Features: []suite.SupportedFeature{suite.SupportHTTPRoutePathRewrite}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + suite.SupportHTTPRoutePathRewrite, + }, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := "gateway-conformance-infra" routeNN := types.NamespacedName{Name: "rewrite-path", Namespace: ns} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: ns} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) testCases := []http.ExpectedResponse{ { @@ -66,6 +71,52 @@ var HTTPRouteRewritePath = suite.ConformanceTest{ Backend: "infra-backend-v1", Namespace: ns, }, + { + Request: http.Request{ + Path: "/full/rewrite-path-and-modify-headers/test", + Headers: map[string]string{ + "X-Header-Remove": "remove-val", + "X-Header-Add-Append": "append-val-1", + "X-Header-Set": "set-val", + }, + }, + ExpectedRequest: &http.ExpectedRequest{ + Request: http.Request{ + Path: "/test", + Headers: map[string]string{ + "X-Header-Add": "header-val-1", + "X-Header-Add-Append": "append-val-1,header-val-2", + "X-Header-Set": "set-overwrites-values", + }, + }, + AbsentHeaders: []string{"X-Header-Remove"}, + }, + Backend: "infra-backend-v1", + Namespace: ns, + }, + { + Request: http.Request{ + Path: "/prefix/rewrite-path-and-modify-headers/one", + Headers: map[string]string{ + "X-Header-Remove": "remove-val", + "X-Header-Add-Append": "append-val-1", + "X-Header-Set": "set-val", + }, + }, + ExpectedRequest: &http.ExpectedRequest{ + Request: http.Request{ + Path: "/prefix/one", + Headers: map[string]string{ + "X-Header-Add": "header-val-1", + "X-Header-Add-Append": "append-val-1,header-val-2", + "X-Header-Set": "set-overwrites-values", + }, + }, + AbsentHeaders: []string{"X-Header-Remove"}, + }, + Backend: "infra-backend-v1", + Namespace: ns, + }, } for i := range testCases { // Declare tc here to avoid loop variable diff --git a/conformance/tests/httproute-rewrite-path.yaml b/conformance/tests/httproute-rewrite-path.yaml index 5f94a8d20d..d09dd8afb3 100644 --- a/conformance/tests/httproute-rewrite-path.yaml +++ b/conformance/tests/httproute-rewrite-path.yaml @@ -33,3 +33,53 @@ spec: backendRefs: - name: infra-backend-v1 port: 8080 + - matches: + - path: + type: PathPrefix + value: /full/rewrite-path-and-modify-headers + filters: + - type: URLRewrite + urlRewrite: + path: + type: ReplaceFullPath + replaceFullPath: /test + - type: RequestHeaderModifier + requestHeaderModifier: + set: + - name: X-Header-Set + value: set-overwrites-values + add: + - name: X-Header-Add + value: header-val-1 + - name: X-Header-Add-Append + value: header-val-2 + remove: + - X-Header-Remove + backendRefs: + - name: infra-backend-v1 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /prefix/rewrite-path-and-modify-headers + filters: + - type: URLRewrite + urlRewrite: + path: + type: ReplacePrefixMatch + replacePrefixMatch: /prefix + - type: RequestHeaderModifier + requestHeaderModifier: + set: + - name: X-Header-Set + value: set-overwrites-values + add: + - name: X-Header-Add + value: header-val-1 + - name: X-Header-Add-Append + value: header-val-2 + remove: + - X-Header-Remove + backendRefs: + - name: infra-backend-v1 + port: 8080 diff --git a/conformance/tests/httproute-simple-same-namespace.go b/conformance/tests/httproute-simple-same-namespace.go index 910b901d52..c60bb40cb8 100644 --- a/conformance/tests/httproute-simple-same-namespace.go +++ b/conformance/tests/httproute-simple-same-namespace.go @@ -34,12 +34,17 @@ func init() { var HTTPRouteSimpleSameNamespace = suite.ConformanceTest{ ShortName: "HTTPRouteSimpleSameNamespace", Description: "A single HTTPRoute in the gateway-conformance-infra namespace attaches to a Gateway in the same namespace", - Manifests: []string{"tests/httproute-simple-same-namespace.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportHTTPRoute, + }, + Manifests: []string{"tests/httproute-simple-same-namespace.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { ns := v1beta1.Namespace("gateway-conformance-infra") routeNN := types.NamespacedName{Name: "gateway-conformance-infra-test", Namespace: string(ns)} gwNN := types.NamespacedName{Name: "same-namespace", Namespace: string(ns)} gwAddr := kubernetes.GatewayAndHTTPRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + kubernetes.HTTPRouteMustHaveResolvedRefsConditionsTrue(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN) t.Run("Simple HTTP request should reach infra-backend", func(t *testing.T) { http.MakeRequestAndExpectEventuallyConsistentResponse(t, suite.RoundTripper, suite.TimeoutConfig, gwAddr, http.ExpectedResponse{ diff --git a/conformance/tests/mesh-basic.go b/conformance/tests/mesh-basic.go new file mode 100644 index 0000000000..b583a3aaa3 --- /dev/null +++ b/conformance/tests/mesh-basic.go @@ -0,0 +1,58 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "sigs.k8s.io/gateway-api/conformance/utils/echo" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, MeshBasic) +} + +var MeshBasic = suite.ConformanceTest{ + ShortName: "MeshBasic", + Description: "A mesh client can communicate with a mesh server. This tests basic reachability with no configuration applied.", + Features: []suite.SupportedFeature{ + suite.SupportMesh, + }, + Manifests: []string{}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + client := echo.ConnectToApp(t, s, echo.MeshAppEchoV1) + cases := []http.ExpectedResponse{{ + Request: http.Request{ + Host: "echo", + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + }, + }} + for i := range cases { + // Declare tc here to avoid loop variable + // reuse issues across parallel tests. + tc := cases[i] + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + client.MakeRequestAndExpectEventuallyConsistentResponse(t, tc, s.TimeoutConfig) + }) + } + }, +} diff --git a/conformance/tests/mesh-consumer-route.go b/conformance/tests/mesh-consumer-route.go new file mode 100644 index 0000000000..062e4a2ea7 --- /dev/null +++ b/conformance/tests/mesh-consumer-route.go @@ -0,0 +1,86 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "sigs.k8s.io/gateway-api/conformance/utils/echo" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, MeshConsumerRoute) +} + +var MeshConsumerRoute = suite.ConformanceTest{ + ShortName: "MeshConsumerRoute", + Description: "An HTTPRoute in a namespace other than its parentRef's namespace only affects requests from the route's namespace", + Features: []suite.SupportedFeature{ + suite.SupportMesh, + suite.SupportHTTPRoute, + suite.SupportHTTPResponseHeaderModification, + }, + Manifests: []string{"tests/mesh-consumer-route.yaml"}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + consumerClient := echo.ConnectToAppInNamespace(t, s, echo.MeshAppEchoV1, "gateway-conformance-mesh-consumer") + consumerCases := []http.ExpectedResponse{ + { + TestCaseName: "request from consumer route's namespace modified by HTTPRoute", + Request: http.Request{ + Host: "echo-v1.gateway-conformance-mesh", + Method: "GET", + Path: "/", + }, + Response: http.Response{ + StatusCode: 200, + Headers: map[string]string{ + "X-Header-Set": "set", + }, + }, + Backend: "echo-v1", + }, + } + producerClient := echo.ConnectToAppInNamespace(t, s, echo.MeshAppEchoV1, "gateway-conformance-mesh") + producerCases := []http.ExpectedResponse{ + { + TestCaseName: "request not from consumer route's namespace not modified by HTTPRoute", + Request: http.Request{ + Host: "echo-v1.gateway-conformance-mesh", + Method: "GET", + Path: "/", + }, + Response: http.Response{ + StatusCode: 200, + AbsentHeaders: []string{"X-Header-Set"}, + }, + Backend: "echo-v1", + }, + } + for i, tc := range consumerCases { + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + consumerClient.MakeRequestAndExpectEventuallyConsistentResponse(t, tc, s.TimeoutConfig) + }) + } + for i, tc := range producerCases { + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + producerClient.MakeRequestAndExpectEventuallyConsistentResponse(t, tc, s.TimeoutConfig) + }) + } + }, +} diff --git a/conformance/tests/mesh-consumer-route.yaml b/conformance/tests/mesh-consumer-route.yaml new file mode 100644 index 0000000000..1258674fec --- /dev/null +++ b/conformance/tests/mesh-consumer-route.yaml @@ -0,0 +1,21 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mesh-echo-add-header + namespace: gateway-conformance-mesh-consumer +spec: + parentRefs: + - kind: Service + name: echo-v1 + namespace: gateway-conformance-mesh + rules: + - filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: X-Header-Set + value: set + backendRefs: + - name: echo-v1 + namespace: gateway-conformance-mesh + port: 80 diff --git a/conformance/tests/mesh-frontend-hostname.go b/conformance/tests/mesh-frontend-hostname.go new file mode 100644 index 0000000000..a616eb1b07 --- /dev/null +++ b/conformance/tests/mesh-frontend-hostname.go @@ -0,0 +1,84 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "sigs.k8s.io/gateway-api/conformance/utils/echo" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, MeshFrontendHostname) +} + +var MeshFrontendHostname = suite.ConformanceTest{ + ShortName: "MeshFrontendHostname", + Description: "Mesh parentRef matches Service IP (not Host)", + Features: []suite.SupportedFeature{ + suite.SupportMesh, + }, + Manifests: []string{"tests/mesh-frontend.yaml"}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + client := echo.ConnectToApp(t, s, echo.MeshAppEchoV1) + cases := []http.ExpectedResponse{ + { + TestCaseName: "Send to service with wrong hostname", + Request: http.Request{ + Host: "echo-v2", + Headers: map[string]string{ + "Host": "echo-v1", + }, + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + // Make sure the route actually did something + Headers: map[string]string{ + "X-Header-Set": "set", + }, + }, + Backend: "echo-v2", + }, + { + TestCaseName: "Send to other service with matching hostname", + Request: http.Request{ + Host: "echo-v1", + Headers: map[string]string{ + "Host": "echo-v2", + }, + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + AbsentHeaders: []string{"X-Header-Set"}, + }, + Backend: "echo-v1", + }, + } + for i := range cases { + // Declare tc here to avoid loop variable + // reuse issues across parallel tests. + tc := cases[i] + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + client.MakeRequestAndExpectEventuallyConsistentResponse(t, tc, s.TimeoutConfig) + }) + } + }, +} diff --git a/conformance/tests/mesh-frontend.go b/conformance/tests/mesh-frontend.go new file mode 100644 index 0000000000..c9df4b923d --- /dev/null +++ b/conformance/tests/mesh-frontend.go @@ -0,0 +1,81 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "sigs.k8s.io/gateway-api/conformance/utils/echo" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, MeshFrontend) +} + +var MeshFrontend = suite.ConformanceTest{ + ShortName: "MeshFrontend", + Description: "Mesh rules should only apply to the associated frontend", + Features: []suite.SupportedFeature{ + suite.SupportMesh, + suite.SupportHTTPRoute, + suite.SupportHTTPResponseHeaderModification, + }, + Manifests: []string{"tests/mesh-frontend.yaml"}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + client := echo.ConnectToApp(t, s, echo.MeshAppEchoV1) + v2 := echo.ConnectToApp(t, s, echo.MeshAppEchoV2) + cases := []http.ExpectedResponse{ + { + TestCaseName: "Send to service", + Request: http.Request{ + Host: "echo-v2", + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + // Make sure the route actually did something + Headers: map[string]string{ + "X-Header-Set": "set", + }, + }, + Backend: "echo-v2", + }, + { + TestCaseName: "Send to pod IP", + Request: http.Request{ + Host: v2.Address, + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + AbsentHeaders: []string{"X-Header-Set"}, + }, + Backend: "echo-v2", + }, + } + for i := range cases { + // Declare tc here to avoid loop variable + // reuse issues across parallel tests. + tc := cases[i] + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + client.MakeRequestAndExpectEventuallyConsistentResponse(t, tc, s.TimeoutConfig) + }) + } + }, +} diff --git a/conformance/tests/mesh-frontend.yaml b/conformance/tests/mesh-frontend.yaml new file mode 100644 index 0000000000..75d9e96451 --- /dev/null +++ b/conformance/tests/mesh-frontend.yaml @@ -0,0 +1,19 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mesh-split-v1 + namespace: gateway-conformance-mesh +spec: + parentRefs: + - kind: Service + name: echo-v2 + rules: + - filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: X-Header-Set + value: set + backendRefs: + - name: echo-v2 + port: 80 diff --git a/conformance/tests/mesh-ports.go b/conformance/tests/mesh-ports.go new file mode 100644 index 0000000000..14de24d6d3 --- /dev/null +++ b/conformance/tests/mesh-ports.go @@ -0,0 +1,109 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "sigs.k8s.io/gateway-api/conformance/utils/echo" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, MeshPorts) +} + +var MeshPorts = suite.ConformanceTest{ + ShortName: "MeshPorts", + Description: "A mesh route can optionally configure 'port' in parentRef", + Features: []suite.SupportedFeature{ + suite.SupportMesh, + suite.SupportHTTPRoute, + suite.SupportHTTPResponseHeaderModification, + }, + Manifests: []string{"tests/mesh-ports.yaml"}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + client := echo.ConnectToApp(t, s, echo.MeshAppEchoV1) + cases := []http.ExpectedResponse{ + { + TestCaseName: "Explicit port set, send to that port", + Request: http.Request{ + Host: "echo-v1", + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + // Make sure the route actually did something + Headers: map[string]string{ + "X-Header-Set": "v1", + }, + }, + Backend: "echo-v1", + }, + { + TestCaseName: "Explicit port, send to an excluded port", + Request: http.Request{ + Host: "echo-v1:8080", + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + // Route should not apply + AbsentHeaders: []string{"X-Header-Set"}, + }, + Backend: "echo-v1", + }, + { + TestCaseName: "No port set", + Request: http.Request{ + Host: "echo-v2", + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + Headers: map[string]string{ + "X-Header-Set": "v2", + }, + }, + Backend: "echo-v2", + }, + { + TestCaseName: "No port set", + Request: http.Request{ + Host: "echo-v2:8080", + Method: "GET", + }, + Response: http.Response{ + StatusCode: 200, + Headers: map[string]string{ + "X-Header-Set": "v2", + }, + }, + Backend: "echo-v2", + }, + } + for i := range cases { + // Declare tc here to avoid loop variable + // reuse issues across parallel tests. + tc := cases[i] + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + client.MakeRequestAndExpectEventuallyConsistentResponse(t, tc, s.TimeoutConfig) + }) + } + }, +} diff --git a/conformance/tests/mesh-ports.yaml b/conformance/tests/mesh-ports.yaml new file mode 100644 index 0000000000..9432469726 --- /dev/null +++ b/conformance/tests/mesh-ports.yaml @@ -0,0 +1,40 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mesh-split-v1 + namespace: gateway-conformance-mesh +spec: + parentRefs: + - kind: Service + name: echo-v1 + port: 80 + rules: + - filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: X-Header-Set + value: v1 + backendRefs: + - name: echo-v1 + port: 80 +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mesh-split-v2 + namespace: gateway-conformance-mesh +spec: + parentRefs: + - kind: Service + name: echo-v2 + rules: + - filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: X-Header-Set + value: v2 + backendRefs: + - name: echo-v2 + port: 80 diff --git a/conformance/tests/mesh-split.go b/conformance/tests/mesh-split.go new file mode 100644 index 0000000000..2d52a698ee --- /dev/null +++ b/conformance/tests/mesh-split.go @@ -0,0 +1,73 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + "sigs.k8s.io/gateway-api/conformance/utils/echo" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, MeshTrafficSplit) +} + +var MeshTrafficSplit = suite.ConformanceTest{ + ShortName: "MeshTrafficSplit", + Description: "A mesh client can send traffic to a Service which is split between two versions", + Features: []suite.SupportedFeature{ + suite.SupportMesh, + }, + Manifests: []string{"tests/mesh-split.yaml"}, + Test: func(t *testing.T, s *suite.ConformanceTestSuite) { + client := echo.ConnectToApp(t, s, echo.MeshAppEchoV1) + cases := []http.ExpectedResponse{ + { + Request: http.Request{ + Host: "echo", + Method: "GET", + Path: "/v1", + }, + Response: http.Response{ + StatusCode: 200, + }, + Backend: "echo-v1", + }, + { + Request: http.Request{ + Host: "echo", + Method: "GET", + Path: "/v2", + }, + Response: http.Response{ + StatusCode: 200, + }, + Backend: "echo-v2", + }, + } + for i := range cases { + // Declare tc here to avoid loop variable + // reuse issues across parallel tests. + tc := cases[i] + t.Run(tc.GetTestCaseName(i), func(t *testing.T) { + client.MakeRequestAndExpectEventuallyConsistentResponse(t, tc, s.TimeoutConfig) + }) + } + }, +} diff --git a/conformance/tests/mesh-split.yaml b/conformance/tests/mesh-split.yaml new file mode 100644 index 0000000000..293eef9faf --- /dev/null +++ b/conformance/tests/mesh-split.yaml @@ -0,0 +1,24 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mesh-split + namespace: gateway-conformance-mesh +spec: + parentRefs: + - kind: Service + name: echo + rules: + - matches: + - path: + type: Exact + value: /v1 + backendRefs: + - name: echo-v1 + port: 80 + - matches: + - path: + type: Exact + value: /v2 + backendRefs: + - name: echo-v2 + port: 80 diff --git a/conformance/tests/tlsroute-invalid-reference-grant.go b/conformance/tests/tlsroute-invalid-reference-grant.go new file mode 100644 index 0000000000..0f9b097111 --- /dev/null +++ b/conformance/tests/tlsroute-invalid-reference-grant.go @@ -0,0 +1,59 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package tests + +import ( + "testing" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + + "sigs.k8s.io/gateway-api/apis/v1beta1" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +func init() { + ConformanceTests = append(ConformanceTests, TLSRouteInvalidReferenceGrant) +} + +var TLSRouteInvalidReferenceGrant = suite.ConformanceTest{ + ShortName: "TLSRouteInvalidReferenceGrant", + Description: "A single TLSRoute in the gateway-conformance-infra namespace, with a backendRef in another namespace without valid ReferenceGrant, should have the ResolvedRefs condition set to False", + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportTLSRoute, + suite.SupportReferenceGrant, + }, + Manifests: []string{"tests/tlsroute-invalid-reference-grant.yaml"}, + Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { + routeNN := types.NamespacedName{Name: "gateway-conformance-infra-test", Namespace: "gateway-conformance-infra"} + gwNN := types.NamespacedName{Name: "gateway-tlsroute-referencegrant", Namespace: "gateway-conformance-infra"} + + kubernetes.GatewayAndTLSRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) + + t.Run("TLSRoute with BackendRef in another namespace and no ReferenceGrant covering the Service has a ResolvedRefs Condition with status False and Reason RefNotPermitted", func(t *testing.T) { + resolvedRefsCond := metav1.Condition{ + Type: string(v1beta1.RouteConditionResolvedRefs), + Status: metav1.ConditionFalse, + Reason: string(v1beta1.RouteReasonRefNotPermitted), + } + + kubernetes.TLSRouteMustHaveCondition(t, suite.Client, suite.TimeoutConfig, routeNN, gwNN, resolvedRefsCond) + }) + }, +} diff --git a/conformance/tests/tlsroute-invalid-reference-grant.yaml b/conformance/tests/tlsroute-invalid-reference-grant.yaml new file mode 100644 index 0000000000..13d6040977 --- /dev/null +++ b/conformance/tests/tlsroute-invalid-reference-grant.yaml @@ -0,0 +1,141 @@ +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-namespace + namespace: gateway-conformance-infra +spec: + from: + - group: gateway.networking.k8s.io + kind: TLSRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Service + name: tls-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-from-group + namespace: gateway-conformance-app-backend +spec: + from: + - group: not-the-group-youre-looking-for + kind: TLSRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Service + name: tls-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-from-kind + namespace: gateway-conformance-app-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: Gateway + namespace: gateway-conformance-infra + to: + - group: "" + kind: Service + name: tls-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-from-namespace + namespace: gateway-conformance-app-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: TLSRoute + namespace: not-the-namespace-youre-looking-for + to: + - group: "" + kind: Service + name: tls-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-to-group + namespace: gateway-conformance-app-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: TLSRoute + namespace: gateway-conformance-infra + to: + - group: not-the-group-youre-looking-for + kind: Service + name: tls-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-to-kind + namespace: gateway-conformance-app-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: TLSRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Secret + name: tls-backend +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: reference-grant-wrong-to-name + namespace: gateway-conformance-app-backend +spec: + from: + - group: gateway.networking.k8s.io + kind: TLSRoute + namespace: gateway-conformance-infra + to: + - group: "" + kind: Service + name: not-the-service-youre-looking-for +--- +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TLSRoute +metadata: + name: gateway-conformance-infra-test + namespace: gateway-conformance-infra +spec: + parentRefs: + - name: gateway-tlsroute-referencegrant + hostnames: + - abc.example.com + rules: + - backendRefs: + - name: tls-backend + namespace: gateway-conformance-app-backend + port: 443 +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway-tlsroute-referencegrant + namespace: gateway-conformance-infra +spec: + gatewayClassName: "{GATEWAY_CLASS_NAME}" + listeners: + - name: https + port: 443 + protocol: TLS + hostname: "*.example.com" + allowedRoutes: + namespaces: + from: Same + kinds: + - kind: TLSRoute + tls: + mode: Passthrough diff --git a/conformance/tests/tlsroute-simple-same-namespace.go b/conformance/tests/tlsroute-simple-same-namespace.go index 9002c16e5f..87481d13d9 100644 --- a/conformance/tests/tlsroute-simple-same-namespace.go +++ b/conformance/tests/tlsroute-simple-same-namespace.go @@ -27,7 +27,6 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/gateway-api/apis/v1beta1" "sigs.k8s.io/gateway-api/conformance/utils/http" "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" "sigs.k8s.io/gateway-api/conformance/utils/suite" @@ -41,12 +40,18 @@ func init() { var TLSRouteSimpleSameNamespace = suite.ConformanceTest{ ShortName: "TLSRouteSimpleSameNamespace", Description: "A single TLSRoute in the gateway-conformance-infra namespace attaches to a Gateway in the same namespace", - Manifests: []string{"tests/tlsroute-simple-same-namespace.yaml"}, + Features: []suite.SupportedFeature{ + suite.SupportGateway, + suite.SupportTLSRoute, + }, + Manifests: []string{"tests/tlsroute-simple-same-namespace.yaml"}, Test: func(t *testing.T, suite *suite.ConformanceTestSuite) { - ns := v1beta1.Namespace("gateway-conformance-infra") - routeNN := types.NamespacedName{Name: "gateway-conformance-infra-test", Namespace: string(ns)} - gwNN := types.NamespacedName{Name: "gateway-tlsroute", Namespace: string(ns)} - certNN := types.NamespacedName{Name: "tls-passthrough-checks-certificate", Namespace: string(ns)} + ns := "gateway-conformance-infra" + routeNN := types.NamespacedName{Name: "gateway-conformance-infra-test", Namespace: ns} + gwNN := types.NamespacedName{Name: "gateway-tlsroute", Namespace: ns} + certNN := types.NamespacedName{Name: "tls-passthrough-checks-certificate", Namespace: ns} + + kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, []string{ns}) gwAddr, hostnames := kubernetes.GatewayAndTLSRoutesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, suite.ControllerName, kubernetes.NewGatewayRef(gwNN), routeNN) if len(hostnames) != 1 { diff --git a/conformance/tests/tlsroute-simple-same-namespace.yaml b/conformance/tests/tlsroute-simple-same-namespace.yaml index fa50abe571..c2fe72f6cc 100644 --- a/conformance/tests/tlsroute-simple-same-namespace.yaml +++ b/conformance/tests/tlsroute-simple-same-namespace.yaml @@ -22,14 +22,14 @@ metadata: spec: gatewayClassName: "{GATEWAY_CLASS_NAME}" listeners: - - name: https - port: 443 - protocol: TLS - hostname: "*.example.com" - allowedRoutes: - namespaces: - from: Same - kinds: - - kind: TLSRoute - tls: - mode: Passthrough + - name: https + port: 443 + protocol: TLS + hostname: "*.example.com" + allowedRoutes: + namespaces: + from: Same + kinds: + - kind: TLSRoute + tls: + mode: Passthrough diff --git a/conformance/utils/config/timeout.go b/conformance/utils/config/timeout.go index d1064e7066..dbaf2bec0f 100644 --- a/conformance/utils/config/timeout.go +++ b/conformance/utils/config/timeout.go @@ -51,6 +51,10 @@ type TimeoutConfig struct { // Max value for conformant implementation: None HTTPRouteMustHaveCondition time.Duration + // TLSRouteMustHaveCondition represents the maximum time for an TLSRoute to have the supplied Condition. + // Max value for conformant implementation: None + TLSRouteMustHaveCondition time.Duration + // RouteMustHaveParents represents the maximum time for an xRoute to have parents in status that match the expected parents. // Max value for conformant implementation: None RouteMustHaveParents time.Duration @@ -63,7 +67,10 @@ type TimeoutConfig struct { // Max value for conformant implementation: 30 seconds MaxTimeToConsistency time.Duration - // NamespacesMustBeReady represents the maximum time for all Pods and Gateways in a namespaces to be marked as ready. + // NamespacesMustBeReady represents the maximum time for the following to happen within + // specified namespace(s): + // * All Pods to be marked as "Ready" + // * All Gateways to be marked as "Accepted" and "Programmed" // Max value for conformant implementation: None NamespacesMustBeReady time.Duration @@ -71,6 +78,10 @@ type TimeoutConfig struct { // Max value for conformant implementation: None RequestTimeout time.Duration + // LatestObservedGenerationSet represents the maximum time for an ObservedGeneration to bump. + // Max value for conformant implementation: None + LatestObservedGenerationSet time.Duration + // RequiredConsecutiveSuccesses is the number of requests that must succeed in a row // to consider a response "consistent" before making additional assertions on the response body. // If this number is not reached within MaxTimeToConsistency, the test will fail. @@ -88,11 +99,13 @@ func DefaultTimeoutConfig() TimeoutConfig { GWCMustBeAccepted: 180 * time.Second, HTTPRouteMustNotHaveParents: 60 * time.Second, HTTPRouteMustHaveCondition: 60 * time.Second, + TLSRouteMustHaveCondition: 60 * time.Second, RouteMustHaveParents: 60 * time.Second, ManifestFetchTimeout: 10 * time.Second, MaxTimeToConsistency: 30 * time.Second, NamespacesMustBeReady: 300 * time.Second, RequestTimeout: 10 * time.Second, + LatestObservedGenerationSet: 60 * time.Second, RequiredConsecutiveSuccesses: 3, } } @@ -138,4 +151,10 @@ func SetupTimeoutConfig(timeoutConfig *TimeoutConfig) { if timeoutConfig.RequestTimeout == 0 { timeoutConfig.RequestTimeout = defaultTimeoutConfig.RequestTimeout } + if timeoutConfig.LatestObservedGenerationSet == 0 { + timeoutConfig.LatestObservedGenerationSet = defaultTimeoutConfig.LatestObservedGenerationSet + } + if timeoutConfig.TLSRouteMustHaveCondition == 0 { + timeoutConfig.TLSRouteMustHaveCondition = defaultTimeoutConfig.TLSRouteMustHaveCondition + } } diff --git a/conformance/utils/echo/parse.go b/conformance/utils/echo/parse.go new file mode 100644 index 0000000000..a175df926c --- /dev/null +++ b/conformance/utils/echo/parse.go @@ -0,0 +1,273 @@ +/* +Copyright 2020 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package echo + +import ( + "fmt" + "net/http" + "regexp" + "sort" + "strings" +) + +// Field is a list of fields returned in responses from the Echo server. +type Field string + +const ( + RequestIDField Field = "X-Request-Id" + ServiceVersionField Field = "ServiceVersion" + ServicePortField Field = "ServicePort" + StatusCodeField Field = "StatusCode" + URLField Field = "URL" + ForwarderURLField Field = "Url" + ForwarderMessageField Field = "Echo" + ForwarderHeaderField Field = "Header" + HostField Field = "Host" + HostnameField Field = "Hostname" + MethodField Field = "Method" + ProtocolField Field = "Proto" + AlpnField Field = "Alpn" + RequestHeaderField Field = "RequestHeader" + ResponseHeaderField Field = "ResponseHeader" + ClusterField Field = "Cluster" + IPField Field = "IP" // The Requester’s IP Address. + LatencyField Field = "Latency" + ActiveRequestsField Field = "ActiveRequests" + DNSProtocolField Field = "Protocol" + DNSQueryField Field = "Query" + DNSServerField Field = "DnsServer" + CipherField Field = "Cipher" + TLSVersionField Field = "Version" + TLSServerName Field = "ServerName" +) + +var ( + requestIDFieldRegex = regexp.MustCompile("(?i)" + string(RequestIDField) + "=(.*)") + serviceVersionFieldRegex = regexp.MustCompile(string(ServiceVersionField) + "=(.*)") + servicePortFieldRegex = regexp.MustCompile(string(ServicePortField) + "=(.*)") + statusCodeFieldRegex = regexp.MustCompile(string(StatusCodeField) + "=(.*)") + hostFieldRegex = regexp.MustCompile(string(HostField) + "=(.*)") + hostnameFieldRegex = regexp.MustCompile(string(HostnameField) + "=(.*)") + requestHeaderFieldRegex = regexp.MustCompile(string(RequestHeaderField) + "=(.*)") + responseHeaderFieldRegex = regexp.MustCompile(string(ResponseHeaderField) + "=(.*)") + URLFieldRegex = regexp.MustCompile(string(URLField) + "=(.*)") + ClusterFieldRegex = regexp.MustCompile(string(ClusterField) + "=(.*)") + IPFieldRegex = regexp.MustCompile(string(IPField) + "=(.*)") + methodFieldRegex = regexp.MustCompile(string(MethodField) + "=(.*)") + protocolFieldRegex = regexp.MustCompile(string(ProtocolField) + "=(.*)") + alpnFieldRegex = regexp.MustCompile(string(AlpnField) + "=(.*)") +) + +// Response represents a response to a single echo request. +type Response struct { + // RequestURL is the requested URL. This differs from URL, which is the just the path. + // For example, RequestURL=http://foo/bar, URL=/bar + RequestURL string + // Method used (for HTTP). + Method string + // Protocol used for the request. + Protocol string + // Alpn value (for HTTP). + Alpn string + // RawContent is the original unparsed content for this response + RawContent string + // ID is a unique identifier of the resource in the response + ID string + // URL is the url the request is sent to + URL string + // Version is the version of the resource in the response + Version string + // Port is the port of the resource in the response + Port string + // Code is the response code + Code string + // Host is the host called by the request + Host string + // Hostname is the host that responded to the request + Hostname string + // The cluster where the server is deployed. + Cluster string + // IP is the requester's ip address + IP string + // rawBody gives a map of all key/values in the body of the response. + rawBody map[string]string + RequestHeaders http.Header + ResponseHeaders http.Header +} + +func ParseResponse(output string) Response { + out := Response{ + RawContent: output, + RequestHeaders: make(http.Header), + ResponseHeaders: make(http.Header), + } + + match := requestIDFieldRegex.FindStringSubmatch(output) + if match != nil { + out.ID = match[1] + } + + match = methodFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Method = match[1] + } + + match = protocolFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Protocol = match[1] + } + + match = alpnFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Alpn = match[1] + } + + match = serviceVersionFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Version = match[1] + } + + match = servicePortFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Port = match[1] + } + + match = statusCodeFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Code = match[1] + } + + match = hostFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Host = match[1] + } + + match = hostnameFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Hostname = match[1] + } + + match = URLFieldRegex.FindStringSubmatch(output) + if match != nil { + out.URL = match[1] + } + + match = ClusterFieldRegex.FindStringSubmatch(output) + if match != nil { + out.Cluster = match[1] + } + + match = IPFieldRegex.FindStringSubmatch(output) + if match != nil { + out.IP = match[1] + } + + out.rawBody = map[string]string{} + + matches := requestHeaderFieldRegex.FindAllStringSubmatch(output, -1) + for _, kv := range matches { + sl := strings.SplitN(kv[1], ":", 2) + if len(sl) != 2 { + continue + } + out.RequestHeaders.Set(sl[0], sl[1]) + } + + matches = responseHeaderFieldRegex.FindAllStringSubmatch(output, -1) + for _, kv := range matches { + sl := strings.SplitN(kv[1], ":", 2) + if len(sl) != 2 { + continue + } + out.ResponseHeaders.Set(sl[0], sl[1]) + } + + for _, l := range strings.Split(output, "\n") { + prefixSplit := strings.Split(l, "body] ") + if len(prefixSplit) != 2 { + continue + } + kv := strings.SplitN(prefixSplit[1], "=", 2) + if len(kv) != 2 { + continue + } + out.rawBody[kv[0]] = kv[1] + } + + return out +} + +// HeaderType is a helper enum for retrieving Headers from a Response. +type HeaderType string + +const ( + RequestHeader HeaderType = "request" + ResponseHeader HeaderType = "response" +) + +// GetHeaders returns the appropriate headers for the given type. +func (r Response) GetHeaders(hType HeaderType) http.Header { + switch hType { + case RequestHeader: + return r.RequestHeaders + case ResponseHeader: + return r.ResponseHeaders + default: + panic("invalid HeaderType enum: " + hType) + } +} + +// Body returns the lines of the response body, in order +func (r Response) Body() []string { + type keyValue struct { + k, v string + } + var keyValues []keyValue + // rawBody is in random order, so get the order back via sorting. + for k, v := range r.rawBody { + keyValues = append(keyValues, keyValue{k, v}) + } + sort.Slice(keyValues, func(i, j int) bool { + return keyValues[i].k < keyValues[j].k + }) + var resp []string + for _, kv := range keyValues { + resp = append(resp, kv.v) + } + return resp +} + +func (r Response) String() string { + out := "" + out += fmt.Sprintf("RawContent: %s\n", r.RawContent) + out += fmt.Sprintf("ID: %s\n", r.ID) + out += fmt.Sprintf("Method: %s\n", r.Method) + out += fmt.Sprintf("Protocol: %s\n", r.Protocol) + out += fmt.Sprintf("Alpn: %s\n", r.Alpn) + out += fmt.Sprintf("URL: %s\n", r.URL) + out += fmt.Sprintf("Version: %s\n", r.Version) + out += fmt.Sprintf("Port: %s\n", r.Port) + out += fmt.Sprintf("Code: %s\n", r.Code) + out += fmt.Sprintf("Host: %s\n", r.Host) + out += fmt.Sprintf("Hostname: %s\n", r.Hostname) + out += fmt.Sprintf("Cluster: %s\n", r.Cluster) + out += fmt.Sprintf("IP: %s\n", r.IP) + out += fmt.Sprintf("Request Headers: %v\n", r.RequestHeaders) + out += fmt.Sprintf("Response Headers: %v\n", r.ResponseHeaders) + + return out +} diff --git a/conformance/utils/echo/pod.go b/conformance/utils/echo/pod.go new file mode 100644 index 0000000000..c57642e76d --- /dev/null +++ b/conformance/utils/echo/pod.go @@ -0,0 +1,177 @@ +/* +Copyright 2020 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package echo + +import ( + "bytes" + "context" + "fmt" + "strings" + "testing" + "time" + + v1 "k8s.io/api/core/v1" + klabels "k8s.io/apimachinery/pkg/labels" + "k8s.io/client-go/kubernetes/scheme" + "k8s.io/client-go/rest" + "k8s.io/client-go/tools/remotecommand" + "sigs.k8s.io/controller-runtime/pkg/client" + + "sigs.k8s.io/gateway-api/conformance/utils/config" + "sigs.k8s.io/gateway-api/conformance/utils/http" + "sigs.k8s.io/gateway-api/conformance/utils/suite" +) + +// MeshPod represents a connection to a specific pod running in the mesh. +// This can be used to trigger requests *from* that pod. +type MeshPod struct { + Name string + Namespace string + Address string + rc rest.Interface + rcfg *rest.Config +} + +type MeshApplication string + +const ( + MeshAppEchoV1 MeshApplication = "app=echo,version=v1" + MeshAppEchoV2 MeshApplication = "app=echo,version=v2" +) + +func (m *MeshPod) MakeRequestAndExpectEventuallyConsistentResponse(t *testing.T, exp http.ExpectedResponse, timeoutConfig config.TimeoutConfig) { + t.Helper() + + req := makeRequest(exp.Request) + + http.AwaitConvergence(t, timeoutConfig.RequiredConsecutiveSuccesses, timeoutConfig.MaxTimeToConsistency, func(elapsed time.Duration) bool { + resp, err := m.request(makeRequest(exp.Request)) + if err != nil { + t.Logf("Request failed, not ready yet: %v (after %v)", err.Error(), elapsed) + return false + } + t.Logf("Got resp %v", resp) + if err := compareRequest(exp, resp); err != nil { + t.Logf("Response expectation failed for request: %v not ready yet: %v (after %v)", req, err, elapsed) + return false + } + return true + }) + + t.Logf("Request passed") +} + +func makeRequest(r http.Request) []string { + protocol := strings.ToLower(r.Protocol) + if protocol == "" { + protocol = "http" + } + args := []string{"client", fmt.Sprintf("%s://%s%s", protocol, r.Host, r.Path)} + if r.Method != "" { + args = append(args, "--method="+r.Method) + } + for k, v := range r.Headers { + args = append(args, "-H", fmt.Sprintf("%v: %v", k, v)) + } + return args +} + +func compareRequest(exp http.ExpectedResponse, resp Response) error { + want := exp.Response + if fmt.Sprint(want.StatusCode) != resp.Code { + return fmt.Errorf("wanted status code %v, got %v", want.StatusCode, resp.Code) + } + for _, name := range want.AbsentHeaders { + if v := resp.ResponseHeaders.Values(name); len(v) != 0 { + return fmt.Errorf("expected no header %q, got %v", name, v) + } + } + for k, v := range want.Headers { + if got := resp.ResponseHeaders.Get(k); got != v { + return fmt.Errorf("expected header %v=%v, got %v", k, v, got) + } + } + if !strings.HasPrefix(resp.Hostname, exp.Backend) { + return fmt.Errorf("expected pod name to start with %s, got %s", exp.Backend, resp.Hostname) + } + return nil +} + +func (m *MeshPod) request(args []string) (Response, error) { + container := "echo" + + req := m.rc.Post(). + Resource("pods"). + Name(m.Name). + Namespace(m.Namespace). + SubResource("exec"). + Param("container", container). + VersionedParams(&v1.PodExecOptions{ + Container: container, + Command: args, + Stdin: false, + Stdout: true, + Stderr: true, + TTY: false, + }, scheme.ParameterCodec) + + exec, err := remotecommand.NewSPDYExecutor(m.rcfg, "POST", req.URL()) + if err != nil { + return Response{}, err + } + + var stdoutBuf, stderrBuf bytes.Buffer + err = exec.StreamWithContext(context.Background(), remotecommand.StreamOptions{ + Stdin: nil, + Stdout: &stdoutBuf, + Stderr: &stderrBuf, + Tty: false, + }) + if err != nil { + return Response{}, err + } + + return ParseResponse(stdoutBuf.String()), nil +} + +func ConnectToApp(t *testing.T, s *suite.ConformanceTestSuite, app MeshApplication) MeshPod { + return ConnectToAppInNamespace(t, s, app, "gateway-conformance-mesh") +} + +func ConnectToAppInNamespace(t *testing.T, s *suite.ConformanceTestSuite, app MeshApplication, ns string) MeshPod { + lbls, _ := klabels.Parse(string(app)) + + podsList := v1.PodList{} + err := s.Client.List(context.Background(), &podsList, client.InNamespace(ns), client.MatchingLabelsSelector{Selector: lbls}) + if err != nil { + t.Fatalf("failed to query pods in app %v", app) + } + if len(podsList.Items) == 0 { + t.Fatalf("no pods found in app %v", app) + } + pod := podsList.Items[0] + podName := pod.Name + podNamespace := pod.Namespace + + return MeshPod{ + Name: podName, + Namespace: podNamespace, + Address: pod.Status.PodIP, + rc: s.Clientset.CoreV1().RESTClient(), + rcfg: s.RestConfig, + } +} diff --git a/conformance/utils/flags/experimental_flags.go b/conformance/utils/flags/experimental_flags.go new file mode 100644 index 0000000000..a53350ddf2 --- /dev/null +++ b/conformance/utils/flags/experimental_flags.go @@ -0,0 +1,35 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// flags contains command-line flag definitions for the conformance +// profile certification. They're in this package so they can be shared +// among the various suites that are all run by the same Makefile invocation. +package flags + +import "flag" + +var ( + ImplementationOrganization = flag.String("organization", "", "Implementation's Organization to issue conformance to") + ImplementationProject = flag.String("project", "", "Implementation's project to issue conformance to") + ImplementationUrl = flag.String("url", "", "Implementation's url to issue conformance to") + ImplementationVersion = flag.String("version", "", "Implementation's version to issue conformance to") + ImplementationContact = flag.String("contact", "", "Comma-separated list of contact information for the maintainers") + ConformanceProfiles = flag.String("conformance-profiles", "", "Comma-separated list of the conformance profiles to run") + ReportOutput = flag.String("report-output", "", "The file where to write the conformance report") +) diff --git a/conformance/utils/flags/flags.go b/conformance/utils/flags/flags.go index 5159773987..bf09d93551 100644 --- a/conformance/utils/flags/flags.go +++ b/conformance/utils/flags/flags.go @@ -24,9 +24,12 @@ import ( ) var ( - GatewayClassName = flag.String("gateway-class", "gateway-conformance", "Name of GatewayClass to use for tests") - ShowDebug = flag.Bool("debug", false, "Whether to print debug logs") - CleanupBaseResources = flag.Bool("cleanup-base-resources", true, "Whether to cleanup base test resources after the run") - SupportedFeatures = flag.String("supported-features", "", "Supported features included in conformance tests suites") - ExemptFeatures = flag.String("exempt-features", "", "Exempt Features excluded from conformance tests suites") + GatewayClassName = flag.String("gateway-class", "gateway-conformance", "Name of GatewayClass to use for tests") + ShowDebug = flag.Bool("debug", false, "Whether to print debug logs") + CleanupBaseResources = flag.Bool("cleanup-base-resources", true, "Whether to cleanup base test resources after the run") + SupportedFeatures = flag.String("supported-features", "", "Supported features included in conformance tests suites") + SkipTests = flag.String("skip-tests", "", "Comma-separated list of tests to skip") + ExemptFeatures = flag.String("exempt-features", "", "Exempt Features excluded from conformance tests suites") + EnableAllSupportedFeatures = flag.Bool("all-features", false, "Whether to enable all supported features for conformance tests") + NamespaceLabels = flag.String("namespace-labels", "", "Comma-separated list of name=value labels to add to test namespaces") ) diff --git a/conformance/utils/http/http.go b/conformance/utils/http/http.go index 93f72dabbd..a86ea48e09 100644 --- a/conformance/utils/http/http.go +++ b/conformance/utils/http/http.go @@ -18,6 +18,7 @@ package http import ( "fmt" + "net" "net/url" "strings" "testing" @@ -51,6 +52,9 @@ type ExpectedResponse struct { Backend string Namespace string + // MirroredTo is the destination pod of the mirrored request. + MirroredTo string + // User Given TestCase name TestCaseName string } @@ -64,6 +68,7 @@ type Request struct { Path string Headers map[string]string UnfollowRedirect bool + Protocol string } // ExpectedRequest defines expected properties of a request that reaches a backend. @@ -106,14 +111,15 @@ func MakeRequest(t *testing.T, expected *ExpectedResponse, gwAddr, protocol, sch expected.Response.StatusCode = 200 } - t.Logf("Making %s request to %s://%s%s", expected.Request.Method, scheme, gwAddr, expected.Request.Path) - path, query, _ := strings.Cut(expected.Request.Path, "?") + reqURL := url.URL{Scheme: scheme, Host: calculateHost(gwAddr, scheme), Path: path, RawQuery: query} + + t.Logf("Making %s request to %s", expected.Request.Method, reqURL.String()) req := roundtripper.Request{ Method: expected.Request.Method, Host: expected.Request.Host, - URL: url.URL{Scheme: scheme, Host: gwAddr, Path: path, RawQuery: query}, + URL: reqURL, Protocol: protocol, Headers: map[string][]string{}, UnfollowRedirect: expected.Request.UnfollowRedirect, @@ -134,6 +140,36 @@ func MakeRequest(t *testing.T, expected *ExpectedResponse, gwAddr, protocol, sch return req } +// calculateHost will calculate the Host header as per [HTTP spec]. To +// summarize, host will not include any port if it is implied from the scheme. In +// case of any error, the input gwAddr will be returned as the default. +// +// [HTTP spec]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.23 +func calculateHost(gwAddr, scheme string) string { + host, port, err := net.SplitHostPort(gwAddr) + if err != nil { + return gwAddr + } + if strings.ToLower(scheme) == "http" && port == "80" { + return ipv6SafeHost(host) + } + if strings.ToLower(scheme) == "https" && port == "443" { + return ipv6SafeHost(host) + } + return gwAddr +} + +func ipv6SafeHost(host string) string { + // We assume that host is a literal IPv6 address if host has + // colons. + // Per https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.2. + // This is like net.JoinHostPort, but we don't need a port. + if strings.Contains(host, ":") { + return "[" + host + "]" + } + return host +} + // AwaitConvergence runs the given function until it returns 'true' `threshold` times in a row. // Each failed attempt has a 1s delay; successful attempts have no delay. func AwaitConvergence(t *testing.T, threshold int, maxTimeToConsistency time.Duration, fn func(elapsed time.Duration) bool) { @@ -182,8 +218,8 @@ func WaitForConsistentResponse(t *testing.T, r roundtripper.RoundTripper, req ro return false } - if err := CompareRequest(&req, cReq, cRes, expected); err != nil { - t.Logf("Response expectation failed for request: %v not ready yet: %v (after %v)", req, err, elapsed) + if err := CompareRequest(t, &req, cReq, cRes, expected); err != nil { + t.Logf("Response expectation failed for request: %+v not ready yet: %v (after %v)", req, err, elapsed) return false } @@ -192,7 +228,7 @@ func WaitForConsistentResponse(t *testing.T, r roundtripper.RoundTripper, req ro t.Logf("Request passed") } -func CompareRequest(req *roundtripper.Request, cReq *roundtripper.CapturedRequest, cRes *roundtripper.CapturedResponse, expected ExpectedResponse) error { +func CompareRequest(t *testing.T, req *roundtripper.Request, cReq *roundtripper.CapturedRequest, cRes *roundtripper.CapturedResponse, expected ExpectedResponse) error { if expected.Response.StatusCode != cRes.StatusCode { return fmt.Errorf("expected status code to be %d, got %d", expected.Response.StatusCode, cRes.StatusCode) } @@ -235,7 +271,6 @@ func CompareRequest(req *roundtripper.Request, cReq *roundtripper.CapturedReques } else if strings.Join(actualVal, ",") != expectedVal { return fmt.Errorf("expected %s header to be set to %s, got %s", name, expectedVal, strings.Join(actualVal, ",")) } - } } @@ -296,19 +331,33 @@ func CompareRequest(req *roundtripper.Request, cReq *roundtripper.CapturedReques setRedirectRequestDefaults(req, cRes, &expected) if expected.RedirectRequest.Host != cRes.RedirectRequest.Host { - return fmt.Errorf("expected redirected hostname to be %s, got %s", expected.RedirectRequest.Host, cRes.RedirectRequest.Host) + return fmt.Errorf("expected redirected hostname to be %q, got %q", expected.RedirectRequest.Host, cRes.RedirectRequest.Host) } - if expected.RedirectRequest.Port != cRes.RedirectRequest.Port { - return fmt.Errorf("expected redirected port to be %s, got %s", expected.RedirectRequest.Port, cRes.RedirectRequest.Port) + gotPort := cRes.RedirectRequest.Port + if expected.RedirectRequest.Port == "" { + // If the test didn't specify any expected redirect port, we'll try to use + // the scheme to determine sensible defaults for the port. Well known + // schemes like "http" and "https" MAY skip setting any port. + if strings.ToLower(cRes.RedirectRequest.Scheme) == "http" && gotPort != "80" && gotPort != "" { + return fmt.Errorf("for http scheme, expected redirected port to be 80 or not set, got %q", gotPort) + } + if strings.ToLower(cRes.RedirectRequest.Scheme) == "https" && gotPort != "443" && gotPort != "" { + return fmt.Errorf("for https scheme, expected redirected port to be 443 or not set, got %q", gotPort) + } + t.Logf("Can't validate redirectPort for unrecognized scheme %v", cRes.RedirectRequest.Scheme) + } else if expected.RedirectRequest.Port != gotPort { + // An expected port was specified in the tests but it didn't match with + // gotPort. + return fmt.Errorf("expected redirected port to be %q, got %q", expected.RedirectRequest.Port, gotPort) } if expected.RedirectRequest.Scheme != cRes.RedirectRequest.Scheme { - return fmt.Errorf("expected redirected scheme to be %s, got %s", expected.RedirectRequest.Scheme, cRes.RedirectRequest.Scheme) + return fmt.Errorf("expected redirected scheme to be %q, got %q", expected.RedirectRequest.Scheme, cRes.RedirectRequest.Scheme) } if expected.RedirectRequest.Path != cRes.RedirectRequest.Path { - return fmt.Errorf("expected redirected path to be %s, got %s", expected.RedirectRequest.Path, cRes.RedirectRequest.Path) + return fmt.Errorf("expected redirected path to be %q, got %q", expected.RedirectRequest.Path, cRes.RedirectRequest.Path) } } return nil @@ -316,7 +365,6 @@ func CompareRequest(req *roundtripper.Request, cReq *roundtripper.CapturedReques // GetTestCaseName gets the user-defined test case name or generates one from expected response to a given request. func (er *ExpectedResponse) GetTestCaseName(i int) string { - // If TestCase name is provided then use that or else generate one. if er.TestCaseName != "" { return er.TestCaseName @@ -344,10 +392,6 @@ func setRedirectRequestDefaults(req *roundtripper.Request, cRes *roundtripper.Ca expected.RedirectRequest.Host = cRes.RedirectRequest.Host } - if expected.RedirectRequest.Port == "" { - expected.RedirectRequest.Port = req.URL.Port() - } - if expected.RedirectRequest.Scheme == "" { expected.RedirectRequest.Scheme = req.URL.Scheme } diff --git a/conformance/utils/http/mirror.go b/conformance/utils/http/mirror.go new file mode 100644 index 0000000000..6700b619d9 --- /dev/null +++ b/conformance/utils/http/mirror.go @@ -0,0 +1,59 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package http + +import ( + "fmt" + "regexp" + "testing" + "time" + + "github.com/stretchr/testify/require" + clientset "k8s.io/client-go/kubernetes" + "sigs.k8s.io/controller-runtime/pkg/client" + + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" +) + +func ExpectMirroredRequest(t *testing.T, client client.Client, clientset clientset.Interface, ns, mirrorPod, path string) { + if mirrorPod == "" { + t.Fatalf("MirroredTo wasn't provided in the testcase, this test should only check http request mirror.") + } + + require.Eventually(t, func() bool { + var mirrored bool + mirrorLogRegexp := regexp.MustCompile(fmt.Sprintf("Echoing back request made to \\%s to client", path)) + + t.Log("Searching for the mirrored request log") + t.Logf("Reading \"%s/%s\" logs", ns, mirrorPod) + logs, err := kubernetes.DumpEchoLogs(ns, mirrorPod, client, clientset) + if err != nil { + t.Logf("could not read \"%s/%s\" logs: %v", ns, mirrorPod, err) + return false + } + + for _, log := range logs { + if mirrorLogRegexp.MatchString(string(log)) { + mirrored = true + break + } + } + return mirrored + }, 60*time.Second, time.Second, "Mirrored request log wasn't found") + + t.Log("Mirrored request log found") +} diff --git a/conformance/utils/kubernetes/apply.go b/conformance/utils/kubernetes/apply.go index da1f5bfe6f..d7343d8181 100644 --- a/conformance/utils/kubernetes/apply.go +++ b/conformance/utils/kubernetes/apply.go @@ -19,6 +19,7 @@ package kubernetes import ( "bytes" "context" + "embed" "errors" "fmt" "io" @@ -33,8 +34,6 @@ import ( "k8s.io/apimachinery/pkg/util/yaml" "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/gateway-api/apis/v1beta1" - "sigs.k8s.io/gateway-api/conformance" "sigs.k8s.io/gateway-api/conformance/utils/config" ) @@ -42,50 +41,21 @@ import ( // them to the Kubernetes cluster. type Applier struct { NamespaceLabels map[string]string - // ValidUniqueListenerPorts maps each listener port of each Gateway in the - // manifests to a valid, unique port. There must be as many - // ValidUniqueListenerPorts as there are listeners in the set of manifests. - // For example, given two Gateways, each with 2 listeners, there should be - // four ValidUniqueListenerPorts. - // If empty or nil, ports are not modified. - ValidUniqueListenerPorts []v1beta1.PortNumber // GatewayClass will be used as the spec.gatewayClassName when applying Gateway resources GatewayClass string // ControllerName will be used as the spec.controllerName when applying GatewayClass resources ControllerName string + + // FS is the filesystem to use when reading manifests. + FS embed.FS } -// prepareGateway adjusts both listener ports and the gatewayClassName. It -// returns an index pointing to the next valid listener port. -func (a Applier) prepareGateway(t *testing.T, uObj *unstructured.Unstructured, portIndex int) int { +// prepareGateway adjusts the gatewayClassName. +func (a Applier) prepareGateway(t *testing.T, uObj *unstructured.Unstructured) { err := unstructured.SetNestedField(uObj.Object, a.GatewayClass, "spec", "gatewayClassName") require.NoErrorf(t, err, "error setting `spec.gatewayClassName` on %s Gateway resource", uObj.GetName()) - - if len(a.ValidUniqueListenerPorts) > 0 { - listeners, _, err := unstructured.NestedSlice(uObj.Object, "spec", "listeners") - require.NoErrorf(t, err, "error getting `spec.listeners` on %s Gateway resource", uObj.GetName()) - - for i, uListener := range listeners { - require.Less(t, portIndex, len(a.ValidUniqueListenerPorts), "not enough unassigned valid ports for `spec.listeners[%d]` on %s Gateway resource", i, uObj.GetName()) - - listener, ok := uListener.(map[string]interface{}) - require.Truef(t, ok, "unexpected type at `spec.listeners[%d]` on %s Gateway resource", i, uObj.GetName()) - - nextPort := a.ValidUniqueListenerPorts[portIndex] - err = unstructured.SetNestedField(listener, int64(nextPort), "port") - require.NoErrorf(t, err, "error setting `spec.listeners[%d].port` on %s Gateway resource", i, uObj.GetName()) - - portIndex++ - listeners[i] = listener - } - - err = unstructured.SetNestedSlice(uObj.Object, listeners, "spec", "listeners") - require.NoErrorf(t, err, "error setting `spec.listeners` on %s Gateway resource", uObj.GetName()) - } - - return portIndex } // prepareGatewayClass adjust the spec.controllerName on the resource @@ -95,11 +65,11 @@ func (a Applier) prepareGatewayClass(t *testing.T, uObj *unstructured.Unstructur } // prepareNamespace adjusts the Namespace labels. -func prepareNamespace(t *testing.T, uObj *unstructured.Unstructured, namespaceLabels map[string]string) { +func (a Applier) prepareNamespace(t *testing.T, uObj *unstructured.Unstructured) { labels, _, err := unstructured.NestedStringMap(uObj.Object, "metadata", "labels") require.NoErrorf(t, err, "error getting labels on Namespace %s", uObj.GetName()) - for k, v := range namespaceLabels { + for k, v := range a.NamespaceLabels { if labels == nil { labels = map[string]string{} } @@ -119,10 +89,6 @@ func prepareNamespace(t *testing.T, uObj *unstructured.Unstructured, namespaceLa func (a Applier) prepareResources(t *testing.T, decoder *yaml.YAMLOrJSONDecoder) ([]unstructured.Unstructured, error) { var resources []unstructured.Unstructured - // portIndex is incremented for each listener we see. For a manifest file - // with 2 gateways, each with 2 listeners, it will be incremented 4 times. - portIndex := 0 - for { uObj := unstructured.Unstructured{} if err := decoder.Decode(&uObj); err != nil { @@ -139,11 +105,11 @@ func (a Applier) prepareResources(t *testing.T, decoder *yaml.YAMLOrJSONDecoder) a.prepareGatewayClass(t, &uObj) } if uObj.GetKind() == "Gateway" { - portIndex = a.prepareGateway(t, &uObj, portIndex) + a.prepareGateway(t, &uObj) } if uObj.GetKind() == "Namespace" && uObj.GetObjectKind().GroupVersionKind().Group == "" { - prepareNamespace(t, &uObj, a.NamespaceLabels) + a.prepareNamespace(t, &uObj) } resources = append(resources, uObj) @@ -184,7 +150,7 @@ func (a Applier) MustApplyObjectsWithCleanup(t *testing.T, c client.Client, time // provided YAML file and registers a cleanup function for resources it created. // Note that this does not remove resources that already existed in the cluster. func (a Applier) MustApplyWithCleanup(t *testing.T, c client.Client, timeoutConfig config.TimeoutConfig, location string, cleanup bool) { - data, err := getContentsFromPathOrURL(location, timeoutConfig) + data, err := getContentsFromPathOrURL(a.FS, location, timeoutConfig) require.NoError(t, err) decoder := yaml.NewYAMLOrJSONDecoder(data, 4096) @@ -218,7 +184,9 @@ func (a Applier) MustApplyWithCleanup(t *testing.T, c client.Client, timeoutConf defer cancel() t.Logf("Deleting %s %s", uObj.GetName(), uObj.GetKind()) err = c.Delete(ctx, uObj) - require.NoErrorf(t, err, "error deleting resource") + if !apierrors.IsNotFound(err) { + require.NoErrorf(t, err, "error deleting resource") + } }) } continue @@ -234,7 +202,9 @@ func (a Applier) MustApplyWithCleanup(t *testing.T, c client.Client, timeoutConf defer cancel() t.Logf("Deleting %s %s", uObj.GetName(), uObj.GetKind()) err = c.Delete(ctx, uObj) - require.NoErrorf(t, err, "error deleting resource") + if !apierrors.IsNotFound(err) { + require.NoErrorf(t, err, "error deleting resource") + } }) } require.NoErrorf(t, err, "error updating resource") @@ -243,7 +213,7 @@ func (a Applier) MustApplyWithCleanup(t *testing.T, c client.Client, timeoutConf // getContentsFromPathOrURL takes a string that can either be a local file // path or an https:// URL to YAML manifests and provides the contents. -func getContentsFromPathOrURL(location string, timeoutConfig config.TimeoutConfig) (*bytes.Buffer, error) { +func getContentsFromPathOrURL(fs embed.FS, location string, timeoutConfig config.TimeoutConfig) (*bytes.Buffer, error) { if strings.HasPrefix(location, "http://") { return nil, fmt.Errorf("data can't be retrieved from %s: http is not supported, use https", location) } else if strings.HasPrefix(location, "https://") { @@ -272,7 +242,7 @@ func getContentsFromPathOrURL(location string, timeoutConfig config.TimeoutConfi } return manifests, nil } - b, err := conformance.Manifests.ReadFile(location) + b, err := fs.ReadFile(location) if err != nil { return nil, err } diff --git a/conformance/utils/kubernetes/apply_test.go b/conformance/utils/kubernetes/apply_test.go index 70f2593623..0c75753de3 100644 --- a/conformance/utils/kubernetes/apply_test.go +++ b/conformance/utils/kubernetes/apply_test.go @@ -24,7 +24,6 @@ import ( "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured" "k8s.io/apimachinery/pkg/util/yaml" - "sigs.k8s.io/gateway-api/apis/v1beta1" _ "sigs.k8s.io/gateway-api/conformance/utils/flags" ) @@ -105,7 +104,7 @@ metadata: }, }}, }, { - name: "no listener ports given", + name: "setting the gatewayClassName", applier: Applier{}, given: ` apiVersion: gateway.networking.k8s.io/v1beta1 @@ -146,119 +145,6 @@ spec: }, }, }}, - }, { - name: "multiple gateways each with multiple listeners", - applier: Applier{ - ValidUniqueListenerPorts: []v1beta1.PortNumber{8000, 8001, 8002, 8003}, - }, - given: ` -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: Gateway -metadata: - name: test -spec: - gatewayClassName: {GATEWAY_CLASS_NAME} - listeners: - - name: http - port: 80 - protocol: HTTP - allowedRoutes: - namespaces: - from: Same - - name: https - port: 443 - protocol: HTTPS - allowedRoutes: - namespaces: - from: Same ---- -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: Gateway -metadata: - name: test2 -spec: - gatewayClassName: {GATEWAY_CLASS_NAME} - listeners: - - name: http - port: 80 - protocol: HTTP - allowedRoutes: - namespaces: - from: Same - - name: https - port: 443 - protocol: HTTPS - allowedRoutes: - namespaces: - from: Same -`, - expected: []unstructured.Unstructured{{ - Object: map[string]interface{}{ - "apiVersion": "gateway.networking.k8s.io/v1beta1", - "kind": "Gateway", - "metadata": map[string]interface{}{ - "name": "test", - }, - "spec": map[string]interface{}{ - "gatewayClassName": "test-class", - "listeners": []interface{}{ - map[string]interface{}{ - "name": "http", - "port": int64(8000), - "protocol": "HTTP", - "allowedRoutes": map[string]interface{}{ - "namespaces": map[string]interface{}{ - "from": "Same", - }, - }, - }, - map[string]interface{}{ - "name": "https", - "port": int64(8001), - "protocol": "HTTPS", - "allowedRoutes": map[string]interface{}{ - "namespaces": map[string]interface{}{ - "from": "Same", - }, - }, - }, - }, - }, - }, - }, { - Object: map[string]interface{}{ - "apiVersion": "gateway.networking.k8s.io/v1beta1", - "kind": "Gateway", - "metadata": map[string]interface{}{ - "name": "test2", - }, - "spec": map[string]interface{}{ - "gatewayClassName": "test-class", - "listeners": []interface{}{ - map[string]interface{}{ - "name": "http", - "port": int64(8002), - "protocol": "HTTP", - "allowedRoutes": map[string]interface{}{ - "namespaces": map[string]interface{}{ - "from": "Same", - }, - }, - }, - map[string]interface{}{ - "name": "https", - "port": int64(8003), - "protocol": "HTTPS", - "allowedRoutes": map[string]interface{}{ - "namespaces": map[string]interface{}{ - "from": "Same", - }, - }, - }, - }, - }, - }, - }}, }, { name: "setting the controllerName for a GatewayClass", applier: Applier{}, diff --git a/conformance/utils/kubernetes/certificate.go b/conformance/utils/kubernetes/certificate.go index 1671cd12fd..93bdfea2b9 100644 --- a/conformance/utils/kubernetes/certificate.go +++ b/conformance/utils/kubernetes/certificate.go @@ -27,7 +27,6 @@ import ( "io" "math/big" "net" - "strings" "testing" "time" @@ -50,9 +49,7 @@ func MustCreateSelfSignedCertSecret(t *testing.T, namespace, secretName string, var serverKey, serverCert bytes.Buffer - host := strings.Join(hosts, ",") - - require.NoError(t, generateRSACert(host, &serverKey, &serverCert), "failed to generate RSA certificate") + require.NoError(t, generateRSACert(hosts, &serverKey, &serverCert), "failed to generate RSA certificate") data := map[string][]byte{ corev1.TLSCertKey: serverCert.Bytes(), @@ -72,7 +69,7 @@ func MustCreateSelfSignedCertSecret(t *testing.T, namespace, secretName string, } // generateRSACert generates a basic self signed certificate valid for a year -func generateRSACert(host string, keyOut, certOut io.Writer) error { +func generateRSACert(hosts []string, keyOut, certOut io.Writer) error { priv, err := rsa.GenerateKey(rand.Reader, rsaBits) if err != nil { return fmt.Errorf("failed to generate key: %w", err) @@ -82,7 +79,6 @@ func generateRSACert(host string, keyOut, certOut io.Writer) error { serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128) serialNumber, err := rand.Int(rand.Reader, serialNumberLimit) - if err != nil { return fmt.Errorf("failed to generate serial number: %w", err) } @@ -101,7 +97,6 @@ func generateRSACert(host string, keyOut, certOut io.Writer) error { BasicConstraintsValid: true, } - hosts := strings.Split(host, ",") for _, h := range hosts { if ip := net.ParseIP(h); ip != nil { template.IPAddresses = append(template.IPAddresses, ip) @@ -111,7 +106,6 @@ func generateRSACert(host string, keyOut, certOut io.Writer) error { } derBytes, err := x509.CreateCertificate(rand.Reader, &template, &template, &priv.PublicKey, priv) - if err != nil { return fmt.Errorf("failed to create certificate: %w", err) } diff --git a/conformance/utils/kubernetes/helpers.go b/conformance/utils/kubernetes/helpers.go index d5446506e6..880f0223de 100644 --- a/conformance/utils/kubernetes/helpers.go +++ b/conformance/utils/kubernetes/helpers.go @@ -32,7 +32,6 @@ import ( v1 "k8s.io/api/core/v1" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/types" - "k8s.io/apimachinery/pkg/util/sets" "k8s.io/apimachinery/pkg/util/wait" "sigs.k8s.io/controller-runtime/pkg/client" @@ -84,10 +83,7 @@ func gwcMustBeAccepted(t *testing.T, c client.Client, timeoutConfig config.Timeo t.Helper() var controllerName string - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.GWCMustBeAccepted, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.GWCMustBeAccepted, true, func(ctx context.Context) (bool, error) { gwc := &v1beta1.GatewayClass{} err := c.Get(ctx, types.NamespacedName{Name: gwcName}, gwc) if err != nil { @@ -109,14 +105,27 @@ func gwcMustBeAccepted(t *testing.T, c client.Client, timeoutConfig config.Timeo return controllerName } -// GatewayMustHaveLatestConditions will fail the test if there are -// conditions that were not updated -func GatewayMustHaveLatestConditions(t *testing.T, gw *v1beta1.Gateway) { +// GatewayMustHaveLatestConditions waits until the specified Gateway has +// all conditions updated with the latest observed generation. +func GatewayMustHaveLatestConditions(t *testing.T, c client.Client, timeoutConfig config.TimeoutConfig, gwNN types.NamespacedName) { t.Helper() - if err := ConditionsHaveLatestObservedGeneration(gw, gw.Status.Conditions); err != nil { - t.Fatalf("Gateway %v", err) - } + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.LatestObservedGenerationSet, true, func(ctx context.Context) (bool, error) { + gw := &v1beta1.Gateway{} + err := c.Get(ctx, gwNN, gw) + if err != nil { + return false, fmt.Errorf("error fetching Gateway: %w", err) + } + + if err := ConditionsHaveLatestObservedGeneration(gw, gw.Status.Conditions); err != nil { + t.Logf("Gateway %s latest conditions not set yet: %v", gwNN.String(), err) + return false, nil + } + + return true, nil + }) + + require.NoErrorf(t, waitErr, "error waiting for Gateway %s to have Latest ObservedGeneration to be set: %v", gwNN.String(), waitErr) } // GatewayClassMustHaveLatestConditions will fail the test if there are @@ -148,13 +157,14 @@ func ConditionsHaveLatestObservedGeneration(obj metav1.Object, conditions []meta return nil } + wantGeneration := obj.GetGeneration() var b strings.Builder - fmt.Fprint(&b, "expected observedGeneration to be updated for all conditions") + fmt.Fprintf(&b, "expected observedGeneration to be updated to %d for all conditions", wantGeneration) fmt.Fprintf(&b, ", only %d/%d were updated.", len(conditions)-len(staleConditions), len(conditions)) fmt.Fprintf(&b, " stale conditions are: ") for i, c := range staleConditions { - fmt.Fprintf(&b, c.Type) + fmt.Fprintf(&b, "%s (generation %d)", c.Type, c.ObservedGeneration) if i != len(staleConditions)-1 { fmt.Fprintf(&b, ", ") } @@ -175,16 +185,13 @@ func FilterStaleConditions(obj metav1.Object, conditions []metav1.Condition) []m return stale } -// NamespacesMustBeAccepted waits until all Pods are marked ready and all Gateways -// are marked accepted in the provided namespaces. This will cause the test to -// halt if the specified timeout is exceeded. -func NamespacesMustBeAccepted(t *testing.T, c client.Client, timeoutConfig config.TimeoutConfig, namespaces []string) { +// NamespacesMustBeReady waits until all Pods are marked Ready and all Gateways +// are marked Accepted and Programmed in the specified namespace(s). This will +// cause the test to halt if the specified timeout is exceeded. +func NamespacesMustBeReady(t *testing.T, c client.Client, timeoutConfig config.TimeoutConfig, namespaces []string) { t.Helper() - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.NamespacesMustBeReady, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.NamespacesMustBeReady, true, func(ctx context.Context) (bool, error) { for _, ns := range namespaces { gwList := &v1beta1.GatewayList{} err := c.List(ctx, gwList, client.InNamespace(ns)) @@ -195,13 +202,19 @@ func NamespacesMustBeAccepted(t *testing.T, c client.Client, timeoutConfig confi gw := gw if err = ConditionsHaveLatestObservedGeneration(&gw, gw.Status.Conditions); err != nil { - t.Log(err) + t.Logf("Gateway %s/%s %v", ns, gw.Name, err) return false, nil } // Passing an empty string as the Reason means that any Reason will do. if !findConditionInList(t, gw.Status.Conditions, string(v1beta1.GatewayConditionAccepted), "True", "") { - t.Logf("%s/%s Gateway not ready yet", ns, gw.Name) + t.Logf("%s/%s Gateway not Accepted yet", ns, gw.Name) + return false, nil + } + + // Passing an empty string as the Reason means that any Reason will do. + if !findConditionInList(t, gw.Status.Conditions, string(v1beta1.GatewayConditionProgrammed), "True", "") { + t.Logf("%s/%s Gateway not Programmed yet", ns, gw.Name) return false, nil } } @@ -213,7 +226,8 @@ func NamespacesMustBeAccepted(t *testing.T, c client.Client, timeoutConfig confi } for _, pod := range podList.Items { if !findPodConditionInList(t, pod.Status.Conditions, "Ready", "True") && - pod.Status.Phase != v1.PodSucceeded { + pod.Status.Phase != v1.PodSucceeded && + pod.DeletionTimestamp == nil { t.Logf("%s/%s Pod not ready yet", ns, pod.Name) return false, nil } @@ -276,10 +290,7 @@ func WaitForGatewayAddress(t *testing.T, client client.Client, timeoutConfig con t.Helper() var ipAddr, port string - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.GatewayMustHaveAddress, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.GatewayMustHaveAddress, true, func(ctx context.Context) (bool, error) { gw := &v1beta1.Gateway{} err := client.Get(ctx, gwName, gw) if err != nil { @@ -312,10 +323,10 @@ func WaitForGatewayAddress(t *testing.T, client client.Client, timeoutConfig con // may indicate a single listener with zero attached routes or no listeners. func GatewayMustHaveZeroRoutes(t *testing.T, client client.Client, timeoutConfig config.TimeoutConfig, gwName types.NamespacedName) { var gotStatus *v1beta1.GatewayStatus - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.GatewayStatusMustHaveListeners, func() (bool, error) { + + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.GatewayStatusMustHaveListeners, true, func(ctx context.Context) (bool, error) { gw := &v1beta1.Gateway{} - ctx, cancel := context.WithTimeout(context.Background(), timeoutConfig.GetTimeout) - defer cancel() + err := client.Get(ctx, gwName, gw) require.NoError(t, err, "error fetching Gateway") @@ -350,10 +361,7 @@ func HTTPRouteMustHaveNoAcceptedParents(t *testing.T, client client.Client, time var actual []v1beta1.RouteParentStatus emptyChecked := false - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.HTTPRouteMustNotHaveParents, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.HTTPRouteMustNotHaveParents, true, func(ctx context.Context) (bool, error) { route := &v1beta1.HTTPRoute{} err := client.Get(ctx, routeName, route) if err != nil { @@ -398,10 +406,7 @@ func HTTPRouteMustHaveParents(t *testing.T, client client.Client, timeoutConfig t.Helper() var actual []v1beta1.RouteParentStatus - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.RouteMustHaveParents, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.RouteMustHaveParents, true, func(ctx context.Context) (bool, error) { route := &v1beta1.HTTPRoute{} err := client.Get(ctx, routeName, route) if err != nil { @@ -430,10 +435,7 @@ func TLSRouteMustHaveParents(t *testing.T, client client.Client, timeoutConfig c var actual []v1beta1.RouteParentStatus var route v1alpha2.TLSRoute - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.RouteMustHaveParents, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.RouteMustHaveParents, true, func(ctx context.Context) (bool, error) { err := client.Get(ctx, routeName, &route) if err != nil { return false, fmt.Errorf("error fetching TLSRoute: %w", err) @@ -464,7 +466,6 @@ func parentsForRouteMatch(t *testing.T, routeName types.NamespacedName, expected return false } if !reflect.DeepEqual(aParent.ParentRef.Group, eParent.ParentRef.Group) { - t.Logf("Route %s/%s expected ParentReference.Group to be %v, got %v", routeName.Namespace, routeName.Name, eParent.ParentRef.Group, aParent.ParentRef.Group) return false } @@ -498,10 +499,7 @@ func GatewayStatusMustHaveListeners(t *testing.T, client client.Client, timeoutC t.Helper() var actual []v1beta1.ListenerStatus - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.GatewayStatusMustHaveListeners, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.GatewayStatusMustHaveListeners, true, func(ctx context.Context) (bool, error) { gw := &v1beta1.Gateway{} err := client.Get(ctx, gwNN, gw) if err != nil { @@ -524,10 +522,7 @@ func GatewayStatusMustHaveListeners(t *testing.T, client client.Client, timeoutC func HTTPRouteMustHaveCondition(t *testing.T, client client.Client, timeoutConfig config.TimeoutConfig, routeNN types.NamespacedName, gwNN types.NamespacedName, condition metav1.Condition) { t.Helper() - waitErr := wait.PollImmediate(1*time.Second, timeoutConfig.HTTPRouteMustHaveCondition, func() (bool, error) { - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) - defer cancel() - + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.HTTPRouteMustHaveCondition, true, func(ctx context.Context) (bool, error) { route := &v1beta1.HTTPRoute{} err := client.Get(ctx, routeNN, route) if err != nil { @@ -538,7 +533,6 @@ func HTTPRouteMustHaveCondition(t *testing.T, client client.Client, timeoutConfi var conditionFound bool for _, parent := range parents { if err := ConditionsHaveLatestObservedGeneration(route, parent.Conditions); err != nil { - t.Logf("HTTPRoute(parentRef=%v) %v", parentRefToString(parent.ParentRef), err) return false, nil } @@ -556,6 +550,16 @@ func HTTPRouteMustHaveCondition(t *testing.T, client client.Client, timeoutConfi require.NoErrorf(t, waitErr, "error waiting for HTTPRoute status to have a Condition matching expectations") } +// HTTPRouteMustHaveResolvedRefsConditionsTrue checks that the supplied HTTPRoute has the resolvedRefsCondition +// set to true. +func HTTPRouteMustHaveResolvedRefsConditionsTrue(t *testing.T, client client.Client, timeoutConfig config.TimeoutConfig, routeNN types.NamespacedName, gwNN types.NamespacedName) { + HTTPRouteMustHaveCondition(t, client, timeoutConfig, routeNN, gwNN, metav1.Condition{ + Type: string(v1beta1.RouteConditionResolvedRefs), + Status: metav1.ConditionTrue, + Reason: string(v1beta1.RouteReasonResolvedRefs), + }) +} + func parentRefToString(p v1beta1.ParentReference) string { if p.Namespace != nil && *p.Namespace != "" { return fmt.Sprintf("%v/%v", p.Namespace, p.Name) @@ -611,6 +615,39 @@ func GatewayAndTLSRoutesMustBeAccepted(t *testing.T, c client.Client, timeoutCon return gwAddr, hostnames } +// TLSRouteMustHaveCondition checks that the supplied TLSRoute has the supplied Condition, +// halting after the specified timeout is exceeded. +func TLSRouteMustHaveCondition(t *testing.T, client client.Client, timeoutConfig config.TimeoutConfig, routeNN types.NamespacedName, gwNN types.NamespacedName, condition metav1.Condition) { + t.Helper() + + waitErr := wait.PollUntilContextTimeout(context.Background(), 1*time.Second, timeoutConfig.TLSRouteMustHaveCondition, true, func(ctx context.Context) (bool, error) { + route := &v1alpha2.TLSRoute{} + err := client.Get(ctx, routeNN, route) + if err != nil { + return false, fmt.Errorf("error fetching TLSRoute: %w", err) + } + + parents := route.Status.Parents + var conditionFound bool + for _, parent := range parents { + if err := ConditionsHaveLatestObservedGeneration(route, parent.Conditions); err != nil { + t.Logf("TLSRoute(parentRef=%v) %v", parentRefToString(parent.ParentRef), err) + return false, nil + } + + if parent.ParentRef.Name == v1beta1.ObjectName(gwNN.Name) && (parent.ParentRef.Namespace == nil || string(*parent.ParentRef.Namespace) == gwNN.Namespace) { + if findConditionInList(t, parent.Conditions, condition.Type, string(condition.Status), condition.Reason) { + conditionFound = true + } + } + } + + return conditionFound, nil + }) + + require.NoErrorf(t, waitErr, "error waiting for TLSRoute status to have a Condition matching expectations") +} + // TODO(mikemorris): this and parentsMatch could possibly be rewritten as a generic function? func listenersMatch(t *testing.T, expected, actual []v1beta1.ListenerStatus) bool { t.Helper() @@ -620,11 +657,16 @@ func listenersMatch(t *testing.T, expected, actual []v1beta1.ListenerStatus) boo return false } - // TODO(mikemorris): Allow for arbitrarily ordered listeners - for i, eListener := range expected { - aListener := actual[i] - if aListener.Name != eListener.Name { - t.Logf("Name doesn't match") + for _, eListener := range expected { + var aListener *v1beta1.ListenerStatus + for i := range actual { + if actual[i].Name == eListener.Name { + aListener = &actual[i] + break + } + } + if aListener == nil { + t.Logf("Expected status for listener %s to be present", eListener.Name) return false } @@ -636,12 +678,29 @@ func listenersMatch(t *testing.T, expected, actual []v1beta1.ListenerStatus) boo // Ensure that the expected Listener.SupportedKinds items are present in actual Listener.SupportedKinds // Find the items instead of performing an exact match of the slice because the implementation // might support more Kinds than defined in the test - eSupportedKindsSet := sets.New(eListener.SupportedKinds...) - aSupportedKindsSet := sets.New(aListener.SupportedKinds...) - if !aSupportedKindsSet.IsSuperset(eSupportedKindsSet) { - t.Logf("Expected %v kinds to be present in SupportedKinds", eSupportedKindsSet.Difference(aSupportedKindsSet)) - return false + for _, eKind := range eListener.SupportedKinds { + found := false + + for _, aKind := range aListener.SupportedKinds { + if eKind.Group == nil { + eKind.Group = (*v1beta1.Group)(&v1beta1.GroupVersion.Group) + } + + if aKind.Group == nil { + aKind.Group = (*v1beta1.Group)(&v1beta1.GroupVersion.Group) + } + + if *eKind.Group == *aKind.Group && eKind.Kind == aKind.Kind { + found = true + break + } + } + if !found { + t.Logf("Expected Group:%s Kind:%s to be present in SupportedKinds", *eKind.Group, eKind.Kind) + return false + } } + if aListener.AttachedRoutes != eListener.AttachedRoutes { t.Logf("Expected AttachedRoutes to be %v, got %v", eListener.AttachedRoutes, aListener.AttachedRoutes) return false diff --git a/conformance/utils/kubernetes/helpers_test.go b/conformance/utils/kubernetes/helpers_test.go new file mode 100644 index 0000000000..3029bd4a23 --- /dev/null +++ b/conformance/utils/kubernetes/helpers_test.go @@ -0,0 +1,321 @@ +/* +Copyright 2022 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package kubernetes + +import ( + "fmt" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + corev1 "k8s.io/api/core/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "k8s.io/utils/pointer" + + "sigs.k8s.io/gateway-api/apis/v1alpha2" + "sigs.k8s.io/gateway-api/apis/v1beta1" +) + +// ----------------------------------------------------------------------------- +// Test - Public Functions +// ----------------------------------------------------------------------------- + +func TestNewGatewayRef(t *testing.T) { + tests := []struct { + name string + nsn types.NamespacedName + listenerNames []string + }{ + { + name: "verifying the contents of a GatewayRef with no provided listeners", + nsn: types.NamespacedName{Namespace: corev1.NamespaceDefault, Name: "fake-gateway"}, + }, + { + name: "verifying the contents of a GatewayRef listeners with one listener provided", + nsn: types.NamespacedName{Namespace: corev1.NamespaceDefault, Name: "fake-gateway"}, + listenerNames: []string{"fake-listener-1"}, + }, + { + name: "verifying the contents of a GatewayRef listeners with multiple listeners provided", + nsn: types.NamespacedName{Namespace: corev1.NamespaceDefault, Name: "fake-gateway"}, + listenerNames: []string{ + "fake-listener-1", + "fake-listener-2", + "fake-listener-3", + }, + }, + } + + for i := 0; i < len(tests); i++ { + test := tests[i] + t.Run(test.name, func(t *testing.T) { + ref := NewGatewayRef(test.nsn, test.listenerNames...) + require.IsType(t, GatewayRef{}, ref) + if test.listenerNames == nil { + require.Len(t, ref.listenerNames, 1) + assert.Equal(t, "", string(*ref.listenerNames[0])) + } else { + require.Len(t, ref.listenerNames, len(test.listenerNames)) + for i := 0; i < len(ref.listenerNames); i++ { + assert.Equal(t, test.listenerNames[i], string(*ref.listenerNames[i])) + } + } + assert.Equal(t, test.nsn, ref.NamespacedName) + }) + } +} + +func TestVerifyConditionsMatchGeneration(t *testing.T) { + tests := []struct { + name string + obj metav1.Object + conditions []metav1.Condition + expected error + }{ + {}, + { + name: "if no conditions are provided this technically passes verification", + }, + { + name: "conditions where all match the generation pass verification", + obj: &v1beta1.Gateway{ObjectMeta: metav1.ObjectMeta{Name: "fake-gateway", Generation: 20}}, + conditions: []metav1.Condition{ + {Type: "FakeCondition1", ObservedGeneration: 20}, + {Type: "FakeCondition2", ObservedGeneration: 20}, + {Type: "FakeCondition3", ObservedGeneration: 20}, + }, + }, + { + name: "conditions where one does not match the generation fail verification", + obj: &v1beta1.Gateway{ObjectMeta: metav1.ObjectMeta{Name: "fake-gateway", Generation: 20}}, + conditions: []metav1.Condition{ + {Type: "FakeCondition1", ObservedGeneration: 20}, + {Type: "FakeCondition2", ObservedGeneration: 19}, + {Type: "FakeCondition3", ObservedGeneration: 20}, + }, + expected: fmt.Errorf("expected observedGeneration to be updated to 20 for all conditions, only 2/3 were updated. stale conditions are: FakeCondition2 (generation 19)"), + }, + { + name: "conditions where most do not match the generation fail verification", + obj: &v1beta1.Gateway{ObjectMeta: metav1.ObjectMeta{Name: "fake-gateway", Generation: 20}}, + conditions: []metav1.Condition{ + {Type: "FakeCondition1", ObservedGeneration: 18}, + {Type: "FakeCondition2", ObservedGeneration: 18}, + {Type: "FakeCondition3", ObservedGeneration: 14}, + {Type: "FakeCondition4", ObservedGeneration: 20}, + {Type: "FakeCondition5", ObservedGeneration: 16}, + {Type: "FakeCondition6", ObservedGeneration: 15}, + }, + expected: fmt.Errorf("expected observedGeneration to be updated to 20 for all conditions, only 1/6 were updated. stale conditions are: FakeCondition1 (generation 18), FakeCondition2 (generation 18), FakeCondition3 (generation 14), FakeCondition5 (generation 16), FakeCondition6 (generation 15)"), + }, + } + + for i := 0; i < len(tests); i++ { + test := tests[i] + t.Run(test.name, func(t *testing.T) { + err := ConditionsHaveLatestObservedGeneration(test.obj, test.conditions) + assert.Equal(t, test.expected, err) + }) + } +} + +// ----------------------------------------------------------------------------- +// Test - Private Functions +// ----------------------------------------------------------------------------- + +func Test_listenersMatch(t *testing.T) { + tests := []struct { + name string + expected []v1beta1.ListenerStatus + actual []v1beta1.ListenerStatus + want bool + }{ + { + name: "listeners do not match if a different number of actual and expected listeners are provided", + expected: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("GRPCRoute"), + }, + }, + }, + }, + actual: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + want: false, + }, + { + name: "SupportedKinds: expected empty and actual is non empty", + expected: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{}, + }, + }, + actual: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + want: false, + }, + { + name: "SupportedKinds: expected and actual are equal", + expected: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + actual: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + want: true, + }, + { + name: "SupportedKinds: expected and actual are equal values, Group pointers are different", + expected: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + actual: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(pointer.String("gateway.networking.k8s.io")), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + want: true, + }, + { + name: "SupportedKinds: expected kind not found in actual", + expected: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + actual: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1alpha2.GroupVersion.Group), + Kind: v1beta1.Kind("GRPCRoute"), + }, + }, + }, + }, + want: false, + }, + { + name: "SupportedKinds: expected is a subset of actual", + expected: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + actual: []v1beta1.ListenerStatus{ + { + SupportedKinds: []v1beta1.RouteGroupKind{ + { + Group: (*v1beta1.Group)(&v1alpha2.GroupVersion.Group), + Kind: v1beta1.Kind("GRPCRoute"), + }, + { + Group: (*v1beta1.Group)(&v1beta1.GroupVersion.Group), + Kind: v1beta1.Kind("HTTPRoute"), + }, + }, + }, + }, + want: true, + }, + { + name: "expected and actual can be in different orders", + expected: []v1beta1.ListenerStatus{ + {Name: "listener-2"}, + {Name: "listener-3"}, + {Name: "listener-1"}, + }, + actual: []v1beta1.ListenerStatus{ + {Name: "listener-1"}, + {Name: "listener-2"}, + {Name: "listener-3"}, + }, + want: true, + }, + } + + for _, test := range tests { + test := test + t.Run(test.name, func(t *testing.T) { + assert.Equal(t, test.want, listenersMatch(t, test.expected, test.actual)) + }) + } +} diff --git a/conformance/utils/kubernetes/logs.go b/conformance/utils/kubernetes/logs.go new file mode 100644 index 0000000000..c0cd18ce5a --- /dev/null +++ b/conformance/utils/kubernetes/logs.go @@ -0,0 +1,64 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package kubernetes + +import ( + "context" + "io" + + corev1 "k8s.io/api/core/v1" + "k8s.io/apimachinery/pkg/labels" + clientset "k8s.io/client-go/kubernetes" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +// DumpEchoLogs returns logs of the echoserver pod in +// in the given namespace and with the given name. +func DumpEchoLogs(ns, name string, c client.Client, cs clientset.Interface) ([][]byte, error) { + var logs [][]byte + + pods := new(corev1.PodList) + podListOptions := &client.ListOptions{ + LabelSelector: labels.SelectorFromSet(map[string]string{"app": name}), + Namespace: ns, + } + if err := c.List(context.TODO(), pods, podListOptions); err != nil { + return nil, err + } + + podLogOptions := &corev1.PodLogOptions{ + Container: name, + } + for _, pod := range pods.Items { + if pod.Status.Phase == corev1.PodFailed { + continue + } + req := cs.CoreV1().Pods(ns).GetLogs(pod.Name, podLogOptions) + logStream, err := req.Stream(context.TODO()) + if err != nil { + continue + } + defer logStream.Close() + logBytes, err := io.ReadAll(logStream) + if err != nil { + continue + } + logs = append(logs, logBytes) + } + + return logs, nil +} diff --git a/conformance/utils/roundtripper/roundtripper.go b/conformance/utils/roundtripper/roundtripper.go index cc3aa7b937..edfcfe2671 100644 --- a/conformance/utils/roundtripper/roundtripper.go +++ b/conformance/utils/roundtripper/roundtripper.go @@ -23,7 +23,7 @@ import ( "encoding/json" "fmt" "io" - iou "io/ioutil" + "net" "net/http" "net/http/httputil" "net/url" @@ -51,6 +51,20 @@ type Request struct { Server string } +// String returns a printable version of Request for logging. Note that the +// CertPem and KeyPem are truncated. +func (r Request) String() string { + return fmt.Sprintf("{URL: %+v, Host: %v, Protocol: %v, Method: %v, Headers: %v, UnfollowRedirect: %v, Server: %v, CertPem: , KeyPem: }", + r.URL, + r.Host, + r.Protocol, + r.Method, + r.Headers, + r.UnfollowRedirect, + r.Server, + ) +} + // CapturedRequest contains request metadata captured from an echoserver // response. type CapturedRequest struct { @@ -85,8 +99,9 @@ type CapturedResponse struct { // DefaultRoundTripper is the default implementation of a RoundTripper. It will // be used if a custom implementation is not specified. type DefaultRoundTripper struct { - Debug bool - TimeoutConfig config.TimeoutConfig + Debug bool + TimeoutConfig config.TimeoutConfig + CustomDialContext func(context.Context, string, string) (net.Conn, error) } // CaptureRoundTrip makes a request with the provided parameters and returns the @@ -94,7 +109,6 @@ type DefaultRoundTripper struct { // there is an error running the function but not if an HTTP error status code // is received. func (d *DefaultRoundTripper) CaptureRoundTrip(request Request) (*CapturedRequest, *CapturedResponse, error) { - cReq := &CapturedRequest{} client := &http.Client{} if request.UnfollowRedirect { @@ -103,35 +117,17 @@ func (d *DefaultRoundTripper) CaptureRoundTrip(request Request) (*CapturedReques } } - // Setup TLS transport if there are CertPem, KeyPem, and Server in the request + transport := &http.Transport{ + DialContext: d.CustomDialContext, + } if request.Server != "" && len(request.CertPem) != 0 && len(request.KeyPem) != 0 { - // Create a certificate from the provided cert and key - cert, err := tls.X509KeyPair(request.CertPem, request.KeyPem) + tlsConfig, err := tlsClientConfig(request.Server, request.CertPem, request.KeyPem) if err != nil { - return nil, nil, fmt.Errorf("unexpected error creating cert: %w", err) - } - - // Add the provided cert as a trusted CA - certPool := x509.NewCertPool() - if !certPool.AppendCertsFromPEM(request.CertPem) { - return nil, nil, fmt.Errorf("unexpected error adding trusted CA: %w", err) - } - - if request.Server == "" { - return nil, nil, fmt.Errorf("unexpected error, server name required for TLS") - } - - // Create the Transport for this provided host, cert, and trusted CA - client.Transport = &http.Transport{ - // Disable G402: TLS MinVersion too low. (gosec) - // #nosec G402 - TLSClientConfig: &tls.Config{ - Certificates: []tls.Certificate{cert}, - ServerName: request.Server, - RootCAs: certPool, - }, + return nil, nil, err } + transport.TLSClientConfig = tlsConfig } + client.Transport = transport method := "GET" if request.Method != "" { @@ -168,9 +164,7 @@ func (d *DefaultRoundTripper) CaptureRoundTrip(request Request) (*CapturedReques if err != nil { return nil, nil, err } - defer func(Body io.ReadCloser) { - _ = Body.Close() - }(resp.Body) + defer resp.Body.Close() if d.Debug { var dump []byte @@ -182,7 +176,12 @@ func (d *DefaultRoundTripper) CaptureRoundTrip(request Request) (*CapturedReques fmt.Printf("Received Response:\n%s\n\n", formatDump(dump, "< ")) } - body, _ := iou.ReadAll(resp.Body) + cReq := &CapturedRequest{} + + body, err := io.ReadAll(resp.Body) + if err != nil { + return nil, nil, err + } // we cannot assume the response is JSON if resp.Header.Get("Content-type") == "application/json" { @@ -215,6 +214,33 @@ func (d *DefaultRoundTripper) CaptureRoundTrip(request Request) (*CapturedReques return cReq, cRes, nil } +func tlsClientConfig(server string, certPem []byte, keyPem []byte) (*tls.Config, error) { + // Create a certificate from the provided cert and key + cert, err := tls.X509KeyPair(certPem, keyPem) + if err != nil { + return nil, fmt.Errorf("unexpected error creating cert: %w", err) + } + + // Add the provided cert as a trusted CA + certPool := x509.NewCertPool() + if !certPool.AppendCertsFromPEM(certPem) { + return nil, fmt.Errorf("unexpected error adding trusted CA: %w", err) + } + + if server == "" { + return nil, fmt.Errorf("unexpected error, server name required for TLS") + } + + // Create the tls Config for this provided host, cert, and trusted CA + // Disable G402: TLS MinVersion too low. (gosec) + // #nosec G402 + return &tls.Config{ + Certificates: []tls.Certificate{cert}, + ServerName: server, + RootCAs: certPool, + }, nil +} + // IsRedirect returns true if a given status code is a redirect code. func IsRedirect(statusCode int) bool { switch statusCode { diff --git a/conformance/utils/suite/experimental_profiles.go b/conformance/utils/suite/experimental_profiles.go new file mode 100644 index 0000000000..1efd054dc6 --- /dev/null +++ b/conformance/utils/suite/experimental_profiles.go @@ -0,0 +1,138 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package suite + +import ( + "fmt" + + "k8s.io/apimachinery/pkg/util/sets" +) + +// ----------------------------------------------------------------------------- +// Conformance Profiles - Public Types +// ----------------------------------------------------------------------------- + +// ConformanceProfile is a group of features that have a related purpose, e.g. +// to cover specific protocol support or a specific feature present in Gateway +// API. +// +// For more details see the relevant GEP: https://gateway-api.sigs.k8s.io/geps/gep-1709/ +type ConformanceProfile struct { + Name ConformanceProfileName + CoreFeatures sets.Set[SupportedFeature] + ExtendedFeatures sets.Set[SupportedFeature] +} + +type ConformanceProfileName string + +const ( + // HTTPConformanceProfileName indicates the name of the conformance profile + // which covers HTTP functionality, such as the HTTPRoute API. + HTTPConformanceProfileName ConformanceProfileName = "HTTP" + + // TLSConformanceProfileName indicates the name of the conformance profile + // which covers TLS stream functionality, such as the TLSRoute API. + TLSConformanceProfileName ConformanceProfileName = "TLS" + + // MeshConformanceProfileName indicates the name of the conformance profile + // which covers service mesh functionality. + MeshConformanceProfileName ConformanceProfileName = "MESH" +) + +// ----------------------------------------------------------------------------- +// Conformance Profiles - Public Vars +// ----------------------------------------------------------------------------- + +var ( + // HTTPConformanceProfile is a ConformanceProfile that covers testing HTTP + // related functionality with Gateways. + HTTPConformanceProfile = ConformanceProfile{ + Name: HTTPConformanceProfileName, + CoreFeatures: sets.New( + SupportGateway, + SupportReferenceGrant, + SupportHTTPRoute, + ), + ExtendedFeatures: HTTPExtendedFeatures, + } + + // TLSConformanceProfile is a ConformanceProfile that covers testing TLS + // related functionality with Gateways. + TLSConformanceProfile = ConformanceProfile{ + Name: TLSConformanceProfileName, + CoreFeatures: sets.New( + SupportGateway, + SupportReferenceGrant, + SupportTLSRoute, + ), + } + + // MeshConformanceProfile is a ConformanceProfile that covers testing + // service mesh related functionality. + MeshConformanceProfile = ConformanceProfile{ + Name: MeshConformanceProfileName, + CoreFeatures: sets.New( + SupportMesh, + ), + } +) + +// ----------------------------------------------------------------------------- +// Conformance Profiles - Private Profile Mapping Helpers +// ----------------------------------------------------------------------------- + +// conformanceProfileMap maps short human-readable names to their respective +// ConformanceProfiles. +var conformanceProfileMap = map[ConformanceProfileName]ConformanceProfile{ + HTTPConformanceProfileName: HTTPConformanceProfile, + TLSConformanceProfileName: TLSConformanceProfile, + MeshConformanceProfileName: MeshConformanceProfile, +} + +// getConformanceProfileForName retrieves a known ConformanceProfile by it's simple +// human readable ConformanceProfileName. +func getConformanceProfileForName(name ConformanceProfileName) (ConformanceProfile, error) { + profile, ok := conformanceProfileMap[name] + if !ok { + return profile, fmt.Errorf("%s is not a valid conformance profile", name) + } + + return profile, nil +} + +// getConformanceProfilesForTest retrieves the ConformanceProfiles a test belongs to. +func getConformanceProfilesForTest(test ConformanceTest, conformanceProfiles sets.Set[ConformanceProfileName]) (sets.Set[*ConformanceProfile], error) { + matchingConformanceProfiles := sets.New[*ConformanceProfile]() + for _, conformanceProfileName := range conformanceProfiles.UnsortedList() { + cp := conformanceProfileMap[conformanceProfileName] + hasAllFeatures := true + for _, feature := range test.Features { + if !cp.CoreFeatures.Has(feature) && !cp.ExtendedFeatures.Has(feature) { + hasAllFeatures = false + break + } + } + if hasAllFeatures { + matchingConformanceProfiles.Insert(&cp) + } + } + + return matchingConformanceProfiles, nil +} diff --git a/conformance/utils/suite/experimental_reports.go b/conformance/utils/suite/experimental_reports.go new file mode 100644 index 0000000000..f5efba1dca --- /dev/null +++ b/conformance/utils/suite/experimental_reports.go @@ -0,0 +1,191 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package suite + +import ( + "k8s.io/apimachinery/pkg/util/sets" + confv1a1 "sigs.k8s.io/gateway-api/conformance/apis/v1alpha1" +) + +// ----------------------------------------------------------------------------- +// ConformanceReport - Private Types +// ----------------------------------------------------------------------------- + +type testResult struct { + test ConformanceTest + result resultType +} + +type resultType string + +var ( + testSucceeded resultType = "SUCCEEDED" + testFailed resultType = "FAILED" + testSkipped resultType = "SKIPPED" + testNotSupported resultType = "NOT_SUPPORTED" +) + +type profileReportsMap map[ConformanceProfileName]confv1a1.ProfileReport + +func newReports() profileReportsMap { + return make(profileReportsMap) +} + +func (p profileReportsMap) addTestResults(conformanceProfile ConformanceProfile, result testResult) { + // initialize the profile report if not already initialized + if _, ok := p[conformanceProfile.Name]; !ok { + p[conformanceProfile.Name] = confv1a1.ProfileReport{ + Name: string(conformanceProfile.Name), + } + } + + testIsExtended := isTestExtended(conformanceProfile, result.test) + report := p[conformanceProfile.Name] + + switch result.result { + case testSucceeded: + if testIsExtended { + if report.Extended == nil { + report.Extended = &confv1a1.ExtendedStatus{} + } + report.Extended.Statistics.Passed++ + + } else { + report.Core.Statistics.Passed++ + } + case testFailed: + if testIsExtended { + if report.Extended == nil { + report.Extended = &confv1a1.ExtendedStatus{} + } + if report.Extended.FailedTests == nil { + report.Extended.FailedTests = []string{} + } + report.Extended.FailedTests = append(report.Extended.FailedTests, result.test.ShortName) + report.Extended.Statistics.Failed++ + } else { + report.Core.Statistics.Failed++ + if report.Core.FailedTests == nil { + report.Core.FailedTests = []string{} + } + report.Core.FailedTests = append(report.Core.FailedTests, result.test.ShortName) + } + case testSkipped: + if testIsExtended { + if report.Extended == nil { + report.Extended = &confv1a1.ExtendedStatus{} + } + report.Extended.Statistics.Skipped++ + if report.Extended.SkippedTests == nil { + report.Extended.SkippedTests = []string{} + } + report.Extended.SkippedTests = append(report.Extended.SkippedTests, result.test.ShortName) + } else { + report.Core.Statistics.Skipped++ + if report.Core.SkippedTests == nil { + report.Core.SkippedTests = []string{} + } + report.Core.SkippedTests = append(report.Core.SkippedTests, result.test.ShortName) + } + } + p[conformanceProfile.Name] = report +} + +func (p profileReportsMap) list() (profileReports []confv1a1.ProfileReport) { + for _, profileReport := range p { + profileReports = append(profileReports, profileReport) + } + return +} + +func (p profileReportsMap) compileResults(supportedFeaturesMap map[ConformanceProfileName]sets.Set[SupportedFeature], unsupportedFeaturesMap map[ConformanceProfileName]sets.Set[SupportedFeature]) { + for key, report := range p { + // report the overall result for core features + if report.Core.Failed > 0 { + report.Core.Result = confv1a1.Failure + } else if report.Core.Skipped > 0 { + report.Core.Result = confv1a1.Partial + } else { + report.Core.Result = confv1a1.Success + } + + if report.Extended != nil { + // report the overall result for extended features + if report.Extended.Failed > 0 { + report.Extended.Result = confv1a1.Failure + } else if report.Extended.Skipped > 0 { + report.Extended.Result = confv1a1.Partial + } else { + report.Extended.Result = confv1a1.Success + } + } + p[key] = report + + supportedFeatures := supportedFeaturesMap[ConformanceProfileName(report.Name)] + if report.Extended != nil { + if supportedFeatures != nil { + if report.Extended.SupportedFeatures == nil { + report.Extended.SupportedFeatures = make([]string, 0) + } + for _, f := range supportedFeatures.UnsortedList() { + report.Extended.SupportedFeatures = append(report.Extended.SupportedFeatures, string(f)) + } + } + } + + unsupportedFeatures := unsupportedFeaturesMap[ConformanceProfileName(report.Name)] + if report.Extended != nil { + if unsupportedFeatures != nil { + if report.Extended.UnsupportedFeatures == nil { + report.Extended.UnsupportedFeatures = make([]string, 0) + } + for _, f := range unsupportedFeatures.UnsortedList() { + report.Extended.UnsupportedFeatures = append(report.Extended.UnsupportedFeatures, string(f)) + } + } + } + } +} + +// ----------------------------------------------------------------------------- +// ConformanceReport - Private Helper Functions +// ----------------------------------------------------------------------------- + +// isTestExtended determines if a provided test is considered to be supported +// at an extended level of support given the provided conformance profile. +// +// TODO: right now the tests themselves don't indicate the conformance +// support level associated with them. The only way we have right now +// in this prototype to know whether a test belongs to any particular +// conformance level is to compare the features needed for the test to +// the conformance profiles known list of core vs extended features. +// Later if we move out of Prototyping/Provisional it would probably +// be best to indicate the conformance support level of each test, but +// for now this hack works. +func isTestExtended(profile ConformanceProfile, test ConformanceTest) bool { + for _, supportedFeature := range test.Features { + // if ANY of the features needed for the test are extended features, + // then we consider the entire test extended level support. + if profile.ExtendedFeatures.Has(supportedFeature) { + return true + } + } + return false +} diff --git a/conformance/utils/suite/experimental_suite.go b/conformance/utils/suite/experimental_suite.go new file mode 100644 index 0000000000..18ce0262f7 --- /dev/null +++ b/conformance/utils/suite/experimental_suite.go @@ -0,0 +1,320 @@ +//go:build experimental +// +build experimental + +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package suite + +import ( + "errors" + "fmt" + "strings" + "sync" + "testing" + "time" + + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/util/sets" + "sigs.k8s.io/gateway-api/conformance" + confv1a1 "sigs.k8s.io/gateway-api/conformance/apis/v1alpha1" + "sigs.k8s.io/gateway-api/conformance/utils/config" + "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" + "sigs.k8s.io/gateway-api/conformance/utils/roundtripper" +) + +// ----------------------------------------------------------------------------- +// Conformance Test Suite - Public Types +// ----------------------------------------------------------------------------- + +// ConformanceTestSuite defines the test suite used to run Gateway API +// conformance tests. +// This is experimental for now and can be used as an alternative to the +// ConformanceTestSuite. Once this won't be experimental any longer, +// the two of them will be merged. +type ExperimentalConformanceTestSuite struct { + ConformanceTestSuite + + // implementation contains the details of the implementation, such as + // organization, project, etc. + implementation confv1a1.Implementation + + // conformanceProfiles is a compiled list of profiles to check + // conformance against. + conformanceProfiles sets.Set[ConformanceProfileName] + + // running indicates whether the test suite is currently running + running bool + + // results stores the pass or fail results of each test that was run by + // the test suite, organized by the tests unique name. + results map[string]testResult + + // extendedSupportedFeatures is a compiled list of named features that were + // marked as supported, and is used for reporting the test results. + extendedSupportedFeatures map[ConformanceProfileName]sets.Set[SupportedFeature] + + // extendedUnsupportedFeatures is a compiled list of named features that were + // marked as not supported, and is used for reporting the test results. + extendedUnsupportedFeatures map[ConformanceProfileName]sets.Set[SupportedFeature] + + // lock is a mutex to help ensure thread safety of the test suite object. + lock sync.RWMutex +} + +// Options can be used to initialize a ConformanceTestSuite. +type ExperimentalConformanceOptions struct { + Options + + Implementation confv1a1.Implementation + ConformanceProfiles sets.Set[ConformanceProfileName] +} + +// NewExperimentalConformanceTestSuite is a helper to use for creating a new ExperimentalConformanceTestSuite. +func NewExperimentalConformanceTestSuite(s ExperimentalConformanceOptions) (*ExperimentalConformanceTestSuite, error) { + config.SetupTimeoutConfig(&s.TimeoutConfig) + + roundTripper := s.RoundTripper + if roundTripper == nil { + roundTripper = &roundtripper.DefaultRoundTripper{Debug: s.Debug, TimeoutConfig: s.TimeoutConfig} + } + + suite := &ExperimentalConformanceTestSuite{ + results: make(map[string]testResult), + extendedUnsupportedFeatures: make(map[ConformanceProfileName]sets.Set[SupportedFeature]), + extendedSupportedFeatures: make(map[ConformanceProfileName]sets.Set[SupportedFeature]), + conformanceProfiles: s.ConformanceProfiles, + implementation: s.Implementation, + } + + // test suite callers are required to provide a conformance profile OR at + // minimum a list of features which they support. + if s.SupportedFeatures == nil && s.ConformanceProfiles.Len() == 0 && !s.EnableAllSupportedFeatures { + return nil, fmt.Errorf("no conformance profile was selected for test run, and no supported features were provided so no tests could be selected") + } + + // test suite callers can potentially just run all tests by saying they + // cover all features, if they don't they'll need to have provided a + // conformance profile or at least some specific features they support. + if s.EnableAllSupportedFeatures { + s.SupportedFeatures = AllFeatures + } else { + if s.SupportedFeatures == nil { + s.SupportedFeatures = sets.New[SupportedFeature]() + } + + for _, conformanceProfileName := range s.ConformanceProfiles.UnsortedList() { + conformanceProfile, err := getConformanceProfileForName(conformanceProfileName) + if err != nil { + return nil, fmt.Errorf("failed to retrieve conformance profile: %w", err) + } + // the use of a conformance profile implicitly enables any features of + // that profile which are supported at a Core level of support. + for _, f := range conformanceProfile.CoreFeatures.UnsortedList() { + if !s.SupportedFeatures.Has(f) { + s.SupportedFeatures.Insert(f) + } + } + for _, f := range conformanceProfile.ExtendedFeatures.UnsortedList() { + if s.SupportedFeatures.Has(f) { + if suite.extendedSupportedFeatures[conformanceProfileName] == nil { + suite.extendedSupportedFeatures[conformanceProfileName] = sets.New[SupportedFeature]() + } + suite.extendedSupportedFeatures[conformanceProfileName].Insert(f) + } else { + if suite.extendedUnsupportedFeatures[conformanceProfileName] == nil { + suite.extendedUnsupportedFeatures[conformanceProfileName] = sets.New[SupportedFeature]() + } + suite.extendedUnsupportedFeatures[conformanceProfileName].Insert(f) + } + } + } + } + + if s.FS == nil { + s.FS = &conformance.Manifests + } + + suite.ConformanceTestSuite = ConformanceTestSuite{ + Client: s.Client, + Clientset: s.Clientset, + RestConfig: s.RestConfig, + RoundTripper: roundTripper, + GatewayClassName: s.GatewayClassName, + Debug: s.Debug, + Cleanup: s.CleanupBaseResources, + BaseManifests: s.BaseManifests, + MeshManifests: s.MeshManifests, + Applier: kubernetes.Applier{ + NamespaceLabels: s.NamespaceLabels, + }, + SupportedFeatures: s.SupportedFeatures, + TimeoutConfig: s.TimeoutConfig, + SkipTests: sets.New(s.SkipTests...), + FS: *s.FS, + } + + // apply defaults + if suite.BaseManifests == "" { + suite.BaseManifests = "base/manifests.yaml" + } + if suite.MeshManifests == "" { + suite.MeshManifests = "mesh/manifests.yaml" + } + + return suite, nil +} + +// ----------------------------------------------------------------------------- +// Conformance Test Suite - Public Methods +// ----------------------------------------------------------------------------- + +// Setup ensures the base resources required for conformance tests are installed +// in the cluster. It also ensures that all relevant resources are ready. +func (suite *ExperimentalConformanceTestSuite) Setup(t *testing.T) { + suite.ConformanceTestSuite.Setup(t) +} + +// Run runs the provided set of conformance tests. +func (suite *ExperimentalConformanceTestSuite) Run(t *testing.T, tests []ConformanceTest) error { + // verify that the test suite isn't already running, don't start a new run + // until the previous run finishes + suite.lock.Lock() + if suite.running { + suite.lock.Unlock() + return fmt.Errorf("can't run the test suite multiple times in parallel: the test suite is already running.") + } + + // if the test suite is not currently running, reset reporting and start a + // new test run. + suite.running = true + suite.results = nil + suite.lock.Unlock() + + // run all tests and collect the test results for conformance reporting + results := make(map[string]testResult) + for _, test := range tests { + succeeded := t.Run(test.ShortName, func(t *testing.T) { + test.Run(t, &suite.ConformanceTestSuite) + }) + res := testSucceeded + if suite.SkipTests.Has(test.ShortName) { + res = testSkipped + } + if !suite.SupportedFeatures.HasAll(test.Features...) { + res = testNotSupported + } + + if !succeeded { + res = testFailed + } + + results[test.ShortName] = testResult{ + test: test, + result: res, + } + } + + // now that the tests have completed, mark the test suite as not running + // and report the test results. + suite.lock.Lock() + suite.running = false + suite.results = results + suite.lock.Unlock() + + return nil +} + +// Report emits a ConformanceReport for the previously completed test run. +// If no run completed prior to running the report, and error is emitted. +func (suite *ExperimentalConformanceTestSuite) Report() (*confv1a1.ConformanceReport, error) { + suite.lock.RLock() + if suite.running { + suite.lock.RUnlock() + return nil, fmt.Errorf("can't generate report: the test suite is currently running") + } + defer suite.lock.RUnlock() + + profileReports := newReports() + for _, testResult := range suite.results { + conformanceProfiles, err := getConformanceProfilesForTest(testResult.test, suite.conformanceProfiles) + if err != nil { + return nil, err + } + + for _, profile := range conformanceProfiles.UnsortedList() { + profileReports.addTestResults(*profile, testResult) + } + } + + profileReports.compileResults(suite.extendedSupportedFeatures, suite.extendedUnsupportedFeatures) + + return &confv1a1.ConformanceReport{ + TypeMeta: v1.TypeMeta{ + APIVersion: "gateway.networking.k8s.io/v1alpha1", + Kind: "ConformanceReport", + }, + Date: time.Now().Format(time.RFC3339), + Implementation: suite.implementation, + GatewayAPIVersion: "TODO", + ProfileReports: profileReports.list(), + }, nil +} + +// ParseImplementation parses implementation-specific flag arguments and +// creates a *confv1a1.Implementation. +func ParseImplementation(org, project, url, version, contact string) (*confv1a1.Implementation, error) { + if org == "" { + return nil, errors.New("implementation's organization can not be empty") + } + if project == "" { + return nil, errors.New("implementation's project can not be empty") + } + if url == "" { + return nil, errors.New("implementation's url can not be empty") + } + if version == "" { + return nil, errors.New("implementation's version can not be empty") + } + contacts := strings.SplitN(contact, ",", -1) + if len(contacts) == 0 { + return nil, errors.New("implementation's contact can not be empty") + } + + // TODO: add data validation https://github.com/kubernetes-sigs/gateway-api/issues/2178 + + return &confv1a1.Implementation{ + Organization: org, + Project: project, + URL: url, + Version: version, + Contact: contacts, + }, nil +} + +// ParseConformanceProfiles parses flag arguments and converts the string to +// sets.Set[ConformanceProfileName]. +func ParseConformanceProfiles(p string) sets.Set[ConformanceProfileName] { + res := sets.Set[ConformanceProfileName]{} + if p == "" { + return res + } + + for _, value := range strings.Split(p, ",") { + res.Insert(ConformanceProfileName(value)) + } + return res +} diff --git a/conformance/utils/suite/features.go b/conformance/utils/suite/features.go new file mode 100644 index 0000000000..2e0da4721d --- /dev/null +++ b/conformance/utils/suite/features.go @@ -0,0 +1,197 @@ +/* +Copyright 2022 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package suite + +import "k8s.io/apimachinery/pkg/util/sets" + +// ----------------------------------------------------------------------------- +// Features - Types +// ----------------------------------------------------------------------------- + +// SupportedFeature allows opting in to additional conformance tests at an +// individual feature granularity. +type SupportedFeature string + +// ----------------------------------------------------------------------------- +// Features - Standard (Core) +// ----------------------------------------------------------------------------- + +const ( + // This option indicates support for ReferenceGrant (core conformance). + // Opting out of this requires an implementation to have clearly implemented + // and documented equivalent safeguards. + SupportReferenceGrant SupportedFeature = "ReferenceGrant" + // This option indicates support for Gateway (core conformance). + // Opting out of this is allowed only for GAMMA-only implementations + SupportGateway SupportedFeature = "Gateway" +) + +// StandardCoreFeatures are the features that are required to be conformant with +// the Core API features (e.g. GatewayClass, Gateway, e.t.c.). +// +// TODO: we need clarity for standard vs experimental features. +// See: https://github.com/kubernetes-sigs/gateway-api/issues/1891 +var StandardCoreFeatures = sets.New( + SupportReferenceGrant, + SupportGateway, +) + +// ----------------------------------------------------------------------------- +// Features - Standard (Extended) +// ----------------------------------------------------------------------------- + +const ( + // This option indicates that the Gateway can also use port 8080 + SupportGatewayPort8080 SupportedFeature = "GatewayPort8080" +) + +// StandardExtendedFeatures are extra generic features that implementations may +// choose to support as an opt-in. +// +// TODO: we need clarity for standard vs experimental features. +// See: https://github.com/kubernetes-sigs/gateway-api/issues/1891 +var StandardExtendedFeatures = sets.New( + SupportGatewayPort8080, +).Insert(StandardCoreFeatures.UnsortedList()...) + +// ----------------------------------------------------------------------------- +// Features - Experimental (Extended) +// ----------------------------------------------------------------------------- + +const ( + // This option indicates support for Destination Port matching. + SupportRouteDestinationPortMatching SupportedFeature = "RouteDestinationPortMatching" +) + +// ExperimentalExtendedFeatures are extra generic features that are currently +// only available in our experimental release channel, and at an extended +// support level. +// +// TODO: we need clarity for standard vs experimental features. +// See: https://github.com/kubernetes-sigs/gateway-api/issues/1891 +var ExperimentalExtendedFeatures = sets.New( + SupportRouteDestinationPortMatching, +) + +// ----------------------------------------------------------------------------- +// Features - HTTPRoute Conformance (Core) +// ----------------------------------------------------------------------------- + +const ( + // This option indicates support for HTTPRoute + SupportHTTPRoute SupportedFeature = "HTTPRoute" +) + +// HTTPCoreFeatures includes all SupportedFeatures needed to be conformant with +// the HTTPRoute. +var HTTPCoreFeatures = sets.New( + SupportHTTPRoute, +) + +// ----------------------------------------------------------------------------- +// Features - HTTPRoute Conformance (Extended) +// ----------------------------------------------------------------------------- + +const ( + // This option indicates support for HTTPRoute query param matching (extended conformance). + SupportHTTPRouteQueryParamMatching SupportedFeature = "HTTPRouteQueryParamMatching" + + // This option indicates support for HTTPRoute method matching (extended conformance). + SupportHTTPRouteMethodMatching SupportedFeature = "HTTPRouteMethodMatching" + + // This option indicates support for HTTPRoute response header modification (extended conformance). + SupportHTTPResponseHeaderModification SupportedFeature = "HTTPResponseHeaderModification" + + // This option indicates support for HTTPRoute port redirect (extended conformance). + SupportHTTPRoutePortRedirect SupportedFeature = "HTTPRoutePortRedirect" + + // This option indicates support for HTTPRoute scheme redirect (extended conformance). + SupportHTTPRouteSchemeRedirect SupportedFeature = "HTTPRouteSchemeRedirect" + + // This option indicates support for HTTPRoute path redirect (experimental conformance). + SupportHTTPRoutePathRedirect SupportedFeature = "HTTPRoutePathRedirect" + + // This option indicates support for HTTPRoute host rewrite (experimental conformance) + SupportHTTPRouteHostRewrite SupportedFeature = "HTTPRouteHostRewrite" + + // This option indicates support for HTTPRoute path rewrite (experimental conformance) + SupportHTTPRoutePathRewrite SupportedFeature = "HTTPRoutePathRewrite" + + // This option indicates support for HTTPRoute request mirror (extended conformance). + SupportHTTPRouteRequestMirror SupportedFeature = "HTTPRouteRequestMirror" +) + +// HTTPExtendedFeatures includes all the supported features for HTTPRoute +// conformance and can be used to opt-in to run all HTTPRoute tests, including +// extended features. +var HTTPExtendedFeatures = sets.New( + SupportHTTPRouteQueryParamMatching, + SupportHTTPRouteMethodMatching, + SupportHTTPResponseHeaderModification, + SupportHTTPRoutePortRedirect, + SupportHTTPRouteSchemeRedirect, + SupportHTTPRoutePathRedirect, + SupportHTTPRouteHostRewrite, + SupportHTTPRoutePathRewrite, + SupportHTTPRouteRequestMirror, +) + +// ----------------------------------------------------------------------------- +// Features - TLSRoute Conformance (Core) +// ----------------------------------------------------------------------------- + +const ( + // This option indicates support for TLSRoute + SupportTLSRoute SupportedFeature = "TLSRoute" +) + +// TLSCoreFeatures includes all the supported features for the TLSRoute API at +// a Core level of support. +var TLSCoreFeatures = sets.New( + SupportTLSRoute, +) + +// ----------------------------------------------------------------------------- +// Features - Mesh Conformance (Core) +// ----------------------------------------------------------------------------- + +const ( + // This option indicates general support for service mesh + SupportMesh SupportedFeature = "Mesh" +) + +// MeshCoreFeatures includes all the supported features for the service mesh at +// a Core level of support. +var MeshCoreFeatures = sets.New( + SupportMesh, +) + +// ----------------------------------------------------------------------------- +// Features - Compilations +// ----------------------------------------------------------------------------- + +// AllFeatures contains all the supported features and can be used to run all +// conformance tests with `all-features` flag. +// +// NOTE: as new feature sets are added they should be inserted into this set. +var AllFeatures = sets.New[SupportedFeature](). + Insert(StandardExtendedFeatures.UnsortedList()...). + Insert(ExperimentalExtendedFeatures.UnsortedList()...). + Insert(HTTPCoreFeatures.UnsortedList()...). + Insert(HTTPExtendedFeatures.UnsortedList()...). + Insert(TLSCoreFeatures.UnsortedList()...). + Insert(MeshCoreFeatures.UnsortedList()...) diff --git a/conformance/utils/suite/suite.go b/conformance/utils/suite/suite.go index 616d787370..e95f69c37e 100644 --- a/conformance/utils/suite/suite.go +++ b/conformance/utils/suite/suite.go @@ -17,107 +17,66 @@ limitations under the License. package suite import ( + "embed" + "strings" "testing" "k8s.io/apimachinery/pkg/util/sets" + clientset "k8s.io/client-go/kubernetes" + "k8s.io/client-go/rest" "sigs.k8s.io/controller-runtime/pkg/client" - "sigs.k8s.io/gateway-api/apis/v1beta1" + "sigs.k8s.io/gateway-api/conformance" "sigs.k8s.io/gateway-api/conformance/utils/config" "sigs.k8s.io/gateway-api/conformance/utils/kubernetes" "sigs.k8s.io/gateway-api/conformance/utils/roundtripper" ) -// SupportedFeature allows opting in to additional conformance tests at an -// individual feature granularity. -type SupportedFeature string - -const ( - // This option indicates support for ReferenceGrant (core conformance). - // Opting out of this requires an implementation to have clearly implemented - // and documented equivalent safeguards. - SupportReferenceGrant SupportedFeature = "ReferenceGrant" - - // This option indicates support for TLSRoute (extended conformance). - SupportTLSRoute SupportedFeature = "TLSRoute" - - // This option indicates support for HTTPRoute query param matching (extended conformance). - SupportHTTPRouteQueryParamMatching SupportedFeature = "HTTPRouteQueryParamMatching" - - // This option indicates support for HTTPRoute method matching (extended conformance). - SupportHTTPRouteMethodMatching SupportedFeature = "HTTPRouteMethodMatching" - - // This option indicates support for HTTPRoute response header modification (extended conformance). - SupportHTTPResponseHeaderModification SupportedFeature = "HTTPResponseHeaderModification" - - // This option indicates support for Destination Port matching (extended conformance). - SupportRouteDestinationPortMatching SupportedFeature = "RouteDestinationPortMatching" - - // This option indicates GatewayClass will update the observedGeneration in it's conditions when reconciling - SupportGatewayClassObservedGenerationBump SupportedFeature = "GatewayClassObservedGenerationBump" - - // This option indicates support for HTTPRoute port redirect (extended conformance). - SupportHTTPRoutePortRedirect SupportedFeature = "HTTPRoutePortRedirect" - - // This option indicates support for HTTPRoute scheme redirect (extended conformance). - SupportHTTPRouteSchemeRedirect SupportedFeature = "HTTPRouteSchemeRedirect" - - // This option indicates support for HTTPRoute path redirect (experimental conformance). - SupportHTTPRoutePathRedirect SupportedFeature = "HTTPRoutePathRedirect" - - // This option indicates support for HTTPRoute host rewrite (experimental conformance) - SupportHTTPRouteHostRewrite SupportedFeature = "HTTPRouteHostRewrite" - - // This option indicates support for HTTPRoute path rewrite (experimental conformance) - SupportHTTPRoutePathRewrite SupportedFeature = "HTTPRoutePathRewrite" -) - -// StandardCoreFeatures are the features that are required to be conformant with -// the Core API features that are part of the Standard release channel. -var StandardCoreFeatures = map[SupportedFeature]bool{ - SupportReferenceGrant: true, -} - // ConformanceTestSuite defines the test suite used to run Gateway API // conformance tests. type ConformanceTestSuite struct { Client client.Client + Clientset clientset.Interface + RESTClient *rest.RESTClient + RestConfig *rest.Config RoundTripper roundtripper.RoundTripper GatewayClassName string ControllerName string Debug bool Cleanup bool BaseManifests string + MeshManifests string Applier kubernetes.Applier - SupportedFeatures map[SupportedFeature]bool + SupportedFeatures sets.Set[SupportedFeature] TimeoutConfig config.TimeoutConfig SkipTests sets.Set[string] + FS embed.FS } // Options can be used to initialize a ConformanceTestSuite. type Options struct { Client client.Client + Clientset clientset.Interface + RestConfig *rest.Config GatewayClassName string Debug bool RoundTripper roundtripper.RoundTripper BaseManifests string + MeshManifests string NamespaceLabels map[string]string - // ValidUniqueListenerPorts maps each listener port of each Gateway in the - // manifests to a valid, unique port. There must be as many - // ValidUniqueListenerPorts as there are listeners in the set of manifests. - // For example, given two Gateways, each with 2 listeners, there should be - // four ValidUniqueListenerPorts. - // If empty or nil, ports are not modified. - ValidUniqueListenerPorts []v1beta1.PortNumber // CleanupBaseResources indicates whether or not the base test // resources such as Gateways should be cleaned up after the run. - CleanupBaseResources bool - SupportedFeatures map[SupportedFeature]bool - TimeoutConfig config.TimeoutConfig + CleanupBaseResources bool + SupportedFeatures sets.Set[SupportedFeature] + ExemptFeatures sets.Set[SupportedFeature] + EnableAllSupportedFeatures bool + TimeoutConfig config.TimeoutConfig // SkipTests contains all the tests not to be run and can be used to opt out // of specific tests SkipTests []string + + FS *embed.FS } // New returns a new ConformanceTestSuite. @@ -129,36 +88,51 @@ func New(s Options) *ConformanceTestSuite { roundTripper = &roundtripper.DefaultRoundTripper{Debug: s.Debug, TimeoutConfig: s.TimeoutConfig} } - if s.SupportedFeatures == nil { + switch { + case s.EnableAllSupportedFeatures == true: + s.SupportedFeatures = AllFeatures + case s.SupportedFeatures == nil: s.SupportedFeatures = StandardCoreFeatures - } else { - for feature, val := range StandardCoreFeatures { - if _, ok := s.SupportedFeatures[feature]; !ok { - s.SupportedFeatures[feature] = val - } + default: + for feature := range StandardCoreFeatures { + s.SupportedFeatures.Insert(feature) } } + for feature := range s.ExemptFeatures { + s.SupportedFeatures.Delete(feature) + } + + if s.FS == nil { + s.FS = &conformance.Manifests + } + suite := &ConformanceTestSuite{ Client: s.Client, + Clientset: s.Clientset, + RestConfig: s.RestConfig, RoundTripper: roundTripper, GatewayClassName: s.GatewayClassName, Debug: s.Debug, Cleanup: s.CleanupBaseResources, BaseManifests: s.BaseManifests, + MeshManifests: s.MeshManifests, Applier: kubernetes.Applier{ - NamespaceLabels: s.NamespaceLabels, - ValidUniqueListenerPorts: s.ValidUniqueListenerPorts, + NamespaceLabels: s.NamespaceLabels, }, SupportedFeatures: s.SupportedFeatures, TimeoutConfig: s.TimeoutConfig, SkipTests: sets.New(s.SkipTests...), + FS: *s.FS, } // apply defaults if suite.BaseManifests == "" { suite.BaseManifests = "base/manifests.yaml" } + if suite.MeshManifests == "" { + suite.MeshManifests = "mesh/manifests.yaml" + } return suite } @@ -166,30 +140,48 @@ func New(s Options) *ConformanceTestSuite { // Setup ensures the base resources required for conformance tests are installed // in the cluster. It also ensures that all relevant resources are ready. func (suite *ConformanceTestSuite) Setup(t *testing.T) { - t.Logf("Test Setup: Ensuring GatewayClass has been accepted") - suite.ControllerName = kubernetes.GWCMustHaveAcceptedConditionTrue(t, suite.Client, suite.TimeoutConfig, suite.GatewayClassName) - - suite.Applier.GatewayClass = suite.GatewayClassName - suite.Applier.ControllerName = suite.ControllerName - - t.Logf("Test Setup: Applying base manifests") - suite.Applier.MustApplyWithCleanup(t, suite.Client, suite.TimeoutConfig, suite.BaseManifests, suite.Cleanup) - - t.Logf("Test Setup: Applying programmatic resources") - secret := kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-web-backend", "certificate", []string{"*"}) - suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup) - secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-infra", "tls-validity-checks-certificate", []string{"*"}) - suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup) - secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-infra", "tls-passthrough-checks-certificate", []string{"abc.example.com"}) - suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup) - - t.Logf("Test Setup: Ensuring Gateways and Pods from base manifests are ready") - namespaces := []string{ - "gateway-conformance-infra", - "gateway-conformance-app-backend", - "gateway-conformance-web-backend", - } - kubernetes.NamespacesMustBeAccepted(t, suite.Client, suite.TimeoutConfig, namespaces) + suite.Applier.FS = suite.FS + + if suite.SupportedFeatures.Has(SupportGateway) { + t.Logf("Test Setup: Ensuring GatewayClass has been accepted") + suite.ControllerName = kubernetes.GWCMustHaveAcceptedConditionTrue(t, suite.Client, suite.TimeoutConfig, suite.GatewayClassName) + + suite.Applier.GatewayClass = suite.GatewayClassName + suite.Applier.ControllerName = suite.ControllerName + + t.Logf("Test Setup: Applying base manifests") + suite.Applier.MustApplyWithCleanup(t, suite.Client, suite.TimeoutConfig, suite.BaseManifests, suite.Cleanup) + + t.Logf("Test Setup: Applying programmatic resources") + secret := kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-web-backend", "certificate", []string{"*"}) + suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup) + secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-infra", "tls-validity-checks-certificate", []string{"*", "*.org"}) + suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup) + secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-infra", "tls-passthrough-checks-certificate", []string{"abc.example.com"}) + suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup) + secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-app-backend", "tls-passthrough-checks-certificate", []string{"abc.example.com"}) + suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup) + + t.Logf("Test Setup: Ensuring Gateways and Pods from base manifests are ready") + namespaces := []string{ + "gateway-conformance-infra", + "gateway-conformance-app-backend", + "gateway-conformance-web-backend", + } + kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, namespaces) + } + if suite.SupportedFeatures.Has(SupportMesh) { + t.Logf("Test Setup: Applying base manifests") + suite.Applier.MustApplyWithCleanup(t, suite.Client, suite.TimeoutConfig, suite.MeshManifests, suite.Cleanup) + t.Logf("Test Setup: Ensuring Gateways and Pods from mesh manifests are ready") + namespaces := []string{ + "gateway-conformance-mesh", + "gateway-conformance-mesh-consumer", + "gateway-conformance-app-backend", + "gateway-conformance-web-backend", + } + kubernetes.NamespacesMustBeReady(t, suite.Client, suite.TimeoutConfig, namespaces) + } } // Run runs the provided set of conformance tests. @@ -222,15 +214,14 @@ func (test *ConformanceTest) Run(t *testing.T, suite *ConformanceTestSuite) { // Check that all features exercised by the test have been opted into by // the suite. for _, feature := range test.Features { - if supported, ok := suite.SupportedFeatures[feature]; !ok || !supported { + if !suite.SupportedFeatures.Has(feature) { t.Skipf("Skipping %s: suite does not support %s", test.ShortName, feature) } } // check that the test should not be skipped if suite.SkipTests.Has(test.ShortName) { - t.Logf("Skipping %s", test.ShortName) - return + t.Skipf("Skipping %s: test explicitly skipped", test.ShortName) } for _, manifestLocation := range test.Manifests { @@ -240,3 +231,41 @@ func (test *ConformanceTest) Run(t *testing.T, suite *ConformanceTestSuite) { test.Test(t, suite) } + +// ParseSupportedFeatures parses flag arguments and converts the string to +// sets.Set[suite.SupportedFeature] +func ParseSupportedFeatures(f string) sets.Set[SupportedFeature] { + if f == "" { + return nil + } + res := sets.Set[SupportedFeature]{} + for _, value := range strings.Split(f, ",") { + res.Insert(SupportedFeature(value)) + } + return res +} + +// ParseNamespaceLables parses flag arguments and converts the string to +// map[string]string containing label key/value pairs. +func ParseNamespaceLabels(f string) map[string]string { + if f == "" { + return nil + } + res := map[string]string{} + for _, kv := range strings.Split(f, ",") { + parts := strings.Split(kv, "=") + if len(parts) == 2 { + res[parts[0]] = parts[1] + } + } + return res +} + +// ParseSkipTests parses flag arguments and converts the string to +// []string containing the tests to be skipped. +func ParseSkipTests(t string) []string { + if t == "" { + return nil + } + return strings.Split(t, ",") +} diff --git a/conformance/utils/suite/suite_test.go b/conformance/utils/suite/suite_test.go new file mode 100644 index 0000000000..5bfacf4ae7 --- /dev/null +++ b/conformance/utils/suite/suite_test.go @@ -0,0 +1,69 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package suite + +import ( + "reflect" + "testing" + + "k8s.io/apimachinery/pkg/util/sets" +) + +func TestParseSupportedFeatures(t *testing.T) { + flags := []string{ + "", + "a", + "b,c,d", + } + + s1 := sets.Set[SupportedFeature]{} + s1.Insert(SupportedFeature("a")) + s2 := sets.Set[SupportedFeature]{} + s2.Insert(SupportedFeature("b")) + s2.Insert(SupportedFeature("c")) + s2.Insert(SupportedFeature("d")) + features := []sets.Set[SupportedFeature]{nil, s1, s2} + + for i, f := range flags { + expect := features[i] + got := ParseSupportedFeatures(f) + if !reflect.DeepEqual(got, expect) { + t.Errorf("Unexpected features from flags '%s', expected: %v, got: %v", f, expect, got) + } + } +} + +func TestParseNamespaceLabels(t *testing.T) { + flags := []string{ + "", + "a=b", + "b=c,d=e,f=g", + } + labels := []map[string]string{ + nil, + {"a": "b"}, + {"b": "c", "d": "e", "f": "g"}, + } + + for i, f := range flags { + expect := labels[i] + got := ParseNamespaceLabels(f) + if !reflect.DeepEqual(got, expect) { + t.Errorf("Unexpected labels from flags '%s', expected: %v, got: %v", f, expect, got) + } + } +} diff --git a/conformance/utils/tls/tls.go b/conformance/utils/tls/tls.go index 16e2326726..653e999a5b 100644 --- a/conformance/utils/tls/tls.go +++ b/conformance/utils/tls/tls.go @@ -53,8 +53,8 @@ func WaitForConsistentTLSResponse(t *testing.T, r roundtripper.RoundTripper, req return false } - if err := http.CompareRequest(&req, cReq, cRes, expected); err != nil { - t.Logf("Response expectation failed for request: %v not ready yet: %v (after %v)", req, err, elapsed) + if err := http.CompareRequest(t, &req, cReq, cRes, expected); err != nil { + t.Logf("Response expectation failed for request: %+v not ready yet: %v (after %v)", req, err, elapsed) return false } diff --git a/docker/Dockerfile.echo b/docker/Dockerfile.echo new file mode 100644 index 0000000000..3164bc5162 --- /dev/null +++ b/docker/Dockerfile.echo @@ -0,0 +1,2 @@ +# TODO(https://github.com/kubernetes-sigs/gateway-api/issues/1895): build it ourselves +FROM gcr.io/istio-release/app:1.17.1 diff --git a/Dockerfile b/docker/Dockerfile.webhook similarity index 95% rename from Dockerfile rename to docker/Dockerfile.webhook index 5ef9216234..2ecaab7bbc 100644 --- a/Dockerfile +++ b/docker/Dockerfile.webhook @@ -13,7 +13,7 @@ # limitations under the License. ARG BUILDPLATFORM=linux/amd64 -FROM --platform=$BUILDPLATFORM golang:1.19 AS build-env +FROM --platform=$BUILDPLATFORM golang:1.20.5 AS build-env RUN mkdir -p /go/src/sig.k8s.io/gateway-api WORKDIR /go/src/sig.k8s.io/gateway-api COPY . . diff --git a/examples/experimental/grpc-filter.yaml b/examples/experimental/grpc-filter.yaml index c8d061eb1b..3b84930bdf 100644 --- a/examples/experimental/grpc-filter.yaml +++ b/examples/experimental/grpc-filter.yaml @@ -9,12 +9,12 @@ spec: - my.filter.com rules: - filters: - - type: RequestHeaderModifier - requestHeaderModifier: - add: - - name: my-header - value: foo + - type: RequestHeaderModifier + requestHeaderModifier: + add: + - name: my-header + value: foo backendRefs: - - name: my-filter-svc1 - weight: 1 - port: 50051 + - name: my-filter-svc1 + weight: 1 + port: 50051 diff --git a/examples/experimental/http-redirect-rewrite/httproute-redirect-full.yaml b/examples/experimental/http-redirect-rewrite/httproute-redirect-full.yaml deleted file mode 100644 index 3b97f2f52b..0000000000 --- a/examples/experimental/http-redirect-rewrite/httproute-redirect-full.yaml +++ /dev/null @@ -1,25 +0,0 @@ -#$ Used in: -#$ - site-src/api-types/httproute.md -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: http-filter-redirect -spec: - hostnames: - - redirect.example - rules: - - matches: - - path: - type: PathPrefix - value: /cayenne - filters: - - type: RequestRedirect - requestRedirect: - path: - type: ReplaceFullPath - replaceFullPath: /paprika - statusCode: 302 - backendRefs: - - name: example-svc - weight: 1 - port: 80 diff --git a/examples/experimental/http-redirect-rewrite/httproute-redirect-prefix.yaml b/examples/experimental/http-redirect-rewrite/httproute-redirect-prefix.yaml deleted file mode 100644 index 5c04bca224..0000000000 --- a/examples/experimental/http-redirect-rewrite/httproute-redirect-prefix.yaml +++ /dev/null @@ -1,25 +0,0 @@ -#$ Used in: -#$ - site-src/api-types/httproute.md -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: http-filter-redirect -spec: - hostnames: - - redirect.example - rules: - - matches: - - path: - type: PathPrefix - value: /cayenne - filters: - - type: RequestRedirect - requestRedirect: - path: - type: ReplacePrefixMatch - replacePrefixMatch: /paprika - statusCode: 302 - backendRefs: - - name: example-svc - weight: 1 - port: 80 diff --git a/examples/experimental/http-redirect-rewrite/httproute-rewrite-path.yaml b/examples/experimental/http-redirect-rewrite/httproute-rewrite-path.yaml deleted file mode 100644 index e5aaafc65d..0000000000 --- a/examples/experimental/http-redirect-rewrite/httproute-rewrite-path.yaml +++ /dev/null @@ -1,25 +0,0 @@ -#$ Used in: -#$ - site-src/api-types/httproute.md -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: http-filter-rewrite -spec: - hostnames: - - rewrite.example - rules: - - matches: - - path: - type: PathPrefix - value: /cardamom - filters: - - type: URLRewrite - urlRewrite: - hostname: elsewhere.example - path: - type: ReplaceFullPath - replaceFullPath: /fennel - backendRefs: - - name: example-svc - weight: 1 - port: 80 diff --git a/examples/experimental/http-redirect-rewrite/httproute-rewritepath.yaml b/examples/experimental/http-redirect-rewrite/httproute-rewritepath.yaml deleted file mode 100644 index bf1d50c622..0000000000 --- a/examples/experimental/http-redirect-rewrite/httproute-rewritepath.yaml +++ /dev/null @@ -1,21 +0,0 @@ -#$ Used in: -#$ - site-src/api-types/httproute.md -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: http-filter-rewrite -spec: - hostnames: - - rewrite.example - rules: - - filters: - - type: URLRewrite - urlRewrite: - hostname: elsewhere.example - path: - type: ReplacePrefixMatch - replacePrefixMatch: /fennel - backendRefs: - - name: example-svc - weight: 1 - port: 80 diff --git a/examples/experimental/http-response-header.yaml b/examples/experimental/http-response-header.yaml new file mode 100644 index 0000000000..a3693f9f1f --- /dev/null +++ b/examples/experimental/http-response-header.yaml @@ -0,0 +1,25 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: response-header-modifier +spec: + parentRefs: + - name: acme-gw + rules: + - matches: + - path: + type: PathPrefix + value: /add-multiple-response-headers + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + add: + - name: X-Header-Add-1 + value: header-add-1 + - name: X-Header-Add-2 + value: header-add-2 + - name: X-Header-Add-3 + value: header-add-3 + backendRefs: + - name: echo + port: 8080 diff --git a/examples/standard/default-match-http.yaml b/examples/standard/default-match-http.yaml index 88f27f750d..0e70dc20ad 100644 --- a/examples/standard/default-match-http.yaml +++ b/examples/standard/default-match-http.yaml @@ -30,7 +30,7 @@ spec: parentRefs: - name: default-match-gw hostnames: - - default-match.com + - default-match.com rules: - matches: - headers: diff --git a/examples/standard/gateway-addresses.yaml b/examples/standard/gateway-addresses.yaml new file mode 100644 index 0000000000..bb543a422c --- /dev/null +++ b/examples/standard/gateway-addresses.yaml @@ -0,0 +1,27 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: gateway-addresses +spec: + gatewayClassName: acme-lb + addresses: + - value: 1200:0000:AB00:1234:0000:2552:7777:1313 + - value: 21DA:D3:0:2F3B:2AA:FF:FE28:9C5A + - value: "2001:db8:3c4d:15:0:d234:3eee::" + - value: "1234::" + - value: "1.1.1.1" + - value: "1.2.3.4" + - value: "0.0.0.0" + - value: "9.255.255.255" + - value: "11.0.0.0" + - type: IPAddress + value: "255.255.255.255" + - type: "Hostname" + value: "example.com" + listeners: + - protocol: HTTP + port: 80 + name: prod-web-gw + allowedRoutes: + namespaces: + from: Same diff --git a/examples/standard/http-filter.yaml b/examples/standard/http-filter.yaml index 042e9b488e..293817931d 100644 --- a/examples/standard/http-filter.yaml +++ b/examples/standard/http-filter.yaml @@ -9,12 +9,12 @@ spec: - my.filter.com rules: - filters: - - type: RequestHeaderModifier - requestHeaderModifier: - add: - - name: my-header - value: foo + - type: RequestHeaderModifier + requestHeaderModifier: + add: + - name: my-header + value: foo backendRefs: - - name: my-filter-svc1 - weight: 1 - port: 80 + - name: my-filter-svc1 + weight: 1 + port: 80 diff --git a/examples/experimental/http-redirect.yaml b/examples/standard/http-redirect-path.yaml similarity index 55% rename from examples/experimental/http-redirect.yaml rename to examples/standard/http-redirect-path.yaml index a97d15f397..2a0a751ca2 100644 --- a/examples/experimental/http-redirect.yaml +++ b/examples/standard/http-redirect-path.yaml @@ -8,11 +8,11 @@ spec: - name: my-filter-gateway sectionName: http hostnames: - - my-filter.example.com + - my-filter.example.com rules: - - filters: - - type: RequestRedirect - requestRedirect: - path: - type: ReplaceFullPath - replaceFullPath: /foo + - filters: + - type: RequestRedirect + requestRedirect: + path: + type: ReplaceFullPath + replaceFullPath: /foo diff --git a/examples/standard/http-redirect-rewrite/httproute-redirect-full.yaml b/examples/standard/http-redirect-rewrite/httproute-redirect-full.yaml new file mode 100644 index 0000000000..51a1e0d374 --- /dev/null +++ b/examples/standard/http-redirect-rewrite/httproute-redirect-full.yaml @@ -0,0 +1,21 @@ +#$ Used in: +#$ - site-src/api-types/httproute.md +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-filter-redirect +spec: + hostnames: + - redirect.example + rules: + - matches: + - path: + type: PathPrefix + value: /cayenne + filters: + - type: RequestRedirect + requestRedirect: + path: + type: ReplaceFullPath + replaceFullPath: /paprika + statusCode: 302 diff --git a/examples/experimental/http-redirect-rewrite/httproute-redirect-https.yaml b/examples/standard/http-redirect-rewrite/httproute-redirect-https.yaml similarity index 53% rename from examples/experimental/http-redirect-rewrite/httproute-redirect-https.yaml rename to examples/standard/http-redirect-rewrite/httproute-redirect-https.yaml index 3729f67d47..c876184575 100644 --- a/examples/experimental/http-redirect-rewrite/httproute-redirect-https.yaml +++ b/examples/standard/http-redirect-rewrite/httproute-redirect-https.yaml @@ -9,11 +9,7 @@ spec: - redirect.example rules: - filters: - - type: RequestRedirect - requestRedirect: - scheme: https - statusCode: 301 - backendRefs: - - name: example-svc - weight: 1 - port: 80 + - type: RequestRedirect + requestRedirect: + scheme: https + statusCode: 301 diff --git a/examples/standard/http-redirect-rewrite/httproute-redirect-prefix.yaml b/examples/standard/http-redirect-rewrite/httproute-redirect-prefix.yaml new file mode 100644 index 0000000000..64d71f72a2 --- /dev/null +++ b/examples/standard/http-redirect-rewrite/httproute-redirect-prefix.yaml @@ -0,0 +1,21 @@ +#$ Used in: +#$ - site-src/api-types/httproute.md +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-filter-redirect +spec: + hostnames: + - redirect.example + rules: + - matches: + - path: + type: PathPrefix + value: /cayenne + filters: + - type: RequestRedirect + requestRedirect: + path: + type: ReplacePrefixMatch + replacePrefixMatch: /paprika + statusCode: 302 diff --git a/examples/standard/http-redirect-rewrite/httproute-rewrite-path.yaml b/examples/standard/http-redirect-rewrite/httproute-rewrite-path.yaml new file mode 100644 index 0000000000..4d019f1604 --- /dev/null +++ b/examples/standard/http-redirect-rewrite/httproute-rewrite-path.yaml @@ -0,0 +1,25 @@ +#$ Used in: +#$ - site-src/api-types/httproute.md +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-filter-rewrite +spec: + hostnames: + - rewrite.example + rules: + - matches: + - path: + type: PathPrefix + value: /cardamom + filters: + - type: URLRewrite + urlRewrite: + hostname: elsewhere.example + path: + type: ReplaceFullPath + replaceFullPath: /fennel + backendRefs: + - name: example-svc + weight: 1 + port: 80 diff --git a/examples/experimental/http-redirect-rewrite/httproute-rewrite.yaml b/examples/standard/http-redirect-rewrite/httproute-rewrite.yaml similarity index 59% rename from examples/experimental/http-redirect-rewrite/httproute-rewrite.yaml rename to examples/standard/http-redirect-rewrite/httproute-rewrite.yaml index 96cc5a4a85..b5652109df 100644 --- a/examples/experimental/http-redirect-rewrite/httproute-rewrite.yaml +++ b/examples/standard/http-redirect-rewrite/httproute-rewrite.yaml @@ -9,10 +9,10 @@ spec: - rewrite.example rules: - filters: - - type: URLRewrite - urlRewrite: - hostname: elsewhere.example + - type: URLRewrite + urlRewrite: + hostname: elsewhere.example backendRefs: - - name: example-svc - weight: 1 - port: 80 + - name: example-svc + weight: 1 + port: 80 diff --git a/examples/standard/http-redirect-rewrite/httproute-rewritepath.yaml b/examples/standard/http-redirect-rewrite/httproute-rewritepath.yaml new file mode 100644 index 0000000000..30299818bb --- /dev/null +++ b/examples/standard/http-redirect-rewrite/httproute-rewritepath.yaml @@ -0,0 +1,21 @@ +#$ Used in: +#$ - site-src/api-types/httproute.md +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-filter-rewrite +spec: + hostnames: + - rewrite.example + rules: + - filters: + - type: URLRewrite + urlRewrite: + hostname: elsewhere.example + path: + type: ReplacePrefixMatch + replacePrefixMatch: /fennel + backendRefs: + - name: example-svc + weight: 1 + port: 80 diff --git a/examples/standard/http-redirect.yaml b/examples/standard/http-redirect.yaml index 7e34c45273..e4f2bbbc4c 100644 --- a/examples/standard/http-redirect.yaml +++ b/examples/standard/http-redirect.yaml @@ -30,9 +30,9 @@ spec: port: 443 tls: certificateRefs: - - kind: Secret - group: "" - name: example-com-cert + - kind: Secret + group: "" + name: example-com-cert --- apiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute @@ -41,15 +41,15 @@ metadata: namespace: gateway-api-example-ns1 spec: parentRefs: - - name: my-filter-gateway - sectionName: http + - name: my-filter-gateway + sectionName: http hostnames: - my-filter.example.com rules: - filters: - - type: RequestRedirect - requestRedirect: - scheme: https + - type: RequestRedirect + requestRedirect: + scheme: https --- apiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute @@ -58,16 +58,16 @@ metadata: namespace: gateway-api-example-ns1 spec: parentRefs: - - name: my-filter-gateway - sectionName: https + - name: my-filter-gateway + sectionName: https hostnames: - my-filter.example.com rules: - matches: - - path: - type: PathPrefix - value: / + - path: + type: PathPrefix + value: / backendRefs: - - name: my-filter-svc1 - weight: 1 - port: 80 + - name: my-filter-svc1 + weight: 1 + port: 80 diff --git a/examples/standard/http-request-header-add.yaml b/examples/standard/http-request-header-add.yaml new file mode 100644 index 0000000000..b8ba4066f0 --- /dev/null +++ b/examples/standard/http-request-header-add.yaml @@ -0,0 +1,21 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: header-http-echo +spec: + parentRefs: + - name: acme-gw + rules: + - matches: + - path: + type: PathPrefix + value: /add-a-request-header + filters: + - type: RequestHeaderModifier + requestHeaderModifier: + add: + - name: my-header-name + value: my-header-value + backendRefs: + - name: echo + port: 8080 diff --git a/examples/standard/http-request-header-remove.yaml b/examples/standard/http-request-header-remove.yaml new file mode 100644 index 0000000000..1cc29dbe52 --- /dev/null +++ b/examples/standard/http-request-header-remove.yaml @@ -0,0 +1,20 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: header-http-echo +spec: + parentRefs: + - name: acme-gw + rules: + - matches: + - path: + type: PathPrefix + value: /remove-a-request-header + filters: + - type: RequestHeaderModifier + requestHeaderModifier: + remove: + - x-request-id + backendRefs: + - name: echo + port: 8080 diff --git a/examples/standard/http-request-header-set.yaml b/examples/standard/http-request-header-set.yaml new file mode 100644 index 0000000000..45a69d68af --- /dev/null +++ b/examples/standard/http-request-header-set.yaml @@ -0,0 +1,21 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: header-http-echo +spec: + parentRefs: + - name: acme-gw + rules: + - matches: + - path: + type: PathPrefix + value: /edit-a-request-header + filters: + - type: RequestHeaderModifier + requestHeaderModifier: + set: + - name: my-header-name + value: my-new-header-value + backendRefs: + - name: echo + port: 8080 diff --git a/examples/experimental/http-rewrite.yaml b/examples/standard/http-rewrite.yaml similarity index 56% rename from examples/experimental/http-rewrite.yaml rename to examples/standard/http-rewrite.yaml index eb580f9a41..272b97be18 100644 --- a/examples/experimental/http-rewrite.yaml +++ b/examples/standard/http-rewrite.yaml @@ -8,11 +8,11 @@ spec: - name: my-filter-gateway sectionName: http hostnames: - - my-filter.example.com + - my-filter.example.com rules: - - filters: - - type: URLRewrite - urlRewrite: - path: - type: ReplaceFullPath - replaceFullPath: /foo + - filters: + - type: URLRewrite + urlRewrite: + path: + type: ReplaceFullPath + replaceFullPath: /foo diff --git a/examples/standard/http-route-attachment/gateway-namespaces.yaml b/examples/standard/http-route-attachment/gateway-namespaces.yaml index 9e23ac65fc..e751adbba7 100644 --- a/examples/standard/http-route-attachment/gateway-namespaces.yaml +++ b/examples/standard/http-route-attachment/gateway-namespaces.yaml @@ -10,10 +10,10 @@ spec: listeners: - name: prod-web port: 80 - protocol: HTTP + protocol: HTTP allowedRoutes: - kinds: - - kind: HTTPRoute + kinds: + - kind: HTTPRoute namespaces: from: Selector selector: diff --git a/examples/standard/http-route-attachment/gateway-strict.yaml b/examples/standard/http-route-attachment/gateway-strict.yaml index 7d5f55b0b0..fc3bf9f1b6 100644 --- a/examples/standard/http-route-attachment/gateway-strict.yaml +++ b/examples/standard/http-route-attachment/gateway-strict.yaml @@ -12,8 +12,8 @@ spec: port: 80 protocol: HTTP allowedRoutes: - kinds: - - kind: HTTPRoute + kinds: + - kind: HTTPRoute namespaces: from: Selector selector: diff --git a/examples/standard/httproute.yaml b/examples/standard/httproute.yaml new file mode 100644 index 0000000000..bd78dc0a4c --- /dev/null +++ b/examples/standard/httproute.yaml @@ -0,0 +1,28 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: my-app +spec: + rules: + - matches: + - path: + type: PathPrefix + value: /mypath + backendRefs: + - name: my-service-1 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /mypath-012 + backendRefs: + - name: my-service-2 + port: 8080 + - matches: + - path: + type: PathPrefix + value: /my%20path/123 + backendRefs: + - name: my-service-3 + port: 8080 + diff --git a/examples/standard/multicluster/0-namespaces.yaml b/examples/standard/multicluster/0-namespaces.yaml new file mode 100644 index 0000000000..13be3bc235 --- /dev/null +++ b/examples/standard/multicluster/0-namespaces.yaml @@ -0,0 +1,9 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: foo +--- +apiVersion: v1 +kind: Namespace +metadata: + name: bar diff --git a/examples/standard/multicluster/httproute-gamma.yaml b/examples/standard/multicluster/httproute-gamma.yaml new file mode 100644 index 0000000000..aba076df18 --- /dev/null +++ b/examples/standard/multicluster/httproute-gamma.yaml @@ -0,0 +1,19 @@ +#$ Used in: +#$ - geps/gep-1748.md +kind: HTTPRoute +apiVersion: gateway.networking.k8s.io/v1beta1 +metadata: + name: store +spec: + parentRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: store + rules: + - matches: + - path: + value: "/cart" + backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: cart diff --git a/examples/standard/multicluster/httproute-hybrid.yaml b/examples/standard/multicluster/httproute-hybrid.yaml new file mode 100644 index 0000000000..fe1c468ed1 --- /dev/null +++ b/examples/standard/multicluster/httproute-hybrid.yaml @@ -0,0 +1,20 @@ +#$ Used in: +#$ - geps/gep-1748.md +kind: HTTPRoute +apiVersion: gateway.networking.k8s.io/v1beta1 +metadata: + name: store +spec: + parentRefs: + - name: external-http + rules: + - backendRefs: + - kind: Service + name: store + port: 8080 + weight: 90 + - group: multicluster.x-k8s.io + kind: ServiceImport + name: store-global + port: 8080 + weight: 10 diff --git a/examples/standard/multicluster/httproute-location.yaml b/examples/standard/multicluster/httproute-location.yaml new file mode 100644 index 0000000000..443afb1ebc --- /dev/null +++ b/examples/standard/multicluster/httproute-location.yaml @@ -0,0 +1,33 @@ +#$ Used in: +#$ - geps/gep-1748.md +kind: HTTPRoute +apiVersion: gateway.networking.k8s.io/v1beta1 +metadata: + name: store +spec: + parentRefs: + - name: external-http + rules: + - matches: + - path: + type: PathPrefix + value: /west + backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: store-west + port: 8080 + - matches: + - path: + type: PathPrefix + value: /east + backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: store-east + port: 8080 + - backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: store + port: 8080 diff --git a/examples/standard/multicluster/httproute-method.yaml b/examples/standard/multicluster/httproute-method.yaml new file mode 100644 index 0000000000..e4075dc55c --- /dev/null +++ b/examples/standard/multicluster/httproute-method.yaml @@ -0,0 +1,24 @@ +#$ Used in: +#$ - geps/gep-1748.md +kind: HTTPRoute +apiVersion: gateway.networking.k8s.io/v1beta1 +metadata: + name: api +spec: + parentRefs: + - name: api-gw + rules: + - matches: + - method: POST + - method: PUT + - method: DELETE + backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: api-primary + port: 8080 + - backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: api-replicas + port: 8080 diff --git a/examples/standard/multicluster/httproute-referencegrant.yaml b/examples/standard/multicluster/httproute-referencegrant.yaml new file mode 100644 index 0000000000..5850778c51 --- /dev/null +++ b/examples/standard/multicluster/httproute-referencegrant.yaml @@ -0,0 +1,32 @@ +#$ Used in: +#$ - geps/gep-1748.md +kind: HTTPRoute +apiVersion: gateway.networking.k8s.io/v1beta1 +metadata: + name: foo + namespace: foo +spec: + rules: + - matches: + - path: + type: PathPrefix + value: /bar + backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: bar + namespace: bar +--- +kind: ReferenceGrant +apiVersion: gateway.networking.k8s.io/v1beta1 +metadata: + name: bar + namespace: bar +spec: + from: + - group: gateway.networking.k8s.io + kind: HTTPRoute + namespace: foo + to: + - group: multicluster.x-k8s.io + kind: ServiceImport diff --git a/examples/standard/multicluster/httproute-simple.yaml b/examples/standard/multicluster/httproute-simple.yaml new file mode 100644 index 0000000000..9e115dabc8 --- /dev/null +++ b/examples/standard/multicluster/httproute-simple.yaml @@ -0,0 +1,15 @@ +#$ Used in: +#$ - geps/gep-1748.md +kind: HTTPRoute +apiVersion: gateway.networking.k8s.io/v1beta1 +metadata: + name: store +spec: + parentRefs: + - name: external-http + rules: + - backendRefs: + - group: multicluster.x-k8s.io + kind: ServiceImport + name: store + port: 8080 diff --git a/examples/standard/simple-gateway/gateway.yaml b/examples/standard/simple-gateway/gateway.yaml index 19924bdad0..9b19a20116 100644 --- a/examples/standard/simple-gateway/gateway.yaml +++ b/examples/standard/simple-gateway/gateway.yaml @@ -7,7 +7,7 @@ metadata: name: prod-web spec: gatewayClassName: acme-lb - listeners: + listeners: - protocol: HTTP port: 80 name: prod-web-gw diff --git a/site-src/geps/OWNERS b/geps/OWNERS similarity index 100% rename from site-src/geps/OWNERS rename to geps/OWNERS diff --git a/site-src/geps/gep-1016.md b/geps/gep-1016.md similarity index 99% rename from site-src/geps/gep-1016.md rename to geps/gep-1016.md index 656132a7a4..2031a26087 100644 --- a/site-src/geps/gep-1016.md +++ b/geps/gep-1016.md @@ -3,6 +3,12 @@ * Issue: [#1016](https://github.com/kubernetes-sigs/gateway-api/issues/1016) * Status: Experimental +> **Note**: This GEP is exempt from the [Probationary Period][expprob] rules of +> our GEP overview as it existed before those rules did, and so it has been +> explicitly grandfathered in. + +[expprob]:https://gateway-api.sigs.k8s.io/geps/overview/#probationary-period + ## Goal Add an idiomatic GRPCRoute for routing gRPC traffic. diff --git a/site-src/geps/gep-1282.md b/geps/gep-1282.md similarity index 96% rename from site-src/geps/gep-1282.md rename to geps/gep-1282.md index 1c7dd49d36..c8a54f74a1 100644 --- a/site-src/geps/gep-1282.md +++ b/geps/gep-1282.md @@ -1,7 +1,14 @@ # GEP-1282: Describing Backend Properties * Issue: [#1282](https://github.com/kubernetes-sigs/gateway-api/issues/1282) -* Status: Provisional +* Status: Declined + + +## Explanation for Declined status + +This GEP is declined because, after spending a lot of time discussing it, we felt that it was too large, and had too many crosscutting concerns. + +It's superseded for TLS Backend Properties by [GEP-1897](https://github.com/kubernetes-sigs/gateway-api/issues/1897). ## TLDR diff --git a/site-src/geps/gep-1323.md b/geps/gep-1323.md similarity index 96% rename from site-src/geps/gep-1323.md rename to geps/gep-1323.md index 12f0a1871a..3baa6cee8e 100644 --- a/site-src/geps/gep-1323.md +++ b/geps/gep-1323.md @@ -1,7 +1,13 @@ # GEP 1323: Response Header Filter * Issue: [#1323](https://github.com/kubernetes-sigs/gateway-api/issues/1323) -* Status: Experimental +* Status: Standard + +> **Note**: This GEP is exempt from the [Probationary Period][expprob] rules of +> our GEP overview as it existed before those rules did, and so it has been +> explicitly grandfathered in. + +[expprob]:https://gateway-api.sigs.k8s.io/geps/overview/#probationary-period ## TLDR Similar to how we have `RequestHeaderModifier` in `HTTPRouteFilter`, which lets users modify request headers before the request is forwarded to a backend (or a group of backends), it’d be helpful to have a `ResponseHeaderModifier` field which would let users modify response headers before they are returned to the client. diff --git a/site-src/geps/gep-1324.md b/geps/gep-1324.md similarity index 98% rename from site-src/geps/gep-1324.md rename to geps/gep-1324.md index a4b6ca0d1a..fac3f90af1 100644 --- a/site-src/geps/gep-1324.md +++ b/geps/gep-1324.md @@ -50,10 +50,10 @@ These use-cases are presented as an aid for discussion, and as frames of referen 1. I want to enforce that all traffic within my cluster is encrypted. 2. I want to have strict isolation and control at namespace level, so a bug/malicious user can't impact other namespaces 3. I want to be able to allow app owners to gradually opt-in to a mesh (no mesh, L4 only, L7 enabled) so they can choose the right fit for their applications’ performance and compatibility goals. - 4. Since mesh can be multi-tenant and hosting multiple services (e.g. foo or bar), as a mesh administrator I need to make sure a client can discover different services. Here are few possible ways - 1. Each service is allocated a unique IP and port - 2. Or Each service must use unique Host name - 3. Or a unique port and protocol, (80:http, 443 tls) + 4. Since mesh can be multi-tenant and hosting multiple services (e.g. foo or bar), as a mesh administrator I need to make sure a client can discover different services. Here are a few possible ways: + 1. Each service is allocated a unique IP and port + 2. Or Each service must use a unique hostname + 3. Or a unique port and protocol (80:http, 443:tls) ## Glossary diff --git a/site-src/geps/gep-1364.md b/geps/gep-1364.md similarity index 100% rename from site-src/geps/gep-1364.md rename to geps/gep-1364.md diff --git a/site-src/geps/gep-1426.md b/geps/gep-1426.md similarity index 80% rename from site-src/geps/gep-1426.md rename to geps/gep-1426.md index 27b5f1a878..79678364b4 100644 --- a/site-src/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -1,12 +1,12 @@ # GEP-1426: xRoutes Mesh Binding * Issue: [#1294](https://github.com/kubernetes-sigs/gateway-api/issues/1294) -* Status: Provisional +* Status: Implementable ## Overview Similar to how `xRoutes` bind to `Gateways` and manage North/South traffic flows in Gateway API’s ingress use-case, it would be natural to adopt a similar model for traffic routing concerns in service mesh deployments. The purpose of this GEP is to add a mechanism to the Gateway API spec for the purpose of associating the various `xRoute` types to a service mesh and offering a model for service owners to manage traffic splitting configurations. - + This GEP is intended to establish an implementable, but experimental, baseline for supporting basic service mesh traffic routing functionality through the Gateway API spec. ## Personas @@ -43,11 +43,15 @@ This GEP uses the [roles and personas](https://gateway-api.sigs.k8s.io/concepts/ It is proposed that an application owner should configure traffic rules for a mesh service by configuring an `xRoute` with a Kubernetes `Service` resource as a `parentRef`. -This approach is dependent on both the "frontend" role of the Kubernetes `Service` resource as defined in [GEP-1324: Service Mesh in Gateway API](https://gateway-api.sigs.k8s.io/geps/gep-1324/#service) when used as a `parentRef` and the "backend" role of `Service` when used as a `backendRef`. It would use the Kubernetes service name to match traffic for meshes implementing "transparent proxy" functionality, but the `backendRef` endpoints would ultimately be used for the canonical IP address(es) to which traffic should be redirected by rules defined in this `xRoute`. This approach leverages the existing points of extensibility within the Gateway API spec, and would not require introducing any API changes or new resources, only defining expected behavior. +This approach is dependent on both the "frontend" role of the Kubernetes `Service` resource as defined in [GEP-1324: Service Mesh in Gateway API](https://gateway-api.sigs.k8s.io/geps/gep-1324/#service) when used as a `parentRef` and the "backend" role of `Service` when used as a `backendRef`. The conformant implementation would use the Kubernetes `Service` name to match traffic for meshes, but the `backendRef` endpoints would ultimately be used for the canonical IP address(es) to which traffic should be redirected by rules defined in this `xRoute`. This approach leverages the existing points of extensibility within the Gateway API spec, and would not require introducing any API changes or new resources, only defining expected behavior. + +### Why Service? + +The GAMMA initiative has been working to bring service mesh use-cases to the Gateway API spec, taking the best practices and learnings from mesh implementations and codifying them in a spec. Most mesh users are familiar with using the Kubernetes `Service` resource as the foundation for traffic routing. Generaly, this architecture makes perfect sense; unfortunately, `Service` is far too coupled of a resource. It orchestrates IP address allocation, DNS, endpoint collection and propagation, load balancing, etc. For this reason, it **cannot** be the right long-term answer for `parentRef` binding; however, it is the only feasible option that Kubernetes has for mesh implementations today. We expect this to change (indeed, we hope to be a part of that change), but in the interest of developing a spec now, we must once again lean on the `Service` resource. However, we will provide provisions to support additional resources as a `parentRef`. ## API -``` +```yaml metadata: name: foo-route namespace: store @@ -67,29 +71,9 @@ spec: In the example above, routing rules have been configured to direct 90% of traffic for the `foo` `Service` to the default "backend" endpoints specified by the `foo` `Service` [`selector`](https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service) field, and 10% to the `foo-v2` `Service`. This is determined based on the `ClusterIP` (for `Service`) and `ClusterSetIP` (for `ServiceImport`) matching, and for "transparent proxy" mesh implementations would match all requests to `foo.svc.cluster.local` (or arbitrary custom suffix, as the hostname is not specified manually) from within the same namespace, all requests to `foo.store.svc.cluster.local` from other namespaces, and all requests to `foo.store.svc.clusterset.local` for multicluster services, within the scope of the service mesh. -### Omitting `backendRefs` - -Implementations SHOULD support a terse syntax which allows omitting `backendRefs` to avoid unnecessary redundancy for simple configurations. If no `backendRefs` are specified, implementations should direct traffic to the default endpoints of the `parentRef` service. The behavior should be exactly the same as if the `parentRef` service was explicitly defined as the only `backendRef`, so if the `parentRef` does not have any endpoints, implementations MUST return an HTTP 503 response code (see discussion in [#1210](https://github.com/kubernetes-sigs/gateway-api/pull/1210)). +### Route presence -This syntax has limited utility currently, but could become more relevant if additional functionality beyond traffic splitting is added to the `xRoute` resource. `Routes` configured in this way MAY be used to manually enroll services into a mesh (which could trigger behavior like injecting a sidecar proxy) if they have not already been enrolled by some other mechanism (as proposed in [GEP-1291: Mesh Representation](https://docs.google.com/document/d/1oyA9uUH7pNNxxwy3WZGSWx-edHDBLrujcezr8q3el70/edit) or similar). - -``` -metadata: - name: foo-route - namespace: store -spec: - parentRefs: - - kind: Service - name: foo - rules: - matches: - - path: - value: "/bar" -``` - -The example above would drop all incoming traffic for HTTP paths other than `/bar` to the `foo` `Service`, following the existing spec to return HTTP 404 response codes for unmatched requests, and HTTP 500 response codes for requests excluded due to an [`HTTPRouteFilter`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPRouteFilter). - -When no `HTTPRoute` resources or AuthZ configuration are defined, all traffic should implicitly work - this is just how Kubernetes functions. When you create an `HTTPRoute` targeting a service as a `parentRef` you are replacing that implicit logic - not adding to it. Therefore, you may be reshaping or restricting traffic via an `HTTPRoute` configuration (which should be noted is distinct from disallowing traffic by AuthZ). +When no `xRoute` resources are defined, all traffic should implicitly work - this is just how Kubernetes functions. When you create an `xRoute` targeting a service as a `parentRef` you are replacing that implicit logic - not adding to it. Therefore, you may be reshaping or restricting traffic via an `xRoute` configuration (which should be noted, is distinct from *disallowing* traffic by AuthZ). ### Allowed service types @@ -109,11 +93,60 @@ Services supported as `backendRefs` SHOULD be consistent with expectations for N An alternate pattern additionally supported by this approach would be to target a `Service` without selectors as the `parentRef`. This could be a clean way to create a pure routing construct and abstract a logical frontend, as traffic would resolve to a `backendRef` `Service` with selectors defined on the `HTTPRoute`, or receive a 4xx/5xx error response if no matching path or valid backend was found. -#### Multicluster support with `ServiceImport` +### `parentRef` Conformance Levels + +Currently (v0.7.0), this spec only considers the `Service` resource to be under Core conformance as a `parentRef`. However, Service is not the only resource that can fulfill the frontend role. While the Gateway API spec couldn’t possibly enumerate every existing (and future) frontend-like resource, it can specify a subset of resources that implementations MUST support as parentRefs under as a part of core conformance. Meshes MAY support other implementation-specific resources as parentRefs. The spec maintainers also reserve the right to add additional resources to core conformance as the spec evolves. -`ServiceImport` resources allocate a virtual IP in the cluster, so MAY be allowed as a `parentRef`. +#### Extended Conformance -`ServiceImport` would remain a valid backend option for `xRoute` resources (but _not_ currently a requirement for core conformance), and could be specified alongside a `Service` `backendRef` to split traffic across clusters within a `ClusterSet` (as defined in the [Multi-cluster Service (MCS) APIs project](https://github.com/kubernetes-sigs/mcs-api)). This could be a way to solve the need described in [A use case for using Gateway APIs in Multi-Cluster](https://docs.google.com/document/d/1bjr0uAVMmEtTX4mpU_aGXXapQFsEz_Qt0OnDoVswEEI/edit). +In addition to Service, there are other optional parentRef resources that, if used by implementations, MUST adhere to the spec’s prescriptions. At the time of writing (v0.7.0), there is one resource in extended conformance: `ServiceImport` (part of the [MCS API](https://github.com/kubernetes-sigs/mcs-api), currently in alpha). The semantics of `ServiceImport` `parentRef` binding can be found in [GEP-1748](https://gateway-api.sigs.k8s.io/geps/gep-1748/) (Note: Headless `ServiceImport` is out of scope and not currently a part of the spec). + +##### Why not `IPAddress` + +In Kubernetes 1.27, there will be a new IPAddress resource added to networking.k8s.io/v1alpha1 as part of [KEP 1880](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1880-multiple-service-cidrs#proposal). Naturally, it makes sense to examine whether or not this new resource makes sense as a GAMMA aware parentRef. At first glance, IPAddress seems to be an appropriate abstraction for the “frontend” role we’ve been discussing; every Kubernetes Service is accessed over the network via one of its ip addresses. Furthermore, the fact that the Service resource auto-creates an IPAddress is encouraging. However, the fact that the name of the IPAddress is simply the decimal/hex ip address and not a human-readable Service name makes the UX untenable as a spec-supported parentRef. However, `IPAddress` is NOT disallowed; implementations may use it if they wish. + +#### Implementation-specific `parentRef`s + +If mesh implementations wish to enable an implementation-specific resource as a parentRef, they may do so as long as that resource meets certain conditions. Recall that the frontend role of a (generic) service is how one calls the service. In the service mesh transparent proxy context, the frontend role (and parentRef by extension) is effectively the matching mechanism for the specified route. For the Service parentRef, this means that the mesh should apply a particular xRoute’s configuration if the destination ip address for a given connection is the ClusterIP of that parentRef Service. If a mesh wishes to use an implementation-specific resource for parentRef, that resource MUST contain layer-appropriate information suitable for traffic matching (e.g. no Host header capture in TCPRoute). For example, the following HTTPRoute with an Istio `ServiceEntry` as a parentRef would be a valid implementation-specific reference: + +```yaml +apiVersion: networking.istio.io/v1beta1 +kind: ServiceEntry +metadata: + name: internal-svc-httpbin + namespace : egress +spec: + hosts: + - example.com + exportTo: + - "." + location: MESH_INTERNAL + ports: + - number: 80 + name: http + protocol: HTTP + resolution: DNS +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mongo-internal +spec: + parentRefs: + - kind: ServiceEntry + group: networking.istio.io/v1beta1 + name: internal-svc-httpbin + namespace: egress + sectionName: http # referencing the port name + rules: + - backendRefs: + - name: internal-example + port: 80 +``` + +##### `Gateways` + +There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef if an implementation wishes to do so. ### Route types @@ -427,3 +460,10 @@ For convenience, `ServiceProjection` could have a `meshRef` field that, when set * May require reconfiguring `Gateway` `HTTPRoutes` to specify `ServiceProjections` as `backendRefs`. * Verbose boilerplate for each service. + +### Implicit backendRef + +An initial iteration of this GEP had the ability to omit a `backendRef` and have it implicitly be set the same as the `parentRef`. +This has been removed due to inconsistency with Gateway `parentRefs` and tight coupling of the "frontend" and "backend" roles. + +Implementations MUST respect the standard `backendRef` rules as defined by the existing spec. diff --git a/geps/gep-1619.md b/geps/gep-1619.md new file mode 100644 index 0000000000..6190ffd2dc --- /dev/null +++ b/geps/gep-1619.md @@ -0,0 +1,242 @@ +# GEP-1619: Session Persistence and Session Affinity + +* Issue: [#1619](https://github.com/kubernetes-sigs/gateway-api/issues/1619) +* Status: Provisional + +(See definitions in [GEP Status][/contributing/gep#status].) + +## TLDR + +This GEP proposes a definition of session persistence and session affinity as the foundation of future session persistence related improvements. This GEP also outlines the ways session persistence could be achieved by implementations. + +## Goals +- Define session persistence and session affinity to establish a common language +- Identify differences in session persistence functionality between implementations + +## Non-Goals +- Mandate a default or expected session persistence functionality for implementations + +## Introduction + +### Defining Session Persistence + +Session persistence is when a client request is directed to the same backend server for the duration of a "session". It is achieved when a client directly provides information, such as a header, that a proxy uses as a reference to direct traffic to a specific server. Persistence is an exception to load balancing: a persistent client request bypasses the proxy's load balancing algorithm, going directly to a backend server it has previously established a session with. + +Session persistence enables more efficient application workflows: +1. Better performance: Maintaining a single session allows a server to cache information about a client locally reducing the need for servers to exchange session data and overall storage needs. +2. Seamless client experience: Clients can reconnect to the same server without re-authenticating or re-entering their information. + +Some of the concerns of session persistence are the duration and expiration of the session, security of the transaction stream, and storage of the context or state. + +Session affinity, not to be confused with session persistence, uses an existing attribute of the request to consistently send to the same backend. Session affinity can be considered a weaker form of session persistence: it is not guaranteed to persist a connection to the same backend server if certain attributes of the request or the backends are changed. + +### Security and Privacy Implications + +Session persistence can introduce security and privacy vulnerabilities if not properly implemented. These vulnerabilities can include: + +1. Session hijacking: Attackers intercepting or predicting a valid session token to gain unauthorized access. +2. Session fixation: Attackers setting a client's session ID to a known value, which they can then use to hijack the session. +3. Session replay attacks: Attackers capturing and resending a client's message with a valid session ID. +4. Data leakage: Attackers can exploit sensitive session information cached on servers if not properly secured. +5. Denial of service attacks: Attackers can use up server resources by creating and maintaining large numbers of sessions. + +To mitigate these security concerns, it is important to implement session persistence using secure practices, such as using strong session ID generation algorithms, implementing session timeouts, encrypting sensitive data, and monitoring server resources for unusual activity. + +IP address reuse may also be a security or privacy concern when using session persistence or session affinity. If Kubernetes reuses an IP address of previously shutdown pod, the new pod may receive session persistent traffic meant for the old pod. + +Session affinity introduces fewer security and privacy vulnerabilities since there are no session tokens to protect or exploit. + +### Achieving Session Persistence + +Session persistence is achieved using attributes residing in the application layer. The following are mechanisms for achieving session persistence: + +**1. Cookie-Based Session Persistence** + +The most common mechanism is by using cookies (described by [RFC6265](https://www.rfc-editor.org/rfc/rfc6265)) with the set-cookie HTTP response header. A client will use the provided value in the set-cookie response header in a cookie request header in subsequent requests. Proxies can use this cookie header to maintain a persistent connection to a single backend server on behalf of the client. + +**2. Header-Based Session Persistence** + +Header-based stateful sessions are achieved by a backend or gateway providing an HTTP response header and the client using the same header in subsequent HTTP requests. Proxies can use this header to maintain a persistent connection to a single backend server on behalf of the client. + +**3. URL-Encoded Session Persistence** + +Session information can be also encoded into the request URL to establish a persistent session. The server rewrites the client's URL to encode the new session information automatically. The server then decodes the session information from the URL to identify the session. + +**Session Initiation** + +For both header-based and cookie-based sessions, either the gateway or the backend server can initiate establishing the session via the appropriate header or set-cookie attributes. The following rules apply based on who initiates the session: + +- If the backend initiates the session, the gateway should allow this and not force persistence connections, unless specifically configured to. The gateway may decode and alter the cookie established by the backend to achieve session persistence. +- If the gateway initiates the session, the backend will be presented with session attributes regardless if it enabled them. + +The client application also plays in a role in session initiation. If a client doesn't want a request to belong an existing session, it can remove the session cookie or identifier to signal the gateway or backend to recreate the session. + +### Achieving Session Affinity + +While session persistence uses attributes in the application layer, session affinity often uses, but is not limited to, attributes below the application layer. Session affinity doesn't require a session identifier like session persistence (e.g. a cookie), but instead uses existing connection attributes to establish a consistent hashing load balancing algorithm. + +Session affinity can be achieved by deterministic load balancing algorithms or a proxy feature that tracks IP-to-backend associations such as [HAProxy's stick tables](https://www.haproxy.com/blog/introduction-to-haproxy-stick-tables/) or [Cilium's session affinity](https://docs.cilium.io/en/v1.12/gettingstarted/kubeproxy-free/#id2). + +### The Relationship of Session Persistence and Session Affinity + +Although session persistence and session affinity may appear to be quite similar, it is important to recognize and understand their distinctions. As mentioned above, their distinctions are defined by what information they use to establish session consistency and whether consistency is guaranteed. + +We can also examine how session persistence and session affinity functionally work together, by framing the relationship into a two tiered logical decision made by the data plane: +1. If the request contains a session persistence identity (e.g. a cookie or header), then route it directly to the backend it has previously established a session with. +2. If no session persistence identity is present, load balance as per load balancing configuration, taking into account the session affinity configuration (e.g. by utilizing a hashing algorithm that is deterministic). + +This tiered decision-based logic is consistent with the idea that session persistence is an exception to load balancing. Though there are different ways to frame this relationship, this design will influence the separation between persistence and affinity API design. + +### Implementations +In this section, we will describe how implementations achieve session persistence and affinity, along with a breakdown of related configuration options. Input from implementations is appreciated to complete this information. + +In the following tables, we will example two types of APIs: +1. Dataplane APIs +2. Implementation APIs + +Generally, the implementation API programs the dataplane API; however these two are not always clearly separated. The two types of APIs can use different API structures for configuring the same feature. Examining the dataplane APIs helps to remove the layer of API abstraction that implementations provide. Removing this layer avoids situations where implementations don’t fully implement all capabilities of a dataplane API or obfuscate certain configuration around session persistence. On the other hand, examining implementation APIs provides valuable data points in what implementations are interested in configuring. + +**Session Persistence** + +| **Technology** | **Technology Type** | **Session Persistence Type** | **Configuration Options** | **Configuration Association (Global, Gateway, Route, or Backends)** | **Notes** | +|--- |--- |--- |--- |--- |--- | +| Acnodal EPIC | Implementation (Envoy) | N/A | Supports Gateway API Only* | N/A | *Acnodal Epic solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. [Acnodal EPIC Docs](https://www.epick8sgw.io/docs/) | +| Apache APISIX | Implementation (Nginx) | [Cookie-Based](https://apisix.apache.org/docs/apisix/admin-api/#upstream) | hash_on=[vars \| header \| cookie \| consumer]
key=cookie_name | [Upstream](https://apisix.apache.org/docs/apisix/admin-api/#upstream) (Route or Backends) | N/A | +| | Implementation (Nginx) | [Header-Based](https://apisix.apache.org/docs/apisix/terminology/upstream/#header) | hash_on=[vars \| header \| cookie \| consumer]
key=header_name | [Upstream](https://apisix.apache.org/docs/apisix/admin-api/#upstream) (Route or Backends) | N/A | +| Apache httpd | Web Server | [Cookie-Based / URL-Encoded](https://httpd.apache.org/docs/2.4/mod/mod_proxy_balancer.html) | Cookie Attributes | N/A | N/A | +| Cilium | Implementation / Dataplane | None | None | None | Cilium has no documented way of doing session persistence. [Cilium Docs](https://cilium.io/) | +| Contour | Implementation (Envoy) | [Cookie-Based](https://projectcontour.io/docs/1.24/config/api/#projectcontour.io/v1.CookieRewritePolicy) | Name=name
pathRewrite=path
domainRewrite=domain
secure
sameSite | [Route](https://projectcontour.io/docs/1.24/config/api/#projectcontour.io/v1.Route) and [Service](https://projectcontour.io/docs/1.24/config/api/#projectcontour.io/v1.Service) (Backends) | Envoy does not natively support cookie attribute rewriting nor adding attributes other than path and TTL, but rewriting and adding additional attributes is possible via Lua ([Contour design reference](https://github.com/projectcontour/contour/blob/main/design/cookie-rewrite-design.md), [Envoy Issue](https://github.com/envoyproxy/envoy/issues/15612)). | +| Emissary-Ingress | Implementation (Envoy) | [Cookie-Based](https://www.getambassador.io/docs/emissary/latest/topics/running/load-balancer#cookie) | Name=name
Path=path
TTL=duration | [Module or Mapping](https://www.getambassador.io/docs/emissary/latest/topics/running/load-balancer#cookie) (Global or Route) | N/A | +| | | [Header-Based](https://www.getambassador.io/docs/emissary/latest/topics/running/load-balancer#header) | Name=name | [Module or Mapping](https://www.getambassador.io/docs/emissary/latest/topics/running/load-balancer#cookie) (Global or Route) | N/A | +| Envoy | Dataplane | [Cookie-Based](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/http/stateful_session/cookie/v3/cookie.proto) | Name=name
Path=path
TTL=duration | [HttpConnectionManager](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto) (Route) | Envoy does not natively support cookie attribute rewriting nor adding attributes other than path and TTL, but rewriting and adding additional attributes is possible via Lua ([Contour design reference](https://github.com/projectcontour/contour/blob/main/design/cookie-rewrite-design.md), [Envoy Issue](https://github.com/envoyproxy/envoy/issues/15612)). | +| | | [Header-Based](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/http/stateful_session/header/v3/header.proto) | Name=name | [HttpConnectionManager](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/network/http_connection_manager/v3/http_connection_manager.proto) (Route) | N/A | +| Envoy Gateway | Implementation (Envoy) | N/A | Supports Gateway API Only* | N/A | *Envoy Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. [Envoy Gateway Docs](https://gateway.envoyproxy.io/v0.3.0/index.html) | +| Flomesh Service Mesh | Implementation / Dataplane (Pipy) | ? | ? | ? | ? | +| Gloo Edge 2.0 | Implementation (Envoy) | [Cookie-Based](https://docs.solo.io/gloo-edge/latest/reference/api/envoy/api/v2/route/route.proto.sk/#cookie) | Name=name
Path=path
TTL=duration | [Route](https://docs.solo.io/gloo-edge/latest/reference/api/envoy/api/v2/route/route.proto.sk/#route) (Route) | N/A | +| | | [Header-Based](https://docs.solo.io/gloo-edge/latest/reference/api/envoy/api/v2/route/route.proto.sk/#hashpolicy) | Name=name | [Route](https://docs.solo.io/gloo-edge/latest/reference/api/envoy/api/v2/route/route.proto.sk/#route) (Route) | N/A | +| Google CloudRun | Implementation / Dataplane | [Cookie-Based](https://cloud.google.com/run/docs/configuring/session-affinity) | Enabled / Disabled | [Service](https://cloud.google.com/run/docs/configuring/session-affinity) (Backends) | Only allowed to turn off or on, no other configuration items | +| Google Kubernetes Engine | Implementation / Dataplane | [Cookie-Based](https://cloud.google.com/load-balancing/docs/backend-service#session_affinity) | GENERATED_COOKIE or HTTP_COOKIE=name
cookieTtlSec | [Backend Policy](https://cloud.google.com/kubernetes-engine/docs/how-to/configure-gateway-resources#session_affinity) (Backends) | Google Kubernetes Engine [lists](https://cloud.google.com/load-balancing/docs/backend-service#bs-session-affinity) the products that can do persistence/affinity mode. All persistence/affinity options are exclusive and can’t be used at the same time.
Note: Google Kubernetes Engine defines everything (persistence and affinity) as session affinity. | +| | | [Header-Based](https://cloud.google.com/load-balancing/docs/backend-service#header_field_affinity) | httpHeaderName=name | [Backend Policy](https://cloud.google.com/kubernetes-engine/docs/how-to/configure-gateway-resources#session_affinity) (Backends) | N/A | +| HAProxy | Dataplane | [Cookie-Based](https://docs.haproxy.org/2.6/configuration.html#4.2-cookie) | name=name
[rewrite \| insert \| prefix ]
indirect
nocache
postonly
preserve
httponly
secure
domain=domain
maxidle=idle
maxlife=life
dynamic
attr=value | [Default or Backends](https://docs.haproxy.org/2.6/configuration.html#4.2-cookie) (Global or Backends) | HAProxy allows for operational cookie strategy configuration (i.e. when/how HAProxy should inject cookies) | +| HAProxy Ingress | Implementation (HAProxy) | [Cookie-Based](https://haproxy-ingress.github.io/docs/configuration/keys/#affinity) | affinity (enable/disable)
cookie-key=key
session-cookie-domain=domain
session-cookie-dynamic=[true \| false]
session-cookie-keywords=keywords
session-cookie-name=name
session-cookie-preserve=[true \| false]
session-cookie-same-site=[true \| false]
session-cookie-shared=[true \| false]
session-cookie-strategy=strategy
session-cookie-value-strategy=value_strategy | [Backend](https://haproxy-ingress.github.io/docs/configuration/keys/#affinity) (Backends) | N/A | +| Hashicorp Consul | Implementation (Envoy) | N/A | Supports Gateway API Only* | N/A | *Hashicorp Consul solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. [Hashicorp Consul API Gateway Docs](https://developer.hashicorp.com/consul/docs/api-gateway) | +| Istio | Implementation (Envoy) | [Cookie-Based](https://istio.io/latest/docs/reference/config/networking/destination-rule/#LoadBalancerSettings-ConsistentHashLB-HTTPCookie) | Name=name
Path=path
TTL=duration | [ConsistentHashLB](https://istio.io/latest/docs/reference/config/networking/destination-rule/#LoadBalancerSettings-ConsistentHashLB) (Backends) | Istio also supports turning on cookie-based session persistence via the Pilot ENV variable [PILOT_PERSISTENT_SESSION_LABEL](https://istio.io/latest/docs/reference/commands/pilot-discovery/#envvars). | +| | Implementation (Envoy) | [Header-Based](https://istio.io/latest/docs/reference/config/networking/destination-rule/#LoadBalancerSettings-ConsistentHashLB) | Name=name | [ConsistentHashLB](https://istio.io/latest/docs/reference/config/networking/destination-rule/#LoadBalancerSettings-ConsistentHashLB) (Backends) | N/A | +| Java Servlet | Web Server | [Cookie-Based / URL-Encoded](https://docs.oracle.com/javaee/7/api/javax/servlet/http/HttpSession.html) | invalidate()
setAttribute(String name, Object value)
setMaxInactiveInterval(int interval) | N/A | Java Servlets do not natively support proxy functions. | +| Kong | Implementation / Dataplane | [Cookie-Based](https://docs.konghq.com/hub/kong-inc/session/) | cookie_name=name
rolling_timeout=timeout
absolute_timeout=timeout
idling_timeout=timeout
cookie_path=path
cookie_domain=domain
cookie_same_site=[Strict \| Lax \| None \| off]
cookie_http_only
cookie_secure=[true \| false]
stale_ttl=duration
cookie_persistent=[true \| false]
storage=storage_type | [Route, Service, Global](https://docs.konghq.com/hub/kong-inc/session/) (Route or Backends or Global) | N/A | +| | | [Header-Based](https://docs.konghq.com/gateway/latest/how-kong-works/load-balancing/#balancing-algorithms) | name | [Upstreams](https://docs.konghq.com/gateway/3.2.x/admin-api/#add-upstream) (Backends) | N/A | +| Kuma | Implementation (Envoy) | None | None | None | Kuma has no documentation on how it supports session persistence or cookies. [Kuma Docs](https://kuma.io/docs/2.1.x/) | +| Nginx | Dataplane | [Cookie-Based (Nginx Plus Only)](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#enabling-session-persistence) | Name=name
Expires=time
Domain=domain
HttpOnly
SameSite = [strict \| lax \| none \| $variable]
Secure
path=path | [Upstream](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#enabling-session-persistence) (Backends) | See also [Sticky Cookie](https://nginx.org/en/docs/http/ngx_http_upstream_module.html?&_ga=2.184452070.1306763907.1680031702-1761609832.1671225057#sticky_cookie) | +| NGINX Kubernetes Gateway | Implementation (Nginx) | N/A | Supports Gateway API Only* | N/A | *NGINX Kubernetes Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session persistence. [Nginx Kubernetes Gateway Docs](https://github.com/nginxinc/nginx-kubernetes-gateway) | +| Traefik | Implementation / Dataplane | [Cookie-Based](https://doc.traefik.io/traefik/routing/services/#sticky-sessions) | name=name
secure
httpOnly
sameSite=[none \| lax \| strict ] | [Services](https://doc.traefik.io/traefik/routing/services/#sticky-sessions) (Backends) | N/A | + +**Session Affinity** + +| **Technology** | **Technology Type** | **Session Affinity Type** | **Notes** | +|--- |--- |--- |--- | +| Acnodal EPIC | Implementation (Envoy) | Supports Gateway API Only* | *Acnodal Epic solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. [Acnodal EPIC Docs](https://www.epick8sgw.io/docs/) | +| Apache APISIX | Implementation (Nginx) | [Consistent Hashing](https://apisix.apache.org/docs/apisix/plugins/traffic-split/#attributes) via consumer ID or other variables | N/A | +| Cilium | Implementation / Dataplane | [Source IP Address](https://docs.cilium.io/en/v1.12/gettingstarted/kubeproxy-free/#id2) (Default) | N/A | +| Contour | Implementation (Envoy) | Consistent Hashing via [RequestHash](https://projectcontour.io/docs/1.24/config/request-routing/#load-balancing-strategy) | N/A | +| Emissary-Ingress | Implementation (Envoy) | Consistent Hashing via [sourceIP](https://www.getambassador.io/docs/emissary/latest/topics/running/load-balancer#source-ip) | N/A | +| Envoy | Dataplane | Consistent Hashing via [Maglev](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/load_balancers#maglev) and [Ring Hash](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/load_balancing/load_balancers#ring-hash) | N/A | +| Envoy Gateway | Implementation (Envoy) | Supports Gateway API Only* | *Envoy Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. [Envoy Gateway Docs](https://gateway.envoyproxy.io/v0.3.0/index.html) | +| Flomesh Service Mesh | Implementation / Dataplane (Pipy) | ? | ? | +| Gloo Edge 2.0 | Implementation (Envoy) | Consistent Hashing via HashPolicy of [SourceIP](https://docs.solo.io/gloo-edge/latest/reference/api/envoy/api/v2/route/route.proto.sk/#connectionproperties) | N/A | +| Google CloudRun | Implementation / Dataplane | None | [Google CloudRun Docs](https://cloud.google.com/run/docs) | +| Google Kubernetes Engine | Implementation / Dataplane | [CLIENT_IP_NO_DESTINATION](https://cloud.google.com/load-balancing/docs/backend-service#client-ip-no-destination-affinity) and [CLIENT_IP](https://cloud.google.com/load-balancing/docs/backend-service#client_ip_affinity) | Google Kubernetes Engine [lists](https://cloud.google.com/load-balancing/docs/backend-service#bs-session-affinity) the products that can do persistence/affinity mode. Session affinity settings are fulfilled only if the load balancing locality policy (LocalityLbPolicy) is set to RING_HASH or MAGLEV. All persistence/affinity options are exclusive and can’t be used at the same time.
Note: Google Kubernetes Engine defines everything (persistence and affinity) as session affinity. | +| HAProxy | Dataplane | [Stick Tables](https://cbonte.github.io/haproxy-dconv/2.4/configuration.html#4-stick-table) using connection attributes | N/A | +| HAProxy Ingress | Implementation (HAProxy) | None | [Haproxy Ingress Docs](https://haproxy-ingress.github.io/docs/configuration/keys/) | +| Hashicorp Consul | Implementation (Envoy) | Supports Gateway API Only* | *Hashicorp Consul solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. [Hashicorp Consul API Gateway Docs](https://developer.hashicorp.com/consul/docs/api-gateway) | +| Istio | Implementation (Envoy) | Consistent Hashing via [ConsistentHashLB](https://istio.io/latest/docs/reference/config/networking/destination-rule/#LoadBalancerSettings-ConsistentHashLB) with Maglev or Ring Hash | N/A | +| Kong | Implementation / Dataplane | [Consistent Hashing](https://docs.konghq.com/gateway/latest/how-kong-works/load-balancing/#balancing-algorithms) via Source IP or Consumer ID | N/A | +| Kuma | Implementation (Envoy) | Consistent Hashing via [Ring Hash](https://kuma.io/docs/2.1.x/policies/traffic-route/#load-balancer-types) | N/A | +| Nginx | Dataplane | [Consistent Hashing](https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash) via IP or other connection attributes (Nginx Open Source) | N/A | +| NGINX Kubernetes Gateway | Implementation (Nginx) | Supports Gateway API Only* | *NGINX Kubernetes Gateway solely uses Gateway API; therefore, it doesn’t yet have a way to configure session affinity. [Nginx Kubernetes Gateway Docs](https://github.com/nginxinc/nginx-kubernetes-gateway) | +| Traefik | Implementation / Dataplane | None | [Traefik Docs](https://doc.traefik.io/traefik/) | + +### Sessions in Java + +Java application servers such as Tomcat and Jetty, were the first to standardize the API around persistent sessions. These Java applications introduced the “jsessionid” cookie as well as more advanced features such as session migration, replication, and on demand session activation. It’s important for Gateway API to examine persistent session use cases and history from Java APIs to ensure the API is designed appropriately. + +## API + +### Session Persistence API + +The majority of implementations and dataplanes support configuring cookies for session persistence while support for header-based session persistence is limited. + +To organize potential Gateway API configuration of cookie functionality, the next section will break down configuration into specific categories: +- Session Cookie Configuration +- Session Cookie Strategy + +**Session Cookie Configuration** + +Session cookie configuration are the parameters and attributes the dataplane uses to create a cookie for establishing a session. + +A set-cookie request can be broken down into [3 main components](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie): +- Cookie Name +- Cookie Value +- Cookie Attributes + +Only the Cookie Name and Cookie Value are required. All Cookie Attributes are optional. Attributes are simply key=value pairs with the value as optional for some attributes. + +The set-cookies attributes defined by [rfc6265](https://www.rfc-editor.org/rfc/rfc6265#section-5.2) are: +- Expires=_date_ +- Max-Age=_number_ +- Domain=_domain_ +- Path=_path-value_ +- Secure +- HttpOnly + +However, there are de-facto standard attributes that have been proposed as updates to [rfc6265](https://www.rfc-editor.org/rfc/rfc6265#section-5.2): +- SameSite=_[Strict|Lax|None]_ ([Draft RFC Update](https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-cookie-same-site-00)) +- Partitioned ([Draft RFC Update](https://www.ietf.org/archive/id/draft-cutler-httpbis-partitioned-cookies-01.html)) + +**Session Cookie Strategy** + +Session cookie strategy is configuration specific to the creation of the cookie by the dataplane. These are not specific configuration parameters of the cookie itself, but instead the operation to take when inserting the cookie such as: +- Rewriting an existing cookie value +- Inserting a new cookie +- Prefixing an existing cookie +- Preserving an existing cookie value +- Only adding cookies on specific HTTP methods (POST/GET) + +### Session Affinity API + +TBD - See [open question](#open-questions) about session affinity API + +### Open Questions + +- Should we include both session persistence and session affinity API configurations in this GEP or just focus on session persistence? +- Should both session persistence and session affinity be designed as common load balancing policy API? Should they be configured as separately in the API? +- What happens when session persistence is broken because the backend is not up or healthy? If that's an error case, how should that be handled? Should the API dictate the http error code? Or should the API dictate fall back behavior? + +## TODO +The following are items that we intend to resolve before we consider this GEP implementable: + +- We need to identify the needs and use cases from end users for specifying session persistence. +- We need to identify and document requirements regarding session draining and migration. How do implementations drain established sessions during backend upgrades without disruption? +- We need to document sessions with Java in greater detail. Java standardized the API and behavior of session persistence long ago and would be worth examining. +- We need to add a small section on compliance regarding the browser and client relationship. +- We need to design and document an API for session persistence and/or session affinity in the Gateway API spec (see [open question](#open-questions) about API design) + +## Alternatives + +### Alternate Naming + +This GEP describes session persistence and session affinity as the idea of strong and weak connection persistence respectively. Other technologies use different names or define persistence and affinity differently: + +- Envoy defines [stateful sessions](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/http/stateful_session/cookie/v3/cookie.proto) as what we've defined as session persistence +- Google Cloud defines [session affinity](https://cloud.google.com/run/docs/configuring/session-affinity) as what we've defined as session persistence +- Nginx defines [session persistence](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/#enabling-session-persistence) as what we've defined as both session persistence and affinity +- Traefik defines [sticky sessions](https://doc.traefik.io/traefik/routing/services/#sticky-sessions) as what we've defined as session persistence +- Apache httpd defines [sticky sessions or stickyness](https://httpd.apache.org/docs/2.4/mod/mod_proxy_balancer.html) as what we've defined as session persistence +- Kubernetes defines [session affinity](https://kubernetes.io/docs/reference/networking/virtual-ips/#session-affinity) based on client IP hashing (same as our session affinity) + +Though session persistence is a ubiquitous name, session affinity is more inconsistently used. An alternate decision could be made to use a different name for session affinity based on the prevalence of other naming conventions. + +## References + +- [LBPolicy](https://static.sched.com/hosted_files/kccnceu2023/c4/Autoscaling%20Elastic%20Kubernetes%20Infrastructure%20for%20Stateful%20Applications%20using%20Proxyless%20gRPC%20and%20Istio.pdf#page=25) (proposed extension for session persistence API) +- [gRPC Stateful Session Affinity Proposal](https://github.com/grpc/proposal/blob/master/A55-xds-stateful-session-affinity.md) (info on session draining and session persistence in gRPC) +- [Kube-Proxy Session Affinity](https://kubernetes.io/docs/reference/networking/virtual-ips/#session-affinity) diff --git a/geps/gep-1651.md b/geps/gep-1651.md new file mode 100644 index 0000000000..92865578d5 --- /dev/null +++ b/geps/gep-1651.md @@ -0,0 +1,411 @@ +# GEP-1651: Gateway Routability + +* Issue: [#1651](https://github.com/kubernetes-sigs/gateway-api/issues/1651) +* Status: Provisional + +(See status definitions [here](overview.md#status).) + +## TLDR + +Allow users to configure a Gateway so that it is only routable within +a specific scope (ie. public/private/cluster) + +## Goals + +- Define a mechanic to set the routability on a Gateway +- Provide a default set of routability options +- Provide a way for vendors to support custom options + +## Non-Goals + +- Per-request/route scope +- Not a lightweight service mesh + +## Introduction + +One of the early feature requests for Knative was the ability to deploy an +application using Knative's HTTP routing support, but make it only available +within the cluster. I want to be able to specify both the "cluster" +(service.namespace.svc) and "external" (service.namespace.example.com). +Gateways using the same GatewayClass on the cluster, but ensure that the +"cluster" service is only routable within the cluster. This would greatly +simplify deployment for users over the instructions we have today. + +Likewise another use case is to provide load balancing capabilities within a virtual +private network. Different IaaS providers offer private load balancers to support +these use cases. + +## API + +We propose adding a new `routability` field under the `spec.infrastructure` stanza of a Gateway. + +### Predefined Routability Values + +Implementations MAY implement the following values for 'routability' and MUST abide by +their defined semantics. + +Value | Scope +-|- +`Public`|The address is routable on the public internet +`Private`|The address is routable inside a private network larger than a single cluster (ie. VPC) and MAY include RFC1918 address space +`Cluster`|The address is routable inside the [cluster's network](https://kubernetes.io/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-network-model) + +Values can be compared semantically - `Public` has a larger scope than `Private`, while `Private` has a larger scope than `Cluster`. + +### Vendor prefixed values + +Implementations can define custom 'routability' values by specifying a vendor prefix followed +by a slash `/` and a custom name ie. `com.example.com/my-routability`. + +Comparing vendor prefixed scopes with the pre-defined ones in implementation specific. + +### Default Routability + +The default value of `routability` is implementation specific. It is RECOMMENDED that +the default `routability` remains consistent for Gateways with the same +`gatewayClassName`. + +Implementations MUST signal the default routability using the Gateway's `status.addresses`. See 'Status Addresses` +for more details. + +### Mutability + +Implementations MAY prevent end-users from updating the `routability` value of a Gateway. If +updates are allowed the semantics and behaviour will depend on the underlying implementation. + +If a Gateway is mutated but does not support the desired routability it MUST set the conditions +`Accepted`, `Programmed` to `False` with `Reason` set to `UnsupportedRoutability`. Implementations +MAY choose to leave the old Gateway running with the previous generation's configuration. + +### Go + +```go + +// GatewayRoutability represents the routability of a Gateway +// +// The pre-defined values listed in this package can be compared semantically. +// `Public` has a larger scope than `Private`, while `Private` has a larger scope than +// `Cluster`. +// +// Implementations can define custom routability values by specifying a vendor +// prefix followed by a slash '/' and a custom name ie. `dev.example.com/my-routability`. +// +// +kubebuilder:validation:MinLength=1 +// +kubebuilder:validation:MaxLength=253 +// +kubebuilder:validation:Pattern=`^Public|Private|Cluster|[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*\/[A-Za-z0-9\/\-_]+$` +type GatewayRoutability string + +const ( + // GatewayRoutabilityPublic means the Gateway's address MUST + // be routable on the public internet + // + // Implementations MAY support this routability + GatewayRoutabilityPublic GatewayRoutability = "Public" + + // GatewayRoutabilityPrivate means the Gateway's address MUST + // only be routable inside a private network larger than a single + // cluster (ie. VPC) and MAY include the RFC1918 address space + // + // Implementations MAY support this routability + GatewayRoutabilityPrivate GatewayRoutability = "Private" + + // GatewayRoutabilityCluster means the Gateway's address MUST + // only be routable inside the [cluster's network] + // + // Implementations MAY support this routability + // + // [cluster's network](https://kubernetes.io/docs/concepts/cluster-administration/networking/#how-to-implement-the-kubernetes-network-model) + GatewayRoutabilityCluster GatewayRoutability = "Cluster" +) + +type GatewaySpec struct { + // Infrastructure defines infrastructure level attributes about this Gateway instance. + Infrastructure GatewayInfrastructure `json:"infrastructure"` + // ... +} +type GatewayInfrastructure struct { + // Routability allows the Gateway to specify the accessibility of its addresses. Setting + // this property will override the default value defined by the GatewayClass. + // + // If the desired Gateway routability is incompatible with the GatewayClass implementations + // MUST set the condition `Accepted` to `False` with `Reason` set to `UnsupportedRoutability`. + + // The default value of routability is implementation specific and MUST remains consistent for + // Gateways with the same gatewayClassName + // + // Implementations MAY prevent end-users from updating the routability value of a Gateway. + // If updates are allowed the semantics and behaviour will depend on the underlying implementation. + // If a Gateway is mutated but does not support the desired routability it MUST set `Accepted` + // and `Programmed` conditions to `False` with `Reason` set to `UnsupportedRoutability`. + // + // It is RECOMMENDED that in-cluster gateways SHOULD NOT support 'Private' routability. + // Kubernetes doesn't have a concept of 'Private' routability for Services. In the future this may + // change upstream. + // + // +optional + Routability *GatewayRoutability `json:"routability,omitempty"` +} + +type GatewayStatus struct { + // Addresses lists the IP addresses that have actually been + // bound to the Gateway. These addresses may differ from the + // addresses in the Spec, e.g. if the Gateway automatically + // assigns an address from a reserved pool. + // + // Implementations that support Gateway routability MUST include an address + // that has the same routable semantics as defined in the Gateway spec. + // + // Implementations MAY add additional addresses in status, but they MUST be + // semantically less than the scope of the requested scope. For example if a + // user requests a `Private` routable Gateway then an additional address MAY + // have a routability of `Cluster` but MUST NOT include `Public`. + // + // +optional + // +kubebuilder:validation:MaxItems=16 + Addresses []GatewayStatusAddress `json:"addresses,omitempty"` + // ... +} + +type GatewayStatusAddress struct { + // Routability specifies the routable bounds of this address + // Predefined values are: 'Private', 'Public', Cluster + // Other values MUST have a vendor prefix. + // + // Implementations that support Routability MUST populate this + // field + // + // +optional + Routability *GatewayRoutability `json:"routability,omitempty"` + + // ... +} + +type GatewayClassStatus struct { + // Routabilities specifies a list of supported routabilities offered by + // the GatewayClass. The first entry in this list will be the default + // routability used when Gateways of this class are created. + // + // Implemenations MAY provide a pre-defined set of GatewayClasses that + // limit the routability choices of a Gateway. + // + // Implementations that support routability MUST populate this list with + // a subset of the pre-defined GatewayRoutability values or vendored + // prefix values. + // + // +optional + // +kubebuilder:validation:MaxItems=8 + // + Routabilities []GatewayRoutability `json:"routabilities"` +} +``` + +### YAML +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: prod-web +spec: + gatewayClassName: acme-lb + infrastructure: + routability: Public + listeners: + - protocol: HTTP + port: 80 +``` + +### Semantics + +#### Interaction with GatewayClass + +An infrastructure provider MAY provide a pre-defined set of GatewayClasses that limit the +routability choices of a Gateway. If the desired Gateway routability is incompatible with the +GatewayClass it MUST set the condition `Accepted` to `False` with `Reason` set to `UnsupportedRoutability`. + +If an implementation supports 'routability' then the GatewayClass MUST list the supported +routabilities in the status stanza. The `status.routabilities` MUST contain either +a subset of the pre-defined values mentioned above or contain vendored prefixed values. + +The first value in the list will be used as the default value when Gateways of this class +are created. This can be overriden by setting the Gateway's `spec.infrastructure.routability`. + +#### Unsupported routability & address values + +If a Gateway is unable to provide an address for the desired routability it MUST set the condition `Accepted` +to `False` with `Reason` set to `UnsupportedRoutability` + +#### Status.Addresses + +If a Gateway supports the desired 'routability' implementations MUST populate the `status.addresses` with +an address that has the same routable semantics. The GatewayAddress field `routability` MUST be populated. + +Implementations MAY add additional addresses in status, but they MUST be semantically less than the scope +of the requested scope. For example if a user requests a `Cluster` routable Gateway then the list of addresses +MUST NOT have a routability of `Public` or `Private`. + +We plan on introducing a new type `GatewayStatusAddress` and change Gateway's `status.addresses` to be +`[]GatewayStatusAddress`. This will allow the status address type to evolve separately from the spec address. + +#### In-cluster Gateways and 'Private' Routability + +It is RECOMMENDED that in-cluster gateways SHOULD NOT support 'Private' routability. Kubernetes doesn't have +a concept of 'Private' routability for Services. In the future this may change upstream. + +## Examples + +#### 1. Request a GatewayAddress that is routable within the same cluster + +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: prod-web +spec: + gatewayClassName: acme-lb + infrastructure: + routability: Cluster + listeners: + - protocol: HTTP + port: 80 +``` + +#### 2. Request a GatewayAddress with a specific routability and address +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: prod-web +spec: + gatewayClassName: acme-lb + infrastructure: + routability: Cluster + listeners: + - protocol: HTTP + port: 80 + addresses: + - value: 10.0.0.8 +``` + +#### 3. Request a GatewayAddress that is routable on the public internet +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: prod-web +spec: + gatewayClassName: acme-lb + infrastructure: + routability: Public + listeners: + - protocol: HTTP + port: 80 +``` + +#### 4. Request a GatewayAddress that is a cloud provider's VPC +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: prod-web +spec: + gatewayClassName: acme-lb + infrastructure: + routability: Private + listeners: + - protocol: HTTP + port: 80 +``` + +## Alternatives + +### Introducing new GatewayAddress Types + +We could introduce additional `AddressTypes` (ie. `ClusterLocalIPAddress`) but +this would lead to a combinatorial explosion as new dimensions (ie. IPv6) are +introduced. + +From: [https://github.com/kubernetes-sigs/gateway-api/pull/1653#issuecomment-1451246877](https://github.com/kubernetes-sigs/gateway-api/pull/1653#issuecomment-1451246877) + +> Although this makes sense in isolation, I'm worried about the long term impacts this could have. In my opinion, ClusterLocal is a modifier, not exactly an address type. For example, it's possible in the future that we'll have a way to provision cluster-local DNS names, we may want to use the same kind of mechanism to request a ClusterLocal DNS name for the Gateway. +> +> It's also possible that users will want to explicitly request an IP Families (v4, v6, or both). I'd really hate to get into a situation where we have the following options: +> +> - IPAddress +> - IPv4Address +> - IPv6Address +> - ClusterLocalIPAddress +> - ClusterLocalIPv4Address +> - ClusterLocalIPv6Address +> +> For each dimension we avoid adding a separate field for and instead try to embed into a single name, we risk this kind of name explosion. Of course none of the above even begins to cover my idea of NetworkLocal which could further complicate this. + +### Scope/reachability/routability field on GatewayAddress + +This would allow Gateways to have multiple scopes. + +From: [https://github.com/kubernetes-sigs/gateway-api/pull/1653#issuecomment-1486271913](https://github.com/kubernetes-sigs/gateway-api/pull/1653#issuecomment-1486271913) +> The obvious application for multiple scopes seems to be saving on boilerplate, which is a win, but are there are any other advantages to allowing one Gateway to have multiple scopes? +> +> Multiple scopes Pros: +> +> Allows a single Gateway to express multiple networks, saving on needing to attach HTTPRoutes to multiple Gateways for each network scope. +> +> Multiple scopes Cons: +> +> Complicates the Gateway's purpose. Instead of one Gateway being one set of Listeners, now a Gateway is two sets of listeners that have a totally different scope (and presumably, security context). Personally, I'm also concerned how this will interact with other features like merging and preprovisioning that GEP-1867: Per-Gateway Infrastructure #1868 will allow. + +### Adding `routability` attribute to GatewayClass + +See [Prior Art - Multiple Gateways Classes](#multiple-gateway-classes) + +## Survey of Prior Art + +These alternatives are a survey of existing approaches to support cluster +local Gateways. Most are implementation specific and are not portable. + +### Special annotation/label + +Istio let's you specify an annotation `networking.istio.io/service-type` to +change the underlying Kubernetes Service type to make it a ClusterIP type. + +### Re-use of AddressType Hostname + +Istio let's you re-use existing Gateway deployments by setting the address +type to `Hostname` and the value to the Istio ingress Kubernetes Service. If an +operator configures the Istio deployment to support cluster local traffic a +Gateway implementation can select it using the `HostName` attribute. + +### Multiple Gateway Classes + +Some implementations support multiple deployments on a single cluster where each maps to a +GatewayClass. One of these deployments can be configured to serve cluster local traffic. This is +sub-optimal because this is implementation specific and the end-user is effectively managing the +deployments themselves rather than infrastructure being automatically provisioned. + +Likewise, infrastructure providers may provide a fixed set of GatewayClasses with unique and fixed +routability. Thus GatewayClass name is a viable option to control routability. There may be a +non-zero cost when requiring additional GatewayClasses - but this depends on the implementation. + +Additionally, if more attributes are added to GatewayClass to constrain Gateways in some +form this leads to a combinatorial number of GatewayClassNames. For example, `foo-public` and +`foo-cluster` are two GatewayClasses surfacing the values of a single attribute `routability`. +Let's say we want to enforce address types to just IP then our `gatewayClassName` would be: + +- `foo-public-ipv4` +- `foo-public-ipv6` +- `foo-cluster-ipv4` +- `foo-cluster-ipv6` + +This may not be as flexible for end-users compared to configuring `routability` when creating +a Gateway. + +As howardjohn mentioned [here](https://github.com/kubernetes-sigs/gateway-api/pull/1653#issuecomment-1429992160): +> having the ability to configure things at a higher level seems nice for Gateway, but being able to configure them on a per-Gateway basis remains important. + +## References + +- [Knative - Private Services](https://knative.dev/docs/serving/services/private-services/#configuring-private-services) +- [Initial Gateway GitHub Discussion](https://github.com/kubernetes-sigs/gateway-api/discussions/1247) +- [Istio Support for Private Gateways](https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/#automated-deployment) +- [Envoy Gateway Support for Private Gateways](https://gateway.envoyproxy.io/latest/api/config_types.html#kubernetesservicespec) + diff --git a/geps/gep-1686.md b/geps/gep-1686.md new file mode 100644 index 0000000000..0f2e0c292e --- /dev/null +++ b/geps/gep-1686.md @@ -0,0 +1,108 @@ +# GEP-1686: Mesh conformance testing plan + +- Issue: [#1686](https://github.com/kubernetes-sigs/gateway-api/issues/1686) +- Status: Implementable + +## TLDR + +This testing plan specifies a new set of tests to define a "Mesh" [conformance profile](https://github.com/kubernetes-sigs/gateway-api/issues/1709). + +## Goals + +* Define a strategy for segmenting GAMMA tests from the existing conformance test suite +* Define a set of test scenarios to capture conformance with the GAMMA spec +* Rely on existing tests for non-GAMMA-specific Gateway API conformance + +## Focus + +Currently the GAMMA spec consists of two Gateway API GEPs [defining terminology and goals of Gateway API for service meshes](https://gateway-api.sigs.k8s.io/geps/gep-1324/) +and specifically [how route resources work in a service mesh context](https://gateway-api.sigs.k8s.io/geps/gep-1426/). +The goal of the initial conformance testing is to check the essential behavior as defined by GEP-1426, as it differs from the wider Gateway API spec. This GEP focuses on using a `Service` object as an `xRoute` `parentRef` to control how the GAMMA implementation directs traffic to the endpoints specified by the `Services` in `backendRefs` and how the traffic is filtered and modified. + +## Conformance Profile + +GAMMA intends to introduce a "Mesh" [conformance profile](https://gateway-api.sigs.k8s.io/geps/gep-1709/) to isolate tests specific to East/West functionality from both existing tests focused on North/South functionality and common Gateway API functionality shared by N/S and E/W implementations. A conformance profile is a set of tests that implementations can run to check their conformance to some subset of the Gateway API spec. + +This appropach will enable service meshes to certify that an implementation follows the GAMMA spec without requiring a North/South implementation, and importantly avoid any expectation that North/South Gateway API implementations expand their scope to understand GAMMA and E/W traffic flows. + +Leveraging existing tests for common functionality between N/S and E/W implementations will both ensure consistency across Gateway API implementations and help limit the maintence burden for the conformance testing suite. + +### Support Levels + +Using a conformance profile will enable granular conformance definitions for GAMMA, splitting functionality along the existing Gateway API [support levels](https://gateway-api.sigs.k8s.io/concepts/conformance/?h=conformance+levels#2-support-levels), with required functionality as Core, standardized but optional functionality as Extended, and Implementation-specific for things beyond the current or intended scope of GAMMA configuration. It is expected that some capabilities will begin as Implementation-specific and eventually migrate to Extended or Core conformance as GAMMA matures. + +## Tests + +Testing GAMMA implementations requires both a new suite of test cases as well as refactoring the existing test framework setup. + +### Runner and Setup + +The existing Gateway API conformance tests use a relatively simple implementation to send requests from outside a Kubernetes cluster to a gateway sitting at the edge, [capture the request and response](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/utils/roundtripper/roundtripper.go), and [assert a match against an expected response](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/utils/http/http.go). + +GAMMA conformance tests should still be based around a request/expected response suite, but requests will need to originate from _inside the cluster_, from either the same or different namespace as the target service. Adopting or developing tooling to enable this is being explored in [gateway-api#1340](https://github.com/kubernetes-sigs/gateway-api/issues/1340). + +### Scenarios + +All requests are sent from a client inside the same cluster/mesh and the same `Namespace` +as the `Service` under test. +Test scenarios are largely focused on the `backendRefs` and the +`Namespace` of an `xRoute` resource. + +#### `Service` as `parentRef` + +- Given a simple `HTTPRoute` with a single `backendRef` + - With an explicit `port` in `parentRef` + - Assert that only requests to this `Service` and `port` are directed to the + backend + - Without a `port` in `parentRef` + - Assert that all requests to this `Service` are directed to the backend + +#### Omitted `backendRefs` + +- Given a simple `HTTPRoute` without `backendRefs` + - Assert that requests are directed to the endpoints defined by the `Service` + `parentRef` in its backend role + +#### Only `Services` as frontends are affected + +- Given a simple `HTTPRoute` with a single `backendRef` + - Send requests directly the endpoints of the `parentRef` `Service`'s backend + - Assert that traffic is not affected by the `HTTPRoute` resource + +#### `Namespace`-dependent behavior, producer vs consumer + +A producer `HTTPRoute` is in the same namespace as the `parentRef` `Service` (the +producer). + +- Given a producer `HTTPRoute` + - Assert that traffic from a client in the producer `Namespace` is routed by the + `HTTPRoute` + - Assert that traffic from a client in a different `Namespace` is routed by the + `HTTPRoute` + +A consumer `HTTPRoute` is in the same `Namespace` as the the request sender (the +consumer), a different `Namespace` as the `parentRef` `Service`. + +- Given a consumer `HTTPRoute` + - Assert that traffic from the consumer client is routed by the `HTTPRoute` + - Assert that traffic from a client in a different `Namespace` is _not_ routed by the + `HTTPRoute` + +Consumer routes have priority over producer routes. + +- Given both a consumer `HTTPRoute` and a producer `HTTPRoute` + - Assert that traffic from the consumer client is routed by the consumer `HTTPRoute` + - Assert that traffic from a client in a different `Namespace` is routed by + the producer `HTTPRoute` + +#### `xRoute`-specific + +- Given multiple `xRoutes` of different types + - Assert that routes take affect according to the specificity as defined in the spec +- Given an `HTTPRoute` without `matches`, all requests are received at the `Service` endpoints as if no `HTTPRoute` existed +- Given an `HTTPRoute` with `matches`, unmatched requests are dropped with a 404 + +#### Filters + +Filters have the same effects on requests as any implementation. Gateway API conformance test framework can be +refactored to extract checks on filter behavior for use on both GAMMA and Gateway API tests. diff --git a/geps/gep-1709.md b/geps/gep-1709.md new file mode 100644 index 0000000000..74d7593d9a --- /dev/null +++ b/geps/gep-1709.md @@ -0,0 +1,633 @@ +# GEP-1709: Conformance Profiles + +* Issue: [#1709](https://github.com/kubernetes-sigs/gateway-api/issues/1709) +* Status: Experimental +* Probationary Period: Re-evaluate in February 2024 + +## TLDR + +Add high level profiles for conformance tests which implementations can select +when running the conformance test suite. Also add opt-in automated conformance +reporting to the conformance test suite to report conformance results back to +the Gateway API project and receive recognition (e.g. badges). + +## Goals + +- Add high level profiles which downstream implementations can subscribe to in + order to run tests for the associated supported feature sets. +- Add a reporting mechanism where conformance results can be reported back to + the Gateway API project and provide "badges" to visibly decorate the + implementations as conformant according to their profiles. +- Expand conformance testing documentation significantly so that it becomes the + "landing zone" for new prospective implementations and provides a clear and + iterative process for how to get started implementing Gateway API support. + +## Non-Goals + +- We want to avoid adding any infrastructure for the reporting mechanism if + feasible. +- For this iteration we don't want to add configuration files for conformance + tests, instead leaving that to future iterations and working on the raw + machinery here (see [alternatives considered](#alternatives-considered)). +- For this iteration we don't want to add container images for conformance test + runs, instead leaving that to future iterations (see + [alternatives considered](#alternatives-considered). + +## Introduction + +Since our conformance test suite was conceived of it's been our desire to +provide simple high level profiles that downstream implementations can +subscribe to. + +Today we have `SupportedFeatures` which get us some of what we want in terms of +easily configuring the conformance test suite, but in this GEP we will describe +taking that a step further (and a level higher) to create named profiles which +indicate a "level of conformance" which implementations can prove they satisfy +and receive certification for. + +An API will be provided as the format for conformance test reports. We will +provide tooling to assist with the reporting and certification process of +submitting those reports and displaying the results. + +## API + +The API for conformance profiles will include an API resource called +`ConformanceReport` which will be at the center of a workflow that +implementations can opt into to generate and submit those resources. + +The workflow implementers will follow will include the following high-level +steps: + +1. select a [profile](#profiles) +2. [integrate](#integration) tests in the downstream project +3. [report results and get certified](#certification) + +The goal is to make selecting a conformance profile as simple and automatic of a +process as feasible and support both the existing command line integration +approach (e.g. `go test`) as well as a [Golang][go] approach using the +conformance suite as a library. + +[go]:https://go.dev + +### Profiles + +"Profiles" are effectively categories which represent the high level grouping of +tests related to some feature (or feature set) of Gateway API. When conformance +is reported using one of these profiles extra features can be covered according +to support levels: + +- `core` +- `extended` + +> **NOTE**: `implementation-specific` doesn't really have much in the way of +> tests today, but it is something users want to be able to display. We leave +> door open for it later and mention it in the [alternatives +> considered](#alternatives-considered) section below. + +We will start with the following named profiles: + +- `HTTP` +- `TLSPassthrough` + +These profiles correspond with `*Route` type APIs that we currently have tests +for. As the tests roll in, we'll also eventually have: + +- `UDP` +- `TCP` +- `GRPC` + +> **NOTE**: In time we may have higher level groupings, like `Layer4` (which +> would include at least `TCP` and `UDP`) but feedback from the community has +> been strong for a preference on the `*Route` level (see the +> [alternatives considered](#alternatives-considered) for some more notes on +> this) for the moment. + +> **NOTE**: APIs that are referenced to or by `*Route` APIs will be tested as +> a part of a profile. For instance, running the `HTTP` profile will also run +> tests for `GatewayClass` and `Gateway` implicitly as these are required +> components of supporting `HTTP`. + +The technical implementation of these profiles is very simple: effectively a +"profile" is a static compilation of existing [SupportedFeatures][feat] which +represent the named category. Features that aren't covered under a "core" level +of support are opt-in. + +[feat]:https://github.com/kubernetes-sigs/gateway-api/blob/c61097edaa3b1fad29721e787fee4b02c35e3103/conformance/utils/suite/suite.go#L33 + +### Integration + +Integrating the test suite into your implementation can be done using one of +the following methods: + +- The [go test][go-test] command line interface which enables projects of any + language to run the test suite on any platform [Golang][go] supports. +- Using the conformance test suite as a [Golang library][lib] within an already + existing test suite. + +> **NOTE**: Usage as a library is already an established colloquialism in the +> community, this effort simply intends to make that more official. + +Conformance profiles are passed as arguments when running the test suite. For +instance when running via command line: + +```console +$ go test ./conformance/... -args -gateway-class=acme -conformance-profile=Layer7 +``` + +Or the equivalent configuration using the Golang library: + +```go +cSuite, err := suite.New(suite.Options{ + GatewayClassName: "acme", + Profiles: sets.New(Layer7), + // other options +}) +require.NoError(t, err, "misconfigured conformance test suite") +cSuite.Setup(t) + +for i := 0; i < len(tests.ConformanceTests); i++ { + test := tests.ConformanceTests[i] + test.Run(t, cSuite) +} +``` + +> **NOTE**: In the `suite.Options` above it's still possible to add `SkipTests` +> but when used in conjunction with `Profile` this will result in a report that +> the profile is not valid for reporting. Implementations in this state may be +> able to report themselves as "in progress", see the +> [certification section](#certification) for details. + +Alternatively for an `Extended` conformance profile where not all of the +features are implemented (as described in the [profiles](#profiles) section +above): + +```console +$ go test ./conformance/... -args \ + -gateway-class=acme \ + -conformance-profiles=HTTP,TCP \ + -unsupported-features=HTTPResponseHeaderModification,HTTPRouteMethodMatching,HTTPRouteQueryParamMatching, +``` + +Or the equivalent configuration using the Golang library: + +```go +cSuite, err := suite.New(suite.Options{ + GatewayClassName: "acme", + Profiles: sets.New( + HTTP, + TCP, + ), + UnsupportedFeatures: sets.New( + suite.SupportHTTPResponseHeaderModification, + suite.SupportHTTPRouteMethodMatching, + suite.SupportHTTPRouteQueryParamMatching, + ), + // other options +}) +require.NoError(t, err, "misconfigured conformance test suite") +cSuite.Setup(t) + +for i := 0; i < len(tests.ConformanceTests); i++ { + test := tests.ConformanceTests[i] + test.Run(t, cSuite) +} +``` + +> **NOTE**: You can't disable features that are `Core` conformance as `Core` is +> a minimum requirement for the profile to be considered fulfilled. + +Some implementations may support more or less extended features than others, +so in some cases it could be cumbersome to have to list ALL features that you +_don't_ support so we optionally and inversely allow `SupportedFeatures` so +you can pick which option makes sense to you, and under the hood the +expressions will compile to the same overall list: + +```go +cSuite, err := suite.New(suite.Options{ + GatewayClassName: "acme", + Profiles: sets.New( + HTTP, + TCP, + ), + SupportedFeatures: sets.New( + suite.SupportHTTPRouteMethodMatching, + ), + // other options +}) +``` + +> **NOTE**: The `UnsupportedFeatures` and `SupportedFeatures` fields are +> mutually exclusive. + +So to have your YAML report include details about extended features you support +you must either opt-in using `SupportedFeatures` to the exact features you +support, or opt-out of the features you _don't_ support using +`UnsupportedFeatures`. + +Once an implementation has integrated with the conformance test suite, they can +move on to [certification](#certification) to report the results. + +[go-test]:https://go.dev/doc/tutorial/add-a-test +[go]:https://go.dev +[lib]:https://pkg.go.dev/sigs.k8s.io/gateway-api@v0.6.2/conformance/utils/suite + +### Gateway API version and channel + +The certification is related to a specific API version and a specific channel, +therefore such information must be included in the final report. At test suite +setup time, the conformance profile machinery gets all the CRDs with the field +`.spec.group` equal to `gateway.networking.k8s.io`, and for each of them checks +the annotations `gateway.networking.k8s.io/bundle-version` and +`gateway.networking.k8s.io/channel`. If there are `CRD`s with different +versions, the certification fails specifying that it's not possible to run the +tests as there are different Gateway API versions installed in the cluster. If +there are CRDs with different channels, the certification fails specifying that +it's not possible to run the tests as there are different Gateway API channels +installed in the cluster. If all the Gateway API `CRD`s have the same version +and the same channel, the tests can be run and the detected version and channel +will be set in the `GatewayAPIVersion` and `gatewayAPIChannel` fields of the +final report. Furthermore, the suite must run all the experimental tests when +the channel is `experimental`, and the related features are enabled. + +In addition to the `CRD`s version, the suite needs to check its version in +relation to the `CRD`s one. To do so, a new `.go` file containing the current +Gateway API version is introduced in the project and compiled with the +conformance profile suite: + +```go +const GatewayAPIVersion = "0.7.0" +``` + +At test suite setup time the conformance profile suite checks the `CRD`s version +and the suite version; if the two versions differ, the certification fails. A +new generator will be introduced in the project to generate the aforementioned +`.go` file starting from a VERSION file contained in the root folder. Such a +VERSION file contains the semver of the latest release and is manually bumped at +release time. The script hack/verify-all.sh will be updated to ensure the +generated `.go` file is up to date with the VERSION file. + +### Certification + +Implementations will be able to report their conformance testing results using +our [reporting process](#reporting-process). Implementations will be able to +visibly demonstrate their conformance results on their downstream projects and +repositories using our [certification process](#certification-process). + +#### Reporting Process + +When conformance tests are executed an argument can be provided to the test +suite to emit `ConformanceReport` resource with the test results. This resource +can be configured to emit to the test output itself, or to a specific file. + +The following is an example report: + +```yaml +apiVersion: v1alpha1 +kind: ConformanceReport +implementation: + organization: acme + project: operator + url: https://acme.com + version: v1.0.0 + contact: + - @acme/maintainers +date: "2023-02-28 20:29:41+00:00" +gatewayAPIVersion: v0.7.0 +gatewayAPIChannel: standard +profiles: + - name: tcp + core: + result: success + summary: "all core functionality passed" + statistics: + passed: 4 + skipped: 0 + failed: 0 + extended: + result: skipped + summary: "no extended features supported" + statistics: + passed: 0 + skipped: 6 + failed: 0 + unsupportedFeatures: + - ExtendedFeature1 + - ExtendedFeature2 + - ExtendedFeature3 + - name: http + core: + result: success + summary: "all core functionality passed" + statistics: + passed: 20 + skipped: 0 + failed: 0 + extended: + result: success + summary: "all extended features supported" + statistics: + passed: 8 + skipped: 0 + failed: 0 + supportedFeatures: + - ExtendedFeature1 + - ExtendedFeature2 + - ExtendedFeature3 + - ExtendedFeature4 + - ExtendedFeature5 +``` + +> **WARNING**: It is an important clarification that this is NOT a full +> Kubernetes API. It uses `TypeMeta` for some fields that made sense to re-use +> and were familiar, but otherwise has it's own structure. It is not a [Custom +> Resource Definition (CRD)][crd] nor will it be made available along with our +> CRDs. It will be used only by conformance test tooling. + +> **NOTE**: The `implementation` field in the above example includes an +> `organization` and `project` field. Organizations can be an open source +> organization, an individual, a company, e.t.c.. Organizations can +> theoretically have multiple projects and should submit separate reports for +> each of them. + +> **NOTE**: The `contact` field indicates the Github usernames or team +> names of those who are responsible for maintaining this file, so they can be +> easily contacted when needed (e.g. for relevant release announcements +> regarding conformance, e.t.c.). Optionally, it can be an email address or +> a support URL (e.g. Github new issue page). + +The above report describes an implemenation that just released `v1` and has +`Core` support for `TCP` functionality and fully supports both `Core` and +`Extended` `HTTP` functionality. + +`ConformanceReports` can be stored as a list of reports in chronological order. +The following shows previous releases of the `acme`/`operator` implementation and +its feature progression: + +```yaml +apiVersion: v1alpha1 +kind: ConformanceReport +implementation: + organization: acme + project: operator + url: https://acme.com + version: v0.91.0 + contact: + - @acme/maintainers +date: "2022-09-28 20:29:41+00:00" +gatewayAPIVersion: v0.6.2 +gatewayAPIChannel: standard +profiles: + - name: tcp + core: + result: partial + summary: "some tests were manually skipped" + statistics: + passed: 2 + skipped: 2 + failed: 0 + skippedTests: + - TCPRouteBasics + - UDPRouteBasics + extended: + result: skipped + summary: "no extended features supported" + statistics: + passed: 0 + skipped: 4 + failed: 0 + unsupportedFeatures: + - ExtendedFeature1 + - ExtendedFeature2 + - ExtendedFeature3 + - name: http + core: + result: success + summary: "all core functionality passed" + statistics: + passed: 20 + skipped: 0 + failed: 0 + extended: + result: success + summary: "all extended features supported" + statistics: + passed: 5 + skipped: 3 + failed: 0 + supportedFeatures: + - ExtendedFeature1 + - ExtendedFeature2 + - ExtendedFeature3 + unsupportedFeatures: + - ExtendedFeature4 + - ExtendedFeature5 +--- +apiVersion: v1alpha1 +kind: ConformanceReport +implementation: + organization: acme + project: operator + url: https://acmeorg.com + version: v0.90.0 + contact: + - @acme/maintainers +date: "2022-08-28 20:29:41+00:00" +gatewayAPIVersion: v0.6.1 +gatewayAPIChannel: standard +profiles: + - name: tcp + core: + result: failed + summary: "all tests are failing" + statistics: + passed: 0 + skipped: 0 + failed: 4 + failedTests: + - TCPRouteExampleTest1 + - TCPRouteExampleTest2 + - TCPRouteExampleTest3 + - TCPRouteExampleTest4 + - name: http + core: + result: success + summary: "all core functionality passed" + statistics: + passed: 20 + skipped: 0 + failed: 0 + extended: + result: skipped + summary: "no extended features supported" + statistics: + passed: 2 + skipped: 6 + failed: 0 + supportedFeatures: + - ExtendedFeature1 + unsupportedFeatures: + - ExtendedFeature2 + - ExtendedFeature3 + - ExtendedFeature4 + - ExtendedFeature5 +--- +apiVersion: v1alpha1 +kind: ConformanceReport +implementation: + organization: acme + project: operator + url: https://acmeorg.com + version: v0.89.0 + contact: + - @acme/maintainers +date: "2022-07-28 20:29:41+00:00" +gatewayAPIVersion: v0.6.0 +gatewayAPIChannel: standard +profiles: + - name: http + core: + result: partial + summary: "some tests were skipped" + statistics: + passed: 16 + skipped: 2 + failed: 0 + skippedTests: + - HTTPRouteTestExample1 + - HTTPRouteTestExample2 + extended: + result: skipped + summary: "no extended features supported" + statistics: + passed: 0 + skipped: 8 + failed: 0 + unsupportedFeatures: + - ExtendedFeature1 + - ExtendedFeature2 + - ExtendedFeature3 + - ExtendedFeature4 + - ExtendedFeature5 +``` + +> **NOTE**: In the above you can see the `acme` implementation's progression. In +> their release `v0.89.0` they had started adding `HTTP` support and added the +> conformance tests to CI, but they were still skipping some core tests. In +> their next release `v0.90.0` they completed adding `HTTP` `Core` +> functionality (and even added one extended feature), and also starting adding +> `TCP` functionality during `v0.90.0` (but it was failing at that time). In +> `v0.91.0` they had completed core `HTTP` supported and added two more +> `Extended` features, and also started to get their `TCP` functionality to +> partially pass. + +Implementers can submit their reports upstream by creating a pull request to +the Gateway API repository and adding new reports to a file specific to their +implementation's project name: + +```console +conformance/results/-.yaml +``` + +Creating a pull request to add the `ConformanceReport` will start the +[certification process](#certification-process). + +> **NOTE**: No verification process (to prevent intentionally incorrect +> conformance results) will be implemented at this time. We expect that this wont +> be an issue in our community and even if someone were to try and "cheat" on +> the reporting the reputation loss for being caught would make them look very +> bad and would not be worth it. + +[crd]:https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/ + +#### Certification Process + +For this initial iteration the raw report data of the `ConformanceReports` will +live in its own directory and _is predominantly meant for machine consumption_. +Report data will be compiled into human-friendly displays during the an +automated certification process. + +Certification starts with the pull request described during the [reporting +process](#reporting-process). Once the `ConformanceReport` is created or +updated a display layer in the implementations page will need to be updated to +point to the new data. + +> **NOTE**: The `ConformanceReport` API will be defined in Golang like our +> other `apis/` so that we can utilize build tags from kubebuilder for defaults +> and validation, and so that there exists a common Golang type for it in the +> conformance test suite. When PRs are created the Gateway API repositories' +> CI will run linting and validation against the reports. + +CI will provide [badges][bdg] to contributors at the end of the process which +link to the implementations page for that specific implementation and can be +easily added via markdown to Git repositories. + +[impl]:https://gateway-api.sigs.k8s.io/implementations/ +[bdg]:https://shields.io + +## Alternatives Considered + +### Conformance Test Configuration File + +Conformance testing is currently done mainly through command line with +`go test` or via use of the conformance test suite as a library in Golang +projects. We considered whether adding the alternative to provide a +configuration file to the test suite would be nice, but we haven't heard +any specific demand for that from implementors yet so we're leaving the +idea here for later iterations to consider. + +### Conformance Test Container Image + +Providing a container image which could be fed deployment instructions for an +implementation was considered while working on this GET but it seemed like a +scope all unto itself so we're avoiding it here and perhaps a later iteration +could take a look at that if there are asks in the community. + +### Implementation-Specific Reporting + +Users have mentioned the desire to report on `implementation-specific` features +they support as a part of conformance. At the time of writing, there's not much +in the way of structure or testing for us to do this with but we remain open to +the idea. The door is left open in the `ConformanceReport` API for a future +iteration to add this if desired, but it probably warrants its own GEP as we +need to make sure we have buy-in from multiple stakeholders with different +implementations that are implementing those features. + +### High Level Profiles + +We originally started with two high level profiles: + +- `Layer4` +- `Layer7` + +However the overwhelming feedback from the community was to go a step down and +define profiles at the level of each individual API (e.g. `HTTPRoute`, +`TCPRoute`, `GRPCRoute`, e.t.c.). One of the main reasons for this was that we +already have multiple known implementations of Gateway API which only support +a single route type (`UDPRoute`, in particular as it turns out). + +We may consider in the future doing some of these higher level profiles if +there's a technical reason or strong desire from implementers. + +## Graduation Criteria + +The following are items that **MUST** be resolved to move this GEP to +`Standard` status (and before the end of the probationary period): + +- [ ] some kind of basic level of display for the report data needs to exist. + It's OK for a more robust display layer to be part of a follow-up effort. +- [ ] initially we were OK with storing reports in the Git repository as files. + While this is probably sufficient during the `Experimental` phase, we need to + re-evaluate this before `Standard` and see if this remains sufficient or if + we want to store the data elsewhere. +- [ ] We have been actively [gathering feedback from SIG + Arch][sig-arch-feedback]. Some time during the `experimental` phase needs to + be allowed to continue to engage with SIG Arch and incorporate their feedback + into the test suite. + +[sig-arch-feedback]:https://groups.google.com/g/kubernetes-sig-architecture/c/YjrVZ4NJQiA/m/7Qg7ScddBwAJ + +## References + +- https://github.com/kubernetes-sigs/gateway-api/issues/1709 +- https://github.com/kubernetes-sigs/gateway-api/issues/1329 + diff --git a/geps/gep-1742.md b/geps/gep-1742.md new file mode 100644 index 0000000000..c3b7ddb3c2 --- /dev/null +++ b/geps/gep-1742.md @@ -0,0 +1,498 @@ +# GEP-1742: HTTPRoute Timeouts + +* Issue: [#1742](https://github.com/kubernetes-sigs/gateway-api/issues/1742) +* Status: Implementable + +(See status definitions [here](overview.md#status).) + +## TLDR + +Create some sort of design so that Gateway API objects can be used to configure +timeouts for different types of connection. + +## Goals + +- Create some method to configure some timeouts. +- Timeout config must be applicable to most if not all Gateway API implementations. + +## Non-Goals + +- A standard API for every possible timeout that implementations may support. + +## Introduction + +In talking about Gateway API objects, particularly HTTPRoute, we've mentioned +timeout configuration many times in the past as "too hard" to find the common +ground necessary to make more generic configuration. This GEP intends firstly +to make this process less difficult, then to find common timeouts that we can +build into Gateway API. + +For this initial round, we'll focus on Layer 7 HTTP traffic, while acknowledging +that Layer 4 connections have their own interesting timeouts as well. + +The following sections will review all the implementations, then document what +timeouts are _available_ for the various data planes. + +### Background on implementations + +Most implementations that handle HTTPRoute objects use a proxy as the data plane +implementation, that actually forwards flows as directed by Gateway API configuration. + +The following table is a review of all the listed implementations of Gateway API +at the time of writing, with the data plane they use for Layer 7, based on what information +could be found online. If there are errors here, or if the implementation doesn't +support layer 7, please feel free to correct them. + +| Implementation | Data Plane | +|----------------|------------| +| Acnodal EPIC | Envoy | +| Apache APISIX | Nginx | +| BIG-IP Kubernetes Gateway| F5 BIG-IP | +| Cilium | Envoy | +| Contour | Envoy | +| Emissary Ingress| Envoy | +| Envoy Gateway | Envoy | +| Flomesh Service Mesh | Pipy | +| Gloo Edge | Envoy | +| Google Kubernetes Engine (GKE) | Similar to Envoy Timeouts | +| HAProxy Ingress | HAProxy | +| Hashicorp Consul | Envoy | +| Istio | Envoy | +| Kong | Nginx | +| Kuma | Envoy | +| Litespeed | Litespeed WebADC | +| Nginx Kubernetes Gateway | Nginx | +| Traefik | Traefik | + + +### Flow diagrams with available timeouts + +The following flow diagrams are based off the basic diagram below, with all the +timeouts I could find included. + +In general, timeouts are recorded with the setting name or similar that the data +plane uses for them, and are correct as far as I've parsed the documentation +correctly. + +Idle timeouts are marked as such. + +```mermaid +sequenceDiagram + participant C as Client + participant P as Proxy + participant U as Upstream + C->>P: Connection Started + C->>P: Starts sending Request + C->>P: Finishes Headers + C->>P: Finishes request + P->>U: Connection Started + P->>U: Starts sending Request + P->>U: Finishes request + P->>U: Finishes Headers + U->>P: Starts Response + U->>P: Finishes Headers + U->>P: Finishes Response + P->>C: Starts Response + P->>C: Finishes Headers + P->>C: Finishes Response + Note right of P: Repeat if connection sharing + U->>C: Connection ended +``` + +#### Envoy Timeouts + +For Envoy, some timeouts are configurable at either the HTTP Connection Manager +(very, very roughly equivalent to a Listener), the Route (equivalent to a HTTPRoute) +level, or the Cluster (usually close to the Service) or some combination. These +are noted in the below diagram with a `CM`, `R`, or `Cluster` prefix respectively. + +```mermaid +sequenceDiagram + participant C as Client + participant P as Envoy + participant U as Upstream + C->>P: Connection Started + activate P + Note left of P: transport_socket_connect_timeout for TLS + deactivate P + C->>P: Starts sending Request + activate C + activate P + activate P + C->>P: Finishes Headers + note left of P: CM request_headers_timeout + C->>P: Finishes request + deactivate P + activate U + note left of U: Cluster connect_timeout + deactivate U + P->>U: Connection Started + activate U + note right of U: CM idle_timeout
CM max_connection_duration + P->>U: Starts sending Request + P->>U: Finishes Headers + note left of P: CM request_timeout + P->>U: Finishes request + deactivate P + activate U + U->>P: Starts Response + U->>P: Finishes Headers + note right of U: R timeout
R per_try_timeout
R per_try_idle_timeout + U->>P: Finishes Response + deactivate U + P->>C: Starts Response + P->>C: Finishes Headers + P->>C: Finishes Response + Note left of C: CM stream_idle_timeout
R idle_timeout
CM,R max_stream_duration
TCP proxy idle_timeout
TCP protocol idle_timeout + deactivate C + Note right of P: Repeat if connection sharing + U->>C: Connection ended + deactivate U +``` + +#### Nginx timeouts + +Nginx allows setting of GRPC and general HTTP timeouts separately, although the +purposes seem to be roughly equivalent. + +```mermaid +sequenceDiagram + participant C as Client + participant P as Nginx + participant U as Upstream + C->>P: Connection Started + activate P + C->>P: Starts sending Request + C->>P: Finishes Headers + Note right of P: client_headers_timeout + deactivate P + activate P + C->>P: Finishes request + deactivate P + Note right of P: client_body_timeout + activate U + note left of U: proxy_connect_timeout
grpc_connect_timeout + deactivate U + P->>U: Connection Started + Activate U + Activate U + P->>U: Starts sending Request + P->>U: Finishes Headers + P->>U: Finishes request + Note right of U: (between write operations)
proxy_send_timeout
grpc_send_timeout + deactivate U + activate U + U->>P: Starts Response + U->>P: Finishes Headers + Note right of U: (between read operations)
proxy_read_timeout
grpc_read_timeout + U->>P: Finishes Response + deactivate U + activate P + P->>C: Starts Response + P->>C: Finishes Headers + P->>C: Finishes Response + deactivate P + Note left of P: send_timeout (only between two successive write operations) + Note left of C: Repeat if connection is shared until server's keepalive_timeout is hit + Note Right of U: upstream's keepalive_timeout (if keepalive enabled) + U->>C: Connection ended + deactivate U +``` + +#### HAProxy timeouts + +```mermaid +sequenceDiagram + participant C as Client + participant P as Proxy + participant U as Upstream + + C->>P: Connection Started + activate U + activate C + activate P + note left of P: timeout client (idle) + C->>P: Starts sending Request + C->>P: Finishes Headers + C->>P: Finishes request + note left of C: timeout http-request + deactivate C + activate C + note left of C: timeout client-fin + deactivate C + deactivate P + activate U + note left of U: timeout queue
(wait for available server) + deactivate U + + P->>U: Connection Started + activate U + P->>U: Starts sending Request + activate U + P->>U: Finishes Headers + P->>U: Finishes request + + note right of U: timeout connect + deactivate U + note left of U: timeout server
(idle timeout) + deactivate U + activate U + note left of U: timeout server-fin + deactivate U + U->>P: Starts Response + U->>P: Finishes Headers + U->>P: Finishes Response + P->>C: Starts Response + P->>C: Finishes Headers + P->>C: Finishes Response + activate C + note left of C: timeout http-keep-alive + deactivate C + Note right of P: Repeat if connection sharing + Note right of U: timeout tunnel
(for upgraded connections) + deactivate U + U->>C: Connection ended + +``` + +#### Traefik timeouts + +```mermaid +sequenceDiagram + participant C as Client + participant P as Proxy + participant U as Upstream + C->>P: Connection Started + activate U + C->>P: Starts sending Request + activate P + C->>P: Finishes Headers + Note right of P: respondingTimeouts
readTimeout + C->>P: Finishes request + deactivate P + P->>U: Connection Started + activate U + Note right of U: forwardingTimeouts
dialTimeout + deactivate U + P->>U: Starts sending Request + P->>U: Finishes request + P->>U: Finishes Headers + U->>P: Starts Response + activate U + note right of U: forwardingTimeouts
responseHeaderTimeout + U->>P: Finishes Headers + deactivate U + U->>P: Finishes Response + P->>C: Starts Response + activate P + P->>C: Finishes Headers + Note right of P: respondingTimeouts
writeTimeout + P->>C: Finishes Response + deactivate P + Note right of P: Repeat if connection sharing + Note right of U: respondingTimeouts
idleTimeout
Keepalive connections only + deactivate U + U->>C: Connection ended + +``` +#### F5 BIG-IP Timeouts + +Could not find any HTTP specific timeouts. PRs welcomed. 😊 + +#### Pipy Timeouts + +Could not find any HTTP specific timeouts. PRs welcomed. 😊 + +#### Litespeed WebADC Timeouts + +Could not find any HTTP specific timeouts. PRs welcomed. 😊 + +## API + +The above diagrams show that there are many different kinds of configurable timeouts +supported by Gateway implementations: connect, idle, request, upstream, downstream. +Although there may be opportunity for the specification of a common API for more of +them in the future, this GEP will focus on the L7 timeouts in HTTPRoutes that are +most valuable to clients. + +From the above analysis, it appears that most implementations are capable of +supporting the configuration of simple client downstream request timeouts on HTTPRoute +rules. This is a relatively small addition that would benefit many users. + +Some implementations support configuring a timeout for individual backend requests, +separate from the overall client request timeout. This is particularly useful if a +client HTTP request to a gateway can result in more than one call from the gateway +to the destination backend service, for example, if automatic retries are supported. +Adding support for this would also benefit many users. + +### Timeout values + +There are 2 kinds of timeouts that can be configured in an `HTTPRouteRule`: + +1. `timeouts.request` is the timeout for the Gateway API implementation to send a + response to a client HTTP request. Whether the gateway starts the timeout before + or after the entire client request stream has been received, is implementation dependent. + This field is optional `Extended` support. + +1. `timeouts.backendRequest` is a timeout for a single request from the gateway to a backend. + This field is optional `Extended` support. Typically used in conjuction with retry configuration, + if supported by an implementation. + Note that retry configuration will be the subject of a separate GEP (GEP-1731). + +```mermaid +sequenceDiagram + participant C as Client + participant P as Proxy + participant U as Upstream + C->>P: Connection Started + note left of P: timeouts.request start time (min) + C->>P: Starts sending Request + C->>P: Finishes Headers + C->>P: Finishes request + note left of P: timeouts.request start time (max) + P->>U: Connection Started + note right of P: timeouts.backendRequest start time + P->>U: Starts sending Request + P->>U: Finishes request + P->>U: Finishes Headers + U->>P: Starts Response + U->>P: Finishes Headers + note right of P: timeouts.backendRequest end time + note left of P: timeouts.request end time + U->>P: Finishes Response + note right of P: Repeat if retry + P->>C: Starts Response + P->>C: Finishes Headers + P->>C: Finishes Response + Note right of P: Repeat if connection sharing + U->>C: Connection ended +``` + +Both timeout fields are string duration values as specified by +[Golang time.ParseDuration](https://pkg.go.dev/time#ParseDuration) and MUST be >= 1ms +or 0 to disable (no timeout). + +### GO + +```go +type HTTPRouteRule struct { + // Timeouts defines the timeouts that can be configured for an HTTP request. + // + // Support: Extended + // + // +optional + // + Timeouts *HTTPRouteTimeouts `json:"timeouts,omitempty"` + + // ... +} + +// HTTPRouteTimeouts defines timeouts that can be configured for an HTTPRoute. +// Timeout values are formatted like 1h/1m/1s/1ms as parsed by Golang time.ParseDuration +// and MUST BE >= 1ms or 0 to disable (no timeout). +type HTTPRouteTimeouts struct { + // Request specifies the duration for processing an HTTP client request after which the + // gateway will time out if unable to send a response. + // + // For example, setting the `rules.timeouts.request` field to the value `10s` in an + // `HTTPRoute` will cause a timeout if a client request is taking longer than 10 seconds + // to complete. + // + // This timeout is intended to cover as close to the whole request-response transaction + // as possible although an implementation MAY choose to start the timeout after the entire + // request stream has been received instead of immediately after the transaction is + // initiated by the client. + // + // When this field is unspecified, request timeout behavior is implementation-dependent. + // + // Support: Extended + // + // +optional + // +kubebuilder:validation:Format=duration + Request *metav1.Duration `json:"request,omitempty"` + + // BackendRequest specifies a timeout for an individual request from the gateway + // to a backend service. This covers the time from when the request first starts being + // sent from the gateway to when the full response has been received from the backend. + // + // An entire client HTTP transaction with a gateway, covered by the Request timeout, + // may result in more than one call from the gateway to the destination backend service, + // for example, if automatic retries are supported. + // + // Because the Request timeout encompasses the BackendRequest timeout, + // the value of BackendRequest defaults to and must be <= the value of Request timeout. + // + // Support: Extended + // + // +optional + // +kubebuilder:validation:Format=duration + BackendRequest *metav1.Duration `json:"backendRequest,omitempty"` +} +``` + +### YAML + +```yaml +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: timeout-example +spec: + ... + rules: + - backendRefs: + - name: some-service + port: 8080 + timeouts: + request: 10s + backendRequest: 2s +``` + +### Conformance Details + +Gateway implementations can indicate support for the optional behavior in this GEP using +the following feature names: + +- `HTTPRouteRequestTimeout`: supports `rules.timeouts.request` in an `HTTPRoute`. +- `HTTPRouteBackendTimeout`: supports `rules.timeouts.backendRequest` in an `HTTPRoute`. + +## Alternatives + +Timeouts could be configured using policy attachments or in objects other than `HTTPRouteRule`. + +### Policy Attachment + +Instead of configuring timeouts directly on an API object, they could be configured using policy +attachments. The advantage to this approach would be that timeout policies can be not only +configured for an `HTTPRouteRule`, but can also be added/overriden at a more fine +(e.g., `HTTPBackendRef`) or course (e.g. `HTTPRoute`) level of granularity. + +The downside, however, is complexity introduced for the most common use case, adding a simple +timeout for an HTTP request. Setting a single field in the route rule, instead of needing to +create a policy resource, for this simple case seems much better. + +In the future, we could consider using policy attachments to configure less common kinds of +timeouts that may be needed, but it would probably be better to instead extend the proposed API +to support those timeouts as well. + +The default values of the proposed timeout fields could also be overriden +using policy attachments in the future. For example, a policy attachment could be used to set the +default value of `rules.timeouts.request` for all routes under an `HTTPRoute` or `Gateway`. + +### Other API Objects + +The new timeouts field could be added to a different API struct, instead of `HTTPRouteRule`. + +Putting it on an `HTTPBackendRef`, for example, would allow users to set different timeouts for different +backends. This is a feature that we believe has not been requested by existing proxy or service mesh +clients and is also not implementable using available timeouts of most proxies. + +Another alternative is to move the timeouts configuration up a level in the API to `HTTPRoute`. This +would be convenient when a user wants the same timeout on all rules, but would be overly restrictive. +Using policy attachments to override the default timeout value for all rules, as described in the +previous section, is likely a better way to handle timeout configuration above the route rule level. + +## References + +(Add any additional document links. Again, we should try to avoid +too much content not in version control to avoid broken links) diff --git a/geps/gep-1748.md b/geps/gep-1748.md new file mode 100644 index 0000000000..7123d62e8e --- /dev/null +++ b/geps/gep-1748.md @@ -0,0 +1,145 @@ +# GEP-1748: Gateway API Interaction with Multi-Cluster Services + +* Issue: [#1748](https://github.com/kubernetes-sigs/gateway-api/issues/1748) +* Status: Experimental + +> **Note**: This GEP is exempt from the [Probationary Period][expprob] rules of +> our GEP overview as it existed before those rules did, and so it has been +> explicitly grandfathered in. + +[expprob]:https://gateway-api.sigs.k8s.io/geps/overview/#probationary-period + +## TLDR + +The Kubernetes Multi-Cluster Services API enables Services to span multiple +clusters. Gateway API enables advanced traffic routing, serving as the next +generation Ingress, Load Balancing, and Mesh API. This document describes how +these APIs can be used together. + +## Goals + +* Define the interaction between Gateway API and Multi-Cluster Services +* Define any situations where Gateway API may span multiple clusters without the + Multi-Cluster Services API + +## Non-Goals + +* Make significant changes to either API + +## Introduction + +Gateway API and the Multi-Cluster Services API represent two of the newest +Kubernetes networking APIs. As they’ve been developed in parallel, there’s been +some cross-project discussion about how they can interact, but that has never +formally been written down. This GEP aims to change that. + +## Overview + +Multi-Cluster Services can be used within Gateway API wherever Services can be +used. The difference is that Services refer only to cluster-local endpoints while +Multi-Cluster Services can refer to endpoints throughout an entire ClusterSet. + +### ServiceImport as a Backend + +A Route can forward traffic to the endpoints attached to an imported Service. +This behaves identically to how forwarding to Services work in Kubernetes, with +the exception that the endpoints attached to a ServiceImport may span multiple +clusters. For example, the following HTTPRoute would forward traffic to +endpoints attached to the "store" ServiceImport: + +```yaml +{% include 'standard/multicluster/httproute-simple.yaml' %} +``` + +#### Routing to Specific Clusters + +In some cases, it may be helpful to route certain paths to a specific cluster +(or region). Similar to single-cluster Services, this can be accomplished by +creating multiple Multi-Cluster Services, one for each subset of endpoints you +would like to route to. For example, the following configuration will send +requests with paths prefixed with “/west” to the store-west ServiceImport, and +“/east” to the store-east ServiceImport. Requests that don’t match either of +these paths will be routed to the “store” ServiceImport which represents a +superset of the “store-west” and “store-east” ServiceImports. + +```yaml +{% include 'standard/multicluster/httproute-location.yaml' %} +``` + +#### Advanced Routing With ServiceImports + +All Routing capabilities in Gateway API should apply equally whether the backend +is a Service or ServiceImport. For example, when routing to a system with +multiple read replicas, it could be beneficial to route requests based on HTTP +Method. In the following example, requests with POST, PUT, and DELETE methods +are routed to `api-primary` while the rest are routed to `api-replicas`: + +```yaml +{% include 'standard/multicluster/httproute-method.yaml' %} +``` + +#### Routing to Both Services and ServiceImports + +There are some situations where it will be useful to split traffic between a +Service and ServiceImport. In the following example, 90% of traffic would go to +endpoints attached to the cluster-local "store" Service, and the remaining 10% +would go to endpoints attached to the Multi-Cluster "store-global" Service: + +```yaml +{% include 'standard/multicluster/httproute-hybrid.yaml' %} +``` + +#### Cross-Namespace References with ReferenceGrant + +It is possible to use ReferenceGrant to enable cross-namespace references to +ServiceImports. For example, the following HTTPRoute would forward traffic to +endpoints attached to the “bar” Multi-Cluster Service in the “bar” namespace: + +```yaml +{% include 'standard/multicluster/httproute-referencegrant.yaml' %} +``` + +### Mesh: ServiceImport as Parent + +In some cases, you may want to override traffic destined for a Multi-Cluster +Service with a mesh. As part of the broader GAMMA initiative, ServiceImport can +be used in the same way that Service is used as a ParentRef. When a Service is +specified as a parent, meshes will intercept traffic destined for the ClusterIP +of the Service and apply any policies or routing decisions defined by the Route. +Similarly, when a ServiceImport is specified as a parent, meshes will intercept +traffic destined for the ClusterSetIP and apply any policies or routing +decisions defined by the Route. In the following example, the mesh would +intercept traffic destined for the store ClusterSetIP matching the `/cart` path +and redirect it to the `cart` Multi-Cluster Service. + +```yaml +{% include 'standard/multicluster/httproute-gamma.yaml' %} +``` + +### Services vs ServiceImports + +It is important that all implementations provide a consistent experience. That +means that references to Services SHOULD always be interpreted as references to +endpoints within the same cluster for that Service. References to ServiceImports +MUST be interpreted as routing to Multi-Cluster endpoints across the ClusterSet +for the given ServiceImport. In practice, that means that users should use +“Service” when they want to reference cluster-local endpoints, and +“ServiceImport” when they want to route to Multi-Cluster endpoints across the +ClusterSet for the given ServiceImport. This behavior should be analogous to +using `.cluster.local` versus `.clusterset.local` DNS for a given Service. + +## API Changes + +* ServiceImport is recognized as backend with “Extended” conformance +* ServiceImport is included in GAMMA GEP(s) with “Extended” conformance +* Clarification that Services refer to endpoints within the same cluster + +## Alternatives + +### Develop Custom Multi-Cluster Concepts Independently from Multi-Cluster Services + +We could theoretically develop an entirely new way to handle multi-cluster routing. We’re choosing not to do that because the existing APIs are sound and can work well together. + +## References + +* [Original Doc](https://docs.google.com/document/d/1akwzBKtMKkkUV8tX-O7tPcI4BPMOLZ-gmS7Iz-7AOQE/edit#) diff --git a/geps/gep-1867.md b/geps/gep-1867.md new file mode 100644 index 0000000000..3326671220 --- /dev/null +++ b/geps/gep-1867.md @@ -0,0 +1,134 @@ +# GEP-1867: Per-Gateway Infrastructure + +* Status: Provisional +* Issue: [#1867](https://github.com/kubernetes-sigs/gateway-api/issues/1867) + +## Overview + +`Gateway`s represent a piece of infrastructure implemented by cloud load balancers, in-cluster deployments, or other mechanisms. +These often need vendor-specific configuration outside the scope of existing APIs (e.g. "size" or "version" of the infrastructure to provision). + +Today `GatewayClass.spec.parametersRef` is available to attach arbitrary configuration to a `GatewayClass`. + +This GEP will explain why that is not sufficient to meet common use cases, and introduce a new field - `infrastructure` - to address these cases. + +Related discussions: +* [Support cluster-local Gateways](https://github.com/kubernetes-sigs/gateway-api/discussions/1247) +* [Scaling Gateway Resources](https://github.com/kubernetes-sigs/gateway-api/discussions/1355) +* [Manual deployments](https://github.com/kubernetes-sigs/gateway-api/issues/1687) +* [Merging Gateways](https://github.com/kubernetes-sigs/gateway-api/pull/1863) +* [In Cluster Gateway Deployments](https://github.com/kubernetes-sigs/gateway-api/pull/1757) + +## Goals + +* Provide the ability to configure arbitrary (implementation specific) attributes about a **specific Gateway**. +* Provide the ability to configure a standardized set of attributes about a **specific Gateway**. + +## Why not GatewayClass parameters? + +`GatewayClass.spec.parametersRef` is the existing mechanism to configure arbitrary fields on a Gateway. +However, this introduces operational challenges when configuring Gateways. + +### Scope + +As a `Gateway` manager (with RBAC permissions to a specific `Gateway`) I should be able to declaratively make changes to that `Gateway` without the need for access to cluster-scoped resources (`GatewayClass`) and without affecting other `Gateways` managed by the same `GatewayClass`. +This has been previously discussed in [this issue](https://github.com/kubernetes-sigs/gateway-api/issues/567). + +As a cluster scoped resource, `GatewayClass` does not meet this requirement. +This restricts customization use cases to either a few pre-provisioned classes by the admin, or running in an environment where the "Infrastructure Provider" and "Cluster Operator" are the same roles. +The distinction between these roles is explicitly called out on the [homepage](https://gateway-api.sigs.k8s.io/#what-is-the-gateway-api). + +### Custom Resource + +`parametersRef` is entirely a generic implementation-specific meaning. +This means implementations will either need a custom CRD or use untyped resources like ConfigMap. +Neither of these have any consistency between implementations. +While there will always be some vendor-specific requirements, there are also a number of configuration aspects of a Gateway that are common between implementations. +However, these cannot currently be expressed in a vendor-neutral way. + +The original motivation behind `parametersRef` was for implementation specific concepts, while portable comments could be added into the API as first-class fields, but this has not been done (yet). + +Additionally, there is hesitancy to use a CRD (which leads to CRD proliferation), which pushes users towards untyped ConfigMaps which are not much better than annotations. +The scoping, as mentioned above, is also a bit awkward of a cluster scoped resource pointing to a namespaced object. + +### Separation of concerns + +While there is value out of providing class-wide options as defaults, there is also value in providing these options on the object (Gateway) directly. + +Some parallels in existing APIs: + +[Policy Attachment](https://gateway-api.sigs.k8s.io/references/policy-attachment) offers a hierarchy of defaults and overrides, allowing attachment to GatewayClass and Gateway. +This is similar to our needs here, but representing infrastructure configuration as a "Policy" is a bit problematic, and the existing mechanisms have no hierarchy. + +In core Kubernetes, Pods declare their requirements (for example, CPU requests) inline in the Pod resource; there is not a `ResourceClass` API that abstracts these further. +These higher level abstractions are handled by layered APIs (whether this is a CRD, an admission webhook, CI/CD tooling, etc). +This allows users the flexibility to easily configure things per-pod basis. +If the infrastructure admin wants to impose defaults or requirements on this flexibility, they are able to do so (in fact, `LimitRanger` provides a built in mechanism to do so). + +### Dynamic Changes + +Currently, the spec recommends `GatewayClass` to be used as a *template*. +Changes to it are not expected to change deployed `Gateway`s. + +This makes usage problematic in a declarative way. +For example, if I wanted to represent a `version` field and change that to trigger an upgrade, I would need to create an entirely new `Gateway`. + +## API + +In order to address the concerns above, I propose a standard `infrastructure` API is added to `Gateway` and `GatewayClass`. +Note the important part of this is the `Gateway` change; the `GatewayClass` aspect is mostly for consistency. + +The exact fields are out of scope for this GEP and will be handled by additional GEPs. +Some example GEPs already depending on this are [GEP-1713](/geps/gep-1713.md), [GEP-1651](/geps/gep-1651.md), and [GEP-1762](/geps/gep-1762.md) + +The fields as defined below are, of course, not useful. +This is intended as a basis for other PRs, not to provide value on its own. +This GEP will remain in provisional until at least one field is ready to be promoted. + +```go +type GatewaySpec struct { + // Infrastructure defines infrastructure level attributes about this Gateway instance. + Infrastructure GatewayInfrastructure `json:"infrastructure"` + // ... +} + +type GatewayClassSpec struct { + // Infrastructure defines infrastructure level attributes for all Gateways in this class. + // A Gateway may provide configuration for the same values; as all fields in GatewayInfrastructure are implementation specific, + // the merging logic between these is as well. However, the GatewayClass is generally expected to be providing defaults + // rather than overrides. + Infrastructure GatewayClassInfrastructure `json:"infrastructure"` + // ... +} + +type GatewayInfrastructure struct { + // ParametersRef provides a arbitrary implementation-specific configuration for + // fields not expressed directly in this struct. + // This follows the same semantics as GatewayClass's ParametersRef, but lives on the Gateway. + ParametersRef ParametersReference +} + +type GatewayClassInfrastructure struct { +} +``` + +### API Principles + +For any given field, we will need to make two decisions: +* whether this should be a first-class field or a generic `parametersRef`. +* whether this field should be configurable on a Gateway and/or GatewayClass level + +The choice to use an extension (`parametersRef`) or first-class field is a well known problem across the API, and the same logic will be used here. +Fields that are generally portable across implementations and have wide-spread demand and use cases will be promoted to first-class fields, +while vendor specific or niche fields will remain extensions. +Because infrastructure is somewhat inherently implementation specific, it is likely most fields will be Extended or ImplementationSpecific. +However, there are still a variety of concepts that have some meaning between implementations that can provide value to users. + +Introduction at Gateway or GatewayClass level will depend on the specific field and use cases for the field. +In general, it makes sense to provide defaults (GatewayClass) and specific settings (Gateway) for most fields, but +this will be evaluated on a case-by-case basis. + +### Status + +The API should likely expose some status. However, it is not yet clear what that will look like. +This will be addressed prior to promotion beyond "Provisional". diff --git a/geps/gep-1897.md b/geps/gep-1897.md new file mode 100644 index 0000000000..5d067f99a3 --- /dev/null +++ b/geps/gep-1897.md @@ -0,0 +1,199 @@ +# GEP-1897: TLS from Gateway to Backend for ingress + +* Issue: [#1897](https://github.com/kubernetes-sigs/gateway-api/issues/1897) +* Status: Provisional + +## TLDR + +This document specifically addresses the topic of conveying HTTPS from the Gateway +dataplane to the backend (backend TLS termination), and intends to satisfy the single +use case “As a client implementation of Gateway API, I need to know how to connect to +a backend pod that has its own certificate”. TLS configuration can be a nebulous topic, +so in order to drive resolution this GEP focuses only on this single piece of functionality. + +## Immediate Goals + +1. The solution must satisfy the following use case: the backend pod has its own +certificate and the gateway implementation client needs to know how to connect to the +backend pod. (Use case #4 in [Gateway API TLS Use Cases](#references)) +2. In terms of the Gateway API personas, only the application developer persona in this +solution. The application developer should control the gateway to backend TLS settings, +not the cluster operator, as requiring a cluster operator to manage certificate renewals +and revocations would be extremely cumbersome. +3. The solution should consider client certificate settings used in the TLS handshake **from +Gateway to backend**, such as TLS versions and cipher suites. (Use case #5 in [Gateway API TLS Use Cases](#references)) + +## Longer Term Goals + +These are worthy goals, but may need a different GEP for proper attention. + +1. TCPRoute use cases (completed by GA) +2. Mutual TLS use cases +3. Service mesh use cases + +## Non-Goals + +These are worthy goals, but will not be covered by this GEP. + +1. Changes to the existing mechanisms for edge or passthrough TLS termination +2. Providing a mechanism to decorate multiple route instances +3. TLSRoute use cases +4. UDPRoute use cases +5. Controlling certificates used by more than one workload (#6 in [Gateway API TLS Use Cases](#references)) +6. Client certificate settings used in TLS **from external clients to the +Listener** (#7 in [Gateway API TLS Use Cases](#references)) +7. Providing a mechanism for the cluster operator to override gateway to backend TLS settings. + +## Already Solved TLS Use Cases + +These are worthy goals that are already solved and thus will not be modified by the implementation. + +1. Termination of TLS for HTTP routing (#1 in [Gateway API TLS Use Cases](#references)) +2. HTTPS passthrough use cases (#2 in [Gateway API TLS Use Cases](#references)) +3. Termination of TLS for non-HTTP TCP streams (#3 in [Gateway API TLS Use Cases](#references)) + +## Overview - what do we want to do? + +Given that the current ingress solution specifies **edge** TLS termination (from the client to +the gateway), and how to handle **passthrough** TLS (from the client to the backend pod), this +proposed ingress solution specifies TLS origination to the **backend** (from the gateway to the +backend pod). As mentioned, this solution satisfies the use case in which the backend pod +has its own certificate and the gateway client needs to know how to connect to the backend pod. + +![image depicting TLS termination types](images/1897-TLStermtypes.png "TLS termination types") + +Gateway API is missing a mechanism for separately providing the details for the backend TLS handshake, +including: +* use of TLS +* destination CA (certificate authority) or CA bundle +* SANs for validating upstream service (server authentication) +* client certificate of the gateway (client authentication) + +## Purpose - why do we want to this? + +This proposal is _very_ tightly scoped because we have tried and failed to address this well-known +gap in the API specification. The lack of support for this fundamental concept is holding back +Gateway API adoption by users that require a solution to the use case. One of the recurring themes +that has held up the prior art has been interest related to service mesh and as such this proposal +focuses only on the ingress use case to avoid contention there. Another reason for the tight scope +is that we have been too focused on a generic representation of everything that TLS can do, which +covers too much ground to address in a single GEP. + +## The history of backend TLS + +Work on this topic has spanned over three years, as documented in our repositories and other references, +and summarized below. + +In January 2020, in issue [TLS Termination Policy #52](https://github.com/kubernetes-sigs/gateway-api/issues/52), +this use case was discussed. The discussion ended after being diverted by +[KEP: Adding AppProtocol to Services and Endpoints #1422](https://github.com/kubernetes/enhancements/pull/1422), +which was implemented and later reverted. + +In February 2020, [HTTPRoute: Add Reencrypt #81](https://github.com/kubernetes-sigs/gateway-api/pull/81) +added the dataplane feature as “reencrypt”, but it went stale and was closed in favor of the work done in the +next paragraph, which unfortunately didn’t implement the backend TLS termination feature. + +In August 2020, it resurfaced with a [comment](https://github.com/kubernetes-sigs/gateway-api/pull/256/files#r472734392) +on this pull request: [tls: introduce mode and sni to cert matching behavior](https://github.com/kubernetes-sigs/gateway-api/pull/256/files#top). +The backend TLS termination feature was deferred at that time. Other TLS discussion was documented in +[[SIG-NETWORK] TLS config in service-apis](https://docs.google.com/document/d/15fkzMrhN_7tA-i2mHKwZpqcjN1o2Pe9Am9Qt828x1lo/edit#heading=h.wym7wehwll44) +, a list of TLS features that had been collected in June 2020, itself based on spreadsheet +[Service API: TLS related issues](https://docs.google.com/spreadsheets/d/18KE61Y6InCmoQHZcbrYYRZS5Cnt7n33s5dTxUlhHgIA/edit#gid=0). + +In December 2021, this was discussed as a beta blocker in issue +[Docs mentions Reencrypt for HTTPRoute and TLSRoute is available #968](https://github.com/kubernetes-sigs/gateway-api/issues/968). + +A March 2022 issue documents another request for it: [Provide a way to configure TLS from a Gateway to Backends #1067](https://github.com/kubernetes-sigs/gateway-api/issues/1067) + +A June 2022 issue documents a documentation issue related to it: +[Unclear how to specify upstream (webserver) HTTP protocol #1244](https://github.com/kubernetes-sigs/gateway-api/discussions/1244) + +A July 2022 discussion [Specify Re-encrypt TLS Termination (i.e., Upstream TLS) #1285](https://github.com/kubernetes-sigs/gateway-api/discussions/1285) +collected most of the historical context preceding the backend TLS termination feature, with the intention of +collecting evidence that this feature is still unresolved. This was followed by +[GEP: Describe Backend Properties #1282](https://github.com/kubernetes-sigs/gateway-api/issues/1282). + +In August 2022, [Add Provisional GEP-1282 document #1333](https://github.com/kubernetes-sigs/gateway-api/pull/1333) +was created, and in October 2022, a GEP update with proposed implementation +[GEP-1282 Backend Properties - Update implementation #1430](https://github.com/kubernetes-sigs/gateway-api/pull/1430) +was followed by intense discussion and closed in favor of a downsize in scope. + +In January 2023 we closed GEP-1282 and began a new discussion on enumerating TLS use cases in +[Gateway API TLS Use Cases](#references), for the purposes of a clear definition and separation of concerns. +This GEP is the outcome of the TLS use cases #4 and #5 in +[Gateway API TLS Use Cases](#references) as mentioned in the Goals section above. + +## API + +Details deferred until we reach consensus on what we want to do, and why we want to do this. + +## Prior Art + +TLS from gateway to backend for ingress exists in several implementations, and was developed independently. + +### Istio Gateway supports this with a DestinationRule: + +* A secret representing a certificate/key pair, where the certificate is valid for the route host +* Set Gateway spec.servers[].port.protocol: HTTPS, spec.servers[].tls.mode=SIMPLE, spec.servers[].tls.credentialName +* Set DestinationRule spec.trafficPolicy.tls.mode: SIMPLE + +Ref: [Istio / Understanding TLS Configuration](https://istio.io/latest/docs/ops/configuration/traffic-management/tls-configuration/#gateways) +and [Istio / Destination Rule](https://istio.io/latest/docs/reference/config/networking/destination-rule/#ClientTLSSettings) + +### OpenShift Route (comparable to GW API Gateway) supports this with the following route configuration items: + +* A certificate/key pair, where the certificate is valid for the route host +* A separate destination CA certificate enables the Ingress Controller to trust the destination’s certificate +* An optional, separate CA certificate that completes the certificate chain + +Ref: [Secured routes - Configuring Routes | Networking | OpenShift Container Platform 4.12](https://docs.openshift.com/container-platform/4.12/networking/routes/secured-routes.html#nw-ingress-creating-a-reencrypt-route-with-a-custom-certificate_secured-routes) + +### Contour supports this from Envoy to the backend using: + +* An Envoy client certificate +* A CA certificate and SubjectName which are both used to verify the backend endpoint’s identity +* Kubernetes Service annotation: projectcontour.io/upstream-protocol.tls + +Ref: [Upstream TLS](https://projectcontour.io/docs/v1.21.1/config/upstream-tls/) + +### GKE supports a way to encrypt traffic to the backend pods using: + +* `AppProtocol` on Service set to HTTPS +* Load balancer does not verify the certificate used by backend pods + +Ref: [Secure a Gateway](https://cloud.google.com/kubernetes-engine/docs/how-to/secure-gateway#load-balancer-tls) + +### Emissary supports encrypted traffic to services + +* In the `Mapping` definition, set https:// in the spec.service field +* A spec.tls in the `Mapping` definition, with the name of a `TLSContext` +* A `TLSContext` to provide a client certificate, set minimum TLS version support, SNI + +Ref: [TLS Origination](https://www.getambassador.io/docs/emissary/latest/topics/running/tls/origination) + +### NGINX implementation through CRDs (Comparable to Route or Policy of Gateway API) supports both TLS and mTLS + +* In the Upstream section of a VirtualServer or VirtualServerRoute (equivalent to HTTPRoute) there is a simple toggle to enable TLS. This does not validate the certificate of the backend and implictly trusts the backend in order to form the SSL tunnel. This is not about validating the certificate but obfuscating the traffic with TLS/SSL. +* A Policy attachment can be provided when certification validation is required that is called egressMTLS (egress from the proxy to the upstream). This can be tuned to perform various certificate validation tests. It was created as a Policy becuase it implies some type of AuthN/AuthZ due to the additional checks. This was also compatible with Open Service Mesh and NGINX Service Mesh and removed the need for a sidecar at the ingress controller. +* A corresponding 'IngressMTLS' policy also exists for mTLS verification of client connections to the proxy. The Policy object is used for anything that implies AuthN/AuthZ. + +Ref: [Upstream.TLS](https://docs.nginx.com/nginx-ingress-controller/configuration/virtualserver-and-virtualserverroute-resources/#upstreamtls) +Ref: [EgressMTLS](https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/#egressmtls) +Ref: [IngressMTLS](https://docs.nginx.com/nginx-ingress-controller/configuration/policy-resource/#ingressmtls) + +## Open Questions (TODO) + +This section is to record issues that should be discussed in the implementation section before this GEP moves +out of `Provisional` status. + +1. Bowei recommended that we mention the approach of cross-namespace referencing between Route and Service. +Be explicit about using the standard rules with respect to attaching policies to resources. +2. Costin recommended that Gateway SHOULD authenticate with either a JWT with audience or client cert +or some other means - so gateway added headers can be trusted, etc. +3. Costin mentioned we need to answer the question - is configuring the connection to a backend and TLS +something the route author decides - or the backend owner? Same for SANs. However, providing a mechanism +for the cluster operator to override gateway to backend TLS settings is already listed as a Non-Goal. + +## References + +[Gateway API TLS Use Cases](https://docs.google.com/document/d/17sctu2uMJtHmJTGtBi_awGB0YzoCLodtR6rUNmKMCs8/edit#heading=h.cxuq8vo8pcxm) diff --git a/site-src/geps/gep-696.md b/geps/gep-696.md similarity index 74% rename from site-src/geps/gep-696.md rename to geps/gep-696.md index 06ba6b1439..e1fb1d2a24 100644 --- a/site-src/geps/gep-696.md +++ b/geps/gep-696.md @@ -27,6 +27,14 @@ the content into the GEP as online documents are easier to lose (... details, can point to PR with changes) +## Conformance Details + +(This section describes the names to be used for the feature or +features in conformance tests and profiles. + +These should be `CamelCase` names that specify the feature as +precisely as possible, and are particularly important for +Extended features, since they may be surfaced to users.) ## Alternatives diff --git a/site-src/geps/gep-709.md b/geps/gep-709.md similarity index 100% rename from site-src/geps/gep-709.md rename to geps/gep-709.md diff --git a/geps/gep-713.md b/geps/gep-713.md new file mode 100644 index 0000000000..520c18130b --- /dev/null +++ b/geps/gep-713.md @@ -0,0 +1,1142 @@ +# GEP-713: Metaresources and Policy Attachment_metaresource_ + +* Issue: [#713](https://github.com/kubernetes-sigs/gateway-api/issues/713) +* Status: Experimental + +> **Note**: This GEP is exempt from the [Probationary Period][expprob] rules of +> our GEP overview as it existed before those rules did, and so it has been +> explicitly grandfathered in. + +[expprob]:https://gateway-api.sigs.k8s.io/geps/overview/#probationary-period + +## TLDR + +This GEP aims to standardize terminology and processes around using one Kubernetes +object to modify the functions of one or more other objects. + +This GEP defines some terms, firstly: _Metaresource_. + +A Kubernetes object that that _augments_ the behavior of an object +in a standard way is called a _Metaresource_. + +This document proposes controlling the creation of configuration in the underlying +Gateway data plane using two types of Policy Attachment. +A "Policy Attachment" is a specific type of _metaresource_ that can affect specific +settings across either one object (this is "Direct Policy Attachment"), or objects +in a hierarchy (this is "Inherited Policy Attachment"). + +Individual policy APIs: +- must be their own CRDs (e.g. `TimeoutPolicy`, `RetryPolicy` etc), +- can be included in the Gateway API API group and installation or be defined by + implementations +- and must include a common `TargetRef` struct in their specification to identify + how and where to apply that policy. +- _may_ include either a `defaults` section, an `overrides` section, or both. If + these are included, the Policy is an Inherited Policy, and should use the + inheritance rules defined in this document. + +For Inherited Policies, this GEP also describes a set of expected behaviors +for how settings can flow across a defined hierarchy. + + +## Goals + +* Establish a pattern for Policy resources which will be used for any policies + included in the Gateway API spec +* Establish a pattern for Policy attachment, whether Direct or Inherited, + which must be used for any implementation specific policies used with + Gateway API resources +* Provide a way to distinguish between required and default values for all + policy API implementations +* Enable policy attachment at all relevant scopes, including Gateways, Routes, + Backends, along with how values should flow across a hierarchy if necessary +* Ensure the policy attachment specification is generic and forward thinking + enough that it could be easily adapted to other grouping mechanisms like + Namespaces in the future +* Provide a means of attachment that works for both ingress and mesh + implementations of this API +* Provide a consistent specification that will ensure familiarity between both + included and implementation-specific policies so they can both be interpreted + the same way. + +## Out of scope + +* Define all potential policies that may be attached to resources +* Design the full structure and configuration of policies + +## Background and concepts + +When designing Gateway API, one of the things we’ve found is that we often need to be +able change the behavior of objects without being able to make changes to the spec +of those objects. Sometimes, this is because we can’t change the spec of the object +to hold the information we need ( ReferenceGrant, from +[GEP-709](https://gateway-api.sigs.k8s.io/geps/gep-709/), affecting Secrets +and Services is an example, as is Direct Policy Attachment), and sometimes it’s +because we want the behavior change to flow across multiple objects +(this is what Inherited Policy Attachment is for). + +To put this another way, sometimes we need ways to be able to affect how an object +is interpreted in the API, without representing the description of those effects +inside the spec of the object. + +This document describes the ways we design objects to meet these two use cases, +and why you might choose one or the other. + +We use the term “metaresource” to describe the class of objects that _only_ augment +the behavior of another Kubernetes object, regardless of what they are targeting. + +“Meta” here is used in its Greek sense of “more comprehensive” +or “transcending”, and “resource” rather than “object” because “metaresource” +is more pronounceable than “metaobject”. Additionally, a single word is better +than a phrase like “wrapper object” or “wrapper resource” overall, although both +of those terms are effectively synonymous with “metaresource”. + +A "Policy Attachment" is a metaresource that affects the fields in existing objects +(like Gateway or Routes), or influences the configuration that's generated in an +underlying data plane. + +"Direct Policy Attachment" is when a Policy object references a single object _only_, +and only modifies the fields of or the configuration associated with that object. + +"Inherited Policy Attachment" is when a Policy object references a single object +_and any child objects of that object_ (according to some defined hierarchy), and +modifies fields of the child objects, or configuration associated with the child +objects. + +In either case, a Policy may either affect an object by controlling the value +of one of the existing _fields_ in the `spec` of an object, or it may add +additional fields that are _not_ in the `spec` of the object. + +### Direct Policy Attachment + +A Direct Policy Attachment is tightly bound to one instance of a particular +Kind within a single namespace (or to an instance of a single Kind at cluster scope), +and only modifies the behavior of the object that matches its binding. + +As an example, one use case that Gateway API currently does not support is how +to configure details of the TLS required to connect to a backend (in other words, +if the process running inside the backend workload expects TLS, not that some +automated infrastructure layer is provisioning TLS as in the Mesh case). + +A hypothetical TLSConnectionPolicy that targets a Service could be used for this, +using the functionality of the Service as describing a set of endpoints. (It +should also be noted this is not the only way to solve this problem, just an +example to illustrate Direct Policy Attachment.) + +The TLSConnectionPolicy would look something like this: + +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: TLSConnectionPolicy +metadata: + name: tlsport8443 + namespace: foo +spec: + targetRef: # This struct is defined as part of Gateway API + group: "" # Empty string means core - this is a standard convention + kind: Service + name: fooService + tls: + certificateAuthorityRefs: + - name: CAcert + port: 8443 + +``` + +All this does is tell an implementation, that for connecting to port `8443` on the +Service `fooService`, it should assume that the connection is TLS, and expect the +service's certificate to be validated by the chain in the `CAcert` Secret. + +Importantly, this would apply to _every_ usage of that Service across any HTTPRoutes +in that namespace, which could be useful for a Service that is reused in a lot of +HTTPRoutes. + +With these two examples in mind, here are some guidelines for when to consider +using Direct Policy Attachment: + +* The number or scope of objects to be modified is limited or singular. Direct + Policy Attachments must target one specific object. +* The modifications to be made to the objects don’t have any transitive information - + that is, the modifications only affect the single object that the targeted + metaresource is bound to, and don’t have ramifications that flow beyond that + object. +* In terms of status, it should be reasonably easy for a user to understand that + everything is working - basically, as long as the targeted object exists, and + the modifications are valid, the metaresource is valid, and this should be + straightforward to communicate in one or two Conditions. Note that at the time + of writing, this is *not* completed. +* Direct Policy Attachment _should_ only be used to target objects in the same + namespace as the Policy object. Allowing cross-namespace references brings in + significant security concerns, and/or difficulties about merging cross-namespace + policy objects. Notably, Mesh use cases may need to do something like this for + consumer policies, but in general, Policy objects that modify the behavior of + things outside their own namespace should be avoided unless it uses a handshake + of some sort, where the things outside the namespace can opt–out of the behavior. + (Notably, this is the design that we used for ReferenceGrant). + +### Inherited Policy Attachment: It's all about the defaults and overrides + +Because a Inherited Policy is a metaresource, it targets some other resource +and _augments_ its behavior. + +But why have this distinct from other types of metaresource? Because Inherited +Policy resources are designed to have a way for settings to flow down a hierarchy. + +Defaults set the default value for something, and can be overridden by the +“lower” objects (like a connection timeout default policy on a Gateway being +overridable inside a HTTPRoute), and Overrides cannot be overridden by “lower” +objects (like setting a maximum client timeout to some non-infinite value at the +Gateway level to stop HTTPRoute owners from leaking connections over time). + +Here are some guidelines for when to consider using a Inherited Policy object: + +* The settings or configuration are bound to one containing object, but affect + other objects attached to that one (for example, affecting HTTPRoutes attached + to a single Gateway, or all HTTPRoutes in a GatewayClass). +* The settings need to able to be defaulted, but can be overridden on a per-object + basis. +* The settings must be enforced by one persona, and not modifiable or removable + by a lesser-privileged persona. (The owner of a GatewayClass may want to restrict + something about all Gateways in a GatewayClass, regardless of who owns the Gateway, + or a Gateway owner may want to enforce some setting across all attached HTTPRoutes). +* In terms of status, a good accounting for how to record that the Policy is + attached is easy, but recording what resources the Policy is being applied to + is not, and needs to be carefully designed to avoid fanout apiserver load. + (This is not built at all in the current design either). + +When multiple Inherited Policies are used, they can interact in various ways, +which are governed by the following rules, which will be expanded on later in +in this document. + +* If a Policy does not affect an object's fields directly, then the resultant + Policy should be the set of all distinct fields inside the relevant Policy objects, + as set out by the rules below. +* For Policies that affect an object's existing fields, multiple instances of the + same Policy Kind affecting an object's fields will be evaluated as + though only a single Policy "wins" the right to affect each field. This operation + is performed on a _per-distinct-field_ basis. +* Settings in `overrides` stanzas will win over the same setting in a `defaults` + stanza. +* `overrides` settings operate in a "less specific beats more specific" fashion - + Policies attached _higher_ up the hierarchy will beat the same type of Policy + attached further down the hierarchy. +* `defaults` settings operate in a "more specific beats less specific" fashion - + Policies attached _lower down_ the hierarchy will beat the same type of Policy + attached further _up_ the hierarchy. +* For `defaults`, the _most specific_ value is the one _inside the object_ that + the Policy applies to; that is, if a Policy specifies a `default`, and an object + specifies a value, the _object's_ value will win. +* Policies interact with the fields they are controlling in a "replace value" + fashion. + * For fields where the `value` is a scalar, (like a string or a number) + should have their value _replaced_ by the value in the Policy if it wins. + Notably, this means that a `default` will only ever replace an empty or unset + value in an object. + * For fields where the value is an object, the Policy should include the fields + in the object in its definition, so that the replacement can be on simple fields + rather than complex ones. + * For fields where the final value is non-scalar, but is not an _object_ with + fields of its own, the value should be entirely replaced, _not_ merged. This + means that lists of strings or lists of ints specified in a Policy will overwrite + the empty list (in the case of a `default`) or any specified list (in the case + of an `override`). The same applies to `map[string]string` fields. An example + here would be a field that stores a map of annotations - specifying a Policy + that overrides annotations will mean that a final object specifying those + annotations will have its value _entirely replaced_ by an `override` setting. +* In the case that two Policies of the same type specify different fields, then + _all_ of the specified fields should take effect on the affected object. + +Examples to further illustrate these rules are given below. + +## API + +This approach is building on concepts from all of the alternatives discussed +below. This is very similar to the (now removed) BackendPolicy resource in the API, +but also borrows some concepts from the [ServicePolicy +proposal](https://github.com/kubernetes-sigs/gateway-api/issues/611). + +### Policy Attachment for Ingress +Attaching a Directly Attached Policy to Gateway resources for ingress use cases +is relatively straightforward. A policy can reference the resource it wants to +apply to. + +Access is granted with RBAC - anyone that has access to create a RetryPolicy in +a given namespace can attach it to any resource within that namespace. + +![Simple Ingress Example](images/713-ingress-simple.png) + +An Inherited Policy can attach to a parent resource, and then each policy +applies to the referenced resource and everything below it in terms of hierarchy. +Although this example is likely more complex than many real world +use cases, it helps demonstrate how policy attachment can work across +namespaces. + +![Complex Ingress Example](images/713-ingress-complex.png) + +### Policy Attachment for Mesh +Although there is a great deal of overlap between ingress and mesh use cases, +mesh enables more complex policy attachment scenarios. For example, you may want +to apply policy to requests from a specific namespace to a backend in another +namespace. + +![Simple Mesh Example](images/713-mesh-simple.png) + +Policy attachment can be quite simple with mesh. Policy can be applied to any +resource in any namespace but it can only apply to requests from the same +namespace if the target is in a different namespace. + +At the other extreme, policy can be used to apply to requests from a specific +workload to a backend in another namespace. A route can be used to intercept +these requests and split them between different backends (foo-a and foo-b in +this case). + +![Complex Mesh Example](images/713-mesh-complex.png) + +### Policy TargetRef API + +Each Policy resource MUST include a single `targetRef` field. It must not +target more than one resource at a time, but it can be used to target larger +resources such as Gateways or Namespaces that may apply to multiple child +resources. + +As with most APIs, there are countless ways we could choose to expand this in +the future. This includes supporting multiple targetRefs and/or label selectors. +Although this would enable compelling functionality, it would increase the +complexity of an already complex API and potentially result in more conflicts +between policies. Although we may choose to expand the targeting capabilities +in the future, at this point it is strongly preferred to start with a simpler +pattern that still leaves room for future expansion. + +The `targetRef` field MUST have the following structure: + +```go +// PolicyTargetReference identifies an API object to apply policy to. +type PolicyTargetReference struct { + // Group is the group of the target resource. + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Group string `json:"group"` + + // Kind is kind of the target resource. + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Kind string `json:"kind"` + + // Name is the name of the target resource. + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + Name string `json:"name"` + + // Namespace is the namespace of the referent. When unspecified, the local + // namespace is inferred. Even when policy targets a resource in a different + // namespace, it may only apply to traffic originating from the same + // namespace as the policy. + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + // +optional + Namespace string `json:"namespace,omitempty"` +} +``` + +### Sample Policy API +The following structure can be used as a starting point for any Policy resource +using this API pattern. Note that the PolicyTargetReference struct defined above +will be distributed as part of the Gateway API. + +```go +// ACMEServicePolicy provides a way to apply Service policy configuration with +// the ACME implementation of the Gateway API. +type ACMEServicePolicy struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata,omitempty"` + + // Spec defines the desired state of ACMEServicePolicy. + Spec ACMEServicePolicySpec `json:"spec"` + + // Status defines the current state of ACMEServicePolicy. + Status ACMEServicePolicyStatus `json:"status,omitempty"` +} + +// ACMEServicePolicySpec defines the desired state of ACMEServicePolicy. +type ACMEServicePolicySpec struct { + // TargetRef identifies an API object to apply policy to. + TargetRef gatewayv1a2.PolicyTargetReference `json:"targetRef"` + + // Override defines policy configuration that should override policy + // configuration attached below the targeted resource in the hierarchy. + // +optional + Override *ACMEPolicyConfig `json:"override,omitempty"` + + // Default defines default policy configuration for the targeted resource. + // +optional + Default *ACMEPolicyConfig `json:"default,omitempty"` +} + +// ACMEPolicyConfig contains ACME policy configuration. +type ACMEPolicyConfig struct { + // Add configurable policy here +} + +// ACMEServicePolicyStatus defines the observed state of ACMEServicePolicy. +type ACMEServicePolicyStatus struct { + // Conditions describe the current conditions of the ACMEServicePolicy. + // + // +optional + // +listType=map + // +listMapKey=type + // +kubebuilder:validation:MaxItems=8 + Conditions []metav1.Condition `json:"conditions,omitempty"` +} +``` + +### Hierarchy +Each policy MAY include default or override values. Default values are given +precedence from the bottom up, while override values are top down. That means +that a default attached to a Backend will have the highest precedence among +default values while an override value attached to a GatewayClass will have the +highest precedence overall. + +![Ingress and Sidecar Hierarchy](images/713-hierarchy.png) + +To illustrate this, consider 3 resources with the following hierarchy: +A > B > C. When attaching the concept of defaults and overrides to that, the +hierarchy would be expanded to this: + +A override > B override > C override > C default > B default > A default. + +Note that the hierarchy is reversed for defaults. The rationale here is that +overrides usually need to be enforced top down while defaults should apply to +the lowest resource first. For example, if an admin needs to attach required +policy, they can attach it as an override to a Gateway. That would have +precedence over Routes and Services below it. On the other hand, an app owner +may want to set a default timeout for their Service. That would have precedence +over defaults attached at higher levels such as Route or Gateway. + +If using defaults _and_ overrides, each policy resource MUST include 2 structs +within the spec. One with override values and the other with default values. + +In the following example, the policy attached to the Gateway requires cdn to +be enabled and provides some default configuration for that. The policy attached +to the Route changes the value for one of those fields (includeQueryString). + +```yaml +kind: CDNCachingPolicy # Example of implementation specific policy name +spec: + override: + cdn: + enabled: true + default: + cdn: + cachePolicy: + includeHost: true + includeProtocol: true + includeQueryString: true + targetRef: + kind: Gateway + name: example +--- +kind: CDNCachingPolicy +spec: + default: + cdn: + cachePolicy: + includeQueryString: false + targetRef: + type: direct + kind: HTTPRoute + name: example +``` + +In this final example, we can see how the override attached to the Gateway has +precedence over the default drainTimeout value attached to the Route. At the +same time, we can see that the default connectionTimeout attached to the Route +has precedence over the default attached to the Gateway. + +Also note how the different resources interact - fields that are not common across +objects _may_ both end up affecting the final object. + +![Inherited Policy Example](images/713-policy-hierarchy.png) + +#### Supported Resources +It is important to note that not every implementation will be able to support +policy attachment to each resource described in the hierarchy above. When that +is the case, implementations MUST clearly document which resources a policy may +be attached to. + +#### Attaching Policy to GatewayClass +GatewayClass may be the trickiest resource to attach policy to. Policy +attachment relies on the policy being defined within the same scope as the +target. This ensures that only users with write access to a policy resource in a +given scope will be able to modify policy at that level. Since GatewayClass is a +cluster scoped resource, this means that any policy attached to it must also be +cluster scoped. + +GatewayClass parameters provide an alternative to policy attachment that may be +easier for some implementations to support. These parameters can similarly be +used to set defaults and requirements for an entire GatewayClass. + +### Targeting External Services +In some cases (likely limited to mesh) we may want to apply policies to requests +to external services. To accomplish this, implementations can choose to support +a reference to a virtual resource type: + +```yaml +apiVersion: networking.acme.io/v1alpha1 +kind: RetryPolicy +metadata: + name: foo +spec: + default: + maxRetries: 5 + targetRef: + group: networking.acme.io + kind: ExternalService + name: foo.com +``` + +### Merging into existing `spec` fields + +It's possible (even likely) that configuration in a Policy may need to be merged +into an existing object's fields somehow, particularly for Inherited policies. + +When merging into an existing fields inside an object, Policy objects should +merge values at a scalar level, not at a struct or object level. + +For example, in the `CDNCachingPolicy` example above, the `cdn` struct contains +a `cachePolicy` struct that contains fields. If an implementation was merging +this configuration into an existing object that contained the same fields, it +should merge the fields at a scalar level, with the `includeHost`, +`includeProtocol`, and `includeQueryString` values being defaulted if they were +not specified in the object being controlled. Similarly, for `overrides`, the +values of the innermost scalar fields should overwrite the scalar fields in the +affected object. + +Implementations should not copy any structs from the Policy object directly into the +affected object, any fields that _are_ overridden should be overridden on a per-field +basis. + +In the case that the field in the Policy affects a struct that is a member of a list, +each existing item in the list in the affected object should have each of its +fields compared to the corresponding fields in the Policy. + +For non-scalar field _values_, like a list of strings, or a `map[string]string` +value, the _entire value_ must be overwritten by the value from the Policy. No +merging should take place. This mainly applies to `overrides`, since for +`defaults`, there should be no value present in a field on the final object. + +This table shows how this works for various types: + +|Type|Object config|Override Policy config|Result| +|----|-------------|----------------------|------| +|string| `key: "foo"` | `key: "bar"` | `key: "bar"` | +|list| `key: ["a","b"]` | `key: ["c","d"]` | `key: ["c","d"]` | +|`map[string]string`| `key: {"foo": "a", "bar": "b"}` | `key: {"foo": "c", "bar": "d"}` | `key: {"foo": "c", "bar": "d"}` | + + +### Conflict Resolution +It is possible for multiple policies to target the same object _and_ the same +fields inside that object. If multiple policy resources target +the same resource _and_ have an identical field specified with different values, +precedence MUST be determined in order of the following criteria, continuing on +ties: + +* Direct Policies override Inherited Policies. If preventing settings from + being overwritten is important, implementations should only use Inherited + Policies, and the `override` stanza that implies. Note also that it's not + intended that Direct and Inherited Policies should overlap, so this should + only come up in exceptional circumstances. +* Inside Inherited Policies, the same setting in `overrides` beats the one in + `defaults`. +* The oldest Policy based on creation timestamp. For example, a Policy with a + creation timestamp of "2021-07-15 01:02:03" is given precedence over a Policy + with a creation timestamp of "2021-07-15 01:02:04". +* The Policy appearing first in alphabetical order by `{namespace}/{name}`. For + example, foo/bar is given precedence over foo/baz. + +For a better user experience, a validating webhook can be implemented to prevent +these kinds of conflicts all together. + +### Kubectl Plugin +To help improve UX and standardization, a kubectl plugin will be developed that +will be capable of describing the computed sum of policy that applies to a given +resource, including policies applied to parent resources. + +Each Policy CRD that wants to be supported by this plugin will need to follow +the API structure defined above and add a `gateway.networking.k8s.io/policy: +true` label to the CRD. + +### Status + +In the current iteration of this GEP, metaresources and Policy objects don't +have any standard way to record what they're attaching to, or applying settings +to in the case of Policy Attachment. There are some recommended Condition types +defined below, but further work on the status design is required to ensure that +some problems are resolved: + +* When multiple controllers are implementing the same Route and recognize a + policy, it must be possible to determine which controller was + responsible for adding that policy reference to status. Adding Conditions to + status on the Policy instead can be helpful here, but we're still lacking a way + for the Route or Gateway owner to find _all_ the Policies that are influencing + their object. +* For this to be somewhat scalable, we must limit the number of status updates + that can result from a metaresource update. +* Since we only control some of the resources a policy might be attached to, + adding policies to status would only be possible on the policy objects themselves + or on Gateway API resources, not Services or other kinds of backends. + +Previous experience in the Kubernetes API has made it clear that having a single +object that can cause status updates to occur across many other objects can have +a big performance impact, so the status design must be very carefully done to +avoid these kind of fanout problems. + +However, the whole purpose of having a standardized Policy API structure and +patterns is intended to make this problem solvable both for human users and with +tooling. + +This is currently a _very_ open question. A discussion is ongoing at +[#1531](https://github.com/kubernetes-sigs/gateway-api/discussions/1531), and this +GEP will be updated with any outcomes. + +### Conditions +Controllers using the Gateway API policy attachment model SHOULD populate the +following condition and reasons on policy resources to provide a consistent +experience across implementations. + +```go +// PolicyConditionType is a type of condition for a policy. +type PolicyConditionType string + +// PolicyConditionReason is a reason for a policy condition. +type PolicyConditionReason string + +const ( + // PolicyConditionAccepted indicates whether the policy has been accepted or rejected + // by a targeted resource, and why. + // + // Possible reasons for this condition to be True are: + // + // * "Accepted" + // + // Possible reasons for this condition to be False are: + // + // * "Conflicted" + // * "Invalid" + // * "TargetNotFound" + // + PolicyConditionAccepted PolicyConditionType = "Accepted" + + // PolicyReasonAccepted is used with the "Accepted" condition when the policy has been + // accepted by the targeted resource. + PolicyReasonAccepted PolicyConditionReason = "Accepted" + + // PolicyReasonConflicted is used with the "Accepted" condition when the policy has not + // been accepted by a targeted resource because there is another policy that targets the same + // resource and a merge is not possible. + PolicyReasonConflicted PolicyConditionReason = "Conflicted" + + // PolicyReasonInvalid is used with the "Accepted" condition when the policy is syntactically + // or semantically invalid. + PolicyReasonInvalid PolicyConditionReason = "Invalid" + + // PolicyReasonTargetNotFound is used with the "Accepted" condition when the policy is attached to + // an invalid target resource + PolicyReasonTargetNotFound PolicyConditionReason = "TargetNotFound" +) +``` + +### Interaction with Custom Filters and other extension points +There are multiple methods of custom extension in the Gateway API. Policy +attachment and custom Route filters are two of these. Policy attachment is +designed to provide arbitrary configuration fields that decorate Gateway API +resources. Route filters provide custom request/response filters embedded inside +Route resources. Both are extension methods for fields that cannot easily be +standardized as core or extended fields of the Gateway API. The following +guidance should be considered when introducing a custom field into any Gateway +controller implementation: + +1. For any given field that a Gateway controller implementation needs, the + possibility of using core or extended should always be considered before + using custom policy resources. This is encouraged to promote standardization + and, over time, to absorb capabilities into the API as first class fields, + which offer a more streamlined UX than custom policy attachment. + +2. Although it's possible that arbitrary fields could be supported by custom + policy, custom route filters, and core/extended fields concurrently, it is + recommended that implementations only use multiple mechanisms for + representing the same fields when those fields really _need_ the defaulting + and/or overriding behavior that Policy Attachment provides. For example, a + custom filter that allowed the configuration of Authentication inside a + HTTPRoute object might also have an associated Policy resource that allowed + the filter's settings to be defaulted or overridden. It should be noted that + doing this in the absence of a solution to the status problem is likely to + be *very* difficult to troubleshoot. + +### Conformance Level +This policy attachment pattern is associated with an "EXTENDED" conformance +level. The implementations that support this policy attachment model will have +the same behavior and semantics, although they may not be able to support +attachment of all types of policy at all potential attachment points. + +### Apply Policies to Sections of a Resource (Future Extension) +Although initially out of scope, it would be helpful to be able to target +specific matches within nested objects. For example, it may be useful to attach +policies to a specific Gateway listener or Route rule. This section explores +what that could look like. + +Each Route rule or Gateway listener should be expanded with an optional name +field. The target ref would be expanded with an optional sectionName field that +could be used to refer to that specific section of the resource. It would refer +to the following concepts on these resources: + +* Gateway.Listeners.Name +* xRoute.Rules.Name +* Service.Ports.Name + +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: HTTPRoute +metadata: + name: http-app-1 + labels: + app: foo +spec: + hostnames: + - "foo.com" + rules: + - name: bar + matches: + - path: + type: Prefix + value: /bar + forwardTo: + - serviceName: my-service1 + port: 8080 +--- +apiVersion: networking.acme.io/v1alpha2 +kind: RetryPolicy +metadata: + name: foo +spec: + maxRetries: 5 + targetRef: + name: foo + group: gateway.networking.k8s.io + kind: HTTPRoute + sectionName: bar +``` + +This would require adding a `SectionName` field to the PolicyTargetReference: +```go +type PolicyTargetReference struct { + // SectionName is the name of a section within the target resource. When + // unspecified, this targets the entire resource. In the following + // resources, SectionName is interpreted as the following: + // * Gateway: Listener Name + // * Route: Rule Name + // * Service: Port Name + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + // +optional + SectionName string `json:"sectionName,omitempty"` + // ... +} +``` + +This would also require adding a `Name` field to Gateway listeners and Route +rules: + +```go +type Listener struct { + // Name is the name of the Listener. If more than one Listener is present + // each Listener MUST specify a name. The names of Listeners MUST be unique + // within a Gateway. + // + // Support: Core + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + // +optional + Name string `json:"name,omitempty"` + // ... +} +``` + +```go +type RouteRule struct { + // Name is the name of the Route rule. If more than one Route Rule is + // present, each Rule MUST specify a name. The names of Rules MUST be unique + // within a Route. + // + // Support: Core + // + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + // +optional + Name string `json:"name,omitempty"` + // ... +} +``` + +### Advantages +* Incredibly flexible approach that should work well for both ingress and mesh +* Conceptually similar to existing ServicePolicy proposal and BackendPolicy + pattern +* Easy to attach policy to resources we don’t control (Service, ServiceImport, + etc) +* Minimal API changes required +* Simplifies packaging an application for deployment as policy references do not + need to be part of the templating + +### Disadvantages +* May be difficult to understand which policies apply to a request + +## Examples + +This section provides some examples of various types of Policy objects, and how +merging, `defaults`, `overrides`, and other interactions work. + +### Direct Policy Attachment + +The following Policy sets the minimum TLS version required on a Gateway Listener: +```yaml +apiVersion: networking.example.io/v1alpha1 +kind: TLSMinimumVersionPolicy +metadata: + name: minimum12 + namespace: appns +spec: + minimumTLSVersion: 1.2 + targetRef: + name: internet + group: gateway.networking.k8s.io + kind: Gateway +``` + +Note that because there is no version controlling the minimum TLS version in the +Gateway `spec`, this is an example of a non-field Policy. + +### Inherited Policy Attachment + +It also could be useful to be able to _default_ the `minimumTLSVersion` setting +across multiple Gateways. + +This version of the above Policy allows this: +```yaml +apiVersion: networking.example.io/v1alpha1 +kind: TLSMinimumVersionPolicy +metadata: + name: minimum12 + namespace: appns +spec: + defaults: + minimumTLSVersion: 1.2 + targetRef: + name: appns + group: "" + kind: namespace +``` + +This Inherited Policy is using the implicit hierarchy that all resources belong +to a namespace, so attaching a Policy to a namespace means affecting all possible +resources in a namespace. Multiple hierarchies are possible, even within Gateway +API, for example Gateway -> Route, Gateway -> Route -> Backend, Gateway -> Route +-> Service. GAMMA Policies could conceivably use a hierarchy of Service -> Route +as well. + +Note that this will not be very discoverable for Gateway owners in the absence of +a solution to the Policy status problem. This is being worked on and this GEP will +be updated once we have a design. + +Conceivably, a security or admin team may want to _force_ Gateways to have at least +a minimum TLS version of `1.2` - that would be a job for `overrides`, like so: + +```yaml +apiVersion: networking.example.io/v1alpha1 +kind: TLSMinimumVersionPolicy +metadata: + name: minimum12 + namespace: appns +spec: + overrides: + minimumTLSVersion: 1.2 + targetRef: + name: appns + group: "" + kind: namespace +``` + +This will make it so that _all Gateways_ in the `default` namespace _must_ use +a minimum TLS version of `1.2`, and this _cannot_ be changed by Gateway owners. +Only the Policy owner can change this Policy. + +### Handling non-scalar values + +In this example, we will assume that at some future point, HTTPRoute has grown +fields to configure retries, including a field called `retryOn` that reflects +the HTTP status codes that should be retried. The _value_ of this field is a +list of strings, being the HTTP codes that must be retried. The `retryOn` field +has no defaults in the field definitions (which is probably a bad design, but we +need to show this interaction somehow!) + +We also assume that a Inherited `RetryOnPolicy` exists that allows both +defaulting and overriding of the `retryOn` field. + +A full `RetryOnPolicy` to default the field to the codes `501`, `502`, and `503` +would look like this: +```yaml +apiVersion: networking.example.io/v1alpha1 +kind: RetryOnPolicy +metadata: + name: retryon5xx + namespace: appns +spec: + defaults: + retryOn: + - "501" + - "502" + - "503" + targetRef: + kind: Gateway + group: gateway.networking.k8s.io + name: we-love-retries +``` + +This means that, for HTTPRoutes that do _NOT_ explicitly set this field to something +else, (in other words, they contain an empty list), then the field will be set to +a list containing `501`, `502`, and `503`. (Notably, because of Go zero values, this +would also occur if the user explicitly set the value to the empty list.) + +However, if a HTTPRoute owner sets any value other than the empty list, then that +value will remain, and the Policy will have _no effect_. These values are _not_ +merged. + +If the Policy used `overrides` instead: +```yaml +apiVersion: networking.example.io/v1alpha1 +kind: RetryOnPolicy +metadata: + name: retryon5xx + namespace: appns +spec: + overrides: + retryOn: + - "501" + - "502" + - "503" + targetRef: + kind: Gateway + group: gateway.networking.k8s.io + name: you-must-retry +``` + +Then no matter what the value is in the HTTPRoute, it will be set to `501`, `502`, +`503` by the Policy override. + +### Interactions between defaults, overrides, and field values + +All HTTPRoutes that attach to the `YouMustRetry` Gateway will have any value +_overwritten_ by this policy. The empty list, or any number of values, will all +be replaced with `501`, `502`, and `503`. + +Now, let's also assume that we use the Namespace -> Gateway hierarchy on top of +the Gateway -> HTTPRoute hierarchy, and allow attaching a `RetryOnPolicy` to a +_namespace_. The expectation here is that this will affect all Gateways in a namespace +and all HTTPRoutes that attach to those Gateways. (Note that the HTTPRoutes +themselves may not necessarily be in the same namespace though.) + +If we apply the default policy from earlier to the namespace: +```yaml +apiVersion: networking.example.io/v1alpha1 +kind: RetryOnPolicy +metadata: + name: retryon5xx + namespace: appns +spec: + defaults: + retryOn: + - "501" + - "502" + - "503" + targetRef: + kind: Namespace + group: "" + name: appns +``` + +Then this will have the same effect as applying that Policy to every Gateway in +the `default` namespace - namely that every HTTPRoute that attaches to every +Gateway will have its `retryOn` field set to `501`, `502`, `503`, _if_ there is no +other setting in the HTTPRoute itself. + +With two layers in the hierarchy, we have a more complicated set of interactions +possible. + +Let's look at some tables for a particular HTTPRoute, assuming that it does _not_ +configure the `retryOn` field, for various types of Policy at different levels. + +#### Overrides interacting with defaults for RetryOnPolicy, empty list in HTTPRoute + +||None|Namespace override|Gateway override|HTTPRoute override| +|----|-----|-----|----|----| +|No default|Empty list|Namespace override| Gateway override Policy| HTTPRoute override| +|Namespace default| Namespace default| Namespace override | Gateway override | HTTPRoute override | +|Gateway default| Gateway default | Namespace override | Gateway override | HTTPRoute override | +|HTTPRoute default| HTTPRoute default | Namespace override | Gateway override | HTTPRoute override| + +#### Overrides interacting with other overrides for RetryOnPolicy, empty list in HTTPRoute +||No override|Namespace override A|Gateway override A|HTTPRoute override A| +|----|-----|-----|----|----| +|No override|Empty list|Namespace override| Gateway override| HTTPRoute override| +|Namespace override B| Namespace override B| Namespace override
first created wins
otherwise first alphabetically | Namespace override B | Namespace override B| +|Gateway override B| Gateway override B | Namespace override A| Gateway override
first created wins
otherwise first alphabetically | Gateway override B| +|HTTPRoute override B| HTTPRoute override B | Namespace override A| Gateway override A| HTTPRoute override
first created wins
otherwise first alphabetically| + +#### Defaults interacting with other defaults for RetryOnPolicy, empty list in HTTPRoute +||No default|Namespace default A|Gateway default A|HTTPRoute default A| +|----|-----|-----|----|----| +|No default|Empty list|Namespace default| Gateway default| HTTPRoute default A| +|Namespace default B| Namespace default B| Namespace default
first created wins
otherwise first alphabetically | Gateway default A | HTTPRoute default A| +|Gateway default B| Gateway default B| Gateway default B| Gateway default
first created wins
otherwise first alphabetically | HTTPRoute default A| +|HTTPRoute default B| HTTPRoute default B| HTTPRoute default B| HTTPRoute default B| HTTPRoute default
first created wins
otherwise first alphabetically| + + +Now, if the HTTPRoute _does_ specify a RetryPolicy, +it's a bit easier, because we can basically disregard all defaults: + +#### Overrides interacting with defaults for RetryOnPolicy, value in HTTPRoute + +||None|Namespace override|Gateway override|HTTPRoute override| +|----|-----|-----|----|----| +|No default| Value in HTTPRoute|Namespace override| Gateway override | HTTPRoute override| +|Namespace default| Value in HTTPRoute| Namespace override | Gateway override | HTTPRoute override | +|Gateway default| Value in HTTPRoute | Namespace override | Gateway override | HTTPRoute override | +|HTTPRoute default| Value in HTTPRoute | Namespace override | Gateway override | HTTPRoute override| + +#### Overrides interacting with other overrides for RetryOnPolicy, value in HTTPRoute +||No override|Namespace override A|Gateway override A|HTTPRoute override A| +|----|-----|-----|----|----| +|No override|Value in HTTPRoute|Namespace override A| Gateway override A| HTTPRoute override A| +|Namespace override B| Namespace override B| Namespace override
first created wins
otherwise first alphabetically | Namespace override B| Namespace override B| +|Gateway override B| Gateway override B| Namespace override A| Gateway override
first created wins
otherwise first alphabetically | Gateway override B| +|HTTPRoute override B| HTTPRoute override B | Namespace override A| Gateway override A| HTTPRoute override
first created wins
otherwise first alphabetically| + +#### Defaults interacting with other defaults for RetryOnPolicy, value in HTTPRoute +||No default|Namespace default A|Gateway default A|HTTPRoute default A| +|----|-----|-----|----|----| +|No default|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute| +|Namespace default B|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute| +|Gateway default B|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute| +|HTTPRoute default B|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute|Value in HTTPRoute| + + +## Removing BackendPolicy +BackendPolicy represented the initial attempt to cover policy attachment for +Gateway API. Although this proposal ended up with a similar structure to +BackendPolicy, it is not clear that we ever found sufficient value or use cases +for BackendPolicy. Given that this proposal provides more powerful ways to +attach policy, BackendPolicy was removed. + +## Alternatives + +### 1. ServiceBinding for attaching Policies and Routes for Mesh +A new ServiceBinding resource has been proposed for mesh use cases. This would +provide a way to attach policies, including Routes to a Service. + +Most notably, these provide a way to attach different policies to requests +coming from namespaces or specific Gateways. In the example below, a +ServiceBinding in the consumer namespace would be applied to the selected +Gateway and affect all requests from that Gateway to the foo Service. Beyond +policy attachment, this would also support attaching Routes as policies, in this +case the attached HTTPRoute would split requests between the foo-a and foo-b +Service instead of the foo Service. + +![Simple Service Binding Example](images/713-servicebinding-simple.png) + +This approach can be used to attach a default set of policies to all requests +coming from a namespace. The example below shows a ServiceBinding defined in the +producer namespace that would apply to all requests from within the same +namespace or from other namespaces that did not have their own ServiceBindings +defined. + +![Complex Service Binding Example](images/713-servicebinding-complex.png) + +#### Advantages +* Works well for mesh and any use cases where requests don’t always transit + through Gateways and Routes. +* Allows policies to apply to an entire namespace. +* Provides very clear attachment of polices, routes, and more to a specific + Service. +* Works well for ‘shrink-wrap application developers’ - the packaged app does + not need to know about hostnames or policies or have extensive templates. +* Works well for ‘dynamic’ / programmatic creation of workloads ( Pods,etc - see + CertManager) +* It is easy to understand what policy applies to a workload - by listing the + bindings in the namespace. + +#### Disadvantages +* Unclear how this would work with an ingress model. If Gateways, Routes, and + Backends are all in different namespaces, and each of those namespaces has + different ServiceBindings applying different sets of policies, it’s difficult + to understand which policy would be applied. +* Unclear if/how this would interact with existing the ingress focused policy + proposal described below. If both coexisted, would it be possible for a user + to understand which policies were being applied to their requests? +* Route status could get confusing when Routes were referenced as a policy by + ServiceBinding +* Introduces a new mesh specific resource. + +### 2. Attaching Policies for Ingress +An earlier proposal for policy attachment in the Gateway API suggested adding +policy references to each Resource. This works very naturally for Ingress use +cases where all requests follow a path through Gateways, Routes, and Backends. +Adding policy attachment at each level enables different roles to define +defaults and allow overrides at different levels. + +![Simple Ingress Attachment Example](images/713-ingress-attachment.png) + +#### Advantages +* Consistent policy attachment at each level +* Clear which policies apply to each component +* Naturally translates to hierarchical Ingress model with ability to delegate + policy decisions to different roles + +#### Disadvantages +* Policy overrides could become complicated +* At least initially, policy attachment on Service would have to rely on Service + annotations or references from policy to Service(s) +* No way to attach policy to other resources such as namespace or ServiceImport +* May be difficult to modify Routes and Services if other components/roles are + managing them (eg Knative) + +### 3. Shared Policy Resource +This is really just a slight variation or extension of the main proposal in this +GEP. We would introduce a shared policy resource. This resource would follow the +guidelines described above, including the `targetRef` as defined as well as +`default` and `override` fields. Instead of carefully crafted CRD schemas for +each of the `default` and `override` fields, we would use more generic +`map[string]string` values. This would allow similar flexibility to annotations +while still enabling the default and override concepts that are key to this +proposal. + +Unfortunately this would be difficult to validate and would come with many of +the downsides of annotations. A validating webhook would be required for any +validation which could result in just as much or more work to maintain than +CRDs. At this point we believe that the best experience will be from +implementations providing their own policy CRDs that follow the patterns +described in this GEP. We may want to explore tooling or guidance to simplify +the creation of these policy CRDs to help simplify implementation and extension +of this API. + +## References + +**Issues** +* [Extensible Service Policy and Configuration](https://github.com/kubernetes-sigs/gateway-api/issues/611) + +**Docs** +* [Policy Attachment and Binding](https://docs.google.com/document/d/13fyptUtO9NV_ZAgkoJlfukcBf2PVGhsKWG37yLkppJo/edit?resourcekey=0-Urhtj9gBkGBkSL1gHgbWKw) diff --git a/site-src/geps/gep-718.md b/geps/gep-718.md similarity index 100% rename from site-src/geps/gep-718.md rename to geps/gep-718.md diff --git a/site-src/geps/gep-724.md b/geps/gep-724.md similarity index 100% rename from site-src/geps/gep-724.md rename to geps/gep-724.md diff --git a/site-src/geps/gep-726.md b/geps/gep-726.md similarity index 99% rename from site-src/geps/gep-726.md rename to geps/gep-726.md index ef7406dd0b..7e4a04bd4b 100644 --- a/site-src/geps/gep-726.md +++ b/geps/gep-726.md @@ -1,7 +1,7 @@ # GEP-726: Add Path Redirects and Rewrites * Issue: [#726](https://github.com/kubernetes-sigs/gateway-api/issues/726) -* Status: Experimental +* Status: Standard ## TLDR diff --git a/site-src/geps/gep-735.md b/geps/gep-735.md similarity index 100% rename from site-src/geps/gep-735.md rename to geps/gep-735.md diff --git a/site-src/geps/gep-746.md b/geps/gep-746.md similarity index 100% rename from site-src/geps/gep-746.md rename to geps/gep-746.md diff --git a/site-src/geps/gep-820.md b/geps/gep-820.md similarity index 100% rename from site-src/geps/gep-820.md rename to geps/gep-820.md diff --git a/site-src/geps/gep-851.md b/geps/gep-851.md similarity index 100% rename from site-src/geps/gep-851.md rename to geps/gep-851.md diff --git a/geps/gep-91.md b/geps/gep-91.md new file mode 100644 index 0000000000..13fee63e4a --- /dev/null +++ b/geps/gep-91.md @@ -0,0 +1,22 @@ +# GEP-91: Client Certificate Validation for TLS terminating at the Gateway Listener + +* Issue: [#91](https://github.com/kubernetes-sigs/gateway-api/issues/91) +* Status: Provisional + +(See definitions in [GEP Status][/contributing/gep#status].) + +## TLDR + +This GEP proposes a way to validate the TLS certificate presented by the downstream client to the server +(Gateway Listener in this case) during a [TLS Handshake Protocol][], also commonly referred to as mutual TLS (mTLS). + +## Goals +- Define an API field to specify the CA Certificate within the Gateway Listener configuration that can be used as a trusted anchor to validate the certificates presented by the client. + +## Non-Goals +- Define other fields that can be used to verify the client certificate such as the Cerificate Hash or Subject Alt Name. + +## References + +[TLS Handshake Protocol]: https://www.rfc-editor.org/rfc/rfc5246#section-7.4 +[Certificate Path Validation]: https://www.rfc-editor.org/rfc/rfc5280#section-6 diff --git a/site-src/geps/gep-917.md b/geps/gep-917.md similarity index 100% rename from site-src/geps/gep-917.md rename to geps/gep-917.md diff --git a/site-src/geps/gep-922.md b/geps/gep-922.md similarity index 100% rename from site-src/geps/gep-922.md rename to geps/gep-922.md diff --git a/site-src/geps/gep-957.md b/geps/gep-957.md similarity index 94% rename from site-src/geps/gep-957.md rename to geps/gep-957.md index d24d326754..8834919539 100644 --- a/site-src/geps/gep-957.md +++ b/geps/gep-957.md @@ -3,6 +3,12 @@ * Issue: [#957](https://github.com/kubernetes-sigs/gateway-api/issues/957) * Status: Experimental +> **Note**: This GEP is exempt from the [Probationary Period][expprob] rules of +> our GEP overview as it existed before those rules did, and so it has been +> explicitly grandfathered in. + +[expprob]:https://gateway-api.sigs.k8s.io/geps/overview/#probationary-period + ## TLDR Add a new `port` field to ParentRef to support port matching in Routes. diff --git a/site-src/geps/images/1324-backend-ref.png b/geps/images/1324-backend-ref.png similarity index 100% rename from site-src/geps/images/1324-backend-ref.png rename to geps/images/1324-backend-ref.png diff --git a/site-src/geps/images/1324-decomposed-view-of-service.png b/geps/images/1324-decomposed-view-of-service.png similarity index 100% rename from site-src/geps/images/1324-decomposed-view-of-service.png rename to geps/images/1324-decomposed-view-of-service.png diff --git a/site-src/geps/images/1324-resource-view-of-service.png b/geps/images/1324-resource-view-of-service.png similarity index 100% rename from site-src/geps/images/1324-resource-view-of-service.png rename to geps/images/1324-resource-view-of-service.png diff --git a/site-src/geps/images/1324-service-frontend.png b/geps/images/1324-service-frontend.png similarity index 100% rename from site-src/geps/images/1324-service-frontend.png rename to geps/images/1324-service-frontend.png diff --git a/geps/images/1897-TLStermtypes.png b/geps/images/1897-TLStermtypes.png new file mode 100644 index 0000000000..53e258ff78 Binary files /dev/null and b/geps/images/1897-TLStermtypes.png differ diff --git a/site-src/geps/images/709-inline.png b/geps/images/709-inline.png similarity index 100% rename from site-src/geps/images/709-inline.png rename to geps/images/709-inline.png diff --git a/site-src/geps/images/709-referencegrant.png b/geps/images/709-referencegrant.png similarity index 100% rename from site-src/geps/images/709-referencegrant.png rename to geps/images/709-referencegrant.png diff --git a/site-src/geps/images/713-hierarchy.png b/geps/images/713-hierarchy.png similarity index 100% rename from site-src/geps/images/713-hierarchy.png rename to geps/images/713-hierarchy.png diff --git a/site-src/geps/images/713-ingress-attachment.png b/geps/images/713-ingress-attachment.png similarity index 100% rename from site-src/geps/images/713-ingress-attachment.png rename to geps/images/713-ingress-attachment.png diff --git a/site-src/geps/images/713-ingress-complex.png b/geps/images/713-ingress-complex.png similarity index 100% rename from site-src/geps/images/713-ingress-complex.png rename to geps/images/713-ingress-complex.png diff --git a/site-src/geps/images/713-ingress-simple.png b/geps/images/713-ingress-simple.png similarity index 100% rename from site-src/geps/images/713-ingress-simple.png rename to geps/images/713-ingress-simple.png diff --git a/site-src/geps/images/713-mesh-complex.png b/geps/images/713-mesh-complex.png similarity index 100% rename from site-src/geps/images/713-mesh-complex.png rename to geps/images/713-mesh-complex.png diff --git a/site-src/geps/images/713-mesh-simple.png b/geps/images/713-mesh-simple.png similarity index 100% rename from site-src/geps/images/713-mesh-simple.png rename to geps/images/713-mesh-simple.png diff --git a/site-src/geps/images/713-policy-hierarchy.png b/geps/images/713-policy-hierarchy.png similarity index 100% rename from site-src/geps/images/713-policy-hierarchy.png rename to geps/images/713-policy-hierarchy.png diff --git a/site-src/geps/images/713-servicebinding-complex.png b/geps/images/713-servicebinding-complex.png similarity index 100% rename from site-src/geps/images/713-servicebinding-complex.png rename to geps/images/713-servicebinding-complex.png diff --git a/site-src/geps/images/713-servicebinding-simple.png b/geps/images/713-servicebinding-simple.png similarity index 100% rename from site-src/geps/images/713-servicebinding-simple.png rename to geps/images/713-servicebinding-simple.png diff --git a/site-src/geps/images/724-alt1.png b/geps/images/724-alt1.png similarity index 100% rename from site-src/geps/images/724-alt1.png rename to geps/images/724-alt1.png diff --git a/site-src/geps/images/724-alt2.png b/geps/images/724-alt2.png similarity index 100% rename from site-src/geps/images/724-alt2.png rename to geps/images/724-alt2.png diff --git a/site-src/geps/images/724-proposal.png b/geps/images/724-proposal.png similarity index 100% rename from site-src/geps/images/724-proposal.png rename to geps/images/724-proposal.png diff --git a/geps/overview.md b/geps/overview.md new file mode 100644 index 0000000000..7e1b35543d --- /dev/null +++ b/geps/overview.md @@ -0,0 +1,214 @@ +# Gateway Enhancement Proposal (GEP) + +Gateway Enhancement Proposals (GEPs) serve a similar purpose to the [KEP][kep] +process for the main Kubernetes project: + +1. Ensure that changes to the API follow a known process and discussion + in the OSS community. +1. Make changes and proposals discoverable (current and future). +1. Document design ideas, tradeoffs, decisions that were made for + historical reference. + +## Process + +This diagram shows the state diagram of the GEP process at a high level, but the details are below. + +
+ +```mermaid +flowchart TD + D([Discuss with
the community]) --> C + C([Issue Created]) --> Provisional + Provisional -->|GEP Doc PR
done| Implementable + Provisional -->|If practical
work needed| Prototyping + Prototyping -->|GEP Doc PR
done| Implementable + Implementable -->|Gateway API
work completed| Experimental + Experimental -->|Supported in
multiple implementations
+ Conformance tests| Standard + Standard -->|Entire change is GA or implemented| Completed +``` + +
+ +### 1. Discuss with the community + +Before creating a GEP, share your high level idea with the community. There are +several places this may be done: + +- A [new GitHub Discussion](https://github.com/kubernetes-sigs/gateway-api/discussions/new) +- On our [Slack Channel](https://kubernetes.slack.com/archives/CR0H13KGA) +- On one of our [community meetings](https://gateway-api.sigs.k8s.io/contributing/?h=meetings#meetings) + +Please default to GitHub discussions: they work a lot like GitHub issues which +makes them easy to search. + +### 2. Create an Issue +[Create a GEP issue](https://github.com/kubernetes-sigs/gateway-api/issues/new?assignees=&labels=kind%2Ffeature&template=enhancement.md) in the repo describing your change. +At this point, you should copy the outcome of any other conversations or documents +into this document. + +### 3. Agree on the Goals +Although it can be tempting to start writing out all the details of your +proposal, it's important to first ensure we all agree on the goals. The first +version of your GEP should aim for a "Provisional" status and leave out any +implementation details, focusing primarily on "Goals" and "Non-Goals". + +### 3. Document Implementation Details +Now that everyone agrees on the goals, it is time to start writing out your +proposed implementation details. These implementation details should be very +thorough, including the proposed API spec, and covering any relevant edge cases. +Note that it may be helpful to use a shared doc for part of this phase to enable +faster iteration on potential designs. + +It is likely that throughout this process, you will discuss a variety of +alternatives. Be sure to document all of these in the GEP, and why we decided +against them. At this stage, the GEP should be targeting the "Implementable" +stage. + +### 4. Implement the GEP as "Experimental" + +With the GEP marked as "Implementable", it is time to actually make those +proposed changes in our API. In some cases, these changes will be documentation +only, but in most cases, some API changes will also be required. It is important +that every new feature of the API is marked as "Experimental" when it is +introduced. Within the API, we use `` tags to denote +experimental fields. Within Golang packages (conformance tests, CLIs, e.t.c.) we +use the `experimental` Golang build tag to denote experimental functionality. + +Some other requirements must be met before marking a GEP `Experimental`: + +- the graduation criteria to reach `Standard` MUST be filled out +- a proposed probationary period (see next section) must be included in the GEP + and approved by maintainers. + +Before changes are released they MUST be documented. GEPs that have not been +both implemented and documented before a release cut off will be excluded from +the release. + +#### Probationary Period + +Any GEP in the `Experimental` phase is automatically under a "probationary +period" where it will come up for re-assessment if its graduation criteria are +not met within a given time period. GEPs that wish to move into `Experimental` +status MUST document a proposed period (6 months is the suggested default) that +MUST be approved by maintainers. Maintainers MAY select an alternative time +duration for a probationary period if deemed appropriate, and will document +their reasoning. + +> **Rationale**: This probationary period exists to avoid GEPs getting "stale" +> and to provide guidance to implementations about how relevant features should +> be used, given that they are not guaranteed to become supported. + +At the end of a probationary period if the GEP has not been able to resolve +its graduation criteria it will move to "Rejected" status. In extenuating +circumstances an extension of that period may be accepted by approval from +maintainers. GEPs which are `Rejected` in this way are removed from the +experimental CRDs and more or less put on hold. GEPs may be allowed to move back +into `Experimental` status from `Rejected` for another probationary period if a +new strategy for achieving their graduation criteria can be established. Any +such plan to take a GEP "off the shelf" must be reviewed and accepted by the +maintainers. + +> **Warning**: It is extremely important** that projects which implement +> `Experimental` features clearly document that these features may be removed in +> future releases. + +### 5. Graduate the GEP to "Standard" + +Once this feature has met the [graduation criteria](/concepts/versioning/#graduation-criteria), it is +time to graduate it to the "Standard" channel of the API. Depending on the feature, this may include +any of the following: + +1. Graduating the resource to beta +2. Graduating fields to "standard" by removing `` tags +3. Graduating a concept to "standard" by updating documentation + +### 6. Close out the GEP issue + +The GEP issue should only be closed once the feature has: +- Moved to the standard channel for distribution (if necessary) +- Moved to a "v1" `apiVersion` for CRDs +- been completely implemented and has wide acceptance (for process changes). + +In short, the GEP issue should only be closed when the work is "done" (whatever +that means for that GEP). + +## Status + +Each GEP has a status field that defines it's current state. Each transition +will require a PR to update the GEP and should be discussed at a community +meeting before merging. Most GEPS will proceed through the following states: + +* **Provisional:** The goals described by this GEP have consensus but + implementation details have not been agreed to yet. +* **Prototyping:** An extension of `Provisional` which can be opted in to in + order to indicate to the community that there are some active practical tests + and experiments going on which are intended to be a part of the development + of this GEP. This may include APIs or code, but that content _must_ not be + distributed with releases. +* **Implementable:** The goals and implementation details described by this GEP + have consensus but have not been fully implemented yet. +* **Experimental:** This GEP has been implemented and is part of the + "Experimental" release channel. Breaking changes are still possible, up to + and including complete removal and moving to `Rejected`. +* **Standard:** This GEP has been implemented and is part of the + "Standard" release channel. It should be quite stable. + +Although less common, some GEPs may end up in one of the following states: + +* **Deferred:** We do not currently have bandwidth to handle this GEP, it + may be revisited in the future. +* **Rejected:** This proposal was considered by the community but ultimately + rejected. +* **Replaced:** This proposal was considered by the community but ultimately + replaced by a newer proposal. +* **Withdrawn:** This proposal was considered by the community but ultimately + withdrawn by the author. + +## Format + +GEPs should match the format of the template found in [GEP-696](/geps/gep-696.md). + +## Out of scope + +What is out of scope: see [text from KEP][kep-when-to-use]. Examples: + +* Bug fixes +* Small changes (API validation, documentation, fixups). It is always + possible that the reviewers will determine a "small" change ends up + requiring a GEP. + +## FAQ + +* Q: Why is it named GEP? + * A: To avoid potential confusion if people start following the cross + references to the full KEP process. +* Q: Why have a different process than mainline? + * A: We would like to keep the machinery to an absolute minimum for now -- + this may change as we move to v1. +* Q: Is it ok to discuss using shared docs, scratch docs etc? + * A: Yes, this can be a helpful intermediate step when iterating on design + details. It is important that all major feedback, discussions, and + alternatives considered in that step are represented in the GEP though. A + key goal of GEPs is to show why we made a decision and which alternatives + were considered. If separate docs are used, it's important that we can + still see all relevant context and decisions in the final GEP. +* Q: When should I mark a GEP as `Prototyping` as opposed to `Provisional`? + * A: The `Prototyping` status carries the same base meaning as `Provisional` + in that consensus is not complete between stakeholders and we're not ready + to move toward releasing content yet. You should use `Prototyping` to + indicate to your fellow community members that we're in a state of active + practical tests and experiments which are intended to help us learn and + iterate on the GEP. These can include distributing content, but not under + any release channel. +* Q: Should I implement support for `Experimental` channel features? + * A: Ultimately one of the main ways to get something into `Standard` is for + it to mature through the `Experimental` phase, so we really _need_ people + to implement these features and provide feedback in order to have progress. + That said, the graduation of a feature past `Experimental` is not a forgone + conclusion. Before implementing an experimental feature, you should: + + * Clearly document that support for the feature is experimental and may disappear in the future. + * Have a plan in place for how you would handle the removal of this feature from the API. + +[kep]: https://github.com/kubernetes/enhancements +[kep-when-to-use]: https://github.com/kubernetes/enhancements/tree/master/keps#do-i-have-to-use-the-kep-process diff --git a/go.mod b/go.mod index 70fb8d8f17..67707e783d 100644 --- a/go.mod +++ b/go.mod @@ -5,66 +5,74 @@ go 1.19 require ( github.com/ahmetb/gen-crd-api-reference-docs v0.3.0 github.com/lithammer/dedent v1.1.0 - github.com/stretchr/testify v1.8.1 - k8s.io/api v0.26.1 - k8s.io/apiextensions-apiserver v0.26.1 - k8s.io/apimachinery v0.26.1 - k8s.io/client-go v0.26.1 - k8s.io/code-generator v0.26.1 - k8s.io/klog/v2 v2.90.0 - sigs.k8s.io/controller-runtime v0.14.2 - sigs.k8s.io/controller-tools v0.11.2 + github.com/stretchr/testify v1.8.4 + k8s.io/api v0.27.4 + k8s.io/apiextensions-apiserver v0.27.4 + k8s.io/apimachinery v0.27.4 + k8s.io/client-go v0.27.4 + k8s.io/code-generator v0.27.4 + k8s.io/klog/v2 v2.100.1 + k8s.io/utils v0.0.0-20230209194617-a36077c30491 + sigs.k8s.io/controller-runtime v0.15.1 + sigs.k8s.io/controller-tools v0.12.1 sigs.k8s.io/yaml v1.3.0 ) +require ( + go.uber.org/atomic v1.9.0 // indirect + go.uber.org/multierr v1.8.0 // indirect +) + require ( github.com/davecgh/go-spew v1.1.1 // indirect github.com/emicklei/go-restful/v3 v3.9.0 // indirect github.com/evanphx/json-patch v4.12.0+incompatible // indirect github.com/evanphx/json-patch/v5 v5.6.0 // indirect - github.com/fatih/color v1.13.0 // indirect - github.com/go-logr/logr v1.2.3 // indirect - github.com/go-openapi/jsonpointer v0.19.5 // indirect - github.com/go-openapi/jsonreference v0.20.0 // indirect - github.com/go-openapi/swag v0.19.14 // indirect - github.com/gobuffalo/flect v0.3.0 // indirect + github.com/fatih/color v1.15.0 // indirect + github.com/go-logr/logr v1.2.4 // indirect + github.com/go-openapi/jsonpointer v0.19.6 // indirect + github.com/go-openapi/jsonreference v0.20.1 // indirect + github.com/go-openapi/swag v0.22.3 // indirect + github.com/gobuffalo/flect v1.0.2 // indirect github.com/gogo/protobuf v1.3.2 // indirect - github.com/golang/protobuf v1.5.2 // indirect + github.com/golang/protobuf v1.5.3 // indirect github.com/google/gnostic v0.5.7-v3refs // indirect github.com/google/go-cmp v0.5.9 // indirect github.com/google/gofuzz v1.1.0 // indirect + github.com/google/uuid v1.3.0 // indirect github.com/imdario/mergo v0.3.6 // indirect - github.com/inconshreveable/mousetrap v1.0.1 // indirect + github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/josharian/intern v1.0.0 // indirect github.com/json-iterator/go v1.1.12 // indirect - github.com/mailru/easyjson v0.7.6 // indirect - github.com/mattn/go-colorable v0.1.9 // indirect - github.com/mattn/go-isatty v0.0.14 // indirect + github.com/kr/pretty v0.3.1 // indirect + github.com/mailru/easyjson v0.7.7 // indirect + github.com/mattn/go-colorable v0.1.13 // indirect + github.com/mattn/go-isatty v0.0.17 // indirect + github.com/moby/spdystream v0.2.0 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/pkg/errors v0.9.1 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect github.com/russross/blackfriday/v2 v2.1.0 // indirect - github.com/spf13/cobra v1.6.1 // indirect + github.com/spf13/cobra v1.7.0 // indirect github.com/spf13/pflag v1.0.5 // indirect - golang.org/x/mod v0.7.0 // indirect - golang.org/x/net v0.4.0 // indirect - golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b // indirect - golang.org/x/sys v0.3.0 // indirect - golang.org/x/term v0.3.0 // indirect - golang.org/x/text v0.5.0 // indirect + golang.org/x/mod v0.10.0 // indirect + golang.org/x/net v0.10.0 // indirect + golang.org/x/oauth2 v0.5.0 // indirect + golang.org/x/sys v0.8.0 // indirect + golang.org/x/term v0.8.0 // indirect + golang.org/x/text v0.9.0 // indirect golang.org/x/time v0.3.0 // indirect - golang.org/x/tools v0.4.0 // indirect + golang.org/x/tools v0.9.1 // indirect google.golang.org/appengine v1.6.7 // indirect - google.golang.org/protobuf v1.28.1 // indirect + google.golang.org/protobuf v1.30.0 // indirect gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect k8s.io/gengo v0.0.0-20220902162205-c0856e24416d // indirect k8s.io/klog v0.2.0 // indirect - k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 // indirect - k8s.io/utils v0.0.0-20221128185143-99ec85e7a448 // indirect - sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 // indirect + k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f // indirect + sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect sigs.k8s.io/structured-merge-diff/v4 v4.2.3 // indirect ) diff --git a/go.sum b/go.sum index eb7a6782a3..930eb597c8 100644 --- a/go.sum +++ b/go.sum @@ -1,48 +1,12 @@ cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU= -cloud.google.com/go v0.44.1/go.mod h1:iSa0KzasP4Uvy3f1mN/7PiObzGgflwredwwASm/v6AU= -cloud.google.com/go v0.44.2/go.mod h1:60680Gw3Yr4ikxnPRS/oxxkBccT6SA1yMk63TGekxKY= -cloud.google.com/go v0.45.1/go.mod h1:RpBamKRgapWJb87xiFSdk4g1CME7QZg3uwTez+TSTjc= -cloud.google.com/go v0.46.3/go.mod h1:a6bKKbmY7er1mI7TEI4lsAkts/mkhTSZK8w33B4RAg0= -cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6To= -cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4= -cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M= -cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc= -cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk= -cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs= -cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc= -cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY= -cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o= -cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE= -cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc= -cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg= -cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc= -cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ= -cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE= -cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk= -cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I= -cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw= -cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA= -cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU= -cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw= -cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos= -cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk= -cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs= -cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0= -dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo= github.com/ahmetb/gen-crd-api-reference-docs v0.3.0 h1:+XfOU14S4bGuwyvCijJwhhBIjYN+YXS18jrCY2EzJaY= github.com/ahmetb/gen-crd-api-reference-docs v0.3.0/go.mod h1:TdjdkYhlOifCQWPs1UdTma97kQQMozf5h26hTuG70u8= +github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/cespare/xxhash/v2 v2.1.2 h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE= -github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= -github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= -github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= +github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= -github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc= github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= @@ -51,128 +15,93 @@ github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSs github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE= github.com/emicklei/go-restful/v3 v3.9.0 h1:XwGDlfxEnQZzuopoqxwSEllNcCOM9DhhFyhFIIGKwxE= github.com/emicklei/go-restful/v3 v3.9.0/go.mod h1:6n3XBCmQQb25CM2LCACGz8ukIrRry+4bhvbpWn3mrbc= -github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= -github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98= github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH5pOlLGNtQ5lPWQu84= github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk= github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww= github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4= -github.com/fatih/color v1.13.0 h1:8LOYc1KYPPmyKMuN8QV2DNRWNbLo6LZ0iLs8+mlH53w= -github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk= +github.com/fatih/color v1.15.0 h1:kOqh6YHBtK8aywxGerMG2Eq3H6Qgoqeo13Bk2Mv/nBs= +github.com/fatih/color v1.15.0/go.mod h1:0h5ZqXfHYED7Bhv2ZJamyIOUej9KtShiJESRwBDUSsw= github.com/fsnotify/fsnotify v1.6.0 h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY= -github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU= -github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= -github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8= github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/logr v1.2.3 h1:2DntVwHkVopvECVRSlL5PSo9eG+cAkDCuckLubN+rq0= -github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= -github.com/go-logr/zapr v1.2.3 h1:a9vnzlIBPQBBkeaR9IuMUfmVOrQlkoC4YfPoFkX3T7A= -github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= -github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY= -github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg= -github.com/go-openapi/jsonreference v0.20.0 h1:MYlu0sBgChmCfJxxUKZ8g1cPWFOB37YSZqewK7OKeyA= -github.com/go-openapi/jsonreference v0.20.0/go.mod h1:Ag74Ico3lPc+zR+qjn4XBUmXymS4zJbYVCZmcgkasdo= -github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh66Z9tfKk= -github.com/go-openapi/swag v0.19.14 h1:gm3vOOXfiuw5i9p5N9xJvfjvuofpyvLA9Wr6QfK5Fng= -github.com/go-openapi/swag v0.19.14/go.mod h1:QYRuS/SOXUCsnplDa677K7+DxSOj6IPNl/eQntq43wQ= -github.com/gobuffalo/flect v0.3.0 h1:erfPWM+K1rFNIQeRPdeEXxo8yFr/PO17lhRnS8FUrtk= -github.com/gobuffalo/flect v0.3.0/go.mod h1:5pf3aGnsvqvCj50AVni7mJJF8ICxGZ8HomberC3pXLE= +github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ= +github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A= +github.com/go-logr/zapr v1.2.4 h1:QHVo+6stLbfJmYGkQ7uGHUCu5hnAFAj6mDe6Ea0SeOo= +github.com/go-openapi/jsonpointer v0.19.6 h1:eCs3fxoIi3Wh6vtgmLTOjdhSpiqphQ+DaPn38N2ZdrE= +github.com/go-openapi/jsonpointer v0.19.6/go.mod h1:osyAmYz/mB/C3I+WsTTSgw1ONzaLJoLCyoi6/zppojs= +github.com/go-openapi/jsonreference v0.20.1 h1:FBLnyygC4/IZZr893oiomc9XaghoveYTrLC1F86HID8= +github.com/go-openapi/jsonreference v0.20.1/go.mod h1:Bl1zwGIM8/wsvqjsOQLJ/SH+En5Ap4rVB5KVcIDZG2k= +github.com/go-openapi/swag v0.22.3 h1:yMBqmnQ0gyZvEb/+KzuWZOXgllrXT4SADYbvDaXHv/g= +github.com/go-openapi/swag v0.22.3/go.mod h1:UzaqsxGiab7freDnrUUra0MwWfN/q7tE4j+VcZ0yl14= +github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI= +github.com/gobuffalo/flect v1.0.2 h1:eqjPGSo2WmjgY2XlpGwo2NXgL3RucAKo4k4qQMNA5sA= +github.com/gobuffalo/flect v1.0.2/go.mod h1:A5msMlrHtLqh9umBSnvabjsMrCcCpAyzglnDvkbYKHs= github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= -github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc= github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y= -github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw= -github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw= -github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk= github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= -github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= -github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= -github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= -github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/golang/protobuf v1.5.3 h1:KhyjKVUg7Usr/dYsdSqoFveMYd5ko72D+zANwlG1mmg= +github.com/golang/protobuf v1.5.3/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= github.com/google/gnostic v0.5.7-v3refs h1:FhTMOKj2VhjpouxvWJAV1TL304uMlb9zcDqkl6cEI54= github.com/google/gnostic v0.5.7-v3refs/go.mod h1:73MKFl6jIHelAJNaBGFzt3SPtZULs9dYrGFt8OiIsHQ= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g= github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= -github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= -github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0= -github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= -github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc= -github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM= -github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI= -github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y= -github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg= -github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk= -github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= -github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc= +github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 h1:K6RDEckDVWvDI9JAJYCmNdQXq6neHJOYx3V6jnqNEec= +github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I= +github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28= github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA= -github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc= -github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI= github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY= github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= -github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= -github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= -github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.0/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= +github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= +github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk= github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ= github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/lithammer/dedent v1.1.0 h1:VNzHMVCBNG1j0fh3OrsFRkVUwStdDArbgBWoPAffktY= github.com/lithammer/dedent v1.1.0/go.mod h1:jrXYCQtgg0nJiN+StA2KgR7w6CiQNv9Fd/Z9BP0jIOc= -github.com/mailru/easyjson v0.0.0-20190614124828-94de47d64c63/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= -github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc= -github.com/mailru/easyjson v0.7.6 h1:8yTIVnZgCoiM1TgqoeTl+LfU5Jg6/xL3QhGQnimLYnA= -github.com/mailru/easyjson v0.7.6/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= -github.com/mattn/go-colorable v0.1.9 h1:sqDoxXbdeALODt0DAeJCVp38ps9ZogZEAXjus69YV3U= -github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc= -github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU= -github.com/mattn/go-isatty v0.0.14 h1:yVuAays6BHfxijgZPzw+3Zlu5yQgKGP2/hcQbHb7S9Y= -github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94= -github.com/matttproud/golang_protobuf_extensions v1.0.2 h1:hAHbPm5IJGijwng3PWk09JkG9WeqChjprR5s9bBZ+OM= +github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0= +github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc= +github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA= +github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg= +github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= +github.com/mattn/go-isatty v0.0.17 h1:BTarxUcIeDqL27Mc+vyvdWYSL28zpIhv3RoTdsLMPng= +github.com/mattn/go-isatty v0.0.17/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= +github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo= +github.com/moby/spdystream v0.2.0 h1:cjW1zVyyoiM0T7b6UoySUFqzXMoqRckQtXwGPiBhOM8= +github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= @@ -180,29 +109,29 @@ github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9G github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= -github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs= -github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno= github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE= github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE= -github.com/onsi/ginkgo/v2 v2.6.0 h1:9t9b9vRUbFq3C4qKFCGkVuq/fIHji802N1nrtkh1mNc= -github.com/onsi/gomega v1.24.2 h1:J/tulyYK6JwBldPViHJReihxxZ+22FHs0piGjQAvoUE= +github.com/onsi/ginkgo/v2 v2.9.5 h1:+6Hr4uxzP4XIUyAkg61dWBw8lb/gc4/X5luuxN/EC+Q= +github.com/onsi/gomega v1.27.7 h1:fVih9JD6ogIiHUN6ePK7HJidyEDpWGVB5mzM7cWNXoU= +github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_golang v1.14.0 h1:nJdhIvne2eSX/XRAFV9PcvFFRbrjbcTUj0VP62TMhnw= +github.com/prometheus/client_golang v1.15.1 h1:8tXpTmJbyH5lydzFPoxSIJ0J46jdh3tylbvM1xCv0LI= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/prometheus/client_model v0.3.0 h1:UBgGFHqYdG/TPFD1B1ogZywDqEkwp3fBMvqdiQ7Xew4= -github.com/prometheus/common v0.37.0 h1:ccBbHCgIiT9uSoFY0vX8H3zsNR5eLt17/RQLUvn8pXE= -github.com/prometheus/procfs v0.8.0 h1:ODq8ZFEaYeCaZOJlZZdJA2AbQR98dSHSM1KW/You5mo= -github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4= +github.com/prometheus/client_model v0.4.0 h1:5lQXD3cAg1OXBf4Wq03gTrXHeaV0TQvGfUooCfx1yqY= +github.com/prometheus/common v0.42.0 h1:EKsfXEYo4JpWMHH5cg+KOUWeuJSov1Id8zGR8eeI1YM= +github.com/prometheus/procfs v0.9.0 h1:wzCHvIvM5SxWqYvwgVL7yJY8Lz3PKn49KQtpgMYJfhI= +github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs= +github.com/rogpeppe/go-internal v1.11.0 h1:cWPaGQEPrBb5/AsnsZesgZZ9yb1OQ+GOISoDNXVBh4M= github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= -github.com/spf13/cobra v1.6.1 h1:o94oiPyS4KD1mPy2fmcYYHHfCxLqYjJOhGsCHFZtEzA= -github.com/spf13/cobra v1.6.1/go.mod h1:IOw/AERYS7UzyrGinqmz6HLUo219MORXGxhbaJUqzrY= +github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I= +github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0= github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8= @@ -210,293 +139,111 @@ github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+ github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= -github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA= -github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= -github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= -github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= +github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= -go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU= -go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8= -go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= -go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw= -go.uber.org/multierr v1.6.0 h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4= +go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= +go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE= +go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc= +go.uber.org/multierr v1.8.0 h1:dg6GjLku4EH+249NNmoIciG9N/jURbDG+pFlTkhzIC8= +go.uber.org/multierr v1.8.0/go.mod h1:7EAYxJLBy9rStEaz58O2t4Uvip6FSURkq8/ppBp95ak= go.uber.org/zap v1.24.0 h1:FiJd5l1UOLj0wCgbSE0rwwXHzEdAZS6hiiSnxJN/D60= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= -golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8= -golang.org/x/exp v0.0.0-20190829153037-c13cbed26979/go.mod h1:86+5VVa7VpoJ4kLfm080zCjGlMRFzhUhsZKEZO7MGek= -golang.org/x/exp v0.0.0-20191030013958-a1ab85dbe136/go.mod h1:JXzH8nQsPlswgeRAPE3MuO9GYsAcnJvJ4vnMwN/5qkY= -golang.org/x/exp v0.0.0-20191129062945-2f5052295587/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4= -golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM= -golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU= -golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js= -golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0= golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= -golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190909230951-414d861bb4ac/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs= -golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= -golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE= -golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o= -golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc= -golang.org/x/mod v0.1.0/go.mod h1:0QHyrYULN0/3qlju5TqG8bIK38QM8yzMo5ekMj3DlcY= -golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= -golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= -golang.org/x/mod v0.7.0 h1:LapD9S96VoQRhi/GrNTqeBJFrUjs5UHCAtTlgwA5oZA= -golang.org/x/mod v0.7.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.10.0 h1:lFO9qtOdlre5W1jxS3r/4szv2/6iXxScdzjoBMXNhYk= +golang.org/x/mod v0.10.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190501004415-9ce7a6920f09/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= -golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A= -golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= -golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= -golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= -golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su4UZFWDARRcnwPjda9FqA0JpMk= -golang.org/x/net v0.4.0 h1:Q5QPcMlvfxFTAPV0+07Xz/MpK9NTXu2VDUuy0FeMfaU= -golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE= +golang.org/x/net v0.10.0 h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M= +golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg= golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= -golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw= -golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b h1:clP8eMhB30EHdc0bd2Twtq6kgU7yl5ub2cQLSdrv1Dg= -golang.org/x/oauth2 v0.0.0-20220223155221-ee480838109b/go.mod h1:DAh4E804XQdzx2j+YRIaUnCqCV2RuMz24cGBJ5QYIrc= +golang.org/x/oauth2 v0.5.0 h1:HuArIo48skDwlrvM3sEdHXElYslAMsf3KwRkkW4MC4s= +golang.org/x/oauth2 v0.5.0/go.mod h1:9/XBHVqLaWO3/BRHs5jbpYCnOZVjj5V0ndyaAM7KB4I= golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o= +golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190606165138-5da285871e9c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191001151750-bb3f8db39f24/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191204072324-ce4227a45e2e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20191228213918-04cbcbbfeed8/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200113162924-86b910548bc1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.3.0 h1:w8ZOecv6NaNa/zC8944JTU3vz4u6Lagfk4RPQxv92NQ= -golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= -golang.org/x/term v0.3.0 h1:qoo4akIqOcDME5bhc/NgxUdovd6BSS2uMsVjB56q1xI= -golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA= -golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU= +golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/term v0.8.0 h1:n5xxQn2i3PC0yLAbjTpNT85q/Kgzcr2gIoX9OrJUols= +golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= -golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= -golang.org/x/text v0.5.0 h1:OLmvp0KP+FVG99Ct/qFiL/Fhk4zp4QQnZ7b2U+5piUM= -golang.org/x/text v0.5.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= -golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE= +golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8= golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4= golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190312151545-0bb0c0a6e846/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc= -golang.org/x/tools v0.0.0-20190816200558-6889da9d5479/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20190911174233-4f2ddba30aff/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191012152004-8de300cfc20a/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191113191852-77e3bb0ad9e7/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191115202509-3a792d9c32b2/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191125144606-a911d9008d1f/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191130070609-6e064ea0cf2d/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= -golang.org/x/tools v0.0.0-20191216173652-a0e659d51361/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20191227053925-7b8e75db28f4/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200117161641-43d50277825c/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200122220014-bf1340f18c4a/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= -golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= -golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw= -golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8= -golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200505023115-26f46d2f7ef8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= -golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= -golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= -golang.org/x/tools v0.4.0 h1:7mTAgkunk3fr4GAloyyCasadO6h9zSsQZbwvcaIciV4= -golang.org/x/tools v0.4.0/go.mod h1:UE5sM2OK9E/d67R0ANs2xJizIymRP5gJU295PvKXxjQ= +golang.org/x/tools v0.9.1 h1:8WMNJAz3zrtPmnYC7ISf5dEn3MT0gY7jBJfw27yrrLo= +golang.org/x/tools v0.9.1/go.mod h1:owI94Op576fPu3cIGQeHs3joujW/2Oc6MtlxbF5dfNc= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -gomodules.xyz/jsonpatch/v2 v2.2.0 h1:4pT439QV83L+G9FkcCriY6EkpcK6r6bK+A5FBUMI7qY= -google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE= -google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M= -google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= -google.golang.org/api v0.9.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg= -google.golang.org/api v0.13.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI= -google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE= -google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= -google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE= -google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM= -google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc= +gomodules.xyz/jsonpatch/v2 v2.3.0 h1:8NFhfS6gzxNqjLIYnZxg319wZ5Qjnx4m/CcX+Klzazc= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0= -google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= -google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c= google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc= google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= -google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE= -google.golang.org/genproto v0.0.0-20190801165951-fa694d86fc64/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= -google.golang.org/genproto v0.0.0-20190911173649-1774047e7e51/go.mod h1:IbNlFCBrqXvoKpeg0TB2l7cyZUmoaFKYIwrEpbDKLA8= -google.golang.org/genproto v0.0.0-20191108220845-16a3f7862a1a/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191115194625-c23dd37a84c9/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191216164720-4f79533eabd1/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20191230161307-f3c370f40bfb/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200115191322-ca5a22157cba/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc= -google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA= -google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c= -google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U= google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= -google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA= -google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= -google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= -google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38= -google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM= google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= -google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY= -google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60= -google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk= -google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= -google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak= google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4= -google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc= -google.golang.org/protobuf v1.28.1 h1:d0NfwRgPtno5B1Wa6L2DAG+KivqkdutMf1UhdNx175w= -google.golang.org/protobuf v1.28.1/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= +google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng= +google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f h1:BLraFXnmrev5lT+xlilqcH8XK9/i0At2xKjWk4p6zsU= -gopkg.in/check.v1 v1.0.0-20200227125254-8fa46927fb4f/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= -gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= +gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/inf.v0 v0.9.1 h1:73M5CoZyi3ZLMOyDlQh031Cx6N9NDJ2Vvfl76EDAgDc= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ= @@ -506,46 +253,39 @@ gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= +gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg= -honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= -k8s.io/api v0.26.1 h1:f+SWYiPd/GsiWwVRz+NbFyCgvv75Pk9NK6dlkZgpCRQ= -k8s.io/api v0.26.1/go.mod h1:xd/GBNgR0f707+ATNyPmQ1oyKSgndzXij81FzWGsejg= -k8s.io/apiextensions-apiserver v0.26.1 h1:cB8h1SRk6e/+i3NOrQgSFij1B2S0Y0wDoNl66bn8RMI= -k8s.io/apiextensions-apiserver v0.26.1/go.mod h1:AptjOSXDGuE0JICx/Em15PaoO7buLwTs0dGleIHixSM= -k8s.io/apimachinery v0.26.1 h1:8EZ/eGJL+hY/MYCNwhmDzVqq2lPl3N3Bo8rvweJwXUQ= -k8s.io/apimachinery v0.26.1/go.mod h1:tnPmbONNJ7ByJNz9+n9kMjNP8ON+1qoAIIC70lztu74= -k8s.io/client-go v0.26.1 h1:87CXzYJnAMGaa/IDDfRdhTzxk/wzGZ+/HUQpqgVSZXU= -k8s.io/client-go v0.26.1/go.mod h1:IWNSglg+rQ3OcvDkhY6+QLeasV4OYHDjdqeWkDQZwGE= -k8s.io/code-generator v0.26.1 h1:dusFDsnNSKlMFYhzIM0jAO1OlnTN5WYwQQ+Ai12IIlo= -k8s.io/code-generator v0.26.1/go.mod h1:OMoJ5Dqx1wgaQzKgc+ZWaZPfGjdRq/Y3WubFrZmeI3I= +k8s.io/api v0.27.4 h1:0pCo/AN9hONazBKlNUdhQymmnfLRbSZjd5H5H3f0bSs= +k8s.io/api v0.27.4/go.mod h1:O3smaaX15NfxjzILfiln1D8Z3+gEYpjEpiNA/1EVK1Y= +k8s.io/apiextensions-apiserver v0.27.4 h1:ie1yZG4nY/wvFMIR2hXBeSVq+HfNzib60FjnBYtPGSs= +k8s.io/apiextensions-apiserver v0.27.4/go.mod h1:KHZaDr5H9IbGEnSskEUp/DsdXe1hMQ7uzpQcYUFt2bM= +k8s.io/apimachinery v0.27.4 h1:CdxflD4AF61yewuid0fLl6bM4a3q04jWel0IlP+aYjs= +k8s.io/apimachinery v0.27.4/go.mod h1:XNfZ6xklnMCOGGFNqXG7bUrQCoR04dh/E7FprV6pb+E= +k8s.io/client-go v0.27.4 h1:vj2YTtSJ6J4KxaC88P4pMPEQECWMY8gqPqsTgUKzvjk= +k8s.io/client-go v0.27.4/go.mod h1:ragcly7lUlN0SRPk5/ZkGnDjPknzb37TICq07WhI6Xc= +k8s.io/code-generator v0.27.4 h1:bw2xFEBnthhCSC7Bt6FFHhPTfWX21IJ30GXxOzywsFE= +k8s.io/code-generator v0.27.4/go.mod h1:DPung1sI5vBgn4AGKtlPRQAyagj/ir/4jI55ipZHVww= k8s.io/gengo v0.0.0-20201203183100-97869a43a9d9/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/gengo v0.0.0-20220902162205-c0856e24416d h1:U9tB195lKdzwqicbJvyJeOXV7Klv+wNAWENRnXEGi08= k8s.io/gengo v0.0.0-20220902162205-c0856e24416d/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E= k8s.io/klog v0.2.0 h1:0ElL0OHzF3N+OhoJTL0uca20SxtYt4X4+bzHeqrB83c= k8s.io/klog v0.2.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk= k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= -k8s.io/klog/v2 v2.90.0 h1:VkTxIV/FjRXn1fgNNcKGM8cfmL1Z33ZjXRTVxKCoF5M= -k8s.io/klog/v2 v2.90.0/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= -k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280 h1:+70TFaan3hfJzs+7VK2o+OGxg8HsuBr/5f6tVAjDu6E= -k8s.io/kube-openapi v0.0.0-20221012153701-172d655c2280/go.mod h1:+Axhij7bCpeqhklhUTe3xmOn6bWxolyZEeyaFpjGtl4= -k8s.io/utils v0.0.0-20221128185143-99ec85e7a448 h1:KTgPnR10d5zhztWptI952TNtt/4u5h3IzDXkdIMuo2Y= -k8s.io/utils v0.0.0-20221128185143-99ec85e7a448/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= -rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8= -rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0= -rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA= -sigs.k8s.io/controller-runtime v0.14.2 h1:P6IwDhbsRWsBClt/8/h8Zy36bCuGuW5Op7MHpFrN/60= -sigs.k8s.io/controller-runtime v0.14.2/go.mod h1:WqIdsAY6JBsjfc/CqO0CORmNtoCtE4S6qbPc9s68h+0= -sigs.k8s.io/controller-tools v0.11.2 h1:3GOMW8Ha4P0v1wt2bXbWOGKl186y2ijsjNFv+Rmi3Yw= -sigs.k8s.io/controller-tools v0.11.2/go.mod h1:qcfX7jfcfYD/b7lAhvqAyTbt/px4GpvN88WKLFFv7p8= -sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2 h1:iXTIw73aPyC+oRdyqqvVJuloN1p0AC/kzH07hu3NE+k= -sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= +k8s.io/klog/v2 v2.100.1 h1:7WCHKK6K8fNhTqfBhISHQ97KrnJNFZMcQvKp7gP/tmg= +k8s.io/klog/v2 v2.100.1/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0= +k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f h1:2kWPakN3i/k81b0gvD5C5FJ2kxm1WrQFanWchyKuqGg= +k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f/go.mod h1:byini6yhqGC14c3ebc/QwanvYwhuMWF6yz2F8uwW8eg= +k8s.io/utils v0.0.0-20230209194617-a36077c30491 h1:r0BAOLElQnnFhE/ApUsg3iHdVYYPBjNSSOMowRZxxsY= +k8s.io/utils v0.0.0-20230209194617-a36077c30491/go.mod h1:OLgZIPagt7ERELqWJFomSt595RzquPNLL48iOWgYOg0= +sigs.k8s.io/controller-runtime v0.15.1 h1:9UvgKD4ZJGcj24vefUFgZFP3xej/3igL9BsOUTb/+4c= +sigs.k8s.io/controller-runtime v0.15.1/go.mod h1:7ngYvp1MLT+9GeZ+6lH3LOlcHkp/+tzA/fmHa4iq9kk= +sigs.k8s.io/controller-tools v0.12.1 h1:GyQqxzH5wksa4n3YDIJdJJOopztR5VDM+7qsyg5yE4U= +sigs.k8s.io/controller-tools v0.12.1/go.mod h1:rXlpTfFHZMpZA8aGq9ejArgZiieHd+fkk/fTatY8A2M= +sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo= +sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= sigs.k8s.io/structured-merge-diff/v4 v4.2.3 h1:PRbqxJClWWYMNV1dhaG4NsibJbArud9kFxnAMREiWFE= sigs.k8s.io/structured-merge-diff/v4 v4.2.3/go.mod h1:qjx8mGObPmV2aSZepjQjbmb2ihdVs8cGKBraizNC69E= sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc= diff --git a/hack/build-and-push.sh b/hack/build-and-push.sh index 2f36ab754a..b8d43de7b3 100755 --- a/hack/build-and-push.sh +++ b/hack/build-and-push.sh @@ -78,4 +78,15 @@ docker buildx build \ --build-arg "TAG=${BINARY_TAG}" \ --platform ${BUILDX_PLATFORMS} \ --push \ + -f docker/Dockerfile.webhook \ + . + +echo "Building and pushing echo-server image...${BUILDX_PLATFORMS}" + +docker buildx build \ + -t ${REGISTRY}/echo-server:${GIT_TAG} \ + -t ${REGISTRY}/echo-server:${VERSION_TAG} \ + --platform ${BUILDX_PLATFORMS} \ + --push \ + -f docker/Dockerfile.echo \ . diff --git a/hack/cel-validation/gateway_test.go b/hack/cel-validation/gateway_test.go new file mode 100644 index 0000000000..a3fbad060d --- /dev/null +++ b/hack/cel-validation/gateway_test.go @@ -0,0 +1,513 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package main + +import ( + "context" + "fmt" + "strings" + "testing" + "time" + + gatewayv1b1 "sigs.k8s.io/gateway-api/apis/v1beta1" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func TestValidateGateway(t *testing.T) { + ctx := context.Background() + baseGateway := gatewayv1b1.Gateway{ + ObjectMeta: metav1.ObjectMeta{ + Name: "foo", + Namespace: metav1.NamespaceDefault, + }, + Spec: gatewayv1b1.GatewaySpec{ + GatewayClassName: "foo", + Listeners: []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("http"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + }, + }, + }, + } + + testCases := []struct { + desc string + mutate func(gw *gatewayv1b1.Gateway) + mutateStatus func(gw *gatewayv1b1.Gateway) + wantErrors []string + }{ + { + desc: "tls config present with http protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("http"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + TLS: &gatewayv1b1.GatewayTLSConfig{}, + }, + } + }, + wantErrors: []string{"tls must not be specified for protocols ['HTTP', 'TCP', 'UDP']"}, + }, + { + desc: "tls config present with tcp protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("tcp"), + Protocol: gatewayv1b1.TCPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + TLS: &gatewayv1b1.GatewayTLSConfig{}, + }, + } + }, + wantErrors: []string{"tls must not be specified for protocols ['HTTP', 'TCP', 'UDP']"}, + }, + { + desc: "tls config not set with https protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("https"), + Protocol: gatewayv1b1.HTTPSProtocolType, + Port: gatewayv1b1.PortNumber(8443), + }, + } + }, + wantErrors: []string{"tls must be specified for protocols ['HTTPS', 'TLS']"}, + }, + { + desc: "tls config not set with tls protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("tls"), + Protocol: gatewayv1b1.TLSProtocolType, + Port: gatewayv1b1.PortNumber(8443), + }, + } + }, + wantErrors: []string{"tls must be specified for protocols ['HTTPS', 'TLS']"}, + }, + { + desc: "tls config not set with http protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("http"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + }, + } + }, + }, + { + desc: "tls config not set with tcp protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("tcp"), + Protocol: gatewayv1b1.TCPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + }, + } + }, + }, + { + desc: "tls config not set with udp protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("udp"), + Protocol: gatewayv1b1.UDPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + }, + } + }, + }, + { + desc: "hostname present with tcp protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + hostname := gatewayv1b1.Hostname("foo") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("tcp"), + Protocol: gatewayv1b1.TCPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + Hostname: &hostname, + }, + } + }, + wantErrors: []string{"hostname must not be specified for protocols ['TCP', 'UDP']"}, + }, + { + desc: "hostname present with udp protocol", + mutate: func(gw *gatewayv1b1.Gateway) { + hostname := gatewayv1b1.Hostname("foo") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("udp"), + Protocol: gatewayv1b1.UDPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + Hostname: &hostname, + }, + } + }, + wantErrors: []string{"hostname must not be specified for protocols ['TCP', 'UDP']"}, + }, + { + desc: "certificateRefs not set with https protocol and TLS terminate mode", + mutate: func(gw *gatewayv1b1.Gateway) { + tlsMode := gatewayv1b1.TLSModeType("Terminate") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("https"), + Protocol: gatewayv1b1.HTTPSProtocolType, + Port: gatewayv1b1.PortNumber(8443), + TLS: &gatewayv1b1.GatewayTLSConfig{ + Mode: &tlsMode, + }, + }, + } + }, + wantErrors: []string{"certificateRefs must be specified when TLSModeType is Terminate"}, + }, + { + desc: "certificateRefs not set with tls protocol and TLS terminate mode", + mutate: func(gw *gatewayv1b1.Gateway) { + tlsMode := gatewayv1b1.TLSModeType("Terminate") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("tls"), + Protocol: gatewayv1b1.TLSProtocolType, + Port: gatewayv1b1.PortNumber(8443), + TLS: &gatewayv1b1.GatewayTLSConfig{ + Mode: &tlsMode, + }, + }, + } + }, + wantErrors: []string{"certificateRefs must be specified when TLSModeType is Terminate"}, + }, + { + desc: "certificateRefs set with tls protocol and TLS terminate mode", + mutate: func(gw *gatewayv1b1.Gateway) { + tlsMode := gatewayv1b1.TLSModeType("Terminate") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("tls"), + Protocol: gatewayv1b1.TLSProtocolType, + Port: gatewayv1b1.PortNumber(8443), + TLS: &gatewayv1b1.GatewayTLSConfig{ + Mode: &tlsMode, + CertificateRefs: []gatewayv1b1.SecretObjectReference{ + {Name: gatewayv1b1.ObjectName("foo")}, + }, + }, + }, + } + }, + }, + { + desc: "names are not unique within the Gateway", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("http"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + }, + { + Name: gatewayv1b1.SectionName("http"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8000), + }, + { + Name: gatewayv1b1.SectionName("http"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + }, + } + }, + wantErrors: []string{"Listener name must be unique within the Gateway"}, + }, + { + desc: "names are unique within the Gateway", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("http-1"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + }, + { + Name: gatewayv1b1.SectionName("http-2"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8000), + }, + { + Name: gatewayv1b1.SectionName("http-3"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8080), + }, + } + }, + }, + { + desc: "combination of port, protocol, and hostname are not unique for each listener", + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("foo"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + Hostname: &hostnameFoo, + }, + { + Name: gatewayv1b1.SectionName("bar"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + Hostname: &hostnameFoo, + }, + } + }, + wantErrors: []string{"Combination of port, protocol and hostname must be unique for each listener"}, + }, + { + desc: "combination of port and protocol are not unique for each listener when hostnames not set", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("foo"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + }, + { + Name: gatewayv1b1.SectionName("bar"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + }, + } + }, + wantErrors: []string{"Combination of port, protocol and hostname must be unique for each listener"}, + }, + { + desc: "port is unique when protocol and hostname are the same", + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("foo"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + Hostname: &hostnameFoo, + }, + { + Name: gatewayv1b1.SectionName("bar"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8000), + Hostname: &hostnameFoo, + }, + } + }, + }, + { + desc: "hostname is unique when protocol and port are the same", + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + hostnameBar := gatewayv1b1.Hostname("bar.com") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("foo"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + Hostname: &hostnameFoo, + }, + { + Name: gatewayv1b1.SectionName("bar"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(80), + Hostname: &hostnameBar, + }, + } + }, + }, + { + desc: "protocol is unique when port and hostname are the same", + mutate: func(gw *gatewayv1b1.Gateway) { + hostnameFoo := gatewayv1b1.Hostname("foo.com") + gw.Spec.Listeners = []gatewayv1b1.Listener{ + { + Name: gatewayv1b1.SectionName("foo"), + Protocol: gatewayv1b1.HTTPProtocolType, + Port: gatewayv1b1.PortNumber(8000), + Hostname: &hostnameFoo, + }, + { + Name: gatewayv1b1.SectionName("bar"), + Protocol: gatewayv1b1.HTTPSProtocolType, + Port: gatewayv1b1.PortNumber(8000), + Hostname: &hostnameFoo, + TLS: &gatewayv1b1.GatewayTLSConfig{ + CertificateRefs: []gatewayv1b1.SecretObjectReference{ + {Name: gatewayv1b1.ObjectName("foo")}, + }, + }, + }, + } + }, + }, + { + desc: "ip address and hostname in addresses are valid", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Addresses = []gatewayv1b1.GatewayAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4", + }, + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1111:2222:3333:4444::", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "foo.bar", + }, + } + }, + }, + { + desc: "ip address and hostname in addresses are invalid", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Addresses = []gatewayv1b1.GatewayAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4:8080", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "*foo/bar", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "12:34:56::", + }, + } + }, + wantErrors: []string{"Invalid value: \"1.2.3.4:8080\": spec.addresses[0].value in body must be of type ipv4"}, + }, + { + desc: "ip address and hostname in status addresses are valid", + mutateStatus: func(gw *gatewayv1b1.Gateway) { + gw.Status.Addresses = []gatewayv1b1.GatewayStatusAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4", + }, + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1111:2222:3333:4444::", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "foo.bar", + }, + } + }, + }, + { + desc: "ip address and hostname in status addresses are invalid", + mutateStatus: func(gw *gatewayv1b1.Gateway) { + gw.Status.Addresses = []gatewayv1b1.GatewayStatusAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4:8080", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "*foo/bar", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "12:34:56::", + }, + } + }, + wantErrors: []string{"Invalid value: \"1.2.3.4:8080\": status.addresses[0].value in body must be of type ipv4"}, + }, + { + desc: "duplicate ip address or hostname", + mutate: func(gw *gatewayv1b1.Gateway) { + gw.Spec.Addresses = []gatewayv1b1.GatewayAddress{ + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4", + }, + { + Type: ptrTo(gatewayv1b1.IPAddressType), + Value: "1.2.3.4", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "foo.bar", + }, + { + Type: ptrTo(gatewayv1b1.HostnameAddressType), + Value: "foo.bar", + }, + } + }, + wantErrors: []string{"IPAddress values must be unique", "Hostname values must be unique"}, + }, + } + + for _, tc := range testCases { + t.Run(tc.desc, func(t *testing.T) { + gw := baseGateway.DeepCopy() + gw.Name = fmt.Sprintf("foo-%v", time.Now().UnixNano()) + + if tc.mutate != nil { + tc.mutate(gw) + } + err := k8sClient.Create(ctx, gw) + + if tc.mutateStatus != nil { + tc.mutateStatus(gw) + err = k8sClient.Status().Update(ctx, gw) + } + + if (len(tc.wantErrors) != 0) != (err != nil) { + t.Fatalf("Unexpected response while creating Gateway; got err=\n%v\n;want error=%v", err, tc.wantErrors != nil) + } + + var missingErrorStrings []string + for _, wantError := range tc.wantErrors { + if !strings.Contains(strings.ToLower(err.Error()), strings.ToLower(wantError)) { + missingErrorStrings = append(missingErrorStrings, wantError) + } + } + if len(missingErrorStrings) != 0 { + t.Errorf("Unexpected response while creating Gateway; got err=\n%v\n;missing strings within error=%q", err, missingErrorStrings) + } + }) + } +} diff --git a/hack/cel-validation/gatewayclass_test.go b/hack/cel-validation/gatewayclass_test.go new file mode 100644 index 0000000000..90d9c7de93 --- /dev/null +++ b/hack/cel-validation/gatewayclass_test.go @@ -0,0 +1,80 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package main + +import ( + "context" + "fmt" + "strings" + "testing" + "time" + + gatewayv1b1 "sigs.k8s.io/gateway-api/apis/v1beta1" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +func TestValidateGatewayClassUpdate(t *testing.T) { + ctx := context.Background() + baseGatewayClass := gatewayv1b1.GatewayClass{ + ObjectMeta: metav1.ObjectMeta{ + Name: "foo", + }, + Spec: gatewayv1b1.GatewayClassSpec{ + ControllerName: "example.net/gateway-controller", + }, + } + + testCases := []struct { + desc string + creationMutate func(gw *gatewayv1b1.GatewayClass) + updationMutate func(gw *gatewayv1b1.GatewayClass) + wantError string + }{ + { + desc: "cannot upgrade controllerName", + creationMutate: func(gwc *gatewayv1b1.GatewayClass) { + gwc.Spec.ControllerName = "example.net/gateway-controller-1" + }, + updationMutate: func(gwc *gatewayv1b1.GatewayClass) { + gwc.Spec.ControllerName = "example.net/gateway-controller-2" + }, + wantError: "Value is immutable", + }, + } + + for _, tc := range testCases { + t.Run(tc.desc, func(t *testing.T) { + gwc := baseGatewayClass.DeepCopy() + gwc.Name = fmt.Sprintf("foo-%v", time.Now().UnixNano()) + + tc.creationMutate(gwc) + if err := k8sClient.Create(ctx, gwc); err != nil { + t.Fatalf("Failed to create GatewayClass: %v", err) + } + tc.updationMutate(gwc) + err := k8sClient.Update(ctx, gwc) + + if (tc.wantError != "") != (err != nil) { + t.Fatalf("Unexpected error while updating GatewayClass; got err=\n%v\n;want error=%v", err, tc.wantError != "") + } + if tc.wantError != "" && !strings.Contains(strings.ToLower(err.Error()), strings.ToLower(tc.wantError)) { + t.Fatalf("Unexpected error while updating GatewayClass; got err=\n%v\n;want substring within error=%q", err, tc.wantError) + } + }) + } +} diff --git a/hack/cel-validation/httproute_test.go b/hack/cel-validation/httproute_test.go new file mode 100644 index 0000000000..9af3b84367 --- /dev/null +++ b/hack/cel-validation/httproute_test.go @@ -0,0 +1,1393 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package main + +import ( + "context" + "fmt" + "strings" + "testing" + "time" + + gatewayv1b1 "sigs.k8s.io/gateway-api/apis/v1beta1" + + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +//////////////////////////////////////////////////////////////////////////////// +//////////////////////////////////////////////////////////////////////////////// +// +// How are tests named? Where to add new tests? +// +// Ensure that tests for newly added CEL validations are added in the correctly +// named test function. For example, if you added a test at the +// `HTTPRouteFilter` hierarchy (i.e. either at the struct level, or on one of +// the immediate descendent fields), then the test will go in the +// TestHTTPRouteFilter function. If the appropriate test function does not +// exist, please create one. +// +//////////////////////////////////////////////////////////////////////////////// +//////////////////////////////////////////////////////////////////////////////// + +func TestHTTPPathMatch(t *testing.T) { + tests := []struct { + name string + wantErrors []string + path *gatewayv1b1.HTTPPathMatch + }{ + { + name: "invalid because path does not start with '/'", + wantErrors: []string{"value must be an absolute path and start with '/' when type one of ['Exact', 'PathPrefix']"}, + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("foo"), + }, + }, + { + name: "invalid httpRoute prefix (/.)", + wantErrors: []string{"must not end with '/.' when type one of ['Exact', 'PathPrefix']"}, + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/."), + }, + }, + { + name: "invalid exact (/./)", + wantErrors: []string{"must not contain '/./' when type one of ['Exact', 'PathPrefix']"}, + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("Exact")), + Value: ptrTo("/foo/./bar"), + }, + }, + { + name: "invalid type", + wantErrors: []string{"type must be one of ['Exact', 'PathPrefix', 'RegularExpression']"}, + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("FooBar")), + Value: ptrTo("/path"), + }, + }, + { + name: "valid because type is RegularExpression but would not be valid for Exact", + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("RegularExpression")), + Value: ptrTo("/foo/./bar"), + }, + }, + { + name: "valid httpRoute prefix", + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/path"), + }, + }, + { + name: "valid path with some special characters", + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("Exact")), + Value: ptrTo("/abc/123'/a-b-c/d@gmail/%0A"), + }, + }, + { + name: "invalid prefix path (/[])", + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/[]"), + }, + wantErrors: []string{"must only contain valid characters (matching ^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$) for types ['Exact', 'PathPrefix']"}, + }, + { + name: "invalid exact path (/^)", + path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("Exact")), + Value: ptrTo("/^"), + }, + wantErrors: []string{"must only contain valid characters (matching ^(?:[-A-Za-z0-9/._~!$&'()*+,;=:@]|[%][0-9a-fA-F]{2})+$) for types ['Exact', 'PathPrefix']"}, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + route := &gatewayv1b1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("foo-%v", time.Now().UnixNano()), + Namespace: metav1.NamespaceDefault, + }, + Spec: gatewayv1b1.HTTPRouteSpec{ + Rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{{ + Path: tc.path, + }}, + BackendRefs: []gatewayv1b1.HTTPBackendRef{{ + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: gatewayv1b1.ObjectName("test"), + Port: ptrTo(gatewayv1b1.PortNumber(8080)), + }, + }, + }}, + }}, + }, + } + validateHTTPRoute(t, route, tc.wantErrors) + }) + } +} + +func TestBackendObjectReference(t *testing.T) { + portPtr := func(n int) *gatewayv1b1.PortNumber { + p := gatewayv1b1.PortNumber(n) + return &p + } + + groupPtr := func(g string) *gatewayv1b1.Group { + p := gatewayv1b1.Group(g) + return &p + } + + kindPtr := func(k string) *gatewayv1b1.Kind { + p := gatewayv1b1.Kind(k) + return &p + } + + tests := []struct { + name string + wantErrors []string + rules []gatewayv1b1.HTTPRouteRule + backendRef gatewayv1b1.BackendObjectReference + }{ + { + name: "default groupkind with port", + backendRef: gatewayv1b1.BackendObjectReference{ + Name: "backend", + Port: portPtr(99), + }, + }, + { + name: "default groupkind with no port", + wantErrors: []string{"Must have port for Service reference"}, + backendRef: gatewayv1b1.BackendObjectReference{ + Name: "backend", + }, + }, + { + name: "explicit service with port", + backendRef: gatewayv1b1.BackendObjectReference{ + Group: groupPtr(""), + Kind: kindPtr("Service"), + Name: "backend", + Port: portPtr(99), + }, + }, + { + name: "explicit service with no port", + wantErrors: []string{"Must have port for Service reference"}, + backendRef: gatewayv1b1.BackendObjectReference{ + Group: groupPtr(""), + Kind: kindPtr("Service"), + Name: "backend", + }, + }, + { + name: "explicit ref with no port", + backendRef: gatewayv1b1.BackendObjectReference{ + Group: groupPtr("foo.example.com"), + Kind: kindPtr("Foo"), + Name: "backend", + }, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + route := &gatewayv1b1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("foo-%v", time.Now().UnixNano()), + Namespace: metav1.NamespaceDefault, + }, + Spec: gatewayv1b1.HTTPRouteSpec{ + Rules: []gatewayv1b1.HTTPRouteRule{{ + BackendRefs: []gatewayv1b1.HTTPBackendRef{{ + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: tc.backendRef, + }, + }}, + }}, + }, + } + validateHTTPRoute(t, route, tc.wantErrors) + }) + } +} + +func TestHTTPRouteFilter(t *testing.T) { + tests := []struct { + name string + wantErrors []string + routeFilter gatewayv1b1.HTTPRouteFilter + }{ + { + name: "valid HTTPRouteFilterRequestHeaderModifier route filter", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Set: []gatewayv1b1.HTTPHeader{{Name: "name", Value: "foo"}}, + Add: []gatewayv1b1.HTTPHeader{{Name: "add", Value: "foo"}}, + Remove: []string{"remove"}, + }, + }, + }, + { + name: "invalid HTTPRouteFilterRequestHeaderModifier type filter with non-matching field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{}, + }, + wantErrors: []string{"filter.requestHeaderModifier must be specified for RequestHeaderModifier filter.type", "filter.requestMirror must be nil if the filter.type is not RequestMirror"}, + }, + { + name: "invalid HTTPRouteFilterRequestHeaderModifier type filter with empty value field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + }, + wantErrors: []string{"filter.requestHeaderModifier must be specified for RequestHeaderModifier filter.type"}, + }, + { + name: "valid HTTPRouteFilterRequestMirror route filter", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestMirror, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{BackendRef: gatewayv1b1.BackendObjectReference{ + Group: ptrTo(gatewayv1b1.Group("group")), + Kind: ptrTo(gatewayv1b1.Kind("kind")), + Name: "name", + Namespace: ptrTo(gatewayv1b1.Namespace("ns")), + Port: ptrTo(gatewayv1b1.PortNumber(22)), + }}, + }, + }, + { + name: "invalid HTTPRouteFilterRequestMirror type filter with non-matching field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestMirror, + RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{}, + }, + wantErrors: []string{"filter.requestHeaderModifier must be nil if the filter.type is not RequestHeaderModifier", "filter.requestMirror must be specified for RequestMirror filter.type"}, + }, + { + name: "invalid HTTPRouteFilterRequestMirror type filter with empty value field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestMirror, + }, + wantErrors: []string{"filter.requestMirror must be specified for RequestMirror filter.type"}, + }, + { + name: "valid HTTPRouteFilterRequestRedirect route filter", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Scheme: ptrTo("http"), + Hostname: ptrTo(gatewayv1b1.PreciseHostname("hostname")), + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.FullPathHTTPPathModifier, + ReplaceFullPath: ptrTo("path"), + }, + Port: ptrTo(gatewayv1b1.PortNumber(8080)), + StatusCode: ptrTo(302), + }, + }, + }, + { + name: "invalid HTTPRouteFilterRequestRedirect type filter with non-matching field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{}, + }, + wantErrors: []string{"filter.requestMirror must be nil if the filter.type is not RequestMirror", "filter.requestRedirect must be specified for RequestRedirect filter.type"}, + }, + { + name: "invalid HTTPRouteFilterRequestRedirect type filter with empty value field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + }, + wantErrors: []string{"filter.requestRedirect must be specified for RequestRedirect filter.type"}, + }, + { + name: "valid HTTPRouteFilterExtensionRef filter", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterExtensionRef, + ExtensionRef: &gatewayv1b1.LocalObjectReference{ + Group: "group", + Kind: "kind", + Name: "name", + }, + }, + }, + { + name: "invalid HTTPRouteFilterExtensionRef type filter with non-matching field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterExtensionRef, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{}, + }, + wantErrors: []string{"filter.requestMirror must be nil if the filter.type is not RequestMirror", "filter.extensionRef must be specified for ExtensionRef filter.type"}, + }, + { + name: "invalid HTTPRouteFilterExtensionRef type filter with empty value field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterExtensionRef, + }, + wantErrors: []string{"filter.extensionRef must be specified for ExtensionRef filter.type"}, + }, + { + name: "valid HTTPRouteFilterURLRewrite route filter", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Hostname: ptrTo(gatewayv1b1.PreciseHostname("hostname")), + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.FullPathHTTPPathModifier, + ReplaceFullPath: ptrTo("path"), + }, + }, + }, + }, + { + name: "invalid HTTPRouteFilterURLRewrite type filter with non-matching field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{}, + }, + wantErrors: []string{"filter.requestMirror must be nil if the filter.type is not RequestMirror", "filter.urlRewrite must be specified for URLRewrite filter.type"}, + }, + { + name: "invalid HTTPRouteFilterURLRewrite type filter with empty value field", + routeFilter: gatewayv1b1.HTTPRouteFilter{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + }, + wantErrors: []string{"filter.urlRewrite must be specified for URLRewrite filter.type"}, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + route := &gatewayv1b1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("foo-%v", time.Now().UnixNano()), + Namespace: metav1.NamespaceDefault, + }, + Spec: gatewayv1b1.HTTPRouteSpec{ + Rules: []gatewayv1b1.HTTPRouteRule{{ + Filters: []gatewayv1b1.HTTPRouteFilter{tc.routeFilter}, + }}, + }, + } + validateHTTPRoute(t, route, tc.wantErrors) + }) + } +} + +func TestHTTPRouteRule(t *testing.T) { + testService := gatewayv1b1.ObjectName("test-service") + tests := []struct { + name string + wantErrors []string + rules []gatewayv1b1.HTTPRouteRule + }{ + { + name: "valid httpRoute with no filters", + rules: []gatewayv1b1.HTTPRouteRule{ + { + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(8080)), + }, + Weight: ptrTo(int32(100)), + }, + }, + }, + }, + }, + }, + { + name: "valid httpRoute with 1 filter", + rules: []gatewayv1b1.HTTPRouteRule{ + { + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestMirror, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ + BackendRef: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(8081)), + }, + }, + }, + }, + }, + }, + }, + { + name: "valid httpRoute with duplicate ExtensionRef filters", + rules: []gatewayv1b1.HTTPRouteRule{ + { + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Set: []gatewayv1b1.HTTPHeader{ + { + Name: "special-header", + Value: "foo", + }, + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterRequestMirror, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ + BackendRef: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(8080)), + }, + }, + }, + { + Type: "ExtensionRef", + ExtensionRef: &gatewayv1b1.LocalObjectReference{ + Kind: "Service", + Name: "test", + }, + }, + { + Type: "ExtensionRef", + ExtensionRef: &gatewayv1b1.LocalObjectReference{ + Kind: "Service", + Name: "test", + }, + }, + { + Type: "ExtensionRef", + ExtensionRef: &gatewayv1b1.LocalObjectReference{ + Kind: "Service", + Name: "test", + }, + }, + }, + }, + }, + }, + { + name: "valid redirect path modifier", + rules: []gatewayv1b1.HTTPRouteRule{ + { + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.FullPathHTTPPathModifier, + ReplaceFullPath: ptrTo("foo"), + }, + }, + }, + }, + }, + }, + }, + { + name: "valid rewrite path modifier", + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{{ + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/bar"), + }, + }}, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "multiple actions for different request headers", + rules: []gatewayv1b1.HTTPRouteRule{{ + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Add: []gatewayv1b1.HTTPHeader{ + { + Name: gatewayv1b1.HTTPHeaderName("x-vegetable"), + Value: "carrot", + }, + { + Name: gatewayv1b1.HTTPHeaderName("x-grain"), + Value: "rye", + }, + }, + Set: []gatewayv1b1.HTTPHeader{ + { + Name: gatewayv1b1.HTTPHeaderName("x-fruit"), + Value: "watermelon", + }, + { + Name: gatewayv1b1.HTTPHeaderName("x-spice"), + Value: "coriander", + }, + }, + }, + }}, + }}, + }, + { + name: "multiple actions for different response headers", + rules: []gatewayv1b1.HTTPRouteRule{{ + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterResponseHeaderModifier, + ResponseHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Add: []gatewayv1b1.HTTPHeader{{ + Name: gatewayv1b1.HTTPHeaderName("x-example"), + Value: "blueberry", + }}, + Set: []gatewayv1b1.HTTPHeader{{ + Name: gatewayv1b1.HTTPHeaderName("x-different"), + Value: "turnip", + }}, + }, + }}, + }}, + }, + { + name: "backendref with request redirect httpRoute filter", + wantErrors: []string{"RequestRedirect filter must not be used together with backendRefs"}, + rules: []gatewayv1b1.HTTPRouteRule{ + { + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Scheme: ptrTo("https"), + StatusCode: ptrTo(301), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + }, + }, + }, + }, + }, + { + name: "request redirect without backendref in httpRoute filter", + rules: []gatewayv1b1.HTTPRouteRule{ + { + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Scheme: ptrTo("https"), + StatusCode: ptrTo(301), + }, + }, + }, + }, + }, + }, + { + name: "backendref without request redirect filter", + rules: []gatewayv1b1.HTTPRouteRule{ + { + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Set: []gatewayv1b1.HTTPHeader{{Name: "name", Value: "foo"}}, + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + }, + }, + }, + }, + }, + { + name: "backendref without any filter", + rules: []gatewayv1b1.HTTPRouteRule{ + { + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + }, + }, + }, + }, + }, + { + name: "valid use of URLRewrite filter", + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "invalid URLRewrite filter because too many path matches", + wantErrors: []string{"When using URLRewrite filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + { // Cannot have multiple path matches. + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/bar"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "invalid URLRewrite filter because too many path matches", + wantErrors: []string{"When using URLRewrite filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType(gatewayv1b1.FullPathHTTPPathModifier)), // Incorrect Patch match Type for URLRewrite filter with ReplacePrefixMatch. + Value: ptrTo("/foo"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "valid use of RequestRedirect filter", + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "invalid RequestRedirect filter because too many path matches", + wantErrors: []string{"When using RequestRedirect filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + { // Cannot have multiple path matches. + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/bar"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "invalid RequestRedirect filter because path match has type ReplaceFullPath", + wantErrors: []string{"When using RequestRedirect filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType(gatewayv1b1.FullPathHTTPPathModifier)), // Incorrect Patch match Type for RequestRedirect filter with ReplacePrefixMatch. + Value: ptrTo("/foo"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "valid use of URLRewrite filter (within backendRefs)", + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }, + }, + }}, + }, + { + name: "invalid URLRewrite filter (within backendRefs) because too many path matches", + wantErrors: []string{"Within backendRefs, When using URLRewrite filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + { // Cannot have multiple path matches. + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/bar"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }, + }, + }}, + }, + { + name: "invalid URLRewrite filter (within backendRefs) because path match has type ReplaceFullPath", + wantErrors: []string{"Within backendRefs, When using URLRewrite filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType(gatewayv1b1.FullPathHTTPPathModifier)), // Incorrect Patch match Type for URLRewrite filter with ReplacePrefixMatch. + Value: ptrTo("/foo"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }, + }, + }}, + }, + { + name: "valid use of RequestRedirect filter (within backendRefs)", + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }, + }, + }}, + }, + { + name: "invalid RequestRedirect filter (within backendRefs) because too many path matches", + wantErrors: []string{"Within backendRefs, when using RequestRedirect filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/foo"), + }, + }, + { // Cannot have multiple path matches. + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchPathPrefix), + Value: ptrTo("/bar"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }, + }, + }}, + }, + { + name: "invalid RequestRedirect filter (within backendRefs) because path match has type ReplaceFullPath", + wantErrors: []string{"Within backendRefs, when using RequestRedirect filter with path.replacePrefixMatch, exactly one PathPrefix match must be specified"}, + rules: []gatewayv1b1.HTTPRouteRule{{ + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType(gatewayv1b1.FullPathHTTPPathModifier)), // Incorrect Patch match Type for RequestRedirect filter with ReplacePrefixMatch. + Value: ptrTo("/foo"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }, + }, + }}, + }, + { + name: "rewrite and redirect filters combined (invalid)", + wantErrors: []string{"May specify either httpRouteFilterRequestRedirect or httpRouteFilterRequestRewrite, but not both"}, // errCount: 3, + rules: []gatewayv1b1.HTTPRouteRule{{ + Filters: []gatewayv1b1.HTTPRouteFilter{{ + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }, { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }}, + }}, + }, + { + name: "invalid because repeated URLRewrite filter", + wantErrors: []string{"URLRewrite filter cannot be repeated"}, + rules: []gatewayv1b1.HTTPRouteRule{ + { + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("bar"), + }, + }, + }, + }, + }, + }, + }, + { + name: "invalid because repeated RequestHeaderModifier filter among mix of filters", + wantErrors: []string{"RequestHeaderModifier filter cannot be repeated"}, + rules: []gatewayv1b1.HTTPRouteRule{ + { + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Set: []gatewayv1b1.HTTPHeader{ + { + Name: "special-header", + Value: "foo", + }, + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterRequestMirror, + RequestMirror: &gatewayv1b1.HTTPRequestMirrorFilter{ + BackendRef: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(8080)), + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterRequestHeaderModifier, + RequestHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Add: []gatewayv1b1.HTTPHeader{ + { + Name: "my-header", + Value: "bar", + }, + }, + }, + }, + }, + }, + }, + }, + { + name: "invalid because multiple filters are repeated", + wantErrors: []string{"ResponseHeaderModifier filter cannot be repeated", "RequestRedirect filter cannot be repeated"}, + rules: []gatewayv1b1.HTTPRouteRule{ + { + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/"), + }, + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterResponseHeaderModifier, + ResponseHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Set: []gatewayv1b1.HTTPHeader{ + { + Name: "special-header", + Value: "foo", + }, + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterResponseHeaderModifier, + ResponseHeaderModifier: &gatewayv1b1.HTTPHeaderFilter{ + Add: []gatewayv1b1.HTTPHeader{ + { + Name: "my-header", + Value: "bar", + }, + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.FullPathHTTPPathModifier, + ReplaceFullPath: ptrTo("foo"), + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.FullPathHTTPPathModifier, + ReplaceFullPath: ptrTo("bar"), + }, + }, + }, + }, + }, + }, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + route := &gatewayv1b1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("foo-%v", time.Now().UnixNano()), + Namespace: metav1.NamespaceDefault, + }, + Spec: gatewayv1b1.HTTPRouteSpec{Rules: tc.rules}, + } + validateHTTPRoute(t, route, tc.wantErrors) + }) + } +} + +func TestHTTPBackendRef(t *testing.T) { + testService := gatewayv1b1.ObjectName("test-service") + tests := []struct { + name string + wantErrors []string + rules []gatewayv1b1.HTTPRouteRule + }{ + { + name: "invalid because repeated URLRewrite filter within backendRefs", + wantErrors: []string{"URLRewrite filter cannot be repeated"}, + rules: []gatewayv1b1.HTTPRouteRule{ + { + Matches: []gatewayv1b1.HTTPRouteMatch{ + { + Path: &gatewayv1b1.HTTPPathMatch{ + Type: ptrTo(gatewayv1b1.PathMatchType("PathPrefix")), + Value: ptrTo("/"), + }, + }, + }, + BackendRefs: []gatewayv1b1.HTTPBackendRef{ + { + BackendRef: gatewayv1b1.BackendRef{ + BackendObjectReference: gatewayv1b1.BackendObjectReference{ + Name: testService, + Port: ptrTo(gatewayv1b1.PortNumber(80)), + }, + }, + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("foo"), + }, + }, + }, + { + Type: gatewayv1b1.HTTPRouteFilterURLRewrite, + URLRewrite: &gatewayv1b1.HTTPURLRewriteFilter{ + Path: &gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("bar"), + }, + }, + }, + }, + }, + }, + }, + }, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + route := &gatewayv1b1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("foo-%v", time.Now().UnixNano()), + Namespace: metav1.NamespaceDefault, + }, + Spec: gatewayv1b1.HTTPRouteSpec{Rules: tc.rules}, + } + validateHTTPRoute(t, route, tc.wantErrors) + }) + } +} + +func TestHTTPPathModifier(t *testing.T) { + tests := []struct { + name string + wantErrors []string + pathModifier gatewayv1b1.HTTPPathModifier + }{ + { + name: "valid ReplaceFullPath", + pathModifier: gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.FullPathHTTPPathModifier, + ReplaceFullPath: ptrTo("foo"), + }, + }, + { + name: "replaceFullPath must be specified when type is set to 'ReplaceFullPath'", + wantErrors: []string{"replaceFullPath must be specified when type is set to 'ReplaceFullPath'"}, + pathModifier: gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.FullPathHTTPPathModifier, + }, + }, + { + name: "type must be 'ReplaceFullPath' when replaceFullPath is set", + wantErrors: []string{"type must be 'ReplaceFullPath' when replaceFullPath is set"}, + pathModifier: gatewayv1b1.HTTPPathModifier{ + ReplaceFullPath: ptrTo("foo"), + }, + }, + { + name: "valid ReplacePrefixMatch", + pathModifier: gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + ReplacePrefixMatch: ptrTo("/foo"), + }, + }, + { + name: "replacePrefixMatch must be specified when type is set to 'ReplacePrefixMatch'", + wantErrors: []string{"replacePrefixMatch must be specified when type is set to 'ReplacePrefixMatch'"}, + pathModifier: gatewayv1b1.HTTPPathModifier{ + Type: gatewayv1b1.PrefixMatchHTTPPathModifier, + }, + }, + { + name: "type must be 'ReplacePrefixMatch' when replacePrefixMatch is set", + wantErrors: []string{"type must be 'ReplacePrefixMatch' when replacePrefixMatch is set"}, + pathModifier: gatewayv1b1.HTTPPathModifier{ + ReplacePrefixMatch: ptrTo("/foo"), + }, + }, + } + + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + route := &gatewayv1b1.HTTPRoute{ + ObjectMeta: metav1.ObjectMeta{ + Name: fmt.Sprintf("foo-%v", time.Now().UnixNano()), + Namespace: metav1.NamespaceDefault, + }, + Spec: gatewayv1b1.HTTPRouteSpec{ + Rules: []gatewayv1b1.HTTPRouteRule{ + { + Filters: []gatewayv1b1.HTTPRouteFilter{ + { + Type: gatewayv1b1.HTTPRouteFilterRequestRedirect, + RequestRedirect: &gatewayv1b1.HTTPRequestRedirectFilter{ + Path: &tc.pathModifier, + }, + }, + }, + }, + }, + }, + } + validateHTTPRoute(t, route, tc.wantErrors) + }) + } +} + +func validateHTTPRoute(t *testing.T, route *gatewayv1b1.HTTPRoute, wantErrors []string) { + t.Helper() + + ctx := context.Background() + err := k8sClient.Create(ctx, route) + + if (len(wantErrors) != 0) != (err != nil) { + t.Fatalf("Unexpected response while creating HTTPRoute %q; got err=\n%v\n;want error=%v", fmt.Sprintf("%v/%v", route.Namespace, route.Name), err, wantErrors != nil) + } + + var missingErrorStrings []string + for _, wantError := range wantErrors { + if !strings.Contains(strings.ToLower(err.Error()), strings.ToLower(wantError)) { + missingErrorStrings = append(missingErrorStrings, wantError) + } + } + if len(missingErrorStrings) != 0 { + t.Errorf("Unexpected response while creating HTTPRoute %q; got err=\n%v\n;missing strings within error=%q", fmt.Sprintf("%v/%v", route.Namespace, route.Name), err, missingErrorStrings) + } +} diff --git a/hack/cel-validation/main_test.go b/hack/cel-validation/main_test.go new file mode 100644 index 0000000000..73d3e6b4a7 --- /dev/null +++ b/hack/cel-validation/main_test.go @@ -0,0 +1,57 @@ +/* +Copyright 2023 The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package main + +import ( + "fmt" + "os" + "path" + "testing" + + "sigs.k8s.io/gateway-api/apis/v1alpha2" + "sigs.k8s.io/gateway-api/apis/v1beta1" + + "k8s.io/client-go/tools/clientcmd" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +var k8sClient client.Client + +func TestMain(m *testing.M) { + kubeconfig := os.Getenv("KUBECONFIG") + if kubeconfig == "" { + kubeconfig = path.Join(os.Getenv("HOME"), ".kube/config") + } + + restConfig, err := clientcmd.BuildConfigFromFlags("", kubeconfig) + if err != nil { + panic(fmt.Sprintf("Failed to get restConfig from BuildConfigFromFlags: %v", err)) + } + + k8sClient, err = client.New(restConfig, client.Options{}) + if err != nil { + panic(fmt.Sprintf("Error initializing Kubernetes client: %v", err)) + } + v1alpha2.AddToScheme(k8sClient.Scheme()) + v1beta1.AddToScheme(k8sClient.Scheme()) + + os.Exit(m.Run()) +} + +func ptrTo[T any](a T) *T { + return &a +} diff --git a/hack/invalid-examples/v1alpha2/httproute/invalid-filter-duplicate.yaml b/hack/invalid-examples/v1alpha2/httproute/invalid-filter-duplicate.yaml index 05d8f84443..016c8afbf2 100644 --- a/hack/invalid-examples/v1alpha2/httproute/invalid-filter-duplicate.yaml +++ b/hack/invalid-examples/v1alpha2/httproute/invalid-filter-duplicate.yaml @@ -8,11 +8,11 @@ spec: - type: RequestHeaderModifier requestHeaderModifier: add: - - name: my-header - value: foo + - name: my-header + value: foo - type: RequestHeaderModifier requestHeaderModifier: add: - - name: my-header - value: bar + - name: my-header + value: bar diff --git a/hack/invalid-examples/v1beta1/gateway/invalid-addresses.yaml b/hack/invalid-examples/v1beta1/gateway/invalid-addresses.yaml new file mode 100644 index 0000000000..caa52c7acf --- /dev/null +++ b/hack/invalid-examples/v1beta1/gateway/invalid-addresses.yaml @@ -0,0 +1,28 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: Gateway +metadata: + name: invalid-addresses +spec: + gatewayClassName: acme-lb + addresses: + - value: 1200:0000:::AB00:1234:0000:2552:7777:1313 + - value: 21DA:D3:0:2F3B:2AY:FF:FE28:9C5A + - value: "2001:db8:3c4d:15:0:d234:3eee:" + - value: "2001:db8:3c4d:15:0:d234:3eee:::" + - value: ":::1234::" + - value: "1.1.1" + - value: "1.a.3.4" + - value: "foo.com" + - type: IPAddress + value: "256.255.255.255" + - type: "Hostname" + value: "foo.com:80" + - type: "example.com/custom" + value: "anything goes" + listeners: + - protocol: HTTP + port: 80 + name: prod-web-gw + allowedRoutes: + namespaces: + from: Same diff --git a/hack/invalid-examples/v1beta1/httproute/invalid-filter-duplicate.yaml b/hack/invalid-examples/v1beta1/httproute/invalid-filter-duplicate.yaml index f7034f8a90..ba3be3fb41 100644 --- a/hack/invalid-examples/v1beta1/httproute/invalid-filter-duplicate.yaml +++ b/hack/invalid-examples/v1beta1/httproute/invalid-filter-duplicate.yaml @@ -8,11 +8,11 @@ spec: - type: RequestHeaderModifier requestHeaderModifier: add: - - name: my-header - value: foo + - name: my-header + value: foo - type: RequestHeaderModifier requestHeaderModifier: add: - - name: my-header - value: bar + - name: my-header + value: bar diff --git a/hack/invalid-examples/v1beta1/httproute/invalid-path-alphanum-specialchars-mix.yaml b/hack/invalid-examples/v1beta1/httproute/invalid-path-alphanum-specialchars-mix.yaml new file mode 100644 index 0000000000..7a1e34b2e7 --- /dev/null +++ b/hack/invalid-examples/v1beta1/httproute/invalid-path-alphanum-specialchars-mix.yaml @@ -0,0 +1,10 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: invalid-path-alphanum-specialchars-mix +spec: + rules: + - matches: + - path: + type: PathPrefix + value: /my[/]path01 diff --git a/hack/invalid-examples/v1beta1/httproute/invalid-path-specialchars.yaml b/hack/invalid-examples/v1beta1/httproute/invalid-path-specialchars.yaml new file mode 100644 index 0000000000..2d4b255791 --- /dev/null +++ b/hack/invalid-examples/v1beta1/httproute/invalid-path-specialchars.yaml @@ -0,0 +1,10 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: invalid-path-specialchars +spec: + rules: + - matches: + - path: + type: PathPrefix + value: /[] diff --git a/hack/invalid-examples/v1beta1/httproute/invalid-request-redirect-with-backendref.yaml b/hack/invalid-examples/v1beta1/httproute/invalid-request-redirect-with-backendref.yaml new file mode 100644 index 0000000000..127463bc55 --- /dev/null +++ b/hack/invalid-examples/v1beta1/httproute/invalid-request-redirect-with-backendref.yaml @@ -0,0 +1,16 @@ +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: http-filter-rewrite +spec: + hostnames: + - rewrite.example + rules: + - filters: + - type: RequestRedirect + requestRedirect: + scheme: https + statusCode: 301 + backendRefs: + - name: example-svc + port: 80 diff --git a/hack/mkdocs-copy-geps.py b/hack/mkdocs-copy-geps.py new file mode 100644 index 0000000000..81fdc3a33c --- /dev/null +++ b/hack/mkdocs-copy-geps.py @@ -0,0 +1,24 @@ +# Copyright 2023 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import shutil +import logging +from mkdocs import plugins + +log = logging.getLogger('mkdocs') + +@plugins.event_priority(100) +def on_pre_build(config, **kwargs): + log.info("copying geps") + shutil.copytree("geps","site-src/geps", dirs_exist_ok=True) diff --git a/hack/update-webhook-yaml.sh b/hack/update-webhook-yaml.sh index 0f115b8887..af47137278 100755 --- a/hack/update-webhook-yaml.sh +++ b/hack/update-webhook-yaml.sh @@ -36,7 +36,7 @@ then for yaml in `ls config/webhook/*.yaml` do echo Replacing in $yaml - sed -i -E "s/admission-server:[a-z0-9\.-]+/admission-server:${BASE_REF}/g" $yaml + sed -i -E "s/image:.+admission-server:[a-z0-9\.-]+/image: registry.k8s.io\/gateway-api\/admission-server:${BASE_REF}/g" $yaml done else echo "No version requested with BASE_REF, nothing to do." diff --git a/hack/verify-examples-kind.sh b/hack/verify-examples-kind.sh index 5b8fd89fbd..6a372449a3 100755 --- a/hack/verify-examples-kind.sh +++ b/hack/verify-examples-kind.sh @@ -22,7 +22,7 @@ readonly GO111MODULE="on" readonly GOFLAGS="-mod=readonly" readonly GOPATH="$(mktemp -d)" readonly CLUSTER_NAME="verify-gateway-api" -readonly ADMISSION_WEBHOOK_VERSION="v0.6.0" +readonly LOCAL_IMAGE="registry.k8s.io/gateway-api/admission-server:latest" export KUBECONFIG="${GOPATH}/.kubeconfig" export GOFLAGS GO111MODULE GOPATH @@ -35,7 +35,7 @@ cleanup() { return fi - rm config/webhook/kustomization.yaml + rm -f config/webhook/kustomization.yaml if [ "${KIND_CREATE_ATTEMPTED:-}" = true ]; then kind delete cluster --name "${CLUSTER_NAME}" || true @@ -43,30 +43,53 @@ cleanup() { CLEANED_UP=true } -trap cleanup INT TERM +trap cleanup INT TERM EXIT # For exit code res=0 # Install kind -(cd $GOPATH && go install sigs.k8s.io/kind@v0.17.0) || res=$? +(cd $GOPATH && go install sigs.k8s.io/kind@v0.20.0) || res=$? # Create cluster KIND_CREATE_ATTEMPTED=true -kind create cluster --name "${CLUSTER_NAME}" || res=$? +kind create cluster --name "${CLUSTER_NAME}" + +# Verify CEL validations before installing webhook. +for CHANNEL in experimental standard; do + # Install CRDs. + kubectl apply -f "config/crd/${CHANNEL}/gateway*.yaml" + + # Run tests. + go test -timeout=120s -count=1 sigs.k8s.io/gateway-api/hack/cel-validation + + # Delete CRDs to reset environment. + kubectl delete -f "config/crd/${CHANNEL}/gateway*.yaml" +done cat <config/webhook/kustomization.yaml resources: - 0-namespace.yaml - certificate_config.yaml - admission_webhook.yaml -images: - - name: gcr.io/k8s-staging-gateway-api/admission-server:${ADMISSION_WEBHOOK_VERSION} - newTag: latest +patches: + - patch: |- + - op: replace + path: /spec/template/spec/containers/0/image + value: ${LOCAL_IMAGE} + - op: replace + path: /spec/template/spec/containers/0/imagePullPolicy + value: IfNotPresent + target: + group: apps + version: v1 + kind: Deployment + name: gateway-api-admission-server EOF # Install webhook -docker build -t gcr.io/k8s-staging-gateway-api/admission-server:latest . +docker build -t ${LOCAL_IMAGE} -f docker/Dockerfile.webhook . +kind load docker-image ${LOCAL_IMAGE} --name "${CLUSTER_NAME}" kubectl apply -k config/webhook/ # Wait for webhook to be ready diff --git a/hack/verify-golint.sh b/hack/verify-golint.sh index 14c157d4ad..c23ad4b6d1 100755 --- a/hack/verify-golint.sh +++ b/hack/verify-golint.sh @@ -18,7 +18,7 @@ set -o errexit set -o nounset set -o pipefail -readonly VERSION="v1.46.2" +readonly VERSION="v1.52.2" readonly KUBE_ROOT=$(dirname "${BASH_SOURCE}")/.. cd "${KUBE_ROOT}" diff --git a/hack/verify-yamllint.sh b/hack/verify-yamllint.sh new file mode 100755 index 0000000000..e8ceb02dc8 --- /dev/null +++ b/hack/verify-yamllint.sh @@ -0,0 +1,31 @@ +#!/bin/bash + +# Copyright 2014 The Kubernetes Authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +set -o errexit +set -o nounset +set -o pipefail + +readonly VERSION="1.29.0" +readonly KUBE_ROOT=$(dirname "${BASH_SOURCE}")/.. + +cd "${KUBE_ROOT}" + +# See configuration file in ${KUBE_ROOT}/.yamllint.yaml. +pip3 install yamllint==$VERSION + +yamllint -c .yamllint.yaml . + +# ex: ts=2 sw=2 et filetype=sh diff --git a/mkdocs.yml b/mkdocs.yml index 79c2b9a7b9..7baaeafdd7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -3,6 +3,10 @@ repo_url: https://github.com/kubernetes-sigs/gateway-api repo_name: kubernetes-sigs/gateway-api site_dir: site docs_dir: site-src +hooks: +- hack/mkdocs-copy-geps.py +watch: +- geps theme: name: material icon: @@ -24,34 +28,48 @@ plugins: redirect_maps: 'guides/getting-started.md': 'guides/index.md' 'contributing/community.md': 'contributing/index.md' + 'contributing/gamma.md': 'concepts/gamma.md#contributing' + - mermaid2 markdown_extensions: - admonition - meta - pymdownx.emoji: emoji_index: !!python/name:materialx.emoji.twemoji emoji_generator: !!python/name:materialx.emoji.to_svg + - pymdownx.details - pymdownx.highlight - pymdownx.inlinehilite - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.snippets - toc: permalink: true + - tables nav: - Overview: - Introduction: index.md - Concepts: API Overview: concepts/api-overview.md + GAMMA: concepts/gamma.md Conformance: concepts/conformance.md - Security Model: concepts/security-model.md Implementation Guidelines: concepts/guidelines.md + Roles and Personas: concepts/roles-and-personas.md + Service Facets: concepts/service-facets.md + Security Model: concepts/security-model.md + Use Cases: concepts/use-cases.md Versioning: concepts/versioning.md - Implementations: implementations.md - FAQ: faq.md + - Glossary: concepts/glossary.md - Guides: - Getting started: guides/index.md - Simple Gateway: guides/simple-gateway.md - HTTP routing: guides/http-routing.md - HTTP redirects and rewrites: guides/http-redirect-rewrite.md + - HTTP header modifier: guides/http-header-modifier.md - HTTP traffic splitting: guides/traffic-splitting.md - Cross-Namespace routing: guides/multiple-ns.md - TLS: guides/tls.md @@ -71,31 +89,41 @@ nav: - Overview: geps/overview.md - Declined: - geps/gep-735.md + - geps/gep-1282.md - Provisional: - - geps/gep-1426.md - geps/gep-1324.md - - geps/gep-1282.md + - geps/gep-1619.md + - geps/gep-1897.md + - geps/gep-1867.md + - Implementable: + - geps/gep-1742.md + - geps/gep-1686.md + - geps/gep-1426.md - Experimental: - - geps/gep-1323.md + - geps/gep-1748.md + - geps/gep-1651.md - geps/gep-1016.md - geps/gep-957.md - - geps/gep-726.md - geps/gep-713.md + - geps/gep-1709.md - Standard: - geps/gep-1364.md + - geps/gep-1323.md - geps/gep-922.md - geps/gep-917.md - geps/gep-851.md - geps/gep-820.md - geps/gep-746.md + - geps/gep-726.md - geps/gep-724.md - geps/gep-718.md - geps/gep-709.md + - Glossary: concepts/glossary.md - Contributing: - How to Get Involved: contributing/index.md - Developer Guide: contributing/devguide.md - Enhancement Requests: contributing/enhancement-requests.md - - GAMMA: contributing/gamma.md + - Contributor Ladder: contributing/contributor-ladder.md - Blog: - Index: blog/index.md - 2022: diff --git a/pkg/admission/server.go b/pkg/admission/server.go index ab072ff305..f3e21ac49b 100644 --- a/pkg/admission/server.go +++ b/pkg/admission/server.go @@ -151,7 +151,6 @@ func ServeHTTP(w http.ResponseWriter, r *http.Request) { } func handleValidation(request admission.AdmissionRequest) (*admission.AdmissionResponse, error) { - var ( response admission.AdmissionResponse deserializer = codecs.UniversalDeserializer() diff --git a/pkg/admission/server_test.go b/pkg/admission/server_test.go index d896446e64..95e4d8ec04 100644 --- a/pkg/admission/server_test.go +++ b/pkg/admission/server_test.go @@ -235,7 +235,7 @@ func TestServeHTTPSubmissions(t *testing.T) { }, }, { - name: "invalid v1alpha2 HTTPRoute resource with two request mirror filters", + name: "valid v1alpha2 HTTPRoute resource with two request mirror filters", reqBody: dedent.Dedent(`{ "kind": "AdmissionReview", "apiVersion": "` + apiVersion + `", @@ -285,9 +285,9 @@ func TestServeHTTPSubmissions(t *testing.T) { } } ], - "forwardTo": [ + "backendRefs": [ { - "serviceName": "my-service1", + "name": "RequestMirror", "port": 8080 } ] @@ -301,11 +301,8 @@ func TestServeHTTPSubmissions(t *testing.T) { wantRespCode: http.StatusOK, wantSuccessResponse: admission.AdmissionResponse{ UID: "7313cd05-eddc-4150-b88c-971a0d53b2ab", - Allowed: false, - Result: &metav1.Status{ - Code: 400, - Message: "spec.rules[0].filters: Invalid value: \"RequestMirror\": cannot be used multiple times in the same rule", - }, + Allowed: true, + Result: &metav1.Status{}, }, }, { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gateway.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gateway.go index a6d0594831..2313be1bf3 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gateway.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gateway.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeGateways struct { ns string } -var gatewaysResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "gateways"} +var gatewaysResource = v1alpha2.SchemeGroupVersion.WithResource("gateways") -var gatewaysKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "Gateway"} +var gatewaysKind = v1alpha2.SchemeGroupVersion.WithKind("Gateway") // Get takes name of the gateway, and returns the corresponding gateway object, and an error if there is any. func (c *FakeGateways) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.Gateway, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gatewayclass.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gatewayclass.go index 25dcc9851c..606fe6d783 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gatewayclass.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_gatewayclass.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -35,9 +34,9 @@ type FakeGatewayClasses struct { Fake *FakeGatewayV1alpha2 } -var gatewayclassesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "gatewayclasses"} +var gatewayclassesResource = v1alpha2.SchemeGroupVersion.WithResource("gatewayclasses") -var gatewayclassesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "GatewayClass"} +var gatewayclassesKind = v1alpha2.SchemeGroupVersion.WithKind("GatewayClass") // Get takes name of the gatewayClass, and returns the corresponding gatewayClass object, and an error if there is any. func (c *FakeGatewayClasses) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.GatewayClass, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_grpcroute.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_grpcroute.go index 33650a5c58..6311067274 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_grpcroute.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_grpcroute.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeGRPCRoutes struct { ns string } -var grpcroutesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "grpcroutes"} +var grpcroutesResource = v1alpha2.SchemeGroupVersion.WithResource("grpcroutes") -var grpcroutesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "GRPCRoute"} +var grpcroutesKind = v1alpha2.SchemeGroupVersion.WithKind("GRPCRoute") // Get takes name of the gRPCRoute, and returns the corresponding gRPCRoute object, and an error if there is any. func (c *FakeGRPCRoutes) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.GRPCRoute, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_httproute.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_httproute.go index 7e73f537c6..cf3bf1b4ac 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_httproute.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_httproute.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeHTTPRoutes struct { ns string } -var httproutesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "httproutes"} +var httproutesResource = v1alpha2.SchemeGroupVersion.WithResource("httproutes") -var httproutesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "HTTPRoute"} +var httproutesKind = v1alpha2.SchemeGroupVersion.WithKind("HTTPRoute") // Get takes name of the hTTPRoute, and returns the corresponding hTTPRoute object, and an error if there is any. func (c *FakeHTTPRoutes) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.HTTPRoute, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_referencegrant.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_referencegrant.go index 86d6686cad..1089d6e1ca 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_referencegrant.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_referencegrant.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeReferenceGrants struct { ns string } -var referencegrantsResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "referencegrants"} +var referencegrantsResource = v1alpha2.SchemeGroupVersion.WithResource("referencegrants") -var referencegrantsKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "ReferenceGrant"} +var referencegrantsKind = v1alpha2.SchemeGroupVersion.WithKind("ReferenceGrant") // Get takes name of the referenceGrant, and returns the corresponding referenceGrant object, and an error if there is any. func (c *FakeReferenceGrants) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.ReferenceGrant, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tcproute.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tcproute.go index 6b5c3dc6a1..c6ecb38186 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tcproute.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tcproute.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeTCPRoutes struct { ns string } -var tcproutesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "tcproutes"} +var tcproutesResource = v1alpha2.SchemeGroupVersion.WithResource("tcproutes") -var tcproutesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "TCPRoute"} +var tcproutesKind = v1alpha2.SchemeGroupVersion.WithKind("TCPRoute") // Get takes name of the tCPRoute, and returns the corresponding tCPRoute object, and an error if there is any. func (c *FakeTCPRoutes) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.TCPRoute, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tlsroute.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tlsroute.go index 6d17c60bdc..f9c0115d5e 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tlsroute.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_tlsroute.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeTLSRoutes struct { ns string } -var tlsroutesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "tlsroutes"} +var tlsroutesResource = v1alpha2.SchemeGroupVersion.WithResource("tlsroutes") -var tlsroutesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "TLSRoute"} +var tlsroutesKind = v1alpha2.SchemeGroupVersion.WithKind("TLSRoute") // Get takes name of the tLSRoute, and returns the corresponding tLSRoute object, and an error if there is any. func (c *FakeTLSRoutes) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.TLSRoute, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_udproute.go b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_udproute.go index 1441f64108..88a139e78b 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_udproute.go +++ b/pkg/client/clientset/versioned/typed/apis/v1alpha2/fake/fake_udproute.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeUDPRoutes struct { ns string } -var udproutesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Resource: "udproutes"} +var udproutesResource = v1alpha2.SchemeGroupVersion.WithResource("udproutes") -var udproutesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1alpha2", Kind: "UDPRoute"} +var udproutesKind = v1alpha2.SchemeGroupVersion.WithKind("UDPRoute") // Get takes name of the uDPRoute, and returns the corresponding uDPRoute object, and an error if there is any. func (c *FakeUDPRoutes) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1alpha2.UDPRoute, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gateway.go b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gateway.go index f01ba0331c..722587f684 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gateway.go +++ b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gateway.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeGateways struct { ns string } -var gatewaysResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1beta1", Resource: "gateways"} +var gatewaysResource = v1beta1.SchemeGroupVersion.WithResource("gateways") -var gatewaysKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1beta1", Kind: "Gateway"} +var gatewaysKind = v1beta1.SchemeGroupVersion.WithKind("Gateway") // Get takes name of the gateway, and returns the corresponding gateway object, and an error if there is any. func (c *FakeGateways) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1beta1.Gateway, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gatewayclass.go b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gatewayclass.go index 1832406154..7215742cb6 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gatewayclass.go +++ b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_gatewayclass.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -35,9 +34,9 @@ type FakeGatewayClasses struct { Fake *FakeGatewayV1beta1 } -var gatewayclassesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1beta1", Resource: "gatewayclasses"} +var gatewayclassesResource = v1beta1.SchemeGroupVersion.WithResource("gatewayclasses") -var gatewayclassesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1beta1", Kind: "GatewayClass"} +var gatewayclassesKind = v1beta1.SchemeGroupVersion.WithKind("GatewayClass") // Get takes name of the gatewayClass, and returns the corresponding gatewayClass object, and an error if there is any. func (c *FakeGatewayClasses) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1beta1.GatewayClass, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_httproute.go b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_httproute.go index 2e022aa04b..93f4edf362 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_httproute.go +++ b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_httproute.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeHTTPRoutes struct { ns string } -var httproutesResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1beta1", Resource: "httproutes"} +var httproutesResource = v1beta1.SchemeGroupVersion.WithResource("httproutes") -var httproutesKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1beta1", Kind: "HTTPRoute"} +var httproutesKind = v1beta1.SchemeGroupVersion.WithKind("HTTPRoute") // Get takes name of the hTTPRoute, and returns the corresponding hTTPRoute object, and an error if there is any. func (c *FakeHTTPRoutes) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1beta1.HTTPRoute, err error) { diff --git a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_referencegrant.go b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_referencegrant.go index f63fc2512a..c103c1c10f 100644 --- a/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_referencegrant.go +++ b/pkg/client/clientset/versioned/typed/apis/v1beta1/fake/fake_referencegrant.go @@ -23,7 +23,6 @@ import ( v1 "k8s.io/apimachinery/pkg/apis/meta/v1" labels "k8s.io/apimachinery/pkg/labels" - schema "k8s.io/apimachinery/pkg/runtime/schema" types "k8s.io/apimachinery/pkg/types" watch "k8s.io/apimachinery/pkg/watch" testing "k8s.io/client-go/testing" @@ -36,9 +35,9 @@ type FakeReferenceGrants struct { ns string } -var referencegrantsResource = schema.GroupVersionResource{Group: "gateway.networking.k8s.io", Version: "v1beta1", Resource: "referencegrants"} +var referencegrantsResource = v1beta1.SchemeGroupVersion.WithResource("referencegrants") -var referencegrantsKind = schema.GroupVersionKind{Group: "gateway.networking.k8s.io", Version: "v1beta1", Kind: "ReferenceGrant"} +var referencegrantsKind = v1beta1.SchemeGroupVersion.WithKind("ReferenceGrant") // Get takes name of the referenceGrant, and returns the corresponding referenceGrant object, and an error if there is any. func (c *FakeReferenceGrants) Get(ctx context.Context, name string, options v1.GetOptions) (result *v1beta1.ReferenceGrant, err error) { diff --git a/pkg/generator/main.go b/pkg/generator/main.go index 252758c149..7f07fb6f04 100644 --- a/pkg/generator/main.go +++ b/pkg/generator/main.go @@ -35,8 +35,8 @@ const ( channelAnnotation = "gateway.networking.k8s.io/channel" // These values must be updated during the release process - bundleVersion = "v0.6.0" - approvalLink = "https://github.com/kubernetes-sigs/gateway-api/pull/1538" + bundleVersion = "v0.7.1-dev" + approvalLink = "https://github.com/kubernetes-sigs/gateway-api/pull/1923" ) var standardKinds = map[string]bool{ @@ -93,6 +93,7 @@ func main() { if channel == "standard" && !standardKinds[groupKind.Kind] { continue } + log.Printf("generating %s CRD for %v\n", channel, groupKind) parser.NeedCRDFor(groupKind, nil) @@ -111,7 +112,7 @@ func main() { channelCrd := crdRaw.DeepCopy() for _, version := range channelCrd.Spec.Versions { - version.Schema.OpenAPIV3Schema.Properties = channelTweaks(channel, version.Schema.OpenAPIV3Schema.Properties) + version.Schema.OpenAPIV3Schema.Properties = gatewayTweaks(channel, version.Schema.OpenAPIV3Schema.Properties) } conv, err := crd.AsVersion(*channelCrd, apiext.SchemeGroupVersion) @@ -133,9 +134,37 @@ func main() { } } -func channelTweaks(channel string, props map[string]apiext.JSONSchemaProps) map[string]apiext.JSONSchemaProps { +// Custom Gateway API Tweaks for tags prefixed with `") { + jsonProps.Items.Schema.OneOf = []apiext.JSONSchemaProps{{ + Properties: map[string]apiext.JSONSchemaProps{ + "type": { + Enum: []apiext.JSON{{Raw: []byte("\"IPAddress\"")}}, + }, + "value": { + AnyOf: []apiext.JSONSchemaProps{{ + Format: "ipv4", + }, { + Format: "ipv6", + }}, + }, + }, + }, { + Properties: map[string]apiext.JSONSchemaProps{ + "type": { + Not: &apiext.JSONSchemaProps{ + Enum: []apiext.JSON{{Raw: []byte("\"IPAddress\"")}}, + }, + }, + }, + }} + } + if channel == "standard" && strings.Contains(jsonProps.Description, "") { delete(props, name) continue @@ -158,13 +187,13 @@ func channelTweaks(channel string, props map[string]apiext.JSONSchemaProps) map[ } } - experimentalRe := regexp.MustCompile(``) - jsonProps.Description = experimentalRe.ReplaceAllLiteralString(jsonProps.Description, "") + gatewayRe := regexp.MustCompile(``) + jsonProps.Description = gatewayRe.ReplaceAllLiteralString(jsonProps.Description, "") if len(jsonProps.Properties) > 0 { - jsonProps.Properties = channelTweaks(channel, jsonProps.Properties) + jsonProps.Properties = gatewayTweaks(channel, jsonProps.Properties) } else if jsonProps.Items != nil && jsonProps.Items.Schema != nil { - jsonProps.Items.Schema.Properties = channelTweaks(channel, jsonProps.Items.Schema.Properties) + jsonProps.Items.Schema.Properties = gatewayTweaks(channel, jsonProps.Items.Schema.Properties) } props[name] = jsonProps } diff --git a/requirements.txt b/requirements.txt index 3338c6e30b..caf05dfac8 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,4 @@ -Click==8.1.3 +Click==8.1.6 htmlmin==0.1.12 Jinja2==3.1.2 jsmin==3.0.1 @@ -6,16 +6,17 @@ livereload==2.6.3 # mkdocs 2.4.1 requires Markdown < 3.4.0 # https://github.com/kubernetes-sigs/gateway-api/pull/1671#issuecomment-1400586465 Markdown==3.3.7 -MarkupSafe==2.1.2 -mkdocs==1.4.2 -mkdocs-awesome-pages-plugin==2.8.0 -mkdocs-macros-plugin==0.7.0 -mkdocs-material==9.0.8 +MarkupSafe==2.1.3 +mkdocs==1.5.2 +mkdocs-awesome-pages-plugin==2.9.1 +mkdocs-macros-plugin==1.0.2 +mkdocs-material==9.1.21 mkdocs-material-extensions==1.1.1 -mkdocs-redirects==1.2.0 +mkdocs-redirects==1.2.1 +mkdocs-mermaid2-plugin==1.0.6 pep562==1.1 -Pygments==2.14.0 -pymdown-extensions==9.9.2 -PyYAML==6.0 +Pygments==2.16.1 +pymdown-extensions==10.1 +PyYAML==6.0.1 six==1.16.0 -tornado==6.2 +tornado==6.3.2 diff --git a/site-src/OWNERS b/site-src/OWNERS new file mode 100644 index 0000000000..f866d12bd1 --- /dev/null +++ b/site-src/OWNERS @@ -0,0 +1,10 @@ +# See the OWNERS docs at https://go.k8s.io/owners +# See the OWNERS_ALIASES file at https://github.com/kubernetes-sigs/gateway-api/blob/main/OWNERS_ALIASES for a list of members for each alias. + +approvers: + - sig-network-leads + - gateway-api-maintainers + +reviewers: + - gateway-api-maintainers + - gateway-api-mesh-leads diff --git a/site-src/api-types/gateway.md b/site-src/api-types/gateway.md index e138b7abbc..d487880977 100644 --- a/site-src/api-types/gateway.md +++ b/site-src/api-types/gateway.md @@ -1,5 +1,9 @@ # Gateway +??? success "Standard Channel in v0.5.0+" + + The `Gateway` resource is Beta and part of the Standard Channel in `v0.5.0+`. + A `Gateway` is 1:1 with the life cycle of the configuration of infrastructure. When a user creates a `Gateway`, some load balancing infrastructure is provisioned or configured (see below for details) by the `GatewayClass` diff --git a/site-src/api-types/gatewayclass.md b/site-src/api-types/gatewayclass.md index 36c99a42c7..b4576923b5 100644 --- a/site-src/api-types/gatewayclass.md +++ b/site-src/api-types/gatewayclass.md @@ -1,5 +1,9 @@ # GatewayClass +??? success "Standard Channel in v0.5.0+" + + The `GatewayClass` resource is Beta and part of the Standard Channel in `v0.5.0+`. + [GatewayClass][gatewayclass] is cluster-scoped resource defined by the infrastructure provider. This resource represents a class of Gateways that can be instantiated. diff --git a/site-src/api-types/grpcroute.md b/site-src/api-types/grpcroute.md index ddc5f3fe60..a795e840fe 100644 --- a/site-src/api-types/grpcroute.md +++ b/site-src/api-types/grpcroute.md @@ -1,5 +1,9 @@ # GRPCRoute +??? example "Experimental Channel in v0.6.0+" + + The `GRPCRoute` resource is Alpha and part of the Experimental Channel in `v0.6.0+`. + !!! info "Experimental Channel" The `GRPCRoute` resource described below is currently only included in the @@ -184,10 +188,11 @@ Conformance levels are defined by the filter type: Specifying a core filter multiple times has unspecified or custom conformance. -All filters are expected to be compatible with each other. If an implementation -cannot support other combinations of filters, they must clearly document that -limitation. In all cases where incompatible or unsupported filters are -specified, implementations MUST add a warning condition to status. +If an implementation can not support a combinations of filters, they must clearly +document that limitation. In cases where incompatible or unsupported +filters are specified and cause the `Accepted` condition to be set to status +`False`, implementations may use the `IncompatibleFilters` reason to specify +this configuration error. #### BackendRefs (optional) diff --git a/site-src/api-types/httproute.md b/site-src/api-types/httproute.md index 3ae861ef63..88a0b86cd0 100644 --- a/site-src/api-types/httproute.md +++ b/site-src/api-types/httproute.md @@ -1,5 +1,9 @@ # HTTPRoute +??? success "Standard Channel in v0.5.0+" + + The `HTTPRoute` resource is Beta and part of the Standard Channel in `v0.5.0+`. + [HTTPRoute][httproute] is a Gateway API type for specifying routing behavior of HTTP requests from a Gateway listener to an API object, i.e. Service. @@ -81,14 +85,16 @@ Take the following matches configuration as an example: apiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute ... -matches: - - path: - value: "/foo" - headers: - values: - version: "2" - - path: - value: "/v2/foo" +spec: + rules: + - matches: + - path: + value: "/foo" + headers: + - name: "version" + value: "2" + - path: + value: "/v2/foo" ``` For a request to match against this rule, it must satisfy EITHER of the @@ -122,7 +128,7 @@ Conformance levels are defined by the filter type: - All "core" filters MUST be supported by implementations. - Implementers are encouraged to support "extended" filters. - - "Custom" filters have no API guarantees across implementations. + - "Implementation-specific" filters have no API guarantees across implementations. Specifying a core filter multiple times has unspecified or implementation-specific conformance. @@ -130,8 +136,10 @@ implementation-specific conformance. All filters are expected to be compatible with each other except for the URLRewrite and RequestRedirect filters, which may not be combined. If an implementation can not support other combinations of filters, they must clearly -document that limitation. In all cases where incompatible or unsupported -filters are specified, implementations MUST add a warning condition to status. +document that limitation. In cases where incompatible or unsupported +filters are specified and cause the `Accepted` condition to be set to status +`False`, implementations may use the `IncompatibleFilters` reason to specify +this configuration error. #### BackendRefs (optional) diff --git a/site-src/api-types/referencegrant.md b/site-src/api-types/referencegrant.md index e263cb8cf8..e9718b3a17 100644 --- a/site-src/api-types/referencegrant.md +++ b/site-src/api-types/referencegrant.md @@ -1,10 +1,8 @@ # ReferenceGrant -!!! info "Experimental Channel" +??? success "Standard Channel in v0.6.0+" - The `ReferenceGrant` resource described below is currently only included in the - "Experimental" channel of Gateway API. For more information on release - channels, refer to the [related documentation](https://gateway-api.sigs.k8s.io/concepts/versioning). + The `ReferenceGrant` resource is Beta and part of the Standard Channel in `v0.6.0+`. !!! note This resource was originally named "ReferencePolicy". It was renamed @@ -53,8 +51,7 @@ spec: rules: - matches: - path: /bar - forwardTo: - backend: + backendRefs: - name: bar namespace: bar --- @@ -101,7 +98,7 @@ When communicating the status of a cross-namespace reference, implementations MUST NOT expose information about the existence of a resource in another namespace unless a ReferenceGrant exists allowing the reference to occur. This means that if a cross-namespace reference is made without a ReferenceGrant to a -resource that doesn't exist. Any status conditions or warning messages need to +resource that doesn't exist, any status conditions or warning messages need to focus on the fact that a ReferenceGrant does not exist to allow this reference. No hints should be provided about whether or not the referenced resource exists. diff --git a/site-src/blog/2022/graduating-to-beta.md b/site-src/blog/2022/graduating-to-beta.md index ca6cecd3d7..84c37a0deb 100644 --- a/site-src/blog/2022/graduating-to-beta.md +++ b/site-src/blog/2022/graduating-to-beta.md @@ -146,7 +146,7 @@ possible integration. We are pleased to announce that the service mesh community, including representatives from Cilium Service Mesh, Consul, Istio, Kuma, Linkerd, NGINX Service Mesh and Open Service Mesh, is coming together to form the [GAMMA -Initiative](https://gateway-api.sigs.k8s.io/contributing/gamma/), a dedicated +Initiative](https://gateway-api.sigs.k8s.io/concepts/gamma/), a dedicated workstream within the Gateway API subproject focused on Gateway API for Mesh Management and Administration. @@ -169,13 +169,13 @@ As we continue to mature the API for production use cases, here are some of the - [Route delegation][pr1085] - Layer 4 API maturity: Graduating [TCPRoute][tcpr], [UDPRoute][udpr] and [TLSRoute][tlsr] to beta -- [GAMMA Initiative](https://gateway-api.sigs.k8s.io/contributing/gamma/) - Gateway API for Service Mesh +- [GAMMA Initiative](https://gateway-api.sigs.k8s.io/concepts/gamma/) - Gateway API for Service Mesh If there's something on this list you want to get involved in, or there's something not on this list that you want to advocate for to get on the roadmap please join us in the #sig-network-gateway-api channel on Kubernetes Slack or our weekly [community calls](https://gateway-api.sigs.k8s.io/contributing/community/#meetings). -[gep1016]:https://github.com/kubernetes-sigs/gateway-api/blob/master/site-src/geps/gep-1016.md +[gep1016]:https://github.com/kubernetes-sigs/gateway-api/blob/master/geps/gep-1016.md [grpc]:https://grpc.io/ [pr1085]:https://github.com/kubernetes-sigs/gateway-api/pull/1085 [tcpr]:https://github.com/kubernetes-sigs/gateway-api/blob/main/apis/v1alpha2/tcproute_types.go diff --git a/site-src/concepts/api-overview.md b/site-src/concepts/api-overview.md index 79effef97b..747f6430f3 100644 --- a/site-src/concepts/api-overview.md +++ b/site-src/concepts/api-overview.md @@ -2,18 +2,15 @@ This document provides an overview of Gateway API. -## Roles and personas. +## Roles and personas -There are 3 primary roles in Gateway API: +There are 3 primary roles in Gateway API, as described in [roles and personas]: -- Infrastructure Provider -- Cluster Operator -- Application Developer +- **Ian** (he/him): Infrastructure Provider +- **Chihiro** (they/them): Cluster Operator +- **Ana** (she/her): Application Developer -There could be a fourth role of Application Admin in some use cases. - -Please refer to the [roles and personas](/concepts/security-model#roles-and-personas) -section in the Security model for details. +[roles and personas]:/concepts/roles-and-personas ## Resource model @@ -86,6 +83,7 @@ that are implementation-specific are encouraged for other protocols. New route types may be added to the API in future. #### HTTPRoute + HTTPRoute is for multiplexing HTTP or terminated HTTPS connections. It's intended for use in cases where you want to inspect the HTTP stream and use HTTP request data for either routing or modification, for example using HTTP Headers for routing, or @@ -118,7 +116,7 @@ use to choose different backends on the same port, so each TCPRoute really needs different port on the listener (in general, anyway). You can terminate TLS in which case the unencrypted byte stream is passed through to the backend. You can choose to not terminate TLS, in which case the encrypted byte stream -is passed through to the backend. +is passed through to the backend. #### GRPCRoute @@ -133,6 +131,7 @@ GRPCRoute are required to support HTTP/2 without an initial upgrade from HTTP/1, so gRPC traffic is guaranteed to flow properly. #### Route summary table + The "Routing Discriminator" column below refers to what information can be used to allow multiple Routes to share ports on the Listener. @@ -150,7 +149,7 @@ to configure that with existing Gateway API resources, but implementations may provide custom configuration for this until there is a standardized approach defined by Gateway API. -### Attaching Routes to Gateways +## Attaching Routes to Gateways !!! note This section has changed significantly between v1alpha1 and v1alpha2. This @@ -177,20 +176,22 @@ different relationships that Gateways and Routes can have: ### Example -A Kubernetes cluster admin has deployed a Gateway `shared-gw` in the `Infra` -Namespace to be used by different application teams for exposing their -applications outside the cluster. Teams A and B (in Namespaces `A` and `B` -respectively) attach their Routes to this Gateway. They are unaware of each -other and as long as their Route rules do not conflict with each other they -can continue operating in isolation. Team C has special networking needs -(perhaps performance, security, or criticality) and they need a dedicated -Gateway to proxy their application to the outside world. Team C deploys their -own Gateway `dedicated-gw` in the `C` Namespace that can only be used by apps -in the `C` Namespace. +[Chihiro] has deployed a Gateway `shared-gw` in the `infra` Namespace to be +used by different application teams for exposing their applications outside +the cluster. Teams A and B (in Namespaces `A` and `B` respectively) attach +their Routes to this Gateway. They are unaware of each other and as long as +their Route rules do not conflict with each other they can continue operating +in isolation. Team C has special networking needs (perhaps performance, +security, or criticality) and they need a dedicated Gateway to proxy their +application to the outside world. Team C deploys their own Gateway +`dedicated-gw` in the `C` Namespace that can only be used by apps in the `C` +Namespace. ![route binding](/images/gateway-route-binding.png) +[Chihiro]:/concepts/roles-and-personas#Chihiro + ### How it Works The following is required for a Route to be attached to a Gateway: @@ -277,29 +278,52 @@ relationships between the different resources: ![schema](/images/schema-uml.svg) -## Request flow +### Request flow -A typical client/gateway API request flow for a gateway implemented using a +A typical [north/south] API request flow for a gateway implemented using a reverse proxy is: - 1. A client makes a request to http://foo.example.com. - 2. DNS resolves the name to a `Gateway` address. - 3. The reverse proxy receives the request on a `Listener` and uses the [Host - header](https://tools.ietf.org/html/rfc7230#section-5.4) to match an - `HTTPRoute`. - 5. Optionally, the reverse proxy can perform request header and/or path - matching based on `match` rules of the `HTTPRoute`. - 6. Optionally, the reverse proxy can modify the request, i.e. add/remove - headers, based on `filter` rules of the `HTTPRoute`. - 7. Lastly, the reverse proxy forwards the request to one or more objects, i.e. - `Service`, in the cluster based on `forwardTo` rules of the `HTTPRoute`. +1. A client makes a request to . +2. DNS resolves the name to a `Gateway` address. +3. The reverse proxy receives the request on a `Listener` and uses the [Host + header](https://tools.ietf.org/html/rfc7230#section-5.4) to match an + `HTTPRoute`. +4. Optionally, the reverse proxy can perform request header and/or path + matching based on `match` rules of the `HTTPRoute`. +5. Optionally, the reverse proxy can modify the request, i.e. add/remove + headers, based on `filter` rules of the `HTTPRoute`. +6. Lastly, the reverse proxy forwards the request to one or more objects, i.e. + `Service`, in the cluster based on `backendRefs` rules of the `HTTPRoute`. + +[north/south]:/concepts/glossary#northsouth-traffic -## TLS Configuration +### TLS Configuration TLS is configured on Gateway listeners, and may be referred to across namespaces. Please refer to the [TLS details](/guides/tls) guide for a deep dive on TLS. +## Attaching Routes to Services + +!!! danger "Experimental in v0.8.0" + + The [GAMMA initiative][gamma] work for supporting service mesh use cases + is _experimental_ in `v0.8.0`. It is possible that it will change; we do + not recommend it in production at this point. + + In particular, binding Routes directly to Services seems to be the current + best choice for configuring mesh routing, but it is still **experimental** + and thus **subject to change**. + +When using the Gateway API to configure a [service mesh], the Route will +attach directly to a Service, representing configuration meant to be applied +to any traffic directed to the Service. How and which Routes attach to a given +Service is controlled by the Routes themselves (working with Kubernetes RBAC), +as covered in the [GAMMA routing documentation]. + +[GAMMA]:/concepts/gamma +[GAMMA routing documentation]:/concepts/gamma#gateway-api-for-mesh +[service mesh]:/concepts/glossary#service-mesh ## Extension points @@ -324,4 +348,3 @@ Whenever you are using an extension point without any prior art, please let the community know. As we learn more about usage of extension points, we would like to find the common denominators and promote the features to core/extended API conformance. - diff --git a/site-src/concepts/conformance.md b/site-src/concepts/conformance.md index 8f43f433ab..e8c9a5f825 100644 --- a/site-src/concepts/conformance.md +++ b/site-src/concepts/conformance.md @@ -13,8 +13,8 @@ Within Gateway API, release channels are used to indicate the stability of a field or resource. The "standard" channel of the API includes fields and resources that have graduated to "beta". The "experimental" channel of the API includes everything in the "standard" channel, along with experimental fields -and resources that may still be changed in breaking ways or removed altogether. -For more information on this concept, refer to our +and resources that may still be changed in breaking ways **or removed +altogether**. For more information on this concept, refer to our [versioning](/concepts/versioning) documentation. ## 2. Support Levels @@ -74,27 +74,98 @@ capabilities. ### Running Tests -By default, conformance tests will expect a GatewayClass named `gateway-conformance` -to be installed in the cluster, and tests will be run against that. Most often, -you'll use a different class, which can be specified with the `-gateway-class` flag along with the -corresponding test command. Check your instance for the `gateway-class` name to use. +There are two main contrasting sets of conformance tests: + +* Gateway related tests (can also be thought of as ingress tests) +* Service Mesh related tests + +For `Gateway` tests you must enable the `Gateway` test feature, and then +opt-in to any other specific tests you want to run (e.g. `HTTPRoute`). For +Mesh related tests you must enable `Mesh`. + +We'll cover each use case separately, but it's also possible to combine these +if your implementation implements both. There are also options which pertain +to the entire test suite regardless of which tests you're running. + +#### Gateway Tests + +By default `Gateway` oriented conformance tests will expect a GatewayClass +named `gateway-conformance` to be installed in the cluster, and tests will be +run against that. Most often, you'll use a different class, which can be +specified with the `-gateway-class` flag along with the corresponding test +command. Check your instance for the `gateway-class` name to use. You must +also enable `Gateway` support and test support for any `*Routes` your +implementation supports. + +The following runs all the tests relevant to `Gateway`, `HTTPRoute`, and +`ReferenceGrant`: ```shell -go test ./conformance/... -args -gateway-class=my-gateway-class +go test ./conformance/... -args \ + -gateway-class=my-gateway-class \ + -supported-features=Gateway,HTTPRoute ``` -Other useful flags may be found in -[conformance flags](https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/utils/flags/flags.go). -For example, if you'd like to examine the objects in Kubernetes after your test runs, you can pass a flag to -suppress cleanup: +Other useful flags may be found in [conformance flags][cflags]. + +[cflags]:https://github.com/kubernetes-sigs/gateway-api/blob/main/conformance/utils/flags/flags.go + +#### Mesh Tests + +Mesh tests can be run by simply enabling the `Mesh` feature: + ```shell -go test ./conformance/... -args -gateway-class=istio -cleanup-base-resources=false +go test ./conformance/... -args -supported-features=Mesh ``` -If you'd like to run a single test instead of the entire conformance suite, find your test name -`(suite.ConformanceTest.ShortName)` and use it like this: + +If your mesh also includes ingress support with an API such as `HTTPRoute`, you +can run the relevant tests in the same test run by enabling the `Gateway` +feature and any relevant API features, e.g: + ```shell -go test ./conformance/... --run TestConformance/YOURTESTNAME --gateway-class=istio +go test ./conformance/... -args -supported-features=Mesh,Gateway,HTTPRoute ``` + +#### Namespace Labels + +If labels are needed on namespaces used for testing, you can use +the `-namespace-labels` flag to pass one or more `name=value` labels to +set on the test namespaces. For mesh testing, this flag can be used if +an implementation requires labels on namespaces that host mesh workloads, +for example, to enable sidecar injection. + +#### Excluding Tests + +The `Gateway` and `ReferenceGrant` features are enabled by default. +You do not need to explicitly list them using the `-supported-features` flag. +However, if you don't want to run them, you will need to disable them using +the `-exempt-features` flag. For example, to run only the `Mesh` tests, +and nothing else: + +```shell +go test ./conformance/... -args \ + -supported-features=Mesh \ + -exempt-features=Gateway,ReferenceGrant +``` + +#### Suite Level Options + +When running tests of any kind you may not want the test suite to cleanup the +test resources when it completes (i.e. so that you can inspect the cluster +state in the event of a failure). You can skip cleanup with: + +```shell +go test ./conformance/... -args -cleanup-base-resources=false +``` + +It may be helpful (particularly when working on implementing a specific +feature) to run a very specific test by name. This can be done using the +`ShortName` of that test: + +```shell +go test ./conformance/... --run TestConformance/ +``` + ## Contributing to Conformance Many implementations run conformance tests as part of their full e2e test suite. @@ -111,4 +182,4 @@ handles those manifests appropriately. Issues related to conformance are [labeled with "area/conformance"](https://github.com/kubernetes-sigs/gateway-api/issues?q=is%3Aissue+is%3Aopen+label%3Aarea%2Fconformance). These often cover adding new tests to improve our test coverage or fixing flaws -or limitations in our existing tests. \ No newline at end of file +or limitations in our existing tests. diff --git a/site-src/concepts/gamma.md b/site-src/concepts/gamma.md new file mode 100644 index 0000000000..21ce8562d8 --- /dev/null +++ b/site-src/concepts/gamma.md @@ -0,0 +1,233 @@ +# The GAMMA initiative (Gateway API for Service Mesh) + +!!! danger "Experimental in v0.8.0" + + GAMMA support for service mesh use cases is _experimental_ in `v0.8.0`. + It is possible that it will change; we do not recommend it in production + at this point. + +The Gateway API was originally designed to manage traffic from clients outside +the cluster to services inside the cluster -- the _ingress_ or +[_north/south_][north/south traffic] case. Over time, though, interest from +[service mesh] users prompted the creation of the GAMMA (**G**ateway **A**PI +for **M**esh **M**anagement and **A**dministration) initiative in 2022 to +define how the Gateway API could also be used for inter-service or +[_east/west_ traffic][east/west traffic] within the same cluster. + +The GAMMA initiative is a dedicated workstream within the Gateway API +subproject, rather than being a separate subproject. GAMMA’s goal is to define +how the Gateway API can be used to configure a service mesh, with the +intention of making minimal changes to the Gateway API and always preserving +the [role-oriented] nature of the Gateway API. Additionally, we strive to +advocate for consistency between implementations of the Gateway API by service +mesh projects, regardless of their technology stack or proxy. + +## Deliverables + +The work of the GAMMA intiative will be captured in [Gateway Enhancement +Proposals][geps] that extend or refine the Gateway API specification to cover +mesh and mesh-adjacent use cases. To date, these have been relatively small +changes (albeit sometimes with relatively large impacts!) and we expect that +to continue. Governance of the Gateway API specification remains solely with +the maintainers of the Gateway API subproject. + +The ideal final outcome of the GAMMA initiative is that service mesh use cases +become a first-party concern of the Gateway API, at which point there will be +no further need for a separate initiative. + +## Contributing + +We welcome contributors of all levels! There are [many ways to +contribute][contributor-ladder] to the Gateway API and GAMMA, both technical +and non-technical. + +The simplest way to get started is to attend one of GAMMA's regular meetings, +which happen every two weeks on Tuesdays for 1 hour (as found on the +[sig-network calendar]), alternating between 3pm PT and 8am PT slots to try to +be as time-zone inclusive as possible. GAMMA meetings will be moderated by the +[GAMMA leads] with notes taken by a volunteer. Community members should feel +free to attend both GAMMA and Gateway API meetings, but are by no means +obligated to do so. + +[contributor-ladder]:/contributing/contributor-ladder +[east/west traffic]:/concepts/glossary#eastwest-traffic +[GAMMA leads]:https://github.com/kubernetes-sigs/gateway-api/blob/main/OWNERS_ALIASES#L23 +[geps]:/geps/overview +[north/south traffic]:/concepts/glossary#northsouth-traffic +[service mesh]:/concepts/glossary#service-mesh +[sig-network calendar]:/contributing/community/#meetings +[role-oriented]:/concepts/roles-and-personas + +## An Overview of the Gateway API for Service Mesh + +!!! danger "Experimental in v0.8.0" + + GAMMA support for service mesh use cases is _experimental_ in `v0.8.0`. + It is possible that it will change; we do not recommend it in production + at this point. + +GAMMA has, to date, been able to define service mesh support in the Gateway +API with relatively small changes. The most significant change that GAMMA has +introduced to date is that, when configuring a service mesh, individual route +resources (such as [HTTPRoute]) are [associated directly with Service +resources](#gateway-api-for-mesh). + +This is primarily because there will typically only be one mesh active in the +cluster, so the [Gateway] and [GatewayClass] resources are not used when +working with a mesh. In turn, this leaves the Service resource as the most +universal binding point for routing information. + +Since the Service resource is unfortunately complex, with several overloaded +or underspecified aspects, GAMMA has also found it critical to formally define +the [Service _frontend_ and _backend_ facets][service-facets]. In brief: + +- The Service frontend is its name and cluster IP, and +- The Service backend is its collection of endpoint IPs. + +This distinction helps the Gateway API to be exact about how routing within a +mesh functions, without requiring new resources that largely duplicate the +Service. + +[GatewayClass]: /api-types/gatewayclass +[Gateway]: /api-types/gateway +[HTTPRoute]: /api-types/httproute +[TCPRoute]: /api-types/tcproute +[Service]: https://kubernetes.io/docs/concepts/services-networking/service/ +[service-mesh]:/concepts/glossary#service-mesh +[service-facets]:/concepts/service-facets + +## How the Gateway API works for Service Mesh + +GAMMA specifies that individual Route resources attach directly to a Service, +representing configuration meant to be applied to _any traffic directed to the +Service_. + +!!! danger "Experimental in v0.8.0" + + Binding Routes directly to Services seems to be the current best choice + for configuring mesh routing, but it is still **experimental** and thus + **subject to change**. + +When one or more Routes are attached to a Service, **requests that do not +match at least one of the Routes will be rejected**. If no Routes are attached +to a Service, requests to the Service simply proceed per the mesh's default +behavior (usually resulting in the request being forwarded as if the mesh were +not present). + +Which Routes attach to a given Service is controlled by the Routes themselves +(working with Kubernetes RBAC): the Route simply specifies a `parentRef` that +is a Service, rather than a Gateway. + +```yaml +kind: HTTPRoute +metadata: + name: smiley-route + namespace: faces +spec: + parentRefs: + - name: smiley + kind: Service + group: core + port: 80 + rules: + ... +``` + +The relationship between the Route's Namespace and the Service's Namespace is +important: + +- Same Namespace + + !!! note "Work in Progress" + + There is ongoing work around the relationship between producer + routes and consumer routes. + + A Route in the same Namespace as its Service is called a [producer route], + since it is typically created by the creator of the workload in order to + define acceptable usage of the workload (for example, [Ana] would deploy + both the workload and the Route). All requests from any client of the + workload, from any Namespace, will be affected by this Route. + + The Route shown above is a producer route. + +- Different Namespaces + + !!! note "Work in Progress" + + There is ongoing work around the relationship between producer + routes and consumer routes. + + A Route in a different Namespace than its Service is called a [consumer + route]. Typically, this is a Route meant to refine how a consumer of a + given workload makes request of that workload (for example, configuring + custom timeouts for that consumer's use of the workload). This Route will + only affect requests from workloads in the same Namespace as the Route. + + For example, this HTTPRoute would cause all clients of the `smiley` + workload in the `fast-clients` Namespace to have a 100ms timeout: + + ```yaml + kind: HTTPRoute + metadata: + name: smiley-route + namespace: fast-clients + spec: + parentRefs: + - name: smiley + namespace: faces + kind: Service + group: core + port: 80 + rules: + ... + timeouts: + request: 100ms + ``` + +One important note about Routes bound to Services is that multiple Routes for +the same Service in a single Namespace - whether producer routes or consumer +routes - will be combined according to the Gateway API [Route merging rules]. +As such, it is not currently possible to define distinct consumer routes for +multiple consumers in the same Namespace. + +For example, if the `blender` workload and the `mixer` workload both live in +the `foodprep` Namespace, and both call the `oven` workload using the same +Service, it is not currently possible for `blender` and `mixer` to use +HTTPRoutes to set different timeouts for their calls to the `oven` workload. +`blender` and `mixer` would need to be moved into separate Namespaces to allow +this. + +[Ana]:/concepts/roles-and-personas#ana +[producer route]:/concepts/glossary#producer-route +[consumer route]:/concepts/glossary#consumer-route +[service mesh]:/concepts/glossary#service-mesh +[Route merging rules]:/api-types/httproute#merging + +## Request Flow + +A typical [east/west] API request flow when a GAMMA-compliant mesh is in use +looks like: + +1. A client workload makes a request to . +2. The mesh data plane intercepts the request and identifies it as traffic for + the Service `foo` in Namespace `ns`. +3. The data plane locates Routes associated with the `foo` Service, then: + + a. If there are no associated Routes, the request is always allowed, and + the `foo` workload itself is considered the destination workload. + + b. If there are associated Routes and the request matches at least one of + them, the `backendRefs` of the highest-priority matching Route are used + to select the destination workload. + + c. If there are associated Routes, but the request matches none of them, + the request is rejected. + +6. The data plane routes the request on to the destination workload (most + likely using [endpoint routing], but it is allowed to use [Service + routing]). + +[east/west]:/concepts/glossary#eastwest-traffic +[endpoint routing]:/concepts/glossary#endpoint-routing +[Service routing]:/concepts/glossary#service-routing diff --git a/site-src/concepts/glossary.md b/site-src/concepts/glossary.md new file mode 100644 index 0000000000..ba9d211c57 --- /dev/null +++ b/site-src/concepts/glossary.md @@ -0,0 +1,69 @@ +# Gateway API Glossary + +### Consumer Route + +A Route bound to a workload's Service by a consumer of a given workload, +refining the specific consumer's use of the workload. + +### Gateway Controller + +A _gateway controller_ is software that manages the infrastructure associated +with routing traffic across contexts using the Gateway API, analogous to the +earlier _ingress controller_ concept. Gateway controllers often, but not +always, run in the cluster where they're managing infrastructure. + +### East/West traffic + +Traffic from workload to workload within a cluster. + +### Endpoint routing + +_Endpoint routing_ is sending requests to a specific Service directly to one +of the endpoints of the Service backend, bypassing routing decisions which +might be made by the underlying network infrastructure. This is commonly +necessary for advanced routing cases like sticky sessions, where the gateway +will need to guarantee that every request for a specific session goes to the +same endpoint. + +### North/South traffic + +Traffic from outside a cluster to inside a cluster (and vice versa). + +### Producer Route + +A Route bound to a workload's Service by the creator of a given workload, +defining what is acceptable use of the workload. Producer routes must always +be in the same Namespace as their workload's Service. + +### Service backend + +The part of a Kubernetes Service resource that is a set of endpoints +associated with Pods and their IPs. Some east/west traffic happens by having +workloads direct requests to specific endpoints within a Service backend. + +### Service frontend + +The part of a Kubernetes Service resource that allocates a DNS record and a +cluster IP. East/west traffic often - but not always - works by having +workloads direct requests to a Service frontend. + +### Service mesh + +A _service mesh_ is software that manages infrastructure providing security, +reliability, and observability for communications between workloads (east/west +traffic). Service meshes generally work by intercepting communications between +workloads at a very low level, often (though not always) by inserting proxies +next to the workload's Pods. + +### Service routing + +_Service routing_ is sending requests to a specific Service to the service +frontend, allowing the underlying network infrastructure (usually `kube-proxy` +or a [service mesh](#service-mesh)) to choose the specific endpoint to which +the request is routed. + +### Workload + +An instance of computation that provides a function within a cluster, +comprising the Pods providing the compute, and the +Deployment/Job/ReplicaSet/etc which owns those Pods. diff --git a/site-src/concepts/guidelines.md b/site-src/concepts/guidelines.md index b16edc0cd6..e282f88523 100644 --- a/site-src/concepts/guidelines.md +++ b/site-src/concepts/guidelines.md @@ -129,8 +129,4 @@ in our CRDs. We use the following rules: - If the field name is a noun, use a plural value. - If the field name is a verb, use a singular value. -So for example, in HTTPRoute, `hostnames` uses a plural, but `forwardTo` is singular, -although they are both lists. - [1]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md - diff --git a/site-src/concepts/roles-and-personas.md b/site-src/concepts/roles-and-personas.md new file mode 100644 index 0000000000..73da3744a9 --- /dev/null +++ b/site-src/concepts/roles-and-personas.md @@ -0,0 +1,80 @@ +# Roles and personas + +## Introduction + +In the original design of Kubernetes, Ingress and Service resources were based +on a usage model in which the developers who create Services and Ingresses +controlled all aspects of defining and exposing their applications to their +users. + +In practice, though, clusters and their infrastructure tend to be shared, +which the original Ingress model doesn't capture very well. A critical factor +is that when infrastructure is shared, not everyone using the infrastructure +has the same concerns, and to be successful, an infrastructure project needs +to address the needs of all the users. + +This raises a fundamental challenge: how do you provide the flexibility needed +by the users of the infrastructure, while also maintaining control by the +owners of the infrastructure? + +The Gateway API defines several distinct roles, each with an associated +_persona_, as a tool for surfacing and discussing the differing needs of +different users in order to balance usability, flexibility, and control. +Design work within the Gateway API is deliberately cast in terms of these +personas. + +Note that, depending on the environment, a single human may end up taking on +multiple roles, as discussed below. + +## Roles and Personas + +The Gateway API defines three roles and personas: + +* **Ian** (he/him) is an _infrastructure provider_, + responsible for the care and feeding of a set of infrastructure that permits + multiple isolated clusters to serve multiple tenants. He is not beholden to + any single tenant; rather, he worries about all of them collectively. Ian + will often work for a cloud provider (AWS, Azure, GCP, ...) or for a PaaS + provider. + +* **Chihiro** (they/them) is a _cluster operator_, + responsible for managing clusters to ensure that they meet the needs of + their several users. Chihiro will typically be concerned with policies, + network access, application permissions, etc. Again, they are beholden to no + single user of any cluster; rather, they need to make sure that the clusters + serve all users as needed. + +* **Ana** (she/her) is an _application developer_, + responsible for creating and managing an application running in a cluster. + From the Gateway API's point of view, Ana will need to manage configuration + (e.g. timeouts, request matching/filter) and Service composition (e.g. path + routing to backends). She is in a unique position among the Gateway API + personas, since her focus is on the business needs her application is meant + to serve, _not_ Kubernetes or the Gateway API. In fact, Ana is likely to + view the Gateway API and Kubernetes as pure friction getting in her way to + get things done. + +Depending on the environment, multiple roles can map to the same user: + +- Giving a single user all the above roles replicates the self-service model, + and may actually happen in a small startup running Kubernetes on bare metal. + +- A more typical small startup would use clusters from a cloud provider. In + this situation, Ana and Chihiro may be embodied in the same person, with Ian + being an employee (or automated process!) within the cloud provider. + +- In a much larger organization, we would expect each persona above to be + embodied by a distinct person (most likely working in different groups, + perhaps with little direct contact). + +## RBAC + +RBAC (role-based access control) is the standard used for Kubernetes +authorization. This allows users to configure who can perform actions on +resources in specific scopes. We anticipate that each persona will map +approximately to a `Role` in the Kubernetes Role-Based Authentication (RBAC) +system and will define resource model responsibility and separation. + +RBAC is discussed further in the [Security Model] description. + +[Security Model]: /concepts/security-model#rbac diff --git a/site-src/concepts/security-model.md b/site-src/concepts/security-model.md index bda7de4fa3..3774fc9867 100644 --- a/site-src/concepts/security-model.md +++ b/site-src/concepts/security-model.md @@ -15,40 +15,16 @@ Gateway API has 3 primary API resources: ## Roles and personas -In the original design of Kubernetes, Ingress and Service resources were based -on a self-service model of usage; developers who create Services and Ingresses -control all aspects of defining and exposing their applications to their users. - -We have found that the self-service model does not fully capture some of the -more complex deployment and team structures that our users are seeing. Gateway -API is designed to target the following personas: - -* **Infrastructure provider**: The infrastructure provider (infra) is - responsible for the overall environment that the cluster(s) are operating in. - Examples include: the cloud provider (AWS, Azure, GCP, ...) or the PaaS - provider in a company. -* **Cluster operator**: The cluster operator (ops) is responsible for - administration of entire clusters. They manage policies, network access, - application permissions. -* **Application developer**: The application developer (dev) is responsible for - defining their application configuration (e.g. timeouts, request - matching/filter) and Service composition (e.g. path routing to backends). - -Although these roles can cover a wide variety of use cases, some organizations -may be structured slightly differently. Many organizations may also have a -fourth role that sits between "cluster operator" and "application developer": - -* **Application admin**: The application admin has administrative access to some - namespaces within a cluster, but not the cluster as a whole. - -We expect that each persona will map approximately to a `Role` in the Kubernetes -Role-Based Authentication (RBAC) system and will define resource model -responsibility and separation. - -Depending on the environment, multiple roles can map to the same user. For -example, giving the user all the above roles replicates the self-service model. +There are 3 primary roles in Gateway API, as described in [roles and personas]: + +- **Ian** (he/him): Infrastructure Provider +- **Chihiro** (they/them): Cluster Operator +- **Ana** (she/her): Application Developer + +[roles and personas]:/concepts/roles-and-personas ### RBAC + RBAC (role-based access control) is the standard used for Kubernetes authorization. This allows users to configure who can perform actions on resources in specific scopes. RBAC can be used to enable each of the roles @@ -113,8 +89,8 @@ Backends (usually Services). In these cases, the required handshake is accomplished with a ReferenceGrant resource. This resource exists within a target namespace and can be used to allow references from other namespaces. -For example, the following ReferenceGrant allows references from Gateways in -the "prod" namespace to HTTPRoutes that are deployed in the same namespace as +For example, the following ReferenceGrant allows references from HTTPRoutes in +the "prod" namespace to Services that are deployed in the same namespace as the ReferenceGrant. ```yaml diff --git a/site-src/concepts/service-facets.md b/site-src/concepts/service-facets.md new file mode 100644 index 0000000000..5e2d39a488 --- /dev/null +++ b/site-src/concepts/service-facets.md @@ -0,0 +1,49 @@ +# The Different Facets of a Service + +The Kubernetes [Service] resource is considerably more complex than people +often realize. When you create a Service, typically the cluster machinery will: + +- Allocate a cluster-wide IP address for the Service itself (its _cluster IP_); +- Allocate a DNS name for the Service, resolving to the cluster IP (its _DNS name_); +- Collect the separate cluster-wide IP addresses assigned to each Pod matched + by the Service's selector (the _endpoint IPs_) into the Service's Endpoints + or EndpointSlices. +- Configure the network such that traffic to the cluster IP will be + load-balanced across all the endpoint IPs. + +Unfortunately, these implementation details become very important when +considering how the Gateway API can work for service meshes! + +In [GAMMA initiative][gamma] work, it has become useful to consider Services +as comprising two separate _facets_: + +- The **frontend** of the Service is the combination of the cluster IP and + its DNS name. + +- The **backend** of the Service is the collection of endpoint IPs. (The Pods + are not part of the Service backend, but they are of course strongly + associated with the endpoint IPs.) + +The distinction between the facets is critical because the [gateway] and the +[mesh] each need to decide whether a request that mentions a given Service +should be directed to the Service's frontend or its backend: + +- Directing the request to the Service's frontend (_Service routing_) leaves + the decision of which endpoint IP to use to the underlying network + infrastructure (which might be `kube-proxy`, a service mesh, or something + else). + +- Directing the request to the Service's backend (_endpoint routing_) is + often necessary to enable more advanced load balancing decisions (for + example, a gateway implementing sticky sessions). + +While Service routing may be the most direct fit for [Ana]'s sense of routing, +endpoint routing can be more predictable when using the Gateway API for both +[north/south] and [east/west traffic]. The [GAMMA initiative][gamma] is working to +formalize guidance for this use case. + +[Service]: https://kubernetes.io/docs/concepts/services-networking/service/ +[north/south]:/concepts/glossary#northsouth-traffic +[east/west traffic]:/concepts/glossary#eastwest-traffic +[gamma]:/concepts/gamma/ +[Ana]:/concepts/roles-and-personas#ana diff --git a/site-src/concepts/use-cases.md b/site-src/concepts/use-cases.md new file mode 100644 index 0000000000..7b5edae7e4 --- /dev/null +++ b/site-src/concepts/use-cases.md @@ -0,0 +1,202 @@ +# Gateway API Use Cases + +The Gateway API covers a _very_ wide range of use cases (which is both a +strength and a weakness!). This page is emphatically _not_ meant to be an +exhaustive list of these use cases: rather, it is meant to provide some +examples that can be helpful to demonstrate how the API can be used. + +In all cases, it's very important to bear in mind the [roles and personas] +used in the Gateway API. The use cases presented here are deliberately +described in terms of [Ana], [Chihiro], and [Ian]: they are the ones for whom +the API must be usable. (It's also important to remember that even though +these roles might be filled by the same person, especially in smaller +organizations, they all have distinct concerns that we need to consider +separately.) + +[roles and personas]:/concepts/roles-and-personas +[Ana]:/concepts/roles-and-personas#ana +[Chihiro]:/concepts/roles-and-personas#chihiro +[Ian]:/concepts/roles-and-personas#ian + +## The Use Cases + +- [Basic north/south use case](#basic-northsouth-use-case) +- [Multiple applications behind a single + Gateway](#multiple-applications-behind-a-single-gateway) +- [Basic east/west use case](#basic-eastwest-use-case) -- **experimental** +- [Gateway and mesh use case](#gateway-and-mesh-use-case) -- **experimental** + +[role and personas]:/concepts/roles-and-personas + +## Basic [north/south] use case + +??? success "Standard Channel in v0.8.0+" + + The [north/south] use case is fully supported by the Standard Channel + in `v0.8.0+`. + +Ana has created a microservice application which she wants to run in +Kubernetes. Her application will be used by clients outside the cluster, and +while Ana has created the application, setting up the cluster is not in her +wheelhouse. + +1. Ana goes to Chihiro to ask them to set up a cluster. Ana tells Chihiro that + her clients will expect her APIs to be available using URLs rooted at + `https://ana.application.com/`. + +2. Chihiro goes to Ian and requests a cluster. + +3. Ian provisions a cluster running a gateway controller with a [GatewayClass] + resource named `basic-gateway-class`. The gateway controller manages the + infrastructure associated with routing traffic from outside the cluster to + inside the cluster. + +4. Ian gives Chihiro credentials to the new cluster, and tells Chihiro that + they can use GatewayClass `basic-gateway-class` to set things up. + +5. Chihiro applies a [Gateway] named `ana-gateway` to the cluster, telling it + to listen on port 443 for TLS traffic, and providing it a TLS certificate + with a Subject CN of `ana.application.com`. They associate this Gateway with the `basic-gateway-class` GatewayClass. + +6. The gateway controller that Ian provisioned in step 3 allocates a load + balancer and an IP address for `ana-gateway`, provisions data-plane + components that can route requests arriving at the load balancer on port + 443, and starts watching for routing resources associated with + `ana-gateway`. + +7. Chihiro gets the IP address of `ana-gateway` and creates a DNS record + outside the cluster for `ana.application.com` to match. + +8. Chihiro tells Ana that she's good to go, using the Gateway named + `ana-gateway`. + +9. Ana writes and applies [HTTPRoute] resources to configure which URL paths + are allowed and which microservices should handle them. She associates + these HTTPRoutes with Gateway `ana-gateway` using the Gateway API [Route + Attachment Process]. + +10. At this point, when requests arrive at the load balancer, they are routed + to Ana's application according to her routing specification. + +This allows Chihiro to enforce centralized policies [such as +TLS](/guides/tls#downstream-tls) at the Gateway, while simultaneously allowing +Ana and her colleagues control over the application's [routing +logic](/guides/http-routing) and rollout plans (e.g. [traffic splitting +rollouts](/guides/traffic-splitting)). + +[north/south]:/concepts/glossary#northsouth-traffic + +## Multiple applications behind a single Gateway + +??? success "Standard Channel in v0.8.0+" + + The [north/south] use case is fully supported by the Standard Channel + in `v0.8.0+`. + +This is remarkably similar to the [basic north/south use +case](#basic-northsouth-use-case), but there are multiple application teams: +Ana and her team are managing a storefront application in the `store` +Namespace, while Allison and her team are managing a website in the `site` +Namespace. + +- Ian and Chihiro work together to provide a cluster, `GatewayClass`, and + `Gateway`, as above. + +- Ana and Allison independently deploy workloads and HTTPRoutes bound to the + same `Gateway` resource. + +Again, this separation of concerns allows Chihiro to enforce centralized +policies [such as TLS](/guides/tls#downstream-tls) can be enforced at the +Gateway. Meanwhile, Ana and Allison run their applications [in their own +Namespaces](/guides/multiple-ns), but attach their Routes to the same shared +Gateway, allowing them to independently control their [routing +logic](/guides/http-routing), [traffic splitting +rollout](/guides/traffic-splitting), etc., while not worrying about the things +that Chihiro and Ian are handling. + +[HTTPRoute]:/api-types/httproute +[GatewayClass]:/api-types/gatewayclass +[Gateway]:/api-types/gateway +[Route Attachment Process]:/concepts/api-overview#attaching-routes-to-gateways + +![Gateway API Roles](/images/gateway-roles.png) + +## Basic [east/west] use case + +!!! danger "Experimental in v0.8.0" + + The [GAMMA initiative][gamma] work for supporting service mesh use cases + is _experimental_ in `v0.8.0`. It is possible that it will change; we do + not recommend it in production at this point. + +In this scenario, Ana has built a workload which is already running in a +cluster with a [GAMMA]-compliant [service mesh]. She wants to use the mesh to +protect her workload by rejecting calls to her workload with incorrect +URL paths, and by enforcing timeouts whenever anyone makes a request of her +workload. + +- Chihiro and Ian have already provided a cluster with a running service mesh. + Ana doesn't need to make any requests of them. + +- Ana writes an HTTPRoute that defines acceptable routes and timeouts and has + a `parentRef` of her workload's Service. + +- Ana applies her HTTPRoute in the same Namespace as her workload. + +- The mesh automatically starts enforcing the routing policy described by + Ana's HTTPRoute. + +In this case, the separation of concerns across roles allows Ana to take +advantage of the service mesh, with custom routing logic, without any +bottlenecks in requests to Chihiro or Ian. + +[east/west]:/concepts/glossary#eastwest-traffic +[GAMMA]:/concepts/gamma/ +[service mesh]:/concepts/glossary#service-mesh + +## Gateway and mesh use case + +!!! danger "Experimental in v0.8.0" + + The [GAMMA initiative][gamma] work for supporting service mesh use cases + is _experimental_ in `v0.8.0`. It is possible that it will change; we do + not recommend it in production at this point. + +This is effectively a combination of the [multiple applications behind a +single Gateway](#multiple-applications-behind-a-single-gateway) and [basic +east/west](#basic-eastwest-use-case) use cases: + +- Chihiro and Ian will provision a cluster, a [GatewayClass], and a [Gateway]. + +- Ana and Allison will deploy their applications in the appropriate + Namespaces. + +- Ana and Allison will then apply HTTPRoute resources as appropriate. + +There are two very important changes in this scenario, though, since a mesh is +involved: + +1. If Chihiro has deployed a [gateway controller] that defaults to [Service + routing], they will probably need to reconfigure it for [endpoint routing]. + (This is an ongoing area of work for [GAMMA], but the expectation is that + endpoint routing will be recommended.) + +2. Ana and/or Allison will need to bind HTTPRoutes to their respective + workloads' Services to configure mesh routing logic. These could be + distinct HTTPRoutes solely for the mesh, or they could apply single + HTTPRoutes that bind to both the Gateway and a Service. + +As always, the ultimate point of separating concerns in this way is that it +permits Chihiro to enforce centralized policies [such as +TLS](/guides/tls#downstream-tls) at the Gateway, while allowing Ana and +Allison to retain independent control of [routing +logic](/guides/http-routing), [traffic splitting +rollout](/guides/traffic-splitting), etc., both for [north/south] and for +[east/west] routing. + + + + +[gateway controller]:/concepts/glossary#gateway-controller +[Service routing]:/concepts/glossary#service-routing +[endpoint routing]:/concepts/glossary#endpoint-routing diff --git a/site-src/contributing/contributor-ladder.md b/site-src/contributing/contributor-ladder.md new file mode 100644 index 0000000000..d498de5b05 --- /dev/null +++ b/site-src/contributing/contributor-ladder.md @@ -0,0 +1,133 @@ +# Contributor Ladder + +Within the Kubernetes community, the concept of a contributor ladder has been +developed to define how individuals can earn formal roles within the project. +The Gateway API contributor ladder largely follows the [roles defined by the +broader Kubernetes +community](https://github.com/kubernetes/community/blob/master/community-membership.md), +though there are some aspects that are unique to this community. + +## Goals + +We hope that this doc will provide an initial step towards the following goals: + +* Ensure the long term health of the Gateway API community +* Encourage new contributors to work towards formal roles and responsibilities + in the project +* Clearly define the path towards leadership roles +* Develop a strong leadership pipeline so we have great candidates to fill + project leadership roles + + +## Scope + +The following repositories are covered by this doc: + +* [kubernetes-sigs/gateway-api](https://github.com/kubernetes-sigs/gateway-api) +* [kubernetes-sigs/ingress2gateway](https://github.com/kubernetes-sigs/ingress2gateway) +* kubernetes-sigs/blixt (once migration is complete) + +Within each of these projects, there are opportunities to become an approver or +reviewer for either the entire project, or a subset of that project. For +example, you could become a reviewer or approver focused on just docs, GEPs, API +changes, or conformance tests. + +## General Guidelines + +### 1. Everyone is welcome + +We appreciate all contributions. You don’t need to have a formal role in the project to make or review pull requests, and help with issues or discussions. Accepting a formal role within the project is entirely optional. + +### 2. These roles require continued contributions + +Applying for one of the roles defined above should only be done if you intend to +continue to contribute at a level that would merit that role. If for any reason +you are unable to continue in one of the roles above, please resign. Members +with an extended period away from the project with no activity will be removed +from the Kubernetes GitHub Organizations and will be required to go through the +org membership process again after re-familiarizing themselves with the current +state. + +### 3. Don’t merge without consensus + +If you have reason to believe that a change may be contentious, please wait for +additional perspectives from others before merging any PRs. Even if you have +access to merge a PR, it doesn’t mean you should. Although we can’t have PRs +blocked indefinitely, we need to make sure everyone has had a chance to present +their perspective. + +### 4. Start a discussion + +If you’re interested in working towards one of these roles, please reach out to +a Gateway API maintainer on Slack. + +## Contributor Ladder + +The Gateway API contributor ladder has the following steps: + +1. Member +2. Reviewer +3. Approver +4. Maintainer + +This is also a GAMMA-specific leadership role that does not fit as cleanly on +this ladder. All of these roles will be defined in more detail below. + +## Member, Reviewer, and Approver + +The first steps on the contributor ladder are already [clearly defined in the +upstream Kubernetes +Community](https://github.com/kubernetes/community/blob/master/community-membership.md#community-membership). +Gateway API follows those guidelines along with the rest of the Kubernetes +community. Within Gateway API, there are a variety of areas one can become a +reviewer or approver, this includes: + +* Conformance +* Documentation +* GEPs +* Webhook Validation + +## Maintainers and GAMMA Leads + +The final steps on the contributor ladder represent large overall leadership +roles within the project as a whole. The spaces available for these roles are +limited (generally 3-4 people in each role is ideal). Wherever possible, we try +to ensure that different companies are represented in these roles. + +### Maintainers + +Gateway API Maintainers are known as [Subproject +Owners](https://github.com/kubernetes/community/blob/master/community-membership.md#subproject-owner) +within the Kubernetes community. To become a Gateway API Maintainer, the most +important things we expect are: + +* Long term, sustained contributions to Gateway API for at least 6 months +* Deep understanding of technical goals and direction of the project +* Successfully authored and led significant enhancement proposals +* Approver for at least 3 months +* Ability to lead community meetings + +In addition to all of the expectations described above, we expect maintainers to +set the technical direction and goals for the project. This role is critical to +the health of the project, maintainers should mentor new approvers and +reviewers, and ensure that there are healthy processes in place for discussion +and decision making. Finally, maintainers are ultimately responsible for +releasing new versions of the API. + +## GAMMA Leads + +The concept of GAMMA Leads does not have a perfect parallel on the upstream +Kubernetes community ladder. They are essentially Subproject Owners, but for the +GAMMA initiative, which is a major initiative within Gateway API. + +To become a GAMMA lead, the most important thing we expect are: + +* Significant experience with Service Mesh implementation(s) +* Deep understanding of technical goals and direction of the project +* Long term, sustained contributions to the GAMMA initiative for at least 6 months +* Ability to lead community meetings + +In addition to all of the expectations described above, we expect GAMMA Leads to +set the technical direction and goals for the GAMMA initiative. They should +ensure that there are healthy processes in place for discussion and decision +making and that the release goals and milestones are clearly defined. diff --git a/site-src/contributing/devguide.md b/site-src/contributing/devguide.md index de85b59836..07bad221e5 100644 --- a/site-src/contributing/devguide.md +++ b/site-src/contributing/devguide.md @@ -154,3 +154,9 @@ INFO - Building documentation... [gateway-api.sigs.k8s.io]: https://gateway-api.sigs.k8s.io [Go environment]: https://golang.org/doc/install [Kubernetes]: https://github.com/kubernetes/community/blob/master/contributors/guide/pull-requests.md + +## Conformance Tests + +To develop or run conformance tests, follow the [Conformance Test Guide]. + +[Conformance Test Guide]: /concepts/conformance/#running-tests diff --git a/site-src/contributing/enhancement-requests.md b/site-src/contributing/enhancement-requests.md index 0170cddaf1..b6306b5b7b 100644 --- a/site-src/contributing/enhancement-requests.md +++ b/site-src/contributing/enhancement-requests.md @@ -10,17 +10,6 @@ Enhancements provides the basis of a community roadmap. Enhancements may be filed by anyone, but require approval from a maintainer to accept the enhancement into the project. -## Quick start - -1. Create an [Issue][issue] and select "Enhancement Request". -2. Follow the instructions in the enhancement request template and submit the - Issue. -3. (depending on size of change) Start a [draft Gateway Enhancement Proposal - (GEP)][gep] - -[issue]: https://github.com/kubernetes-sigs/gateway-api/issues/new/choose -[gep]: /geps/overview - ## What is Considered an Enhancement? An enhancement is generally anything that: @@ -49,6 +38,20 @@ Create an enhancement once you have: - Enhancements may take several releases to complete. - A prototype in your own fork (optional) +## How to Create a New Enhancement + +Once you've circulated your idea and received feedback from the maintainers +that creating a [Gateway Enhancement Proposal (GEP)][gep] is a good next step: + +- create an [Issue][issue] and select "Enhancement Request" +- follow the instructions in the enhancement request template and submit the + Issue. +- create an initial pull request to add your GEP. Follow the directions in the + [GEP documentation][gep]. + +[issue]: https://github.com/kubernetes-sigs/gateway-api/issues/new/choose +[gep]: /geps/overview + ## Why are Enhancements Tracked As the project evolves, it's important that the community understands how the diff --git a/site-src/contributing/gamma.md b/site-src/contributing/gamma.md deleted file mode 100644 index fb7f94e1dc..0000000000 --- a/site-src/contributing/gamma.md +++ /dev/null @@ -1,28 +0,0 @@ -# The GAMMA Initiative - -The GAMMA (Gateway API for Mesh Management and Administration) initiative is a -dedicated workstream within the Gateway API subproject. This group’s goal is to -investigate, design, and track Gateway API resources, semantics, and other -artifacts related to service mesh technology and use-cases. Additionally, we -strive to advocate for consistency between implementations of the Gateway API by -service mesh projects, regardless of their technology stack or proxy. - -## Deliverables - -This group will deliver [Gateway Enhancement Proposals](/geps/overview) -consisting of resources, additions, and modifications to the Gateway API -specification for mesh and mesh-adjacent use-cases. Governance of the Gateway -API specification remains solely with the maintainers of the Gateway API -subproject. Ideally, once service mesh use-cases become a first-party concern of -the spec, there will be no further need for a separate initiative. - -## Meeting mechanics - -GAMMA meetings will occur weekly on Tuesdays for 1 hour, alternating between 3pm -PT and 8am PT slots to try to be as time-zone inclusive as possible. Meetings -will be moderated by the [GAMMA -leads](https://github.com/kubernetes-sigs/gateway-api/blob/main/OWNERS_ALIASES#L23) -with notes taken by a volunteer. Meeting occurrences can be found on the -[sig-network calendar](/contributing/community/#meetings). Community members -should feel free to attend both GAMMA and Gateway API meetings, but are by no -means obligated to do so. diff --git a/site-src/contributing/index.md b/site-src/contributing/index.md index 928c4d6a0e..eb79977e5c 100644 --- a/site-src/contributing/index.md +++ b/site-src/contributing/index.md @@ -1,7 +1,9 @@ # How to Get Involved This page contains links to all of the meeting notes, design docs and related -discussions around the APIs. +discussions around the APIs. If you're interested in working towards a formal +role in the project, refer to the [Contributor +Ladder](/contributing/contributor-ladder). ## Feedback and Questions @@ -35,9 +37,10 @@ questions, discussions. ## Meetings -Meetings discussing the evolution of the Gateway API will alternate times to -accommodate participants from various time zones. This calendar includes all -Gateway API meetings as well as any other SIG-Network meetings. +Gateway API has multiple meetings covering different aspects and topics of the +Gateway API project. The following calendar includes _all_ SIG Network meetings +(which therefore includes all Gateway API meetings which include "Gateway API" +somewhere in their name): -Gateway API community meetings happen weekly on Mondays at 3pm Pacific Time -(23:00 UTC): +### Main Meeting + +The main Gateway API community meetings happen weekly on Mondays at 3pm Pacific +Time (23:00 UTC): * [Zoom link](https://zoom.us/j/441530404) * [Convert to your timezone](http://www.thetimezoneconverter.com/?t=15:00&tz=PT%20%28Pacific%20Time%29) * [Add to your calendar](https://calendar.google.com/event?action=TEMPLATE&tmeid=NXU4OXYyY2pqNzEzYzUwYnVsYmZwdXJzZDlfMjAyMTA1MTBUMjIwMDAwWiA4OGZlMWwzcWZuMmI2cjExazh1bTVhbTc2Y0Bn&tmsrc=88fe1l3qfn2b6r11k8um5am76c%40group.calendar.google.com&scp=ALL) -[GAMMA](/contributing/gamma/) meetings happen weekly on Tuesdays, alternating between 3pm Pacific Time (23:00 UTC) +Being the main meeting for Gateway API, the topics can vary here and often this +is where new topics and ideas are discussed. However if you're simply +interested in the ingress use case, this is the common forum for that. + +### Mesh Meeting (GAMMA) + +[GAMMA](/concepts/gamma/) is the initative within Gateway API to use the +resources provided by Gateway API for service mesh use cases. Meetings happen +every other week on Tuesdays, alternating between 3pm Pacific Time (23:00 UTC) and 8AM Pacific Time (16:00 UTC): * [Zoom link](https://zoom.us/j/96951309977) * Convert to your timezone: [3pm PT](http://www.thetimezoneconverter.com/?t=15:00&tz=PT%20%28Pacific%20Time%29)/[8am PT](http://www.thetimezoneconverter.com/?t=08:00&tz=PT%20%28Pacific%20Time%29) +### Code Jam Meeting + +The Gateway API "Code Jam" is less of a meeting and more of a hangout to discuss +and pair on Gateway API and technologies relevant to the project. This is an +open agenda meeting (feel free to bring your topics) where the following kinds +of activities (focusing on Gateway API related things) are encouraged: + +- Brainstorming +- Code pairing +- Demos +- Getting help + +This meeting puts an emphasis on being fun and laid back: If you're looking to +build further consensus and progress a [GEP][geps] then the _main_ meeting is +likely the place you'll want to bring your topic. However, if you're working on +adding [conformance tests][conformance] to your downstream implementation and +having some trouble and want some help getting things working, this meeting is a +good place for that kind of topic. + +* [Zoom link](https://zoom.us/j/96900767253) +* [8:30am PT](http://www.thetimezoneconverter.com/?t=08:30&tz=PT%20%28Pacific%20Time%29) + +[geps]:https://gateway-api.sigs.k8s.io/geps/overview/ + ### Meeting Notes and Recordings Meeting agendas and notes are maintained in the [meeting notes diff --git a/site-src/faq.md b/site-src/faq.md index 5d8f0954c5..27121d598b 100644 --- a/site-src/faq.md +++ b/site-src/faq.md @@ -16,60 +16,72 @@ and models more infrastructure components to provide better deployment and management options for cluster operators. + For more information, see the [Migrating from Ingress](https://gateway-api.sigs.k8s.io/guides/migrating-from-ingress/) guide. + * **Q: Will there be a default controller implementation (in this repo)?
** A: There is no current plan to have an "official" or "default" implementation. You will see the controller code in this repo be used for testing the support libraries. * **Q: How can I expose custom capabilities through Gateway API?
** - A: There is a lot of diversity in the networking and proxying - ecosystem, and many products will have features that are not directly - supported in the API. However, there are a few mechanisms available + A: There are a few mechanisms available for extending the API with implementation-specific capabilities: - * Decorate Gateway API objects with implementation-specific objects. A + * The [Policy Attachment](https://gateway-api.sigs.k8s.io/references/policy-attachment/) + model allows you to decorate Gateway API objects with implementation-specific CRDs. A policy or configuration object could match the Gateway API object either by name or by using an explicit object reference. - For example, given a `Gateway` object with name "inbound", - creating a `AccessPolicy` object that also has the name "inbound" - could cause an implementation to attach a specified access - control policy. This is an example of matching the object by name. - - * Use implementation-specific values for string fields. In many - places, the fields of Gateway API resources have the type - "string". This allows an implementation to support custom values - for those fields in addition to any values specified in the API. + * Use implementation-specific values for string fields in Gateway API resources. - * Use implementation-specific annotations. For some kinds of - configuration, implementations may choose to support custom - annotations on Gateway API objects. This approach continues - a proud tradition of extending Ingress objects. + * As a last resort, use implementation-specific annotations on Gateway API objects. * Use API-defined extension points. Some Gateway API objects have explicit [extension points](/concepts/api-overview#extension-points) for implementations to use. * **Q: Where can I find Gateway API releases?
** - A: Gateway API releases are tags of the [Github repository][1]. - The [Github releases][2] page shows all the releases. + A: Gateway API releases are tags of the [Github repository][1]. + The [Github releases][2] page shows all the releases. * **Q: How should I think about alpha API versions?
** - A: Similar to upstream Kubernetes, alpha API versions indicate that resources - are still experimental in nature and may either be removed or changed in - breaking ways in future releases of Gateway API. + A: Similar to upstream Kubernetes, alpha API versions indicate that resources + are still experimental in nature and may either be removed or changed in + breaking ways in future releases of Gateway API. + + See the [Versioning](https://gateway-api.sigs.k8s.io/concepts/versioning/) documentation for more info. * **Q: Which Kubernetes versions are supported?
** - A: Generally, we support Kubernetes 1.16+, but certain features like - AppProtocol depend on Kubernetes 1.18 (opt-in) or 1.19 (on by default). - There are not any other exceptions to the 1.16+ guideline right now. - -* **Q: Is SSL Passthrough supported?** - A: SSL Passthrough (wherein a Gateway routes traffic with the [Transport - Layer Security (TLS)][tls] encryption _intact_ to a backend service instead of - terminating it) is supported by [TLSRoutes][tlsroute]. See the - [TLS Guide][tlsguide] for more details about passthrough and other TLS - configurations. + A: See our policy on [Supported Version](https://gateway-api.sigs.k8s.io/concepts/versioning/#supported-versions) + +* **Q: Is SSL Passthrough supported?
** + A: SSL Passthrough (wherein a Gateway routes traffic with the [Transport + Layer Security (TLS)][tls] encryption _intact_ to a backend service instead of + terminating it) is supported by [TLSRoutes][tlsroute]. See the + [TLS Guide][tlsguide] for more details about passthrough and other TLS + configurations. + +* **Q: What's the difference between Gateway API and an API Gateway?
** + A: An API Gateway is a general concept that describes anything that exposes + capabilities of a backend service, while providing extra capabilities for + traffic routing and manipulation, such as load balancing, request and response + transformation, and sometimes more advanced features like authentication and + authorization, rate limiting, and circuit breaking. + + Gateway API is an interface, or set of resources, that model service networking + in Kubernetes. One of the main resources is a `Gateway`, which declares the + Gateway type (or class) to instantiate and its configuration. As a Gateway + Provider, you can implement the Gateway API to model Kubernetes service + networking in an expressive, extensible, and role-oriented way. + + Most Gateway API implementations are API Gateways to some extent, but not all + API Gateways are Gateway API implementations. + +* **Q: Is Gateway API a standard for API Management?
** + A: No. API Management is a much broader concept than what Gateway API aims to + be, or what an API Gateway is intended to provide. An API Gateway can be an + essential part of an API Management solution. Gateway API can be seen as a + way to standardize on that aspect of API Management. [1]: https://github.com/kubernetes-sigs/gateway-api [2]: https://github.com/kubernetes-sigs/gateway-api/releases diff --git a/site-src/geps/gep-713.md b/site-src/geps/gep-713.md deleted file mode 100644 index f8a4efd0f8..0000000000 --- a/site-src/geps/gep-713.md +++ /dev/null @@ -1,633 +0,0 @@ -# GEP-713: Policy Attachment - -* Issue: [#713](https://github.com/kubernetes-sigs/gateway-api/issues/713) -* Status: Experimental - -## TLDR - -This GEP aims to standardize policy attachment to resources associated with -Gateway API by establishing a pattern which defines how `Policy` API types can -have their relevant effects applied to network traffic. Individual policy APIs -(e.g. `TimeoutPolicy`, `RetryPolicy`, etc) will include a common `TargetRef` -field in their specification to identify how and where to apply that policy. -This will be important for providing a consistent experience across -implementations of the API, even for configuration details that may not be fully -portable. - -## Goals - -* Establish a pattern for policy attachment which will be used for any policies - included in the Gateway API spec -* Establish a pattern for policy attachment which should be used for any - implementation specific policies used with Gateway API resources -* Provide a way to distinguish between required and default values for all - policy API implementations -* Enable policy attachment at all relevant scopes, including Gateways, Routes, - Backends -* Ensure the policy attachment specification is generic and forward thinking - enough that it could be easily adapted to other grouping mechanisms like - Namespaces in the future -* Provide a means of attachment that works for both ingress and mesh - implementations of this API -* Provide a consistent specification that will ensure familiarity between both - included and implementation-specific policies so they can both be interpreted - the same way. - -## Out of scope - -* Define all potential policies that may be attached to resources -* Design the full structure and configuration of policies - -## API - -This approach is building on concepts from all of the alternatives discussed -below. This is very similar to the existing BackendPolicy resource in the API, -but also borrows some concepts from the [ServicePolicy -proposal](https://github.com/kubernetes-sigs/gateway-api/issues/611). - -### Policy Attachment for Ingress -Attaching policy to Gateway resources for ingress use cases is relatively -straightforward. A policy can reference the resource it wants to apply to. -Access is granted with RBAC - anyone that has access to create a RetryPolicy in -a given namespace can attach it to any resource within that namespace. - -![Simple Ingress Example](images/713-ingress-simple.png) - -To build on that example, it’s possible to attach policies to more resources. -Each policy applies to the referenced resource and everything below it in terms -of hierarchy. Although this example is likely more complex than many real world -use cases, it helps demonstrate how policy attachment can work across -namespaces. - -![Complex Ingress Example](images/713-ingress-complex.png) - -### Policy Attachment for Mesh -Although there is a great deal of overlap between ingress and mesh use cases, -mesh enables more complex policy attachment scenarios. For example, you may want -to apply policy to requests from a specific namespace to a backend in another -namespace. - -![Simple Mesh Example](images/713-mesh-simple.png) - -Policy attachment can be quite simple with mesh. Policy can be applied to any -resource in any namespace but it can only apply to requests from the same -namespace if the target is in a different namespace. - -At the other extreme, policy can be used to apply to requests from a specific -workload to a backend in another namespace. A route can be used to intercept -these requests and split them between different backends (foo-a and foo-b in -this case). - -![Complex Mesh Example](images/713-mesh-complex.png) - -### Policy TargetRef API - -Each Policy resource MUST include a single `targetRef` field. It must not -target more than one resource at a time, but it can be used to target larger -resources such as Gateways or Namespaces that may apply to multiple child -resources. - -As with most APIs, there are countless ways we could choose to expand this in -the future. This includes supporting multiple targetRefs and/or label selectors. -Although this would enable compelling functionality, it would increase the -complexity of an already complex API and potentially result in more conflicts -between policies. Although we may choose to expand the targeting capabilities -in the future, at this point it is strongly preferred to start with a simpler -pattern that still leaves room for future expansion. - -The `targetRef` field MUST have the following structure: - -```go -// PolicyTargetReference identifies an API object to apply policy to. -type PolicyTargetReference struct { - // Group is the group of the target resource. - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - Group string `json:"group"` - - // Kind is kind of the target resource. - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - Kind string `json:"kind"` - - // Name is the name of the target resource. - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - Name string `json:"name"` - - // Namespace is the namespace of the referent. When unspecified, the local - // namespace is inferred. Even when policy targets a resource in a different - // namespace, it may only apply to traffic originating from the same - // namespace as the policy. - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - // +optional - Namespace string `json:"namespace,omitempty"` -} -``` - -### Sample Policy API -The following structure can be used as a starting point for any Policy resource -using this API pattern. Note that the PolicyTargetReference struct defined above -will be distributed as part of the Gateway API. - -```go -// ACMEServicePolicy provides a way to apply Service policy configuration with -// the ACME implementation of the Gateway API. -type ACMEServicePolicy struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - // Spec defines the desired state of ACMEServicePolicy. - Spec ACMEServicePolicySpec `json:"spec"` - - // Status defines the current state of ACMEServicePolicy. - Status ACMEServicePolicyStatus `json:"status,omitempty"` -} - -// ACMEServicePolicySpec defines the desired state of ACMEServicePolicy. -type ACMEServicePolicySpec struct { - // TargetRef identifies an API object to apply policy to. - TargetRef gatewayv1a2.PolicyTargetReference `json:"targetRef"` - - // Override defines policy configuration that should override policy - // configuration attached below the targeted resource in the hierarchy. - // +optional - Override *ACMEPolicyConfig `json:"override,omitempty"` - - // Default defines default policy configuration for the targeted resource. - // +optional - Default *ACMEPolicyConfig `json:"default,omitempty"` -} - -// ACMEPolicyConfig contains ACME policy configuration. -type ACMEPolicyConfig struct { - // Add configurable policy here -} - -// ACMEServicePolicyStatus defines the observed state of ACMEServicePolicy. -type ACMEServicePolicyStatus struct { - // Conditions describe the current conditions of the ACMEServicePolicy. - // - // +optional - // +listType=map - // +listMapKey=type - // +kubebuilder:validation:MaxItems=8 - Conditions []metav1.Condition `json:"conditions,omitempty"` -} -``` - -### Hierarchy -Each policy MAY include default or override values. Default values are given -precedence from the bottom up, while override values are top down. That means -that a default attached to a Backend will have the highest precedence among -default values while an override value attached to a GatewayClass will have the -highest precedence overall. - -![Ingress and Sidecar Hierarchy](images/713-hierarchy.png) - -To illustrate this, consider 3 resources with the following hierarchy: -A > B > C. When attaching the concept of defaults and overrides to that, the -hierarchy would be expanded to this: - -A override > B override > C override > C default > B default > A default. - -Note that the hierarchy is reversed for defaults. The rationale here is that -overrides usually need to be enforced top down while defaults should apply to -the lowest resource first. For example, if an admin needs to attach required -policy, they can attach it as an override to a Gateway. That would have -precedence over Routes and Services below it. On the other hand, an app owner -may want to set a default timeout for their Service. That would have precedence -over defaults attached at higher levels such as Route or Gateway. - -If using defaults and overrides, each policy resource MUST include 2 structs -within the spec. One with override values and the other with default values. - -In the following example, the policy attached to the Gateway requires cdn to -be enabled and provides some default configuration for that. The policy attached -to the Route changes the value for one of those fields (includeQueryString). - -```yaml -kind: GKEServicePolicy # Example of implementation specific policy name -spec: - override: - cdn: - enabled: true - default: - cdn: - cachePolicy: - includeHost: true - includeProtocol: true - includeQueryString: true - targetRef: - kind: Gateway - name: example ---- -kind: GKEServicePolicy -spec: - default: - cdn: - cachePolicy: - includeQueryString: false - targetRef: - kind: HTTPRoute - name: example -``` - -In this final example, we can see how the override attached to the Gateway has -precedence over the default drainTimeout value attached to the Route. At the -same time, we can see that the default connectionTimeout attached to the Route -has precedence over the default attached to the Gateway. - -![Hierarchical Policy Example](images/713-policy-hierarchy.png) - -#### Supported Resources -It is important to note that not every implementation will be able to support -policy attachment to each resource described in the hierarchy above. When that -is the case, implementations MUST clearly document which resources a policy may -be attached to. - -#### Attaching Policy to GatewayClass -GatewayClass may be the trickiest resource to attach policy to. Policy -attachment relies on the policy being defined within the same scope as the -target. This ensures that only users with write access to a policy resource in a -given scope will be able to modify policy at that level. Since GatewayClass is a -cluster scoped resource, this means that any policy attached to it must also be -cluster scoped. - -GatewayClass parameters provide an alternative to policy attachment that may be -easier for some implementations to support. These parameters can similarly be -used to set defaults and requirements for an entire GatewayClass. - -### Targeting External Services -In some cases (likely limited to mesh) we may want to apply policies to requests -to external services. To accomplish this, implementations can choose to support -a refernce to a virtual resource type: - -```yaml -apiVersion: networking.acme.io/v1alpha1 -kind: RetryPolicy -metadata: - name: foo -spec: - default: - maxRetries: 5 - targetRef: - group: networking.acme.io - kind: ExternalService - name: foo.com -``` - -### Conflict Resolution -It is possible for multiple policies to target the same resource. When this -happens, merging is the preferred outcome. If multiple policy resources target -the same resource _and_ have an identical field specified with different values, -precedence MUST be determined in order of the following criteria, continuing on -ties: - -* The oldest Policy based on creation timestamp. For example, a Policy with a - creation timestamp of "2021-07-15 01:02:03" is given precedence over a Policy - with a creation timestamp of "2021-07-15 01:02:04". -* The Policy appearing first in alphabetical order by `{namespace}/{name}`. For - example, foo/bar is given precedence over foo/baz. - -For a better user experience, a validating webhook can be implemented to prevent -these kinds of conflicts all together. - -### Kubectl Plugin -To help improve UX and standardization, a kubectl plugin will be developed that -will be capable of describing the computed sum of policy that applies to a given -resource, including policies applied to parent resources. - -Each Policy CRD that wants to be supported by this plugin will need to follow -the API structure defined above and add a `gateway.networking.k8s.io/policy: -true` label to the CRD. - -### Status -In the future, we may consider adding a new `Policies` field to status on -Gateways and Routes. This would be a list of `PolicyTargetReference` structs -with the fields instead used to refer to the Policy resource that has been -applied. - -Unfortunately, this may create more confusion than it is worth, here are some of -the key concerns: - -* When multiple controllers are implementing the same Route and recognize a - policy, it would be difficult to determine which controller should be - responsible for adding that policy reference to status. -* For this to be somewhat scalable, we'd need to limit the status entries to - policies that had been directly applied to the resource. This could get - confusing as it would not provide any insight into policies attached above or - below. -* Since we only control some of the resources a policy might be attached to, - adding policies to status would only be possible on Gateway API resources, not - Services or other kinds of backends. - -Although these concerns are not unsolvable, they lead to the conclusion that -a Kubectl plugin should be our primary approach to providing visibility here, -with a possibility of adding policies to status at a later point. - -### Conditions -Controllers using the Gateway API policy attachment model SHOULD populate the -following condition and reasons on policy resources to provide a consistent -experience across implementations. - -```go -// PolicyConditionType is a type of condition for a policy. -type PolicyConditionType string - -// PolicyConditionReason is a reason for a policy condition. -type PolicyConditionReason string - -const ( - // PolicyConditionAccepted indicates whether the policy has been accepted or rejected - // by a targeted resource, and why. - // - // Possible reasons for this condition to be True are: - // - // * "Accepted" - // - // Possible reasons for this condition to be False are: - // - // * "Conflicted" - // * "Invalid" - // * "TargetNotFound" - // - PolicyConditionAccepted PolicyConditionType = "Accepted" - - // PolicyReasonAccepted is used with the "Accepted" condition when the policy has been - // accepted by the targeted resource. - PolicyReasonAccepted PolicyConditionReason = "Accepted" - - // PolicyReasonConflicted is used with the "Accepted" condition when the policy has not - // been accepted by a targeted resource because there is another policy that targets the same - // resource and a merge is not possible. - PolicyReasonConflicted PolicyConditionReason = "Conflicted" - - // PolicyReasonInvalid is used with the "Accepted" condition when the policy is syntactically - // or semantically invalid. - PolicyReasonInvalid PolicyConditionReason = "Invalid" - - // PolicyReasonTargetNotFound is used with the "Accepted" condition when the policy is attached to - // an invalid target resource - PolicyReasonTargetNotFound PolicyConditionReason = "TargetNotFound" -) -``` - -### Interaction with Custom Filters and other extension points -There are multiple methods of custom extension in the Gateway API. Policy -attachment and custom Route filters are two of these. Policy attachment is -designed to provide arbitrary configuration fields that decorate Gateway API -resources. Route filters provide custom request/response filters embedded inside -Route resources. Both are extension methods for fields that cannot easily be -standardized as core or extended fields of the Gateway API. The following -guidance should be considered when introducing a custom field into any Gateway -controller implementation: - -1. For any given field that a Gateway controller implementation needs, the - possibility of using core or extended should always be considered before - using custom policy resources. This is encouraged to promote standardization - and, over time, to absorb capabilities into the API as first class fields, - which offer a more streamlined UX than custom policy attachment. - -2. Although it's possible that arbitrary fields could be supported by custom - policy, custom route filters, and core/extended fields concurrently, it is - strongly recommended that implementations not use multiple mechanisms for - representing the same fields. A given field should only be supported through - a single extension method. An example of potential conflict is policy - precedence and structured hierarchy, which only applies to custom policies. - Allowing a field to exist in custom policies and also other areas of the API, - which are not part of the structured hierarchy, breaks the precedence model. - Note that this guidance may change in the future as we gain a better - understanding for extension mechanisms of the Gateway API can interoperate. - -### Conformance Level -This policy attachment pattern is associated with an "EXTENDED" conformance -level. The implementations that support this policy attachment model will have -the same behavior and semantics, although they may not be able to support -attachment of all types of policy at all potential attachment points. - -### Apply Policies to Sections of a Resource (Future Extension) -Although initially out of scope, it would be helpful to be able to target -specific matches within nested objects. For example, it may be useful to attach -policies to a specific Gateway listener or Route rule. This section explores -what that could look like. - -Each Route rule or Gateway listener should be expanded with an optional name -field. The target ref would be expanded with an optional sectionName field that -could be used to refer to that specific section of the resource. It would refer -to the following concepts on these resources: - -* Gateway.Listeners.Name -* xRoute.Rules.Name -* Service.Ports.Name - -```yaml -apiVersion: gateway.networking.k8s.io/v1alpha2 -kind: HTTPRoute -metadata: - name: http-app-1 - labels: - app: foo -spec: - hostnames: - - "foo.com" - rules: - - name: bar - matches: - - path: - type: Prefix - value: /bar - forwardTo: - - serviceName: my-service1 - port: 8080 ---- -apiVersion: networking.acme.io/v1alpha2 -kind: RetryPolicy -metadata: - name: foo -spec: - maxRetries: 5 - targetRef: - name: foo - group: gateway.networking.k8s.io - kind: HTTPRoute - sectionName: bar -``` - -This would require adding a `SectionName` field to the PolicyTargetReference: -```go -type PolicyTargetReference struct { - // SectionName is the name of a section within the target resource. When - // unspecified, this targets the entire resource. In the following - // resources, SectionName is interpreted as the following: - // * Gateway: Listener Name - // * Route: Rule Name - // * Service: Port Name - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - // +optional - SectionName string `json:"sectionName,omitempty"` - // ... -} -``` - -This would also require adding a `Name` field to Gateway listeners and Route -rules: - -```go -type Listener struct { - // Name is the name of the Listener. If more than one Listener is present - // each Listener MUST specify a name. The names of Listeners MUST be unique - // within a Gateway. - // - // Support: Core - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - // +optional - Name string `json:"name,omitempty"` - // ... -} -``` - -```go -type RouteRule struct { - // Name is the name of the Route rule. If more than one Route Rule is - // present, each Rule MUST specify a name. The names of Rules MUST be unique - // within a Route. - // - // Support: Core - // - // +kubebuilder:validation:MinLength=1 - // +kubebuilder:validation:MaxLength=253 - // +optional - Name string `json:"name,omitempty"` - // ... -} -``` - -### Advantages -* Incredibly flexible approach that should work well for both ingress and mesh -* Conceptually similar to existing ServicePolicy proposal and BackendPolicy - pattern -* Easy to attach policy to resources we don’t control (Service, ServiceImport, - etc) -* Minimal API changes required -* Simplifies packaging an application for deployment as policy references do not - need to be part of the templating - -### Disadvantages -* May be difficult to understand which policies apply to a request - -## Removing BackendPolicy -BackendPolicy represents the initial attempt to cover policy attachment for -Gateway API. Although this proposal ended up with a similar structure to -BackendPolicy, it is not clear that we ever found sufficient value or use cases -for BackendPolicy. Given that this proposal provides more powerful ways to -attach policy, it makes sense to remove BackendPolicy until we have a better -alternative. - -## Alternatives - -### 1. ServiceBinding for attaching Policies and Routes for Mesh -A new ServiceBinding resource has been proposed for mesh use cases. This would -provide a way to attach policies, including Routes to a Service. - -Most notably, these provide a way to attach different policies to requests -coming from namespaces or specific Gateways. In the example below, a -ServiceBinding in the consumer namespace would be applied to the selected -Gateway and affect all requests from that Gateway to the foo Service. Beyond -policy attachment, this would also support attaching Routes as policies, in this -case the attached HTTPRoute would split requests between the foo-a and foo-b -Service instead of the foo Service. - -![Simple Service Binding Example](images/713-servicebinding-simple.png) - -This approach can be used to attach a default set of policies to all requests -coming from a namespace. The example below shows a ServiceBinding defined in the -producer namespace that would apply to all requests from within the same -namespace or from other namespaces that did not have their own ServiceBindings -defined. - -![Complex Service Binding Example](images/713-servicebinding-complex.png) - -#### Advantages -* Works well for mesh and any use cases where requests don’t always transit - through Gateways and Routes. -* Allows policies to apply to an entire namespace. -* Provides very clear attachment of polices, routes, and more to a specific - Service. -* Works well for ‘shrink-wrap application developers’ - the packaged app does - not need to know about hostnames or policies or have extensive templates. -* Works well for ‘dynamic’ / programmatic creation of workloads ( Pods,etc - see - CertManager) -* It is easy to understand what policy applies to a workload - by listing the - bindings in the namespace. - -#### Disadvantages -* Unclear how this would work with an ingress model. If Gateways, Routes, and - Backends are all in different namespaces, and each of those namespaces has - different ServiceBindings applying different sets of policies, it’s difficult - to understand which policy would be applied. -* Unclear if/how this would interact with existing the ingress focused policy - proposal described below. If both coexisted, would it be possible for a user - to understand which policies were being applied to their requests? -* Route status could get confusing when Routes were referenced as a policy by - ServiceBinding -* Introduces a new mesh specific resource. - -### 2. Attaching Policies for Ingress -An earlier proposal for policy attachment in the Gateway API suggested adding -policy references to each Resource. This works very naturally for Ingress use -cases where all requests follow a path through Gateways, Routes, and Backends. -Adding policy attachment at each level enables different roles to define -defaults and allow overrides at different levels. - -![Simple Ingress Attachment Example](images/713-ingress-attachment.png) - -#### Advantages -* Consistent policy attachment at each level -* Clear which policies apply to each component -* Naturally translates to hierarchical Ingress model with ability to delegate - policy decisions to different roles - -#### Disadvantages -* Policy overrides could become complicated -* At least initially, policy attachment on Service would have to rely on Service - annotations or references from policy to Service(s) -* No way to attach policy to other resources such as namespace or ServiceImport -* May be difficult to modify Routes and Services if other components/roles are - managing them (eg Knative) - -### 3. Shared Policy Resource -This is really just a slight variation or extension of the main proposal in this -GEP. We would introduce a shared policy resource. This resource would follow the -guidelines described above, including the `targetRef` as defined as well as -`default` and `override` fields. Instead of carefully crafted CRD schemas for -each of the `default` and `override` fields, we would use more generic -`map[string]string` values. This would allow similar flexibility to annotations -while still enabling the default and override concepts that are key to this -proposal. - -Unfortunately this would be difficult to validate and would come with many of -the downsides of annotations. A validating webhook would be required for any -validation which could result in just as much or more work to maintain than -CRDs. At this point we believe that the best experience will be from -implementations providing their own policy CRDs that follow the patterns -described in this GEP. We may want to explore tooling or guidance to simplify -the creation of these policy CRDs to help simplify implementation and extension -of this API. - -## References - -**Issues** -* [Extensible Service Policy and Configuration](https://github.com/kubernetes-sigs/gateway-api/issues/611) - -**Docs** -* [Policy Attachment and Binding](https://docs.google.com/document/d/13fyptUtO9NV_ZAgkoJlfukcBf2PVGhsKWG37yLkppJo/edit?resourcekey=0-Urhtj9gBkGBkSL1gHgbWKw) diff --git a/site-src/geps/overview.md b/site-src/geps/overview.md deleted file mode 100644 index d31141722a..0000000000 --- a/site-src/geps/overview.md +++ /dev/null @@ -1,117 +0,0 @@ -# Gateway Enhancement Proposal (GEP) - -Gateway Enhancement Proposals (GEPs) serve a similar purpose to the [KEP][kep] -process for the main Kubernetes project: - -1. Ensure that changes to the API follow a known process and discussion - in the OSS community. -1. Make changes and proposals discoverable (current and future). -1. Document design ideas, tradeoffs, decisions that were made for - historical reference. - -## Process - -### 1. Circulate the Idea -Before creating a GEP, share your high level idea with the community. This can -be in one of many forms: - -- A [new GitHub Discussion](https://github.com/kubernetes-sigs/gateway-api/discussions/new) -- On our [Slack Channel](https://kubernetes.slack.com/archives/CR0H13KGA) -- On one of our [community meetings](https://gateway-api.sigs.k8s.io/contributing/?h=meetings#meetings) - -### 2. Agree on the Goals -Although it can be tempting to start writing out all the details of your -proposal, it's important to first ensure we all agree on the goals. The first -version of your GEP should aim for a "Provisional" status and leave out any -implementation details, focusing primarily on "Goals" and "Non-Goals". - -### 3. Document Implementation Details -Now that everyone agrees on the goals, it is time to start writing out your -proposed implementation details. These implementation details should be very -thorough, including the proposed API spec, and covering any relevant edge cases. -Note that it may be helpful to use a shared doc for part of this phase to enable -faster iteration on potential designs. - -It is likely that throughout this process, you will discuss a variety of -alternatives. Be sure to document all of these in the GEP, and why we decided -against them. At this stage, the GEP should be targeting the "Implementable" -stage. - -### 4. Implement the GEP as "Experimental" -With the GEP marked as "Implementable", it is time to actually make those -proposed changes in our API. In some cases, these changes will be documentation -only, but in most cases, some API changes will also be required. It is important -that every new feature of the API is marked as "Experimental" when it is -introduced. Within the API, we use `` tags to denote -experimental fields. - -Before these changes are released, they will also need to be documented. -GEPs that have not been both implemented and documented before a release -cut off will be excluded from the release. - -### 5. Graduate the GEP to "Standard" -Once this feature has met the [graduation criteria](/concepts/versioning/#graduation-criteria), it is -time to graduate it to the "Standard" channel of the API. Depending on the feature, this may include -any of the following: - -1. Graduating the resource to beta -2. Graduating fields to "standard" by removing `` tags -3. Graduating a concept to "standard" by updating documentation - -## Status - -Each GEP has a status field that defines it's current state. Each transition -will require a PR to update the GEP and should be discussed at a community -meeting before merging. Most GEPS will proceed through the following states: - -* **Provisional:** The goals described by this GEP have consensus but - implementation details have not been agreed to yet. -* **Implementable:** The goals and implementation details described by this GEP - have consensus but have not been fully implemented yet. -* **Experimental:** This GEP has been implemented and is part of the - "Experimental" release channel. Breaking changes are still possible. -* **Standard:** This GEP has been implemented and is part of the - "Standard" release channel. It should be quite stable. - -Although less common, some GEPs may end up in one of the following states: - -* **Deferred:** We do not currently have bandwidth to handle this GEP, it - may be revisited in the future. -* **Rejected:** This proposal was considered by the community but ultimately - rejected. -* **Replaced:** This proposal was considered by the community but ultimately - replaced by a newer proposal. -* **Withdrawn:** This proposal was considered by the community but ultimately - withdrawn by the author. - -## Format - -GEPs should match the format of the template found in [GEP-696](/geps/gep-696). - -## Out of scope - -What is out of scope: see [text from KEP][kep-when-to-use]. Examples: - -* Bug fixes -* Small changes (API validation, documentation, fixups). It is always - possible that the reviewers will determine a "small" change ends up - requiring a GEP. - -## FAQ - -* Q: Why is it named GEP? - * A: To avoid potential confusion if people start following the cross - references to the full KEP process. -* Q: Why have a different process than mainline? - * A: We would like to keep the machinery to an absolute minimum for now -- - this may change as we move to v1. -* Q: Is it ok to discuss using shared docs, scratch docs etc? - * A: Yes, this can be a helpful intermediate step when iterating on design - details. It is important that all major feedback, discussions, and - alternatives considered in that step are represented in the GEP though. A - key goal of GEPs is to show why we made a decision and which alternatives - were considered. If separate docs are used, it's important that we can - still see all relevant context and decisions in the final GEP. - -[kep]: https://github.com/kubernetes/enhancements -[kep-when-to-use]: https://github.com/kubernetes/enhancements/tree/master/keps#do-i-have-to-use-the-kep-process diff --git a/site-src/guides/http-header-modifier.md b/site-src/guides/http-header-modifier.md new file mode 100644 index 0000000000..f3af1a7135 --- /dev/null +++ b/site-src/guides/http-header-modifier.md @@ -0,0 +1,65 @@ +# HTTP Header Modifiers + +[HTTPRoute resources](/api-types/httproute) can modify the headers of HTTP requests and the HTTP responses from clients. +There are two types of [filters](/api-types/httproute#filters-optional) available to meet these requirements: `RequestHeaderModifier` and `ResponseHeaderModifier`. + +This guide shows how to use these features. + +Note that these features are compatible. HTTP headers of the incoming requests and the headers of their responses can both be modified using a single [HTTPRoute resource](/api-types/httproute). + +## HTTP Request Header Modifier + +HTTP header modification is the process of adding, removing, or modifying HTTP headers in incoming requests. + +To configure HTTP header modification, define a Gateway object with one or more HTTP filters. Each filter specifies a specific modification to make to incoming requests, such as adding a custom header or modifying an existing header. + +To add a header to a HTTP request, use a filter of the type `RequestHeaderModifier`, with the `add` action and the name and value of the header: + +```yaml +{% include 'standard/http-request-header-add.yaml' %} +``` + +To edit an existing header, use the `set` action and specify the value of the header to be modified and the new header value to be set. + +```yaml + filters: + - type: RequestHeaderModifier + requestHeaderModifier: + set: + - name: my-header-name + value: my-new-header-value +``` + +Headers can also be removed, by using the `remove` keyword and a list of header names. + +```yaml + filters: + - type: RequestHeaderModifier + requestHeaderModifier: + remove: ["x-request-id"] +``` + +Using the example above would remove the `x-request-id` header from the HTTP request. + +### HTTP Response Header Modifier + +Just like editing request headers can be useful, the same goes for response headers. For example, it allows teams to add/remove cookies for only a certain backend, which can help in identifying certain users that were redirected to that backend previously. + +Another potential use case could be when you have a frontend that needs to know whether it’s talking to a stable or a beta version of the backend server, in order to render different UI or adapt its response parsing accordingly. + +Modifying the HTTP header response leverages a very similar syntax to the one used to modify the original request, albeit with a different filter (`ResponseHeaderModifier`). + +Headers can be added, edited and removed. Multiple headers can be added, as shown in this example below: + +```yaml + filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + add: + - name: X-Header-Add-1 + value: header-add-1 + - name: X-Header-Add-2 + value: header-add-2 + - name: X-Header-Add-3 + value: header-add-3 +``` diff --git a/site-src/guides/http-redirect-rewrite.md b/site-src/guides/http-redirect-rewrite.md index 10f48c6dec..b1f5f7d577 100644 --- a/site-src/guides/http-redirect-rewrite.md +++ b/site-src/guides/http-redirect-rewrite.md @@ -10,7 +10,7 @@ use both filter types at once. ## Redirects -Redirects return HTTP 3XX responses to a client, instructing it to retrive a +Redirects return HTTP 3XX responses to a client, instructing it to retrieve a different resource. [`RequestRedirect` rule filters](/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPRequestRedirectFilter) instruct Gateways to emit a redirect response to requests matching a filtered @@ -21,7 +21,7 @@ example, to issue a permanent redirect (301) from HTTP to HTTPS, configure `requestRedirect.statusCode=301` and `requestRedirect.scheme="https"`: ```yaml -{% include 'experimental/http-redirect-rewrite/httproute-redirect-https.yaml' %} +{% include 'standard/http-redirect-rewrite/httproute-redirect-https.yaml' %} ``` Redirects change configured URL components to match the redirect configuration @@ -36,15 +36,15 @@ unchanged. !!! info "Experimental Channel" The `Path` field described below is currently only included in the - "Experimental" channel of Gateway API. For more information on release - channels, refer to the [related documentation](https://gateway-api.sigs.k8s.io/concepts/versioning). + "Experimental" channel of Gateway API. Starting in v0.7.0, this + feature will graduate to the "Standard" channel. Path redirects use an HTTP Path Modifier to replace either entire paths or path prefixes. For example, the HTTPRoute below will issue a 302 redirect to all `redirect.example` requests whose path begins with `/cayenne` to `/paprika`: ```yaml -{% include 'experimental/http-redirect-rewrite/httproute-redirect-full.yaml' %} +{% include 'standard/http-redirect-rewrite/httproute-redirect-full.yaml' %} ``` Both requests to @@ -56,7 +56,7 @@ The other path redirect type, `ReplacePrefixMatch`, replaces only the path portion matching `matches.path.value`. Changing the filter in the above to: ```yaml -{% include 'experimental/http-redirect-rewrite/httproute-redirect-prefix.yaml' %} +{% include 'standard/http-redirect-rewrite/httproute-redirect-prefix.yaml' %} ``` will result in redirects with `location: @@ -65,12 +65,6 @@ https://redirect.example/paprika/teaspoon` response headers. ## Rewrites -!!! info "Experimental Channel" - - The `URLRewrite` filter described below is currently only included in the - "Experimental" channel of Gateway API. For more information on release - channels, refer to the [related documentation](https://gateway-api.sigs.k8s.io/concepts/versioning). - Rewrites modify components of a client request before proxying it upstream. A [`URLRewrite` filter](/references/spec/#gateway.networking.k8s.io/v1beta1.HTTPURLRewriteFilter) @@ -81,7 +75,7 @@ following HTTPRoute will accept a request for rewrite.example`. ```yaml -{% include 'experimental/http-redirect-rewrite/httproute-rewrite.yaml' %} +{% include 'standard/http-redirect-rewrite/httproute-rewrite.yaml' %} ``` Path rewrites also make use of HTTP Path Modifiers. The HTTPRoute below @@ -91,5 +85,5 @@ Instead using `type: ReplacePrefixMatch` and `replacePrefixMatch: /fennel` will request `https://elsewhere.example/fennel/smidgen` upstream. ```yaml -{% include 'experimental/http-redirect-rewrite/httproute-rewritepath.yaml' %} +{% include 'standard/http-redirect-rewrite/httproute-rewritepath.yaml' %} ``` diff --git a/site-src/guides/index.md b/site-src/guides/index.md index e6f2695d7e..98e5bec5ab 100644 --- a/site-src/guides/index.md +++ b/site-src/guides/index.md @@ -14,7 +14,7 @@ _THEN_ - [Routing across Namespaces](/guides/multiple-ns) - [Configuring TLS](/guides/tls) - [TCP routing](/guides/tcp) -- [gRPC routing](/guides/grpc) +- [gRPC routing](/guides/grpc-routing) - [Migrating from Ingress](/guides/migrating-from-ingress) ## Installing a Gateway controller @@ -36,18 +36,18 @@ channels with different levels of stability: ### Install Standard Channel The standard release channel includes all resources that have graduated to beta, -including GatewayClass, Gateway, and HTTPRoute. To install this channel, run the -following kubectl command: +including GatewayClass, Gateway, ReferenceGrant, and HTTPRoute. To install this +channel, run the following kubectl command: ``` -kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.5.1/standard-install.yaml +kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.7.1/standard-install.yaml ``` ### Install Experimental Channel The experimental release channel includes everything in the standard release channel plus some experimental resources and fields. This includes -ReferencePolicy, TCPRoute, TLSRoute, and UDPRoute. +TCPRoute, TLSRoute, UDPRoute and GRPCRoute. Note that future releases of the API could include breaking changes to experimental resources and fields. For example, any experimental resource or @@ -58,7 +58,7 @@ documentation](https://gateway-api.sigs.k8s.io/concepts/versioning/). To install the experimental channel, run the following kubectl command: ``` -kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.5.1/experimental-install.yaml +kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v0.7.1/experimental-install.yaml ``` ### Cleanup diff --git a/site-src/guides/multiple-ns.md b/site-src/guides/multiple-ns.md index a50a18ec7a..e845beb2ed 100644 --- a/site-src/guides/multiple-ns.md +++ b/site-src/guides/multiple-ns.md @@ -22,8 +22,7 @@ are their goals and how they use Gateway API resources to accomplish them: isolate access and configuration across their apps as much as possible to minimize access and failure domains. They use separate HTTPRoutes attached to the same Gateway to isolate routing configurations, such as canary rollouts, -and still share the same but share the same IP address, port, DNS domain, and -TLS certificate. +but still share the same IP address, port, DNS domain, and TLS certificate. - The store team has a single Service called _store_ that they have deployed in the `store-ns` Namespace which also needs to be exposed behind the same IP address and domain. @@ -44,7 +43,7 @@ The logical relationship between the Gateway API resources looks like this: attach to Gateways and program their routing rules. It is especially relevant when there are Routes across Namespaces that share one or more Gateways. Gateway and Route attachment is bidirectional - attachment can only succeed if -the Gateway owner and Route owner owner both agree to the relationship. This +the Gateway owner and Route owner both agree to the relationship. This bi-directional relationship exists for two reasons: - Route owners don't want to overexpose their applications through paths they diff --git a/site-src/guides/tls.md b/site-src/guides/tls.md index 5ee9de45e7..6d0c180e9a 100644 --- a/site-src/guides/tls.md +++ b/site-src/guides/tls.md @@ -57,15 +57,12 @@ listeners: port: 443 tls: mode: Terminate # If protocol is `TLS`, `Passthrough` is a possible mode - certificateRef: - kind: Secret + certificateRefs: + - kind: Secret group: "" name: default-cert ``` -If `hostname.match` is set to `Exact`, then the TLS settings apply to only the -specific hostname that is set in `hostname.name`. - ### Examples #### Listeners with different certificates diff --git a/site-src/implementations.md b/site-src/implementations.md index aa62e8c08d..de246b3937 100644 --- a/site-src/implementations.md +++ b/site-src/implementations.md @@ -4,13 +4,14 @@ This document tracks downstream implementations and integrations of Gateway API Implementors and integrators of Gateway API are encouraged to update this document with status information about their implementations, the versions they cover, and documentation to help users get started. -## Implementation Status - +## Gateway Controller Implementation Status - [Acnodal EPIC][1] (public preview) -- [Apache APISIX][2] (alpha) -- [BIG-IP Kubernetes Gateway][20] -- [Cilium][16] (work in progress) +- [Amazon Elastic Kubernetes Service][23] (alpha) +- [Apache APISIX][2] (beta) +- [Azure Application Gateway for Containers][27] (preview) +- [BIG-IP Kubernetes Gateway][20] (beta) +- [Cilium][16] (beta) - [Contour][3] (beta) - [Emissary-Ingress (Ambassador API Gateway)][4] (alpha) - [Envoy Gateway][18] (alpha) @@ -26,10 +27,21 @@ Implementors and integrators of Gateway API are encouraged to update this docume - [NGINX Kubernetes Gateway][12] - [STUNner][21] (beta) - [Traefik][13] (alpha) +- [WSO2 APK][25] (pre-alpha) + +## Service Mesh Implementation Status + +- [Istio][9] (experimental) +- [Kuma][11] (experimental) +- [Linkerd][28] (experimental) + +## Integrations -## Integration Status - [Flagger][14] (public preview) - [cert-manager][15] (alpha) +- [argo-rollouts][22] (alpha) +- [Knative][24] (alpha) +- [Kuadrant][26] (work in progress) [1]:#acnodal-epic [2]:#apisix @@ -52,6 +64,15 @@ Implementors and integrators of Gateway API are encouraged to update this docume [19]:#litespeed-ingress-controller [20]:#big-ip-kubernetes-gateway [21]:#stunner +[22]:#argo-rollouts +[23]:#amazon-elastic-kubernetes-service +[24]:#knative +[25]:#wso2-apk +[26]:#kuadrant +[27]:#azure-application-gateway-for-containers +[28]:#linkerd + +[gamma]:/concepts/gamma/ ## Implementations @@ -64,26 +85,45 @@ Documentation can be found at [EPIC Application & API Gateway Service][epic] [epic]:https://www.epick8sgw.io +### Amazon Elastic Kubernetes Service + +[Amazon Elastic Kubernetes Service (EKS)][eks] is a managed service that you can use to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane or nodes. EKS's implementation of the Gateway API is through [AWS Gateway API Controller][eks-gateway] which provisions [Amazon VPC Lattice][vpc-lattice] Resources for gateway(s), HTTPRoute(s) in EKS clusters. + +[eks]:https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html +[eks-gateway]:https://github.com/aws/aws-application-networking-k8s +[vpc-lattice]:https://aws.amazon.com/vpc/lattice/ + ### APISIX [Apache APISIX][apisix] is a dynamic, real-time, high-performance API Gateway. APISIX provides rich traffic management features such as load balancing, dynamic upstream, canary release, circuit breaking, authentication, observability, and more. -APISIX currently supports Gateway API `v1alpha2` version of the specification for its [Apache APISIX Ingress Controller][apisix-1]. +APISIX currently supports Gateway API `v1beta1` version of the specification for its [Apache APISIX Ingress Controller][apisix-1]. [apisix]:https://apisix.apache.org/ [apisix-1]:https://github.com/apache/apisix-ingress-controller +### Azure Application Gateway for Containers + +[Application Gateway for Containers][azure-application-gateway-for-containers] is a managed application (layer 7) load balancing solution, providing dynamic traffic management capabilities for workloads running in a Kubernetes cluster in Azure. Follow the [quickstart guide][azure-application-gateway-for-containers-quickstart-controller] to deploy the ALB controller and get started with Gateway API. + +Application Gateway for Containers implements `v1beta1` specification of Gateway API. + +[azure-application-gateway-for-containers]:https://aka.ms/appgwcontainers/docs +[azure-application-gateway-for-containers-quickstart-controller]:https://learn.microsoft.com/azure/application-gateway/for-containers/quickstart-deploy-application-gateway-for-containers-alb-controller + ### BIG-IP Kubernetes Gateway [BIG-IP Kubernetes Gateway][big-ip-kubernetes-gateway] is an open-source project that provides an implementation of the Gateway API using [F5 BIG-IP][f5bigip] as the data plane. It provides enterprises with high-performance Gateway API implementation. -We are actively supporting various features of the Gateway API. For compatibility with the features of the Gateway API, please refer to [here][bigipgwfeatures]. For any questions about this project, welcome to create [Issues][bigipgwissues] or [PR][bigipgwpr]. +We are actively supporting various features of the Gateway API. For compatibility with the features of the Gateway API, please refer to [here][bigipgwfeatures]. For any questions about this project, welcome to create [Issues][bigipgwissues] or [PR][bigipgwpr]. Also, you are welcome to connect with us in the [slack channel][bigipgwslacklink]. + [big-ip-kubernetes-gateway]:https://gateway-api.f5se.io/ [f5bigip]:https://f5.com [bigipgwfeatures]:https://github.com/f5devcentral/bigip-kubernetes-gateway/blob/master/docs/gateway-api-compatibility.md [bigipgwissues]:https://github.com/f5devcentral/bigip-kubernetes-gateway/issues [bigipgwpr]:https://github.com/f5devcentral/bigip-kubernetes-gateway/pulls +[bigipgwslacklink]: https://gateway-api.f5se.io/Support-and-contact/ ### Cilium @@ -92,10 +132,10 @@ solution for Kubernetes and other networking environments. It includes [Cilium Service Mesh][cilium-service-mesh], a highly efficient mesh data plane that can be run in [sidecarless mode][cilium-sidecarless] to dramatically improve performance, and avoid the operational complexity of sidecars. Cilium also -supports the sidecar proxy model, offering choice to users. Cilium is [working on -a Gateway API implementation][cilium-issue]. +supports the sidecar proxy model, offering choice to users. As of [Cilium 1.13][cilium113blog], +Cilium supports Gateway API, passing conformance for v0.5.1. -Cilium is open source and is a CNCF incubation project. +Cilium is open source and is a CNCF incubation project. If you have questions about Cilium Service Mesh the #service-mesh channel on [Cilium Slack][cilium-slack] is a good place to start. For contributing to the development @@ -104,7 +144,7 @@ effort, check out the #development channel or join our [weekly developer meeting [cilium]:https://cilium.io [cilium-service-mesh]:https://docs.cilium.io/en/stable/gettingstarted/#service-mesh [cilium-sidecarless]:https://isovalent.com/blog/post/cilium-service-mesh/ -[cilium-issue]:https://github.com/cilium/cilium/issues/20655 +[cilium113blog]:https://isovalent.com/blog/post/cilium-release-113/ [cilium-slack]:https://cilium.io/slack [cilium-meeting]:https://github.com/cilium/cilium#weekly-developer-meeting @@ -112,9 +152,9 @@ effort, check out the #development channel or join our [weekly developer meeting [Contour][contour] is a CNCF open source Envoy-based ingress controller for Kubernetes. -Contour implements Gateway API v0.5.1, supporting the v1alpha2 and v1beta1 API versions. -All [Standard channel][contour-standard] resources (GatewayClass, Gateway, HTTPRoute), plus ReferenceGrant and TLSRoute, are supported. -Contour's implementation passes all Gateway API conformance tests included in the v0.5.1 release. +Contour [v1.25.0][contour-release] implements Gateway API v0.6.2, supporting the v1alpha2 and v1beta1 API versions. +All [Standard channel][contour-standard] resources (GatewayClass, Gateway, HTTPRoute, ReferenceGrant), plus TLSRoute and GRPCRoute, are supported. +Contour's implementation passes all core and most extended Gateway API conformance tests included in the v0.6.2 release. See the [Contour Gateway API Guide][contour-guide] for information on how to deploy and use Contour's Gateway API implementation. @@ -123,6 +163,7 @@ For help and support with Contour's implementation, [create an issue][contour-is _Some "extended" functionality is not implemented yet, [contributions welcome!][contour-contrib]._ [contour]:https://projectcontour.io +[contour-release]:https://github.com/projectcontour/contour/releases/tag/v1.25.0 [contour-standard]:https://gateway-api.sigs.k8s.io/concepts/versioning/#release-channels-eg-experimental-standard [contour-guide]:https://projectcontour.io/guides/gateway-api/ [contour-issue-new]:https://github.com/projectcontour/contour/issues/new/choose @@ -141,15 +182,15 @@ See [here][emissary-gateway-api] for more details on using the Gateway API with ### Envoy Gateway -[Envoy Gateway][eg-home] is an [Envoy][envoy-org] subproject for managing Envoy-based application gateways. The -[v0.2][eg-02] release includes support for most `v1beta1` Gateway API features and passes core conformance tests -included in the v0.5.1 release. Use the [quickstart][eg-quickstart] to get Envoy Gateway running with Gateway API in a +[Envoy Gateway][eg-home] is an [Envoy][envoy-org] subproject for managing Envoy-based application gateways. The supported +APIs and fields of the Gateway API are outlined [here][eg-supported]. +Use the [quickstart][eg-quickstart] to get Envoy Gateway running with Gateway API in a few simple steps. [eg-home]:https://gateway.envoyproxy.io/ [envoy-org]:https://github.com/envoyproxy -[eg-02]:https://gateway.envoyproxy.io/v0.2.0/releases/v0.2.html -[eg-quickstart]:https://gateway.envoyproxy.io/v0.2.0/user/quickstart.html +[eg-supported]: https://gateway.envoyproxy.io/v0.4.0/design/gatewayapi-support.html +[eg-quickstart]:https://gateway.envoyproxy.io/v0.4.0/user/quickstart.html ### Flomesh Service Mesh (FSM) @@ -195,7 +236,7 @@ HAProxy Ingress v0.13 partially supports the Gateway API's v1alpha1 specificatio ### HashiCorp Consul -[Consul][consul], by [HashiCorp][hashicorp], is an open source control plane for multi-cloud networking. A single Consul deployment can span bare metal, VM and container environments. +[Consul][consul], by [HashiCorp][hashicorp], is an open source control plane for multi-cloud networking. A single Consul deployment can span bare metal, VM and container environments. Consul service mesh works on any Kubernetes distribution, connects multiple clusters, and Consul CRDs provide a Kubernetes native workflow to manage traffic patterns and permissions in the mesh. [Consul API Gateway][consul-api-gw-doocs] supports Gatewway API for managing North-South traffic. @@ -207,18 +248,20 @@ Please see the [Consul API Gateway documentation][consul-api-gw-doocs] for curre ### Istio -[Istio][istio] is an open source [service mesh][mesh] and gateway implementation. +[Istio][istio] is an open source [service mesh][istio-mesh] and gateway implementation. -A light-weight minimal install of Istio can be used to provide a Beta-quality implementation of the Kubernetes Gateway API for cluster ingress traffic control. For service mesh users, -the Istio implementation also lets you start trying out the experimental Gateway API [support for east-west traffic management][gamma] within the mesh. +A light-weight minimal install of Istio can be used to provide a Beta-quality +implementation of the Kubernetes Gateway API for cluster ingress traffic +control. For service mesh users, Istio 1.16 and later support the [GAMMA +initiative's][gamma] experimental Gateway API [support for east-west traffic +management][gamma] within the mesh. Much of Istio's documentation, including all of the [ingress tasks][istio-1] and several mesh-internal traffic management tasks, already includes parallel instructions for configuring traffic using either the Gateway API or the Istio configuration API. Check out the [Gateway API task][istio-2] for more information about the Gateway API implementation in Istio. [istio]:https://istio.io -[mesh]:https://istio.io/latest/docs/concepts/what-is-istio/#what-is-a-service-mesh -[gamma]:https://gateway-api.sigs.k8s.io/contributing/gamma/ +[istio-mesh]:https://istio.io/latest/docs/concepts/what-is-istio/#what-is-a-service-mesh [istio-1]:https://istio.io/latest/docs/tasks/traffic-management/ingress/ [istio-2]:https://istio.io/latest/docs/tasks/traffic-management/ingress/gateway-api/ @@ -246,9 +289,25 @@ For help and support with Kong's implementations please feel free to [create an Kuma is actively working on an implementation of Gateway API specification for the Kuma builtin Gateway. Check the [Gateway API Documentation][kuma-1] for information on how to setup a Kuma builtin gateway using the Gateway API. +Kuma 2.3 and later support the [GAMMA initiative's][gamma] experimental +Gateway API [support for east-west traffic management][gamma] within the mesh. + [kuma]:https://kuma.io [kuma-1]:https://kuma.io/docs/latest/explore/gateway-api/ +### Linkerd + +[Linkerd][linkerd] is the first CNCF graduated [service mesh][linkerd-mesh]. +It is the only major mesh not based on Envoy, instead relying on a +purpose-built Rust micro-proxy to bring security, observability, and +reliability to Kubernetes, without the complexity. + +Linkerd 2.14 and later support the [GAMMA initiative's][gamma] experimental +Gateway API [support for east-west traffic management][gamma] within the mesh. + +[linkerd]:https://linkerd.io/ +[linkerd-mesh]:https://buoyant.io/service-mesh-manifesto + ### LiteSpeed Ingress Controller The [LiteSpeed Ingress Controller](https://litespeedtech.com/products/litespeed-web-adc/features/litespeed-ingress-controller) uses the LiteSpeed WebADC controller to operate as an Ingress Controller and Load Balancer to manage your traffic on your Kubernetes cluster. It implements the full core Gateway API including Gateway, GatewayClass, HTTPRoute and ReferenceGrant and the Gateway functions of cert-manager. Gateway is fully integrated into the LiteSpeed Ingress Controller. @@ -289,6 +348,19 @@ Traefik is currently working on implementing UDP, and ReferenceGrant. Status upd [traefik]:https://traefik.io [traefik-1]:https://doc.traefik.io/traefik/routing/providers/kubernetes-gateway/ +### WSO2 APK + +[WSO2 APK][wso2-apk] is a purpose-built API management solution tailored for Kubernetes environments, delivering seamless integration, flexibility, and scalability to organizations in managing their APIs. + +WSO2 APK implements the Gateway API, encompassing Gateway and HTTPRoute functionalities. Additionally, it provides support for rate limiting, authentication/authorization, and analytics/observability through the use of Custom Resources (CRs). + +For up-to-date information on the supported version and features of the Gateway API, please refer to the [APK Gateway documentation][apk-doc]. If you have any questions or would like to contribute, feel free to create [issues or pull requests][repo]. Join our [Discord channel][discord] to connect with us and engage in discussions. + +[wso2-apk]:https://apk.docs.wso2.com/en/latest/ +[apk-doc]:https://apk.docs.wso2.com/en/latest/catalogs/kubernetes-crds/ +[repo]:https://github.com/wso2/apk +[discord]:https://discord.com/channels/955510916064092180/1113056079501332541 + ## Integrations In this section you will find specific links to blog posts, documentation and other Gateway API references for specific integrations. @@ -297,7 +369,7 @@ In this section you will find specific links to blog posts, documentation and ot [Flagger][flagger] is a progressive delivery tool that automates the release process for applications running on Kubernetes. -Flagger can be used to automate canary deployments and A/B testing using Gateway API. It currently supports the `v1alpha2` spec of Gateway API. You can refer to [this tutorial][flagger-tutorial] to use Flagger with any implementation of Gateway API. +Flagger can be used to automate canary deployments and A/B testing using Gateway API. It supports both the `v1alpha2` and `v1beta1` spec of Gateway API. You can refer to [this tutorial][flagger-tutorial] to use Flagger with any implementation of Gateway API. [flagger]:https://flagger.app [flagger-tutorial]:https://docs.flagger.app/tutorials/gatewayapi-progressive-delivery @@ -310,3 +382,30 @@ cert-manager can generate TLS certificates for Gateway resources. This is config [cert-manager]:https://cert-manager.io/ [cert-manager-docs]:https://cert-manager.io/docs/usage/gateway/ + +### Argo rollouts + +[Argo Rollouts][argo-rollouts] is a progressive delivery controller for Kubernetes. It supports several advanced deployment methods such as blue/green and canaries. Argo Rollouts supports the Gateway API via [a plugin][argo-rollouts-plugin]. + +[argo-rollouts]:https://argo-rollouts.readthedocs.io/en/stable/ +[argo-rollouts-plugin]:https://github.com/argoproj-labs/rollouts-gatewayapi-trafficrouter-plugin/ + +### Knative + +[Knative][knative] is a serverless platform built on Kubernetes. Knative Serving provides a simple API for running stateless containers with automatic management of URLs, traffic splitting between revisions, request-based autoscaling (including scale to zero), and automatic TLS provisioning. Knative Serving supports multiple HTTP routers through a plugin architecture, including a [gateway API plugin][knative-net-gateway-api] which is currently in alpha as not all Knative features are supported. + +[knative]:https://knative.dev/ +[knative-net-gateway-api]:https://github.com/knative-sandbox/net-gateway-api + +### Kuadrant + +[Kuadrant][kuadrant] is an open source multi cluster Gateway API controller that integrates with and provides policies to other Gateway API providers. + +Kuadrant supports Gateway API for defining gateways centrally and attaching policies such as DNS, TLS, Auth and Rate Limiting that apply to all gateway instances in a multi cluster environment. +Kuadrant works with Istio as the underlying gateway provider, with plans to work with other gateway providers such as Envoy Gateway. + +For help and support with Kuadrant's implementation please feel free to [create an issue][kuadrant-issue-new] or ask for help in the [#kuadrant channel on Kubernetes slack][kuadrant-slack]. + +[kuadrant]:https://kuadrant.io/ +[kuadrant-issue-new]:https://github.com/Kuadrant/multicluster-gateway-controller/issues/new +[kuadrant-slack]:https://kubernetes.slack.com/archives/C05J0D0V525 diff --git a/site-src/index.md b/site-src/index.md index c624ba38a7..d763face69 100644 --- a/site-src/index.md +++ b/site-src/index.md @@ -1,111 +1,215 @@ ## What is the Gateway API? -Gateway API is an open source project managed by the [SIG-NETWORK][sig-network] -community. It is a collection of resources that model service networking -in Kubernetes. These resources - `GatewayClass`,`Gateway`, `HTTPRoute`, -`TCPRoute`, `Service`, etc - aim to evolve Kubernetes service networking through -expressive, extensible, and role-oriented interfaces that are implemented by -many vendors and have broad industry support. +Gateway API is an open source project managed by the +[SIG-NETWORK][sig-network] community. It is an API (collection of resources) +that model service networking in Kubernetes. These resources - +[`GatewayClass`][GatewayClass], [`Gateway`][Gateway], +[`HTTPRoute`][HTTPRoute], [`TCPRoute`][TCPRoute], etc., as well as the +Kubernetes [`Service`][Service] resource - aim to evolve Kubernetes service +networking through expressive, extensible, and role-oriented interfaces that +are implemented by many vendors and have broad industry support. + +[GatewayClass]: /api-types/gatewayclass +[Gateway]: /api-types/gateway +[HTTPRoute]: /api-types/httproute +[TCPRoute]: /api-types/tcproute +[Service]: https://kubernetes.io/docs/concepts/services-networking/service/ ![Gateway API Model](./images/api-model.png) +The Gateway API was originally designed to manage traffic from clients outside +the cluster to services inside the cluster -- the _ingress_ or +[_north/south_][north/south traffic] case. Over time, interest from service +mesh users prompted the creation of the [GAMMA initiative][gamma] to define +how the Gateway API could also be used for inter-service or [_east/west_ +traffic][east/west traffic] within the same cluster. + +If you're familiar with the older [Ingress API], you can think of the Gateway +API as analogous to a more-expressive next-generation version of that API. + +## Gateway API for Ingress + +When using the Gateway API to manage ingress traffic, the [Gateway] resource +defines a point of access at which traffic can be routed across multiple +contexts -- for example, from outside the cluster to inside the cluster +([north/south traffic]). + +Each Gateway is associated with a [GatewayClass], which describes the actual +kind of [gateway controller] that will handle traffic for the Gateway; +individual routing resources (such as [HTTPRoute]) are then [associated with +the Gateway resources][gateway-attachment]. Separating these different +concerns into distinct resources is a critical part of the role-oriented +nature of the Gateway API, as well as allowing for multiple kinds of gateway +controllers (represented by GatewayClass resources), each with multiple +instances (represented by Gateway resources), in the same cluster. + +[Ingress API]:https://kubernetes.io/docs/concepts/services-networking/ingress/ +[north/south traffic]:/concepts/glossary#northsouth-traffic +[east/west traffic]:/concepts/glossary#eastwest-traffic +[gateway controller]:/concepts/glossary#gateway-controller +[gateway-attachment]:/concepts/api-overview#attaching-routes-to-gateways + +## Gateway API for Service Mesh (the [GAMMA initiative][gamma]) + +!!! danger "Experimental in v0.8.0" + + The [GAMMA initiative][gamma] work for supporting service mesh use cases + is _experimental_ in `v0.8.0`. It is possible that it will change; we do + not recommend it in production at this point. + +Things are a bit different when using the Gateway API to manage a [service +mesh][service-mesh]. Since there will usually only be one mesh active in the +cluster, the [Gateway] and [GatewayClass] resources are not used; instead, +individual route resources (such as [HTTPRoute]) are [associated directly with +Service resources][mesh-attachment], permitting the mesh to manage traffic +from any traffic directed to that Service while preserving the role-oriented +nature of the Gateway API. + +To date, [GAMMA][gamma] has been able to support mesh functionality with +fairly minimal changes to the Gateway API. One particular area that has +rapidly become critical for GAMMA, though, is the definition of the different +[facets of the Service resource][service-facets]. + +In Gateway API v0.8.0, GAMMA support for service mesh is **experimental**. We +encourage working with it and providing feedback, but you **must** be prepared +for change in the GAMMA APIs. + +[gamma]:/concepts/gamma/ +[service-mesh]:/concepts/glossary#service-mesh +[service-facets]:/concepts/service-facets +[mesh-attachment]:/concepts/gamma#gateway-api-for-mesh + ## Getting started -Whether you are a user interested in using the Gateway API or an implementer -interested in conforming to the API, the following resources will help give +Whether you are a user interested in using the Gateway API or an implementer +interested in conforming to the API, the following resources will help give you the necessary background: - [API overview](/concepts/api-overview) - [User guides](/guides) -- [Gateway controller implementations](/implementations) +- [Gateway controller implementations](/implementations#gateways) +- [Service Mesh implementations](/implementations#meshes) - [API reference spec](/references/spec) - [Community links](/contributing/community) and [developer guide](/contributing/devguide) - ## Gateway API concepts -The following design goals drive the concepts of the Gateway API. These +The following design goals drive the concepts of the Gateway API. These demonstrate how Gateway aims to improve upon current standards like Ingress. - -- **Role-oriented** - Gateway is composed of API resources which model -organizational roles that use and configure Kubernetes service networking. +- **Role-oriented** - Gateway is composed of API resources which model +organizational roles that use and configure Kubernetes service networking. - **Portable** - This isn't an improvement but rather something that should stay the same. Just as Ingress is a universal specification with [numerous implementations](https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/), Gateway API is designed to be a portable specification supported by many implementations. -- **Expressive** - Gateway API resources support core functionality for things -like header-based matching, traffic weighting, and other capabilities that +- **Expressive** - Gateway API resources support core functionality for things +like header-based matching, traffic weighting, and other capabilities that were only possible in Ingress through custom annotations. -- **Extensible** - Gateway API allows for custom resources to be linked at +- **Extensible** - Gateway API allows for custom resources to be linked at various layers of the API. This makes granular customization possible at the appropriate places within the API structure. Some other notable capabilities include: -- **GatewayClasses** - GatewayClasses formalize types of load balancing -implementations. These classes make it easy and explicit for users to -understand what kind of capabilities are available via the Kubernetes resource +- **GatewayClasses** - GatewayClasses formalize types of load balancing +implementations. These classes make it easy and explicit for users to +understand what kind of capabilities are available via the Kubernetes resource model. - **Shared Gateways and cross-Namespace support** - They allow the sharing of load balancers and VIPs by permitting independent Route resources to attach to the same Gateway. This allows teams (even across Namespaces) to share infrastructure safely without direct coordination. -- **Typed Routes and typed backends** - The Gateway API supports typed Route -resources and also different types of backends. This allows the API to be +- **Typed Routes and typed backends** - The Gateway API supports typed Route +resources and also different types of backends. This allows the API to be flexible in supporting various protocols (like HTTP and gRPC) and various backend targets (like Kubernetes Services, storage buckets, or -functions). - +functions). +- Experimental **Service mesh support** with the [GAMMA initiative][gamma] - +The Gateway API supports associating routing resources with Service resources, +to configure service meshes as well as ingress controllers. ## Why does a role-oriented API matter? Whether it’s roads, power, data centers, or Kubernetes clusters, infrastructure is built to be shared. However, shared infrastructure raises a common challenge - how to provide flexibility to users of the infrastructure -while maintaining control by owners of the infrastructure? +while maintaining control by owners of the infrastructure? The Gateway API accomplishes this through a role-oriented design for Kubernetes service networking that strikes a balance between distributed flexibility and centralized control. It allows shared network infrastructure (hardware load balancers, cloud networking, cluster-hosted proxies etc) to be used by many different and non-coordinating teams, all bound by the policies -and constraints set by cluster operators. The following example shows how this -works in practice. +and constraints set by cluster operators. + +The roles used for the Gateway API's design are defined by three personas: + +### Personas + +- **Ian** (he/him) is an _infrastructure provider_. His role is the care and + feeding of a set of infrastructure that permits multiple isolated clusters + to serve multiple tenants. He is not beholden to any single tenant; rather, + he worries about all of them collectively. + +- **Chihiro** (they/them) is a _cluster operator_. Their role is to manage a + single cluster, ensuring that it meets the needs of its several users. + Again, Chihiro is beholden to no single user of their cluster; they need to + make sure that the cluster serves all of them as needed. + +- **Ana** (she/her) is an _application developer_. Ana is in a unique position + among the Gateway API roles: her focus is on the business needs her + application is meant to serve, _not_ Kubernetes or the Gateway API. In fact, + Ana is likely to view the Gateway API and Kubernetes as pure friction + getting in her way to get things done. + +(These three are discussed in more detail in [Roles and +Personas](/concepts/roles-and-personas).) + +It should be clear that while Ana, Chihiro, and Ian do not necessarily see +eye-to-eye about everything, they need to work together to keep things running +smoothly. This is the core challenge of the Gateway API in a nutshell. + +### Use Cases + +The [example use cases][use-cases] show this role-oriented model at work. Its +flexibility allows the API to adapt to vastly different organizational models +and implementations while remaining a portable and standard API. + +The use cases presented are deliberately cast in terms of the roles presented +above. Ultimately the Gateway API is meant for use by humans, which means that +it must fit the uses to which each of Ana, Chihiro, and Ian will put it. + +[use-cases]:/concepts/use-cases -A cluster operator creates a [Gateway](/api-types/gateway) resource derived from a -[GatewayClass](/api-types/gatewayclass). This Gateway deploys or configures the -underlying network resources that it represents. Through the -[Route Attachment Process](/concepts/api-overview#attaching-routes-to-gateways) -between the Gateway and Routes, the cluster operator and specific teams must -agree on what can attach to this Gateway and expose their applications through -it. Centralized policies [such as TLS](/guides/tls#downstream-tls) can -be enforced on the Gateway by the cluster operator. Meanwhile, the store and site -teams run [in their own Namespaces](/guides/multiple-ns), but attach their -Routes to the same shared Gateway, allowing them to independently control -their [routing logic](/guides/http-routing). This separation of concerns -allows the store team to manage their own -[traffic splitting rollout](/guides/traffic-splitting) while -leaving centralized policies and control to the cluster operators. +## What's the difference between Gateway API and an API Gateway? -![Gateway API Roles](./images/gateway-roles.png) +An API Gateway is a general concept that describes anything that exposes +capabilities of a backend service, while providing extra capabilities for +traffic routing and manipulation, such as load balancing, request and response +transformation, and sometimes more advanced features like authentication and +authorization, rate limiting, and circuit breaking. -This flexibility allows the API to adapt to vastly different -organizational models and implementations while remaining a portable and -standard API. +The Gateway API is an interface, or set of resources, that model service +networking in Kubernetes. One of the main resources is a `Gateway`, which +declares the Gateway type (or class) to instantiate and its configuration. As +a Gateway Provider, you can implement the Gateway API to model Kubernetes +service networking in an expressive, extensible, and role-oriented way. +Most Gateway API implementations are API Gateways to some extent, but not all +API Gateways are Gateway API implementations. -## Who is working on Gateway? +## Who is working on Gateway API? The Gateway API is a [SIG-Network](https://github.com/kubernetes/community/tree/master/sig-network) project being built to improve and standardize service networking in Kubernetes. Current and in-progress implementations include Contour, -Emissary-Ingress (Ambassador API Gateway), Google Kubernetes Engine (GKE), Istio, -Kong, and Traefik. Check out the [implementations -reference](implementations.md) to see the latest projects & -products that support Gateway. If you are interested in contributing to or -building an implementation using the Gateway API then don’t hesitate to [get +Emissary-ingress (Ambassador API Gateway), Google Kubernetes Engine (GKE), +Istio, Kong, Linkerd, and Traefik. Check out the [implementations +reference](implementations.md) to see the latest projects & products that +support Gateway. If you are interested in contributing to or building an +implementation using the Gateway API then don’t hesitate to [get involved!](/contributing/community) [sig-network]: https://github.com/kubernetes/community/tree/master/sig-network diff --git a/site-src/references/policy-attachment.md b/site-src/references/policy-attachment.md index 740a9d717e..e78b2cc81b 100644 --- a/site-src/references/policy-attachment.md +++ b/site-src/references/policy-attachment.md @@ -1,305 +1,17 @@ -# Policy Attachment +# Metaresources and Policy Attachment -While features like timeouts, retries, and custom health checks are present in -most implementations, their details vary since there are no standards (RFCs) -around them. This makes these features less portable. So instead of pulling -these into the API, we offer a middle ground: a standard way to plug these -features in the API and offer a uniform UX across implementations. This standard -approach for policy attachment allows implementations to create their own custom -policy resources that can essentially extend Gateway API. +The Gateway API defines a Kubernetes object that _augments_ the behavior of an object +in a standard way as a _Metaresource_. ReferenceGrant +is an example of this general type of metaresource, but it is far from the only +one. -Policies attached to Gateway API resources and implementations must use the -following approach to ensure consistency across implementations of the API. -There are three primary components of this pattern: +Gateway API also defines a pattern called _Policy Attachment_, which augments +the behavior of an object to add additional settings that can't be described +within the spec of that object. -* A standardized means of attaching policy to resources. -* Support for configuring both default and override values within policy - resources. -* A hierarchy to illustrate how default and override values should interact. +A "Policy Attachment" is a specific type of _metaresource_ that can affect specific +settings across either one object (this is "Direct Policy Attachment"), or objects +in a hierarchy (this is "Inherited Policy Attachment"). -This kind of standardization not only enables consistent patterns, it allows -future tooling such as kubectl plugins to be able to visualize all policies that -have been applied to a given resource. - -## Policy Attachment for Ingress -Attaching policy to Gateway resources for ingress use cases is relatively -straightforward. A policy can reference the resource it wants to apply to. -Access is granted with RBAC - for example, anyone that has access to create a -RetryPolicy in a given namespace can attach it to any resource within that -namespace. - -![Simple Ingress Example](/images/policy/ingress-simple.png) - -To build on that example, it’s possible to attach policies to more resources. -Each policy applies to the referenced resource and everything below it in terms -of hierarchy. Although this example is likely more complex than many real world -use cases, it helps demonstrate how policy attachment can work across -namespaces. - -![Complex Ingress Example](/images/policy/ingress-complex.png) - -## Policy Attachment for Mesh -Although there is a great deal of overlap between ingress and mesh use cases, -mesh enables more complex policy attachment scenarios. For example, users may -want to apply policy to requests from a specific namespace to a backend in -another namespace. - -![Simple Mesh Example](/images/policy/mesh-simple.png) - -Policy attachment can be quite simple with mesh. Policy can be applied to any -resource in any namespace but it can only apply to requests from the same -namespace if the target is in a different namespace. - -At the other extreme, policy can be used to apply to requests from a specific -workload to a backend in another namespace. A route can be used to intercept -these requests and split them between different backends (foo-a and foo-b in -this case). - -![Complex Mesh Example](/images/policy/mesh-complex.png) - -## Target Reference API - -Each Policy resource MUST include a single `targetRef` field. It MUST not -target more than one resource at a time, but it can be used to target larger -resources such as Gateways or Namespaces that may apply to multiple child -resources. - -The `targetRef` field MUST be an exact replica of the `PolicyTargetReference` -struct included in the Gateway API. Where possible, it is recommended to use -that struct directly instead of duplicating the type. - -### Policy Boilerplate -The following structure MUST be used as for any Policy resource using this API -pattern. Within the spec, policy resources may omit `Override` or `Default` -fields, but at least one of them MUST be present. - -```go -// ACMEServicePolicy provides a way to apply Service policy configuration with -// the ACME implementation of the Gateway API. -type ACMEServicePolicy struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - // Spec defines the desired state of ACMEServicePolicy. - Spec ACMEServicePolicySpec `json:"spec"` - - // Status defines the current state of ACMEServicePolicy. - Status ACMEServicePolicyStatus `json:"status,omitempty"` -} - -// ACMEServicePolicySpec defines the desired state of ACMEServicePolicy. -type ACMEServicePolicySpec struct { - // TargetRef identifies an API object to apply policy to. - TargetRef gatewayv1a2.PolicyTargetReference `json:"targetRef"` - - // Override defines policy configuration that should override policy - // configuration attached below the targeted resource in the hierarchy. - // +optional - Override *ACMEPolicyConfig `json:"override,omitempty"` - - // Default defines default policy configuration for the targeted resource. - // +optional - Default *ACMEPolicyConfig `json:"default,omitempty"` -} - -// ACMEPolicyConfig contains ACME policy configuration. -type ACMEPolicyConfig struct { - // Add configurable policy here -} - -// ACMEServicePolicyStatus defines the observed state of ACMEServicePolicy. -type ACMEServicePolicyStatus struct { - // Conditions describe the current conditions of the ACMEServicePolicy. - // - // +optional - // +listType=map - // +listMapKey=type - // +kubebuilder:validation:MaxItems=8 - Conditions []metav1.Condition `json:"conditions,omitempty"` -} -``` - -### Hierarchy -Each policy MAY include default or override values. Overrides enable admins to -enforce policy from the top down. Defaults enable app owners to provide default -values from the bottom up for each individual application. - -![Policy Hierarchy](/images/policy/hierarchy.png) - -To illustrate this, consider 3 resources with the following hierarchy: A -(highest) > B > C. When attaching the concept of defaults and overrides to that, -the hierarchy would be expanded to this: - -A override > B override > C override > C default > B default > A default. - -Note that the hierarchy is reversed for defaults. The rationale here is that -overrides usually need to be enforced top down while defaults should apply to -the lowest resource first. For example, if an admin needs to attach required -policy, they can attach it as an override to a Gateway. That would have -precedence over Routes and Services below it. On the other hand, an app owner -may want to set a default timeout for their Service. That would have precedence -over defaults attached at higher levels such as Route or Gateway. - -If using defaults and overrides, each policy resource MUST include 2 structs -within the spec. One with override values and the other with default values. - -In the following example, the policy attached to the Gateway requires cdn to -be enabled and provides some default configuration for that. The policy attached -to the Route changes the value for one of those fields (`includeQueryString`). - -```yaml -kind: AcmeServicePolicy # Example of implementation specific policy name -spec: - override: - cdn: - enabled: true - default: - cdn: - cachePolicy: - includeHost: true - includeProtocol: true - includeQueryString: true - targetRef: - kind: Gateway - name: example ---- -kind: AcmeServicePolicy -spec: - default: - cdn: - cachePolicy: - includeQueryString: false - targetRef: - kind: HTTPRoute - name: example -``` - -In this final example, we can see how the override attached to the Gateway has -precedence over the default `drainTimeout` value attached to the Route. At the -same time, we can see that the default `connectionTimeout` attached to the Route -has precedence over the default attached to the Gateway. - -![Hierarchical Policy Example](images/policy-hierarchy.png) - -#### Attaching Policy to GatewayClass -GatewayClass may be the trickiest resource to attach policy to. Policy -attachment relies on the policy being defined within the same scope as the -target. This ensures that only users with write access to a policy resource in a -given scope will be able to modify policy at that level. Since GatewayClass is a -cluster scoped resource, this means that any policy attached to it must also be -cluster scoped. - -GatewayClass parameters provide an alternative to policy attachment that may be -easier for some implementations to support. These parameters can similarly be -used to set defaults and requirements for an entire GatewayClass. - -### Targeting External Services -In some cases (likely limited to mesh or egress) users may want to apply -policies to requests to external services. To accomplish this, implementations -can choose to support a reference to a virtual resource type: - -```yaml -apiVersion: networking.example.net/v1alpha1 -kind: RetryPolicy -metadata: - name: foo -spec: - default: - maxRetries: 5 - targetRef: - group: networking.example.net - kind: ExternalService - name: foo.com -``` - -### Conflict Resolution -It is possible for multiple policies to target the same resource. When this -happens, merging is the preferred outcome. If multiple policy resources target -the same resource _and_ have an identical field specified with different values, -precedence MUST be determined in order of the following criteria, continuing on -ties: - -* The oldest Policy based on creation timestamp. For example, a Policy with a - creation timestamp of "2021-07-15 01:02:03" is given precedence over a Policy - with a creation timestamp of "2021-07-15 01:02:04". -* The Policy appearing first in alphabetical order by "{namespace}/{name}". For - example, foo/bar is given precedence over foo/baz. - -### Kubectl Plugin -To help improve UX and standardization, a kubectl plugin will be developed that -will be capable of describing the computed sum of policy that applies to a given -resource, including policies applied to parent resources. - -Each Policy CRD that wants to be supported by this plugin will need to follow -the API structure defined above and add a -`gateway.networking.k8s.io/policy-attachment: ""` label to the CRD. - -### Status -In the future, we may consider adding a new `Policies` field to status on -Gateways and Routes. This would be a list of `PolicyTargetReference` structs -with the fields instead used to refer to the Policy resource that has been -applied. - -Unfortunately, this may create more confusion than it is worth, here are some of -the key concerns: - -* When multiple controllers are implementing the same Route and recognize a - policy, it would be difficult to determine which controller should be - responsible for adding that policy reference to status. -* For this to be somewhat scalable, we'd need to limit the status entries to - policies that had been directly applied to the resource. This could get - confusing as it would not provide any insight into policies attached above or - below. -* Since we only control some of the resources a policy might be attached to, - adding policies to status would only be possible on Gateway API resources, not - Services or other kinds of backends. - -Although these concerns are not unsolvable, they lead to the conclusion that -a Kubectl plugin should be our primary approach to providing visibility here, -with a possibility of adding policies to status at a later point. - -### Interaction with Custom Route Filters -Both Policy attachment and custom Route filters provide ways to extend Gateway -API. Although similar in nature, they have slightly different purposes. - -Custom Route filters provide a way to configure request/response modifiers or -middleware embedded inside Route rules or backend references. - -Policy attachment is more broad in scope. In contrast with filters, policies can -be attached to a wide variety of Gateway API resources, and include a concept of -hierarchical defaulting and overrides. Although Policy attachment can be used to -target an entire Route or Backend, it cannot currently be used to target -specific Route rules or backend references. If there are sufficient use cases -for this, policy attachment may be expanded in the future to support this fine -grained targeting. - -The following guidance should be considered when introducing a custom field into -any Gateway controller implementation: - -#### 1. Use core or extended fields if available -For any given field that a Gateway controller implementation needs, the -possibility of using core or extended fields should always be considered -before using custom policy resources. This is encouraged to promote -standardization and, over time, to absorb capabilities into the API as first -class fields, which offer a more streamlined UX than custom policy -attachment. - -#### 2. Custom filters and policies should not overlap -Although it's possible that arbitrary fields could be supported by custom -policy, custom route filters, and core/extended fields concurrently, it is -strongly recommended that implementations not use multiple mechanisms for -representing the same fields. A given field should only be supported through a -single extension method. An example of potential conflict is policy precedence -and structured hierarchy, which only applies to custom policies. Allowing a -field to exist in custom policies and also other areas of the API, which are not -part of the structured hierarchy, breaks the precedence model. Note that this -guidance may change in the future as we gain a better understanding of how -extension mechanisms of the Gateway API can interoperate. - -### Conformance Level -This policy attachment pattern is associated with an "EXTENDED" conformance -level. The implementations that support this policy attachment model will have -the same behavior and semantics, although they may not be able to support -attachment of all types of policy at all potential attachment points. When that -is the case, implementations MUST clearly document which resources a policy may -be attached to. +This pattern is EXPERIMENTAL, and is described in [GEP-713](https://gateway-api.sigs.k8s.io/geps/gep-713/). +Please see that document for technical details.