Reconcile ClusterIP services and prep v0.10.0

CHANGELOG.md

@@ -2,14 +2,18 @@
 
 All notable changes to this project are documented in this file.
 
-## Unreleased 
+## 0.10.0 (2019-03-26) 
 
-Adds support for AWS App Mesh EKS
+Adds support for AWS App Mesh
 
 #### Features
 
 - AWS App Mesh integration [#107](https://github.com/weaveworks/flagger/pull/107)
 
+#### Improvements 
+
+- Reconcile Kubernetes ClusterIP services [#122](https://github.com/weaveworks/flagger/pull/122) 
+
 #### Fixes
 
 - Copy pod labels from canary to primary [#105](https://github.com/weaveworks/flagger/pull/105)

README.md

@@ -23,7 +23,7 @@ Flagger documentation can be found at [docs.flagger.app](https://docs.flagger.ap
 
 * Install
     * [Flagger install on Kubernetes](https://docs.flagger.app/install/flagger-install-on-kubernetes)
-    * [Flagger install on GKE](https://docs.flagger.app/install/flagger-install-on-google-cloud)
+    * [Flagger install on GKE Istio](https://docs.flagger.app/install/flagger-install-on-google-cloud)
     * [Flagger install on EKS App Mesh](https://docs.flagger.app/install/flagger-install-on-eks-appmesh)
 * How it works
     * [Canary custom resource](https://docs.flagger.app/how-it-works#canary-custom-resource)
@@ -35,36 +35,18 @@ Flagger documentation can be found at [docs.flagger.app](https://docs.flagger.ap
     * [Webhooks](https://docs.flagger.app/how-it-works#webhooks)
     * [Load testing](https://docs.flagger.app/how-it-works#load-testing)
 * Usage
-    * [Canary promotions and rollbacks](https://docs.flagger.app/usage/progressive-delivery)
-    * [A/B testing](https://docs.flagger.app/usage/ab-testing)
+    * [Istio canary deployments](https://docs.flagger.app/usage/progressive-delivery)
+    * [Istio A/B testing](https://docs.flagger.app/usage/ab-testing)
+    * [App Mesh canary deployments](https://docs.flagger.app/usage/appmesh-progressive-delivery)
     * [Monitoring](https://docs.flagger.app/usage/monitoring)
     * [Alerting](https://docs.flagger.app/usage/alerting)
 * Tutorials
     * [Canary deployments with Helm charts and Weave Flux](https://docs.flagger.app/tutorials/canary-helm-gitops)
 
-### Install 
-
-Before installing Flagger make sure you have Istio setup up with Prometheus enabled. 
-If you are new to Istio you can follow my [Istio service mesh walk-through](https://github.com/stefanprodan/istio-gke).
-
-Deploy Flagger in the `istio-system` namespace using Helm:
-
-```bash
-# add the Helm repository
-helm repo add flagger https://flagger.app
-
-# install or upgrade
-helm upgrade -i flagger flagger/flagger \
---namespace=istio-system \
---set metricsServer=http://prometheus.istio-system:9090 
-```
-
-Flagger is compatible with Kubernetes >1.11.0 and Istio >1.0.0.
-
 ### Canary CRD
 
 Flagger takes a Kubernetes deployment and optionally a horizontal pod autoscaler (HPA),
-then creates a series of objects (Kubernetes deployments, ClusterIP services and Istio virtual services).
+then creates a series of objects (Kubernetes deployments, ClusterIP services and Istio or App Mesh virtual services).
 These objects expose the application on the mesh and drive the canary analysis and promotion.
 
 Flagger keeps track of ConfigMaps and Secrets referenced by a Kubernetes Deployment and triggers a canary analysis if any of those objects change. 
@@ -169,7 +151,7 @@ For more details on how the canary analysis and promotion works please [read the
 
 ### Roadmap
 
-* Integrate with other service mesh technologies like Linkerd v2 or Consul Mesh
+* Integrate with other service mesh technologies like Linkerd v2, Super Gloo or Consul Mesh
 * Add support for comparing the canary metrics to the primary ones and do the validation based on the derivation between the two
 
 ### Contributing

artifacts/flagger/crd.yaml

@@ -73,6 +73,10 @@ spec:
               properties:
                 port:
                   type: number
+                portName:
+                  type: string
+                meshName:
+                  type: string
                 timeout:
                   type: string
             skipAnalysis:

artifacts/flagger/deployment.yaml

@@ -22,7 +22,7 @@ spec:
       serviceAccountName: flagger
       containers:
       - name: flagger
-        image: quay.io/stefanprodan/flagger:0.9.0
+        image: quay.io/weaveworks/flagger:0.10.0
         imagePullPolicy: IfNotPresent
         ports:
         - name: http

charts/flagger/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 name: flagger
-version: 0.9.0
-appVersion: 0.9.0
+version: 0.10.0
+appVersion: 0.10.0
 kubeVersion: ">=1.11.0-0"
 engine: gotpl
 description: Flagger is a Kubernetes operator that automates the promotion of canary deployments using Istio routing for traffic shifting and Prometheus metrics for canary analysis.

charts/flagger/templates/crd.yaml

@@ -76,6 +76,10 @@ spec:
               properties:
                 port:
                   type: number
+                portName:
+                  type: string
+                meshName:
+                  type: string
                 timeout:
                   type: string
             skipAnalysis:

charts/flagger/values.yaml

@@ -1,8 +1,8 @@
 # Default values for flagger.
 
 image:
-  repository: quay.io/stefanprodan/flagger
-  tag: 0.9.0
+  repository: quay.io/weaveworks/flagger
+  tag: 0.10.0
   pullPolicy: IfNotPresent
 
 metricsServer: "http://prometheus:9090"

charts/grafana/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: grafana
-version: 1.0.0
+version: 1.1.0
 appVersion: 5.4.3
 description: Grafana dashboards for monitoring Flagger canary deployments
 icon: https://raw.githubusercontent.com/weaveworks/flagger/master/docs/logo/flagger-icon.png

charts/grafana/dashboards/appmesh.json

@@ -1243,6 +1243,6 @@
   },
   "timezone": "",
   "title": "AppMesh Canary",
-  "uid": "wI-fAb3mz",
+  "uid": "flagger-appmesh",
   "version": 4
 }

charts/grafana/dashboards/istio.json

@@ -1680,6 +1680,6 @@
   },
   "timezone": "",
   "title": "Istio Canary",
-  "uid": "RdykD7tiz",
+  "uid": "flagger-istio",
   "version": 3
 }

charts/grafana/templates/NOTES.txt

@@ -1,15 +1,3 @@
-1. Get the application URL by running these commands:
-{{- if contains "NodePort" .Values.service.type }}
-  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ template "grafana.fullname" . }})
-  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
-  echo http://$NODE_IP:$NODE_PORT
-{{- else if contains "LoadBalancer" .Values.service.type }}
-     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
-           You can watch the status of by running 'kubectl get svc -w {{ template "grafana.fullname" . }}'
-  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ template "grafana.fullname" . }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
-  echo http://$SERVICE_IP:{{ .Values.service.port }}
-{{- else if contains "ClusterIP" .Values.service.type }}
-  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "grafana.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
-  echo "Visit http://127.0.0.1:8080 to use your application"
-  kubectl port-forward $POD_NAME 8080:80
-{{- end }}
+1. Get the application URL with:
+
+kubectl -n {{ .Release.Namespace }} port-forward svc/{{ .Release.Name }} 3000:80

charts/loadtester/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v1
 name: loadtester
-version: 0.3.0
+version: 0.2.0
 appVersion: 0.2.0
 kubeVersion: ">=1.11.0-0"
 engine: gotpl

charts/loadtester/templates/service.yaml

@@ -13,6 +13,6 @@ spec:
     - port: {{ .Values.service.port }}
       targetPort: http
       protocol: TCP
-      name: 8080
+      name: http
   selector:
     app: {{ include "loadtester.name" . }}

docs/gitbook/install/flagger-install-on-eks-appmesh.md

@@ -16,18 +16,19 @@ The App Mesh integration with EKS is made out of the following components:
 
 Prerequisites:
 
+* jq
 * homebrew
 * openssl
 * kubectl
 * AWS CLI (default region us-west-2)
 
 ### Create a Kubernetes cluster
 
-In order to create an EKS cluster you can use [EKSctl](https://eksctl.io).
-EKSctl is an open source command-line utility made by Weaveworks in collaboration with Amazon, 
-it's written in Go and is based on EKS CloudFormation templates.
+In order to create an EKS cluster you can use [eksctl](https://eksctl.io).
+Eksctl is an open source command-line utility made by Weaveworks in collaboration with Amazon, 
+it’s a Kubernetes-native tool written in Go.
 
-On MacOS you can install EKSctl with Homebrew:
+On MacOS you can install eksctl with Homebrew:
 
 ```bash
 brew tap weaveworks/tap
@@ -183,3 +184,5 @@ You can access Grafana using port forwarding:
 kubectl -n appmesh-system port-forward svc/flagger-grafana 3000:80
 ```
 
+Now that you have Flagger running you can try the
+[App Mesh canary deployments tutorial](https://docs.flagger.app/usage/appmesh-progressive-delivery).

docs/gitbook/usage/ab-testing.md

@@ -8,6 +8,8 @@ This is particularly useful for frontend applications that require session affin
 
 ![Flagger A/B Testing Stages](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/diagrams/flagger-abtest-steps.png)
 
+### Bootstrap
+
 Create a test namespace with Istio sidecar injection enabled:
 
 ```bash
@@ -121,6 +123,8 @@ service/abtest-primary
 virtualservice.networking.istio.io/abtest
 ```
 
+### Automated canary promotion
+
 Trigger a canary deployment by updating the container image:
 
 ```bash
@@ -170,6 +174,8 @@ prod        frontend  Succeeded     0        2019-03-15T16:15:07Z
 prod        backend   Failed        0        2019-03-14T17:05:07Z
 ```
 
+### Automated rollback
+
 During the canary analysis you can generate HTTP 500 errors and high latency to test Flagger's rollback.
 
 Generate HTTP 500 errors:

docs/gitbook/usage/alerting.md

@@ -15,12 +15,12 @@ helm upgrade -i flagger flagger/flagger \
 Once configured with a Slack incoming **webhook**, Flagger will post messages when a canary deployment 
 has been initialised, when a new revision has been detected and if the canary analysis failed or succeeded.
 
-![flagger-slack](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-notifications.png)
+![Slack Notifications](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-notifications.png)
 
 A canary deployment will be rolled back if the progress deadline exceeded or if the analysis reached the 
 maximum number of failed checks:
 
-![flagger-slack-errors](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-failed.png)
+![Slack Notifications](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-failed.png)
 
 ### Prometheus Alert Manager
 

docs/gitbook/usage/appmesh-progressive-delivery.md

@@ -38,7 +38,7 @@ Deploy the load testing service to generate traffic during the canary analysis:
 helm upgrade -i flagger-loadtester flagger/loadtester \
 --namespace=test \
 --set meshName=global.appmesh-system \
---set backends[0]=podinfo.test
+--set "backends[0]=podinfo.test"
 ```
 
 Create a canary custom resource:
@@ -211,7 +211,8 @@ Events:
 
 **Note** that if you apply new changes to the deployment during the canary analysis, Flagger will restart the analysis.
 
-During the analysis the canary’s progress can be monitored with Grafana:
+During the analysis the canary’s progress can be monitored with Grafana. The App Mesh dashboard URL is 
+http://localhost:3000/d/flagger-appmesh/appmesh-canary?refresh=10s&orgId=1&var-namespace=test&var-primary=podinfo-primary&var-canary=podinfo
 
 ![App Mesh Canary Dashboard](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/flagger-grafana-appmesh.png)
 
@@ -228,7 +229,7 @@ prod        backend   Failed        0        2019-03-14T17:05:07Z
 
 If you’ve enabled the Slack notifications, you should receive the following messages:
 
-![flagger-slack](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-notifications.png)
+![Flagger Slack Notifications](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-notifications.png)
 
 ### Automated rollback
 
@@ -282,4 +283,4 @@ Events:
 If you’ve enabled the Slack notifications, you’ll receive a message if the progress deadline is exceeded, 
 or if the analysis reached the maximum number of failed checks:
 
-![flagger-slack-errors](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-failed.png)
+![Flagger Slack Notifications](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/slack-canary-failed.png)

docs/gitbook/usage/monitoring.md

@@ -6,15 +6,13 @@ Flagger comes with a Grafana dashboard made for canary analysis. Install Grafana
 
 ```bash
 helm upgrade -i flagger-grafana flagger/grafana \
---namespace=istio-system \
---set url=http://prometheus:9090 \
---set user=admin \
---set password=admin
+--namespace=istio-system \ # or appmesh-system
+--set url=http://prometheus:9090
 ```
 
 The dashboard shows the RED and USE metrics for the primary and canary workloads:
 
-![canary dashboard](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/grafana-canary-analysis.png)
+![Canary Dashboard](https://raw.githubusercontent.com/weaveworks/flagger/master/docs/screens/grafana-canary-analysis.png)
 
 ### Logging
 

docs/gitbook/usage/progressive-delivery.md

@@ -2,6 +2,8 @@
 
 This guide shows you how to use Istio and Flagger to automate canary deployments.
 
+### Bootstrap
+
 Create a test namespace with Istio sidecar injection enabled:
 
 ```bash
@@ -109,6 +111,8 @@ service/podinfo-primary
 virtualservice.networking.istio.io/podinfo
 ```
 
+### Automated canary promotion
+
 Trigger a canary deployment by updating the container image:
 
 ```bash
@@ -159,6 +163,8 @@ prod        frontend  Succeeded     0        2019-01-15T16:15:07Z
 prod        backend   Failed        0        2019-01-14T17:05:07Z
 ```
 
+### Automated rollback
+
 During the canary analysis you can generate HTTP 500 errors and high latency to test if Flagger pauses the rollout.
 
 Create a tester pod and exec into it:

pkg/controller/deployer.go

@@ -323,7 +323,7 @@ func (c *CanaryDeployer) Scale(cd *flaggerv1.Canary, replicas int32) error {
 	return nil
 }
 
-// Sync creates the primary deployment and hpa
+// Reconcile creates the primary deployment and hpa
 // and scales to zero the canary deployment
 func (c *CanaryDeployer) Sync(cd *flaggerv1.Canary) error {
 	primaryName := fmt.Sprintf("%s-primary", cd.Spec.TargetRef.Name)

pkg/controller/scheduler.go

@@ -99,14 +99,14 @@ func (c *Controller) advanceCanary(name string, namespace string, skipLivenessCh
 	routerFactory := router.NewFactory(c.kubeClient, c.flaggerClient, c.logger, c.istioClient)
 	meshRouter := routerFactory.MeshRouter(c.meshProvider)
 
-	// create ClusterIP services and virtual service if needed
-	if err := routerFactory.KubernetesRouter().Sync(cd); err != nil {
+	// create or update ClusterIP services
+	if err := routerFactory.KubernetesRouter().Reconcile(cd); err != nil {
 		c.recordEventWarningf(cd, "%v", err)
 		return
 	}
 
 	// create or update virtual service
-	if err := meshRouter.Sync(cd); err != nil {
+	if err := meshRouter.Reconcile(cd); err != nil {
 		c.recordEventWarningf(cd, "%v", err)
 		return
 	}