Merge pull request #77 from rushtehrani/master

docs: Update cloud provider documentation

Author: rushtehrani

docs/deployment/configuration/files.md

@@ -231,7 +231,7 @@ nodePool:
 ```
 
 ### artifactRepository
-This section allows you to setup the default object storage for your Workflow and Workspace artifacts, which includes Workflow log storage. Onepanel currently supports any S3 compatible artifact repository such as AWS, GCS and Minio. Support for additional object storages is coming soon.
+This section allows you to set up the default object storage for your Workflow and Workspace artifacts, which includes Workflow log storage. Onepanel currently supports any S3 compatible artifact repository such as AWS, GCS and Minio.
 
 Here's an example AWS S3 configuration:
 
@@ -245,9 +245,33 @@ artifactRepository:
     secretKey: 5bEYu26084qjSFyclM/f2pz4gviSfoOg+mFwBH39
 ```
 
-Here's an example Google Cloud GCS configuration:
+GCS and Minio configurations would also be similar except that the endpoint will be different. For GCS, you would need to [create an HMAC key](https://cloud.google.com/storage/docs/authentication/managing-hmackeys#create) and use the **Access key** and **Secret** accordingly.
+
+Here's an example GCS configuration:
+
+```yaml
+artifactRepository:
+  s3:
+    accessKey: GOOG1EQPAXHU77377T6TGRYGD7NDV6AA3TFYIIKXP2RZLHI3DZB76FIFGDNLQ
+    bucket: pipelines.example.com
+    endpoint: storage.googleapis.com
+    region: us-west-2
+    secretKey: S3hdxSL6HlPGTAAFZYxG/iaKhtlDHVCbyiIBRPxq
+```
+
+And example Minio configuration:
 
 ```yaml
+artifactRepository:
+  s3:
+    accessKey: AKIAJSIE27KKMHXI3BJQ
+    bucket: my-bucket
+    endpoint: my-minio-endpoint.default:9000
+    region: us-west-2
+    secretKey: 5bEYu26084qjSFyclM/f2pz4gviSfoOg+mFwBH39
+```
+
+<!-- ```yaml
 artifactRepository:
  gcs:
     bucket: mygreatbucket
@@ -275,7 +299,7 @@ You can get the serviceAccount JSON via gcloud.
 gcloud iam service-accounts keys create key.json \
    --iam-account ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com
 ```
-:::
+::: -->
 
 :::important
 Onepanel Workflows will automatically upload or download artifacts from `artifacts/{{workflow.namespace}}/{{workflow.name}}/{{pod.name}}`. See [Workflow artifacts](/docs/reference/workflows/templates#artifacts) for more information.

docs/deployment/public/aks.md

@@ -27,55 +27,33 @@ az aks create --resource-group <resource-group> --name <cluster-name> --location
     --node-vm-size Standard_D4s_v3 \
     --node-osdisk-size 100 \
     --min-count 2 \
-    --max-count 2 \
+    --max-count 5 \
     --enable-cluster-autoscaler \
     --network-plugin azure \
     --network-policy azure \
     --enable-addons monitoring \
     --generate-ssh-keys
 ```
 
-:::note
-The `--enable-addons monitoring` flag in the command above enables Azure Monitor for log aggregation which can incur additional charges. You can optionally remove this flag and add `--enable-efk-logging` to `opctl` command below.
-:::
-
-:::note
-You can specify the version of the cluster.
-Get a list of versions by running:
-```shell script
-az aks get-versions --location eastus --output table
-```
-Example output
-```text
-KubernetesVersion    Upgrades
--------------------  -------------------------------------------------
-1.18.2(preview)      None available
-1.18.1(preview)      1.18.2(preview)
-1.17.5(preview)      1.18.1(preview), 1.18.2(preview)
-1.17.4(preview)      1.17.5(preview), 1.18.1(preview), 1.18.2(preview)
-1.16.9               1.17.4(preview), 1.17.5(preview)
-1.16.8               1.16.9, 1.17.4(preview), 1.17.5(preview)
-1.15.11              1.16.8, 1.16.9
-1.15.10              1.15.11, 1.16.8, 1.16.9
-1.14.8               1.15.10, 1.15.11
-1.14.7               1.14.8, 1.15.10, 1.15.11
-```
+You can then get access credentials by running:
 
-Add the flag to the above command:
-```shell script
-az aks create --resource-group <resource-group> --name <cluster-name> \
-    --node-count 2 \
-    --kubernetes-version 1.16.9 \
-    ...
+```bash
+az aks get-credentials --resource-group <resource-group> --name <cluster-name> --admin
 ```
 
-:::
-
-You can then get access credentials by running:
+Optionally, you can add additional auto-scaling node pools to the cluster as follows:
 
+```bash
+az aks nodepool add --resource-group <resource-group> --cluster-name <cluster-name> \
+  --name <nodepool-name> \
+  --node-vm-size <node-vm-size> \
+  --enable-cluster-autoscaler \
+  --node-count 1 \
+  --min-count 0 \
+  --max-count <max-count>
 ```
-az aks get-credentials --resource-group <resource-group> --name <cluster-name> --admin
-```
+
+In step <strong>1.3</strong> below, you can configure Onepanel to automatically scale these nodes as needed.
 
 ## Install Onepanel
 1. Download the latest `opctl` for your operating system from [our release page](https://github.com/onepanelio/core/releases/latest).
@@ -137,6 +115,7 @@ az aks get-credentials --resource-group <resource-group> --name <cluster-name> -
   ```bash
   opctl init --provider aks \
       --artifact-repository-provider s3 \
+      --gpu-device-plugins nvidia \
       --enable-https \
       --enable-cert-manager \
       --dns-provider <dns-provider>
@@ -147,7 +126,7 @@ az aks get-credentials --resource-group <resource-group> --name <cluster-name> -
   :::
 
   :::note
-  If you have GPU nodes, you need to set the `--gpu-device-plugins` flag. Valid values are `nvidia` and `amd` or a comma separated combination of both `nvidia,amd`.
+  Valid options for `--gpu-device-plugins` are `nvidia` and `amd` or a comma separated combination of both `nvidia,amd`.
   :::
 
 3. Populate `params.yaml` by following the instructions in the template, and referring to [configuration file sections](/docs/deployment/configuration/files#sections) for more detailed information.

docs/deployment/public/eks.md

@@ -27,7 +27,7 @@ eksctl create cluster --name=<cluster-name> --region <region> \
     --node-type m5.xlarge \
     --node-volume-size 100 \
     --nodes-min 2 \
-    --nodes-max 2 \
+    --nodes-max 5 \
     --asg-access \
     --managed \
     --ssh-access
@@ -111,6 +111,7 @@ If you are not the person that created the cluster, you will need to be [added t
   ```bash
   opctl init --provider eks \
       --artifact-repository-provider s3 \
+      --gpu-device-plugins nvidia \
       --enable-https \
       --enable-cert-manager \
       --dns-provider <dns-provider>
@@ -121,7 +122,7 @@ If you are not the person that created the cluster, you will need to be [added t
   :::
 
   :::note
-  If you have GPU nodes, you need to set the `--gpu-device-plugins` flag. Valid values are `nvidia` and `amd` or a comma separated combination of both `nvidia,amd`.
+  Valid options for `--gpu-device-plugins` are `nvidia` and `amd` or a comma separated combination of both `nvidia,amd`.
   :::
 
 3. Populate `params.yaml` by following the instructions in the template, and referring to [configuration file sections](/docs/deployment/configuration/files#sections) for more detailed information.

docs/deployment/public/gke.md

@@ -22,12 +22,12 @@ We recommend launching a cluster with 2 `n1-standard-4` nodes to start, with aut
 Here is sample `gcloud` command to create a bare minimum cluster:
 
 ```bash
-gcloud container --project <project-name> clusters create <cluster-name> --zone <zone> --node-locations <single-location> \
+gcloud container --project <project-name> clusters create <cluster-name> --zone <zone> --node-locations <zone> \
     --num-nodes 2 \
     --machine-type n1-standard-4 \
     --disk-size 100 \
-    --min-nodes 0 \
-    --max-nodes 2 \
+    --min-nodes 2 \
+    --max-nodes 5 \
     --enable-autoscaling \
     --enable-network-policy \
     --enable-stackdriver-kubernetes \
@@ -48,6 +48,21 @@ The command above will automatically retrieve your cluster's access credentials
 gcloud container clusters get-credentials <cluster-name> --zone <zone>
 ```
 
+Optionally, you can add additional auto-scaling node pools to the cluster as follows:
+
+```bash
+gcloud container node-pools create <node-pool-name> --cluster <cluster-name> --zone <zone> \
+  --machine-type <machine-type> \
+  --disk-size 100 \
+  --num-nodes 0 \
+  --min-nodes 0 \
+  --max-nodes 5 \
+  --enable-autoscaling \
+  --accelerator 'type=<type>,count=<count>'  # optional, example: 'type=nvidia-tesla-v100,count=1'
+```
+
+In step <strong>1.3</strong> below, you can configure Onepanel to automatically scale these nodes as needed.
+
 ## Install Onepanel
 1. Download the latest `opctl` for your operating system from [our release page](https://github.com/onepanelio/core/releases/latest).
 

docs/getting-started/quickstart.md

@@ -11,7 +11,7 @@ It's easy to get started with Onepanel. First, you install the CLI (`opctl`) and
 Before getting started, take a look at [concepts](/docs/getting-started/concepts/namespaces) to understand the different components in Onepanel.
 
 :::important
-The steps in the quick start allow you to quickly create a Onepanel cluster for testing. To create a production cluster with TLS and auto scaling enabled see [instructions for your cloud provider](/docs/deployment/overview#installing-on-public-cloud).
+The steps in the quick start allow you to quickly create a Onepanel cluster for evaluation. To create a production cluster with TLS and a managed database see [instructions for your cloud provider](/docs/deployment/overview#installing-on-public-cloud).
 
 You can also [add components](/docs/deployment/upgrade/overview) at later time to make this cluster production ready.
 :::
@@ -26,7 +26,7 @@ Let's get started by creating a Kubernetes cluster in one of the following cloud
     { label: 'Azure AKS', value: 'aks', },
     { label: 'Amazon EKS', value: 'eks', },
     { label: 'Google Cloud GKE', value: 'gke', },
-    { label: 'Minikube', value: 'minikube', },
+    // { label: 'Minikube', value: 'minikube', },
     // { label: 'Microk8s', value: 'microk8s', },
   ]
 }>
@@ -44,7 +44,7 @@ az aks create --resource-group <resource-group> --name <cluster-name> --location
     --node-vm-size Standard_D4s_v3 \
     --node-osdisk-size 100 \
     --min-count 2 \
-    --max-count 2 \
+    --max-count 5 \
     --enable-cluster-autoscaler \
     --network-plugin azure \
     --network-policy azure \
@@ -54,10 +54,24 @@ az aks create --resource-group <resource-group> --name <cluster-name> --location
 
 You can then get access credentials by running:
 
-```
+```bash
 az aks get-credentials --resource-group <resource-group> --name <cluster-name> --admin
 ```
 
+Optionally, you can add additional auto-scaling node pools to the cluster as follows:
+
+```bash
+az aks nodepool add --resource-group <resource-group> --cluster-name <cluster-name> \
+  --name <nodepool-name> \
+  --node-vm-size <node-vm-size> \
+  --enable-cluster-autoscaler \
+  --node-count 1 \
+  --min-count 0 \
+  --max-count <max-count>
+```
+
+In step <strong>1.3</strong> below, you can configure Onepanel to automatically scale these nodes as needed.
+
 </TabItem>
 <TabItem value="eks">
 
@@ -73,7 +87,7 @@ eksctl create cluster --name=<cluster-name> --region <region> \
     --node-type m5.xlarge \
     --node-volume-size 100 \
     --nodes-min 2 \
-    --nodes-max 2 \
+    --nodes-max 5 \
     --asg-access \
     --managed \
     --ssh-access
@@ -95,12 +109,12 @@ Make sure [Google Cloud SDK](https://cloud.google.com/sdk/install) (`gcloud`) is
 Run this `gcloud` command to create a bare minimum cluster with 2 `n1-standard-4` nodes:
 
 ```bash
-gcloud container --project <project-name> clusters create <cluster-name> --zone <zone> --node-locations <single-location> \
+gcloud container --project <project-name> clusters create <cluster-name> --zone <zone> --node-locations <zone> \
     --num-nodes 2 \
     --machine-type n1-standard-4 \
     --disk-size 100 \
-    --min-nodes 0 \
-    --max-nodes 2 \
+    --min-nodes 2 \
+    --max-nodes 5 \
     --enable-autoscaling \
     --enable-network-policy \
     --enable-stackdriver-kubernetes \
@@ -113,6 +127,21 @@ The command above will automatically retrieve your cluster's access credentials
 gcloud container clusters get-credentials <cluster-name> --zone <zone>
 ```
 
+Optionally, you can add additional auto-scaling node pools to the cluster as follows:
+
+```bash
+gcloud container node-pools create <node-pool-name> --cluster <cluster-name> --zone <zone> \
+  --machine-type <machine-type> \
+  --disk-size 100 \
+  --num-nodes 0 \
+  --min-nodes 0 \
+  --max-nodes 5 \
+  --enable-autoscaling \
+  --accelerator 'type=<type>,count=<count>'  # optional, example: 'type=nvidia-tesla-v100,count=1'
+```
+
+In step <strong>1.3</strong> below, you can configure Onepanel to automatically scale these nodes as needed.
+
 </TabItem>
 <TabItem value="minikube">
 
@@ -345,53 +374,45 @@ Next, get the kubeconfig by running
       { label: 'Azure AKS', value: 'aks', },
       { label: 'Amazon EKS', value: 'eks', },
       { label: 'Google Cloud GKE', value: 'gke', },
-      { label: 'Minikube', value: 'minikube', },
+      // { label: 'Minikube', value: 'minikube', },
       // { label: 'Microk8s', value: 'microk8s', },
     ]
   }>
   <TabItem value="aks">
 
   ```bash
   opctl init --provider aks \
-    --artifact-repository-provider s3
+    --artifact-repository-provider s3 \
+    --gpu-device-plugins nvidia
   ```
 
   :::note
-  If you have GPU nodes, you need to set the `--gpu-device-plugins` flag. Valid values are `nvidia` and `amd` or a comma separated combination of both `nvidia,amd`.
-
-  Valid options for `--artifact-repository-provider` flag are `s3` or `gcs`. Use `s3` for any S3 compatible object storage like Minio. 
-
-  See [CLI overview](/docs/deployment/configuration/cli) for additional flags that that may apply.
+  Currently, the only valid option for `--artifact-repository-provider` flag is `s3`, which supports any S3 compatible object storage like [Minio](https://docs.min.io/) and [GCS (with HMAC key enabled)](https://cloud.google.com/storage/docs/authentication/managing-hmackeys#create). 
   :::
 
   </TabItem>
   <TabItem value="eks">
 
   ```bash
   opctl init --provider eks \
-    --artifact-repository-provider s3
+    --artifact-repository-provider s3 \
+    --gpu-device-plugins nvidia
   ```
 
   :::note
-  If you have GPU nodes, you need to set the `--gpu-device-plugins` flag. Valid values are `nvidia` and `amd` or a comma separated combination of both `nvidia,amd`.
-
-  Valid options for `--artifact-repository-provider` flag are `s3` or `gcs`. Use `s3` for any S3 compatible object storage like Minio. 
-
-  See [CLI overview](/docs/deployment/configuration/cli) for additional flags that that may apply.
+  Currently, the only valid option for `--artifact-repository-provider` flag is `s3`, which supports any S3 compatible object storage like [Minio](https://docs.min.io/) and [GCS (with HMAC key enabled)](https://cloud.google.com/storage/docs/authentication/managing-hmackeys#create). 
   :::
 
   </TabItem>
   <TabItem value="gke">
 
   ```bash
   opctl init --provider gke \
-      --artifact-repository-provider s3
+    --artifact-repository-provider s3
   ```
 
   :::note
-  Valid options for `--artifact-repository-provider` flag are `s3` or `gcs`. Use `s3` for any S3 compatible object storage like Minio. 
-
-  See [CLI overview](/docs/deployment/configuration/cli) for additional flags that that may apply.
+  Currently, the only valid option for `--artifact-repository-provider` flag is `s3`, which supports any S3 compatible object storage like [Minio](https://docs.min.io/) and [GCS (with HMAC key enabled)](https://cloud.google.com/storage/docs/authentication/managing-hmackeys#create). 
   :::
   </TabItem>
 
@@ -404,11 +425,7 @@ Next, get the kubeconfig by running
   ```
 
   :::note
-  If you have GPU nodes, you need to set the `--gpu-device-plugins` flag. Valid values are `nvidia` and `amd` or a comma separated combination of both `nvidia,amd`.
-
-  Valid options for `--artifact-repository-provider` flag are `s3` or `gcs`. Use `s3` for any S3 compatible object storage like Minio. 
-
-  See [CLI overview](/docs/deployment/configuration/cli) for additional flags that that may apply.
+  Currently, the only valid option for `--artifact-repository-provider` flag is `s3`, which supports any S3 compatible object storage like [Minio](https://docs.min.io/) and [GCS (with HMAC key enabled)](https://cloud.google.com/storage/docs/authentication/managing-hmackeys#create). 
   :::
 
   </TabItem>
@@ -439,7 +456,7 @@ Next, get the kubeconfig by running
       { label: 'Azure AKS', value: 'aks', },
       { label: 'Amazon EKS', value: 'eks', },
       { label: 'Google Cloud GKE', value: 'gke', },
-      { label: 'Minikube', value: 'minikube', },
+      // { label: 'Minikube', value: 'minikube', },
       // { label: 'Microk8s', value: 'microk8s', },
     ]
   }>