Cannot deploy stash with helm@3

[ljani]

I have a helm umbrella chart and I'd like to deploy stash as a part of it. I added stash@0.8.3 as a dependency for my umbrella chart, but when I try to upgrade the chart, I get the following error:

mychart> helm3 upgrade mychart .
Error: UPGRADE FAILED: [the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation., unable to decode "": no kind "APIService" is registered for version "apiregistration.k8s.io/v1beta1" in scheme "k8s.io/client-go/kubernetes/scheme/register.go:65"]

I'm also experimenting with helm@3.

The documentation states I should pass --namespace kube-system, but as far as I know, I cannot tell Helm to install stash into kube-system and the rest into default. Any ideas?

[hossainemruz]

Hi Jani, You can install Stash in any namespace you want. If you don't provide namespace flag during installation, Stash will be installed in default namespace. It seems that Stash was initially installed in kube-system namespace and while upgrading it, no namespace flag was provided. Try uninstalling Stash then install as dependency as your umbrella chart. It should be installed in default namespace. Regards, Md. Emruz Hossain Software Engineer, AppsCode Ltd. Mobile: +8801770582121

[ljani]

It seems that Stash was initially installed in kube-system namespace and while upgrading it, no namespace flag was provided.

Thanks for your quick reply. I have not installed stash previously at all and I'm the single user of the cluster. I found one reference to kube-system. I wonder if that causes problems with helm@3 then, if it should work with helm@2.

[hossainemruz]

Good catch. It seems to be mistakenly hard coded. We are fixing it soon.

UPD: This is not a bug. It is intentional. Check the following comment for more details: #822 (comment)

[ljani]

Great! I was not sure if it was a mistake or not. Anyway, something like this Chart.yaml should reproduce the issue with helm@3:

apiVersion: v1
name: stash-822
description: See issue 822
type: application
version: 0.1.0
appVersion: 0.1.0
dependencies:
- name: stash
  version: 0.8.3
  repository: "https://charts.appscode.com/stable/"

[hossainemruz]

@ljani I have given a deeper look to this hard-coded namespace. It is not a mistake. It is intentional. Stash operator need to have permissions specified in extension-apiserver-authentication-reader rule. Which reside in kube-system namespace. So, we have to create a RoleBinding in kube-system pointing to ServiceAccount stash-operator at Stash's namespace.

We haven't tested Stash with Helm3. Can you confirm that this problem persist with Helm2?

[ljani]

Yeah, I kinda thought it was intentional. I tried helm@2 and it seems to work:

temp> get-content .\Chart.yaml
apiVersion: v1
name: stash-822
version: 0.1.0
temp> get-content .\requirements.yaml
dependencies:
- name: stash
  version: 0.8.3
  repository: "https://charts.appscode.com/stable/"
temp> helm2 install --name stash-822 .
NAME:   stash-822
LAST DEPLOYED: Sun Jun 23 22:04:05 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/ClusterRole
NAME       AGE
stash-822  1s

==> v1/ClusterRoleBinding
NAME                                AGE
stash-822                           1s
stash-822-apiserver-auth-delegator  1s

==> v1/Deployment
NAME       READY  UP-TO-DATE  AVAILABLE  AGE
stash-822  0/1    1           0          1s

==> v1/Pod(related)
NAME                        READY  STATUS             RESTARTS  AGE
stash-822-6c8c659b67-f7nb9  0/2    ContainerCreating  0         1s

==> v1/RoleBinding
NAME                                                        AGE
stash-822-apiserver-extension-server-authentication-reader  1s

==> v1/Secret
NAME                      TYPE    DATA  AGE
stash-822-apiserver-cert  Opaque  2     1s

==> v1/Service
NAME       TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)            AGE
stash-822  ClusterIP  10.43.222.192  <none>       443/TCP,56789/TCP  1s

==> v1/ServiceAccount
NAME       SECRETS  AGE
stash-822  1        1s

==> v1beta1/APIService
NAME                                      AGE
v1alpha1.admission.stash.appscode.com     1s
v1alpha1.repositories.stash.appscode.com  1s

EDIT: I also tried to clean install (ie. without helm upgrade) the very same chart on a new cluster:

temp> helm3 install stash-822 .
Error: the namespace from the provided object "kube-system" does not match the namespace "default". You must pass '--namespace=kube-system' to perform this operation.

[ljani]

Okay, this is a helm issue. Here is a similar bug and it is being tracked here. EDIT: the workaround (kubectl config set-context default --namespace default) mentioned in the links did not work for me either.

@hossainemruz: Thanks for the support though! The project seems very promising even if I'm yet to try it.

[bacongobbler]

cert-manager/cert-manager#1744 and helm/helm#5628 are two separate issues, though people have confused the two as the same ticket because they're both related to namespaces. They're two separate issues though. Setting the namespace context won't fix this issue.

[bacongobbler]

See helm/helm#5953 for the upstream issue and helm/helm#6046 for a fix.

[tamalsaha]

Thanks @bacongobbler !

[ljani]

helm@v3.0.0-alpha.2 seemed to work as I'm now blocked by #826.