Argo CD Repo-Server Fails to Pull ECR OCI Helm Chart with IRSA

[dmytro-freger-mesalvo]

Hello

Issue Summary
We are trying to configure Argo CD to pull private Helm OCI charts from AWS ECR using IRSA, but receive the following error from the argocd-repo-server:

Error: pull access denied, repository does not exist or may require authorization: authorization failed: no basic auth credentials

{"args":"[helm pull oci://123456789.dkr.ecr.eu-central-1.amazonaws.com/dotnet-base --version 0.1.3 --destination /tmp/2534b935-d1e2-4a19-9537-6956c47eb051]","dir":"","level":"info","msg":"Trace","operation_name":"exec helm","time":"2025-06-08T13:07:11Z","time_ms":89.840609}
{"grpc.code":"Unknown","grpc.component":"server","grpc.error":"error pulling OCI chart: failed to pull OCI chart: failed to get command args to log: helm pull oci://123456789.dkr.ecr.eu-central-1.amazonaws.com/dotnet-base --version 0.1.3 --destination /tmp/2534b935-d1e2-4a19-9537-6956c47eb051 failed exit status 1: Error: pull access denied, repository does not exist or may require authorization: authorization failed: no basic auth credentials","grpc.method":"GenerateManifest","grpc.method_type":"unary","grpc.service":"repository.RepoServerService","grpc.start_time":"2025-06-08T13:07:10Z","grpc.time_ms":"1567.563","level":"error","msg":"finished call","peer.address":"10.0.0.75:55070","protocol":"grpc","time":"2025-06-08T13:07:11Z"}

1. We have installed ARGOCD:

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

2. IAM Policy Created

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:DescribeCluster",
"eks:ListClusters",
"eks:AccessKubernetesApi"
],
"Resource": "*"
}
]
}

3. IAM Role Created

{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::123456789:oidc-provider/oidc.eks.eu-central-1.amazonaws.com/id/<OIDC_ID>"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.eu-central-1.amazonaws.com/id/<OIDC_ID>:sub": "system:serviceaccount:argocd:argocd-ecr-sa",
"oidc.eks.eu-central-1.amazonaws.com/id/<OIDC_ID>:aud": "sts.amazonaws.com"
}
}
}

4. Created ServiceAccount:

apiVersion: v1
kind: ServiceAccount
metadata:
name: argocd-ecr-sa
namespace: argocd
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::123456789:role/mesalvo-platform-argocd-role-feb0c61

5. We pached deployment of argocd-repo-server

kubectl patch deployment argocd-repo-server -n argocd --patch
'{"spec":{"template":{"spec":{"serviceAccountName":"argocd-ecr-sa"}}}}'

kubectl patch deployment argocd-repo-server -n argocd --patch
'{"spec":{"template":{"spec":{"automountServiceAccountToken": true}}}}'

kubectl patch deployment argocd-repo-server -n argocd --patch
'{"spec":{"template":{"spec":{"containers":[{"name":"argocd-repo-server","env":[{"name":"AWS_REGION","value":"eu-central-1"}]}]}}}}'

6. We set role as annotation for argocd-repo-server

kubectl annotate serviceaccount argocd-repo-server -n argocd eks.amazonaws.com/role-arn=arn:aws:iam::123456789:role/mesalvo-platform-argocd-role-feb0c61 --overwrite

7. Repository Secret Created

apiVersion: v1
kind: Secret
metadata:
name: ecr-oci-helm
namespace: argocd
labels:
argocd.argoproj.io/secret-type: repository
type: Opaque
stringData:
name: dotnet-base
type: helm
url: 123456789.dkr.ecr.eu-central-1.amazonaws.com
enableOCI: "true"

8. Created the application:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: ql-report-service-remote
namespace: argocd
spec:
destination:
name: in-cluster
namespace: backend
sources:
- repoURL: 123456789.dkr.ecr.eu-central-1.amazonaws.com
chart: dotnet-base
targetRevision: 0.1.3
helm:
valueFiles:
- $values/Solutions.QL.Reporting.Api/Deployment/values-dev.yaml
- repoURL: [email protected]:_____/ql-report-service.git
targetRevision: develop
ref: values
project: default
syncPolicy:
automated:
prune: true
selfHeal: true

9. Environment Variables Confirmed in argocd-repo-server Pod

AWS_ROLE_ARN=arn:aws:iam::123456789:role/mesalvo-platform-argocd-role-feb0c61
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
AWS_REGION=eu-central-1

10. If we try to pull helm chart from fake-pod, is everything ok:

……
Downloading https://get.helm.sh/helm-v3.18.2-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
Helm installed successfully.
Attempting to get ECR login password...
ECR login password obtained. Attempting helm registry login...
Login Succeeded
Successfully logged into Helm registry. Attempting to pull chart...
Pulled: 123456789.dkr.ecr.eu-central-1.amazonaws.com/dotnet-base:0.1.3
Digest: sha256:c79a6945bac1ac99d829df700989684ff35f769ebf663f2ffc64ea628be34d58
Chart pulled successfully.

I believe we have followed all the required steps to configure IRSA for Argo CD to pull private Helm OCI charts from AWS ECR. However, the authorization is still failing. Please review my configuration and let me know:

What might be missing or incorrectly configured?
If there are any unnecessary steps I’ve taken that can be removed or simplified?
If Argo CD requires any additional setup (e.g., helm registry login or sidecar injection) to support ECR authentication using IRSA?
If this setup is officially supported or recommended by Argo CD for ECR-based OCI Helm charts?

Thank you in advance for your help!!!

[blakepettersson]

Currently there is no direct IRSA support for pulling OCI Helm charts at this time - you'll need to work around it by using external-secrets or similar

[dmytro-freger-mesalvo]

thank you

[truongnht]

@blakepettersson I see with Azure, there is a solution using AzureWorkloadIdentity, can such a solution be implemented for AWS?