how to authenticate to eks cluster using the eks-kubectl-run pipe

[martinyung]

pipe version: aws-eks-kubectl-run:1.1.1

My Bitbucket Pipeline

- pipe: atlassian/aws-eks-kubectl-run:1.1.1
  variables:
    AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
    AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
    AWS_DEFAULT_REGION: $REGION
    CLUSTER_NAME: $CLUSTER_NAME
    KUBECTL_COMMAND: 'set image deployment/app-frontend app-frontend=$IMAGE_NAME'

Error Message:
Added new context arn:aws:eks:[region]:[aws_id]:cluster/[cluster_name] to /root/.kube/config✔ 
Successfully updated the kube config.
error: You must be logged in to the server (Unauthorized)✖ 
kubectl set image deployment/app-frontend app-frontend=IMAGE_NAME failed.

can anyone please guide me on how to authenticate using this pipe? as the documentation doesn't say much aside from filling in those required variables.

Thanks 

[martinyung · Accepted solution]

after much research finally found the solution. Apparently in AWS EKS, by default only the cluster creator IAM can interact with the eks cluster through kubectl.

From AWS EKS Documentation

When you create an Amazon EKS cluster, the IAM entity user or role, such as a federated user that creates the cluster, is automatically granted system:masters permissions in the cluster's RBAC configuration. To grant additional AWS users or roles the ability to interact with your cluster, you must edit the aws-auth ConfigMap within Kubernetes.

AWS EKS Documentation: Managing Users or IAM Roles for your Cluster 

[Alexander Zhukov]

Hi @martinyung , thanks for taking time to share your findings. We will update the docs with more information, including this caveat. 

[Rob Jahn]

I had the same issue running on version atlassian/aws-eks-kubectl-run:1.2.1.  By just edited the config map (kubectl edit -n kube-system configmap/aws-auth) and adding the user I configured in my pipeline fixed it.  Thank you @martinyung this saved alot time!  @Alexander Zhukov if you update the docs, please add to the docs what AWS permissions are required for the AWS user.

[soupdiver]

It seems you can also set `ROLE_ARN` as a variable for the pipe. But to find this I had to  look at the source code....

[opicarellieldorado]

Perfect!!! Code here 

1. Configure role:

TRUST="{ \"Version\": \"2012-10-17\", \"Statement\": [ { \"Effect\": \"Allow\", \"Principal\": { \"AWS\": \"arn:aws:iam::${ACCOUNT_ID}:root\" }, \"Action\": \"sts:AssumeRole\" } ] }"

echo '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "eks:Describe*", "Resource": "*" } ] }' > /tmp/iam-role-policy

aws iam create-role --role-name EksCodeBuildKubectlRole --assume-role-policy-document "$TRUST" --output text --query 'Role.Arn'

aws iam put-role-policy --role-name EksCodeBuildKubectlRole --policy-name eks-describe --policy-document file:///tmp/iam-role-policy

2. Configure AWS-AUTH

ROLE="    - rolearn: arn:aws:iam::${ACCOUNT_ID}:role/EksCodeBuildKubectlRole\n      username: build\n      groups:\n        - system:masters"

kubectl get -n kube-system configmap/aws-auth -o yaml | awk "/mapRoles: \|/{print;print \"$ROLE\";next}1" > /tmp/aws-auth-patch.yml

kubectl patch configmap/aws-auth -n kube-system --patch "$(cat /tmp/aws-auth-patch.yml)"

 
3. Add repository environment ROLE_ARN

4. Add ROLE_ARN=$ROLE_ARN in bitbucket-pipeline.yml

Please attlasian forum, complement the doc with this helpful script