Kubernetes Orchestration · Part 5

Managed K8s on EKS

Q: You need to run GPU-based LLM inference (K7) and also have a bursty webhook handler. Which node model fits each on EKS, and why?

GPU inference must run on a managed (EC2) node group with GPU instance types — Fargate has no GPU support, so it's not an option for the model server. The bursty webhook handler is a good fit for Fargate: you pay per Pod's requested resources with no idle-node cost, and each Pod gets its own isolated micro-VM, which suits spiky, low-baseline traffic. Mixing the two — an EC2 GPU node group plus Fargate for bursty stateless work — is a common and deliberate EKS pattern. Verify current GPU/Fargate capabilities against the EKS docs.

Everything so far runs on any cluster. This lesson goes to Amazon EKS — managed Kubernetes on AWS, and the bridge to the AWS AI Automation track. You'll stand up a cluster with eksctl, choose node groups (managed EC2 vs Fargate), give Pods AWS permissions the right way with IRSA (IAM Roles for Service Accounts), push images to ECR, expose apps through the AWS Load Balancer Controller, and face the production concerns EKS makes real: logging, cost, and upgrades.

⏱️ ~100 min☁️ AWS EKS🎯 Intermediate→Professional

🌱 This one costs money — read firstEKS is a cloud service with real charges (control plane hourly fee, EC2/Fargate, load balancers, data transfer). Everything here needs an AWS account and an EKS cluster — you cannot run it offline. Tear resources down when done (eksctl delete cluster). Commands and IAM/annotation details drift; verify every one against the current EKS and eksctl docs. This ties directly into the AWS track (W1–W14).

Learning objectives

Explain what EKS manages for you (the control plane) versus what you still own (nodes, add-ons).
Create a cluster and node group with eksctl, and choose managed node groups vs Fargate.
Grant Pods least-privilege AWS access with IRSA instead of node-wide credentials.
Push a container image to ECR and pull it into the cluster.
Expose a service via the AWS Load Balancer Controller, and reason about logging and cost.

1 · What EKS manages — and what you still own

On EKS, AWS runs the control plane — api-server and etcd, replicated across Availability Zones, patched and backed up for you. That removes the scariest operational burden (running etcd). But EKS is not fully managed: you still own the worker nodes, the add-ons (CNI, CoreDNS, controllers), upgrades of your workloads, and cost.

The shared-responsibility lineThink of it as: AWS keeps the brain alive; you keep the muscle healthy. You never SSH to an api-server, but you do choose node types, patch node AMIs (or let managed node groups do it), install controllers, and pay for every node and load balancer. Managed means fewer 3am pages, not zero responsibility.

2 · Create a cluster with eksctl

eksctl is the quickest path to a working cluster: one config file describes the cluster and its node groups, and eksctl provisions the VPC, control plane, and nodes via CloudFormation.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.

YAML · an eksctl cluster config (needs an AWS account — verify fields vs current eksctl docs)

cluster.yamlapiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: ai-platform
  region: us-east-1
  version: "1.30"                 # pick a currently-supported version
managedNodeGroups:
  - name: general
    instanceType: m6i.large
    minSize: 2
    maxSize: 6
    desiredCapacity: 3
    volumeSize: 40
iam:
  withOIDC: true                  # REQUIRED to use IRSA (section 3)

shell · create the cluster and point kubectl at it (needs an AWS account)

create.sheksctl create cluster -f cluster.yaml     # ~15-20 min; provisions everything
aws eks update-kubeconfig --name ai-platform --region us-east-1
kubectl get nodes                          # your managed nodes, Ready
# when finished, to STOP being billed:
eksctl delete cluster --name ai-platform

NAME                             STATUS   ROLES    AGE   VERSION
ip-10-0-1-23.ec2.internal        Ready    <none>   3m    v1.30.x
ip-10-0-2-41.ec2.internal        Ready    <none>   3m    v1.30.x
ip-10-0-3-88.ec2.internal        Ready    <none>   3m    v1.30.x

3 · Node groups: managed EC2 vs Fargate

Where do your Pods actually run? Two models, and you can mix them:

	Managed node group (EC2)	Fargate
You manage	Instance type, scaling, AMI patching (assisted)	Nothing — AWS runs each Pod
Granularity	Per node (pack many Pods)	Per Pod (one micro-VM each)
Cost shape	Pay per node, even partly idle	Pay per Pod's requested vCPU/memory
Good for	Steady workloads, GPUs, DaemonSets	Bursty, spiky, or isolation-sensitive workloads
GPUs	Yes (GPU instance types)	No GPU support

Mix them deliberatelyA common pattern: a small managed node group for steady baseline services and system add-ons, plus Fargate for bursty or untrusted workloads that shouldn't share a node. GPU workloads (K7) must run on EC2 node groups — Fargate has no GPUs. Verify current instance/Fargate capabilities against the EKS docs before committing.

4 · IRSA — give Pods AWS permissions the right way

Your app needs to call AWS (read S3, invoke Bedrock, write to DynamoDB). The wrong way is to attach broad permissions to the node's instance role — then every Pod on that node inherits them. The right way is IRSA: map a Kubernetes ServiceAccount to a specific IAM role via OIDC, so only Pods using that ServiceAccount get exactly those permissions.

shell + YAML · create an IRSA-backed ServiceAccount (needs an AWS account + withOIDC)

irsa.sh# eksctl creates the IAM role, the trust policy, and the annotated ServiceAccount:
eksctl create iamserviceaccount \
  --cluster ai-platform \
  --namespace ai \
  --name bedrock-caller \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonBedrockReadOnly \
  --approve
# the resulting ServiceAccount carries an annotation like:
#   eks.amazonaws.com/role-arn: arn:aws:iam::<acct>:role/<generated-role>
# then a Pod uses it:
#   spec.serviceAccountName: bedrock-caller

Never put broad AWS permissions on the node roleIf you attach, say, AmazonS3FullAccess to the node instance role, every Pod on that node — including a compromised one — can use it. IRSA scopes credentials to a single ServiceAccount so each workload gets least privilege. This is the single most important EKS security practice. (Verify whether EKS Pod Identity — a newer alternative — better fits your case against the current docs.)

5 · ECR, the AWS Load Balancer Controller, and prod concerns

Two more AWS-native pieces, then the operational realities.

shell · build, push to ECR, and reference the image (needs an AWS account)

ecr.shaws ecr create-repository --repository-name web
aws ecr get-login-password --region us-east-1 \
  | docker login --username AWS --password-stdin <acct>.dkr.ecr.us-east-1.amazonaws.com
docker build -t <acct>.dkr.ecr.us-east-1.amazonaws.com/web:1.0.0 .
docker push <acct>.dkr.ecr.us-east-1.amazonaws.com/web:1.0.0
# then in your Deployment: image: <acct>.dkr.ecr.us-east-1.amazonaws.com/web:1.0.0

The AWS Load Balancer Controller lets an Ingress (K4) provision a real AWS Application Load Balancer, and a LoadBalancer Service provision an NLB. Install it (via Helm, K6) and it fulfills Ingress objects natively on AWS.

YAML · an ALB-backed Ingress via the AWS Load Balancer Controller (verify annotations vs docs)

alb-ingress.yamlapiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service: { name: web, port: { number: 80 } }

Prod concern	On EKS you handle it with
Logging	Ship container logs to CloudWatch (Fluent Bit / the CloudWatch add-on)
Metrics	Container Insights, or Prometheus + Grafana
Cost	Right-size nodes, use Spot for tolerant workloads, Cluster Autoscaler/Karpenter, watch idle nodes
Upgrades	Bump the control-plane version, then node groups; respect PDBs (K3) during rolls
Security	IRSA for Pods, private API endpoint, least-privilege RBAC, image scanning in ECR

The cluster keeps billing whether or not you use itAn EKS control plane charges hourly and idle nodes/load balancers charge continuously — a forgotten test cluster is a real (and common) surprise bill. Delete clusters you're done with, and put cost monitoring on the ones you keep. Verify current pricing and teardown steps against AWS docs.

✓ Checkpoint — you can move on when you can…

State what EKS manages (control plane) and what you still own (nodes, add-ons, cost, upgrades).
Create a cluster with an eksctl config and point kubectl at it.
Choose managed node groups vs Fargate for a given workload, including the GPU constraint.
Explain IRSA and why node-role permissions are the wrong approach.
Push an image to ECR and expose a service via the AWS Load Balancer Controller.

✓ Knowledge check

A teammate grants their Pod access to S3 by attaching AmazonS3FullAccess to the EKS node group's instance role. It works. Why is this a serious mistake, and what's the correct approach on EKS?

Show answer

Attaching the policy to the node role gives that permission to every Pod scheduled on those nodes, not just the intended one — so any other workload (or a compromised container) on the node can read and write all of S3. The correct approach is IRSA: create an IAM role scoped to exactly the needed S3 actions, associate it with a dedicated Kubernetes ServiceAccount via the cluster's OIDC provider, and set serviceAccountName on only the Pods that need it. Now credentials are least-privilege and per-workload. (EKS Pod Identity is a newer alternative — check current docs.)

✓ Knowledge check

You need to run GPU-based LLM inference (K7) and also have a bursty webhook handler. Which node model fits each on EKS, and why?

Show answer

GPU inference must run on a managed (EC2) node group with GPU instance types — Fargate has no GPU support, so it's not an option for the model server. The bursty webhook handler is a good fit for Fargate: you pay per Pod's requested resources with no idle-node cost, and each Pod gets its own isolated micro-VM, which suits spiky, low-baseline traffic. Mixing the two — an EC2 GPU node group plus Fargate for bursty stateless work — is a common and deliberate EKS pattern. Verify current GPU/Fargate capabilities against the EKS docs.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Stand up an EKS cluster and reach itBeginner

Context: Getting from zero to a reachable managed cluster is the first EKS milestone — and knowing how to tear it down is the second.

Your task: Write the eksctl config for a small cluster and the commands to create it, point kubectl at it, and delete it when done.

Requirements:

A managed node group with 2–4 nodes
Enable OIDC (needed for IRSA later)
Show update-kubeconfig and a get nodes check
Show the delete command and say why it matters

💡 Hint: Enabling withOIDC now saves you re-provisioning later for IRSA.

Show solution

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata: { name: demo, region: us-east-1, version: "1.30" }
iam: { withOIDC: true }
managedNodeGroups:
  - { name: general, instanceType: m6i.large, minSize: 2, maxSize: 4, desiredCapacity: 2 }

eksctl create cluster -f cluster.yaml
aws eks update-kubeconfig --name demo --region us-east-1
kubectl get nodes
eksctl delete cluster --name demo    # stops the billing

Why delete matters: the control plane bills hourly and nodes bill continuously, so a forgotten demo cluster quietly runs up a bill. Enabling withOIDC up front means the cluster is ready for IRSA without re-provisioning. Verify the supported version against current EKS docs.

Exercise 2 · Push to ECR and deploy the imageIntermediate

Context: Your registry is where deploys pull from; wiring ECR correctly is a daily EKS task.

Your task: Give the commands to create an ECR repo, authenticate, build and push an image, then the Deployment snippet that references it.

Requirements:

Create the repo and log docker in to ECR
Build and push a version-tagged image (never :latest)
Reference the full ECR image URI in a Deployment

💡 Hint: The image URI is <acct>.dkr.ecr.<region>.amazonaws.com/<repo>:<tag>.

Show solution

aws ecr create-repository --repository-name web
aws ecr get-login-password --region us-east-1 \
  | docker login --username AWS --password-stdin .dkr.ecr.us-east-1.amazonaws.com
docker build -t .dkr.ecr.us-east-1.amazonaws.com/web:1.0.0 .
docker push .dkr.ecr.us-east-1.amazonaws.com/web:1.0.0

Reference the pushed image by its full URI, pinned to the version tag:

      containers:
        - name: app
          image: .dkr.ecr.us-east-1.amazonaws.com/web:1.0.0

EKS nodes can pull from ECR in the same account by default via their instance role; cross-account or private setups need an image-pull configuration — verify against current ECR/EKS docs.

Exercise 3 · Wire IRSA for a Pod that calls AWSAdvanced

Context: IRSA is the correct, least-privilege way to give a Pod AWS access; setting it up is a core EKS competency.

Your task: Give the steps to let only a specific Pod read from Bedrock via IRSA, and explain why this beats node-role permissions.

Requirements:

Ensure the cluster has OIDC enabled
Create an IAM-backed ServiceAccount scoped to the needed policy
Attach it to the Pod and explain the credential path
Contrast with node-role permissions

💡 Hint: eksctl can create the role, trust policy, and annotated ServiceAccount in one command.

Show solution

eksctl create iamserviceaccount \
  --cluster ai-platform --namespace ai --name bedrock-caller \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonBedrockReadOnly --approve

Attach it to the Pod:

    spec:
      serviceAccountName: bedrock-caller

How it works: the ServiceAccount is annotated with an IAM role ARN; via the cluster's OIDC provider, Pods using it receive short-lived, automatically-rotated credentials for exactly that role — the AWS SDK picks them up with no keys in the image or env.

Why it beats node-role permissions: node-role grants apply to every Pod on the node, violating least privilege; IRSA scopes credentials to a single ServiceAccount, so only the intended workload can call Bedrock and a neighbor Pod can't. Verify whether EKS Pod Identity is a better fit against current docs.

Exercise 4 · Expose a service with the AWS Load Balancer ControllerExpert

Context: On AWS, Ingress is fulfilled by the AWS Load Balancer Controller provisioning a real ALB; using it correctly is how EKS apps face the internet.

Your task: Describe how to expose a web Service through an internet-facing ALB via an Ingress, and name the prerequisite and the annotation caveat.

Requirements:

State the controller prerequisite
Write an ALB Ingress with the key annotations
Explain target-type ip vs instance briefly
Note that annotations are version-specific

💡 Hint: The controller must be installed (Helm) and have IRSA permissions to manage ELBs.

Show solution

Prerequisite: install the AWS Load Balancer Controller (via Helm, K6) and grant it IRSA permissions to create/manage load balancers. Without it, the Ingress is inert.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
    - http:
        paths:
          - { path: /, pathType: Prefix, backend: { service: { name: web, port: { number: 80 } } } }

target-type: ip routes the ALB straight to Pod IPs (works with Fargate and is the common modern choice); instance routes to node ports. Caveat: these alb.ingress.* annotations are controller-version-specific — verify each against the controller's current docs rather than copying blindly.

Exercise 5 · Design a cost- and log-aware node topologyProfessional

Context: On EKS, node choices and observability decisions directly drive the bill and your ability to debug; a professional plans both up front.

Your task: Design the node-group topology, logging, and cost controls for a platform with steady services, bursty jobs, and one GPU inference workload.

Requirements:

Assign each workload class to managed nodes or Fargate (with the GPU constraint)
State the logging/metrics pipeline
Name at least three concrete cost controls
Note the upgrade approach and its interaction with PDBs

💡 Hint: GPUs force EC2; bursty stateless favors Fargate; steady baseline favors a small managed group.

Show solution

Node topology: a small managed node group for steady baseline services and system add-ons; Fargate for bursty jobs (pay per Pod, no idle cost, isolation); a GPU managed node group for the inference workload, since Fargate has no GPUs (K7). Autoscale nodes with the Cluster Autoscaler or Karpenter.

Logging/metrics: Fluent Bit (or the CloudWatch add-on) ships container logs to CloudWatch; Container Insights or Prometheus+Grafana for metrics — so you can actually debug what the cluster does.

Cost controls: (1) right-size requests so the scheduler packs nodes tightly; (2) use Spot instances for interruption-tolerant workloads; (3) autoscale to reclaim idle nodes; (4) alarm on idle/forgotten clusters and cap max node counts. Idle nodes and forgotten load balancers are the biggest silent costs.

Upgrades: bump the control-plane version first, then roll node groups; respect PodDisruptionBudgets (K3) so draining nodes for the roll never breaches availability. Verify current pricing, autoscaler, and upgrade steps against AWS docs.

Exercise 6 · Promote a working app from a local cluster to EKSIndustry scenario

Context: Representative scenario: an app runs on kind locally and must go to EKS for production, with proper AWS integration, exposure, and cost discipline.

Your task: Write the migration plan: what changes from the local manifests, how AWS access and image pulls are handled, how it's exposed, and how it's kept observable and affordable.

Requirements:

State what's identical and what changes moving local → EKS
Handle images (ECR) and AWS permissions (IRSA)
Expose it via the AWS Load Balancer Controller
Include logging, cost controls, and teardown discipline

💡 Hint: The workload YAML barely changes; what changes is registry, IAM, ingress, and cost.

Show solution

What stays the same: the Deployment, Service, ConfigMap/Secret, probes, resources, HPA, and PDB manifests (K2–K4) are portable — that's the payoff of standard Kubernetes. The workload definition barely changes.

What changes for EKS:

Cluster: create with eksctl (OIDC enabled), point kubectl at it.
Images: push to ECR and update image URIs to the ECR path (versioned tags).
AWS access: replace any local fake credentials with IRSA — a scoped ServiceAccount per workload that needs AWS (S3/Bedrock/etc.), never node-role permissions.
Exposure: swap the local NodePort/port-forward for an ALB Ingress via the AWS Load Balancer Controller, with TLS (ACM/cert-manager).
Secrets: move real values into AWS Secrets Manager (via a CSI driver) instead of plain manifests; keep only placeholders in git.

Operate it: ship logs to CloudWatch, add Container Insights/Prometheus metrics, right-size requests, use Spot for tolerant workloads, autoscale nodes, and respect PDBs on upgrades. Put a cost alarm on the cluster and tear down anything non-prod when idle. Verify every AWS-specific command and annotation against the current EKS docs — this is the layer that drifts most.

Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.

Learning objectives

1 · What EKS manages — and what you still own

2 · Create a cluster with eksctl

3 · Node groups: managed EC2 vs Fargate

4 · IRSA — give Pods AWS permissions the right way

5 · ECR, the AWS Load Balancer Controller, and prod concerns

✓ Checkpoint — you can move on when you can…

🪜 Practice ladder beginner → industry

More in Kubernetes orchestration