kubernetes - Is there a way to prevent an EKS NodeGroup (EC2 Autoscaling Group) from being used by the EC2 LoadBalancer?

Question

I have an EKS cluster created with eksctl, with two unmanaged nodegroups. ingress-nginx and cluster-autoscaler are deployed and working. ingress-nginx controller has created a Classic LoadBalancer upon deployment.

When either NodeGroup scales up, its instances are added to the LB. (question: what takes this action? It's not the ASG itself, so I assume it's ingress-nginx doing this). Additionally, I see that all instances (from both ASGs) are responding as "healthy" to the TCP healthcheck on the LoadBalancer.

The problem: I need to only whitelist one of the node groups as being eligible for load balancing. The other ASG (and any future ASGs, by default), are batch workers which do not host any internet services, and no web service pods will be ever scheduled on them.

Is this possible to achieve in EKS, and how?

question from:https://stackoverflow.com/questions/65854551/is-there-a-way-to-prevent-an-eks-nodegroup-ec2-autoscaling-group-from-being-us

Answer 1

You are right, that's the default behavior of Kubernetes LoadBalancer services like Nginx Ingress. LoadBalancer under the hood uses NodePort service which is exposed on ALL cluster nodes regardless of EKS node group they belong to.

What you should do is to add externalTrafficPolicy: Local to your Nginx Ingress LoadBalancer like that (this is just an example):

apiVersion: v1
kind: Service
metadata:
  name: example-service
spec:
  selector:
    app: example
  ports:
    - port: 8765
      targetPort: 9376
  externalTrafficPolicy: Local
  type: LoadBalancer

Doing that will cause AWS Load Balancer to target only those nodes who actually run Nginx Ingress pods. After that you may want to use Nginx Ingress nodeSelector to run only on desired EKS node group.