Access Specific EC2 behind ALB

When there are multiple EC2 instances running behind the application load balancer, there is no way (at least as the time of writing) to tell the load balancer which instance you want to use. It is all controlled by the pre-defined load_balancing.algorithm.type.

load_balancing.algorithm.type – The load balancing algorithm determines how the load balancer selects targets when routing requests. The value is round_robin or least_outstanding_requests. The default is round_robin.

Like the following diagram.

1. Users go to https://confluence.jackiechen.org.
2. Application load balancer receives the request then forwards the requests to the backend target group A based on the pre-defined rule (host header is confluence.jackiechen.org).
3. Target group A distributes the traffics to all the healthy instances (01, 02, 03) that are register with it. The load balancing algorithm can be either round robin or least outstanding requests.

However in some scenarios, as an application owner we may need to login into a specific instance. Let’s continue TO use the above diagram as a example, users report that they have some issues with the confluence site from time to time. And based on the analysis, it only happens when they are hitting the instance 03. The node information can be obtained from the Confluence page footer or X-Confluence-Cluster-Node-Name http header.

As an application admin, we would like to check instance 03 to see if we can re-produce the issues that users have. But in the above setup, there is no guarantee that our traffics will be served by the instance 03. Right?

The solution that we worked out is to add an additional target group which is used for this type of purpose. We call it admin target group.

Let me explain how it works:

4. Create an additional target group B behind the same ALB, and add a rule to ALB to forward the traffics to it if the host header is confluence-admin.jackiechen.org.

5. Register instance 03 to target group B. Optionally we can de-register instance 03 from target group A, so users’s traffics to confluence.jackiechen.org won’t be served by instance 03.

6. Now if we go to https://confluence-admin.jackiechen.org, the requests will be only served by instance 03. So we are able to poke around to do the troubleshooting. Just please be aware of that most web applications check cross-site scripting (XSS). For example, some operations could not be done in Confluence due to the failed cross-site request forgery check (XSRF check) when the request referral URL does not match its base URL. The workaround is to configure the Origin header in the browser extension (e.g ModHeader)

AWS CLI samples:

# Get target group B arn
TARGET_GROUP_B=$(aws elbv2 describe-target-groups --query 'TargetGroups[?contains(TargetGroupName, `confluence-admin`)].TargetGroupArn' --output text)

# Instance 03 ID
INSTANCE_03_ID=i-00000000000000003

# Register instance 03 to target group B
aws elbv2 register-targets --target-group-arn ${TARGET_GROUP_B} --targets Id=${INSTANCE_03_ID}

# Get the registered instance with target group B.
aws elbv2 describe-target-health --target-group-arn ${TARGET_GROUP_B} --query 'TargetHealthDescriptions[*].Target.Id' --output text

# De-register instance 03 to target group B
aws elbv2 deregister-targets --target-group-arn ${TARGET_GROUP_B} --targets Id=${INSTANCE_03_ID}