Automate Salt-Minion Registrations on EC2

Posted by Miguel Lopez on Fri 30 March 2018 in automation

Intro

These scripts can be used to register your Salt minion to a Salt-Master upon successful launch.

Whether you're launching single instances or instances as part of an auto-scaling group, I highly recommend using cloud-init scripts. They're easy to use and help you install all necessary packages.

My scripts also include CodeDeploy. If you haven't heard of CodeDeploy, take a moment to read about it here: AWS CodeDeploy.

CodeDeploy can help with:

Repeatable deployments
Automatic code deployments to scaled instances
Stops and rollbacks
Deployment history

Cloud Init Script

Technical Stack: SaltStack, EC2 (CentOS)

Use this as the cloud-init data for an EC2 instance. It works perfectly with auto-scaling groups. You should be able to pass this as the user data script for your EC2 instance.

This script installs CodeDeploy and automatically registers the Salt minion to a Salt-Master.

Dynamic by region
Installs the CodeDeploy agent
Registers the Salt minion to the master
Auto-deploys your latest revision from a CodeDeploy deployment group to this instance

#cloud-config
# Set hostname to match the instance ID, rather than the
# automatic hostname based on the IP address.
# In these three commands _GRP_ is a placeholder and
# should be changed to your Auto Scaling Group name.
bootcmd:
  # Dynamically fetch region for EC2 in aws
  - "region=$(curl http://169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/.$//')"
  - "sudo yum -y install ruby wget jq"
  # Install codedeploy https://aws.amazon.com/codedeploy/
  - "sudo cd /home/ec2-user"
  - "sudo wget https://aws-codedeploy-${region}.s3.amazonaws.com/latest/install"
  - "sudo chmod +x ./install"
  - "sudo ./install auto"
  - "sudo service codedeploy-agent start"
  # BOX_NAME fetches the EC2 tag for "Name" - name used to register with salt master
  - "INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id)"
  - "BOX_NAME=$(aws ec2 describe-tags --region $region --filters \"Name=resource-id,Values=$INSTANCE_ID\" | jq '.Tags[] | select(.Key == \"Name\") | .Value' | sed s/\\\"//g)"
  # Change hostnames on VM
  - "cloud-init-per instance my_set_hostname sh -xc \"echo $BOX_NAME-$INSTANCE_ID > /etc/hostname; hostname -F /etc/hostname\""
  - "cloud-init-per instance my_etc_hosts sh -xc \"sed -i -e '/^127.0.0.1/d' /etc/hosts; sed -i -e '/^::1/d' /etc/hosts; echo 127.0.0.1 $BOX_NAME-$INSTANCE_ID >> /etc/hosts\""
  # Install and bootstrap salt-minion to saltmaster
  - "SALT_MASTER_IP={IP-TO-SALT-MASTER-HERE}"
  - "mkdir -p /etc/salt/; $BOX_NAME-$INSTANCE_ID > /etc/salt/minion_id"
  - "sudo curl -o /tmp/bootstrap-salt.sh -L https://bootstrap.saltstack.com"
  - "sudo sh /tmp/bootstrap-salt.sh -i $BOX_NAME-$INSTANCE_ID -A $SALT_MASTER_IP"
  - "sudo rm -f /tmp/bootstrap-salt.sh"
# Preserve the hostname file since we've had to manually edit it
preserve_hostname: true
# Don't let cloud-init update the hosts file since we have edited it manually
manage_etc_hosts: false

You'll notice some curls to http://169.254.169.254. This is an internal API used by EC2 instances to fetch metadata about your instance.

Replace $SALT_MASTER_IP with the IP of your Salt-Master. Don't forget to tag your EC2 instance with a "Name" tag. Naming is important when defining Salt environments.

For example: - stage-api - stage-www - test-api - test-www

These are all great examples of "Name" tags for instances because they allow you to apply Salt states by patterns like *www, *api, stage*, or test*.

This can be extremely useful for defining how you run Salt commands. For example, the following command:

salt 'stage*' state.show_top

would only apply Salt states to environments tagged with stage in their name. In this example, that would include the stage-api and stage-www servers.

Shell Script Equivalent

Technical Stack: SaltStack, EC2 (CentOS)

This script installs CodeDeploy and automatically registers the Salt minion to a Salt-Master. Use this script only if you'd like your packages to be installed post-creation.

Dynamic by region
Installs the CodeDeploy agent
Registers the Salt Minion to the master

Run this as a bootstrapping script on an EC2 instance.

# Dynamically fetch region for EC2 in aws
region=$(curl http://169.254.169.254/latest/meta-data/placement/availability-zone | sed ’s/.$//‘);

# gr8 packages
sudo yum -y install ruby wget jq;

# Install codedeploy https://aws.amazon.com/codedeploy/
sudo cd /home/ec2-user;
sudo wget https://aws-codedeploy-${region}.s3.amazonaws.com/latest/install;
sudo chmod +x ./install;
sudo ./install auto;
sudo service codedeploy-agent start;

# Name used to register with salt master
BOX_NAME=$(aws ec2 describe-tags --region $region --filters \"Name=resource-id,Values=$INSTANCE_ID\" | jq '.Tags[] | select(.Key == \"Name\") | .Value' | sed s/\\\"//g);
INSTANCE_ID=$(curl http://169.254.169.254/latest/meta-data/instance-id);

# Change hostnames on VM
echo $BOX_NAME-$INSTANCE_ID > /etc/hostname;
sed -i -e '/^127.0.0.1/d' /etc/hosts; 
sed -i -e '/^::1/d' /etc/hosts; 
echo 127.0.0.1 $BOX_NAME-$INSTANCE_ID >> /etc/hosts;

# Install and bootstrap salt-minion to saltmaster
SALT_MASTER_IP={IP-TO-SALT-MASTER-HERE}
mkdir -p /etc/salt/; $BOX_NAME-$INSTANCE_ID > /etc/salt/minion_id;
sudo curl -o /tmp/bootstrap-salt.sh -L https://bootstrap.saltstack.com;
sudo sh /tmp/bootstrap-salt.sh -i $BOX_NAME-$INSTANCE_ID -A $SALT_MASTER_IP;
sudo rm -f /tmp/bootstrap-salt.sh;

You'll notice some curls to http://169.254.169.254. This is an internal API used by EC2 instances to fetch metadata about your instance.

Replace $SALT_MASTER_IP with your own variables.