Slowly I'm learning more and more AWS services. Today I'll be writing about an awesome service called Systems Manager (SSM) and how it can be used to maintain and manage a fleet of EC2 instances.
SSM is not one function, but a collection of other functions that can be used in synergy. Think EC2 and all of the functions you can perform within that service (AMI, security groups, instance types, and etc).
SSM can automate patch updates, manage terminal sessions, run "one-off" commands, and provides a secure location for sensitive credentials used in your AWS environment! If you're an avid reader of my blog, you may have noticed I used Parameter Store in my blog's migration to AWS. Well Parameter Store was just one function of Systems Manager, I'll show you all some more functions within Systems Manager.
Creating an EC2 Instance Role
First, I'll create a new role this role will have the ability to interact (read and write) with SSM. Along with that, I'll also allow the role to write to S3. This is important for audit reasons in case you want to output executions of SSM into a log file. More on this in the Run Command section of this post.
Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:DescribeAssociation",
"ssm:GetDeployablePatchSnapshotForInstance",
"ssm:GetDocument",
"ssm:DescribeDocument",
"ssm:GetManifest",
"ssm:GetParameter",
"ssm:GetParameters",
"ssm:ListAssociations",
"ssm:ListInstanceAssociations",
"ssm:PutInventory",
"ssm:PutComplianceItems",
"ssm:PutConfigurePackageResult",
"ssm:UpdateAssociationStatus",
"ssm:UpdateInstanceAssociationStatus",
"ssm:UpdateInstanceInformation"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ssmmessages:CreateControlChannel",
"ssmmessages:CreateDataChannel",
"ssmmessages:OpenControlChannel",
"ssmmessages:OpenDataChannel"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ec2messages:AcknowledgeMessage",
"ec2messages:DeleteMessage",
"ec2messages:FailMessage",
"ec2messages:GetEndpoint",
"ec2messages:GetMessages",
"ec2messages:SendReply"
],
"Resource": "*"
}
]
}Installing the SSM-Agent
Download and still the AWS SSM agent. The ssm agent will run on the instance and allow me to interact with it from the Systems Manager console.
sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpmStart and enable the ssm-agent service.
sudo systemctl start amazon-ssm-agent && sudo systemctl enable amazon-ssm-agentIf you're using the standard Amazon Linux ami, the ssm-agent should already be installed by default. However as Jan 2020, this is not true for ECS ami.
Session Manager
Time to test one of SSM's functions! The first function I'll play with is Session Manager.
Before I start a new session I will edit the preferences. Click on the preferences tab, in here I'll write the session output to an S3 bucket named sessmgrlogs. Doing this will allow me to audit actions done on my instances. Handy if you work for an organization that routinely does audits of their systems.
Now head back into the sessions tab and search for your instance. I only have a single instance in this region which makes it easy to find.
Next select the instance and click on the "Start session" button.
It will open a different window showing you a terminal connected to your instance!
Now type in whoami and press enter. It should return an output saying that it logged you into your instance as the ssm-user.
sh-4.2$ whoami
ssm-userRun Command
The Run Command is by far my favorite function within SSM! It allows you to execute a command against a single EC2 instance or your entire fleet.
Go ahead and navigate to the Run Command within Systems Manager and click on the "Run command" button.
In the next page, you will be able to choose a pre-made document. You can create your own document, but I'll be using AWS-RunShellScript.
Click on AWS-RunShellScript, the next page will show you details such as the version, platform it will run on, and owner.
You can even read through the contents and see how the document was created in JSON.
Once you're ready to execute this document, click on the "Run command" button and it will take you to a new page where you can configure the actual command.
For the command parameters, I'll simply echo hello world and redirect the output to /tmp/hello.txt.
To target your instance with the Run Command, you'll need to select them. You can either specify the instances using tags which is what I'll be using or manually choose the instance.
If you choose to go with specifying the targets with tags, you'll be able to direct Run Command to execute the script on multiple instances.
When you're ready to run the command click on the top right button "Run command". A new window will appear, in this window you can see the status of the command and the instance it targeted.
In the picture above, it correctly targeted my ECS instance, but let's check if it created the hello.txt file in the /tmp directory. Connect to the instance using Session Manager and run a cat on /tmp/hello.txt.
sh-4.2$ cat /tmp/hello.txt
Hello world!Great it worked! Now you can use this to quickly manage your fleet without ever having to ssh into them.
Patch Manager
If you need to ensure your instances have the correct package versions installed, Patch Manager is there to help! This function is similar to configuration management tools like Chef, Salt, and Ansible. It will allow you to setup maintenance windows and patch your instances.
Dive into the Patch Manger section and click on "Create patch baseline".
Give your patch baseline a name and an optional description. Make sure to select the appropriate operating system for your instances. Mine is using Amazon Linux 2. I set mine as a default baseline, if you do not want Patch Manager updating other instances, I recommend that you do not enable this feature.
Select All for the product, severity, and classification.
You can leave the patch exceptions blank. This section allows you to control which patches can be installed or rejected.
You can also select where Patch Manager will get it's patches. I left this section blank so it will use the default Amazon Linux 2 patch sources.
Now it's time to create a maintenance window. Navigate to the maintenance windows section on the left sidebar and then create a new maintenance window.
In the new page that appears, go to the targets tab and register your EC2 instance(s).
I chose to manually select my instance, however you can choose based on tags and resource group.
Once the targets have been configured, click on the actions button and select "Register Run command task".
Scroll down to the command document section and search for AWS-RunPatchBaseline.
Register your target instances in the window. Set the concurrency to one and error threshold 0. This will only allow one instance to be patched at a time and to never fail the whole process if an error is received.
For the parameters I selected Install for the operation. If you only want Patch Manager to lookup version info, select Scan. I want Patch Manager to install newer versions of patches. Keep the other options default.
After all of that you can finally click on the "Register Run command task" button. Now depending on how you configured your maintenance window Patch Manager may take some time to execute. I configured mine to execute every thirty minutes.
After waiting thirty minutes, I then proceeded to the history tab of my maintenance window and observed one of the executions. Here you are able to see the status of the patch applied on your instance.
Going forward, I think I need to edit the parameters for the maintenance window. I don't want my instance rebooting without a manual approval. However this is fine for now and is working as intended!
There are many more functions within Systems Manager that I need to explore, but these are a good sampling of the capabilities within SSM. Definitely a cool utility for traditional operators and devops focused individuals and teams! I'm certainly going to incorporate this into my release pipelines.