Ansible and AWS Automation

Posted in: Cloud, DevOps, MySQL, Open Source, Technical Track

Overview

The goal was to create and manage a MySQL RDS instance and learn how to control AWS resources by using Ansible. With Ansible interacting with AWS it’s best to determine how hard or easy it is to build and remove items in EC2, to determine the validity of Ansible for automation purposes in AWS. The big drive for a lot of companies is to implement some or all of their infrastructure into the cloud and with Amazon being one of the largest cloud providers it seemed the logical choice to start with. Ansibles main goal is to make automation of your infrastructure quick and simple, but how does it integrate with AWS?

As a MySQL DBA, the ultimate goal is to develop more automated tasks around RDS and Aurora. This post will help develop a basic or initial understanding of using Ansible with AWS to be able to continue working on further automation opportunities.

AWS Authentication with Ansible

The first requirement is to authenticate to AWS and allow Ansible to execute commands into AWS to discover, build, and destroy objects. This was accomplished pretty quickly. In the interest of time, the export method of the credentials was used for the AWS User (IAM) Key and secret. The more preferred method is to use the Ansible Vault to securely store the credentials:

http://docs.ansible.com/ansible/guide_aws.html
http://docs.ansible.com/ansible/playbooks_vault.html

Credentials

export AWS_ACCESS_KEY_ID=''
export AWS_SECRET_ACCESS_KEY=''

Ansible Site

To help make the roles reusable and easily updated, the variables were placed in the main site.yml file for configuring all of the aspects from the network, bastion, and RDS. This helped centralize the configuration of the playbooks. One key thing to note about Ansible interacting with AWS is that the hosts parameter does not come into effect and it is configured for localhost.

Site.yml

---
# This playbook deploys the whole AWS Network, Bastion, Web, and RDS Instances.  Web Configuration is currently manual
# - Must have credentials exported for AWS IAM
# - RDS may take up to 30 minutes to deploy the instance
 
- name: Deploy RDS Infrastructure
  hosts: localhost
  connection: local
  gather_facts: false
  vars:
 
# Global AWS Variables
    region: us-west-2
    aws_network_name: dev
 
# VPC Variables
    vpc_cidr: 10.4.0.0/16
    public_subnet_cidr: 10.4.0.0/24
    public_subnet_az: us-west-2a
    private_subnet_1_cidr: 10.4.1.0/24
    private_subnet_1_az: us-west-2b
    private_subnet_2_cidr: 10.4.2.0/24
    private_subnet_2_az: us-west-2c
 
# Bastion Variables
    ec2_id: "ami-4836a428"
    key_pair: "pythian-markwardt-west"
 
# RDS Variables
    rds_user: root
    rds_pass: 
    rds_instance_type: db.m1.small
    rds_size_gb: 15
    rds_parameter_engine: mysql5.6
    rds_instance_engine: 5.6.34
    rds_parameters:
      - { param: 'binlog_format', value: 'ROW' }
      - { param: 'general_log', value: '1' }
 
  roles:
  - aws-network
  - aws-bastion
  - aws-rds

Building the AWS Network

Building the base network was also pretty easy for a first timer using Ansible AWS modules. There was a small learning curve, but ultimately it was pretty straight forward and the documentation (and examples) helped simplify a lot of the configuration. This required some pre-existing knowledge of setting up and configuring an AWS network. Below is a list of the primary components that were focused on in order to get an operating network with routing and internet access.

– Build VPC
– Build Subnets
– 1 Public subnet to access the environment
– 2 Private subnets for internal traffic. Two because the RDS Subnet group requires two for redundancy.
– Internet Gateway for the VPC
– Routing Table to allow the three subnets to talk to one another and then send any non subnet traffic to the Internet Gateway
– Security groups to allow specific traffic into specific instances

Building AWS Network Tasks

---
- name: Build VPC
  ec2_vpc_net:
    name: "{{ aws_network_name }}-vpc"
    state: present
    cidr_block: "{{ vpc_cidr }}"
    region: "{{ region }}"
 
- name: Get VPC ID
  ec2_vpc_net_facts:
    region: "{{ region }}"
    filters:
      "tag:Name": "{{ aws_network_name }}-vpc"
  register: vpc_facts
 
- name: VPC ID
  debug: 
    var: vpc_facts['vpcs'][0]['id']
 
- name: Create Private Subnet 1
  ec2_vpc_subnet:
    state: present
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    cidr: "{{ private_subnet_1_cidr }}"
    region: "{{ region }}"
    az: "{{ private_subnet_1_az }}"
    resource_tags:
      Name: "{{ aws_network_name }}-private1"
  register: private_subnet_1
 
- name: subnet MySQL database servers ID 1
  debug: 
    var: private_subnet_1['subnet']['id']
 
- name: Create Private Subnet 2
  ec2_vpc_subnet:
    state: present
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    cidr: "{{ private_subnet_2_cidr }}"
    region: "{{ region }}"
    az: "{{ private_subnet_2_az }}"
    resource_tags:
      Name: "{{ aws_network_name }}-private2"
  register: private_subnet_2
 
- name: subnet MySQL database servers ID 2
  debug: 
    var: private_subnet_2['subnet']['id']
 
- name: Create public subnet
  ec2_vpc_subnet:
    state: present
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    cidr: "{{ public_subnet_cidr }}"
    region: "{{ region }}"
    az: "{{ public_subnet_az }}"
    resource_tags:
      Name: "{{ aws_network_name }}-public"
  register: public_subnet
 
- name: Subnet public ID
  debug: 
    var: public_subnet['subnet']['id']
 
- name: Create internet gateway    
  ec2_vpc_igw:
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    state: present
    region: "{{ region }}"
  register: igw
 
- name: Internet gateway ID
  debug: 
    var: igw['gateway_id']
 
- name: Set up public subnet route table
  ec2_vpc_route_table:
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    region: "{{ region }}"
    state: present
    tags:
      Name: "{{ aws_network_name }}-public"
    subnets:
      - "{{ private_subnet_1['subnet']['id'] }}"
      - "{{ private_subnet_2['subnet']['id'] }}"
      - "{{ public_subnet['subnet']['id'] }}"
    routes:
      - dest: 0.0.0.0/0
        gateway_id: "{{ igw['gateway_id'] }}"
  register: public_route_table
 
- name: Bastion Security group
  ec2_group:
    name: "{{ aws_network_name }}-bastion"
    state: present
    description: Security group for SSH Bastion to get into the servers
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    region: "{{ region }}"
    rules:
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: 0.0.0.0/0
  register: bastion_sg
 
- name: Bastion Security group ID
  debug: 
    var: bastion_sg['group_id']
 
- name: Web Security group
  ec2_group:
    name: "{{ aws_network_name }}-web"
    state: present
    description: Security group for SSH Bastion to get into the servers
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    region: "{{ region }}"
    rules:
      - proto: tcp
        from_port: 80
        to_port: 80
        cidr_ip: 0.0.0.0/0
      - proto: tcp
        from_port: 22
        to_port: 22
        group_id: "{{ bastion_sg['group_id'] }}"
  register: web_sg
 
- name: Web Security group ID
  debug: 
    var: web_sg['group_id']
 
- name: MySQL Security group
  ec2_group:
    name: "{{ aws_network_name }}-private"
    state: present
    description: Security group for private access
    vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
    region: "{{ region }}"
    rules:
      - proto: tcp
        from_port: 80
        to_port: 80
        cidr_ip: 0.0.0.0/0
      - proto: tcp
        from_port: 22
        to_port: 22
        group_id: "{{ bastion_sg['group_id'] }}"
      - proto: tcp
        from_port: 3306
        to_port: 3306
        group_id: "{{ web_sg['group_id'] }}"
  register: mysql_sg
 
- name: MySQL Security group ID
  debug: 
    var: mysql_sg['group_id']

Building the Bastion

Now there’s a working network in place, the next step is to set up a bastion server to SSH in order to access the RDS instance that will be built in the next stage. It’s bad security practice to make the RDS or Database publicly accessible to the internet. When building the bastion, it will be placed into the the public security group and subnet in order to allow access into the network, because the database will only be accessible internally. The bastion is  created using the default AWS EC2 AMI for the particular region that it will reside. A custom AMI can also be used that has pre-built settings. Finally an SSH key or key pair was created and the site.yml updated with both the AMI and key pair being used for the bastion. When the playbook is executed, the output for the bastion creation will provide the public IP that gets assigned, and the public security group that it is placed into allows for port 22 SSH access.

Building AWS Bastion Tasks

---
- name: "Get Public Subnet ID"
  ec2_vpc_subnet_facts:
    region: "{{ region }}"
    filters:
      "tag:Name": "{{ aws_network_name }}-public"
  register: public_subnet
 
- name: Subnet ID
  debug:
    var: public_subnet['subnets'][0]['id']
 
- name: Get bastion SG ID
  ec2_group_facts:
    region: "{{ region }}"
    filters:
      group-name: "{{ aws_network_name }}-bastion"
  register: bastion_sg
 
- name: Bastion SG ID
  debug:
    var: bastion_sg['security_groups'][0]['group_id']
 
- name: Create Bastion
  ec2:
    region: "{{ region }}"
    key_name: "{{ key_pair }}"
    group_id: "{{ bastion_sg['security_groups'][0]['group_id'] }}"
    instance_type: t2.micro
    image: "{{ ec2_id }}"
    wait: yes
    wait_timeout: 500
    exact_count: 1
    instance_tags:
      Name: "{{ aws_network_name }}-bastion"
      Environment: Dev
    count_tag: 
      Name: "{{ aws_network_name }}-bastion"
      Environment: Dev
    vpc_subnet_id: "{{ public_subnet['subnets'][0]['id'] }}"
    assign_public_ip: yes
  register: bastion_facts
 
- debug: "var=bastion_facts"
 
- name: Capture Bastion public IP
  set_fact: 
    bastion_public_ip: "{{ bastion_facts['instances'][0]['public_ip'] }}"
  when: bastion_facts['instances'] | length > 0
 
- name: Capture Bastion public IP
  set_fact: 
    bastion_public_ip: "{{ bastion_facts['tagged_instances'][0]['public_ip'] }}"
  when: bastion_facts['tagged_instances'] | length > 0
 
- name: Display Public IP
  debug:
    var: bastion_public_ip

Building RDS

Now the network is in place and the bastion is built in order to get logged into the network. It’s now time to create the RDS instance. Before creating the RDS instance a subnet group needs to be configured where the RDS instance will reside, and then a new parameters configuration is needed in order to set up the RDS instance. There are only a couple of simple configurations done in this playbook and this is sufficient to validate that the process is working. But if this was for a customer, it would have far more customizations to meet the customer’s needs.

Building RDS Tasks

---
# tasks file for aws-rds
- name: "Get Subnet ID for markwardt-private1"
  ec2_vpc_subnet_facts:
    region: "{{ region }}"
    filters:
      "tag:Name": "{{ aws_network_name }}-private1"
  register: subnet_private1
 
- name: "Get Subnet ID for {{ aws_network_name }}-private2"
  ec2_vpc_subnet_facts:
    region: "{{ region }}"
    filters:
      "tag:Name": "{{ aws_network_name }}-private2"
  register: subnet_private2
 
- name: Get MySQL SG ID
  ec2_group_facts:
    region: "{{ region }}"
    filters:
      group-name: "{{ aws_network_name }}-private"
  register: private_sg
 
- name: Build RDS Subnet Group
  rds_subnet_group:
    region: "{{ region }}"
    state: present
    name: "{{ aws_network_name }}-subnetgroup"
    description: Subnet Group for {{ aws_network_name }}
    subnets:
      - "{{ subnet_private1['subnets'][0]['id'] }}"
      - "{{ subnet_private2['subnets'][0]['id'] }}"
  register: subnet_group_results
 
- name: Subnet group results
  debug:
    var: subnet_group_results
 
- name: Build MySQL Parameters
  rds_param_group:
    state: present
    name: "{{ aws_network_name }}-parameters"
    description: "{{ aws_network_name }} Parameters"
    engine: "{{ rds_parameter_engine }}"
    immediate: no
    region: "{{ region }}"
    params: "{{ item.param }}={{ item.value }}"
  with_items: "{{ rds_parameters }}"
 
- name: Build RDS Instance
  rds:
    command: create
    instance_name: "{{ aws_network_name }}-rds"
    db_engine: MySQL
    size: "{{ rds_size_gb }}"
    instance_type: "{{ rds_instance_type }}"
    username: "{{ rds_user }}"
    password: "{{ rds_pass }}"
    region: "{{ region }}"
    subnet: "{{ aws_network_name }}-subnetgroup"
    parameter_group: "{{ aws_network_name }}-parameters"
    engine_version: "{{ rds_instance_engine }}"
    vpc_security_groups: "{{ private_sg['security_groups'][0]['group_id'] }}"
    wait: yes
    wait_timeout: 1800
    multi_zone: yes
    tags:
      Environment: "{{ aws_network_name }}"
  register: rds_results
 
- name: RDS rds results
  debug:
    var: rds_results

Cleaning up

When working in the cloud, there are a lot of scenarios where cleaning up your infrastructure is a good idea. Below are some playbooks that almost completely removes all of the instances that were created (parameters, subnet groups), and all of the network components. The RDS snapshots were incapable of being cleaned up successfully. These would have to be automated using another method outside of Ansible modules as the current Ansible modules do not allow for discovery to remove the RDS snapshots.

AWS RDS Cleanup Tasks

- hosts: localhost
  connection: local
  vars:
    region: "us-west-2"
    aws_network_name: dev
  tasks:
    - name: Remove RDS Instance
      rds:
        region: "{{ region }}"
        command: delete
        instance_name: "{{ aws_network_name }}-rds"
        wait: yes
        wait_timeout: 1800
    - name: Remove RDS Subnet Group
      rds_subnet_group:
        region: "{{ region }}"
        state: absent
        name: "{{ aws_network_name }}-subnetgroup"
    - name: Build MySQL Parameters
      rds_param_group:
        region: "{{ region }}"
        state: absent
        name: "{{ aws_network_name }}-parameters"

AWS Bastion Cleanup Tasks

- hosts: localhost
  connection: local
  vars:
    region: "us-west-2"
    aws_network_name: dev
  tasks:
    - name: Get Bastion EC2 ID
      ec2_remote_facts:
        region: "{{ region }}"
        filters:
          "tag:Name": "{{ aws_network_name }}-bastion"
      register: ec2_facts
 
    - name: Destroy Bastion
      ec2:
        instance_ids: "{{ item.id }}"
        state: absent
        region: "{{ region }}"
      with_items: "{{ ec2_facts.instances }}"

AWS Network Cleanup Tasks

- hosts: localhost
  connection: local
  vars:
    region: "us-west-2"
    aws_network_name: dev
    vpc_cidr: 10.4.0.0/16
    public_subnet_cidr: 10.4.0.0/24
    private_subnet_1_cidr: 10.4.1.0/24
    private_subnet_2_cidr: 10.4.2.0/24
  gather_facts: False
  tasks:
    - name: Get VPC ID
      ec2_vpc_net_facts:
        region: "{{ region }}"
        filters:
          "tag:Name": "{{ aws_network_name }}-vpc"
      register: vpc_facts
 
    - name: Destroy SG MySQL
      ec2_group:
        name: "{{ aws_network_name }}-private"
        state: absent
        region: "{{ region }}"
 
    - name: Destroy SG Web
      ec2_group:
        name: "{{ aws_network_name }}-web"
        state: absent
        region: "{{ region }}"
 
    - name: Destroy SG Bastion
      ec2_group:
        name: "{{ aws_network_name }}-bastion"
        state: absent
        region: "{{ region }}"
 
    - name: Remove subnet route table
      ec2_vpc_route_table:
        region: "{{ region }}"
        state: absent
        tags:
          Name: "{{ aws_network_name }}-public"
        vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"   
 
    - name: Remove Public Subnet 
      ec2_vpc_subnet:
        state: absent
        cidr: "{{ public_subnet_cidr }}"
        region: "{{ region }}"
        vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
 
    - name: Remove Private Subnet 1
      ec2_vpc_subnet:
        state: absent
        cidr: "{{ private_subnet_1_cidr }}"
        region: "{{ region }}"
        vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
 
    - name: Remove Private Subnet 2
      ec2_vpc_subnet:
        state: absent
        cidr: "{{ private_subnet_2_cidr }}"
        region: "{{ region }}"
        vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
 
    - name: Create internet gateway    
      ec2_vpc_igw:
        vpc_id: "{{ vpc_facts['vpcs'][0]['id'] }}"
        state: absent
        region: "{{ region }}"
 
    - name: Remove VPC
      ec2_vpc_net:
        name: "{{ aws_network_name }}-vpc"
        state: absent
        cidr_block: "{{ vpc_cidr }}"
        region: "{{ region }}"

Summary

Ansible was very enjoyable and easy to work with. The same goes for using Ansible to manage AWS. Even though there are other tools that can be used such as Terraform (https://www.terraform.io/), Ansible was pretty straightforward and intuitive for creating each of the components, discovering them, and then cleaning them up. It made it nice and simple to search for objects by tag, so it was able to tag the AWS pieces appropriately to allow for easy management of those objects dynamically. In coordination with using Ansible to integrate with AWS to build the network and instances, Ansible can then be used to manage and configure the EC2 operating systems as needed. The Ansible AWS modules make it almost a total package for building and managing an infrastructure in AWS.

email

Interested in working with Kevin? Schedule a tech call.

MySQL Database Consultant

No comments

Leave a Reply

Your email address will not be published. Required fields are marked *