Simple steps to perform opatch maintenance with Ansible

Like the Universe, IT growth seems to be infinite, we always have more environments, more servers, more users, more disk usage, more databases to manage and it won't stop. In fact, we are pretty sure that this expansion is going to be faster and faster. We then have to adapt to this new, mutating IT environment being more productive in order to manage more and more targets in less time. How to achieve this goal? Like human beings have always done from the early days - by using tools and by making better tools with the tools we have.
1/ The Ansible Tool
1.1/ A Word on Ansible
Ansible is an open source IT automation tool that was launched in early 2013 and bought by Red Hat in 2015. The most recent 2.3 version was released few days ago.1.2/ Why Ansible?
Other the automation tools are professed to be easy, fast, able to manage thousands of thousands of targets, etc... so why Ansible instead of Puppet or Chef? For me, it'd because Ansible is agentless and does everything through standard SSH (or Paramiko which is a Python SSH implementation). Indeed, 'no agent' really means easy to deploy, no agent to maintain (!), and it is very secure since it uses SSH. I am accustomed to working with companies that have tough security processes and challenging processes for any kind of installations. Be sure that it is easier to quickly deploy everything with these features:- Is it secure? Yes, it goes through SSH.
- Anything to install on the targets? No.
- Do you need root access? No, as long as what I need to do is doable with no root privilege.
- Can it go through sudo? Yes, no worries.
- What do you need then? An SSH key deployed on the targets (which also means that it is very easy to unsetup, you just have to remove that SSH key from the target)
2/ Manage opatch with Ansible
To illustrate how quick and easy it is to use A nsible, I will demonstrate how to update opatch with A nsible. opatch is a very good candidate for Ansible as it needs to be frequently updated, exists in every Oracle home and also needs to be current every time you apply a patch (and for those who read my previous blogs, you know that I like to update opatch :))2.1/ Install Ansible
The best way to install Ansible is to first refer to the official installation documentation . There you will find the specific commands for your favorite platform (note that Ansible is not designed for Windows).2.2/ Configure Ansible
To start, Ansible has to know the hosts you want to manage in a "host" file like:oracle@control:~/work$ cat hosts_dev
[loadbalancer]
lb01
[database]
db01
db02 ansible_host=192.168.135.101
oracle@control:~/work$
We can split the hosts by group like
[loadbalancer],
[database] to have various hosts group. It is also possible that the host you are running
Ansible on cannot resolve a host. We can then use the
ansible_host parameter to specify the IP for it like I did for the
db02 server. In fact,
ansible_host defines the host
Ansible will connect to and the name at the start of the line is an alias used if
ansible_host is not defined Note that I named the hosts file "hosts_dev" in my example. This was done so I would not use the default
ansible hosts file which make it more modular. We then have to tell Ansible that we want to use this file instead of the default file in the
ansible.cfg configuration file.
oracle@control:~/work$ cat ansible.cfg
[defaults]
inventory=./hosts_dev
oracle@control:~/work$
Please remember that Ansible uses SSH connectivity so you'll need to exchange the SSH key of your "control" server to your targets. More extensive documentation on the subject can be found online. Here is an example with
ssh-copy-id (if you don't know the target user password, conduct a Google search for
authorized_keys and you will find how to exchange an SSH key when you don't know the target user password):
oracle@control:~$ ssh-keygen # This will generate your SSH keys
... press ENTER at all prompts) ...
oracle@control:~$ ssh-copy-id oracle@db01
...
Are you sure you want to continue connecting (yes/no)? yes
...
oracle@db01's password: # You will be prompted for the target password once
...
Now try logging into the machine, with: "ssh 'oracle@db01'"
and check to make sure that only the key(s) you wanted were added.
oracle@control:~$ ssh ansible@db01 # Try to connect now
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-112-generic x86_64)
Last login: Thu Apr 20 02:17:24 2017 from control
oracle@db01:~$ # We are now connected with no password
2.3/ A First Playbook
A playbook is a collection of Ansible commands that are used to orchestrate what you want to do. Ansible uses the YAML language (please have a look at the official YAML website) for this purpose. Let's start with a first easy playbook that checks if the /etc/oratab file exists on my [database] hosts:oracle@control:~/work$ cat upgrade_opatch.yml
---
- hosts: database # Specify only the hosts contained in the [database] group
tasks:
- name: Check if /etc/oratab exists # A name for the task
stat: # I will use the stat module to check if /etc/oratab exists
path: /etc/oratab # The file or directory I want to check the presence
register: oratab # Put the return code in a variable named "oratab"
- debug: # A debug task to show an error message if oratab does not exist
msg: "/etc/oratab does not exists" # The debug message
when: oratab.stat.exists == false # The message is printed only when the /etc/oratab file does not exist
oracle@control:~/work$
Let's run it now (we use
ansible-playbook to run a playbook):
oracle@control:~/work$ ansible-playbook upgrade_opatch.yml
PLAY [database] ***************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [db02]
ok: [db01]
TASK [Check if /etc/oratab exists] ********************************************************************************************************************************************************************************
ok: [db02]
ok: [db01]
TASK [debug] ******************************************************************************************************************************************************************************************************
skipping: [db01]
ok: [db02] => {
"changed": false,
"msg": "/etc/oratab does not exists"
}
PLAY RECAP ********************************************************************************************************************************************************************************************************
db01 : ok=2 changed=0 unreachable=0 failed=0
db02 : ok=3 changed=0 unreachable=0 failed=0
oracle@control:~/work$
Since I removed
/etc/oratab from
db02 on purpose, I received the "/etc/oratab does not exists" error message (as expected). Before going further, let's add a test to see if unzip exists (we'll need unzip to unzip the opatch zipfile). Put the db02's oratab file back where it should be and run the playbook again:
oracle@control:~/work$ cat upgrade_opatch.yml
---
- hosts: database
tasks:
- name: Check if /etc/oratab exists
stat:
path: /etc/oratab
register: oratab
- debug:
msg: "/etc/oratab does not exists"
when: oratab.stat.exists == false
- name: Check if unzip exists (if not we wont be able to unzip the opatch zipfile)
shell: "command -v unzip"
register: unzip_exists
- debug:
msg: "unzip cannot be found"
when: unzip_exists == false
oracle@control:~/work$ ansible-playbook upgrade_opatch.yml
PLAY [database] ***************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [db02]
ok: [db01]
TASK [Check if /etc/oratab exists] ********************************************************************************************************************************************************************************
ok: [db01]
ok: [db02]
TASK [debug] ******************************************************************************************************************************************************************************************************
skipping: [db01]
skipping: [db02]
TASK [Check if unzip exists (if not we wont be able to unzip the opatch zipfile)] *********************************************************************************************************************************
changed: [db02]
changed: [db01]
TASK [debug] ******************************************************************************************************************************************************************************************************
skipping: [db01]
skipping: [db02]
PLAY RECAP ********************************************************************************************************************************************************************************************************
db01 : ok=3 changed=1 unreachable=0 failed=0
db02 : ok=3 changed=1 unreachable=0 failed=0
oracle@control:~/work$
Please note that I used the
shell built-in module to test if unzip is present or not.
2.4/ Upgrade opatch
To upgrade opatch, we need to copy the zipfile to the target Oracle home and then unzip it -- easy and straightforward. Let's ask Ansible to do it for us. First, let's use the copy module to copy the opatch zipfile to the target Oracle home:- name: Copy the opatch zipfile to the target oracle home
copy:
src: p6880880_112000_Linux-x86-64.zip
dest: /u01/oracle/11204
Unzip the zipfile in the target Oracle home (I use the
shell module to unzip instead of the unarchive module on purpose. This will trigger a warning during the playbook execution, but I am not a big fan of the
unarchive module... we could discuss that later on):
- name: Upgrade opatch
shell: unzip -o /u01/oracle/11204/p6880880_112000_Linux-x86-64.zip -d /u01/oracle/11204
register: unzip
failed_when: unzip.rc != 0
Let's cleanup the zipfile we copied earlier using the
file module (note that this is the keyword
state: absent which will remove the file), we do not want to leave any leftovers:
- name: Cleanup the zipfile from the target home
file:
name: /u01/oracle/11204/p6880880_112000_Linux-x86-64.zip
state: absent
Now review the whole playbook:
oracle@control:~/work$ cat upgrade_opatch.yml
---
- hosts: database
tasks:
- name: Check if /etc/oratab exists
stat:
path: /etc/oratab
register: oratab
- debug:
msg: "/etc/oratab does not exists"
when: oratab.stat.exists == false
- name: Check if unzip exists (if not we wont be able to unzip the opatch zipfile)
shell: "command -v unzip"
register: unzip_exists
- debug:
msg: "unzip cannot be found"
when: unzip_exists == false
- name: Copy the opatch zipfile to the target oracle home
copy:
src: p6880880_112000_Linux-x86-64.zip
dest: /u01/oracle/11204
- name: Upgrade opatch
shell: unzip -o /u01/oracle/11204/p6880880_112000_Linux-x86-64.zip -d /u01/oracle/11204
register: unzip
failed_when: unzip.rc != 0
- name: Cleanup the zipfile from the target home
file:
name: /u01/oracle/11204/p6880880_112000_Linux-x86-64.zip
state: absent
oracle@control:~/work$
and execute it:
oracle@control:~/work$ ansible-playbook upgrade_opatch.yml
PLAY [database] ***************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [db02]
ok: [db01]
TASK [Check if /etc/oratab exists] ********************************************************************************************************************************************************************************
ok: [db01]
ok: [db02]
TASK [debug] ******************************************************************************************************************************************************************************************************
skipping: [db01]
skipping: [db02]
TASK [Check if unzip exists (if not we wont be able to unzip the opatch zipfile)] *********************************************************************************************************************************
changed: [db02]
changed: [db01]
TASK [debug] ******************************************************************************************************************************************************************************************************
skipping: [db01]
skipping: [db02]
TASK [Copy the opatch zipfile to the target oracle home] **********************************************************************************************************************************************************
changed: [db01]
changed: [db02]
TASK [Upgrade opatch] *********************************************************************************************************************************************************************************************
[WARNING]: Consider using unarchive module rather than running unzip
changed: [db01]
changed: [db02]
TASK [Cleanup the zipfile from the target home] *******************************************************************************************************************************************************************
changed: [db02]
changed: [db01]
PLAY RECAP ********************************************************************************************************************************************************************************************************
db01 : ok=6 changed=4 unreachable=0 failed=0
db02 : ok=6 changed=4 unreachable=0 failed=0
oracle@control:~/work$
We now have a playbook that can update all your opatches in a blink! Please note that this example is a very basic one since this is to give an overview on how to manage opatch with
Ansible. Many features could be implemented here (and are implemented in the code we use here at Pythian) like:
- Check the list of Oracle homes on each server -- there are often many.
- Check the version of each Oracle home's opatch.
- Manager different opatch versions : 11, 12 and 13.
- Use the Ansible roles to make the code more modular and reusable.
- Upgrade opatch only if it needs to and more...