Detect and Notify Linux Server Restart

When Oracle support performs maintenance for Exadata Cloud, communication can sometimes be a bit sparse. Typically, support restarts the host during these windows. Since many applications are not natively RAC-aware, the application team needs a reliable way to know that a host has bounced so they can manually restart their services.

Identifying the Maintenance Gap in Exadata Cloud

The primary challenge is that when the database server goes down and comes back up, the application layer may remain in a disconnected state. To bridge this gap, you need an automated mechanism that fires as soon as the OS is back online.

Leveraging Cron's @reboot Feature

To help with this situation, cron offers a specific option to detect a server reboot: @reboot. By using this in the crontab, you can ensure a shell script is executed immediately upon the server starting up, running under the context of the oracle user.

Crontab Configuration:

# crontab -l @reboot su oracle -c '/home/oracle/scripts/host_restart_alert.sh' > /tmp/host_restart_alert.out 2>&1 

Scripting the Restart Notification

The shell script host_restart_alert.sh is designed to wait for the system to settle before sending an alert to the designated recipient.

host_restart_alert.sh:

#!/bin/bash -x  # Script is being called from root crontab  # uptime reports minutely and needs to sleep for at least 1m after host restart  sleep 63   EMAILMESSAGE="$(hostname) was restarted `uptime -p | awk -F'up' '{print $2}'` ago at `uptime -s`"  echo $EMAILMESSAGE | mailx -s "$HOSTNAME was restarted" email@gmail.com exit  

Example Output and Compatibility

Here is an example of what the generated EMAILMESSAGE will look like once it hits your inbox:

oracle-12201-vagrant was restarted 2 hours, 47 minutes ago at 2021-02-16 14:50:02

I have tested these scripts on Oracle Linux Server release 7.8 and 7.9. Hopefully, this is a useful tip to help you detect and notify your team of a server restart without waiting for a manual check.

Please leave any questions or thoughts in the comments.

Oracle Database Consulting Services

Ready to optimize your Oracle Database for the future?