In today’s world, we are talking about programs or applications getting more intelligent – Self driving cars, M2M IoT communication, Virtual assistant applications like Google bots etc. At the crux of the intelligence lies automation, where applications are doing some of the mundane activities done by a human being with minimal or no human interaction. Automation results in efficient usage of resources – Time, Human resources, Cost etc. As such the topic automation is a very huge and a topic of study by itself. In this blog post, we are going to consider Linux shell scripting as an example and discuss few examples of automation using shell script.
As discussed in the previous blog, shell script is an interpreted language. When a program or set of instructions is written in any language to perform a specific task, it has to be either compiled or interpreted. When a code is compiled using compiler (ex: C programming), language used to write code is said to be compiled language and if code is interpreted line by line using interpreter, the language (ex: Shell scripting) used in writing instructions is said to be interpreted language.
A brief intro to Linux Shell Script :
Shell script is a computer program run by Linux shell. It is a command line interpreter. Operations performed by shell scripts are file manipulation, program execution and printing text. There are various types of shells available for user. They are:
- The Bourne shell (sh)
- The Bourne Again Shell (bash)
- The C shell (csh and tcsh)
- The korn shell (ksh)
If an user wants to perform a task, it can be achieved by writing a script. And if the task has to be repeated daily or in a specific condition / regular interval of time, it can be automated by some facilities provided by the Linux Shell. Linux Shell scripting is very powerful when it comes to command-line automation, as it combines programming constructs (ex: for loop) with linux commands (ex: du command for disk usage) that makes it a natural choice for automation.
Cron :
At the heart of Linux automation lies a powerful utility / application called cron. Cron also called as job scheduler, is a system process which plays a vital role in automation by performing tasks automatically according to a set schedule. The schedule which is set, is said to be crontab. Crontab stands for “crontable” which is the name of the program used in editing the schedule. As an analogy, Cron can be compared with alarm application that is commonly available in mobile phones and crontab can be compared to the table where schedule for the alarm is configured.
Crontab command creates a file called crontab file, that contains instructions and commands for the “cron daemon” to execute. Cron daemon is a long running process, which executes the commands according to the schedule. Similar to Alarm application, jobs can be configured in Crontab for various time intervals. Few examples include:
- Schedule a job for specific time of the day
- Schedule a job for every minute
- Schedule a job for more than one time
- Schedule a job for certain range of time and many more
Thus, with the help of Cron automation of tasks can be achieved. To trigger an automated ask, the crontab file need to be edited (using ‘crontab -e’ command) and time line of the task and application link need to be provided.
For example, in the image shown below a script / application (by name file.sh) is configured to be triggered at 5:15 PM every day. In the similar lines multiple options are available, you may want to refer crontab man page for more details.
Now that crontab understanding and configuration is done, appropriate examples for automation need to be discussed. In this blog post, three examples are taken to demonstrate the automation use-case.
Example 1: Birthday Reminder
Sending automated birthday wishes with personalized message can be one simple use-case to be considered. In order to do this task automatically, instead of individuals remembering them, a Comma Separated File (CSV file) to be created with Name, Email and Date of Birth information. An example CSV file can be as follows:
A Shell script can be written to read all the information from CSV file and compare their month and date information with the system’s current date and month. In case of a match, it indicates that particular individual’s birthday to whom an email can be send with a personalized message. The script code is provided below.
#!/bin/bash
#Script to send Birthday Wishes through email from csv file
TO="to@address.com"
FROM="from@address.com"
#extract day and month from date command
TodayDay=$(date | cut -d" " -f3 )
TodayMonth=$(date | cut -d" " -f2 )
count=$(cat BirthDay.csv | wc -l)
#extract each field from csv
for (( i = 2; i <= count; i++ ));do
Name=$(cut -d, -f1 <BirthDay.csv | sed -n "$i"p)
DobMonth=$(cut -d, -f2 <BirthDay.csv | cut -d- -f2 |sed -n "$i"p)
DobDay=$(cut -d, -f2 <BirthDay.csv | cut -d- -f1 |sed -n "$i"p)
Email=$(cut -d, -f3 <BirthDay.csv | sed -n "$i"p)
if [[ "$todayDay" -eq "$DobDay" ]] && [[ "$TodayMonth" = "$DobMonth" ]];then
echo "happy birthday $Name" | mail -s Subject:Birth_Day_Wishes ${FROM} ${TO}
fi
doneAssume the name of the script file as birthday.sh, which can be configured into the crontab in order to do automated check on a daily basis (say at 9:30 AM) and send the email in a personalized way.
A sample crontab in that case would look as follows:
Understanding system problems and taking preventive actions is one of the key requirement of a system administrator. For example, let us take a use-case of an organization with more than 1000 employees. One of the common system problem is about hard disk getting full, which may result in larger system related problems. What if the system administrator gets an automated alert if any of the employee’s hard disk capacity goes more than 90% occupation. Upon getting such alert, the system administrator can inspect the system and advise the employee by cleaning up / freeing up some space in the hard-disk.
To estimate current disk usage df and du commands are used. Few examples of these command usage is given as follows. Combing this command with shell script and crontab, the pro-active alert system can be set up.
Disk Usage :
Disk Free :
#!/bin/bash
#we are importing the code from the ip_address_mail.sh script in this program
# -using source keyword
#define from and to address : from can be also username@hostname of the computer
TO="to@address.com"
FROM="from@address.com"
#Define some format on text file
echo "filesystem having less 10% free space">text.txt
echo "">>text.txt
echo "filesystem freespace">>text.txt
count=`df --output=size,avail | wc -l`
for (( i = 2; i <= $count; i++ ))
do
used=`df --output=used | sed -n "$i"p`
available=`df --output=avail | sed -n "$i"p`
fileSys=`df --output=source --output=avail | sed -n "$i"p`
final=$(($(($(($available*100))))/$(($available$used))))
#file system less than 10% redirecting to text.txt
if [ $final -le 10 ]
then
echo "$fileSys">>text.txt
fi
done
#contents of text is piped to mail command and defined the subject
cat "text.txt" | mail -s Subject:low_free_space ${FROM} ${TO}
cat text.txt
echo the mail is sentAssume the name of the script file as free.sh, which can be configured into the crontab in order to do automated check on every friday of the week (say at 5:15 PM) and send the email in a personalized way. A sample crontab in that case would look as follows:
Consider we need to find the disk space available in an external device (ex: Raspberry Pi) connected with our machine. The disk space in such devices may be very less, we have to first obtain its IP address to initiate the communication and then use the script mentioned in above example to find disk space. The script below detects the IP address change and initiates appropriate actions based on that, again this can be automated by placing the script in the crontab as called out in previous examples.
#!/bin/bash
#This script need prerequisites as curl and postfix
#(sudo apt-get install postfix curl -y)
#Script to report public IP address change in Raspberry pi
#define from and to address
TO="abc@gmail.com"
FROM="xyz@gmail.com"
#The file that contains the current public IP
#create a test folder in your home directory and touch file ipaddressfile
#Get the current public IP from http://ifconfig.me/ip
#Check file for previous IP address
IP_FILE="$HOME/ipaddressfile"
if [ -f "$IP_FILE" ]; then
KNOWN_IP=$(cat "$IP_FILE")
else
touch "$HOME/ipaddressfile"
fi
CURRENT_IP=$(curl http://ifconfig.me/ip)
#See if the IP has changed
#Subject: is piped to mail command
if [ "$CURRENT_IP" != "$KNOWN_IP" ]; then
echo "$CURRENT_IP" > "$IP_FILE"
echo "Subject: The IP Address at home has changed
The IP address at home has been changed to $CURRENT_IP"| mail -s ${FROM} ${TO}
else
echo "Subject: The IP Address at home is the same
The IP address at home stayed the same $CURRENT_IP"| sendmail -f ${FROM} ${TO}
fi
cat $HOME/ipaddressfile
echo the mail is sentAs called out in three examples above, automation using shell script can be achieved for a normal user (birthday reminder), system administrator (system level alerts) or developer (IP address change). Again, scope of automation is very large considering all possible use-cases is not possible in a single blog post. Combined with scripting examples provided above, there are multiple options provided by Open Source. One example bring “vagrant” software using which development environment of a machine can be automatically configured thereby reducing the time taken to configure each and every machine.
Conclusion :
Linux shell provides multiple facilities to automate tasks. By combining with Linux Kernel facilities like Cron daemon, it gives power to the developer. Along with script there are many tools available and getting built which will help in automation like AutoIT, Selenium, Robotium, TestComplete, Watir, Visual Studio Test Professional, QTP(UFT), SoapUI etc.
Thus because of automation, time spent in doing tasks manually could be utilized fruitfully. Afterall, who want to do the same boring task again-and-again?
Happy Learning!!!
