space=$(df -k / | tail -1 | awk '{print $4}')
echo "free disk space is $space"
if [ $space -lt 510000 ]
then
echo $(date) + " - Purge elasticsearch indexes..."
curl -X DELETE "http://localhost:9200/your_index_name_*"
echo ''
else
echo $(date) + " - disk space seems OK"
fi
Put this in a
crontab
and you are good to go.#linux #sysadmin #bash #script #df #elasticsearch #es
tail
command in Linux
is used to see content of a file from the end. It is usually used for checking log files in server. The interesting thing about tail
is that you can use this command to get the last line. So in a bash script if you want to get last row of the below output:root@server:~# ls -l
total 24
-rw-r--r-- 1 root root 291 May 26 05:19 es_queries
-rw-r--r-- 1 root root 1198 Jun 19 10:34 users.json
-rwxr-xr-x 1 root root 272 Jun 19 11:22 monitor_disk_space.sh
-rwxr-xr-x 1 root root 433 Jun 19 10:00 another_script.sh
You would do:
root@server:~# ls -l | tail -1That's why we have used this command in the previous post on
-rwxr-xr-x 1 root root 433 Jun 19 10:00 another_script.sh
df -k /
.#bash #tail #script #ls
Get the oldest elasticsearch index:
DO NOT PANIC! Just enjoy it :)
First of all we use
If my index name is
This is how we can get the oldest elastic search index. I use this for maintenance of
The possibilities are endless.
Happy bashing :)
#linux #bash #curl #grep #sort #es #elasticsearch #split #awk #script
curl 'http://127.0.0.1:9200/_cat/indices' 2>&1 | awk '{print $3}' | grep "logstash_.*" | sort -t- -k2
DO NOT PANIC! Just enjoy it :)
First of all we use
curl
to get list of indexes from elasticsearch
. By using awk
with fetch just the 3rd column of the output, 3rd column refers to your index names (be careful to give your index name as there are internal indexes too and we do not want to purge them). grep
command will then filter indexes and outputs those that start by logstash_
, if yours are different change it. Finally the sort command sorts the result, but it first gets a delimiter by -t
. sort -t-
will split the column to TWO columns based on dash (-):If my index name is
logstash_data-20180619
, it will exports 2 columns one is logstash_data
and the other is 20180619
. Now we use -k2
in order to sort based on the second column which is the date of the index.This is how we can get the oldest elastic search index. I use this for maintenance of
ES
. In case disk space is almost full, I will delete the oldest elasticsearch
index. You can even send a SLACK
notification using cURL
too.The possibilities are endless.
Happy bashing :)
#linux #bash #curl #grep #sort #es #elasticsearch #split #awk #script
Simple bash script to take nightly
Put it in a crontab for nightly backups. First open crotab:
Create a new line (entry) in crontab and paste the below cron task:
#mongodb #backup #cron #cronjob #coderwall #mongodump #bash
MongoDB
backups:#!/bin/sh
DIR=`date +%m%d%y`
DEST=/db_backups/$DIR
mkdir $DEST
mongodump -h <your_database_host> -d <your_database_name> -u <username> -p <password> -o $DEST
NOTE:
db_backups folder shoud already be created by mkdir /db_backups
.Put it in a crontab for nightly backups. First open crotab:
sudo crontab -e
Create a new line (entry) in crontab and paste the below cron task:
45 1 * * * ../../scripts/db_backup.sh
NOTE:
here our script is called db_backup.sh, should you use your own script name here. and make it executable by chmod +x /your/ full_path/scripts/db_backup.sh
#mongodb #backup #cron #cronjob #coderwall #mongodump #bash
In Linux bash scripting you can check commands exit codes and do appropriate jobs accordingly. For that we will use || and &&.
Let's start by a simple echo command:
If for any reason we want to check the exit code of
You can use code block to run multiple commands:
The is another way that you can check exit code and it is
When
#linux #bash #script #scripting #exit_code
Let's start by a simple echo command:
echo "Hello everybody"
If for any reason we want to check the exit code of
echo
command to see if it is successful or not. We can use the code block:echo "Hello everybody" && echo "Phew! We're good." || echo "echo command FAILED!"
You can use code block to run multiple commands:
echo "Hello everybody" && {
echo "Phew! We're good."
touch ME
} || {
echo "echo command FAILED!"
touch YOURSELF
}
NOTE:
exit code 0 means command execution was successful, and exit code 1 means something nasty happened to the previous command.The is another way that you can check exit code and it is
$?
:cp ME YOURSELF
if [ $? = 0 ] ; then
echo "copy seems OK!"
else
echo "Yuck! File could not get copied! :("
fi
When
cp
command is run $?
will keep the exit code of recent command which has been executed.#linux #bash #script #scripting #exit_code
In
For checking directory existence we use
#bash #linux #directory_existence #file_existence
Linux
we have a command called test
, you can check whether a file/directory exists or not and run commands based on the result. For example let's say we want to check if a folder exists and if it does not exist, create the folder.For checking directory existence we use
test -d
and for file existence we use test -f
, so for our example in order to check if the directory exists we use test -d
and in case the folder does not exists we will create it:directory_to_check="/data/mysql"
test -d $directory_to_check || {
echo "$directory_to_check does not exist, creating the folder..." && mkdir -p $directory_to_check || {
echo "$directory_to_check directory could not be created!"
exit 1
}
}
NOTE:
you can read more about exit codes with hashtag #exit_code#bash #linux #directory_existence #file_existence
If you have multiple statements and you want to time the whole statements in a bash script you can use
That's it. In case you want to just
#linux #bash #time
time
command, but you don't need to use inside of the script. If you have an script called my_long_tasks.sh
you need to just:$ time my_long_tasks.sh
real 0m20.894s
user 0m3.664s
sys 0m0.452s
That's it. In case you want to just
time
part of your statements in your bash script then you need to:start=`date +%s`
stuff
end=`date +%s`
runtime=$((end-start))
#linux #bash #time
Did you know you can test bash scripts line by line? Well,
The content of the bash script is:
#bash #sh #shell #scripting #debug #debugging
bash -x
is here to help:$ bash -x your_script.sh
+ a=10
+ echo 10
10
The content of the bash script is:
#!/bin/bash
a=10
echo $a
#bash #sh #shell #scripting #debug #debugging
Array and loop in bash script
To define an array you can use a structure like below, be careful that we don't use comma in between:
dbs=( 'test1' 'test2' 'test3' )
Now to loop over the array elements use
for
:for your_db in ${dbs[@]}
do
echo $your_db
done
This is it!
#bash #scripting #for #loop #array
How to prepend a string to all file names in a directory in a bash script?
The above one-line will loop over all files in current directory with
So for example a file with name
#python #bash #script #prepend #move #rename #for
for f in *.py; do mv "$f" "old-$f"; done
The above one-line will loop over all files in current directory with
.py
extension and prepend old-
into the files.So for example a file with name
main.py
will be renamed to old-main.py
#python #bash #script #prepend #move #rename #for
I sometimes forgot to pull data from git before start working on a project. To minimize the headache of merge conflict or having to
If you have similar issues and you're solving it in a different manner, I'd be happy to hear about it. :)
#linux #mac #bash #script #git #cron #crontab #cronjob
reset head
and stash
your data you can set a cronjob to run every 10 minute or so. And inside of your bash script CD
into eaach of your folders and issue git pull
command.If you have similar issues and you're solving it in a different manner, I'd be happy to hear about it. :)
#linux #mac #bash #script #git #cron #crontab #cronjob
How to zero-pad a number in
In order to zero-pad a number you need to use do like below:
Here I have used
As simple as that.
#bash #printf #zeropad #zero-pad #zeropadding
bash
?printf
is here to help :)In order to zero-pad a number you need to use do like below:
your_number_var=1
output=$(printf "%02d" $your_number_var)
echo $output # 01
Here I have used
%02d
. the number 2 refers to numbers of padding and d
refers to digit
. So to zero-pad to 5 you can use %05d
.As simple as that.
#bash #printf #zeropad #zero-pad #zeropadding
If you forget to pull your projects from git in a regular interval and many users working on the same projects, then there is a solution for you!
Create a bash script file as follow and make it executable by
Now as a final step, put it in your crontab:
#linux #git #pull #cronjob #crontab #cron #bash
Create a bash script file as follow and make it executable by
chmod +x puller.sh
:puller.sh
file content:#!/bin/bash
echo 'Iterating over folders...'
for dir in *
do
test -d "$dir" && {
cd ${dir}
echo "git pull $dir"
git pull
cd ".."
} || {
echo "------> $dir is not a directory <-------"
}
done
NOTE:
this file should reside in your folder's project root. In my case it is in /Your/Projects/Folder
.Now as a final step, put it in your crontab:
10 * * * * bash -c "cd /Your/Projects/Folder; bash puller.sh >> /var/log/git_pull_output.log"
#linux #git #pull #cronjob #crontab #cron #bash
We have talked before about how to get current month using the below line of code:
It prints out 01, 02, etc.
As per the
So you can remove leading zero by hyphen as below:
It prints out 1, 2, etc.
#linux #bash #date
echo $(date +%m)
It prints out 01, 02, etc.
As per the
GNU date manpage
:By default, date pads numeric fields with zeroes. The following
optional flags may follow '%':
- (hyphen) do not pad the field
So you can remove leading zero by hyphen as below:
echo $(date +%-m)
It prints out 1, 2, etc.
#linux #bash #date
In order to enable bash completion in Kubernetes you can usee the below command in linux bash:
Now to test this enter the below command and you should see the completion:
It should be expanded to
#linux #bash #shell #kubernetes #kubectl
source <(kubectl completion bash)
Now to test this enter the below command and you should see the completion:
kubectl cl<TAB>
It should be expanded to
kubectl cluster-info
.#linux #bash #shell #kubernetes #kubectl