Get the oldest elasticsearch index:

curl 'http://127.0.0.1:9200/_cat/indices' 2>&1 | awk '{print $3}' | grep "logstash_.*" | sort -t- -k2

DO NOT PANIC! Just enjoy it :)

First of all we use curl to get list of indexes from elasticsearch. By using awk with fetch just the 3rd column of the output, 3rd column refers to your index names (be careful to give your index name as there are internal indexes too and we do not want to purge them). grep command will then filter indexes and outputs those that start by logstash_, if yours are different change it. Finally the sort command sorts the result, but it first gets a delimiter by -t. sort -t- will split the column to TWO columns based on dash (-):

If my index name is logstash_data-20180619, it will exports 2 columns one is logstash_data and the other is 20180619. Now we use -k2 in order to sort based on the second column which is the date of the index.

This is how we can get the oldest elastic search index. I use this for maintenance of ES. In case disk space is almost full, I will delete the oldest elasticsearch index. You can even send a SLACK notification using cURL too.

The possibilities are endless.

Happy bashing :)

#linux #bash #curl #grep #sort #es #elasticsearch #split #awk #script

How to convert the below line of string:

data = '15 0 42 50 "some text" "" 4 4 "text"'

into:

[15, 0, 42, 50, 'some text', '', 4, 4, 'text']

As you may have noted you cannot use split as there are texts with spaces in between. For that we can use the below code:

import csv
 import io

 file = io.StringIO(data)  # use io.BytesIO in python 2
 reader = csv.reader(file, delimiter=' ')
 split_data = next(reader)
 parsed_data = [int(x) if x.isdigit() else x for x in split_data]

NOTE: use io.BytesIO instead of io.StringIO in case you are using python 2.

#python #csv #reader #split #stringIO #byteIO