Run newest elasticsearch image on linux using docker:

run -d -p 9200:9200 -v /srv/esdata:/usr/share/elasticsearch/data -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/     elasticsearch/elasticsearch:6.2.4

#docker #es #elasticsearch

Elasticsearch gives below error:

Config: Error 403 Forbidden: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];: [cluster_block_exception] blocked by:   [FORBIDDEN/12/index read-only / allow delete (api)];

This error may happen when server storage is totally full and elasticsearch puts your indexes in read only mode. If you have enough
space now and are sure there is no other matter for elasticsearch and it behaves normally, remove read only mode from index block:

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/.monitoring-*/_settings -d '{"index.blocks.                       read_only_allow_delete": null}'

#elasticsearch #read_only #index #cluster_block_exception

Delete elasticsearch indexes older than 1 month:

#!/bin/bash

 last_month=`date +%Y%m%d --date '1 month ago'`
 old_es_index="faxplus_*-$last_month"
 echo "Deleting ES indexes $old_es_index..."
 curl -X DELETE 'http://localhost:9200/myindex_*-20180520'
 echo ''

NOTE: asterisk in curl command will be anything in between of myindex_ and -20180520. For example myindex_module1-20180520.

#linux #sysadmin #bash #script #es #elasticsearch #DELETE #purge

Apply new basic license on Kibana:

You need to download the license first:
- https://register.elastic.co/xpack_register


The license is a json file that can be applied by cURL, first go to your server where elasticsearch is running and then:

curl -XPUT 'http://172.16.133.102:9200/_xpack/license' -H "Content-Type: application/json" -d @license.json

NOTE: license.json is the file that should be present beside from where you are issuing the cURL command.

#kibana #curl #license #elasticsearch

space=$(df -k / | tail -1 | awk '{print $4}')
 echo "free disk space is $space"

 if [ $space -lt 510000 ]
 then
   echo $(date) + " - Purge elasticsearch indexes..."
   curl -X DELETE "http://localhost:9200/your_index_name_*"
   echo ''
 else
   echo $(date) + " - disk space seems OK"
 fi

Put this in a crontab and you are good to go.

#linux #sysadmin #bash #script #df #elasticsearch #es

Get the oldest elasticsearch index:

curl 'http://127.0.0.1:9200/_cat/indices' 2>&1 | awk '{print $3}' | grep "logstash_.*" | sort -t- -k2

DO NOT PANIC! Just enjoy it :)

First of all we use curl to get list of indexes from elasticsearch. By using awk with fetch just the 3rd column of the output, 3rd column refers to your index names (be careful to give your index name as there are internal indexes too and we do not want to purge them). grep command will then filter indexes and outputs those that start by logstash_, if yours are different change it. Finally the sort command sorts the result, but it first gets a delimiter by -t. sort -t- will split the column to TWO columns based on dash (-):

If my index name is logstash_data-20180619, it will exports 2 columns one is logstash_data and the other is 20180619. Now we use -k2 in order to sort based on the second column which is the date of the index.

This is how we can get the oldest elastic search index. I use this for maintenance of ES. In case disk space is almost full, I will delete the oldest elasticsearch index. You can even send a SLACK notification using cURL too.

The possibilities are endless.

Happy bashing :)

#linux #bash #curl #grep #sort #es #elasticsearch #split #awk #script

https://medium.com/@xoor/indexing-mongodb-with-elasticsearch-2c428b676343

#mongodb #elastic #elasticsearch #indexing #fulltext #fulltext_search #search #ngram #medium