Fixing ‘plugin:elasticsearch [document_already_exists_exception] [config][4.5.1]: document already exists’

Substitute in the version ‘4.5.1’ with the version you are upgrading to. So far I’ve seen it since Kibana 4.1.x to 4.5.1. It seem that if you upgrade Kibana, there is a timing bug in how Kibana note its current version. You will get lots of these errors in Kibana logs: log [08:08:30.649] [error][status][plugin:elasticsearch] Status…

continue reading
No Comments

HOW TO add search-guard-ssl to Elasticsearch

If you have a need to encrypt communication between your Elasticsearch nodes, but do not (yet) need the complicated ACL provided from either Shield (Elastic commercial product) or Search-Guard (open source), then you can use Search-Guard-SSL (open source). I am going to show you how to add Search-Guard-SSL (SG-SSL for short) to Elasticsearch. There are…

continue reading
1 Comment

Kibana 4 with tribe node MasterNotDiscoveredException

I use tribe nodes quite a lot at $work. It’s how we federate disparate ELK clusters and able to search across them. There are many reasons to have distinct ELK clusters in each data center and/or region. Some of these are: 1. Elasticsearch does not work well when there is network latencies, which is guaranteed…

continue reading
No Comments

Elasticsearch util to copy/reindex index(es)

Elasticsearch (and the entire ELK stack) is pretty useful open source piece of software for analyzing large datasets.   I manage a fairly large ELK infrastructure at work — around 90+ ES clusters, 300+ TB of data.   One of things I’ve found myself having to do is copying and/or reindexing one or more index(es).…

continue reading
No Comments

Monitoring Postfix and Dovecot logs in ELK

I’ve been using pflogsumm for the longest time to monitor my postfix logs.   When I used to manage hundreds of domains and many more mailing lists, it was important to keep an eye on my mail servers. These days, it is just my own personal mail server for my dozens of domains.   I…

continue reading
9 Comments

ELK Operational Tips

I’ve been running ELK clusters for over a year now, and want to share tips and tricks that I’ve found to be useful. Feel free to post questions and corrections. I’ll try to answer and update when possible. Elasticsearch Split brained – this is when you have more than one node in your cluster becoming…

continue reading
No Comments

Online debugging/tutorial tools

Online Tools In the course of my career, I’ve jumped from one platform, OS, programming, scripting languages, etc. to another.   I’ve found that what make it easier to transition into a new “whatever” is the quality of tutorial and debugging tools available to me. Besides local tools, there are some awesome web sites that…

continue reading
No Comments
%d bloggers like this: