Category: ELK

Monitoring sendgrid with Elasticsearch

If you are using sendgrid as a service for your outbound email, you would want to monitor and be able to answer questions such as: how much email are you sending status of sent email – success, bounced, delayed, etc. trends etc. We get questions all the time from $WORK customer support folks on whether…

continue reading
No Comments

Courier Fetch Error: unhandled courier request error: Authorization Exception in Chrome/Safari on Kibana 4.5.0

Getting this error in your Kibana? You need to increase your max header size as default netty is only 8KB.   You can change the value in your elasticsearch.yml file. Add the following line (or uncomment it if it is already there). http.max_header_size: 32kb  

continue reading
No Comments

Fixing ‘plugin:elasticsearch [document_already_exists_exception] [config][4.5.1]: document already exists’

Substitute in the version ‘4.5.1’ with the version you are upgrading to. So far I’ve seen it since Kibana 4.1.x to 4.5.1. It seem that if you upgrade Kibana, there is a timing bug in how Kibana note its current version. You will get lots of these errors in Kibana logs: log [08:08:30.649] [error][status][plugin:elasticsearch] Status…

continue reading
No Comments

HOW TO add search-guard-ssl to Elasticsearch

If you have a need to encrypt communication between your Elasticsearch nodes, but do not (yet) need the complicated ACL provided from either Shield (Elastic commercial product) or Search-Guard (open source), then you can use Search-Guard-SSL (open source). I am going to show you how to add Search-Guard-SSL (SG-SSL for short) to Elasticsearch. There are…

continue reading
1 Comment

Kibana 4 with tribe node MasterNotDiscoveredException

I use tribe nodes quite a lot at $work. It’s how we federate disparate ELK clusters and able to search across them. There are many reasons to have distinct ELK clusters in each data center and/or region. Some of these are: 1. Elasticsearch does not work well when there is network latencies, which is guaranteed…

continue reading
No Comments

Elasticsearch util to copy/reindex index(es)

Elasticsearch (and the entire ELK stack) is pretty useful open source piece of software for analyzing large datasets.   I manage a fairly large ELK infrastructure at work — around 90+ ES clusters, 300+ TB of data.   One of things I’ve found myself having to do is copying and/or reindexing one or more index(es).…

continue reading
No Comments

Monitoring Postfix and Dovecot logs in ELK

I’ve been using pflogsumm for the longest time to monitor my postfix logs.   When I used to manage hundreds of domains and many more mailing lists, it was important to keep an eye on my mail servers. These days, it is just my own personal mail server for my dozens of domains.   I…

continue reading
9 Comments
%d bloggers like this: