Monitoring insights

Even when you continuously monitor your Elasticsearch cluster through a tool like Datadog, you may want to quickly ascertain what's happening right now to answer questions like:

What's my cluster's health?
What's the current CPU load, free RAM, and other OS metrics?
What's my largest index and how much HDD space does it consume?

To answer questions like these, it's safe to assume that you'd be already inside of the (linux) terminal, performing commands like du, ps, etc. If you look closely, all these commands respond in a table-like format. But remember that Elasticsearch is a JSON-in, JSON-out interface.

Now, JSON is great… for computers. Even if it’s pretty-printed, trying to find relationships in the data is tedious. Human eyes, especially when looking at a terminal, need compact and aligned text. The compact and aligned text (CAT) APIs aim to meet this need.

Running GET _cat lists all the available commands:

=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
...

Appending ?help to a particular command works similarly to the man command on linux:

GET _cat/allocation?help
shards       | s              | number of shards on node      
disk.indices | di,diskIndices | disk used by ES indices       
disk.used    | du,diskUsed    | disk used (total, not just ES)
disk.avail   | da,diskAvail   | disk available                
disk.total   | dt,diskTotal   | total capacity of all volumes 
disk.percent | dp,diskPercent | percent disk used             
host         | h              | host of node                  
ip           |                | ip of node                    
node         | n              | name of node

Regular inspections

Here are some of the commands I run on a regular basis to get a glimpse of what's going on.

Cluster health

GET _cat/health?v&h=timestamp,cluster,status

timestamp cluster       status
15:13:15  elasticsearch yellow