Using the command line to check redis health

In the forth entry of our series on redis we’ll have a look at its CLI tools. The central question for today: How can we check that redis is working as expected using the command line?

redis-CLI basics

Using redis-cli to check database health sounds easy and, in fact, it is: A simple redis-cli PING on the Redis host returns a PONG. This does not look very reliable, but it is. A redis that returns a PONG is fully functional and works as expected.

To go a bit more into detail: it means that the dataset is completely loaded, and Redis is ready for connections. If something does not work fine, it will display error messages such as “(error) LOADING Redis is loading the dataset in memory”. This will return while loading the rdb/aof file or waiting for replication to be completed.

By using the redis-CLI you can easily use the info command and sub-commands to get some more stats and information.

List of commands

redis-cli info Complete info about the redis-instance
redis-cli info server Server Information, cersions, configs, binary
redis-cli info clients Connected clients
redis-cli info memory Statistics about memory usage and limits
redis-cli info persistence Info about RDB and AOF
redis-cli info stats Connection, network, keyspace statistics
redis-cli info replication Replication settings and status
redis-cli info cpu CPU utilization
redis-cli info cluster Cluster settings and status
redis-cli info keyspace Display dbs and number of keys

To repeat commands you can easily use two options for monitoring: redis-cli -r 5 -i 2 <command> will execute the <command> five times with an interval of two seconds.

With these commands you can easily integrate redis in your monitoring. Just add the redis-CLI commands as custom parameters, e.g. to Zabbix or Nagios. You can also use prometheus exporters to get metrics out of your redis.

Integrated latency monitoring, reporting, slow-logs

Maybe some of you ask yourselves why we need monitoring in the first place, as redis is so incredibly fast. The answer is: you’re right, redis is designed to serve a large amount of queries per second per instance. But unfortunately in most applications there is a strict requirement for the average response time and for a worst case latency.

Nevertheless redis has a huge amount of commands, some commands are fast and run in constant or logarithmic time, other commands are slower, possibly causing latency spikes. Other reasons for this way of monitoring are interaction with the operating system (e.g. disk persistence) and the single-thread architecture of redis.

Since version 2.8.13 latency monitoring helps to troubleshoot possible latency problems. The following mechanics are part of this framework:

  • Latency hooks that sample different latency sensitive code paths.
  • Time series recording of latency spikes split by another event.
  • Reporting engine to fetch raw data from the time series.
  • Analysis engine to provide human readable reports and hints according to the measurements.

How to enable latency monitoring?

During runtime of redis latency monitoring can be easily enabled by config set latency-monitor-threshold 100. The tailing number defines a time in milliseconds. In our example, every event that will take more then 100ms will be logged as a latency spike. Here are some commands to display and reset logged events:

latency latest display the latest latency events logged
latency history event-name display up to 160 elements for the given event name, you get timestamp and latency value
latency reset reset all events at the time-series
latency reset event-name reset only the given event-name
latency graph event-name ascii-art style graph for the event
latency doctor gives details information about latency spikes and advices

What should I do when redis experiences latency spikes?

The next step could be to enable the slow-log. With config set slowlog-log-slower-than 1000 you can log every event that lasts longer then 1000ms. More commands:

slowlog-log-slower-than -1 disable slow log
slowlog-log-slower-than 0 log every command
slowlog-max-len N maximum length of the logfile (ringbuffer-style)
slowlog get N will display the last N events in the log
slowlog len length of the current logfile
slowlog reset delete all information at the log

More on redis

We’ve already published various articles on redis. If you’re new to the subject you best start with our introduction. Have your redis up and running? Learn how to back its data up and save snapshots. Finally, we show you how to make your data highly available with redis.

Read on

Want to find out more about the services we offer in Data Center Automation? Have a look at our website, write an email to info@inovex.de or call +49 721 619 021-0

Join us!

Looking for a job where you can work with cutting edge technology on a daily basis? We’re currently hiring Linux Systems Engineers in Karlsruhe, Pforzheim, Munich, Cologne and Hamburg!

comments powered by Disqus