In the forth entry of our series on redis we’ll have a look at its CLI tools. The central question for today: How can we check that redis is working as expected using the command line?
Using redis-cli to check database health sounds easy and, in fact, it is: A simple redis-cli PING on the Redis host returns a PONG. This does not look very reliable, but it is. A redis that returns a PONG is fully functional and works as expected.
To go a bit more into detail: it means that the dataset is completely loaded, and Redis is ready for connections. If something does not work fine, it will display error messages such as “(error) LOADING Redis is loading the dataset in memory”. This will return while loading the rdb/aof file or waiting for replication to be completed.
By using the redis-CLI you can easily use the info command and sub-commands to get some more stats and information.
List of commands
|latency latest||display the latest latency events logged|
|latency history event-name||display up to 160 elements for the given event name, you get timestamp and latency value|
|latency reset||reset all events at the time-series|
|latency reset event-name||reset only the given event-name|
|latency graph event-name||ascii-art style graph for the event|
|latency doctor||gives details information about latency spikes and advices|
To repeat commands you can easily use two options for monitoring: redis-cli -r 5 -i 2 <command> will execute the <command> five times with an interval of two seconds.
With these commands you can easily integrate redis in your monitoring. Just add the redis-CLI commands as custom parameters, e.g. to Zabbix or Nagios. You can also use prometheus exporters to get metrics out of your redis.
Integrated latency monitoring, reporting, slow-logs
Maybe some of you ask yourselves why we need monitoring in the first place, as redis is so incredibly fast. The answer is: you’re right, redis is designed to serve a large amount of queries per second per instance. But unfortunately in most applications there is a strict requirement for the average response time and for a worst case latency.
Nevertheless redis has a huge amount of commands, some commands are fast and run in constant or logarithmic time, other commands are slower, possibly causing latency spikes. Other reasons for this way of monitoring are interaction with the operating system (e.g. disk persistence) and the single-thread architecture of redis.
Since version 2.8.13 latency monitoring helps to troubleshoot possible latency problems. The following mechanics are part of this framework:
- Latency hooks that sample different latency sensitive code paths.
- Time series recording of latency spikes split by another event.
- Reporting engine to fetch raw data from the time series.
- Analysis engine to provide human readable reports and hints according to the measurements.
How to enable latency monitoring?
During runtime of redis latency monitoring can be easily enabled by config set latency-monitor-threshold 100. The tailing number defines a time in milliseconds. In our example, every event that will take more then 100ms will be logged as a latency spike. Here are some commands to display and reset logged events:
|slowlog-log-slower-than -1||disable slow log|
|slowlog-log-slower-than 0||log every command|
|slowlog-max-len N||maximum length of the logfile (ringbuffer-style)|
|slowlog get N||will display the last N events in the log|
|slowlog len||length of the current logfile|
|slowlog reset||delete all information at the log|
What should I do when redis experiences latency spikes?
The next step could be to enable the slow-log. With config set slowlog-log-slower-than 1000 you can log every event that lasts longer then 1000ms. More commands:
More on redis
We’ve already published various articles on redis. If you’re new to the subject you best start with our introduction. Have your redis up and running? Learn how to back its data up and save snapshots. Finally, we show you how to make your data highly available with redis.
Looking for a job where you can work with cutting edge technology on a daily basis? We’re currently hiring Linux Systems Engineers in Karlsruhe, Pforzheim, Munich, Cologne and Hamburg!