Using Prometheus Recording Rules

Libraries: If you want this lab, consider one of these libraries.
Cloud

Using Prometheus Recording Rules

Prometheus is a great way to collect a variety of metrics for your servers and applications. However, sometimes you need to perform calculations on the raw data Prometheus collects in order to gain actionable information. While queries can help you do this, recording rules provide an additional layer of optimization by allowing you to periodically pre-calculate query results and save them. In this lab, you will have the opportunity to implement some basic recording rules. This will provide you with some hands-on familiarity with the process of creating and managing recording rules.

Get started Contact sales

Lab Info

Level

Intermediate

Last updated

Jun 09, 2025

Duration

45m

Challenge

Implement a Recording Rule to Pre-Calculate CPU Usage
1. Log in to the Prometheus server.
2. Create a directory to store rules files:
  
  sudo mkdir -p /etc/prometheus/rules
3. Edit the Prometheus config and add a rules location:
  
  sudo vi /etc/prometheus/prometheus.yml
  
  ... rule_files: - "/etc/prometheus/rules/*.yml" ...
4. Create a rules file for limedrop-gateway server metrics:
  
  sudo vi /etc/prometheus/rules/limedrop-gateway.yml
5. Implement a recording rule to pre-calculate CPU usage for the server:
  
  groups: - name: limedrop_gateway rules: - record: limedrop_gateway:cpu_usage expr: sum(rate(node_cpu_seconds_total{instance='limedrop-gateway:9100',mode!='idle'}[5m])) * 100 / 2
6. Restart Prometheus to load the new configuration:
  
  sudo systemctl restart prometheus
7. Access the expression browser (http://<PROMETHEUS_SERVER_PUBLIC_IP>:9090/graph) and run a query to verify you can see the pre-calculated data:
  
  limedrop_gateway:cpu_usage[5m]

Challenge

Add a Recording Rule to Pre-Calculate Memory Usage

Edit the rules file:

sudo vi /etc/prometheus/rules/limedrop-gateway.yml

Add a recording rule to pre-calculate memory usage for the server:

groups:
- name: limedrop_gateway
  rules:
  - record: limedrop_gateway:cpu_usage
    expr: sum(rate(node_cpu_seconds_total{instance='limedrop-gateway:9100',mode!='idle'}[5m])) * 100 / 2
  - record: limedrop_gateway:memory_usage
    expr: 100 - (node_memory_MemAvailable_bytes{instance='limedrop-gateway:9100'} / node_memory_MemTotal_bytes{instance='limedrop-gateway:9100'}) * 100

Restart Prometheus to load the new configuration:
```
sudo systemctl restart prometheus
```
Access the expression browser (http://<PROMETHEUS_SERVER_PUBLIC_IP>:9090/graph) and run a query to verify you can see the pre-calculated data:
```
limedrop_gateway:memory_usage[5m]
```
You can also check out the rules interface (http://<PROMETHEUS_SERVER_PUBLIC_IP>:9090/rules) to see the status of your rules.

About the author

Pluralsight Skills

Pluralsight Skills gives leaders confidence they have the skills needed to execute technology strategy. Technology teams can benchmark expertise across roles, speed up release cycles and build reliable, secure products. By leveraging our expert content, skill assessments and one-of-a-kind analytics, keep up with the pace of change, put the right people on the right projects and boost productivity. It's the most effective path to developing tech skills at scale.

Real skill practice before real-world application

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Learn by doing

Engage hands-on with the tools and technologies you’re learning. You pick the skill, we provide the credentials and environment.

Follow your guide

All labs have detailed instructions and objectives, guiding you through the learning process and ensuring you understand every step.

Turn time into mastery

On average, you retain 75% more of your learning if you take time to practice. Hands-on labs set you up for success to make those skills stick.