..
Prometheus and PromQL mental model
Caution: This should not be treated as a starting document for learning Prometheus and PromQL. Rather this can be treated as a supplement while you are reading a few indepth blogs
Prometheus
- Tool for collecting metrics and data from applications hosted on the cloud
- Popular in containerized environments
Components
Targets are your applications hosted on the cloud
- Retrieval
- Retrieval component is responsible for pulling metrics from targets.
- 3 types of metrics are supported in Prometheus (more below)
- For retrieval mechanism to work, target must expose some metrics. You can add these metric definitions in your code and expose them on
/metrics
endpoint. All of this is supported in Prometheus client libraries available for a lot of languages - There are some exporters that automatically export some metrics from your cloud instance and expose them. Eg: Node exporter
- Storage
- After retrieving, prometheus stores the data in time series format in a database
- Query server
- You can query the metics stored in the database using a query language known as PromQL
- A tool that is very famous for querying and visualizing prometheus metrics is Grafana
- Alert manager
- This component is responsible for setting up alerts based on certain conditions
- You can write a query, which when gives certain results, can alert you
Prom QL
To understand querying in PromQL, you need to understand the supported data types, labels and metric types
Data types
- Scalars
- Single numerical values
- Value of a metric at a particular timestemp
- Instant vectors
- Array of scalars
- You get instant vectors by aggregating scalars over single point in time
- You can query instant vectors directly by name
- Range vectors
- Array of instant vectors (effectively array of array)
- You get instant vectors by aggregating instant vectors over different periods of times
- You can query range vectors with a time selector such as
metric_name_total[5m]
Labels
You can add a label to a metric. For example, instead of creating different metrics for request with 2xx
and 4xx
codes, you can create a single metric and add label for 2xx
and 4xx
Metric types
- Counter
- How many times
x
happened - Stores scalar values
- How many times
- Gauge
- What is the current value of
x
- Stores scalar values
- What is the current value of
- Histogram
- How long or how big?
- Useful for plotting time distribution of request times
- Collection of counters
Querying in PromQL is made easy by different functions and operators that you can apply over the metric types
Function and operators
- Aggregation operators
- Converts instant vector to instant vector
- Eg:
sum
,min
,avg
- Binary operators (
*, /, +, -
)- Converts same type of metric type and operates on same type of metrics
- Delta, increase, rate
- Converts range vectors to instant vectors
- Delta is applied over gauge metric type
- Increase and rate are applied over counter metric type
- Always
sum
arate
and never the other way round
- Histogram quantiles
- Applied over histogram metric types
Reference
- Prometheus quick lesson video
- PromQL for Mere Mortals
- Best resource for understanding PromQL in my opinion
- Intro to PromQL
- Another intro to PromQL