> For the complete documentation index, see [llms.txt](https://www.pranaypourkar.co.in/the-programmers-guide/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.pranaypourkar.co.in/the-programmers-guide/system-design/observability/metrics/prometheus/promql.md).
# PromQL
## About
Prometheus is a time-series database and monitoring system that collects metrics from our applications and infrastructure. It uses a powerful query language called **PromQL (Prometheus Query Language)** to extract, filter, and aggregate metrics over time.
In OpenShift, Prometheus is often used as the backend for monitoring the cluster, workloads, nodes, and custom applications. We can execute PromQL queries directly from the **OpenShift Web Console** (under *Observe → Metrics*) or via the Prometheus UI.
## **Purpose of Prometheus Queries in OpenShift**
* Monitor pod or node CPU/memory usage
* Track container restarts or failures
* Measure request rates, latencies, and errors
* Create custom dashboards or alerts
* Troubleshoot performance or resource issues
## **Basic Structure of PromQL**
Prometheus queries work on **metrics**, each optionally labeled with key-value pairs for filtering. A query generally looks like:
```
metric_name{label1="value1", label2="value2"} [range]
```
You can use:
* **Instant vector** queries (current value): `metric_name{...}`
* **Range vector** queries (over time): `metric_name{...}[5m]`
* **Functions** for aggregation, rates, comparisons, etc.
## **Executing Queries in OpenShift Console**
1. **Open OpenShift Console**
2. Navigate to: **Observe → Metrics**
3. In the **"Expression"** box, enter your PromQL query
4. Choose:
* **“Run”** to get instant values
* **“Show Graph”** for visual trends over time
5. Optional: Filter by **namespace**, **label**, or **duration**
## Common Prometheus Queries in OpenShift
### 1. **CPU Usage**
**Query:**
{% code overflow="wrap" %}
```promql
sum(rate(container_cpu_usage_seconds_total{pod="deviceintegrationservice-9f78b7478-pz2cl", namespace="env-dev"}[48h])) by (pod)
```
{% endcode %}
Returns the average CPU usage (in cores) over 5 minutes for the pod. Helps detect if the pod is under or over-utilizing CPU. Useful for auto-scaling and performance monitoring.
### 2. **Memory Usage**
**Query:**
{% code overflow="wrap" %}
```promql
sum(container_memory_usage_bytes{pod="deviceintegrationservice-9f78b7478-pz2cl", namespace="en-devv"}) by (pod)
```
{% endcode %}
Shows how much memory (in bytes) the pod is using. Helps detect memory leaks, over-consumption, or the need for adjusting memory requests/limits.
### 3. **Pod Restarts**
**Query:**
{% code overflow="wrap" %}
```promql
sum(kube_pod_container_status_restarts_total{pod="deviceintegrationservice-9f78b7478-pz2cl", namespace="env-dev"}) by (pod)
```
{% endcode %}
Displays the total number of container restarts for the pod. Useful for identifying unstable or crash-looping containers.
### 4. **Thread Count per JVM State**
**Query:**
{% code overflow="wrap" %}
```promql
sum(jvm_threads_states_threads{job="integration-service", namespace="env-uat"}) by (pod, state)
```
{% endcode %}
Breaks down the number of JVM threads by state (e.g., RUNNABLE, BLOCKED, TIMED\_WAITING). This is useful for diagnosing thread leaks or blocked thread pools in Java applications.
### 5. **Running Pods (for cross-check)**
**Query:**
{% code overflow="wrap" %}
```promql
count(kube_pod_status_phase{phase="Running", pod="integrationservice-8f78b7478-pz2cl", namespace="env-dev"})
```
{% endcode %}
Ensures the pod is currently running. You can use this to validate rollout status, pod health, or replicas during deployments.
### 6. **CPU Requests vs Usage**
**Requested CPU:**
{% code overflow="wrap" %}
```promql
sum(kube_pod_container_resource_requests_cpu_cores{pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}) by (pod)
```
{% endcode %}
**Used CPU:**
{% code overflow="wrap" %}
```promql
sum(rate(container_cpu_usage_seconds_total{pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}[5m])) by (pod)
```
{% endcode %}
These queries help compare how much CPU was requested vs how much is being used. Critical for detecting over-requesting or under-provisioning.
### 7. **Memory Requests vs Usage**
**Requested Memory:**
{% code overflow="wrap" %}
```promql
sum(kube_pod_container_resource_requests_memory_bytes{pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}) by (pod)
```
{% endcode %}
**Used Memory:**
{% code overflow="wrap" %}
```promql
sum(container_memory_usage_bytes{pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}) by (pod)
```
{% endcode %}
Helps understand whether our pod is requesting more or less memory than it actually needs. Important for resource optimization and avoiding OOM errors.
### 8. **Network Usage**
**Receive:**
{% code overflow="wrap" %}
```promql
sum(rate(container_network_receive_bytes_total{pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}[5m])) by (pod)
```
{% endcode %}
**Transmit:**
{% code overflow="wrap" %}
```promql
sum(rate(container_network_transmit_bytes_total{pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}[5m])) by (pod)
```
{% endcode %}
Monitors how much network traffic (in bytes/sec) the pod is receiving and sending. Useful for identifying chatty services, bottlenecks, or unexpected spikes in network load.
### 9. **Container Restarts Over Time**
**Query:**
{% code overflow="wrap" %}
```promql
increase(kube_pod_container_status_restarts_total{pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}[5m])
```
{% endcode %}
Tracks restart count increases over time. Helps correlate instability with specific time windows or deployments.
### 10. **Error Rate (If We Have HTTP Metrics)**
**Query (example, requires instrumentation):**
{% code overflow="wrap" %}
```promql
rate(http_server_requests_seconds_count{status=~"5..", pod="integrationservice-8f78b7478-pz2cl", namespace="env-uat"}[5m])
```
{% endcode %}
Shows how many 5xx errors occurred in the last 5 minutes. Helps identify backend failures, circuit breaker trips, or timeouts in dependent services.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://www.pranaypourkar.co.in/the-programmers-guide/system-design/observability/metrics/prometheus/promql.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.