Prometheus

Overview

Prometheus is a pull-based monitoring system that scrapes metric endpoints and stores time-series data.

Core Concepts

• Scrape jobs and targets
• PromQL query language
• Alert rules
• Service discovery integrations

flowchart LR
    A[Services] --> B[/metrics endpoints]
    B --> C[Prometheus]
    C --> D[Grafana]
    C --> E[Alertmanager]

Practical Example

scrape_configs:
  - job_name: "orders"
    static_configs:
      - targets: ["orders:8080"]

Production Example

A Kubernetes platform uses service discovery and relabeling rules to monitor thousands of pods with team ownership labels.

Common Pitfalls

• Retaining data too long on undersized storage
• Unbounded cardinality from dynamic labels
• Alerting directly from noisy raw metrics

Key Takeaways

• Prometheus offers powerful time-series monitoring.
• Capacity planning and label governance are mandatory.
• Recording rules help reduce query cost and alert noise.