Platform Metrics

Platform metrics measure the health, performance, and efficiency of the platform itself. Comprehensive metrics enable data-driven decisions, early problem detection, and continuous improvement.

Understanding Platform Metrics

Platform metrics track how well the platform serves its users and meets organizational goals.

The Blind Spot Problem

Without metrics, platforms operate blindly:

Operating Without Metrics:
├── Unknown platform health
├── Issues discovered too late
├── No capacity planning data
├── Can't prove platform value
├── Unclear where to invest
├── No SLO compliance tracking
└── Reactive instead of proactive

Metrics-Driven Platform

Data enables informed decisions:

graph LR
    A[Platform Metrics] --> B[Health Monitoring]
    A --> C[Capacity Planning]
    A --> D[Performance Optimization]
    A --> E[Value Demonstration]
    
    B --> F[Proactive Improvements]
    C --> F
    D --> F
    E --> F
    
    style A fill:#e1f5ff
    style F fill:#d4f1d4

Metric Categories

Platform metrics fall into several categories.

System Health Metrics

Core infrastructure health:

# System health metrics
metrics:
  availability:
    - name: platform_uptime_percentage
      description: Platform availability over time period
      target: 99.9%
      measurement: (uptime / total_time) * 100
    
    - name: api_success_rate
      description: Percentage of successful API requests
      target: 99.5%
      measurement: (successful_requests / total_requests) * 100
  
  performance:
    - name: api_response_time_p50
      description: Median API response time
      target: < 200ms
      unit: milliseconds
    
    - name: api_response_time_p99
      description: 99th percentile API response time
      target: < 1000ms
      unit: milliseconds
    
    - name: deployment_duration
      description: Time to complete deployment
      target: < 10 minutes
      unit: minutes
  
  reliability:
    - name: error_rate
      description: Percentage of requests resulting in errors
      target: < 0.5%
      measurement: (error_count / total_requests) * 100
    
    - name: incident_count
      description: Number of incidents per time period
      target: < 5 per month
      unit: count
    
    - name: mttr
      description: Mean time to recovery from incidents
      target: < 30 minutes
      unit: minutes

Collecting system metrics:

# Platform health metrics collector
from prometheus_client import Counter, Histogram, Gauge
import time
from functools import wraps

# Define metrics
api_requests_total = Counter(
    'platform_api_requests_total',
    'Total API requests',
    ['endpoint', 'method', 'status']
)

api_request_duration = Histogram(
    'platform_api_request_duration_seconds',
    'API request duration',
    ['endpoint', 'method']
)

active_deployments = Gauge(
    'platform_active_deployments',
    'Number of active deployments'
)

deployment_duration = Histogram(
    'platform_deployment_duration_seconds',
    'Deployment duration',
    ['service', 'environment']
)

def track_api_metrics(endpoint: str):
    """Decorator to track API metrics"""
    def decorator(func):
        @wraps(func)
        async def wrapper(*args, **kwargs):
            start_time = time.time()
            status = "success"
            
            try:
                result = await func(*args, **kwargs)
                return result
            except Exception as e:
                status = "error"
                raise
            finally:
                duration = time.time() - start_time
                
                # Record metrics
                api_requests_total.labels(
                    endpoint=endpoint,
                    method=request.method,
                    status=status
                ).inc()
                
                api_request_duration.labels(
                    endpoint=endpoint,
                    method=request.method
                ).observe(duration)
        
        return wrapper
    return decorator

# Usage
@track_api_metrics('/api/v1/services')
async def create_service(request):
    # Implementation
    pass

Resource Utilization Metrics

Infrastructure resource consumption:

// Resource utilization metrics
package metrics

import (
    "github.com/prometheus/client_golang/prometheus"
)

type ResourceMetrics struct {
    CPUUsage    *prometheus.GaugeVec
    MemoryUsage *prometheus.GaugeVec
    DiskUsage   *prometheus.GaugeVec
    NetworkIO   *prometheus.CounterVec
}

func NewResourceMetrics() *ResourceMetrics {
    return &ResourceMetrics{
        CPUUsage: prometheus.NewGaugeVec(
            prometheus.GaugeOpts{
                Name: "platform_cpu_usage_percent",
                Help: "CPU usage percentage by service",
            },
            []string{"service", "environment"},
        ),
        MemoryUsage: prometheus.NewGaugeVec(
            prometheus.GaugeOpts{
                Name: "platform_memory_usage_bytes",
                Help: "Memory usage in bytes by service",
            },
            []string{"service", "environment"},
        ),
        DiskUsage: prometheus.NewGaugeVec(
            prometheus.GaugeOpts{
                Name: "platform_disk_usage_bytes",
                Help: "Disk usage in bytes by service",
            },
            []string{"service", "environment", "volume"},
        ),
        NetworkIO: prometheus.NewCounterVec(
            prometheus.CounterOpts{
                Name: "platform_network_bytes_total",
                Help: "Total network bytes transferred",
            },
            []string{"service", "environment", "direction"},
        ),
    }
}

func (rm *ResourceMetrics) CollectKubernetesMetrics(k8s KubernetesClient) {
    // Get all pods
    pods, _ := k8s.ListPods()
    
    for _, pod := range pods {
        metrics := k8s.GetPodMetrics(pod.Name)
        
        // CPU usage
        rm.CPUUsage.WithLabelValues(
            pod.Service,
            pod.Environment,
        ).Set(metrics.CPUPercent)
        
        // Memory usage
        rm.MemoryUsage.WithLabelValues(
            pod.Service,
            pod.Environment,
        ).Set(float64(metrics.MemoryBytes))
        
        // Network I/O
        rm.NetworkIO.WithLabelValues(
            pod.Service,
            pod.Environment,
            "received",
        ).Add(float64(metrics.NetworkBytesReceived))
        
        rm.NetworkIO.WithLabelValues(
            pod.Service,
            pod.Environment,
            "transmitted",
        ).Add(float64(metrics.NetworkBytesTransmitted))
    }
}

Resource efficiency analysis:

# Resource efficiency calculator
from dataclasses import dataclass

@dataclass
class ResourceUsage:
    cpu_requested: float
    cpu_used: float
    memory_requested: float
    memory_used: float

def calculate_resource_efficiency(usage: ResourceUsage) -> dict:
    """Calculate how efficiently resources are used"""
    
    cpu_efficiency = (usage.cpu_used / usage.cpu_requested) * 100
    memory_efficiency = (usage.memory_used / usage.memory_requested) * 100
    
    # Calculate waste
    cpu_waste = usage.cpu_requested - usage.cpu_used
    memory_waste = usage.memory_requested - usage.memory_used
    
    # Determine if over or under provisioned
    cpu_status = "over-provisioned" if cpu_efficiency < 50 else \
                 "under-provisioned" if cpu_efficiency > 90 else \
                 "well-sized"
    
    memory_status = "over-provisioned" if memory_efficiency < 50 else \
                    "under-provisioned" if memory_efficiency > 90 else \
                    "well-sized"
    
    return {
        'cpu_efficiency_percent': cpu_efficiency,
        'memory_efficiency_percent': memory_efficiency,
        'cpu_waste': cpu_waste,
        'memory_waste': memory_waste,
        'cpu_status': cpu_status,
        'memory_status': memory_status,
        'recommendations': generate_recommendations(
            cpu_efficiency,
            memory_efficiency
        )
    }

def generate_recommendations(cpu_eff: float, memory_eff: float) -> list:
    """Generate resource optimization recommendations"""
    recs = []
    
    if cpu_eff < 50:
        recs.append("Reduce CPU requests by 25-50%")
    elif cpu_eff > 90:
        recs.append("Increase CPU requests by 20-30%")
    
    if memory_eff < 50:
        recs.append("Reduce memory requests by 25-50%")
    elif memory_eff > 90:
        recs.append("Increase memory requests by 20-30%")
    
    if not recs:
        recs.append("Resources well-sized, no changes needed")
    
    return recs

Platform Adoption Metrics

Track how teams use the platform:

// Platform adoption metrics
interface AdoptionMetrics {
  totalTeams: number;
  activeTeams: number;
  servicesOnPlatform: number;
  deploymentsPerDay: number;
  uniqueUsersPerWeek: number;
  featureAdoptionRate: Record<string, number>;
}

class AdoptionMetricsCollector {
  async collect(): Promise<AdoptionMetrics> {
    const [
      teams,
      services,
      deployments,
      users,
      features
    ] = await Promise.all([
      this.getTeamStats(),
      this.getServiceStats(),
      this.getDeploymentStats(),
      this.getUserStats(),
      this.getFeatureAdoption()
    ]);
    
    return {
      totalTeams: teams.total,
      activeTeams: teams.active,
      servicesOnPlatform: services.total,
      deploymentsPerDay: deployments.perDay,
      uniqueUsersPerWeek: users.uniquePerWeek,
      featureAdoptionRate: features
    };
  }
  
  async getFeatureAdoption(): Promise<Record<string, number>> {
    const features = [
      'self-service-provisioning',
      'auto-scaling',
      'ci-cd-integration',
      'monitoring-dashboards',
      'secret-management',
      'backup-restore'
    ];
    
    const adoption: Record<string, number> = {};
    
    for (const feature of features) {
      const totalServices = await this.getTotalServices();
      const servicesUsingFeature = await this.getServicesUsingFeature(feature);
      
      adoption[feature] = (servicesUsingFeature / totalServices) * 100;
    }
    
    return adoption;
  }
  
  async calculateGrowthRate(metric: string, days: number): Promise<number> {
    const now = await this.getMetricValue(metric);
    const past = await this.getMetricValue(metric, daysAgo: days);
    
    return ((now - past) / past) * 100;
  }
}

Adoption tracking visualization:

graph TB
    subgraph Month 1
        A1[10 Teams]
        B1[45 Services]
    end
    
    subgraph Month 2
        A2[15 Teams]
        B2[72 Services]
    end
    
    subgraph Month 3
        A3[22 Teams]
        B3[105 Services]
    end
    
    subgraph Month 4
        A4[28 Teams]
        B4[142 Services]
    end
    
    A1 --> A2 --> A3 --> A4
    B1 --> B2 --> B3 --> B4
    
    style A4 fill:#d4f1d4
    style B4 fill:#d4f1d4

Cost Metrics

Track platform costs and efficiency:

# Platform cost metrics
from dataclasses import dataclass
from datetime import datetime

@dataclass
class CostMetrics:
    total_monthly_cost: float
    cost_per_service: float
    cost_per_deployment: float
    cost_per_user: float
    cost_trend: str

class CostAnalyzer:
    def __init__(self):
        self.cloud_provider = CloudProvider()
        self.catalog = ServiceCatalog()
    
    def calculate_platform_costs(self) -> CostMetrics:
        """Calculate comprehensive platform costs"""
        
        # Get infrastructure costs
        compute_cost = self.cloud_provider.get_compute_cost()
        storage_cost = self.cloud_provider.get_storage_cost()
        network_cost = self.cloud_provider.get_network_cost()
        database_cost = self.cloud_provider.get_database_cost()
        
        total_cost = (
            compute_cost +
            storage_cost +
            network_cost +
            database_cost
        )
        
        # Get platform metrics
        service_count = self.catalog.count_services()
        deployment_count = self.get_monthly_deployment_count()
        user_count = self.get_active_user_count()
        
        # Calculate per-unit costs
        cost_per_service = total_cost / service_count
        cost_per_deployment = total_cost / deployment_count
        cost_per_user = total_cost / user_count
        
        # Analyze trend
        last_month_cost = self.get_previous_month_cost()
        cost_change = ((total_cost - last_month_cost) / last_month_cost) * 100
        
        trend = "increasing" if cost_change > 5 else \
                "decreasing" if cost_change < -5 else \
                "stable"
        
        return CostMetrics(
            total_monthly_cost=total_cost,
            cost_per_service=cost_per_service,
            cost_per_deployment=cost_per_deployment,
            cost_per_user=cost_per_user,
            cost_trend=trend
        )
    
    def identify_cost_optimization_opportunities(self) -> list:
        """Find opportunities to reduce costs"""
        opportunities = []
        
        # Check for idle resources
        idle_resources = self.find_idle_resources()
        if idle_resources:
            savings = sum(r.monthly_cost for r in idle_resources)
            opportunities.append({
                'type': 'idle_resources',
                'description': f"Remove {len(idle_resources)} idle resources",
                'potential_savings': savings
            })
        
        # Check for over-provisioned resources
        overprovisioned = self.find_overprovisioned_resources()
        if overprovisioned:
            savings = sum(r.waste_cost for r in overprovisioned)
            opportunities.append({
                'type': 'over_provisioning',
                'description': f"Right-size {len(overprovisioned)} resources",
                'potential_savings': savings
            })
        
        # Check for unattached volumes
        unattached_volumes = self.find_unattached_volumes()
        if unattached_volumes:
            savings = sum(v.monthly_cost for v in unattached_volumes)
            opportunities.append({
                'type': 'unattached_volumes',
                'description': f"Delete {len(unattached_volumes)} unattached volumes",
                'potential_savings': savings
            })
        
        # Check for reserved instance opportunities
        ri_opportunities = self.analyze_reserved_instance_opportunities()
        if ri_opportunities:
            opportunities.append({
                'type': 'reserved_instances',
                'description': "Purchase reserved instances for stable workloads",
                'potential_savings': ri_opportunities['annual_savings']
            })
        
        return sorted(
            opportunities,
            key=lambda x: x['potential_savings'],
            reverse=True
        )

Service Level Objectives (SLOs)

Define and track platform reliability targets.

SLO Definition

Platform SLOs:

# platform-slos.yaml
slos:
  - name: platform-api-availability
    description: Platform API must be available for service management
    sli: successful_requests / total_requests
    target: 99.9%
    window: 30 days
    error_budget: 0.1%
    
  - name: platform-api-latency
    description: Platform API responses must be fast
    sli: request_duration_p99
    target: < 500ms
    window: 30 days
    
  - name: deployment-success-rate
    description: Service deployments must succeed reliably
    sli: successful_deployments / total_deployments
    target: 99%
    window: 30 days
    error_budget: 1%
    
  - name: incident-mttr
    description: Platform incidents must be resolved quickly
    sli: mean_time_to_resolution
    target: < 30 minutes
    window: 30 days

SLO tracking implementation:

// SLO tracking
package slo

import (
    "time"
)

type SLO struct {
    Name         string
    Target       float64
    Window       time.Duration
    ErrorBudget  float64
}

type SLOTracker struct {
    slos map[string]*SLO
}

func (st *SLOTracker) Track(sloName string, success bool) {
    slo := st.slos[sloName]
    
    // Record event
    recordSLIEvent(sloName, success)
    
    // Check if error budget is exhausted
    if st.IsErrorBudgetExhausted(sloName) {
        st.AlertErrorBudgetExhausted(sloName)
    }
}

func (st *SLOTracker) CalculateCompliance(sloName string) float64 {
    slo := st.slos[sloName]
    
    // Get events in window
    events := getEventsInWindow(sloName, slo.Window)
    
    successCount := 0
    for _, event := range events {
        if event.Success {
            successCount++
        }
    }
    
    return float64(successCount) / float64(len(events)) * 100
}

func (st *SLOTracker) GetErrorBudgetRemaining(sloName string) float64 {
    slo := st.slos[sloName]
    
    compliance := st.CalculateCompliance(sloName)
    used := slo.Target - compliance
    remaining := slo.ErrorBudget - used
    
    return remaining
}

func (st *SLOTracker) IsErrorBudgetExhausted(sloName string) bool {
    return st.GetErrorBudgetRemaining(sloName) <= 0
}

Error budget visualization:

graph LR
    A[Error Budget
100%] --> B[Week 1
95% remaining]
    B --> C[Week 2
88% remaining]
    C --> D[Week 3
72% remaining]
    D --> E[Week 4
45% remaining]
    
    style E fill:#fff4cc

Metrics Dashboards

Visualize platform metrics for easy monitoring.

Executive Dashboard

High-level platform health for leadership:

Executive Platform Dashboard
├── Platform Health Score: 98/100
├── Services on Platform: 142 (+12 this month)
├── Monthly Deployments: 1,847 (+23%)
├── Platform Uptime: 99.95%
├── Incident Count: 3 (Target: <5)
├── MTTR: 22 minutes (Target: <30)
├── Monthly Cost: $127K (-5% vs last month)
└── Team Adoption: 85% of engineering teams

Operations Dashboard

Detailed platform operations view:

Platform Operations Dashboard

System Health:
├── API Success Rate: 99.8% (Target: 99.5%)
├── API P99 Latency: 324ms (Target: <500ms)
├── Active Incidents: 0
└── Error Budget Remaining: 72%

Resource Utilization:
├── CPU Usage: 62% (Healthy)
├── Memory Usage: 71% (Healthy)
├── Disk Usage: 45% (Healthy)
└── Network Throughput: 2.3 Gbps

Recent Activity:
├── Deployments (24h): 47 (45 successful, 2 failed)
├── New Services Created: 3
├── API Requests (24h): 2.4M
└── Active Users (24h): 127

Metrics Best Practices

Effective metrics programs follow key principles.

The Four Golden Signals

Focus on what matters:

Golden Signals for Platform:

1. Latency
   - How long do platform operations take?
   - Track: API response time, deployment duration

2. Traffic
   - How much demand is being placed on the platform?
   - Track: API requests, active users, deployments

3. Errors
   - What is the rate of failures?
   - Track: API error rate, failed deployments

4. Saturation
   - How full is the platform?
   - Track: Resource utilization, queue depths

Avoid Vanity Metrics

Focus on actionable metrics:

Vanity Metrics (Avoid):
├── Total API calls
│   └── Better: API calls per service (shows adoption)
├── Total services
│   └── Better: Active services (shows real usage)
└── Total features built
    └── Better: Feature adoption rate (shows value)

Actionable Metrics (Use):
├── Deployment frequency per team (shows velocity)
├── Time to production for new services (shows efficiency)
├── Platform-related incident rate (shows quality)
└── Cost per service (shows efficiency)

Key Takeaways