Building Internal Developer Platforms

An Internal Developer Platform (IDP) is a curated set of tools, services, and workflows that reduces cognitive load and accelerates application development. Building an effective IDP requires understanding developer needs, designing appropriate abstractions, and iterating based on feedback.

IDP Architecture

A well-designed IDP consists of several integrated layers working together.

Core Components

The platform stack includes multiple integrated systems:

graph TB
    subgraph Developer Interface
        A[CLI Tools]
        B[Web Portal]
        C[IDE Plugins]
    end
    
    subgraph Platform Services
        D[Application Lifecycle]
        E[Infrastructure Provisioning]
        F[CI/CD Orchestration]
        G[Observability Stack]
    end
    
    subgraph Foundation
        H[Kubernetes]
        I[Cloud Resources]
        J[Databases]
        K[Message Queues]
    end
    
    A --> D
    B --> D
    C --> D
    
    D --> E
    D --> F
    D --> G
    
    E --> H
    E --> I
    F --> H
    G --> H
    
    H --> J
    H --> K
    
    style D fill:#e1f5ff
    style H fill:#fff4e6

Application Lifecycle Management

Centralized application management:

// Platform API for application management
package platform

type Application struct {
    Name      string
    Team      string
    Runtime   string
    Replicas  int
    Resources ResourceRequirements
    Database  *DatabaseConfig
    Cache     *CacheConfig
    Ingress   *IngressConfig
}

type PlatformController struct {
    k8sClient      kubernetes.Interface
    dbProvisioner  DatabaseProvisioner
    cicdOrchestra  CICDOrchestrator
    monitoringMgr  MonitoringManager
}

func (p *PlatformController) CreateApplication(app Application) error {
    // Create namespace
    if err := p.createNamespace(app.Name, app.Team); err != nil {
        return err
    }
    
    // Provision database if requested
    if app.Database != nil {
        dbEndpoint, err := p.dbProvisioner.Provision(app.Name, app.Database)
        if err != nil {
            return err
        }
        app.Database.Endpoint = dbEndpoint
    }
    
    // Setup CI/CD pipeline
    if err := p.cicdOrchestra.CreatePipeline(app); err != nil {
        return err
    }
    
    // Configure monitoring
    if err := p.monitoringMgr.SetupMonitoring(app); err != nil {
        return err
    }
    
    // Create deployment manifests
    if err := p.createDeployment(app); err != nil {
        return err
    }
    
    return nil
}

Infrastructure Abstraction

Hide cloud complexity behind simple interfaces:

# High-level platform resource definition
apiVersion: platform.example.com/v1
kind: Application
metadata:
  name: order-service
  namespace: commerce
spec:
  type: api
  runtime: python
  version: "3.11"
  
  scaling:
    min: 3
    max: 20
    targetCPU: 70
  
  resources:
    tier: medium  # Platform translates to actual resource values
  
  dependencies:
    - name: postgres
      type: database
      version: "15"
      size: large
    
    - name: redis
      type: cache
      size: medium
    
    - name: orders-queue
      type: rabbitmq
      size: small
  
  networking:
    public: true
    domain: orders.example.com
    tls: auto
  
  observability:
    metrics: true
    logging: true
    tracing: true
    alerts:
      - type: error-rate
        threshold: 5%
      - type: latency-p95
        threshold: 500ms

Platform controller translates this into actual infrastructure.

Building the Platform

Constructing an IDP follows a structured approach.

Discovery Phase

Understand developer pain points:

## Developer Interview Questions

### Current Workflow
1. Walk me through deploying a new service from scratch
2. What are the most time-consuming steps?
3. Where do you get blocked waiting for others?
4. What manual steps do you repeat frequently?

### Pain Points
1. What frustrates you most about current tooling?
2. What takes longer than it should?
3. What information is hard to find?
4. What causes the most production incidents?

### Ideal State
1. If you could wave a magic wand, what would change?
2. What capabilities would save the most time?
3. What would make on-call easier?
4. What would you like to stop doing?

Analyze responses to identify patterns:

Common Pain Points Identified:
├── Infrastructure Provisioning: 8/10 teams
│   └── Average wait time: 3-5 days
├── Database Setup: 7/10 teams
│   └── Manual process, often incorrect configuration
├── Monitoring Setup: 9/10 teams
│   └── Each team builds own dashboards
├── Deployment Complexity: 10/10 teams
│   └── Different process for each team
└── Debugging Production Issues: 10/10 teams
    └── Logs scattered across systems

MVP Definition

Start with highest-impact features:

# Platform MVP Scope
version: 1.0
features:
  - name: One-Click Application Creation
    priority: P0
    description: Create new service with single command
    acceptance:
      - Command completes in <5 minutes
      - Creates namespace, deployment, service
      - Sets up CI/CD pipeline
      - Configures basic monitoring
  
  - name: Self-Service Database Provisioning
    priority: P0
    description: Request database without tickets
    acceptance:
      - Database ready in <10 minutes
      - Connection details automatically injected
      - Backups configured automatically
  
  - name: Integrated Observability
    priority: P0
    description: Automatic dashboards and alerts
    acceptance:
      - Dashboard created on first deployment
      - Standard alerts configured
      - Logs automatically aggregated
  
  - name: Simple Deployment
    priority: P0
    description: Deploy with git push
    acceptance:
      - Push to main triggers deployment
      - Automated testing before deploy
      - Rollback capability
  
deferred:
  - Multi-region support
  - A/B testing
  - Advanced scaling policies
  - Custom resource types

Technology Selection

Choose appropriate technologies:

# Platform Technology Stack
infrastructure:
  orchestration: kubernetes
  version: "1.27"
  distribution: eks  # or gke, aks
  
  gitops:
    tool: argocd
    version: "2.8"
  
  ci_cd:
    tool: github-actions
    runners: self-hosted

observability:
  metrics:
    tool: prometheus
    storage: thanos
    retention: 90d
  
  logging:
    tool: loki
    retention: 30d
  
  tracing:
    tool: tempo
    retention: 7d
  
  dashboards:
    tool: grafana

developer_interface:
  cli:
    language: go
    package_manager: homebrew
  
  portal:
    framework: react
    backend: golang
  
  api:
    protocol: grpc
    rest_gateway: true

infrastructure_as_code:
  provisioning: terraform
  kubernetes_config: kustomize

Platform Capabilities

Key features that make platforms valuable.

Service Templates

Standardized starting points:

# Platform service templates
$ platform templates list

Available Templates:
  api-service-go       - REST API service in Go
  api-service-python   - REST API service in Python
  api-service-node     - REST API service in Node.js
  worker-go            - Background worker in Go
  scheduled-job        - Kubernetes CronJob
  frontend-react       - React frontend application
  frontend-static      - Static website

$ platform create service payment-api --template api-service-go

Creating service from template api-service-go...
✓ Generated repository structure
✓ Created CI/CD pipeline
✓ Configured deployment manifests
✓ Setup monitoring dashboards
✓ Initialized git repository

Repository: git@github.com:company/payment-api
Documentation: https://platform.example.com/services/payment-api

Template structure:

templates/api-service-go/
├── cmd/
│   └── server/
│       └── main.go
├── internal/
│   ├── handlers/
│   ├── models/
│   └── repository/
├── pkg/
│   └── platform/
│       ├── database.go
│       ├── logging.go
│       ├── metrics.go
│       └── tracing.go
├── k8s/
│   ├── base/
│   └── overlays/
├── .github/
│   └── workflows/
│       ├── ci.yaml
│       └── cd.yaml
├── Dockerfile
├── go.mod
└── README.md

Infrastructure Provisioning

Automated resource creation:

# Platform provisioning service
from platform import database, cache, queue

class InfrastructureProvisioner:
    def provision_database(self, app_name: str, config: dict) -> dict:
        """Provision database for application"""
        db_instance = database.create(
            name=f"{app_name}-db",
            engine=config.get('engine', 'postgresql'),
            version=config.get('version', '15'),
            size=config.get('size', 'medium'),
            backup_retention=30,
            multi_az=True,
            encrypted=True
        )
        
        # Create credentials secret
        credentials = database.create_credentials(db_instance.id)
        self.k8s.create_secret(
            namespace=app_name,
            name=f"{app_name}-db-credentials",
            data=credentials
        )
        
        # Setup monitoring
        self.monitoring.create_dashboard(
            name=f"{app_name}-database",
            database_id=db_instance.id
        )
        
        return {
            'endpoint': db_instance.endpoint,
            'port': db_instance.port,
            'secret_name': f"{app_name}-db-credentials"
        }
    
    def provision_cache(self, app_name: str, config: dict) -> dict:
        """Provision Redis cache"""
        cache_instance = cache.create(
            name=f"{app_name}-cache",
            engine='redis',
            version=config.get('version', '7.0'),
            size=config.get('size', 'small'),
            num_replicas=2
        )
        
        return {
            'endpoint': cache_instance.endpoint,
            'port': cache_instance.port
        }

CI/CD Integration

Automated build and deployment:

# Platform-generated CI/CD pipeline
name: Platform CI/CD
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    
    - name: Run tests
      run: |
        make test
    
    - name: Build image
      run: |
        docker build -t ${{ secrets.REGISTRY }}/payment-api:${{ github.sha }} .
    
    - name: Security scan
      uses: aquasecurity/trivy-action@master
      with:
        image-ref: ${{ secrets.REGISTRY }}/payment-api:${{ github.sha }}
        severity: HIGH,CRITICAL
    
    - name: Push image
      if: github.ref == 'refs/heads/main'
      run: |
        docker push ${{ secrets.REGISTRY }}/payment-api:${{ github.sha }}
  
  deploy:
    needs: build
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    steps:
    - name: Update manifest
      uses: platform/update-manifest-action@v1
      with:
        application: payment-api
        image: ${{ secrets.REGISTRY }}/payment-api:${{ github.sha }}
        environment: production
    
    - name: Notify deployment
      uses: platform/notify-action@v1
      with:
        status: success
        application: payment-api

Observability Integration

Automatic monitoring setup:

// Platform observability setup
package observability

func SetupMonitoring(app Application) error {
    // Create Grafana dashboard
    dashboard := createDashboard(app)
    if err := grafana.CreateDashboard(dashboard); err != nil {
        return err
    }
    
    // Create Prometheus ServiceMonitor
    serviceMonitor := &promv1.ServiceMonitor{
        ObjectMeta: metav1.ObjectMeta{
            Name:      app.Name,
            Namespace: app.Namespace,
        },
        Spec: promv1.ServiceMonitorSpec{
            Selector: metav1.LabelSelector{
                MatchLabels: map[string]string{
                    "app": app.Name,
                },
            },
            Endpoints: []promv1.Endpoint{
                {
                    Port:     "metrics",
                    Interval: "30s",
                },
            },
        },
    }
    
    // Create alert rules
    alerts := createAlertRules(app)
    if err := prometheus.CreateRules(alerts); err != nil {
        return err
    }
    
    // Configure log aggregation
    if err := loki.CreateLogStream(app.Name, app.Namespace); err != nil {
        return err
    }
    
    return nil
}

func createDashboard(app Application) Dashboard {
    return Dashboard{
        Title: fmt.Sprintf("%s Overview", app.Name),
        Panels: []Panel{
            {
                Title: "Request Rate",
                Query: fmt.Sprintf(
                    `sum(rate(http_requests_total{app="%s"}[5m]))`,
                    app.Name,
                ),
            },
            {
                Title: "Error Rate",
                Query: fmt.Sprintf(
                    `sum(rate(http_requests_total{app="%s",status=~"5.."}[5m])) / sum(rate(http_requests_total{app="%s"}[5m]))`,
                    app.Name, app.Name,
                ),
            },
            {
                Title: "Response Time (p95)",
                Query: fmt.Sprintf(
                    `histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{app="%s"}[5m])) by (le))`,
                    app.Name,
                ),
            },
        },
    }
}

Platform SDK

Provide libraries for common functionality:

# Platform Python SDK
from platform_sdk import Database, Cache, Logging, Metrics, Tracing

# Automatically configured from environment
db = Database()
cache = Cache()
logger = Logging.get_logger(__name__)
metrics = Metrics()
tracer = Tracing.get_tracer(__name__)

@app.route('/api/orders/<order_id>')
@tracer.start_span('get_order')
def get_order(order_id):
    timer = metrics.timer('order_retrieval_duration')
    timer.start()
    
    try:
        # Check cache
        cached = cache.get(f'order:{order_id}')
        if cached:
            metrics.increment('cache_hits')
            logger.info('Cache hit', order_id=order_id)
            return cached
        
        metrics.increment('cache_misses')
        
        # Query database
        order = db.query_one(
            'SELECT * FROM orders WHERE id = %s',
            [order_id]
        )
        
        if not order:
            logger.warn('Order not found', order_id=order_id)
            metrics.increment('orders_not_found')
            return {'error': 'Not found'}, 404
        
        # Cache result
        cache.set(f'order:{order_id}', order, ttl=300)
        
        logger.info('Order retrieved', order_id=order_id)
        metrics.increment('orders_retrieved')
        
        return order
        
    except Exception as e:
        logger.error('Failed to retrieve order', 
                    order_id=order_id, 
                    error=str(e))
        metrics.increment('order_retrieval_errors')
        raise
    finally:
        timer.stop()

Platform Evolution

Mature platforms continuously improve.

Feedback Loops

Gather input systematically:

# Platform feedback mechanisms
surveys:
  frequency: quarterly
  questions:
    - How satisfied are you with the platform? (1-10)
    - What feature would have the biggest impact?
    - What causes the most friction?
    - What documentation is missing or unclear?

office_hours:
  frequency: weekly
  format: open_forum
  duration: 1_hour
  topics:
    - Platform features
    - Troubleshooting
    - Feature requests
    - Best practices

metrics:
  track:
    - Platform adoption rate
    - Time to first deployment
    - Developer satisfaction score
    - Support ticket volume
    - Feature usage statistics

feedback_channels:
  - Slack channel: #platform-feedback
  - Email: platform-team@example.com
  - Issue tracker: github.com/company/platform/issues

Versioning Strategy

Manage platform evolution:

## Platform Versioning

### Semantic Versioning
- Major: Breaking changes requiring migration
- Minor: New features, backward compatible
- Patch: Bug fixes, no feature changes

### Version Support
- Current version: Full support
- Previous version: Security fixes only (6 months)
- Older versions: No support

### Migration Process
1. Announce deprecation 3 months in advance
2. Provide migration guide and tools
3. Offer migration support during transition
4. Monitor adoption of new version
5. Remove deprecated version after support period

Common Pitfalls

Building in Isolation: Creating platform without developer input results in unused features. Continuously engage with platform users.

Over-Engineering: Adding unnecessary features increases complexity. Focus on solving actual pain points first.

Poor Documentation: Complex platforms without documentation frustrate users. Invest equally in docs and code.

Ignoring Feedback: Dismissing user feedback alienates developers. Treat feedback seriously and act on it.

Key Takeaways